Management Reasoning: Empirical Determination of Key Features and a Conceptual Model

Reasoning on Rounds Volume 2: a Framework for Teaching Management Reasoning in the Inpatient Setting

Management reasoning (MR) is a key domain of clinical reasoning that is distinct from the more heavily studied and taught diagnostic reasoning (DR). Despite MR's importance to patient care, there are few published strategies for incorporating MR education into the clinical learning environment. In this perspective, the authors review key theories and clinical principles relevant to MR and integrate these concepts with previously described tools for teaching MR to provide frontline clinical teachers with practical, theory-informed framework for teaching MR during inpatient rounds.

Virtual Patients Using Large Language Models: Scalable, Contextualized Simulation of Clinician-Patient Dialogue With Feedback

BACKGROUND

Virtual patients (VPs) are computer screen-based simulations of patient-clinician encounters. VP use is limited by cost and low scalability.

OBJECTIVE

We aimed to show that VPs powered by large language models (LLMs) can generate authentic dialogues, accurately represent patient preferences, and provide personalized feedback on clinical performance. We also explored using LLMs to rate the quality of dialogues and feedback.

METHODS

We conducted an intrinsic evaluation study rating 60 VP-clinician conversations. We used carefully engineered prompts to direct OpenAI's generative pretrained transformer (GPT) to emulate a patient and provide feedback. Using 2 outpatient medicine topics (chronic cough diagnosis and diabetes management), each with permutations representing different patient preferences, we created 60 conversations (dialogues plus feedback): 48 with a human clinician and 12 "self-chat" dialogues with GPT role-playing both the VP and clinician. Primary outcomes were dialogue authenticity and feedback quality, rated using novel instruments for which we conducted a validation study collecting evidence of content, internal structure (reproducibility), relations with other variables, and response process. Each conversation was rated by 3 physicians and by GPT. Secondary outcomes included user experience, bias, patient preferences represented in the dialogues, and conversation features that influenced authenticity.

RESULTS

The average cost per conversation was US $0.51 for GPT-4.0-Turbo and US $0.02 for GPT-3.5-Turbo. Mean (SD) conversation ratings, maximum 6, were overall dialogue authenticity 4.7 (0.7), overall user experience 4.9 (0.7), and average feedback quality 4.7 (0.6). For dialogues created using GPT-4.0-Turbo, physician ratings of patient preferences aligned with intended preferences in 20 to 47 of 48 dialogues (42%-98%). Subgroup comparisons revealed higher ratings for dialogues using GPT-4.0-Turbo versus GPT-3.5-Turbo and for human-generated versus self-chat dialogues. Feedback ratings were similar for human-generated versus GPT-generated ratings, whereas authenticity ratings were lower. We did not perceive bias in any conversation. Dialogue features that detracted from authenticity included that GPT was verbose or used atypical vocabulary (93/180, 51.7% of conversations), was overly agreeable (n=56, 31%), repeated the question as part of the response (n=47, 26%), was easily convinced by clinician suggestions (n=35, 19%), or was not disaffected by poor clinician performance (n=32, 18%). For feedback, detractors included excessively positive feedback (n=42, 23%), failure to mention important weaknesses or strengths (n=41, 23%), or factual inaccuracies (n=39, 22%). Regarding validation of dialogue and feedback scores, items were meticulously developed (content evidence), and we confirmed expected relations with other variables (higher ratings for advanced LLMs and human-generated dialogues). Reproducibility was suboptimal, due largely to variation in LLM performance rather than rater idiosyncrasies.

CONCLUSIONS

LLM-powered VPs can simulate patient-clinician dialogues, demonstrably represent patient preferences, and provide personalized performance feedback. This approach is scalable, globally accessible, and inexpensive. LLM-generated ratings of feedback quality are similar to human ratings.

GPT-4 assistance for improvement of physician performance on patient care tasks: a randomized controlled trial

While large language models (LLMs) have shown promise in diagnostic reasoning, their impact on management reasoning, which involves balancing treatment decisions and testing strategies while managing risk, is unknown. This prospective, randomized, controlled trial assessed whether LLM assistance improves physician performance on open-ended management reasoning tasks compared to conventional resources. From November 2023 to April 2024, 92 practicing physicians were randomized to use either GPT-4 plus conventional resources or conventional resources alone to answer five expert-developed clinical vignettes in a simulated setting. All cases were based on real, de-identified patient encounters, with information revealed sequentially to mirror the nature of clinical environments. The primary outcome was the difference in total score between groups on expert-developed scoring rubrics. Secondary outcomes included domain-specific scores and time spent per case. Physicians using the LLM scored significantly higher compared to those using conventional resources (mean difference = 6.5%, 95% confidence interval (CI) = 2.7 to 10.2, P < 0.001). LLM users spent more time per case (mean difference = 119.3 s, 95% CI = 17.4 to 221.2, P = 0.02). There was no significant difference between LLM-augmented physicians and LLM alone (-0.9%, 95% CI = -9.0 to 7.2, P = 0.8). LLM assistance can improve physician management reasoning in complex clinical vignettes compared to conventional resources and should be validated in real clinical practice. ClinicalTrials.gov registration: NCT06208423 .

Exploring management reasoning when discharging high-risk postoperative patients from the post-anaesthesia care unit

INTRODUCTION

Decision-support tools for detecting physiological deterioration are widely used in clinical medicine but have been criticised for fostering a task-oriented culture and reducing the emphasis on clinical reasoning. Little is understood about what influences clinical decisions aided by decision-support tools, including professional standards, policies, and contextual factors. Therefore, we explored management reasoning employed by anaesthesiologists and PACU nurses in the post-anaesthesia care unit during the discharge of high-risk postoperative patients.

METHODS

A qualitative constructivist study, conducting 18 semi-structured with 6 anaesthesiologists and 12 nurses across three Danish teaching hospitals. We analysed data through thematic analysis, utilising Michael Lipsky's theory of "street-level bureaucracy" in combination with David A. Cook's Management Reasoning Framework as a sensitising concept.

RESULTS

Standards are frequently ambiguous, requiring interpretation and prioritisation. This allows for professional discretion by circumventing established policies, reducing task-oriented culture and enhancing the clinical reasoning processes. However, discretion in management reasoning depends on whether the clinician is inclined to uphold or adjust policies to maintain professional standards, influencing discharge decisions.

CONCLUSION

While decision-support tools offer cognitive aid and help standardise patient trajectories, they also limit professional discretion in management reasoning and can potentially compromise care and treatment. This highlights the need for a balanced approach that considers both the benefits and limitations of these tools in clinical decision-making.

Assessment of management reasoning: Design considerations drawn from analysis of simulated outpatient encounters

PURPOSE

Management reasoning is a distinct subset of clinical reasoning. We sought to explore features to be considered when designing assessments of management reasoning.

METHODS

This is a hybrid empirical research study, narrative review, and expert perspective. In 2021, we reviewed and discussed 10 videos of simulated (staged) physician-patient encounters, actively seeking actions that offered insights into assessment of management reasoning. We analyzed our own observations in conjunction with literature on clinical reasoning assessment, using a constant comparative qualitative approach.

RESULTS

Distinguishing features of management reasoning that will influence its assessment include management scripts, shared decision-making, process knowledge, illness-specific knowledge, and tailoring of the encounter and management plan. Performance domains that merit special consideration include communication, integration of patient preferences, adherence to the management script, and prognostication. Additional facets of encounter variation include the clinical problem, clinical and nonclinical patient characteristics (including preferences, values, and resources), team/system characteristics, and encounter features. We cataloged several relevant assessment approaches including written/computer-based, simulation-based, and workplace-based modalities, and a variety of novel response formats.

CONCLUSIONS

Assessment of management reasoning could be improved with attention to the performance domains, facets of variation, and variety of approaches herein identified.

On context specificity and management reasoning: moving beyond diagnosis

OBJECTIVES

Diagnostic error is a global emergency. Context specificity is likely a source of the alarming rate of error and refers to the vexing phenomenon whereby a physician can see two patients with the same presenting complaint, identical history and examination findings, but due to the presence of contextual factors, decides on two different diagnoses. Studies have not empirically addressed the potential role of context specificity in management reasoning and errors with a diagnosis may not consistently translate to actual patient care.

METHODS

We investigated the effect of context specificity on management reasoning in individuals working within a simulated internal medicine environment. Participants completed two ten minute back to back common encounters. The clinical content of each encounter was identical. One encounter featured the presence of carefully controlled contextual factors (CF+ vs. CF-) designed to distract from the correct diagnosis and management. Immediately after each encounter participants completed a post encounter form.

RESULTS

Twenty senior medical students participated. The leading diagnosis score was higher (mean 0.88; SEM 0.07) for the CF- encounter compared with the CF+ encounter (0.58; 0.1; 95 % CI 0.04-0.56; p=0.02). Management reasoning scores were higher (mean 5.48; SEM 0.66) for the CF- encounter compared with the CF+ encounter (3.5; 0.56; 95 % CI 0.69-3.26; p=0.01). We demonstrated context specificity in both diagnostic and management reasoning.

CONCLUSIONS

This study is the first to empirically demonstrate that management reasoning, which directly impacts the patient, is also influenced by context specificity, providing additional evidence of context specificity's role in unwanted variance in health care.

Faculty Reflections on What Makes a Good Surgeon: "The operating Room is Often the Smallest Part of the Puzzle"

OBJECTIVE

Little research has inductively investigated the unique nontechnical qualities required of a surgeon holistic to their practice. This is problematic because there may be additional nuances, or entirely new attributes, that can only be identified in the authentic context of surgical practice. The aim of this study was to investigate the unique nontechnical qualities required of surgeons holistic to their practice.

DESIGN AND SETTING

We conducted a thematic analysis. One-hour in-depth semi-structured interviews were conducted with faculty surgeons from two academic hospitals. Surgeons responded to the question: "What makes you a good surgeon?" Interviews were transcribed and coded. Theory-informing inductive data analysis, utilizing the lens of virtues ethics, allowed for development of an overarching theme.

PARTICIPANT AND RESULTS

Twenty-seven surgeons (25.9% female) participated. Ideas presented by surgeons on what makes a good surgeon were distilled into a novel conceptual framework comprising five virtue couplets. The good surgeon is perceptive and caring; self-reflective and growth-seeking; confident and humble; driven and balance-seeking; and honest and responsible.

CONCLUSIONS

This study indicates a unique set of nontechnical virtues present in the "good surgeon." These virtues offer areas ripe for education and investigation.

Enhancing therapeutic reasoning: key insights and recommendations for education in prescribing

BACKGROUND

Despite efforts to improve undergraduate clinical pharmacology & therapeutics (CPT) education, prescribing errors are still made regularly. To improve CPT education and daily prescribing, it is crucial to understand how therapeutic reasoning works. Therefore, the aim of this study was to gain insight into the therapeutic reasoning process.

METHODS

A narrative literature review has been performed for literature on cognitive psychology and diagnostic and therapeutic reasoning.

RESULTS

Based on these insights, The European Model of Therapeutic Reasoning has been developed, building upon earlier models and insights from cognitive psychology. In this model, it can be assumed that when a diagnosis is made, a primary, automatic response as to what to prescribe arises based on pattern recognition via therapy scripts (type 1 thinking). At some point, this response may be evaluated by the reflective mind (using metacognition). If it is found to be incorrect or incomplete, an alternative response must be formulated through a slower, more analytical and deliberative process, known as type 2 thinking. Metacognition monitors the reasoning process and helps a person to form new therapy scripts after they have chosen an effective therapy. Experienced physicians have more and richer therapy scripts, mostly based on experience and enabling conditions, instead of textbook knowledge, and therefore their type 1 response is more often correct.

CONCLUSION

Because of the important role of metacognition in therapeutic reasoning, more attention should be paid to metacognition in CPT education. Both trainees and teachers should be aware of the possibility to monitor and influence these cognitive processes. Further research is required to investigate the applicability of these insights and the adaptability of educational approaches to therapeutic reasoning.

Reasoning Report: Engineering Case Conferences to Maximize Clinical Reasoning Education for All Learners

Case conferences, specifically those in which an unknown case is presented and discussed, are widely utilized in the delivery of medical education. However, the format of case conferences is not always optimized to engage and challenge audience members' clinical reasoning (CR). Based on the current conception of CR and our experience, we provide recommendations on how to better engineer case conferences to maximize CR education for learners at all levels through case selection, conference format, and intentional case construction.

Large Language Model Influence on Management Reasoning: A Randomized Controlled Trial

Importance

Large language model (LLM) artificial intelligence (AI) systems have shown promise in diagnostic reasoning, but their utility in management reasoning with no clear right answers is unknown.

Objective

To determine whether LLM assistance improves physician performance on open-ended management reasoning tasks compared to conventional resources.

Design

Prospective, randomized controlled trial conducted from 30 November 2023 to 21 April 2024.

Setting

Multi-institutional study from Stanford University, Beth Israel Deaconess Medical Center, and the University of Virginia involving physicians from across the United States.

Participants

92 practicing attending physicians and residents with training in internal medicine, family medicine, or emergency medicine.

Intervention

Five expert-developed clinical case vignettes were presented with multiple open-ended management questions and scoring rubrics created through a Delphi process. Physicians were randomized to use either GPT-4 via ChatGPT Plus in addition to conventional resources (e.g., UpToDate, Google), or conventional resources alone.

Main Outcomes and Measures

The primary outcome was difference in total score between groups on expert-developed scoring rubrics. Secondary outcomes included domain-specific scores and time spent per case.

Results

Physicians using the LLM scored higher compared to those using conventional resources (mean difference 6.5 %, 95% CI 2.7-10.2, p<0.001). Significant improvements were seen in management decisions (6.1%, 95% CI 2.5-9.7, p=0.001), diagnostic decisions (12.1%, 95% CI 3.1-21.0, p=0.009), and case-specific (6.2%, 95% CI 2.4-9.9, p=0.002) domains. GPT-4 users spent more time per case (mean difference 119.3 seconds, 95% CI 17.4-221.2, p=0.02). There was no significant difference between GPT-4-augmented physicians and GPT-4 alone (-0.9%, 95% CI -9.0 to 7.2, p=0.8).

Conclusions and Relevance

LLM assistance improved physician management reasoning compared to conventional resources, with particular gains in contextual and patient-specific decision-making. These findings indicate that LLMs can augment management decision-making in complex cases.

Trial registration

ClinicalTrials.gov Identifier: NCT06208423; https://classic.clinicaltrials.gov/ct2/show/NCT06208423.

'Walking together': How relationships shape physicians' clinical reasoning

INTRODUCTION

The clinical reasoning literature has increasingly considered context as an important influence on physicians' thinking. Physicians' relationships with patients, and their ongoing efforts to maintain these relationships, are important influences on how clinical reasoning is contextualised. The authors sought to understand how physicians' relationships with patients shaped their clinical reasoning.

METHODS

Drawing from constructivist grounded theory, the authors conducted semi-structured interviews with primary care physicians. Participants were asked to reflect on recent challenging clinical experiences, and probing questions were used to explore how participants attended to or leveraged relationships in conjunction with their clinical reasoning. Using constant comparison, three investigators coded transcripts, organising the data into codes and conceptual categories. The research team drew from these codes and categories to develop theory about the phenomenon of interest.

RESULTS

The authors interviewed 15 primary care physicians with a range of experience in practice and identified patient agency as a central influence on participants' clinical reasoning. Participants drew from and managed relationships with patients while attending to patients' agency in three ways. First, participants described how contextualised illness constructions enabled them to individualise their approaches to diagnosis and management. Second, participants managed tensions between enacting their typical approaches to clinical problems and adapting their approaches to foster ongoing relationships with patients. Finally, participants attended to relationships with patients' caregivers, seeing these individuals' contributions as important influences on how their clinical reasoning could be enacted within patients' unique social contexts.

CONCLUSION

Clinical reasoning is influenced in important ways by physicians' efforts to both draw from, and maintain, their relationships with patients and patients' caregivers. Such efforts create tensions between their professional standards of care and their orientations toward patient-centredness. These influences of relationships on physicians' clinical reasoning have important implications for training and clinical practice.

Management reasoning and patient-clinician interactions: Insights from shared decision-making and simulated outpatient encounters

PURPOSE

To expand understanding of patient-clinician interactions in management reasoning.

METHODS

We reviewed 10 videos of simulated patient-clinician encounters to identify instances of problematic and successful communication, then reviewed the videos again through the lens of two models of shared decision-making (SDM): an 'involvement-focused' model and a 'problem-focused' model. Using constant comparative qualitative analysis we explored the connections between these patient-clinician interactions and management reasoning.

RESULTS

Problems in patient-clinician interactions included failures to: encourage patient autonomy; invite the patient's involvement in decision-making; convey the health impact of the problem; explore and address concerns and questions; explore the context of decision-making (including patient preferences); meet the patient where they are; integrate situational preferences and priorities; offer >1 viable option; work with the patient to solve a problem of mutual concern; explicitly agree to a final care plan; and build the patient-clinician relationship. Clinicians' 'management scripts' varied along a continuum of prioritizing clinician vs patient needs. Patients also have their own cognitive scripts that guide their interactions with clinicians. The involvement-focused and problem-focused SDM models illuminated distinct, complementary issues.

CONCLUSIONS

Management reasoning is a deliberative interaction occurring in the space between individuals. Juxtaposing management reasoning alongside SDM generated numerous insights.

Script concordance test to assess diagnostic and management reasoning in acute medicine

Clinical reasoning, an essential skill for patient care, can be difficult to assess. We created and validated a script concordance test (SCT) to assess clinical reasoning in acute medicine. This tool was used to provide feedback and targeted remediation for Postgraduate-Year-1 (PGY1) doctors, guide teaching and learning, and facilitate programme evaluation.

Management reasoning scripts: Qualitative exploration using simulated physician-patient encounters

INTRODUCTION

Management reasoning is distinct from diagnostic reasoning and remains incompletely understood. The authors sought to empirically investigate the concept of management scripts.

METHODS

In November 2020, 4 investigators each reviewed 10 video clips of simulated outpatient physician-patient encounters, and used a coding form to document observations about management reasoning. The team used constant comparative analysis to integrate empirically-grounded insights with theories related to cognitive scripts and Type 1/Type 2 thinking.

RESULTS

Management scripts are precompiled conceptual knowledge structures that represent and connect management options and clinician tasks in a temporal or logical sequence. Management scripts appear to differ substantially from illness scripts. Management scripts varied in quality (in content, sequence, flexibility, and fluency) and generality. The authors empirically identified six key features (components) of management scripts: the problem (diagnosis); management options; preferences, values, and constraints; education needs; interactions; and encounter flow. The authors propose a heuristic framework describing script activation, selection, instantiation with case-specific details, and application to guide development of the management plan. They further propose that management reasoning reflects iterative, back-and-forth involvement of both Type 1 (non-analytic, effortless) and Type 2 (analytic, effortful) thinking. Type 1 thinking likely influences initial script activation, selection, and initial instantiation. Type 2 increasingly influences subsequent script revisions, as activation, selection, and instantiation become more deliberate (effortful) and more hypothetical (involving mental simulation).

DISCUSSION

Management scripts constitute a key feature of management reasoning, and could represent a new target for training in clinical reasoning (distinct from illness scripts).