Clinical vignette-based interactive discussion sessions: feedback from residents

Patient Interpretations of Heart Failure Symptom Recognition and Self-Management Using Vignettes: A Pilot Study

BACKGROUND

Heart failure is a condition with significant symptom burden and high hospitalization rates. Effective self-management, including recognizing symptoms and making behavior changes, is crucial but often inadequately addressed by current educational methods. To improve this, heart failure self-care vignettes were developed to measure knowledge of managing physical and psychological symptoms. However, the vignettes' face validity and reliability have not been thoroughly evaluated.

OBJECTIVE

To ensure the understandability of a novel instrument to measure knowledge of heart failure symptom recognition and self-management and to examine its inter-rater reliability with individuals diagnosed with heart failure.

METHODS

Cognitive interviews were conducted with heart failure patients admitted to an academic hospital in the Midwest U.S. Vignette segments and interview questions were analyzed, totaling 74 items. Five patients aged 65 and older, participated without cognitive, visual, or hearing impairments. The interviews were coded by two independent raters using a cognitive interview coding book. Vignettes were evaluated qualitatively with a 3-point Likert scale (1 = misunderstanding, 2 = partial understanding, and 3 = full understanding). Inter-rater reliability was assessed using percent agreement and Cohen's kappa.

RESULTS

Patients understood an average of 76% of the psychological and 83% of the physical vignette items, indicating acceptable preliminary understandability. Inter-rater reliability was moderate, with Cohen's kappa values of 0.39 (psychological) and 0.43 (physical).

CONCLUSIONS

This pilot study suggests that vignettes could be a useful tool for assessing knowledge of symptom recognition and self-management. Cognitive interviewing helped evaluate how vignette segments were interpreted before using them in future data collection.

Can AI-Generated Clinical Vignettes in Japanese Be Used Medically and Linguistically?

BACKGROUND

Creating clinical vignettes requires considerable effort. Recent developments in generative artificial intelligence (AI) for natural language processing have been remarkable and may allow for the easy and immediate creation of diverse clinical vignettes.

OBJECTIVE

In this study, we evaluated the medical accuracy and grammatical correctness of AI-generated clinical vignettes in Japanese and verified their usefulness.

METHODS

Clinical vignettes were created using the generative AI model GPT-4-0613. The input prompts for the clinical vignettes specified the following seven elements: (1) age, (2) sex, (3) chief complaint and time course since onset, (4) physical findings, (5) examination results, (6) diagnosis, and (7) treatment course. The list of diseases integrated into the vignettes was based on 202 cases considered in the management of diseases and symptoms in Japan's Primary Care Physicians Training Program. The clinical vignettes were evaluated for medical and Japanese-language accuracy by three physicians using a five-point scale. A total score of 13 points or above was defined as "sufficiently beneficial and immediately usable with minor revisions," a score between 10 and 12 points was defined as "partly insufficient and in need of modifications," and a score of 9 points or below was defined as "insufficient."

RESULTS

Regarding medical accuracy, of the 202 clinical vignettes, 118 scored 13 points or above, 78 scored between 10 and 12 points, and 6 scored 9 points or below. Regarding Japanese-language accuracy, 142 vignettes scored 13 points or above, 56 scored between 10 and 12 points, and 4 scored 9 points or below. Overall, 97% (196/202) of vignettes were available with some modifications.

CONCLUSION

Overall, 97% of the clinical vignettes proved practically useful, based on confirmation and revision by Japanese medical physicians. Given the significant effort required by physicians to create vignettes without AI, using GPT is expected to greatly optimize this process.

Using Clinical Vignettes to Understand the Complexity of Diagnosing Type 1 Diabetes in Sub-Saharan Africa

Lack of awareness, access to insulin and diabetes care can result in high levels of morbidity and mortality for children with type 1 diabetes (T1DM) in sub-Saharan Africa (SSA). Improvements in access to insulin and diabetes management have improved outcomes in some settings. However, many people still present in diabetic ketoacidosis (DKA) in parallel to misdiagnosis of children with T1DM in contexts with high rates of communicable diseases. The aim of this study was to highlight the complexity of diagnosing pediatric T1DM in a healthcare environment dominated by infectious diseases and lack of adequate health system resources. This was done by developing clinical vignettes and recreating the hypothetico-deductive process of a clinician confronted with DKA in the absence of identification of pathognomonic elements of diabetes and with limited diagnostic tools. A non-systematic literature search for T1DM and DKA in SSA was conducted and used to construct clinical vignettes for children presenting in DKA. A broad differential diagnosis of the main conditions present in SSA was made, then used to construct a clinician's medical reasoning, and anticipate the results of different actions on the diagnostic process. An examination of the use of the digital based Integrated Management of Childhood Illness diagnostic algorithm was done, and an analysis of the software's efficiency in adequately diagnosing DKA was assessed. The main obstacles to diagnosis were low specificity of non-pathognomonic DKA symptoms and lack of tools to measure blood or urine glucose. Avenues for improvement include awareness of T1DM symptomatology in communities and health systems, and greater availability of diagnostic tests. Through this work clinical vignettes are shown to be a useful tool in analyzing the obstacles to underdiagnosis of diabetes, a technique that could be used for other pathologies in limited settings, for clinical teaching, research, and advocacy.

Effectiveness of Large-Scale Preparedness Training on Electrocardiogram for Medical, Surgical, and Pre-clinical Doctors: A Need-Based Initiative for COVID-19 Patient Care

INTRODUCTION

One of the competencies expected of all doctors posted in coronavirus disease 2019 (COVID-19) wards, is ECG rhythm identification, interpretation, and intervention for immediate management of patients. This study was undertaken to evaluate the effectiveness of the ECG training module as a component of preparedness training to combat COVID-19.

METHODOLOGY

This was a cross-sectional study conducted during training on ECG rhythm identification, interpretation, its management in COVID-19 patients. Study participants included faculty, senior residents, junior residents, and interns of medical, surgical, and paraclinical disciplines. The training session included one hour of didactic lecture and one and half hours of interactive session during which case scenarios were discussed. An objective assessment was conducted through pre-test and post-test. Mean of pre and post-test scores were compared using paired t-test for evaluating statistical significance. Feedback was also taken from participants.

RESULTS

Out of the 800 participants who gave consent, only 682 who completed both pre and post-test were included in the final analysis. Mean pre-test and post-test scores were 9.29/15 (61.9%) and 11.63/15 (77.5%), respectively, with a mean improvement of +2.34/15 (+15.6%). Of the participants, 38.6% obtained low scores in pre-test and 82% of respondents agreed that knowledge and skills gained from training would be useful in providing patient care.

CONCLUSION

 Low baseline knowledge on ECG highlights the need for re-training doctors posted in COVID-19 care on cardiac rhythm identification and interpretation. Interactive training is effective in improving ECG interpretative skills among doctors across disciplines and is the appropriate method to retrain/reskill, especially for large-scale capacity building.