Recognition & management of varicella infections and accuracy of antimicrobial recommendations: Case vignettes study in the US

GP or ChatGPT? Ability of large language models (LLMs) to support general practitioners when prescribing antibiotics

INTRODUCTION

Large language models (LLMs) are becoming ubiquitous and widely implemented. LLMs could also be used for diagnosis and treatment. National antibiotic prescribing guidelines are customized and informed by local laboratory data on antimicrobial resistance.

METHODS

Based on 24 vignettes with information on type of infection, gender, age group and comorbidities, GPs and LLMs were prompted to provide a treatment. Four countries (Ireland, UK, USA and Norway) were included and a GP from each country and six LLMs (ChatGPT, Gemini, Copilot, Mistral AI, Claude and Llama 3.1) were provided with the vignettes, including their location (country). Responses were compared with the country's national prescribing guidelines. In addition, limitations of LLMs such as hallucination, toxicity and data leakage were assessed.

RESULTS

GPs' answers to the vignettes showed high accuracy in relation to diagnosis (96%-100%) and yes/no antibiotic prescribing (83%-92%). GPs referenced (100%) and prescribed (58%-92%) according to national guidelines, but dose/duration of treatment was less accurate (50%-75%). Overall, the GPs' accuracy had a mean of 74%. LLMs scored high in relation to diagnosis (92%-100%), antibiotic prescribing (88%-100%) and the choice of antibiotic (59%-100%) but correct referencing often failed (38%-96%), in particular for the Norwegian guidelines (0%-13%). Data leakage was shown to be an issue as personal information was repeated in the models' responses to the vignettes.

CONCLUSIONS

LLMs may be safe to guide antibiotic prescribing in general practice. However, to interpret vignettes, apply national guidelines and prescribe the right dose and duration, GPs remain best placed.

Validation of the Clinical Performance and Reproducibility of the Savanna HSV 1+2/VZV Assay

Herpes simplex virus 1 (HSV-1), HSV 2 (HSV-2), and varicella-zoster virus (VZV) cause nondescript cutaneous and mucocutaneous lesions requiring rapid, differential identification for appropriate diagnosis and patient counseling. Decentralized multiplex molecular assays may provide more rapid results than existing methodologies but require clinical validation. This multicenter study evaluated the clinical performance of the Savanna HSV 1+2/VZV Assay against the high-complexity Lyra Direct HSV 1+2/VZV real-time PCR nucleic acid test for the detection of HSV-1, HSV-2, and VZV from clinical specimens. The Savanna HSV 1+2/VZV Assay is an automated, moderate-complexity, real-time PCR assay recently cleared by the US Food and Drug Administration for the simultaneous detection and differentiation of HSV-1, HSV-2, and VZV DNA isolated from lesion swabs. In this study, 744 clinical specimens (531 female, 213 male) were evaluated by Savanna and compared with Lyra. Discrepant result analysis was conducted with the moderate-complexity Solana HSV 1+2/VZV isothermal nucleic acid test. For 744 clinical samples, Savanna exhibited overall, positive, and negative percent agreement of 99.5%, 100%, and 99.3% for HSV-1; 99.9%, 100%, and 99.8% for HSV-2; and 100%, 100%, and 100% for VZV. The Savanna HSV 1+2/VZV Assay exhibited excellent performance in a multicenter, clinical study. Savanna can provide laboratory-equivalent results outside of the central laboratory with the potential to deliver accurate results during the patient visit.

Case report: Hospital-acquired chickenpox in a healthcare setting

Chickenpox (varicella) is a rare occurrence in healthcare settings in the USA, but can be transmitted to healthcare workers (HCWs) from patients with herpes zoster who, in turn, can potentially transmit it further to unimmunized, immunosuppressed, at-risk, vulnerable patients. It is uncommon due to the inclusion of varicella vaccination in the recommended immunization schedule for children and screening for varicella immunity in HCWs during employment. We present a case report of hospital-acquired chickenpox in a patient who developed the infection during his prolonged hospital stay through a HCW who had contracted chickenpox after exposure to our patient's roommate with herpes zoster. There was no physical contact between the roommates, but both patients had a common HCW as caregiver. The herpes zoster patient was placed in airborne precautions immediately, but the HCW continued to work and have physical contact with our patient. The HCW initially developed chickenpox 18 days after exposure to the patient with herpes zoster, and our patient developed chickenpox 17 days after the HCW. The timeline and two incubation periods, prior to our patient developing chickenpox, indicate transmission of chickenpox in the HCW from exposure to the herpes zoster patient and subsequently to our patient. The case highlights the potential for nosocomial transmission of chickenpox (varicella) to unimmunized HCWs from exposure to patients with herpes zoster and further transmission to unimmunized patients. Verification of the immunization status of HCWs at the time of employment, mandating immunity, furloughing unimmunized staff after exposure to herpes zoster, and postexposure prophylaxis with vaccination or varicella zoster immunoglobulin (Varizig) will minimize the risk of transmission of communicable diseases like chickenpox in healthcare settings. Additionally, establishing patients' immunity, heightened vigilance and early identification of herpes zoster in hospitalized patients, and initiation of appropriate infection control immediately will further prevent such occurrences and improve patient safety.

This is a case report of a varicella-unimmunized 31-year-old patient who developed chickenpox during his 80-day-long hospitalization. He had different roommates during his long hospital stay but had no physical contact with them and neither had visitors. On most days, the same HCW rendered care to him and his roommates. One of the patient’s roommates was found to have herpes zoster and was immediately moved to a different room with appropriate infection prevention measures. The HCW is presumably unimmunized to varicella and sustained significant exposure to the patient with herpes zoster during routine patient care which involved significant physical contact. The HCW was not furloughed, assessed for immunity, or given postexposure prophylaxis (PEP). The HCW had continued contact with our patient as part of routine care. On day 18, after exposure to the patient with herpes zoster, the HCW developed chickenpox. 17 days thereafter, our patient developed chickenpox. The time interval of chickenpox infection in the HCW after one incubation period after exposure to the patient with herpes zoster followed by a similar infection of chickenpox in our patient after another incubation period suggests the spread of varicella zoster virus (VZV) from the herpes zoster patient to the HCW and further from the HCW to our patient. Assessing the immunity of HCWs to varicella at the time of employment, ensuring only HCWs with immunity take care of herpes zoster and varicella patients, furloughing unimmunized exposed HCWs, offering PEP, and documentation of patients’ immunity to varicella at the time of hospital admission could help prevent VZV transmission in hospital settings. This is an attempt to publish this novel case due to its high educational value and relevant learning points.