Real-World Antimicrobial Stewardship Experience in a Large Academic Medical Center: Using Statistical and Machine Learning Approaches to Identify Intervention “Hotspots” in an Antibiotic Audit and Feedback Program

Artificial intelligence in antimicrobial stewardship: a systematic review and meta-analysis of predictive performance and diagnostic accuracy

The increasing threat of antimicrobial resistance has prompted a need for more effective antimicrobial stewardship programs (AMS). Artificial intelligence (AI) and machine learning (ML) tools have emerged as potential solutions to enhance decision-making and improve patient outcomes in AMS. This systematic review and meta-analysis aims to evaluate the impact of AI in AMS and to assess its predictive performance and diagnostic accuracy. We conducted a comprehensive literature search across PubMed/MEDLINE, Scopus, EMBASE, and Web of Science to identify studies published up to July 2024. Studies included were observational, cohort, or retrospective, focusing on the application of AI/ML in AMS. The outcomes assessed were the area under the curve (AUC), accuracy, sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV). We calculated the mean pooled effect size (ES) and its 95% confidence interval (CI) using a random-effects model. The risk of bias was assessed using the QUADAS-AI tool, and the protocol was registered in PROSPERO. Out of 3,458 retrieved articles, 80 studies met the inclusion criteria. Our meta-analysis demonstrated that ML models exhibited strong predictive performance and diagnostic accuracy, with the following results: AUC [ES: 72.28 (70.42-74.14)], accuracy [ES: 74.97 (73.35-76.58)], sensitivity [ES: 76.89; (71.90-81.89)], specificity [ES: 73.77; (67.87-79.67)], NPV [ES:79.92 (76.54-83.31)], and PPV [ES: 69.41 (60.19-78.63)] across various AMS settings. AI and ML tools offer promising enhancements due to their strong predictive performance. The integration of AI into AMS could lead to more precise antimicrobial prescribing, reduced antimicrobial resistance, and better resource utilization.

Prospective Audit and Feedback of Targeted Antimicrobials Use at a Tertiary Care Hospital in the United Arab Emirates

Background/Objectives: Antimicrobial stewardship programs improve antimicrobial use and help combat antimicrobial resistance. The Infectious Disease Society of America's (IDSA) recommended core interventions include prospective audit and feedback along with formulary restriction and preauthorization. IDSA recommends any one of these interventions be implemented in acute care hospitals to improve antimicrobial stewardship. The objective of this project was to implement a prospective audit and feedback system using selected antimicrobials at a tertiary care hospital in the United Arab Emirates as the foundation to build an antimicrobial stewardship program. Results: A total of 497 patients met the inclusion and exclusion criteria during the study period; the post-intervention group had 260 patients, and the control group had 237 patients. After the implementation of the program, a total of 186 interventions were recommended, and 76% were accepted. The length of stay, length of therapy, and days of therapy were lower in the intervention group compared to the control group (p < 0.05). There was no statistically significant difference in clinical outcome measures (e.g., 30-day readmission, 30-day all-cause mortality, 30-day emergency visit with the same infection, and 60-day readmission). Methods: This single-center quasi-experimental research was conducted from August 2023 to July 2024. A pharmacist-led prospective audit and feedback system was initiated in February 2024 after review and approval of the medical staff, in addition to formulary restrictions. Data from patients receiving the selected antimicrobial before February 2024 were collected from their charts and related medical records without any intervention; this was used by our control group. After implementation, the hospital pharmacy's records were evaluated during the night shift to determine whether they met the inclusion criteria. The records of the eligible patients were then evaluated by the clinical pharmacist. In case of antimicrobial inappropriateness, feedback was provided to the prescriber. If the recommendation was not accepted, succeeding reviews and feedback were provided on subsequent days. The effectiveness of the intervention was measured using clinical and antibiotic use measures. Conclusions: Implementation of a pilot pharmacist-led antimicrobial stewardship program resulted in modification in antimicrobial use measures (i.e., defined daily doses of targeted antimicrobials and days of antimicrobial therapy) without an increase in length of stay or readmissions or mortality.

Artificial intelligence and prescription of antibiotic therapy: present and future

INTRODUCTION

In the past few years, the use of artificial intelligence in healthcare has grown exponentially. Prescription of antibiotics is not exempt from its rapid diffusion, and various machine learning (ML) techniques, from logistic regression to deep neural networks and large language models, have been explored in the literature to support decisions regarding antibiotic prescription.

AREAS COVERED

In this narrative review, we discuss promises and challenges of the application of ML-based clinical decision support systems (ML-CDSSs) for antibiotic prescription. A search was conducted in PubMed up to April 2024.

EXPERT OPINION

Prescribing antibiotics is a complex process involving various dynamic phases. In each of these phases, the support of ML-CDSSs has shown the potential, and also the actual ability in some studies, to favorably impacting relevant clinical outcomes. Nonetheless, before widely exploiting this massive potential, there are still crucial challenges ahead that are being intensively investigated, pertaining to the transparency of training data, the definition of the sufficient degree of prediction explanations when predictions are obtained through black box models, and the legal and ethical framework for decision responsibility whenever an antibiotic prescription is supported by ML-CDSSs.

Explainable and Interpretable Machine Learning for Antimicrobial Stewardship: Opportunities and Challenges

There is growing interest in exploiting the advances in artificial intelligence and machine learning (ML) for improving and monitoring antimicrobial prescriptions in line with antimicrobial stewardship principles. Against this background, the concepts of interpretability and explainability are becoming increasingly essential to understanding how ML algorithms could predict antimicrobial resistance or recommend specific therapeutic agents, to avoid unintended biases related to the "black box" nature of complex models. In this commentary, we review and discuss some relevant topics on the use of ML algorithms for antimicrobial stewardship interventions, highlighting opportunities and challenges, with particular attention paid to interpretability and explainability of employed models. As in other fields of medicine, the exponential growth of artificial intelligence and ML indicates the potential for improving the efficacy of antimicrobial stewardship interventions, at least in part by reducing time-consuming tasks for overwhelmed health care personnel. Improving our knowledge about how complex ML models work could help to achieve crucial advances in promoting the appropriate use of antimicrobials, as well as in preventing antimicrobial resistance selection and dissemination.

Brave New World of Artificial Intelligence: Its Use in Antimicrobial Stewardship-A Systematic Review

Antimicrobial resistance (AMR) is a growing public health problem in the One Health dimension. Artificial intelligence (AI) is emerging in healthcare, since it is helpful to deal with large amounts of data and as a prediction tool. This systematic review explores the use of AI in antimicrobial stewardship programs (ASPs) and summarizes the predictive performance of machine learning (ML) algorithms, compared with clinical decisions, in inpatients and outpatients who need antimicrobial prescriptions. This review includes eighteen observational studies from PubMed, Scopus, and Web of Science. The exclusion criteria comprised studies conducted only in vitro, not addressing infectious diseases, or not referencing the use of AI models as predictors. Data such as study type, year of publication, number of patients, study objective, ML algorithms used, features, and predictors were extracted from the included publications. All studies concluded that ML algorithms were useful to assist antimicrobial stewardship teams in multiple tasks such as identifying inappropriate prescribing practices, choosing the appropriate antibiotic therapy, or predicting AMR. The most extracted performance metric was AUC, which ranged from 0.64 to 0.992. Despite the risks and ethical concerns that AI raises, it can play a positive and promising role in ASP.

Confronting the Disruption of the Infectious Diseases Workforce by Artificial Intelligence: What This Means for Us and What We Can Do About It

With the rapid advancement of artificial intelligence (AI), the field of infectious diseases (ID) faces both innovation and disruption. AI and its subfields including machine learning, deep learning, and large language models can support ID clinicians' decision making and streamline their workflow. AI models may help ensure earlier detection of disease, more personalized empiric treatment recommendations, and allocation of human resources to support higher-yield antimicrobial stewardship and infection prevention strategies. AI is unlikely to replace the role of ID experts, but could instead augment it. However, its limitations will need to be carefully addressed and mitigated to ensure safe and effective implementation. ID experts can be engaged in AI implementation by participating in training and education, identifying use cases for AI to help improve patient care, designing, validating and evaluating algorithms, and continuing to advocate for their vital role in patient care.

Development of machine learning algorithms for scaling-up antibiotic stewardship

BACKGROUND

Antibiotic stewardship programs (ASP) aim to reduce inappropriate use of antibiotics, but their labor-intensive nature impedes their wide adoption. The present study introduces explainable machine learning (ML) models designed to prioritize inpatients who would benefit most from stewardship interventions.

METHODS

A cohort of inpatients who received systemic antibiotics and were monitored by a multidisciplinary ASP team at a tertiary hospital in the Republic of Korea was assembled. Data encompassing over 130,000 patient-days and comprising more than 160 features from multiple domains, including prescription records, laboratory, microbiology results, and patient conditions was collected.Outcome labels were generated using medication administration history: discontinuation, switching from intravenous to oral medication (IV to PO), and early or late de-escalation. The models were trained using Extreme Gradient Boosting (XGB) and light Gradient Boosting Machine (LGBM), with SHapley Additive exPlanations (SHAP) analysis used to explain the model's predictions.

RESULTS

The models demonstrated strong discrimination when evaluated on a hold-out test set(AUROC - IV to PO: 0.81, Early de-escalation: 0.78, Late de-escalation: 0.72, Discontinue: 0.80). The models identified 41%, 16%, 22%, and 17% more cases requiring discontinuation, IV to PO, early and late de-escalation, respectively, compared to the conventional length of therapy strategy, given that the same number of patients were reviewed by the ASP team. The SHAP results explain how each model makes their predictions, highlighting a unique set of important features that are well-aligned with the clinical intuitions of the ASP team.

CONCLUSIONS

The models are expected to improve the efficiency of ASP activities by prioritizing cases that would benefit from different types of ASP interventions along with detailed explanations.

What Is Left to Tackle in Inpatient Antimicrobial Stewardship Practice and Research

Despite widespread uptake of antimicrobial stewardship in acute care hospitals, there is ongoing need for innovation and optimization of ASPs. This article discusses current antimicrobial stewardship practice challenges and ways to improve current antimicrobial stewardship workflows. Additionally, we propose new workflows that further engage front line clinicians in optimizing their own antibiotic decision making.

Principles and Practice of Antimicrobial Stewardship Program Resource Allocation

Antimicrobial Stewardship Programs (ASP) improve individual patient outcomes and clinical care processes while reducing antimicrobial-associated adverse events, optimizing operational priorities, and providing institutional cost savings. ASP composition, resources required, and priority focuses are influenced by myriad factors. Despite robust evidence and broad national support, individual ASPs still face challenges in obtaining appropriate resources. Though understanding the current landscape of ASP resource allocation, factors influencing staffing needs, and strategies required to obtain desired resources is important, acceptance of recommended staffing levels and appropriate ASP resource allocation are much needed to facilitate ASP sustainability and growth across the complex and diverse health care continuum.

Improving the Efficiency of Antimicrobial Stewardship Action in Acute Care Facilities

Inpatient antimicrobial stewardship (AS) programs are quality improvement programs tasked with improving antibiotic practices by augmenting frontline providers' antibiotic prescription. Prospective audit and feedback (PAF) and preauthorization (PRA) are essential activities in the hospital that can be resource intensive for AS teams. Improving efficiency in AS activities is needed when there are limited resources or when programs are looking to expand tasks beyond PAF and PRA, such as broad education or guideline development. Guidance on the creation and maintenance of alerts for the purpose of PAF reviews, modifications of antibiotic restrictions for PRA polices, and overall initiative prioritization strategies are reviewed. In addition, daily prioritization tools, such as the tiered approach, scoring systems, and regression modeling, are available for stewards to prioritize their daily workflow. Using these tools and guidance, AS programs can be productive and impactful in the face of resource limitation or competing priorities in the hospital.