Validation of a clinical decision tree to predict if a patient has a bacteraemia due to a β-lactamase producing organism

Machine learning-based antibiotic resistance prediction models: An updated systematic review and meta-analysis

BACKGROUND

The widespread use of antibiotics has led to a gradual adaptation of bacteria to these drugs, diminishing the effectiveness of treatments.

OBJECTIVE

To comprehensively assess the research progress of antibiotic resistance prediction models based on machine learning (ML) algorithms, providing the latest quantitative analysis and methodological evaluation.

METHODS

Relevant literature was systematically retrieved from databases, including PubMed, Embase and the Cochrane Library, from inception up to December 2023. Studies meeting predefined criteria were selected for inclusion. The prediction model risk of bias assessment tool was employed for methodological quality assessment, and a random-effects model was utilised for meta-analysis.

RESULTS

The systematic review included a total of 22 studies with a combined sample size of 43,628; 10 studies were ultimately included in the meta-analysis. Commonly used ML algorithms included random forest, decision trees and neural networks. Frequently utilised predictive variables encompassed demographics, drug use history and underlying diseases. The overall sensitivity was 0.57 (95% CI: 0.42-0.70; p< 0.001; I2= 99.7%), the specificity was 0.95 (95% CI: 0.79-0.99; p< 0.001; I2 = 99.9%), the positive likelihood ratio was 10.7 (95% CI: 2.9-39.5), the negative likelihood ratio was 0.46 (95% CI: 0.34-0.61), the diagnostic odds ratio was 23 (95% CI: 7-81) and the area under the receiver operating characteristic curve was 0.78 (95% CI: 0.74-0.81; p< 0.001), indicating a good discriminative ability of ML models for antibiotic resistance. However, methodological assessment and funnel plots suggested a high risk of bias and publication bias in the included studies.

CONCLUSION

This meta-analysis provides a current and comprehensive evaluation of ML models for predicting antibiotic resistance, emphasising their potential application in clinical practice. Nevertheless, stringent research design and reporting are warranted to enhance the quality and credibility of future studies. Future research should focus on methodological innovation and incorporate more high-quality studies to further advance this field.

Machine learning to predict antimicrobial resistance: future applications in clinical practice?

INTRODUCTION

Machine learning (ML) is increasingly being used to predict antimicrobial resistance (AMR). This review aims to provide physicians with an overview of the literature on ML as a means of AMR prediction.

METHODS

References for this review were identified through searches of MEDLINE/PubMed, EMBASE, Google Scholar, ACM Digital Library, and IEEE Xplore Digital Library up to December 2023.

RESULTS

Thirty-six studies were included in this review. Thirty-two studies (32/36, 89 %) were based on hospital data and four (4/36, 11 %) on outpatient data. The vast majority of them were conducted in high-resource settings (33/36, 92 %). Twenty-four (24/36, 67 %) studies developed systems to predict drug resistance in infected patients, eight (8/36, 22 %) tested the performances of ML-assisted antibiotic prescription, two (2/36, 6 %) assessed ML performances in predicting colonization with carbapenem-resistant bacteria and, finally, two assessed national and international AMR trends. The most common inputs were demographic characteristics (25/36, 70 %), previous antibiotic susceptibility testing (19/36, 53 %) and prior antibiotic exposure (15/36, 42 %). Thirty-three (92 %) studies targeted prediction of Gram-negative bacteria (GNB) resistance as an output (92 %). The studies included showed moderate to high performances, with AUROC ranging from 0.56 to 0.93.

CONCLUSION

ML can potentially provide valuable assistance in AMR prediction. Although the literature on this topic is growing, future studies are needed to design, implement, and evaluate the use and impact of ML decision support systems.

Using Machine Learning to Predict Antimicrobial Resistance-A Literature Review

Machine learning (ML) algorithms are increasingly applied in medical research and in healthcare, gradually improving clinical practice. Among various applications of these novel methods, their usage in the combat against antimicrobial resistance (AMR) is one of the most crucial areas of interest, as increasing resistance to antibiotics and management of difficult-to-treat multidrug-resistant infections are significant challenges for most countries worldwide, with life-threatening consequences. As antibiotic efficacy and treatment options decrease, the need for implementation of multimodal antibiotic stewardship programs is of utmost importance in order to restrict antibiotic misuse and prevent further aggravation of the AMR problem. Both supervised and unsupervised machine learning tools have been successfully used to predict early antibiotic resistance, and thus support clinicians in selecting appropriate therapy. In this paper, we reviewed the existing literature on machine learning and artificial intelligence (AI) in general in conjunction with antimicrobial resistance prediction. This is a narrative review, where we discuss the applications of ML methods in the field of AMR and their value as a complementary tool in the antibiotic stewardship practice, mainly from the clinician's point of view.

Machine learning in predicting antimicrobial resistance: a systematic review and meta-analysis

INTRODUCTION

Antimicrobial resistance (AMR) is a global health threat; rapid and timely identification of AMR improves patient prognosis and reduces inappropriate antibiotic use.

METHODS

Relevant literature in PubMed, Web of Science, Embase and Institute of Electrical and Electronics Engineers prior to 28 September 2021 was searched. Any study that deployed machine learning (ML) or a risk score as a tool to predict AMR was included in the final review; there were 25 studies that employed the ML algorithm to predict AMR.

RESULTS

Extended spectrum β-lactamases, methicillin-resistant Staphylococcus aureus (MRSA) and carbapenem resistance were the most common outcomes in studies with a specific resistance pattern. The most common algorithms in ML prediction were logistic regression (n = 14 studies), decision tree (n = 14) and random forest (n = 7). The area under the curve (AUC) range for ML prediction was 0.48-0.93. The pooled AUC for ML prediction was 0.82 (0.78-0.85). Compared with risk score, higher specificity [87% (82-91) vs. 37% (25-51)] was indicated for ML prediction, but not sensitivity [67% (62-72) vs. 73% (67-79)].

CONCLUSIONS

Machine learning might be a potential technology for AMR prediction; however, retrospective methodology for model development, nonstandard data processing and scarcity of validation in a randomised controlled trial or real-world study limit the application of these models in clinical practice.

Comparison of the performance of a clinical classification tree versus clinical gestalt in predicting sepsis with extended-spectrum beta-lactamase–producing gram-negative rods

A clinical decision tree was developed using point-of-care characteristics to identify patients with culture-proven sepsis due to extended-spectrum β-lactamase–producing Enterobacterales (ESBL-PE). We compared its performance with the clinical gestalt of emergency department (ED) clinicians and hospital-based clinicians. The developed tree outperformed ED-based clinicians but was comparable to inpatient-based clinicians.

An international prospective cohort study to validate two prediction rules for infections caused by 3rd-generation cephalosporin-resistant Enterobacterales

Background

The possibility of bloodstream infections caused by third-generation cephalosporin-resistant Enterobacterales (3GC-R-BSI) leads to a trade-off between empiric inappropriate treatment (IAT) and unnecessary carbapenem use (UCU). Accurately predicting 3GC-R-BSI could reduce IAT and UCU. We externally validate 2 previously derived prediction rules for community-onset (CO) and hospital-onset (HO) suspected bloodstream infections.

Methods

In 33 hospitals in 13 countries we prospectively enrolled 200 patients per hospital in whom blood cultures were obtained and intravenous antibiotics with coverage for Enterobacterales were empirically started. Cases were defined as 3GC-R-BSI or 3GC-R gram-negative infection (3GC-R-GNI) (analysis 2); all other outcomes served as a comparator. Model discrimination and calibration were assessed. Impact on carbapenem use was assessed at several cutoff points.

Results

4650 CO infection episodes were included and the prevalence of 3GC-R-BSI was 2.1% (n = 97). IAT occurred in 69 of 97 (71.1%) 3GC-R-BSI and UCU in 398 of 4553 non–3GC-R-BSI patients (8.7%). Model calibration was good, and the AUC was .79 (95% CI, .75–.83) for 3GC-R-BSI. The prediction rule potentially reduced IAT to 62% (60/97) while keeping UCU comparable at 8.4% or could reduce UCU to 6.3% (287/4553) while keeping IAT equal. IAT and UCU in all 3GC-R-GNIs (analysis 2) improved at similar percentages. 1683 HO infection episodes were included and the prevalence of 3GC-R-BSI was 4.9% (n = 83). Here model calibration was insufficient.

Conclusions

A prediction rule for CO 3GC-R infection was validated in an international cohort and could improve empirical antibiotic use. Validation of the HO rule yielded suboptimal performance.