Adaptation and External Validation of Pathogenic Urine Culture Prediction in Primary Care Using Machine Learning

Neural networks for the prediction of bacterial and fungal infections: current evidence and implications

In the present narrative review, we discuss the use of artificial neural networks (ANNs) for predicting bacterial and fungal infections based on commonly available clinical and laboratory data, focusing on promises and challenges of these machine learning models. For predicting different bacterial or fungal infections from data commonly found in electronical medical records, ANN models may reach, based on current literature, an acceptable performance for discriminating between infected and non-infected patients, and outperformed other machine learning (ML)-based models in 38.3% of the retrieved studies evaluating at least another ML approach. In the near future, as for other ML models, the use of ANNs could be leveraged to provide real-time support to clinicians in clinical decision-making processes, although further research is needed in terms of quality of data and explainability of ANN model predictions to better understand whether and how these techniques can be safely adopted in everyday clinical practice.

An artificial intelligence-driven support tool for prediction of urine culture test results

BACKGROUND AND AIMS

We aimed to develop an easily deployable artificial intelligence (AI)-driven model for rapid prediction of urine culture test results.

MATERIAL AND METHODS

We utilized a training dataset (n = 34,584 urine samples) and two separate, unseen test sets (n = 10,083 and 9,289 samples). Various machine learning models were compared for diagnostic performance. Predictive parameters included urinalysis results (dipstick and flow cytometry), patient demographics (age and gender), and sample collection method.

RESULTS

Although more complex models achieved the highest AUCs for predicting positive cultures (highest: multilayer perceptron (MLP) with AUC of 0.884, 95% CI 0.878-0.89), multiple logistic regression (MLR) using only flow cytometry parameters achieved a very good AUC (0.858, 95% CI 0.852-0.865). To aid interpretation, prediction results of the MLP and MLR models were categorized based on likelihood ratio (LR) for positivity: highly unlikely (LR 0.1), unlikely (LR 0.3), grey zone (LR 0.9), likely (LR 5.0), and highly likely (LR 40). This resulted in 17%, 28%, 34%, 9%, and 13% of samples falling into each respective category for the MLR model and 20%, 26%, 31%, 7%, and 16% for the MLP model.

CONCLUSIONS

In conclusion, this robust model has the potential to assist clinicians in their decision-making process by providing insights prior to the availability of urine culture results in a significant portion of samples (∼2/3rd).

Artificial intelligence and machine learning applications in urinary tract infections identification and prediction: a systematic review and meta-analysis

BACKGROUND

Urinary tract infections (UTIs) have been one of the most common bacterial infections in clinical practice worldwide. Artificial intelligence (AI) and machine learning (ML) based algorithms have been increasingly applied in UTI case identification and prediction. However, the overall performance of AI/ML algorithms in identifying and predicting UTI has not been evaluated. The purpose of this paper is to quantitatively evaluate the application value of AI/ML in identifying and predicting UTI cases.

METHODS

MEDLINE, EMBASE, Web of Science, and PubMed databases were systematically searched for articles published up to December 31, 2023. Quality Assessment of Diagnostic Accuracy Studies tool (QUADAS-2) and Prediction Model Risk of Bias Assessment Tool (PROBAST) were used to assess the risk of bias. Study characteristics and detailed algorithm information were extracted. Pooled sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were synthesized using a bivariate mix-effects model. Meta-regression and subgroup analysis were conducted to test the source of heterogeneity.

RESULTS

In total, 11 studies with 14 AI/ML models were included in the final meta-analysis. The overall pooled AUC was 0.89 (95%CI 0.86-0.92). Additionally, the pooled Sen, Spe, PLR, NLR, and DOR were 0.78 (95%CI 0.71-0.84), 0.89 (95%CI 0.83-0.93), 6.99 (95%CI 4.38-11.14), 0.25 (95%CI 0.18-0.34) and 28.07 (95%CI 14.27-55.20), respectively. The results of meta-regression suggested that reference standard definitions might be the source of heterogeneity.

CONCLUSION

AI/ML algorithms appear to be promising to help clinicians detect and identify patients at high risk of UTIs. However, further studies are demanded to evaluate the application value of AI/ML more thoroughly.

Smart Diagnosis of Urinary Tract Infections: is Artificial Intelligence the Fast-Lane Solution?

PURPOSE OF REVIEW

Artificial intelligence (AI) can significantly improve physicians' workflow when examining patients with UTI. However, most contemporary reviews are focused on examining the usage of AI with a restricted quantity of data, analyzing only a subset of AI algorithms, or performing narrative work without analyzing all dedicated studies. Given the preceding, the goal of this work was to conduct a mini-review to determine the current state of AI-based systems as a support in UTI diagnosis.

RECENT FINDINGS

There are sufficient publications to comprehend the potential applications of artificial intelligence in the diagnosis of UTIs. Existing research in this field, in general, publishes performance metrics that are exemplary. However, upon closer inspection, many of the available publications are burdened with flaws associated with the improper use of artificial intelligence, such as the use of a small number of samples, their lack of heterogeneity, and the absence of external validation. AI-based models cannot be classified as full-fledged physician assistants in diagnosing UTIs due to the fact that these limitations and flaws represent only a portion of all potential obstacles. Instead, such studies should be evaluated as exploratory, with a focus on the importance of future work that complies with all rules governing the use of AI. AI algorithms have demonstrated their potential for UTI diagnosis. However, further studies utilizing large, heterogeneous, prospectively collected datasets, as well as external validations, are required to define the actual clinical workflow value of artificial intelligence.