Predicting urinary tract infections in the emergency department with machine learning

Role of immature granulocyte and blood biomarkers in predicting perforated acute appendicitis using machine learning model

BACKGROUND

Acute appendicitis (AAp) is a prevalent medical condition characterized by inflammation of the appendix that frequently necessitates urgent surgical procedures. Approximately two-thirds of patients with AAp exhibit characteristic signs and symptoms; hence, negative AAp and complicated AAp are the primary concerns in research on AAp. In other terms, further investigations and algorithms are required for at least one third of patients to predict the clinical condition and distinguish them from uncomplicated patients with AAp.

AIM

To use a Stochastic Gradient Boosting (SGB)-based machine learning (ML) algorithm to tell the difference between AAp patients who are complicated and those who are not, and to find some important biomarkers for both types of AAp by using modeling to get variable importance values.

METHODS

This study analyzed an open access data set containing 140 people, including 41 healthy controls, 65 individuals with uncomplicated AAp, and 34 individuals with complicated AAp. We analyzed some demographic data (age, sex) of the patients and the following biochemical blood parameters: White blood cell (WBC) count, neutrophils, lymphocytes, monocytes, platelet count, neutrophil-to-lymphocyte ratio, lymphocyte-to-monocyte ratio, mean platelet volume, neutrophil-to-immature granulocyte ratio, ferritin, total bilirubin, immature granulocyte count, immature granulocyte percent, and neutrophil-to-immature granulocyte ratio. We tested the SGB model using n-fold cross-validation. It was implemented with an 80-20 training-test split. We used variable importance values to identify the variables that were most effective on the target.

RESULTS

The SGB model demonstrated excellent performance in distinguishing AAp from control patients with an accuracy of 96.3%, a micro aera under the curve (AUC) of 94.7%, a sensitivity of 94.7%, and a specificity of 100%. In distinguishing complicated AAp patients from uncomplicated ones, the model achieved an accuracy of 78.9%, a micro AUC of 79%, a sensitivity of 83.3%, and a specificity of 76.9%. The most useful biomarkers for confirming the AA diagnosis were WBC (100%), neutrophils (95.14%), and the lymphocyte-monocyte ratio (76.05%). On the other hand, the most useful biomarkers for accurate diagnosis of complicated AAp were total bilirubin (100%), WBC (96.90%), and the neutrophil-immature granulocytes ratio (64.05%).

CONCLUSION

The SGB model achieved high accuracy rates in identifying AAp patients while it showed moderate performance in distinguishing complicated AAp patients from uncomplicated AAp patients. Although the model's accuracy in the classification of complicated AAp is moderate, the high variable importance obtained is clinically significant. We need further prospective validation studies, but the integration of such ML algorithms into clinical practice may improve diagnostic processes.

Enhancing clinical decision support with physiological waveforms - A multimodal benchmark in emergency care

BACKGROUND

AI-driven prediction algorithms have the potential to enhance emergency medicine by enabling rapid and accurate decision-making regarding patient status and potential deterioration. However, the integration of multimodal data, including raw waveform signals, remains underexplored in clinical decision support.

METHODS

We present a dataset and benchmarking protocol designed to advance multimodal decision support in emergency care. Our models utilize demographics, biometrics, vital signs, laboratory values, and electrocardiogram (ECG) waveforms as inputs to predict both discharge diagnoses and patient deterioration.

RESULTS

The diagnostic model achieves area under the receiver operating curve (AUROC) scores above 0.8 for 609 out of 1,428 conditions, covering both cardiac (e.g., myocardial infarction) and non-cardiac (e.g., renal disease, diabetes) diagnoses. The deterioration model attains AUROC scores above 0.8 for 14 out of 15 targets, accurately predicting critical events such as cardiac arrest, mechanical ventilation, ICU admission, and mortality.

CONCLUSIONS

Our study highlights the positive impact of incorporating raw waveform data into decision support models, improving predictive performance. By introducing a unique, publicly available dataset and baseline models, we provide a foundation for measurable progress in AI-driven decision support for emergency care.

Comparison of three classifiers in detection of obstruction of the lower urinary tract using recorded sounds of voiding

The aim of this research is to help health care professionals to automatically detect lower urinary tract disorders using sounds of voiding recorded at home. In total 93 patients were diagnosed as obstructed or non-obstructed in a hospital using traditional flow-metering technique. After they went to their houses to collect several micturition recordings (5-13 records per patient) by themselves using their Oppo smart watch. Our proposed method is based on the use of the wavelet scalogram to represent the collected sounds as images, which contains both time and frequency information. A deep learning model, the inception v3 convolutional neural network, is used to classify these recordings of the voiding into the categories of obstructed and non-obstructed. We compared the performance of our approach with classical techniques such as Support Vector Machine (SVM) and Multilayer Perceptron (MLP) using the envelope of the superposed sounds per patient as inputs. These recordings were obtained in home environments. The ground truth was built by physicians' labeling these sound recording. They used the gold standard uroflowmetry test, which gave them all the information to classify the patients as either obstructed or non-obstructed. The performance of the model in terms of the F1 score, accuracy, and area under the curve were 0.897, 0.891 and 0.901, respectively.

Enhancing Antibiotic Stewardship: A Machine Learning Approach to Predicting Antibiotic Resistance in Inpatient Care

Antibiotics have been crucial in advancing medical treatments, but the growing threat of antibiotic resistance challenges these achievements and emphasizes the need for innovative stewardship strategies. In this study, we developed machine learning models, 'personalized antibiograms', to predict antibiotic resistance across five key antibiotics using Stanford's electronic health record data of 49,872 urine, blood, and respiratory infections. We aimed to ascertain the efficacy of these models in predicting antibiotic susceptibility and identify the clinical factors most indicative of resistance. Employing LightGBM, we incorporated demographics, prior resistance, prescriptions, and comorbidities as features. The models demonstrated notable discriminative ability, with AUROCs between 0.74 and 0.78, and highlighted prior resistance and prescriptions as significant predictive factors. The high specificity demonstrates machine learning models' potential to inform antibiotic de-escalation, aiding stewardship without risking safety. By leveraging machine learning with relevant clinical features, we show that it is feasible to improve empirical antibiotic prescribing.

Comparative Analysis of Machine Learning Models for Predicting Hospital- and Community-Associated Urinary Tract Infections Using Demographic, Hospital, and Socioeconomic Predictors

BACKGROUND

Urinary tract infections (UTI) are among the most common infections encountered in both community and healthcare settings. Differentiating between community-associated UTI (CA-UTI) and healthcare-associated UTI (HA-UTI) is crucial for understanding their epidemiology, identifying risk factors, and developing appropriate treatment strategies. Machine learning (ML) techniques have shown significant potential in improving the accuracy of predicting these infections, enabling more effective interventions and better patient outcomes. While previous studies have demonstrated the utility of ML models in various healthcare settings, there is still a need for a comparative analysis of different ML approaches, particularly in distinguishing between CA-UTI and HA-UTI and assessing the risk of UTI among hospitalized patients.

OBJECTIVE

Using 2019-2023 patient demographics, hospital, and socioeconomic data, this study aims to build, validate, and compare machine learning models-Decision Tree (DT), Neural Network (NN), Logistic Regression (LR), Random Forest (RF), and Extreme Gradient Boosting (XGBoost) to differentiate between the incidences of HA-UTI and CA-UTI. Additionally, it seeks to identify key predictors of UTI using demographic, hospital, and socioeconomic variables.

RESULTS

The DT model demonstrated the highest sensitivity, particularly in handling the highly imbalanced data of HAI, with a sensitivity of 87%. LR achieved the best overall accuracy, at 95.9% for HA-UTI and 93.2% for HA-UTI vs. CA-UTI. RF performed best in cross-validation, reaching 99.1% for HA-UTI and 96.2% for HA-UTI vs. CA-UTI. NN showed the highest specificity, at 93.4%, for HA-UTI vs. CA-UTI. The AUC values further supported these findings, ranging from 71.9% for NN to 96% for RF, reflecting the robustness of these models across different annual datasets. Among patient demographics, hospital, and socioeconomic variables, all models consistently identified the nurse units (e.g., inpatient units and mental health units) as the most significant predictors of UTI. In addition to nurse units, LR and DT identified location (e.g., various clinics and medical centres) as a key predictor. For HA-UTI versus CA-UTI, variations were observed across the years, with patient age, median household income, and gender intermittently emerging as key predictors.

CONCLUSION

The predictive accuracy of the machine learning models is relatively the same, with some differences in sensitivity and specificity for identifying both HA-UTI vs. CA-UTI and HA-UTI. Nurse units consistently emerge as the most significant predictors across all years. The importance of all predictors, such as socioeconomic factors and location, varies from year to year, highlighting the need for incorporating those variables in the surveillance systems to optimize the accuracy of predictions.

Integrating socioeconomic deprivation indices and electronic health record data to predict antimicrobial resistance

We developed machine learning models to predict the presence of AMR organisms in blood cultures obtained at the first patient encounter, offering a new and inspiring direction for antimicrobial resistance management. Three supervised machine learning classifiers were used: penalized logistic regression, random forest, and XGBoost, which were used to classify five AMR organisms: ESBL, CRE, AmpC, MRSA, and VRE. The random forest and XGBoost models performed best, with AUC-ROC values of 0.70 and 92.9% negative predictive value, respectively. The multi-class random forest model's AUC-ROC values ranged from 0.80-0.95. Our models highlight how the combination of ADI and SVI increased the predictive power. This approach could reduce costs and mitigate the global public health threat posed by antibiotic-resistant infections. Machine learning techniques can predict antimicrobial-resistant infections in suspected cultures using patient data from EHRs, enabling clinicians to make targeted prescribing decisions and mitigate resistance development.

Comparison of Logistic Regression and Machine Learning Approaches in Predicting Depressive Symptoms: A National-Based Study

OBJECTIVE

Machine learning (ML) has been reported to have better predictive capability than traditional statistical techniques. The aim of this study was to assess the efficacy of ML algorithms and logistic regression (LR) for predicting depressive symptoms during the COVID-19 pandemic.

METHODS

Analyses were carried out in a national cross-sectional study involving 21,916 participants. The ML algorithms in this study included random forest (RF), support vector machine (SVM), neural network (NN), and gradient boosting machine (GBM) methods. The performance indices were sensitivity, specificity, accuracy, precision, F1-score, and area under the receiver operating characteristic curve (AUC).

RESULTS

LR and NN had the best performance in terms of AUCs. The risk of overfitting was found to be negligible for most ML models except for RF, and GBM obtained the highest sensitivity, specificity, accuracy, precision, and F1-score. Therefore, LR, NN, and GBM models ranked among the best models.

CONCLUSION

Compared with ML models, LR model performed comparably to ML models in predicting depressive symptoms and identifying potential risk factors while also exhibiting a lower risk of overfitting.

Cephalosporins for Outpatient Pyelonephritis in the Emergency Department: COPY-ED Study

STUDY OBJECTIVE

The primary objective of our study was to compare the effectiveness of oral cephalosporins versus fluroquinolones and trimethoprim/sulfamethoxazole (TMP-SMX) for the treatment of pyelonephritis in patients discharged home from the emergency department (ED).

METHODS

This was a multicenter, retrospective, observational cohort study of 11 geographically diverse US EDs. Patients aged ≥18 years diagnosed with pyelonephritis and discharged home from the ED between January 1, 2021 and October 31, 2023 were included. The primary outcome was treatment failure at 14 days defined as a composite outcome of the following: (1) recurrence of urinary symptoms, (2) repeat ED visit or hospitalization for a urinary tract infection, (3) receipt of a new antibiotic prescription for urinary tract infection. Secondary outcome was appropriateness of empiric treatment based on urine culture susceptibility.

RESULTS

Among the 851 patients who met inclusion criteria, 647 patients received a cephalosporin, and 204 patients received an Infectious Diseases Society of America guideline-endorsed first-line treatment (fluroquinolones, TMP-SMX). Overall, baseline characteristics were similar between the 2 cohorts. Rates of treatment failure were not significantly different in the cephalosporin group compared with the fluroquinolone/TMP-SMX groups (17.2% of cephalosporin vs 22.5% of fluroquinolone/TMP-SMX group, difference=5.3%, 95% confidence interval -0.118 to 0.01). After adjusting for potential confounders, cephalosporin use was not associated with treatment failure (odds ratio=0.22, 95% confidence interval 0.03 to 1.95). There was no difference in rates of appropriate empiric treatment based on urine culture susceptibility.

CONCLUSION

Oral cephalosporins were associated with similar treatment failure rates compared with Infectious Diseases Society of America guideline-endorsed treatments for the treatment of pyelonephritis in ED patients discharged home.

Development and validation of artificial intelligence models for early detection of postoperative infections (PERISCOPE): a multicentre study using electronic health record data

Background

Postoperative infections significantly impact patient outcomes and costs, exacerbated by late diagnoses, yet early reliable predictors are scarce. Existing artificial intelligence (AI) models for postoperative infection prediction often lack external validation or perform poorly in local settings when validated. We aimed to develop locally valid models as part of the PERISCOPE AI system to enable early detection, safer discharge, and more timely treatment of patients.

Methods

We developed and validated XGBoost models to predict postoperative infections within 7 and 30 days of surgery. Using retrospective pre-operative and intra-operative electronic health record data from 2014 to 2023 across various surgical specialities, the models were developed at Hospital A and validated and updated at Hospitals B and C in the Netherlands and Belgium. Model performance was evaluated before and after updating using the two most recent years of data as temporal validation datasets. Main outcome measures were model discrimination (area under the receiver operating characteristic curve (AUROC)), calibration (slope, intercept, and plots), and clinical utility (decision curve analysis with net benefit).

Findings

The study included 253,010 surgical procedures with 23,903 infections within 30-days. Discriminative performance, calibration properties, and clinical utility significantly improved after updating. Final AUROCs after updating for Hospitals A, B, and C were 0.82 (95% confidence interval (CI) 0.81-0.83), 0.82 (95% CI 0.81-0.83), and 0.91 (95% CI 0.90-0.91) respectively for 30-day predictions on the temporal validation datasets (2022-2023). Calibration plots demonstrated adequate correspondence between observed outcomes and predicted risk. All local models were deemed clinically useful as the net benefit was higher than default strategies (treat all and treat none) over a wide range of clinically relevant decision thresholds.

Interpretation

PERISCOPE can accurately predict overall postoperative infections within 7- and 30-days post-surgery. The robust performance implies potential for improving clinical care in diverse clinical target populations. This study supports the need for approaches to local updating of AI models to account for domain shifts in patient populations and data distributions across different clinical settings.

Funding

This study was funded by a REACT EU grant from European Regional Development Fund (ERDF) and Kansen voor West.

The Role of Artificial Intelligence in Urogynecology: Current Applications and Future Prospects

Artificial intelligence (AI) is the new medical hot topic, being applied mainly in specialties with a strong imaging component. In the domain of gynecology, AI has been tested and shown vast potential in several areas with promising results, with an emphasis on oncology. However, fewer studies have been made focusing on urogynecology, a branch of gynecology known for using multiple imaging exams (IEs) and tests in the management of women's pelvic floor health. This review aims to illustrate the current state of AI in urogynecology, namely with the use of machine learning (ML) and deep learning (DL) in diagnostics and as imaging tools, discuss possible future prospects for AI in this field, and go over its limitations that challenge its safe implementation.

Prediction of urinary tract infection using machine learning methods: a study for finding the most-informative variables

BACKGROUND

Urinary tract infection (UTI) is a frequent health-threatening condition. Early reliable diagnosis of UTI helps to prevent misuse or overuse of antibiotics and hence prevent antibiotic resistance. The gold standard for UTI diagnosis is urine culture which is a time-consuming and also an error prone method. In this regard, complementary methods are demanded. In the recent decade, machine learning strategies that employ mathematical models on a dataset to extract the most informative hidden information are the center of interest for prediction and diagnosis purposes.

METHOD

In this study, machine learning approaches were used for finding the important variables for a reliable prediction of UTI. Several types of machines including classical and deep learning models were used for this purpose.

RESULTS

Eighteen selected features from urine test, blood test, and demographic data were found as the most informative features. Factors extracted from urine such as WBC, nitrite, leukocyte, clarity, color, blood, bilirubin, urobilinogen, and factors extracted from blood test like mean platelet volume, lymphocyte, glucose, red blood cell distribution width, and potassium, and demographic data such as age, gender and previous use of antibiotics were the determinative factors for UTI prediction. An ensemble combination of XGBoost, decision tree, and light gradient boosting machines with a voting scheme obtained the highest accuracy for UTI prediction (AUC: 88.53 (0.25), accuracy: 85.64 (0.20)%), according to the selected features. Furthermore, the results showed the importance of gender and age for UTI prediction.

CONCLUSION

This study highlighted the potential of machine learning strategies for UTI prediction.

A step forward in the diagnosis of urinary tract infections: from machine learning to clinical practice

Objectives

Urinary tract infections (UTIs) are common infections within the Emergency Department (ED), causing increased laboratory workloads and unnecessary antibiotics prescriptions. The aim of this study was to improve UTI diagnostics in clinical practice by application of machine learning (ML) models for real-time UTI prediction.

Methods

In a retrospective study, patient information and outcomes from Emergency Department patients, with positive and negative culture results, were used to design models - 'Random Forest' and 'Neural Network' - for the prediction of UTIs. The performance of these predictive models was validated in a cross-sectional study. In a quasi-experimental study, the impact of UTI risk assessment was investigated by evaluating changes in the behaviour of clinicians, measuring changes in antibiotic prescriptions and urine culture requests.

Results

First, we trained and tested two different predictive models with 8692 cases. Second, we investigated the performance of the predictive models in clinical practice with 962 cases (Area under the curve was between 0.81 to 0.88). The best performance was the combination of both models. Finally, the assessment of the risk for UTIs was implemented into clinical practice and allowed for the reduction of unnecessary urine cultures and antibiotic prescriptions for patients with a low risk of UTI, as well as targeted diagnostics and treatment for patients with a high risk of UTI.

Conclusion

The combination of modern urinalysis diagnostic technologies with digital health solutions can help to further improve UTI diagnostics with positive impact on laboratory workloads and antimicrobial stewardship.

Racial differences in laboratory testing as a potential mechanism for bias in AI: A matched cohort analysis in emergency department visits

AI models are often trained using available laboratory test results. Racial differences in laboratory testing may bias AI models for clinical decision support, amplifying existing inequities. This study aims to measure the extent of racial differences in laboratory testing in adult emergency department (ED) visits. We conducted a retrospective 1:1 exact-matched cohort study of Black and White adult patients seen in the ED, matching on age, biological sex, chief complaint, and ED triage score, using ED visits at two U.S. teaching hospitals: Michigan Medicine, Ann Arbor, MI (U-M, 2015-2022), and Beth Israel Deaconess Medical Center, Boston, MA (BIDMC, 2011-2019). Post-matching, White patients had significantly higher testing rates than Black patients for complete blood count (BIDMC difference: 1.7%, 95% CI: 1.1% to 2.4%, U-M difference: 2.0%, 95% CI: 1.6% to 2.5%), metabolic panel (BIDMC: 1.5%, 95% CI: 0.9% to 2.1%, U-M: 1.9%, 95% CI: 1.4% to 2.4%), and blood culture (BIDMC: 0.9%, 95% CI: 0.5% to 1.2%, U-M: 0.7%, 95% CI: 0.4% to 1.1%). Black patients had significantly higher testing rates for troponin than White patients (BIDMC: -2.1%, 95% CI: -2.6% to -1.6%, U-M: -2.2%, 95% CI: -2.7% to -1.8%). The observed racial testing differences may impact AI models trained using available laboratory results. The findings also motivate further study of how such differences arise and how to mitigate potential impacts on AI models.

Machine learning for predicting device-associated infection and 30-day survival outcomes after invasive device procedure in intensive care unit patients

This study aimed to preliminarily develop machine learning (ML) models capable of predicting the risk of device-associated infection and 30-day outcomes following invasive device procedures in intensive care unit (ICU) patients. The study utilized data from 8574 ICU patients who underwent invasive procedures, sourced from the Medical Information Mart for Intensive Care (MIMIC)-IV version 2.2 database. Patients were allocated into training and validation datasets in a 7:3 ratio. Seven ML models were employed for predicting device-associated infections, while five models were used for predicting 30-day survival outcomes. Model performance was primarily evaluated using the receiver operating characteristic (ROC) curve for infection prediction and the survival model's concordance index (C-index). Top-performing models progressively reduced the number of variables based on their importance, thereby optimizing practical utility. The inclusion of all variables demonstrated that extreme gradient boosting (XGBoost) and extra survival trees (EST) models yielded superior discriminatory performance. Notably, when restricted to the top 10 variables, both models maintained performance levels comparable to when all variables were included. In the validation cohort, the XGBoost model, with the top 10 variables, achieved an area under the curve (AUC) of 0.810 (95% CI 0.808-0.812), an area under the precision-recall curve (AUPRC) of 0.226 (95% CI 0.222-0.230), and a Brier score (BS) of 0.053 (95% CI 0.053-0.054). The EST model, with the top 10 variables, reported a C-index of 0.756 (95% CI 0.754-0.757), a time-dependent AUC of 0.759 (95% CI 0.763-0.775), and an integrated Brier score (IBS) of 0.087 (95% CI 0.087-0.087). Both models are accessible via a web application. The internally evaluated XGBoost and EST models demonstrated exceptional predictive accuracy for device-associated infection risks and 30-day survival outcomes post-invasive procedures in ICU patients. Further validation is required to confirm the clinical utility of these two models in future studies.

Artificial intelligence and prescription of antibiotic therapy: present and future

INTRODUCTION

In the past few years, the use of artificial intelligence in healthcare has grown exponentially. Prescription of antibiotics is not exempt from its rapid diffusion, and various machine learning (ML) techniques, from logistic regression to deep neural networks and large language models, have been explored in the literature to support decisions regarding antibiotic prescription.

AREAS COVERED

In this narrative review, we discuss promises and challenges of the application of ML-based clinical decision support systems (ML-CDSSs) for antibiotic prescription. A search was conducted in PubMed up to April 2024.

EXPERT OPINION

Prescribing antibiotics is a complex process involving various dynamic phases. In each of these phases, the support of ML-CDSSs has shown the potential, and also the actual ability in some studies, to favorably impacting relevant clinical outcomes. Nonetheless, before widely exploiting this massive potential, there are still crucial challenges ahead that are being intensively investigated, pertaining to the transparency of training data, the definition of the sufficient degree of prediction explanations when predictions are obtained through black box models, and the legal and ethical framework for decision responsibility whenever an antibiotic prescription is supported by ML-CDSSs.

An artificial intelligence-driven support tool for prediction of urine culture test results

BACKGROUND AND AIMS

We aimed to develop an easily deployable artificial intelligence (AI)-driven model for rapid prediction of urine culture test results.

MATERIAL AND METHODS

We utilized a training dataset (n = 34,584 urine samples) and two separate, unseen test sets (n = 10,083 and 9,289 samples). Various machine learning models were compared for diagnostic performance. Predictive parameters included urinalysis results (dipstick and flow cytometry), patient demographics (age and gender), and sample collection method.

RESULTS

Although more complex models achieved the highest AUCs for predicting positive cultures (highest: multilayer perceptron (MLP) with AUC of 0.884, 95% CI 0.878-0.89), multiple logistic regression (MLR) using only flow cytometry parameters achieved a very good AUC (0.858, 95% CI 0.852-0.865). To aid interpretation, prediction results of the MLP and MLR models were categorized based on likelihood ratio (LR) for positivity: highly unlikely (LR 0.1), unlikely (LR 0.3), grey zone (LR 0.9), likely (LR 5.0), and highly likely (LR 40). This resulted in 17%, 28%, 34%, 9%, and 13% of samples falling into each respective category for the MLR model and 20%, 26%, 31%, 7%, and 16% for the MLP model.

CONCLUSIONS

In conclusion, this robust model has the potential to assist clinicians in their decision-making process by providing insights prior to the availability of urine culture results in a significant portion of samples (∼2/3rd).

The clinical usefulness of guideline-based strategies with and without the role of nonspecific symptoms to predict urinary tract infections in nursing homes: a decision curve analysis

Objective

The aim of this study was to assess the clinical value of urinary tract infections (UTIs) guideline algorithms and the role of nonspecific symptoms to support clinical decision-making in nursing home residents.

Design

In a preplanned secondary analysis of a cross-sectional study including nursing home residents with a presumed UTI, 2 prediction models were used in a decision curve analysis (DCA): (1) guideline-based and (2) extended: nonspecific symptom(s) added to the guideline model. The stringent outcome definition for "true UTIs" included symptom improvement during adequate antimicrobial therapy, based on susceptibility test results. The outcome of a DCA is the Net Benefit to quantify the performance of the prediction models, visualized in a decision curve.

Setting

Dutch nursing homes (n = 13).

Patients

Nursing home residents with a presumed UTI.

Results

Of the 180 residents with a presumed UTI, 43 fulfilled the definition of "true UTI" (23.9%). The Net Benefit of the guideline-based model was low and the corresponding threshold range was small (21%-28%). The extended model improved the prediction of UTIs. However, the clinical usefulness of the extended model was still limited to a small threshold range (10%-28%).

Conclusions

The clinical usefulness of the current guideline-based algorithm to diagnose UTI in nursing home residents seems limited, and adding nonspecific symptoms does not further improve decision-making due to the small threshold probability. Given the poor performance of the guideline-based model, refinement of the guidelines may be required.

Trial registry

Dutch trial registry: NTR6467; date of first registration, 25/05/2017.

The AI Future of Emergency Medicine

In the coming years, artificial intelligence (AI) and machine learning will likely give rise to profound changes in the field of emergency medicine, and medicine more broadly. This article discusses these anticipated changes in terms of 3 overlapping yet distinct stages of AI development. It reviews some fundamental concepts in AI and explores their relation to clinical practice, with a focus on emergency medicine. In addition, it describes some of the applications of AI in disease diagnosis, prognosis, and treatment, as well as some of the practical issues that they raise, the barriers to their implementation, and some of the legal and regulatory challenges they create.

Artificial intelligence and machine learning applications in urinary tract infections identification and prediction: a systematic review and meta-analysis

BACKGROUND

Urinary tract infections (UTIs) have been one of the most common bacterial infections in clinical practice worldwide. Artificial intelligence (AI) and machine learning (ML) based algorithms have been increasingly applied in UTI case identification and prediction. However, the overall performance of AI/ML algorithms in identifying and predicting UTI has not been evaluated. The purpose of this paper is to quantitatively evaluate the application value of AI/ML in identifying and predicting UTI cases.

METHODS

MEDLINE, EMBASE, Web of Science, and PubMed databases were systematically searched for articles published up to December 31, 2023. Quality Assessment of Diagnostic Accuracy Studies tool (QUADAS-2) and Prediction Model Risk of Bias Assessment Tool (PROBAST) were used to assess the risk of bias. Study characteristics and detailed algorithm information were extracted. Pooled sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were synthesized using a bivariate mix-effects model. Meta-regression and subgroup analysis were conducted to test the source of heterogeneity.

RESULTS

In total, 11 studies with 14 AI/ML models were included in the final meta-analysis. The overall pooled AUC was 0.89 (95%CI 0.86-0.92). Additionally, the pooled Sen, Spe, PLR, NLR, and DOR were 0.78 (95%CI 0.71-0.84), 0.89 (95%CI 0.83-0.93), 6.99 (95%CI 4.38-11.14), 0.25 (95%CI 0.18-0.34) and 28.07 (95%CI 14.27-55.20), respectively. The results of meta-regression suggested that reference standard definitions might be the source of heterogeneity.

CONCLUSION

AI/ML algorithms appear to be promising to help clinicians detect and identify patients at high risk of UTIs. However, further studies are demanded to evaluate the application value of AI/ML more thoroughly.

Risk score models for urinary tract infection hospitalization

Annually, urinary tract infections (UTIs) affect over a hundred million people worldwide. Early detection of high-risk individuals can help prevent hospitalization for UTIs, which imposes significant economic and social burden on patients and caregivers. We present two methods to generate risk score models for UTI hospitalization. We utilize a sample of patients from the insurance claims data provided by the Centers for Medicare and Medicaid Services to develop and validate the proposed methods. Our dataset encompasses a wide range of features, such as demographics, medical history, and healthcare utilization of the patients along with provider quality metrics and community-based metrics. The proposed methods scale and round the coefficients of an underlying logistic regression model to create scoring tables. We present computational experiments to evaluate the prediction performance of both models. We also discuss different features of these models with respect to their impact on interpretability. Our findings emphasize the effectiveness of risk score models as practical tools for identifying high-risk patients and provide a quantitative assessment of the significance of various risk factors in UTI hospitalizations such as admission to ICU in the last 3 months, cognitive disorders and low inpatient, outpatient and carrier costs in the last 6 months.

Automating detection of diagnostic error of infectious diseases using machine learning

Diagnostic error, a cause of substantial morbidity and mortality, is largely discovered and evaluated through self-report and manual review, which is costly and not suitable to real-time intervention. Opportunities exist to leverage electronic health record data for automated detection of potential misdiagnosis, executed at scale and generalized across diseases. We propose a novel automated approach to identifying diagnostic divergence considering both diagnosis and risk of mortality. Our objective was to identify cases of emergency department infectious disease misdiagnoses by measuring the deviation between predicted diagnosis and documented diagnosis, weighted by mortality. Two machine learning models were trained for prediction of infectious disease and mortality using the first 24h of data. Charts were manually reviewed by clinicians to determine whether there could have been a more correct or timely diagnosis. The proposed approach was validated against manual reviews and compared using the Spearman rank correlation. We analyzed 6.5 million ED visits and over 700 million associated clinical features from over one hundred emergency departments. The testing set performances of the infectious disease (Macro F1 = 86.7, AUROC 90.6 to 94.7) and mortality model (Macro F1 = 97.6, AUROC 89.1 to 89.1) were in expected ranges. Human reviews and the proposed automated metric demonstrated positive correlations ranging from 0.231 to 0.358. The proposed approach for diagnostic deviation shows promise as a potential tool for clinicians to find diagnostic errors. Given the vast number of clinical features used in this analysis, further improvements likely need to either take greater account of data structure (what occurs before when) or involve natural language processing. Further work is needed to explain the potential reasons for divergence and to refine and validate the approach for implementation in real-world settings.

Identifying signs and symptoms of urinary tract infection from emergency department clinical notes using large language models

BACKGROUND

Natural language processing (NLP) tools including recently developed large language models (LLMs) have myriad potential applications in medical care and research, including the efficient labeling and classification of unstructured text such as electronic health record (EHR) notes. This opens the door to large-scale projects that rely on variables that are not typically recorded in a structured form, such as patient signs and symptoms.

OBJECTIVES

This study is designed to acquaint the emergency medicine research community with the foundational elements of NLP, highlighting essential terminology, annotation methodologies, and the intricacies involved in training and evaluating NLP models. Symptom characterization is critical to urinary tract infection (UTI) diagnosis, but identification of symptoms from the EHR has historically been challenging, limiting large-scale research, public health surveillance, and EHR-based clinical decision support. We therefore developed and compared two NLP models to identify UTI symptoms from unstructured emergency department (ED) notes.

METHODS

The study population consisted of patients aged ≥ 18 who presented to an ED in a northeastern U.S. health system between June 2013 and August 2021 and had a urinalysis performed. We annotated a random subset of 1250 ED clinician notes from these visits for a list of 17 UTI symptoms. We then developed two task-specific LLMs to perform the task of named entity recognition: a convolutional neural network-based model (SpaCy) and a transformer-based model designed to process longer documents (Clinical Longformer). Models were trained on 1000 notes and tested on a holdout set of 250 notes. We compared model performance (precision, recall, F1 measure) at identifying the presence or absence of UTI symptoms at the note level.

RESULTS

A total of 8135 entities were identified in 1250 notes; 83.6% of notes included at least one entity. Overall F1 measure for note-level symptom identification weighted by entity frequency was 0.84 for the SpaCy model and 0.88 for the Longformer model. F1 measure for identifying presence or absence of any UTI symptom in a clinical note was 0.96 (232/250 correctly classified) for the SpaCy model and 0.98 (240/250 correctly classified) for the Longformer model.

CONCLUSIONS

The study demonstrated the utility of LLMs and transformer-based models in particular for extracting UTI symptoms from unstructured ED clinical notes; models were highly accurate for detecting the presence or absence of any UTI symptom on the note level, with variable performance for individual symptoms.

Electronic Phenotyping of Urinary Tract Infections as a Silver Standard Label for Machine Learning

This study explored the efficacy of electronic phenotyping in data labeling for machine learning with a focus on urinary tract infections (UTIs). We contrasted labels from electronic phenotyping against previously published labels such as urine culture positivity. In comparison, electronic phenotyping showed the potential to enhance specificity in UTI labeling while maintaining similar sensitivity and was easily scaled for application to a large dataset suitable for machine learning, which we used to train and validate a machine learning model. Electronic phenotyping offers a valuable method for machine learning label generation in healthcare, with potential benefits for patient care and antimicrobial stewardship. Further research will expand its application and optimize techniques for increased performance.

A novel score for early prediction of urinary tract infection risk in patients with acute ischemic stroke: a nomogram-based retrospective cohort study

This study aimed to construct and externally validate a user-friendly nomogram-based scoring model for predicting the risk of urinary tract infections (UTIs) in patients with acute ischemic stroke (AIS). A retrospective real-world cohort study was conducted on 1748 consecutive hospitalized patients with AIS. Out of these patients, a total of 1132 participants were ultimately included in the final analysis, with 817 used for model construction and 315 utilized for external validation. Multivariate regression analysis was applied to develop the model. The discriminative capacity, calibration ability, and clinical effectiveness of the model were evaluated. The overall incidence of UTIs was 8.13% (92/1132), with Escherichia coli being the most prevalent causative pathogen in patients with AIS. After multivariable analysis, advanced age, female gender, National Institute of Health Stroke Scale (NIHSS) score ≥ 5, and use of urinary catheters were identified as independent risk factors for UTIs. A nomogram-based SUNA model was constructed using these four factors (Area under the receiver operating characteristic curve (AUC) = 0.810), which showed good discrimination (AUC = 0.788), calibration, and clinical utility in the external validation cohort. Based on four simple and readily available factors, we derived and externally validated a novel and user-friendly nomogram-based scoring model (SUNA score) to predict the risk of UTIs in patients with AIS. The model has a good predictive value and provides valuable information for timely intervention in patients with AIS to reduce the occurrence of UTIs.

Novel Approach for Detecting Respiratory Syncytial Virus in Pediatric Patients Using Machine Learning Models Based on Patient-Reported Symptoms: Model Development and Validation Study

BACKGROUND

Respiratory syncytial virus (RSV) affects children, causing serious infections, particularly in high-risk groups. Given the seasonality of RSV and the importance of rapid isolation of infected individuals, there is an urgent need for more efficient diagnostic methods to expedite this process.

OBJECTIVE

This study aimed to investigate the performance of a machine learning model that leverages the temporal diversity of symptom onset for detecting RSV infections and elucidate its discriminatory ability.

METHODS

The study was conducted in pediatric and emergency outpatient settings in Japan. We developed a detection model that remotely confirms RSV infection based on patient-reported symptom information obtained using a structured electronic template incorporating the differential points of skilled pediatricians. An extreme gradient boosting-based machine learning model was developed using the data of 4174 patients aged ≤24 months who underwent RSV rapid antigen testing. These patients visited either the pediatric or emergency department of Yokohama City Municipal Hospital between January 1, 2009, and December 31, 2015. The primary outcome was the diagnostic accuracy of the machine learning model for RSV infection, as determined by rapid antigen testing, measured using the area under the receiver operating characteristic curve. The clinical efficacy was evaluated by calculating the discriminative performance based on the number of days elapsed since the onset of the first symptom and exclusion rates based on thresholds of reasonable sensitivity and specificity.

RESULTS

Our model demonstrated an area under the receiver operating characteristic curve of 0.811 (95% CI 0.784-0.833) with good calibration and 0.746 (95% CI 0.694-0.794) for patients within 3 days of onset. It accurately captured the temporal evolution of symptoms; based on adjusted thresholds equivalent to those of a rapid antigen test, our model predicted that 6.9% (95% CI 5.4%-8.5%) of patients in the entire cohort would be positive and 68.7% (95% CI 65.4%-71.9%) would be negative. Our model could eliminate the need for additional testing in approximately three-quarters of all patients.

CONCLUSIONS

Our model may facilitate the immediate detection of RSV infection in outpatient settings and, potentially, in home environments. This approach could streamline the diagnostic process, reduce discomfort caused by invasive tests in children, and allow rapid implementation of appropriate treatments and isolation at home. The findings underscore the potential of machine learning in augmenting clinical decision-making in the early detection of RSV infection.

Bioinformatics in urology - molecular characterization of pathophysiology and response to treatment

The application of bioinformatics has revolutionized the practice of medicine in the past 20 years. From early studies that uncovered subtypes of cancer to broad efforts spearheaded by the Cancer Genome Atlas initiative, the use of bioinformatics strategies to analyse high-dimensional data has provided unprecedented insights into the molecular basis of disease. In addition to the identification of disease subtypes - which enables risk stratification - informatics analysis has facilitated the identification of novel risk factors and drivers of disease, biomarkers of progression and treatment response, as well as possibilities for drug repurposing or repositioning; moreover, bioinformatics has guided research towards precision and personalized medicine. Implementation of specific computational approaches such as artificial intelligence, machine learning and molecular subtyping has yet to become widespread in urology clinical practice for reasons of cost, disruption of clinical workflow and need for prospective validation of informatics approaches in independent patient cohorts. Solving these challenges might accelerate routine integration of bioinformatics into clinical settings.

Joint global and local interpretation method for CIN status classification in breast cancer

Breast cancer is among the cancer types with the highest numbers of new cases. The study of this disease from a microscopic perspective has been a prominent research topic. Previous studies have shown that microRNAs (miRNAs) are closely linked to chromosomal instability (CIN). Correctly predicting CIN status from miRNAs can help to improve the survival of breast cancer patients. In this study, a joint global and local interpretation method called GL_XGBoost is proposed for predicting CIN status in breast cancer. GL_XGBoost integrates the eXtreme Gradient Boosting (XGBoost) and SHapley Additive exPlanation (SHAP) methods. XGBoost is used to predict CIN status from miRNA data, whereas SHAP is used to select miRNA features that have strong relationships with CIN. Furthermore, SHAP's rich visualization strategies enhance the interpretability of the entire model at the global and local levels. The performance of GL_XGBoost is validated on the TCGA-BRCA dataset, and it is shown to have an accuracy of 78.57% and an area under the curve value of 0.87. Rich visual analysis is used to explain the relationships between miRNAs and CIN status from different perspectives. Our study demonstrates an intuitive way of exploring the relationship between CIN and cancer from a microscopic perspective.

Future of Artificial Intelligence in Surgery: A Narrative Review

Artificial intelligence (AI) is the capability of a machine to execute cognitive processes that are typically considered to be functions of the human brain. It is the study of algorithms that enable machines to reason and perform mental tasks, including problem-solving, object and word recognition, and decision-making. Once considered science fiction, AI today is a fact and an increasingly prevalent subject in both academic and popular literature. It is expected to reshape medicine, benefiting both healthcare professionals and patients. Machine learning (ML) is a subset of AI that allows machines to learn and make predictions by recognizing patterns, thus empowering the medical team to deliver better care to patients through accurate diagnosis and treatment. ML is expanding its footprint in a variety of surgical specialties, including general surgery, ophthalmology, cardiothoracic surgery, and vascular surgery, to name a few. In recent years, we have seen AI make its way into the operating theatres. Though it has not yet been able to replace the surgeon, it has the potential to become a highly valuable surgical tool. Rest assured that the day is not far off when AI shall play a significant intraoperative role, a projection that is currently marred by safety concerns. This review aims to explore the present application of AI in various surgical disciplines and how it benefits both patients and physicians, as well as the current obstacles and limitations facing its seemingly unstoppable rise.

A study on the risk stratification for patients within 24 hours of admission for risk of hospital-acquired urinary tract infection using Bayesian network models

Early identification of patients at risk of hospital-acquired urinary tract infections (HA-UTI) enables the initiation of timely targeted preventive and therapeutic strategies. Machine learning (ML) models have shown great potential for this purpose. However, existing ML models in infection control have demonstrated poor ability to support explainability, which challenges the interpretation of the result in clinical practice, limiting the adaption of the ML models into a daily clinical routine. In this study, we developed Bayesian Network (BN) models to enable explainable assessment within 24 h of admission for risk of HA-UTI. Our dataset contained 138,250 unique hospital admissions. We included data on admission details, demographics, lifestyle factors, comorbidities, vital parameters, laboratory results, and urinary catheter. Models developed from a reduced set of five features were characterized by transparency compared to models developed from a full set of 50 features. The expert-based clinical BN model over the reduced feature space showed the highest performance (area under the curve = 0.746) compared to the naïve- and tree-augmented-naïve BN models. Moreover, models developed from expert-based knowledge were characterized by enhanced explainability.

Smart Diagnosis of Urinary Tract Infections: is Artificial Intelligence the Fast-Lane Solution?

PURPOSE OF REVIEW

Artificial intelligence (AI) can significantly improve physicians' workflow when examining patients with UTI. However, most contemporary reviews are focused on examining the usage of AI with a restricted quantity of data, analyzing only a subset of AI algorithms, or performing narrative work without analyzing all dedicated studies. Given the preceding, the goal of this work was to conduct a mini-review to determine the current state of AI-based systems as a support in UTI diagnosis.

RECENT FINDINGS

There are sufficient publications to comprehend the potential applications of artificial intelligence in the diagnosis of UTIs. Existing research in this field, in general, publishes performance metrics that are exemplary. However, upon closer inspection, many of the available publications are burdened with flaws associated with the improper use of artificial intelligence, such as the use of a small number of samples, their lack of heterogeneity, and the absence of external validation. AI-based models cannot be classified as full-fledged physician assistants in diagnosing UTIs due to the fact that these limitations and flaws represent only a portion of all potential obstacles. Instead, such studies should be evaluated as exploratory, with a focus on the importance of future work that complies with all rules governing the use of AI. AI algorithms have demonstrated their potential for UTI diagnosis. However, further studies utilizing large, heterogeneous, prospectively collected datasets, as well as external validations, are required to define the actual clinical workflow value of artificial intelligence.

Application of OpenAI GPT-4 for the retrospective detection of catheter-associated urinary tract infections in a fictitious and curated patient data set

The use of the OpenAI GPT-4 model in detecting catheter-associated urinary tract infection (CAUTI) cases in small fictitious and curated patient data sets was investigated. Final analysis of 50 patients including 11 CAUTI cases yielded sensitivity, specificity and positive and negative predictive values of 91%, 92%, 83%, and 96%, respectively.

Development and validation of artificial intelligence models to predict urinary tract infections and secondary bloodstream infections in adult patients

BACKGROUND

Traditional culture methods are time-consuming, making it difficult to utilize the results in the early stage of urinary tract infection (UTI) management, and automated urinalyses alone show insufficient performance for diagnosing UTIs. Several models have been proposed to predict urine culture positivity based on urinalysis. However, most of them have not been externally validated or consisted solely of urinalysis data obtained using one specific commercial analyzer.

METHODS

A total of 259,187 patients were enrolled to develop artificial intelligence (AI) models. AI models were developed and validated for the diagnosis of UTI and urinary tract related-bloodstream infection (UT-BSI). The predictive performance of conventional urinalysis and AI algorithms were assessed by the areas under the receiver operating characteristic curve (AUROC). We also visualized feature importance rankings as Shapley additive explanation bar plots.

RESULTS

In the two cohorts, the positive rates of urine culture tests were 25.2% and 30.4%, and the proportions of cases classified as UT-BSI were 1.8% and 1.6%. As a result of predicting UTI from the automated urinalysis, the AUROC were 0.745 (0.743-0.746) and 0.740 (0.737-0.743), and most AI algorithms presented excellent discriminant performance (AUROC > 0.9). In the external validation dataset, the XGBoost model achieved the best values in predicting both UTI (AUROC 0.967 [0.966-0.968]) and UT-BSI (AUROC 0.955 [0.951-0.959]). A reduced model using ten parameters was also derived.

CONCLUSIONS

We found that AI models can improve the early prediction of urine culture positivity and UT-BSI by combining automated urinalysis with other clinical information. Clinical utilization of the model can reduce the risk of delayed antimicrobial therapy in patients with nonspecific symptoms of UTI and classify patients with UT-BSI who require further treatment and close monitoring.

Attempt to Establish Prognostic Predictive System for Hepatocellular Carcinoma Using Artificial Intelligence for Assistance with Selection of Treatment Modality

INTRODUCTION

Because of recent developments in treatments for hepatocellular carcinoma (HCC), methods for determining suitable therapy for initial or recurrent HCC have become important. This study used artificial intelligence (AI) findings to establish a system for predicting prognosis of HCC patients at time of reoccurrence based on clinical data as a reference for selection of treatment modalities.

METHODS

As a training cohort, 5,701 observations obtained at the initial and each subsequent treatment for recurrence from 1,985 HCC patients at a single center from 2000 to 2021 were used. The validation cohort included 5,692 observations from patients at multiple centers obtained at the time of the initial treatment. An AI calculating system (PRAID) was constructed based on 25 clinical factors noted at each treatment from the training cohort, and then predictive prognostic values for 1- and 3-year survival in both cohorts were evaluated.

RESULTS

After exclusion of patients lacking clinical data regarding albumin-bilirubin (ALBI) grade or tumor-node-metastasis stage of the Liver Cancer Study Group of Japan, 6th edition (TNM-LCSGJ 6th), ALBI-TNM-LCSGJ 6th (ALBI-T) and modified ALBI-T scores confirmed that prognosis for patients in both cohorts was similar. The area under the curve for prediction of both 1- and 3-year survival in the validation cohort was 0.841 (sensitivity 0.933 [95% CI: 0.925-0.940], specificity 0.517 [95% CI: 0.484-0.549]) and 0.796 (sensitivity 0.806 [95% CI: 0.790-0.821], specificity 0.646 [95% CI: 0.624-0.668]), respectively.

CONCLUSION

The present PRAID system might provide useful prognostic information related to short and medium survival for decision-making regarding the best therapeutic modality for both initial and recurrent HCC cases.

Applications of Artificial Intelligence in Urinalysis: Is the Future Already Here?

BACKGROUND

Artificial intelligence (AI) has emerged as a promising and transformative tool in the field of urinalysis, offering substantial potential for advancements in disease diagnosis and the development of predictive models for monitoring medical treatment responses.

CONTENT

Through an extensive examination of relevant literature, this narrative review illustrates the significance and applicability of AI models across the diverse application area of urinalysis. It encompasses automated urine test strip and sediment analysis, urinary tract infection screening, and the interpretation of complex biochemical signatures in urine, including the utilization of cutting-edge techniques such as mass spectrometry and molecular-based profiles.

SUMMARY

Retrospective studies consistently demonstrate good performance of AI models in urinalysis, showcasing their potential to revolutionize clinical practice. However, to comprehensively evaluate the real clinical value and efficacy of AI models, large-scale prospective studies are essential. Such studies hold the potential to enhance diagnostic accuracy, improve patient outcomes, and optimize medical treatment strategies. By bridging the gap between research and clinical implementation, AI can reshape the landscape of urinalysis, paving the way for more personalized and effective patient care.

Machine learning-based prediction model for emergency department visits using prescription information in community-dwelling non-cancer older adults

Older adults are more likely to require emergency department (ED) visits than others, which might be attributed to their medication use. Being able to predict the likelihood of an ED visit using prescription information and readily available data would be useful for primary care. This study aimed to predict the likelihood of ED visits using extensive medication variables generated according to explicit clinical criteria for elderly people and high-risk medication categories by applying machine learning (ML) methods. Patients aged ≥ 65 years were included, and ED visits were predicted with 146 variables, including demographic and comprehensive medication-related factors, using nationwide claims data. Among the eight ML models, the final model was developed using LightGBM, which showed the best performance. The final model incorporated 93 predictors, including six sociodemographic, 28 comorbidity, and 59 medication-related variables. The final model had an area under the receiver operating characteristic curve of 0.689 in the validation cohort. Approximately half of the top 20 strong predictors were medication-related variables. Here, an ED visit risk prediction model for older people was developed and validated using administrative data that can be easily applied in clinical settings to screen patients who are likely to visit an ED.

Development and validation of a parsimonious prediction model for positive urine cultures in outpatient visits

Urine culture is often considered the gold standard for detecting the presence of bacteria in the urine. Since culture is expensive and often requires 24-48 hours, clinicians often rely on urine dipstick test, which is considerably cheaper than culture and provides instant results. Despite its ease of use, urine dipstick test may lack sensitivity and specificity. In this paper, we use a real-world dataset consisting of 17,572 outpatient encounters who underwent urine cultures, collected between 2015 and 2021 at a large multi-specialty hospital in Abu Dhabi, United Arab Emirates. We develop and evaluate a simple parsimonious prediction model for positive urine cultures based on a minimal input set of ten features selected from the patient's presenting vital signs, history, and dipstick results. In a test set of 5,339 encounters, the parsimonious model achieves an area under the receiver operating characteristic curve (AUROC) of 0.828 (95% CI: 0.810-0.844) for predicting a bacterial count ≥ 105 CFU/ml, outperforming a model that uses dipstick features only that achieves an AUROC of 0.786 (95% CI: 0.769-0.806). Our proposed model can be easily deployed at point-of-care, highlighting its value in improving the efficiency of clinical workflows, especially in low-resource settings.

Supervised Machine Learning Methods for Seasonal Influenza Diagnosis

Influenza has been a stationary disease in Mexico since 2009, and this causes a high cost for the national public health system, including its detection using RT-qPCR tests, treatments, and absenteeism in the workplace. Despite influenza's relevance, the main clinical features to detect the disease defined by international institutions like the World Health Organization (WHO) and the United States Centers for Disease Control and Prevention (CDC) do not follow the same pattern in all populations. The aim of this work is to find a machine learning method to facilitate decision making in the clinical differentiation between positive and negative influenza patients, based on their symptoms and demographic features. The research sample consisted of 15480 records, including clinical and demographic data of patients with a positive/negative RT-qPCR influenza tests, from 2010 to 2020 in the public healthcare institutions of Mexico City. The performance of the methods for classifying influenza cases were evaluated with indices like accuracy, specificity, sensitivity, precision, the f1-measure and the area under the curve (AUC). Results indicate that random forest and bagging classifiers were the best supervised methods; they showed promise in supporting clinical diagnosis, especially in places where performing molecular tests might be challenging or not feasible.

Developing and validating clinical features-based machine learning algorithms to predict influenza infection in influenza-like illness patients

BACKGROUND

Seasonal influenza poses a significant risk, and patients can benefit from early diagnosis and treatment. However, underdiagnosis and undertreatment remain widespread. We developed and compared clinical feature-based machine learning (ML) algorithms that can accurately predict influenza infection in emergency departments (EDs) among patients with influenza-like illness (ILI).

MATERIAL AND METHODS

We conducted a prospective cohort study in five EDs in the US and Taiwan from 2015 to 2020. Adult patients visiting the EDs with symptoms of ILI were recruited and tested by real-time RT-PCR for influenza. We evaluated seven ML algorithms and compared their results with previously developed clinical prediction models.

RESULTS

Out of the 2189 enrolled patients, 1104 tested positive for influenza. The eXtreme Gradient Boosting achieved superior performance with an area under the receiver operating characteristic curve of 0.82 (95% confidence interval [CI] = 0.79-0.85), with a sensitivity of 0.92 (95% CI = 0.88-0.95), specificity of 0.89 (95% CI = 0.86-0.92), and accuracy of 0.72 (95% CI = 0.69-0.76) in the testing set over cut-offs of 0.4, 0.6 and 0.5, respectively. These results were superior to those of previously proposed clinical prediction models. The model interpretation revealed that body temperature, cough, rhinorrhea, and exposure history were positively associated with and the days of illness and influenza vaccine were negatively associated with influenza infection. We also found the week of the influenza season, pulse rate, and oxygen saturation to be associated with influenza infection.

CONCLUSIONS

The clinical feature-based ML model outperformed conventional models for predicting influenza infection.

Development, Evaluation, and Multisite Deployment of a Machine Learning Decision Tree Algorithm To Optimize Urinalysis Parameters for Predicting Urine Culture Positivity

PittUDT, a recursive partitioning decision tree algorithm for predicting urine culture (UC) positivity based on macroscopic and microscopic urinalysis (UA) parameters, was developed in support of a broader system-wide diagnostic stewardship initiative to increase appropriateness of UC testing. Reflex algorithm training utilized results from 19,511 paired UA and UC cases (26.8% UC positive); the average patient age was 57.4 years, and 70% of samples were from female patients. Receiver operating characteristic (ROC) analysis identified urine white blood cells (WBCs), leukocyte esterase, and bacteria as the best predictors of UC positivity, with areas under the ROC curve of 0.79, 0.78, and 0.77, respectively. Using the held-out test data set (9,773 cases; 26.3% UC positive), the PittUDT algorithm met the prespecified target of a negative predictive value above 90% and resulted in a 30 to 60% total negative proportion (true-negative plus false-negative predictions). These data show that a supervised rule-based machine learning algorithm trained on paired UA and UC data has adequate predictive ability for triaging urine specimens by identifying low-risk urine specimens, which are unlikely to grow pathogenic organisms, with a false-negative proportion under 5%. The decision tree approach also generates human-readable rules that can be easily implemented across multiple hospital sites and settings. Our work demonstrates how a data-driven approach can be used to optimize UA parameters for predicting UC positivity in a reflex protocol, with the intent of improving antimicrobial stewardship and UC utilization, a potential avenue for cost savings.

Can the application of machine learning to electronic health records guide antibiotic prescribing decisions for suspected urinary tract infection in the Emergency Department?

Urinary tract infections (UTIs) are a major cause of emergency hospital admissions, but it remains challenging to diagnose them reliably. Application of machine learning (ML) to routine patient data could support clinical decision-making. We developed a ML model predicting bacteriuria in the ED and evaluated its performance in key patient groups to determine scope for its future use to improve UTI diagnosis and thus guide antibiotic prescribing decisions in clinical practice. We used retrospective electronic health records from a large UK hospital (2011-2019). Non-pregnant adults who attended the ED and had a urine sample cultured were eligible for inclusion. The primary outcome was predominant bacterial growth ≥104 cfu/mL in urine. Predictors included demography, medical history, ED diagnoses, blood tests, and urine flow cytometry. Linear and tree-based models were trained via repeated cross-validation, re-calibrated, and validated on data from 2018/19. Changes in performance were investigated by age, sex, ethnicity, and suspected ED diagnosis, and compared to clinical judgement. Among 12,680 included samples, 4,677 (36.9%) showed bacterial growth. Relying primarily on flow cytometry parameters, our best model achieved an area under the ROC curve (AUC) of 0.813 (95% CI 0.792-0.834) in the test data, and achieved both higher sensitivity and specificity compared to proxies of clinician's judgement. Performance remained stable for white and non-white patients but was lower during a period of laboratory procedure change in 2015, in patients ≥65 years (AUC 0.783, 95% CI 0.752-0.815), and in men (AUC 0.758, 95% CI 0.717-0.798). Performance was also slightly reduced in patients with recorded suspicion of UTI (AUC 0.797, 95% CI 0.765-0.828). Our results suggest scope for use of ML to inform antibiotic prescribing decisions by improving diagnosis of suspected UTI in the ED, but performance varied with patient characteristics. Clinical utility of predictive models for UTI is therefore likely to differ for important patient subgroups including women <65 years, women ≥65 years, and men. Tailored models and decision thresholds may be required that account for differences in achievable performance, background incidence, and risks of infectious complications in these groups.

Prediction Framework on Early Urine Infection in IoT-Fog Environment Using XGBoost Ensemble Model

Urine infections are one of the most prevalent concerns for the healthcare industry that may impair the functioning of the kidney and other renal organs. As a result, early diagnosis and treatment of such infections are essential to avert any future complications. Conspicuously, in the current work, an intelligent system for the early prediction of urine infections has been presented. The proposed framework uses IoT-based sensors for data collection, followed by data encoding and infectious risk factor computation using the XGBoost algorithm over the fog computing platform. Finally, the analysis results along with the health-related information of users are stored in the cloud repository for future analysis. For performance validation, extensive experiments have been carried out, and results are calculated based on real-time patient data. The statistical findings of accuracy (91.45%), specificity (95.96%), sensitivity (84.79%), precision (95.49%), and f-score(90.12%) reveal the significantly improved performance of the proposed strategy over other baseline techniques.

Utility of artificial intelligence to identify antihyperglycemic agents poisoning in the USA: introducing a practical web application using National Poison Data System (NPDS)

Clinical effects of antihyperglycemic agents poisoning may overlap each other. So, distinguishing exposure to these pharmaceutical drugs may take work. This study examined the application of machine learning techniques in identifying antihyperglycemic agent exposure using the national poisoning database in the USA. In this study, the data of single exposure due to Biguanides and Sulfonylureas (n=6183) was requested from the National Poison Data System (NPDS) for 2014-2018. We have tried five machine learning models (random forest classifier, k-nearest neighbors, Xgboost classifier, logistic regression, neural network Keras). For the multiclass classification modeling, we have divided the dataset into two parts: train (75%) and test (25%). The performance metrics used were accuracy, specificity, precision, recall, and F1-score. The algorithms used to get the classification results of different models to diagnose antihyperglycemic agents were very accurate. The accuracy of our model in determining these two antihyperglycemic agents was 91-93%. The precision-recall curve showed average precision of 0.91, 0.97, 0.97, and 0.98 for k-nearest neighbors, logistic regression, random forest, and XGB, respectively. The logistic regression, random forest, and XGB had the highest AUC (AUC=0.97) among both biguanides and sulfonylureas groups. The negative predictive values (NPV) for all the models were between 89 and 93%. We introduced a practical web application to help physicians distinguish between these agents. Despite variations in accuracy among the different types of algorithms used, all of them could accurately determine the specific exposure to biguanides and sulfonylureas retrospectively. Machine learning can distinguish antihyperglycemic agents, which may be useful for physicians without any background in medical toxicology. Besides, Our suggested ML-based Web application might help physicians in their diagnosis.

Artificial Intelligence: A Next-Level Approach in Confronting the COVID-19 Pandemic

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) which caused coronavirus diseases (COVID-19) in late 2019 in China created a devastating economical loss and loss of human lives. To date, 11 variants have been identified with minimum to maximum severity of infection and surges in cases. Bacterial co-infection/secondary infection is identified during viral respiratory infection, which is a vital reason for morbidity and mortality. The occurrence of secondary infections is an additional burden to the healthcare system; therefore, the quick diagnosis of both COVID-19 and secondary infections will reduce work pressure on healthcare workers. Therefore, well-established support from Artificial Intelligence (AI) could reduce the stress in healthcare and even help in creating novel products to defend against the coronavirus. AI is one of the rapidly growing fields with numerous applications for the healthcare sector. The present review aims to access the recent literature on the role of AI and how its subfamily machine learning (ML) and deep learning (DL) are used to curb the pandemic's effects. We discuss the role of AI in COVID-19 infections, the detection of secondary infections, technology-assisted protection from COVID-19, global laws and regulations on AI, and the impact of the pandemic on public life.

Machine learning to predict occult metastatic lymph nodes along the recurrent laryngeal nerves in thoracic esophageal squamous cell carcinoma

PURPOSE

Esophageal squamous cell carcinoma (ESCC) metastasizes in an unpredictable fashion to adjacent lymph nodes, including those along the recurrent laryngeal nerves (RLNs). This study is to apply machine learning (ML) for prediction of RLN node metastasis in ESCC.

METHODS

The dataset contained 3352 surgically treated ESCC patients whose RLN lymph nodes were removed and pathologically evaluated. Using their baseline and pathological features, ML models were established to predict RLN node metastasis on each side with or without the node status of the contralateral side. Models were trained to achieve at least 90% negative predictive value (NPV) in fivefold cross-validation. The importance of each feature was measured by the permutation score.

RESULTS

Tumor metastases were found in 17.0% RLN lymph nodes on the right and 10.8% on the left. In both tasks, the performance of each model was comparable, with a mean area under the curve ranging from 0.731 to 0.739 (without contralateral RLN node status) and from 0.744 to 0.748 (with contralateral status). All models showed approximately 90% NPV scores, suggesting proper generalizability. The pathology status of chest paraesophgeal nodes and tumor depth had the highest impacts on the risk of RLN node metastasis in both models.

CONCLUSION

This study demonstrated the feasibility of ML in predicting RLN node metastasis in ESCC. These models may potentially be used intraoperatively to spare RLN node dissection in low-risk patients, thereby minimizing adverse events associated with RLN injuries.

A machine learning approach to identifying patients with pulmonary hypertension using real-world electronic health records

BACKGROUND

This study aimed to develop a machine learning (ML) model to identify patients who are likely to have pulmonary hypertension (PH), using a large patient-level US-based electronic health record (EHR) database.

METHODS

A gradient boosting model, XGBoost, was developed using data from Optum's US-based de-identified EHR dataset (2007-2019). PH and disease control adult patients were identified using diagnostic, treatment and procedure codes and were randomly split into the training (90%) or test set (10%). Model features included patient demographics, physician visits, diagnoses, procedures, prescriptions, and laboratory test results. SHapley Additive exPlanations values were used to determine feature importance.

RESULTS

We identified 11,279,478 control and 115,822 PH patients (mean age, respectively: 62 and 68 years, both 53% female). The final model used 165 features, with the most important predictive features including diagnosis of heart failure, shortness of breath and atrial fibrillation. The model predicted PH with an area under the receiver operating characteristic curve (AUROC) of 0.92. AUROC remained above 0.80 for the prediction of PH up to and beyond 18 months before diagnosis. Among the PH patients, we also identified 955 pulmonary arterial hypertension (PAH) and 1432 chronic thromboembolic pulmonary hypertension (CTEPH) patients, and the range of AUROCs obtained for these cohorts was 0.79-0.90 and 0.87-0.96, respectively.

CONCLUSIONS

This model to detect PH based on patients' EHR records is viable and performs well in subgroups of PAH and CTEPH patients. This approach has the potential to improve patient outcomes by reducing diagnostic delay in PH.

Machine learning model for predicting ciprofloxacin resistance and presence of ESBL in patients with UTI in the ED

Increasing antimicrobial resistance in uropathogens is a clinical challenge to emergency physicians as antibiotics should be selected before an infecting pathogen or its antibiotic resistance profile is confirmed. We created a predictive model for antibiotic resistance of uropathogens, using machine learning (ML) algorithms. This single-center retrospective study evaluated patients diagnosed with urinary tract infection (UTI) in the emergency department (ED) between January 2020 and June 2021. Thirty-nine variables were used to train the model to predict resistance to ciprofloxacin and the presence of urinary pathogens' extended-spectrum beta-lactamases. The model was built with Gradient-Boosted Decision Tree (GBDT) with performance evaluation. Also, we visualized feature importance using SHapely Additive exPlanations. After two-step customization of threshold adjustment and feature selection, the final model was compared with that of the original prescribers in the emergency department (ED) according to the ineffectiveness of the antibiotic selected. The probability of using ineffective antibiotics in the ED was significantly lowered by 20% in our GBDT model through customization of the decision threshold. Moreover, we could narrow the number of predictors down to twenty and five variables with high importance while maintaining similar model performance. An ML model is potentially useful for predicting antibiotic resistance improving the effectiveness of empirical antimicrobial treatment in patients with UTI in the ED. The model could be a point-of-care decision support tool to guide clinicians toward individualized antibiotic prescriptions.

Adaptation and External Validation of Pathogenic Urine Culture Prediction in Primary Care Using Machine Learning

BACKGROUND

Urinary tract infection (UTI) symptoms are common in primary care, but antibiotics are appropriate only when an infection is present. Urine culture is the reference standard test for infection, but results take >1 day. A machine learning predictor of urine cultures showed high accuracy for an emergency department (ED) population but required urine microscopy features that are not routinely available in primary care (the NeedMicro classifier).

METHODS

We redesigned a classifier (NoMicro) that does not depend on urine microscopy and retrospectively validated it internally (ED data set) and externally (on a newly curated primary care [PC] data set) using a multicenter approach including 80,387 (ED) and 472 (PC) adults. We constructed machine learning models using extreme gradient boosting (XGBoost), artificial neural networks, and random forests (RFs). The primary outcome was pathogenic urine culture growing ≥100,000 colony forming units. Predictor variables included age; gender; dipstick urinalysis nitrites, leukocytes, clarity, glucose, protein, and blood; dysuria; abdominal pain; and history of UTI.

RESULTS

Removal of microscopy features did not severely compromise performance under internal validation: NoMicro/XGBoost receiver operating characteristic area under the curve (ROC-AUC) 0.86 (95% CI, 0.86-0.87) vs NeedMicro 0.88 (95% CI, 0.87-0.88). Excellent performance in external (PC) validation was also observed: NoMicro/RF ROC-AUC 0.85 (95% CI, 0.81-0.89). Retrospective simulation suggested that NoMicro/RF can be used to safely withhold antibiotics for low-risk patients, thereby avoiding antibiotic overuse.

CONCLUSIONS

The NoMicro classifier appears appropriate for PC. Prospective trials to adjudicate the balance of benefits and harms of using the NoMicro classifier are appropriate.

Developing a machine learning model to predict patient need for computed tomography imaging in the emergency department

Overcrowding is a well-known problem in hospitals and emergency departments (ED) that can negatively impact patients and staff. This study aims to present a machine learning model to detect a patient's need for a Computed Tomography (CT) exam in the emergency department at the earliest possible time. The data for this work was collected from ED at Thunder Bay Regional Health Sciences Centre over one year (05/2016-05/2017) and contained administrative triage information. The target outcome was whether or not a patient required a CT exam. Multiple combinations of text embedding methods, machine learning algorithms, and data resampling methods were experimented with to find the optimal model for this task. The final model was trained with 81, 118 visits and tested on a hold-out test set with a size of 9, 013 visits. The best model achieved a ROC AUC score of 0.86 and had a sensitivity of 87.3% and specificity of 70.9%. The most important factors that led to a CT scan order were found to be chief complaint, treatment area, and triage acuity. The proposed model was able to successfully identify patients needing a CT using administrative triage data that is available at the initial stage of a patient's arrival. By determining that a CT scan is needed early in the patient's visit, the ED can allocate resources to ensure these investigations are completed quickly and patient flow is maintained to reduce overcrowding.