Predicting 30-day mortality of patients with pneumonia in an emergency department setting using machine-learning models

Enhancing pneumonia prognosis in the emergency department: a novel machine learning approach using complete blood count and differential leukocyte count combined with CURB-65 score

BACKGROUND

Pneumonia poses a major global health challenge, necessitating accurate severity assessment tools. However, conventional scoring systems such as CURB-65 have inherent limitations. Machine learning (ML) offers a promising approach for prediction. We previously introduced the Blood Culture Prediction Index (BCPI) model, leveraging solely on complete blood count (CBC) and differential leukocyte count (DC), demonstrating its effectiveness in predicting bacteremia. Nevertheless, its potential in assessing pneumonia remains unexplored. Therefore, this study aims to compare the effectiveness of BCPI and CURB-65 in assessing pneumonia severity in an emergency department (ED) setting and develop an integrated ML model to enhance efficiency.

METHODS

This retrospective study was conducted at a 3400-bed tertiary medical center in Taiwan. Data from 9,352 patients with pneumonia in the ED between 2019 and 2021 were analyzed in this study. We utilized the BCPI model, which was trained on CBC/DC data, and computed CURB-65 scores for each patient to compare their prognosis prediction capabilities. Subsequently, we developed a novel Cox regression model to predict in-hospital mortality, integrating the BCPI model and CURB-65 scores, aiming to assess whether this integration enhances predictive performance.

RESULTS

The predictive performance of the BCPI model and CURB-65 score for the 30-day mortality rate in ED patients and the in-hospital mortality rate among admitted patients was comparable across all risk categories. However, the Cox regression model demonstrated an improved area under the ROC curve (AUC) of 0.713 than that of CURB-65 (0.668) for in-hospital mortality (p<0.001). In the lowest risk group (CURB-65=0), the Cox regression model outperformed CURB-65, with a significantly lower mortality rate (2.9% vs. 7.7%, p<0.001).

CONCLUSIONS

The BCPI model, constructed using CBC/DC data and ML techniques, performs comparably to the widely utilized CURB-65 in predicting outcomes for patients with pneumonia in the ED. Furthermore, by integrating the CURB-65 score and BCPI model into a Cox regression model, we demonstrated improved prediction capabilities, particularly for low-risk patients. Given its simple parameters and easy training process, the Cox regression model may be a more effective prediction tool for classifying patients with pneumonia in the emergency room.

Machine learning-based prediction of in-ICU mortality in pneumonia patients

Conventional severity-of-illness scoring systems have shown suboptimal performance for predicting in-intensive care unit (ICU) mortality in patients with severe pneumonia. This study aimed to develop and validate machine learning (ML) models for mortality prediction in patients with severe pneumonia. This retrospective study evaluated patients admitted to the ICU for severe pneumonia between January 2016 and December 2021. The predictive performance was analyzed by comparing the area under the receiver operating characteristic curve (AU-ROC) of ML models to that of conventional severity-of-illness scoring systems. Three ML models were evaluated: (1) logistic regression with L2 regularization, (2) gradient-boosted decision tree (LightGBM), and (3) multilayer perceptron (MLP). Among the 816 pneumonia patients included, 223 (27.3%) patients died. All ML models significantly outperformed the Simplified Acute Physiology Score II (AU-ROC: 0.650 [0.584-0.716] vs 0.820 [0.771-0.869] for logistic regression vs 0.827 [0.777-0.876] for LightGBM 0.838 [0.791-0.884] for MLP; P < 0.001). In the analysis for NRI, the LightGBM and MLP models showed superior reclassification compared with the logistic regression model in predicting in-ICU mortality in all length of stay in the ICU subgroups; all age subgroups; all subgroups with any APACHE II score, PaO₂/FiO₂ ratio < 200; all subgroups with or without history of respiratory disease; with or without history of CVA or dementia; treatment with mechanical ventilation, and use of inotropic agents. In conclusion, the ML models have excellent performance in predicting in-ICU mortality in patients with severe pneumonia. Moreover, this study highlights the potential advantages of selecting individual ML models for predicting in-ICU mortality in different subgroups.

Haemogram indices are as reliable as CURB-65 to assess 30-day mortality in Covid-19 pneumonia

Background Mortality due to Covid-19 and severe community-acquired pneumonia (CAP) remains high, despite progress in critical care management. We compared the precision of CURB-65 score with monocyte-to-lymphocyte ratio (MLR), neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR) in prediction of mortality among patients with Covid-19 and CAP presenting to the emergency department. Methods We retrospectively analysed two cohorts of patients admitted to the emergency department of Canakkale University Hospital, namely (i) Covid-19 patients with severe acute respiratory symptoms presenting between 23 March 2020 and 31 October 2020, and (ii) all patients with CAP either from bacterial or viral infection within the 36 months preceding the Covid-19 pandemic. Mortality was defined as in-hospital death or death occurring within 30 days after discharge. Results The first study group consisted of 324 Covid-19 patients and the second group of 257 CAP patients. The non-survivor Covid-19 group had significantly higher MLR, NLR and PLR values. In univariate analysis, in Covid-19 patients, a 1-unit increase in NLR and PLR was associated with increased mortality, and in multivariate analysis for Covid-19 patients, age and NLR remained significant in the final step of the model. According to this model, we found that in the Covid-19 group an increase in 1-unit in NLR would result in an increase by 5% and 7% in the probability of mortality, respectively. According to pairwise analysis, NLR and PLR are as reliable as CURB-65 in predicting mortality in Covid-19. Conclusions Our study indicates that NLR and PLR may serve as reliable predictive factors as CURB-65 in Covid-19 pneumonia, which could easily be used to triage and manage severe patients in the emergency department.

Prediction of mortality in pneumonia patients with connective tissue disease treated with glucocorticoids or/and immunosuppressants by machine learning

Objectives

The assessment of accurate mortality risk is essential for managing pneumonia patients with connective tissue disease (CTD) treated with glucocorticoids or/and immunosuppressants. This study aimed to construct a nomogram for predicting 90-day mortality in pneumonia patients using machine learning.

Methods

Data were obtained from the DRYAD database. Pneumonia patients with CTD were screened. The samples were randomly divided into a training cohort (70%) and a validation cohort (30%). A univariate Cox regression analysis was used to screen for prognostic variables in the training cohort. Prognostic variables were entered into the least absolute shrinkage and selection operator (Lasso) and a random survival forest (RSF) analysis was used to screen important prognostic variables. The overlapping prognostic variables of the two algorithms were entered into the stepwise Cox regression analysis to screen the main prognostic variables and construct a model. Model predictive power was assessed using the C-index, the calibration curve, and the clinical subgroup analysis (age, gender, interstitial lung disease, diabetes mellitus). The clinical benefits of the model were assessed using a decision curve analysis (DCA). Similarly, the C-index was calculated and the calibration curve was plotted to verify the model stability in the validation cohort.

Results

A total of 368 pneumonia patients with CTD (training cohort: 247; validation cohort: 121) treated with glucocorticoids or/and immunosuppressants were included. The univariate Cox regression analysis obtained 19 prognostic variables. Lasso and RSF algorithms obtained eight overlapping variables. The overlapping variables were entered into a stepwise Cox regression to obtain five variables (fever, cyanosis, blood urea nitrogen, ganciclovir treatment, and anti-pseudomonas treatment), and a prognostic model was constructed based on the five variables. The C-index of the construction nomogram of the training cohort was 0.808. The calibration curve, DCA results, and clinical subgroup analysis showed that the model also had good predictive power. Similarly, the C-index of the model in the validation cohort was 0.762 and the calibration curve had good predictive value.

Conclusion

In this study, the nomogram developed performed well in predicting the 90-day risk of death in pneumonia patients with CTD treated with glucocorticoids or/and immunosuppressants.

Machine-Learning Model for Mortality Prediction in Patients With Community-Acquired Pneumonia: Development and Validation Study

BACKGROUND

Artificial intelligence tools and techniques such as machine learning (ML) are increasingly seen as a suitable manner in which to increase the prediction capacity of currently available clinical tools, including prognostic scores. However, studies evaluating the efficacy of ML methods in enhancing the predictive capacity of existing scores for community-acquired pneumonia (CAP) are limited. We aimed to apply and validate a causal probabilistic network (CPN) model to predict mortality in patients with CAP.

RESEARCH QUESTION

Is a CPN model able to predict mortality in patients with CAP better than the commonly used severity scores?

STUDY DESIGN AND METHODS

This was a derivation-validation retrospective study conducted in two Spanish university hospitals. The ability of a CPN designed to predict mortality in sepsis (SepsisFinder [SeF]), and adapted for CAP (SeF-ML), to predict 30-day mortality was assessed and compared with other scoring systems (Pneumonia Severity Index [PSI], Sequential Organ Failure Assessment [SOFA], quick Sequential Organ Failure Assessment [qSOFA], and CURB-65 criteria [confusion, urea, respiratory rate, BP, age ≥ 65 years]). The SeF models are proprietary software. Differences between receiver operating characteristic curves were assessed by the DeLong method for correlated receiver operating characteristic curves.

RESULTS

The derivation cohort comprised 4,531 patients, and the validation cohort consisted of 1,034 patients. In the derivation cohort, the areas under the curve (AUCs) of SeF-ML, CURB-65, SOFA, PSI, and qSOFA were 0.801, 0.759, 0.671, 0.799, and 0.642, respectively, for 30-day mortality prediction. In the validation study, the AUC of SeF-ML was 0.826, concordant with the AUC (0.801) in the derivation data (P = .51). The AUC of SeF-ML was significantly higher than those of CURB-65 (0.764; P = .03) and qSOFA (0.729, P = .005). However, it did not differ significantly from those of PSI (0.830; P = .92) and SOFA (0.771; P = .14).

INTERPRETATION

SeF-ML shows potential for improving mortality prediction among patients with CAP, using structured health data. Additional external validation studies should be conducted to support generalizability.

Machine Learning Model Development and Validation for Predicting Outcome in Stage 4 Solid Cancer Patients with Septic Shock Visiting the Emergency Department: A Multi-Center, Prospective Cohort Study

A reliable prognostic score for minimizing futile treatments in advanced cancer patients with septic shock is rare. A machine learning (ML) model to classify the risk of advanced cancer patients with septic shock is proposed and compared with the existing scoring systems. A multi-center, retrospective, observational study of the septic shock registry in patients with stage 4 cancer was divided into a training set and a test set in a 7:3 ratio. The primary outcome was 28-day mortality. The best ML model was determined using a stratified 10-fold cross-validation in the training set. A total of 897 patients were included, and the 28-day mortality was 26.4%. The best ML model in the training set was balanced random forest (BRF), with an area under the curve (AUC) of 0.821 to predict 28-day mortality. The AUC of the BRF to predict the 28-day mortality in the test set was 0.859. The AUC of the BRF was significantly higher than those of the Sequential Organ Failure Assessment score and the Acute Physiology and Chronic Health Evaluation II score (both p < 0.001). The ML model outperformed the existing scores for predicting 28-day mortality in stage 4 cancer patients with septic shock. However, further studies are needed to improve the prediction algorithm and to validate it in various countries. This model might support clinicians in real-time to adopt appropriate levels of care.

Benchmarking emergency department prediction models with machine learning and public electronic health records

The demand for emergency department (ED) services is increasing across the globe, particularly during the current COVID-19 pandemic. Clinical triage and risk assessment have become increasingly challenging due to the shortage of medical resources and the strain on hospital infrastructure caused by the pandemic. As a result of the widespread use of electronic health records (EHRs), we now have access to a vast amount of clinical data, which allows us to develop prediction models and decision support systems to address these challenges. To date, there is no widely accepted clinical prediction benchmark related to the ED based on large-scale public EHRs. An open-source benchmark data platform would streamline research workflows by eliminating cumbersome data preprocessing, and facilitate comparisons among different studies and methodologies. Based on the Medical Information Mart for Intensive Care IV Emergency Department (MIMIC-IV-ED) database, we created a benchmark dataset and proposed three clinical prediction benchmarks. This study provides future researchers with insights, suggestions, and protocols for managing data and developing predictive tools for emergency care.

Incorporation of Suppression of Tumorigenicity 2 into Random Survival Forests for Enhancing Prediction of Short-Term Prognosis in Community-ACQUIRED Pneumonia

(1) Background: Biomarker and model development can help physicians adjust the management of patients with community-acquired pneumonia (CAP) by screening for inpatients with a low probability of cure early in their admission; (2) Methods: We conducted a 30-day cohort study of newly admitted adult CAP patients over 20 years of age. Prognosis models to predict the short-term prognosis were developed using random survival forest (RSF) method; (3) Results: A total of 247 adult CAP patients were studied and 208 (84.21%) of them reached clinical stability within 30 days. The soluble form of suppression of tumorigenicity-2 (sST2) was an independent predictor of clinical stability and the addition of sST2 to the prognosis model could improve the performance of the prognosis model. The C-index of the RSF model for predicting clinical stability was 0.8342 (95% CI, 0.8086–0.8598), which is higher than 0.7181 (95% CI, 0.6933–0.7429) of CURB 65 score, 0.8025 (95% CI, 0.7776–8274) of PSI score, and 0.8214 (95% CI, 0.8080–0.8348) of cox regression. In addition, the RSF model was associated with adverse clinical events during hospitalization, ICU admissions, and short-term mortality; (4) Conclusions: The RSF model by incorporating sST2 was more accurate than traditional methods in assessing the short-term prognosis of CAP patients.

Pneumonia Update for Emergency Clinicians

Purpose of Review

Many new concepts in diagnosis, management, and risk stratification of patients with pneumonia have been described recently. The COVID pandemic made importance of viruses as dangerous pathogens of pneumonia quite clear while several non-invasive measures for patients with respiratory failure gained a more wide-spread usage.

Recent Findings

Studies continue to examine feasibility of bedside ultrasound as a tool in accurate diagnosis of pneumonia in the emergency department, and several new antibiotics have been approved for treatment while others are in late-stage clinical trials. Additionally, the Infectious Diseases Society, American Thoracic Society, and their European counterparts published updated guidelines in recent years. For differences important to emergency medicine clinicians and new emphasis as compared to the prior guidelines, please see Table 1. Several new antibiotics have been approved recently but remain relatively unknown to emergency clinicians as their use is frequently restricted to infectious disease specialists.Table 1

Performance of Machine Learning Algorithms for Predicting Adverse Outcomes in Community-Acquired Pneumonia

Background: The ability to assess adverse outcomes in patients with community-acquired pneumonia (CAP) could improve clinical decision-making to enhance clinical practice, but the studies remain insufficient, and similarly, few machine learning (ML) models have been developed.

Objective: We aimed to explore the effectiveness of predicting adverse outcomes in CAP through ML models.

Methods: A total of 2,302 adults with CAP who were prospectively recruited between January 2012 and March 2015 across three cities in South America were extracted from DryadData. After a 70:30 training set: test set split of the data, nine ML algorithms were executed and their diagnostic accuracy was measured mainly by the area under the curve (AUC). The nine ML algorithms included decision trees, random forests, extreme gradient boosting (XGBoost), support vector machines, Naïve Bayes, K-nearest neighbors, ridge regression, logistic regression without regularization, and neural networks. The adverse outcomes included hospital admission, mortality, ICU admission, and one-year post-enrollment status.

Results: The XGBoost algorithm had the best performance in predicting hospital admission. Its AUC reached 0.921, and accuracy, precision, recall, and F1-score were better than those of other models. In the prediction of ICU admission, a model trained with the XGBoost algorithm showed the best performance with AUC 0.801. XGBoost algorithm also did a good job at predicting one-year post-enrollment status. The results of AUC, accuracy, precision, recall, and F1-score indicated the algorithm had high accuracy and precision. In addition, the best performance was seen by the neural network algorithm when predicting death (AUC 0.831).

Conclusions: ML algorithms, particularly the XGBoost algorithm, were feasible and effective in predicting adverse outcomes of CAP patients. The ML models based on available common clinical features had great potential to guide individual treatment and subsequent clinical decisions.

Calibration-Free Cuffless Blood Pressure Estimation Based on a Population With a Diverse Range of Age and Blood Pressure

This study presents a new blood pressure (BP) estimation algorithm utilizing machine learning (ML). A cuffless device that can measure BP without calibration would be precious for portability, continuous measurement, and comfortability, but unfortunately, it does not currently exist. Conventional BP measurement with a cuff is standard, but this method has various problems like inaccurate BP measurement, poor portability, and painful cuff pressure. To overcome these disadvantages, many researchers have developed cuffless BP estimation devices. However, these devices are not clinically applicable because they require advanced preparation before use, such as calibration, do not follow international standards (81060-1:2007), or have been designed using insufficient data sets. The present study was conducted to combat these issues. We recruited 127 participants and obtained 878 raw datasets. According to international standards, our diverse data set included participants from different age groups with a wide variety of blood pressures. We utilized ML to formulate a BP estimation method that did not require calibration. The present study also conformed to the method required by international standards while calculating the level of error in BP estimation. Two essential methods were applied in this study: (a) grouping the participants into five subsets based on the relationship between the pulse transit time and systolic BP by a support vector machine ensemble with bagging (b) applying the information from the wavelet transformation of the pulse wave and the electrocardiogram to the linear regression BP estimation model for each group. For systolic BP, the standard deviation of error for the proposed BP estimation results with cross-validation was 7.74 mmHg, which was an improvement from 17.05 mmHg, as estimated by the conventional pulse-transit-time-based methods. For diastolic BP, the standard deviation of error was 6.42 mmHg for the proposed BP estimation, which was an improvement from 14.05mmHg. The purpose of the present study was to demonstrate and evaluate the performance of the newly developed BP estimation ML method that meets the international standard for non-invasive sphygmomanometers in a population with a diverse range of age and BP.

Real-time interactive artificial intelligence of things-based prediction for adverse outcomes in adult patients with pneumonia in the emergency department

BACKGROUND

Artificial intelligence of things (AIoT) may be a solution for predicting adverse outcomes in emergency department (ED) patients with pneumonia; however, this issue remains unclear. Therefore, we conducted this study to clarify it.

METHODS

We identified 52,626 adult ED patients with pneumonia from three hospitals between 2010 and 2019 for this study. Thirty-three feature variables from electronic medical records were used to construct an artificial intelligence (AI) model to predict sepsis or septic shock, respiratory failure, and mortality. After comparisons of the predictive accuracies among logistic regression, random forest, support-vector machine (SVM), light gradient boosting machine (LightGBM), multilayer perceptron (MLP), and eXtreme Gradient Boosting (XGBoost), we selected the best one to build the model. We further combined the AI model with the Internet of things as AIoT, added an interactive mode, and implemented it in the hospital information system to assist clinicians with decision making in real time. We also compared the AIoT-based model with the confusion-urea-respiratory rate-blood pressure-65 (CURB-65) and pneumonia severity index (PSI) for predicting mortality.

RESULTS

The best AI algorithms were random forest for sepsis or septic shock (area under the curve [AUC] = 0.781), LightGBM for respiratory failure (AUC = 0.847), and mortality (AUC = 0.835). The AIoT-based model represented better performance than CURB-65 and PSI indicators for predicting mortality (0.835 vs. 0.681 and 0.835 vs. 0.728).

CONCLUSIONS

A real-time interactive AIoT-based model might be a better tool for predicting adverse outcomes in ED patients with pneumonia. Further validation in other populations is warranted.

Evaluation and management of pleural sepsis

Pleural sepsis stems from an infection within the pleural space typically from an underlying bacterial pneumonia leading to development of a parapneumonic effusion. This effusion is traditionally divided into uncomplicated, complicated, and empyema. Poor clinical outcomes and increased mortality can be associated with the development of parapneumonic effusions, reinforcing the importance of early recognition and diagnosis. Management necessitates a multimodal therapeutic strategy consisting of antimicrobials, catheter/tube thoracostomy, and at times, video-assisted thoracoscopic surgery.

Predicting ventilator-associated pneumonia with machine learning

Ventilator-associated pneumonia (VAP) is the most common and fatal nosocomial infection in intensive care units (ICUs). Existing methods for identifying VAP display low accuracy, and their use may delay antimicrobial therapy. VAP diagnostics derived from machine learning (ML) methods that utilize electronic health record (EHR) data have not yet been explored. The objective of this study is to compare the performance of a variety of ML models trained to predict whether VAP will be diagnosed during the patient stay.A retrospective study examined data from 6126 adult ICU encounters lasting at least 48 hours following the initiation of mechanical ventilation. The gold standard was the presence of a diagnostic code for VAP. Five different ML models were trained to predict VAP 48 hours after initiation of mechanical ventilation. Model performance was evaluated with regard to the area under the receiver operating characteristic (AUROC) curve on a 20% hold-out test set. Feature importance was measured in terms of Shapley values.The highest performing model achieved an AUROC value of 0.854. The most important features for the best-performing model were the length of time on mechanical ventilation, the presence of antibiotics, sputum test frequency, and the most recent Glasgow Coma Scale assessment.Supervised ML using patient EHR data is promising for VAP diagnosis and warrants further validation. This tool has the potential to aid the timely diagnosis of VAP.