Clinical- vs. model-based selection of patients suspected of sepsis for direct-from-blood rapid diagnostics in the emergency department: a retrospective study

Machine-Learning Model for Mortality Prediction in Patients With Community-Acquired Pneumonia: Development and Validation Study

BACKGROUND

Artificial intelligence tools and techniques such as machine learning (ML) are increasingly seen as a suitable manner in which to increase the prediction capacity of currently available clinical tools, including prognostic scores. However, studies evaluating the efficacy of ML methods in enhancing the predictive capacity of existing scores for community-acquired pneumonia (CAP) are limited. We aimed to apply and validate a causal probabilistic network (CPN) model to predict mortality in patients with CAP.

RESEARCH QUESTION

Is a CPN model able to predict mortality in patients with CAP better than the commonly used severity scores?

STUDY DESIGN AND METHODS

This was a derivation-validation retrospective study conducted in two Spanish university hospitals. The ability of a CPN designed to predict mortality in sepsis (SepsisFinder [SeF]), and adapted for CAP (SeF-ML), to predict 30-day mortality was assessed and compared with other scoring systems (Pneumonia Severity Index [PSI], Sequential Organ Failure Assessment [SOFA], quick Sequential Organ Failure Assessment [qSOFA], and CURB-65 criteria [confusion, urea, respiratory rate, BP, age ≥ 65 years]). The SeF models are proprietary software. Differences between receiver operating characteristic curves were assessed by the DeLong method for correlated receiver operating characteristic curves.

RESULTS

The derivation cohort comprised 4,531 patients, and the validation cohort consisted of 1,034 patients. In the derivation cohort, the areas under the curve (AUCs) of SeF-ML, CURB-65, SOFA, PSI, and qSOFA were 0.801, 0.759, 0.671, 0.799, and 0.642, respectively, for 30-day mortality prediction. In the validation study, the AUC of SeF-ML was 0.826, concordant with the AUC (0.801) in the derivation data (P = .51). The AUC of SeF-ML was significantly higher than those of CURB-65 (0.764; P = .03) and qSOFA (0.729, P = .005). However, it did not differ significantly from those of PSI (0.830; P = .92) and SOFA (0.771; P = .14).

INTERPRETATION

SeF-ML shows potential for improving mortality prediction among patients with CAP, using structured health data. Additional external validation studies should be conducted to support generalizability.

A comparison of predictors for mortality and bacteraemia in patients suspected of infection

BACKGROUND

Stratification by clinical scores of patients suspected of infection can be used to support decisions on treatment and diagnostic workup. Seven clinical scores, SepsisFinder (SF), National Early Warning Score (NEWS), Sequential Orgen Failure Assessment (SOFA), Mortality in Emergency Department Sepsis (MEDS), quick SOFA (qSOFA), Shapiro Decision Rule (SDR) and Systemic Inflammatory Response Syndrome (SIRS), were evaluated for their ability to predict 30-day mortality and bacteraemia and for their ability to identify a low risk group, where blood culture may not be cost-effective and a high risk group where direct-from-blood PCR (dfbPCR) may be cost effective.

METHODS

Retrospective data from two Danish and an Israeli hospital with a total of 1816 patients were used to calculate the seven scores.

RESULTS

SF had higher Area Under the Receiver Operating curve than the clinical scores for prediction of mortality and bacteraemia, significantly so for MEDS, qSOFA and SIRS. For mortality predictions SF also had significantly higher area under the curve than SDR. In a low risk group identified by SF, consisting of 33% of the patients only 1.7% had bacteraemia and mortality was 4.2%, giving a cost of € 1976 for one positive result by blood culture. This was higher than the cost of € 502 of one positive dfbPCR from a high risk group consisting of 10% of the patients, where 25.3% had bacteraemia and mortality was 24.2%.

CONCLUSION

This may motivate a health economic study of whether resources spent on low risk blood cultures might be better spent on high risk dfbPCR.