External Validation and Recalibration of the CURB-65 and PSI for Predicting 30-Day Mortality and Critical Care Intervention in Multiethnic Patients with COVID-19

Boosting the accuracy of existing models by updating and extending: using a multicenter COVID-19 ICU cohort as a proxy

Most published prediction models for Coronavirus Disease 2019 (COVID-19) were poorly reported, at high risk of bias, and heterogeneous in model performance. To tackle methodological challenges faced in previous prediction studies, we investigated whether model updating and extending improves mortality prediction, using the Intensive Care Unit (ICU) as a proxy. All COVID-19 patients admitted to seven ICUs in the Euregio-Meuse Rhine during the first pandemic wave were included. The 4C Mortality and SEIMC scores were selected as promising prognostic models from an external validation study. Five predictors could be estimated based on cohort size. TRIPOD guidelines were followed and logistic regression analyses with the linear predictor, APACHE II score, and country were performed. Bootstrapping with backward selection was applied to select variables for the final model. Additionally, shrinkage was performed. Model discrimination was displayed as optimism-corrected areas under the ROC curve and calibration by calibration slopes and plots. The mortality rate of the 551 included patients was 36%. Discrimination of the 4C Mortality and SEIMC scores increased from 0.70 to 0.74 and 0.70 to 0.73 and calibration plots improved compared to the original models after updating and extending. Mortality prediction can be improved after updating and extending of promising models.

The Battle of the Pneumonia Predictors: A Comprehensive Meta-Analysis Comparing the Pneumonia Severity Index (PSI) and the CURB-65 Score in Predicting Mortality and the Need for ICU Support

The CURB-65 (confusion, uremia, respiratory rate, blood pressure, age ≥ 65 years) score and the pneumonia severity index (PSI) are widely used and recommended in predicting 30-day mortality and the need for intensive care support in community-acquired pneumonia. This study aims to compare the performance of these two severity scores in both mortality prediction and the need for intensive care support. A systematic review and meta-analysis was carried out, following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) 2020 guidelines, and PubMed, Scopus, ScienceDirect, and Google Scholar were searched for articles published from 2012 to 2022. The reference lists of the included studies were also searched to retrieve possible additional studies. Twenty-five studies reporting prognostic information for CURB 65 and PSI were identified. ReviewManager (RevMan) 5.4.1 was used to produce risk ratios, and a random effects model was used to pool them. Both PSI and CURB-65 showed a high strength in identifying high-risk patients. However, CURB-65 was slightly better in early mortality prediction and had more sensitivity (96.7%) and specificity (89.3%) in predicting admission to intensive care support. Thus, CURB-65 seems to be the preferred tool in predicting mortality and the need for admission into intensive care support.

Severity of Illness Scores and Biomarkers for Prognosis of Patients with Coronavirus Disease 2019

The spectrum of disease severity and the insidiousness of clinical presentation make it difficult to recognize patients with coronavirus disease 2019 (COVID-19) at higher risk of worse outcomes or death when they are seen in the early phases of the disease. There are now well-established risk factors for worse outcomes in patients with COVID-19. These should be factored in when assessing the prognosis of these patients. However, a more precise prognostic assessment in an individual patient may warrant the use of predictive tools. In this manuscript, we conduct a literature review on the severity of illness scores and biomarkers for the prognosis of patients with COVID-19. Several COVID-19-specific scores have been developed since the onset of the pandemic. Some of them are promising and can be integrated into the assessment of these patients. We also found that the well-known pneumonia severity index (PSI) and CURB-65 (confusion, uremia, respiratory rate, BP, age ≥ 65 years) are good predictors of mortality in hospitalized patients with COVID-19. While neither the PSI nor the CURB-65 should be used for the triage of outpatient versus inpatient treatment, they can be integrated by a clinician into the assessment of disease severity and can be used in epidemiological studies to determine the severity of illness in patient populations. Biomarkers also provide valuable prognostic information and, importantly, may depict the main physiological derangements in severe disease. We, however, do not advocate the isolated use of severity of illness scores or biomarkers for decision-making in an individual patient. Instead, we suggest the use of these tools on a case-by-case basis with the goal of enhancing clinician judgment.

Comparison Between Physiological Scores SIPF, CURB-65, and APACHE II as Predictors of Prognosis and Mortality in Hospitalized Patients with COVID-19 Pneumonia: A Multicenter Study, Saudi Arabia

Background

A coronavirus pandemic (COVID-19) is associated with catastrophic effects on the world with high morbidity and mortality. We aimed to evaluate the accuracy of physiological shock index (SIPF) (shock index and hypoxemia), CURB -65, acute physiology, and chronic health assessment II (APACHE II) as predictors of prognosis and in-hospital mortality in patients with COVID-19 pneumonia.

Methods

In Saudi Arabia, a multicenter retrospective study was conducted on hospitalized adult patients confirmed to have COVID-19 pneumonia. Information needed to calculate SIPF, CURB-65, and APACHE II scores were obtained from medical records within 24 hours of admission.

Results

The study included 1131 COVID-19 patients who met the inclusion criteria. They were divided into two groups: (A) the ICU group (n=340; 30.1%) and (B) the ward group (n=791; 69.9%). The most common concomitant diseases of patients at initial ICU admission were hypertension (71.5%) and diabetes (62.4%), and most of them were men (63.8%). The overall mortality was 18.7%, and the mortality rate was higher in the ICU group than in the ward group (39.4% vs 9.6%; p < 0.001). The SIPF score showed a significantly higher ability to predict both ICU admission and mortality in patients with COVID-19 pneumonia compared with APACHE II and CURB -65; (AUC 0.89 vs 0.87; p < 0.001) and (AUC 0.89 vs 0.84; p < 0.001) for ICU admission and (AUC 0.90 vs 0.65; p < 0.001) and (AUC 0.90 vs 0.80; p < 0.001) for mortality, respectively.

Conclusion

The ability of the SIPF score to predict ICU admission and mortality in COVID-19 pneumonia is higher than that of APACHE II and CURB-65. The overall mortality was 18.7%, and the mortality rate was higher in the ICU group than in the ward group (39.4% vs 9.6%; p < 0.001).

A practical scoring model to predict the occurrence of critical illness in hospitalized patients with SARS-CoV-2 omicron infection

Background

The variants of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) have emerged repeatedly, especially the Omicron strain which is extremely infectious, so early identification of patients who may develop critical illness will aid in delivering proper treatment and optimizing use of resources. We aimed to develop and validate a practical scoring model at hospital admission for predicting which patients with Omicron infection will develop critical illness.

Methods

A total of 2,459 patients with Omicron infection were enrolled in this retrospective study. Univariate and multivariate logistic regression analysis were performed to evaluate predictors associated with critical illness. Moreover, the area under the receiver operating characteristic curve (AUROC), continuous net reclassification improvement, and integrated discrimination index were assessed.

Results

The derivation cohort included 1721 patients and the validation cohort included 738 patients. A total of 98 patients developed critical illness. Thirteen variables were independent predictive factors and were included in the risk score: age > 65, C-reactive protein > 10 mg/L, lactate dehydrogenase > 250 U/L, lymphocyte < 0.8*10^⁹/L, white blood cell > 10*10^⁹/L, Oxygen saturation < 90%, malignancy, chronic kidney disease, chronic cardiac disease, chronic obstructive pulmonary disease, diabetes, cerebrovascular disease, and non-vaccination. AUROC in the derivation cohort and validation cohort were 0.926 (95% CI, 0.903-0.948) and 0.907 (95% CI, 0.860-0.955), respectively. Moreover, the critical illness risk scoring model had the highest AUROC compared with CURB-65, sequential organ failure assessment (SOFA) and 4C mortality scores, and always obtained more net benefit.

Conclusion

The risk scoring model based on the characteristics of patients at the time of admission to the hospital may help medical practitioners to identify critically ill patients and take prompt measures.

Comparison of prognostic scores for inpatients with COVID-19: a retrospective monocentric cohort study

Background

The SARS-CoV-2 pandemic led to a steep increase in hospital and intensive care unit (ICU) admissions for acute respiratory failure worldwide. Early identification of patients at risk of clinical deterioration is crucial in terms of appropriate care delivery and resource allocation. We aimed to evaluate and compare the prognostic performance of Sequential Organ Failure Assessment (SOFA), Quick Sequential Organ Failure Assessment (qSOFA), Confusion, Uraemia, Respiratory Rate, Blood Pressure and Age ≥65 (CURB-65), Respiratory Rate and Oxygenation (ROX) index and Coronavirus Clinical Characterisation Consortium (4C) score to predict death and ICU admission among patients admitted to the hospital for acute COVID-19 infection.

Methods and analysis

Consecutive adult patients admitted to the Geneva University Hospitals during two successive COVID-19 flares in spring and autumn 2020 were included. Discriminative performance of these prediction rules, obtained during the first 24 hours of hospital admission, were computed to predict death or ICU admission. We further exluded patients with therapeutic limitations and reported areas under the curve (AUCs) for 30-day mortality and ICU admission in sensitivity analyses.

Results

A total of 2122 patients were included. 216 patients (10.2%) required ICU admission and 303 (14.3%) died within 30 days post admission. 4C score had the best discriminatory performance to predict 30-day mortality (AUC 0.82, 95% CI 0.80 to 0.85), compared with SOFA (AUC 0.75, 95% CI 0.72 to 0.78), qSOFA (AUC 0.59, 95% CI 0.56 to 0.62), CURB-65 (AUC 0.75, 95% CI 0.72 to 0.78) and ROX index (AUC 0.68, 95% CI 0.65 to 0.72). ROX index had the greatest discriminatory performance (AUC 0.79, 95% CI 0.76 to 0.83) to predict ICU admission compared with 4C score (AUC 0.62, 95% CI 0.59 to 0.66), CURB-65 (AUC 0.60, 95% CI 0.56 to 0.64), SOFA (AUC 0.74, 95% CI 0.71 to 0.77) and qSOFA (AUC 0.59, 95% CI 0.55 to 0.62).

Conclusion

Scores including age and/or comorbidities (4C and CURB-65) have the best discriminatory performance to predict mortality among inpatients with COVID-19, while scores including quantitative assessment of hypoxaemia (SOFA and ROX index) perform best to predict ICU admission. Exclusion of patients with therapeutic limitations improved the discriminatory performance of prognostic scores relying on age and/or comorbidities to predict ICU admission.

External validation of risk scores to predict in-hospital mortality in patients hospitalized due to coronavirus disease 2019

BACKGROUND

The coronavirus disease 2019 (COVID-19) presents an urgent threat to global health. Prediction models that accurately estimate mortality risk in hospitalized patients could assist medical staff in treatment and allocating limited resources.

AIMS

To externally validate two promising previously published risk scores that predict in-hospital mortality among hospitalized COVID-19 patients.

METHODS

Two prospective cohorts were available; a cohort of 1028 patients admitted to one of nine hospitals in Lombardy, Italy (the Lombardy cohort) and a cohort of 432 patients admitted to a hospital in Leiden, the Netherlands (the Leiden cohort). The endpoint was in-hospital mortality. All patients were adult and tested COVID-19 PCR-positive. Model discrimination and calibration were assessed.

RESULTS

The C-statistic of the 4C mortality score was good in the Lombardy cohort (0.85, 95CI: 0.82-0.89) and in the Leiden cohort (0.87, 95CI: 0.80-0.94). Model calibration was acceptable in the Lombardy cohort but poor in the Leiden cohort due to the model systematically overpredicting the mortality risk for all patients. The C-statistic of the CURB-65 score was good in the Lombardy cohort (0.80, 95CI: 0.75-0.85) and in the Leiden cohort (0.82, 95CI: 0.76-0.88). The mortality rate in the CURB-65 development cohort was much lower than the mortality rate in the Lombardy cohort. A similar but less pronounced trend was found for patients in the Leiden cohort.

CONCLUSION

Although performances did not differ greatly, the 4C mortality score showed the best performance. However, because of quickly changing circumstances, model recalibration may be necessary before using the 4C mortality score.

A Simple Prognostic Score for Critical COVID-19 Derived from Patients without Comorbidities Performs Well in Unselected Patients

We aimed to search for laboratory predictors of critical COVID-19 in consecutive adults admitted in an academic center between 16 September 2020−20 December 2021. Patients were uniformly treated with low-molecular-weight heparin, and dexamethasone plus remdesivir when SpO2 < 94%. Among consecutive unvaccinated patients without underlying medical conditions (n = 241, 49 year-old median, 71% males), 22 (9.1%) developed critical disease and 2 died (0.8%). White-blood-cell counts, neutrophils, neutrophil-to-lymphocyte ratio, CRP, fibrinogen, ferritin, LDH and γ-GT at admission were each univariably associated with critical disease. ROC-defined cutoffs revealed that CRP > 61.8 mg/L, fibrinogen > 616.5 mg/dL and LDH > 380.5 U/L were each associated with critical disease development, independently of age, sex and days from symptom-onset. A score combining higher-than-cutoff CRP (0/2), LDH (0/1) and fibrinogen (0/1) predicted critical disease (AUC: 0.873, 95% CI: 0.820−0.926). This score performed well in an unselected patient cohort (n = 1228, 100% unvaccinated) predominantly infected by the alpha variant (AUC: 0.718, 95% CI: 0.683−0.753), as well as in a mixed cohort (n = 527, 65% unvaccinated) predominantly infected by the delta variant (AUC: 0.708, 95% CI: 0.656−0.760). Therefore, we propose that a combination of standard biomarkers of acute inflammatory response, cell death and hypercoagulability reflects the severity of COVID-19 per se independently of comorbidities, age and sex, being of value for risk stratification in unselected patients.

Exploratory COVID-19 death risk score based on basic laboratory tests and physiological clinical measurements

BACKGROUND

In the event of a sudden shortage of medical resources, a rapid, simple, and accurate prediction model is essential for the 30-day mortality rate of patients with COVID-19.

METHODS

This retrospective study compared the characteristics of the survivals and non-survivals of 278 patients with COVID-19. Logistic regression analysis was performed to obtain the "COVID-19 death risk score" (CDRS) model. Using the area under the receiver operating characteristic (AUROC) curve and Hosmer-Lemeshow goodness-of-fit test, discrimination and calibration were assessed. Internal validation was conducted using a regular bootstrap method.

RESULTS

A total of 63 (22.66%) of 278 included patients died. The logistic regression analysis revealed that high-sensitivity C-reactive protein (hsCRP; odds ratio [OR]=1.018), D-dimer (OR=1.101), and respiratory rate (RR; OR=1.185) were independently associated with 30-day mortality. CDRS was calculated as follows: CDRS=-10.245+(0.022×hsCRP)+(0.172×D-dimer)+(0.203×RR). CDRS had the same predictive effect as the sequential organ failure assessment (SOFA) and "confusion, uremia, respiratory rate, blood pressure, and age over 65 years" (CURB-65) scores, with AUROCs of 0.984 for CDRS, 0.975 for SOFA, and 0.971 for CURB-65, respectively. And CDRS showed good calibration. The AUROC through internal validations was 0.980 (95% confidence interval [CI]: 0.965-0.995). Regarding the clinical value, the decision curve analysis of CDRS showed a net value similar to that of CURB-65 in this cohort.

CONCLUSION

CDRS is a novel, efficient and accurate prediction model for the early identification of COVID-19 patients with poor outcomes. Although it is not as advanced as the other models, CDRS had a similar performance to that of SOFA and CURB-65.