Prospective validation of the 4C prognostic models for adults hospitalised with COVID-19 using the ISARIC WHO Clinical Characterisation Protocol

External validation of two clinical prediction models for mortality in COVID-19 patients (4C and NEWS2), in three centers in Medellín, Colombia: Assessing the impact of vaccination over time

OBJECTIVES

External validation of the 4C and NEWS2 scores for the prediction of in-hospital mortality in COVID-19 patients, and evaluation of its operational performance in two time periods: before and after the start of the vaccination program in Colombia.

METHODS

Retrospective cohort in three high complexity hospitals in the city of Medellín, Colombia, between June 2020 and April 2022.

RESULTS

The areas under the ROC curve (AUC) for the 4C mortality risk score and the NEWS2 were 0.75 (95% CI 0.73-0.78) and 0.68 (95% CI 0.66-0.71), respectively. For the 4C score, the AUC for the first and second periods was 0.77 (95% CI 0.74-0.80) and 0.75 (95% CI 0.71-0.78); whilst for the NEWS2 score, it was 0.68 (95% CI 0.65-0.71) and 0.69 (95% CI 0.64-0.73). The calibration for both scores was adequate, albeit with reduced performance during the second period.

CONCLUSIONS

The 4C mortality risk score proved to be the more adequate predictor of in-hospital mortality in COVID-19 patients in this Latin American population. The operational performance during both time periods remained similar, which shows its utility notwithstanding major changes, including vaccination, as the pandemic evolved.

External validation of six COVID-19 prognostic models for predicting mortality risk in older populations in a hospital, primary care, and nursing home setting

OBJECTIVES

To systematically evaluate the performance of COVID-19 prognostic models and scores for mortality risk in older populations across three health-care settings: hospitals, primary care, and nursing homes.

STUDY DESIGN AND SETTING

This retrospective external validation study included 14,092 older individuals of ≥70 years of age with a clinical or polymerase chain reaction-confirmed COVID-19 diagnosis from March 2020 to December 2020. The six validation cohorts include three hospital-based (CliniCo, COVID-OLD, COVID-PREDICT), two primary care-based (Julius General Practitioners Network/Academisch network huisartsgeneeskunde/Network of Academic general Practitioners, PHARMO), and one nursing home cohort (YSIS) in the Netherlands. Based on a living systematic review of COVID-19 prediction models using Prediction model Risk Of Bias ASsessment Tool for quality and risk of bias assessment and considering predictor availability in validation cohorts, we selected six prognostic models predicting mortality risk in adults with COVID-19 infection (GAL-COVID-19 mortality, 4C Mortality Score, National Early Warning Score 2-extended model, Xie model, Wang clinical model, and CURB65 score). All six prognostic models were validated in the hospital cohorts and the GAL-COVID-19 mortality model was validated in all three healthcare settings. The primary outcome was in-hospital mortality for hospitals and 28-day mortality for primary care and nursing home settings. Model performance was evaluated in each validation cohort separately in terms of discrimination, calibration, and decision curves. An intercept update was performed in models indicating miscalibration followed by predictive performance re-evaluation.

MAIN OUTCOME MEASURE

In-hospital mortality for hospitals and 28-day mortality for primary care and nursing home setting.

RESULTS

All six prognostic models performed poorly and showed miscalibration in the older population cohorts. In the hospital settings, model performance ranged from calibration-in-the-large -1.45 to 7.46, calibration slopes 0.24-0.81, and C-statistic 0.55-0.71 with 4C Mortality Score performing as the most discriminative and well-calibrated model. Performance across health-care settings was similar for the GAL-COVID-19 model, with a calibration-in-the-large in the range of -2.35 to -0.15 indicating overestimation, calibration slopes of 0.24-0.81 indicating signs of overfitting, and C-statistic of 0.55-0.71.

CONCLUSION

Our results show that most prognostic models for predicting mortality risk performed poorly in the older population with COVID-19, in each health-care setting: hospital, primary care, and nursing home settings. Insights into factors influencing predictive model performance in the older population are needed for pandemic preparedness and reliable prognostication of health-related outcomes in this demographic.

Development and Validation of a Robust and Interpretable Early Triaging Support System for Patients Hospitalized With COVID-19: Predictive Algorithm Modeling and Interpretation Study

BACKGROUND

Robust and accurate prediction of severity for patients with COVID-19 is crucial for patient triaging decisions. Many proposed models were prone to either high bias risk or low-to-moderate discrimination. Some also suffered from a lack of clinical interpretability and were developed based on early pandemic period data. Hence, there has been a compelling need for advancements in prediction models for better clinical applicability.

OBJECTIVE

The primary objective of this study was to develop and validate a machine learning-based Robust and Interpretable Early Triaging Support (RIETS) system that predicts severity progression (involving any of the following events: intensive care unit admission, in-hospital death, mechanical ventilation required, or extracorporeal membrane oxygenation required) within 15 days upon hospitalization based on routinely available clinical and laboratory biomarkers.

METHODS

We included data from 5945 hospitalized patients with COVID-19 from 19 hospitals in South Korea collected between January 2020 and August 2022. For model development and external validation, the whole data set was partitioned into 2 independent cohorts by stratified random cluster sampling according to hospital type (general and tertiary care) and geographical location (metropolitan and nonmetropolitan). Machine learning models were trained and internally validated through a cross-validation technique on the development cohort. They were externally validated using a bootstrapped sampling technique on the external validation cohort. The best-performing model was selected primarily based on the area under the receiver operating characteristic curve (AUROC), and its robustness was evaluated using bias risk assessment. For model interpretability, we used Shapley and patient clustering methods.

RESULTS

Our final model, RIETS, was developed based on a deep neural network of 11 clinical and laboratory biomarkers that are readily available within the first day of hospitalization. The features predictive of severity included lactate dehydrogenase, age, absolute lymphocyte count, dyspnea, respiratory rate, diabetes mellitus, c-reactive protein, absolute neutrophil count, platelet count, white blood cell count, and saturation of peripheral oxygen. RIETS demonstrated excellent discrimination (AUROC=0.937; 95% CI 0.935-0.938) with high calibration (integrated calibration index=0.041), satisfied all the criteria of low bias risk in a risk assessment tool, and provided detailed interpretations of model parameters and patient clusters. In addition, RIETS showed potential for transportability across variant periods with its sustainable prediction on Omicron cases (AUROC=0.903, 95% CI 0.897-0.910).

CONCLUSIONS

RIETS was developed and validated to assist early triaging by promptly predicting the severity of hospitalized patients with COVID-19. Its high performance with low bias risk ensures considerably reliable prediction. The use of a nationwide multicenter cohort in the model development and validation implicates generalizability. The use of routinely collected features may enable wide adaptability. Interpretations of model parameters and patients can promote clinical applicability. Together, we anticipate that RIETS will facilitate the patient triaging workflow and efficient resource allocation when incorporated into a routine clinical practice.

Development and external validation of a prediction model for the transition from mild to moderate or severe form of COVID-19

OBJECTIVES

COVID-19 pandemic seems to be under control. However, despite the vaccines, 5 to 10% of the patients with mild disease develop moderate to critical forms with potential lethal evolution. In addition to assess lung infection spread, chest CT helps to detect complications. Developing a prediction model to identify at-risk patients of worsening from mild COVID-19 combining simple clinical and biological parameters with qualitative or quantitative data using CT would be relevant to organizing optimal patient management.

METHODS

Four French hospitals were used for model training and internal validation. External validation was conducted in two independent hospitals. We used easy-to-obtain clinical (age, gender, smoking, symptoms' onset, cardiovascular comorbidities, diabetes, chronic respiratory diseases, immunosuppression) and biological parameters (lymphocytes, CRP) with qualitative or quantitative data (including radiomics) from the initial CT in mild COVID-19 patients.

RESULTS

Qualitative CT scan with clinical and biological parameters can predict which patients with an initial mild presentation would develop a moderate to critical form of COVID-19, with a c-index of 0.70 (95% CI 0.63; 0.77). CT scan quantification improved the performance of the prediction up to 0.73 (95% CI 0.67; 0.79) and radiomics up to 0.77 (95% CI 0.71; 0.83). Results were similar in both validation cohorts, considering CT scans with or without injection.

CONCLUSION

Adding CT scan quantification or radiomics to simple clinical and biological parameters can better predict which patients with an initial mild COVID-19 would worsen than qualitative analyses alone. This tool could help to the fair use of healthcare resources and to screen patients for potential new drugs to prevent a pejorative evolution of COVID-19.

CLINICAL TRIAL REGISTRATION

NCT04481620.

CLINICAL RELEVANCE STATEMENT

CT scan quantification or radiomics analysis is superior to qualitative analysis, when used with simple clinical and biological parameters, to determine which patients with an initial mild presentation of COVID-19 would worsen to a moderate to critical form.

KEY POINTS

• Qualitative CT scan analyses with simple clinical and biological parameters can predict which patients with an initial mild COVID-19 and respiratory symptoms would worsen with a c-index of 0.70. • Adding CT scan quantification improves the performance of the clinical prediction model to an AUC of 0.73. • Radiomics analyses slightly improve the performance of the model to a c-index of 0.77.

COVID-19: An evaluation of predictive scoring systems in South Africa

Background

| The Coronavirus Disease 2019 (COVID-19) pandemic, caused by SARS-CoV-2, has resulted in more than 700 million cases worldwide. Sepsis and pneumonia severity scores assist in risk assessment of critical outcomes in patients with COVID-19. This allows healthcare workers to triage patients, by using clinical parameters and limited special investigations, thus offering the most appropriate level of care.

Methods

| A retrospective cohort study of 605 adult patients hospitalised with moderate to severe COVID-19, at a tertiary state hospital in South Africa. Evaluating the utility of the CURB65, NEWS2 and ISARIC-4C Mortality Score, in predicting critical outcomes, using clinical characteristics on admission. Outcomes included in-hospital mortality, invasive mechanical ventilation, and intensive care unit admission (ICU). Performance of severity scores and risk factors was assessed by area under the receiver operator characteristics (AUROC) analysis and logistic regression.

Findings |

A total of 605 records were used, 129 (21 %) non-survivors, 101 (17 %) ICU admissions and 77 (13 %) requiring invasive ventilation. Greater odds of mortality was associated with moderate and severe risk groups of the CURB65, ISARIC-4C and NEWS2 score. Mortality AUROC curve analysis for the CURB65 score was 0·76 (95 % CI: 0·71-0·8), 0·77 (95 % CI: 0·73-0·81) for the ISARIC-4C and 0·77 (95 % CI: 0·73-0·82) for the NEWS2 score. The CURB65 score had a sensitivity of 86 % with 12·8 % mortality, ISARIC-4C score a sensitivity of 87·6 % with 8 % mortality and NEWS2 score a sensitivity of 92·2 % with 8·6 % mortality.

Interpretation |

In 605 hospitalised patients with moderate to severe COVID-19, predominantly infected by the ancestral strain, good performance of the NEWS2 and ISARIC-4C score in predicting in-hospital mortality was noted. The CURB65 score had a high mortality rate in its low-risk group suggesting unexplained risk factors, not accounted for in the score, thus limiting its utility in the South African setting.

Head-to-head comparison of six warning scores to predict mortality and clinical impairment in COVID-19 patients in emergency department

The aim was to evaluate the ability of six risk scores (4C, CURB65, SEIMC, mCHOSEN, QuickCSI, and NEWS2) to predict the outcome of patients with COVID-19 during the sixth pandemic wave in Spain. A retrospective observational study was performed to review the electronic medical records in patients ≥ 18 years of age who consulted consecutively in an emergency department with COVID-19 diagnosis throughout 2 months during the sixth pandemic wave. Clinical-epidemiological variables, comorbidities, and their respective outcomes, such as 30-day in-hospital mortality and clinical deterioration risk (a combined outcome considering: mechanical ventilation, intensive care unit admission, and/or 30-day in-hospital mortality), were calculated. The area under the curve for each risk score was calculated, and the resulting curves were compared by the Delong test, concluding with a decision curve analysis. A total of 626 patients (median age 79 years; 49.8% female) fulfilled the inclusion criteria. Two hundred and ninety-three patients (46.8%) had two or more comorbidities. Clinical deterioration risk criteria were present in 10.1% (63 cases), with a 30-day in-hospital mortality rate of 6.2% (39 cases). Comparison of the results showed that score 4C presented the best results for both outcome variables, with areas under the curve for mortality and clinical deterioration risk of 0.931 (95% CI 0.904-0.957) and 0.871 (95% CI 0.833-0.910) (both p < 0.001). The 4C Mortality Score proved to be the best score for predicting mortality or clinical deterioration risk among patients with COVID-19 attended in the emergency department in the following 30 days.

ANCOC Score to Predict Mortality in Different SARS-CoV-2 Variants and Vaccination Status

BACKGROUND

More than three years after the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic outbreak, hospitals worldwide are still affected by coronavirus disease 19 (COVID-19). The availability of a clinical score that can predict the risk of death from the disease at the time of diagnosis and that can be used even if population characteristics change and the virus mutates can be a useful tool for emergency physicians to make clinical decisions. During the first COVID-19 waves, we developed the ANCOC (age, blood urea nitrogen, C-reactive protein, oxygen saturation, comorbidities) score, a clinical score based on five main parameters (age, blood urea nitrogen, C-reactive protein, oxygen saturation, comorbidities) that accurately predicts the risk of death in patients infected with SARS-CoV-2. A score of less than -1 was associated with 0% mortality risk, whereas a score of 6 was associated with 100% risk of death, with an overall accuracy of 0.920. The aim of our study is to internally validate the ANCOC score and evaluate whether it can predict 60-day mortality risk independent of vaccination status and viral variant.

METHODS

We retrospectively enrolled 843 patients admitted to the emergency department (ED) of our hospital with a diagnosis of COVID-19. A total of 515 patients were admitted from July 2021 to September 2021, when the Delta variant was prevalent, and 328 in January 2022, when the Omicron 1 variant was predominant. All patients included in the study had a diagnosis of COVID-19 confirmed by polymerase chain reaction (PCR) on an oropharyngeal swab. Demographic data, comorbidities, vaccination data, and various laboratory, radiographic, and blood gas parameters were collected from all patients to determine differences between the two waves. ANCOC scores were then calculated for each patient, ranging from -6 to 6.

RESULTS

Patients infected with the Omicron variant were significantly older and had a greater number of comorbidities, of which hypertension and chronic obstructive pulmonary disease (COPD) were the most common. Immunization was less common in Delta patients than in Omicron patients (34% and 56%, respectively). To assess the accuracy of mortality prediction, we constructed a receiver operating characteristic (ROC) curve and found that the area under the ROC curve was greater than 0.8 for both variants. These results suggest that the ANCOC score is able to predict 60-day mortality regardless of viral variant and whether the patient is vaccinated or not.

CONCLUSION

In a population with increasingly high vaccination rates, several parameters may be considered prognostic for the risk of fatal outcomes. This study suggests that the ANCOC score can be very useful for the clinician in an emergency setting to quickly understand the patient's evolution and provide proper attention and the most appropriate treatments.

Who to escalate during a pandemic? A retrospective observational study about decision-making during the COVID-19 pandemic in the UK

BACKGROUND

Optimal decision-making regarding who to admit to critical care in pandemic situations remains unclear. We compared age, Clinical Frailty Score (CFS), 4C Mortality Score and hospital mortality in two separate COVID-19 surges based on the escalation decision made by the treating physician.

METHODS

A retrospective analysis of all referrals to critical care during the first COVID-19 surge (cohort 1, March/April 2020) and a late surge (cohort 2, October/November 2021) was undertaken. Patients with confirmed or high clinical suspicion of COVID-19 infection were included. A senior critical care physician assessed all patients regarding their suitability for potential intensive care unit admission. Demographics, CFS, 4C Mortality Score and hospital mortality were compared depending on the escalation decision made by the attending physician.

RESULTS

203 patients were included in the study, 139 in cohort 1 and 64 in cohort 2. There were no significant differences in age, CFS and 4C scores between the two cohorts. Patients deemed suitable for escalation by clinicians were significantly younger with significantly lower CFS and 4C scores compared with patients who were not deemed to benefit from escalation. This pattern was observed in both cohorts. Mortality in patients not deemed suitable for escalation was 61.8% in cohort 1 and 47.4% in cohort 2 (p<0.001).

CONCLUSIONS

Decisions who to escalate to critical care in settings with limited resources pose moral distress on clinicians. 4C score, age and CFS did not change significantly between the two surges but differed significantly between patients deemed suitable for escalation and those deemed unsuitable by clinicians. Risk prediction tools may be useful in a pandemic to supplement clinical decision-making, even though escalation thresholds require adjustments to reflect changes in risk profile and outcomes between different pandemic surges.

Prognostic models in COVID-19 infection that predict severity: a systematic review

Current evidence on COVID-19 prognostic models is inconsistent and clinical applicability remains controversial. We performed a systematic review to summarize and critically appraise the available studies that have developed, assessed and/or validated prognostic models of COVID-19 predicting health outcomes. We searched six bibliographic databases to identify published articles that investigated univariable and multivariable prognostic models predicting adverse outcomes in adult COVID-19 patients, including intensive care unit (ICU) admission, intubation, high-flow nasal therapy (HFNT), extracorporeal membrane oxygenation (ECMO) and mortality. We identified and assessed 314 eligible articles from more than 40 countries, with 152 of these studies presenting mortality, 66 progression to severe or critical illness, 35 mortality and ICU admission combined, 17 ICU admission only, while the remaining 44 studies reported prediction models for mechanical ventilation (MV) or a combination of multiple outcomes. The sample size of included studies varied from 11 to 7,704,171 participants, with a mean age ranging from 18 to 93 years. There were 353 prognostic models investigated, with area under the curve (AUC) ranging from 0.44 to 0.99. A great proportion of studies (61.5%, 193 out of 314) performed internal or external validation or replication. In 312 (99.4%) studies, prognostic models were reported to be at high risk of bias due to uncertainties and challenges surrounding methodological rigor, sampling, handling of missing data, failure to deal with overfitting and heterogeneous definitions of COVID-19 and severity outcomes. While several clinical prognostic models for COVID-19 have been described in the literature, they are limited in generalizability and/or applicability due to deficiencies in addressing fundamental statistical and methodological concerns. Future large, multi-centric and well-designed prognostic prospective studies are needed to clarify remaining uncertainties.

An artificial intelligence approach for predicting death or organ failure after hospitalization for COVID-19: development of a novel risk prediction tool and comparisons with ISARIC-4C, CURB-65, qSOFA, and MEWS scoring systems

BACKGROUND

We applied machine learning (ML) algorithms to generate a risk prediction tool [Collaboration for Risk Evaluation in COVID-19 (CORE-COVID-19)] for predicting the composite of 30-day endotracheal intubation, intravenous administration of vasopressors, or death after COVID-19 hospitalization and compared it with the existing risk scores.

METHODS

This is a retrospective study of adults hospitalized with COVID-19 from March 2020 to February 2021. Patients, each with 92 variables, and one composite outcome underwent feature selection process to identify the most predictive variables. Selected variables were modeled to build four ML algorithms (artificial neural network, support vector machine, gradient boosting machine, and Logistic regression) and an ensemble model to generate a CORE-COVID-19 model to predict the composite outcome and compared with existing risk prediction scores. The net benefit for clinical use of each model was assessed by decision curve analysis.

RESULTS

Of 1796 patients, 278 (15%) patients reached primary outcome. Six most predictive features were identified. Four ML algorithms achieved comparable discrimination (P > 0.827) with c-statistics ranged 0.849-0.856, calibration slopes 0.911-1.173, and Hosmer-Lemeshow P > 0.141 in validation dataset. These 6-variable fitted CORE-COVID-19 model revealed a c-statistic of 0.880, which was significantly (P < 0.04) higher than ISARIC-4C (0.751), CURB-65 (0.735), qSOFA (0.676), and MEWS (0.674) for outcome prediction. The net benefit of the CORE-COVID-19 model was greater than that of the existing risk scores.

CONCLUSION

The CORE-COVID-19 model accurately assigned 88% of patients who potentially progressed to 30-day composite events and revealed improved performance over existing risk scores, indicating its potential utility in clinical practice.

Unfavorable Outcome and Long-Term Sequelae in Cases with Severe COVID-19

Emerging evidence shows that individuals with COVID-19 who survive the acute phase of illness may experience lingering symptoms in the following months. There is no clear indication as to whether these symptoms persist for a short time before resolving or if they persist for a long time. In this review, we will describe the symptoms that persist over time and possible predictors in the acute phase that indicate long-term persistence. Based on the literature available to date, fatigue/weakness, dyspnea, arthromyalgia, depression, anxiety, memory loss, slowing down, difficulty concentrating and insomnia are the most commonly reported persistent long-term symptoms. The extent and persistence of these in long-term follow-up is not clear as there are still no quality studies available. The evidence available today indicates that female subjects and those with a more severe initial disease are more likely to suffer permanent sequelae one year after the acute phase. To understand these complications, and to experiment with interventions and treatments for those at greater risk, we must first understand the physio-pathological mechanisms that sustain them.

A Comparison of ICU Mortality Scoring Systems Applied to COVID-19

Background Over the past three years, COVID-19 has been a major source of mortality in intensive care units around the world. Many scoring systems have been developed to estimate mortality in critically ill patients. Our intent with this study was to compare the efficacy of these systems when applied to COVID-19. Methods The was a multicenter, retrospective cohort study of critically ill patients with COVID-19 admitted to 16 hospitals in Texas from February 2020 to March 2022. The Simplified Acute Physiology Score (SAPS) II, Acute Physiology and Chronic Health Evaluation (APACHE) II, Sequential Organ Failure Assessment (SOFA) score, and 4C Mortality scores were calculated on the initial day of ICU admission. Primary endpoints were all-cause mortality, ICU length of stay, and hospital length of stay. Results Initially, 62,881 patient encounters were assessed, and the cohort of 292 was selected based on the inclusion of the requisite values for each of the scoring systems. The median age was 56 +/- 14.93 years and 61% of patients were male. Mortality was defined as patients who expired or were discharged to hospice and was 78%. The different scoring systems were compared using logistic regression, receiver operating characteristic (ROC) curve, and area under the ROC curve (AUC) analysis to compare the accuracy of prediction of the mortality and length of stay. The multivariate analysis showed that SOFA, APACHE II, SAPS II, and 4C scores were all significant predictors of mortality. The SOFA score had the highest AUC, though the confidence intervals for all of the models overlap therefore one model could not be considered superior to any of the others. Linear regression was performed to evaluate the models' ability to predict ICU and hospital length of stay, and none of the tested systems were found to be significant predictors of length of stay. Conclusion The SOFA, APACHE II, ISARIC 4-C, and SAPS II scores all accurately predicted mortality in critically ill patients with COVID-19. The SOFA score trended to perform the best.

Severity of Illness Scores and Biomarkers for Prognosis of Patients with Coronavirus Disease 2019

The spectrum of disease severity and the insidiousness of clinical presentation make it difficult to recognize patients with coronavirus disease 2019 (COVID-19) at higher risk of worse outcomes or death when they are seen in the early phases of the disease. There are now well-established risk factors for worse outcomes in patients with COVID-19. These should be factored in when assessing the prognosis of these patients. However, a more precise prognostic assessment in an individual patient may warrant the use of predictive tools. In this manuscript, we conduct a literature review on the severity of illness scores and biomarkers for the prognosis of patients with COVID-19. Several COVID-19-specific scores have been developed since the onset of the pandemic. Some of them are promising and can be integrated into the assessment of these patients. We also found that the well-known pneumonia severity index (PSI) and CURB-65 (confusion, uremia, respiratory rate, BP, age ≥ 65 years) are good predictors of mortality in hospitalized patients with COVID-19. While neither the PSI nor the CURB-65 should be used for the triage of outpatient versus inpatient treatment, they can be integrated by a clinician into the assessment of disease severity and can be used in epidemiological studies to determine the severity of illness in patient populations. Biomarkers also provide valuable prognostic information and, importantly, may depict the main physiological derangements in severe disease. We, however, do not advocate the isolated use of severity of illness scores or biomarkers for decision-making in an individual patient. Instead, we suggest the use of these tools on a case-by-case basis with the goal of enhancing clinician judgment.

Utility of the 4C ISARIC mortality score in hospitalized COVID-19 patients at a large tertiary Saudi Arabian center

Background

The International Severe Acute Respiratory and Emerging Infections Consortium (ISARIC) 4C mortality score has been used before as a valuable tool for predicting mortality in COVID-19 patients. We aimed to address the utility of the 4C score in a well-defined Saudi population with COVID-19 admitted to a large tertiary referral hospital in Saudi Arabia.

Methods

A retrospective study was conducted that included all adults COVID‑19 patients admitted to the Armed Forces Hospital Southern Region (AFHSR), between January 2021 and September 2022. The receiver operating characteristic (ROC) curve depicted the diagnostic performance of the 4C Score for mortality prediction.

Results

A total of 1,853 patients were enrolled. The ROC curve of the 4C score had an area under the curve of 0.73 (95% CI: 0.702-0.758), p<0.001. The sensitivity and specificity with scores >8 were 80% and 58%, respectively, the positive and negative predictive values were 28% and 93%, respectively. Three hundred and sixteen (17.1%), 638 (34.4%), 814 (43.9%), and 85 (4.6%) patients had low, intermediate, high, and very high values, respectively. There were significant differences between survivors and non-survivors with regard to all variables used in the calculation of the 4C score. Multivariable logistic regression analysis revealed that all components of the 4C score, except gender and O2 saturation, were independent significant predictors of mortality.

Conclusions

Our data support previous international and Saudi studies that the 4C mortality score is a reliable tool with good sensitivity and specificity in the mortality prediction of COVID-19 patients. All components of the 4C score, except gender and O2 saturation, were independent significant predictors of mortality. Within the 4C score, odds ratios increased proportionately with an increase in the score value. Future multi-center prospective studies are warranted.

Assess and validate predictive performance of models for in-hospital mortality in COVID-19 patients: A retrospective cohort study in the Netherlands comparing the value of registry data with high-granular electronic health records

PURPOSE

To assess, validate and compare the predictive performance of models for in-hospital mortality of COVID-19 patients admitted to the intensive care unit (ICU) over two different waves of infections. Our models were built with high-granular Electronic Health Records (EHR) data versus less-granular registry data.

METHODS

Observational study of all COVID-19 patients admitted to 19 Dutch ICUs participating in both the national quality registry National Intensive Care Evaluation (NICE) and the EHR-based Dutch Data Warehouse (hereafter EHR). Multiple models were developed on data from the first 24 h of ICU admissions from February to June 2020 (first COVID-19 wave) and validated on prospective patients admitted to the same ICUs between July and December 2020 (second COVID-19 wave). We assessed model discrimination, calibration, and the degree of relatedness between development and validation population. Coefficients were used to identify relevant risk factors.

RESULTS

A total of 1533 patients from the EHR and 1563 from the registry were included. With high granular EHR data, the average AUROC was 0.69 (standard deviation of 0.05) for the internal validation, and the AUROC was 0.75 for the temporal validation. The registry model achieved an average AUROC of 0.76 (standard deviation of 0.05) in the internal validation and 0.77 in the temporal validation. In the EHR data, age, and respiratory-system related variables were the most important risk factors identified. In the NICE registry data, age and chronic respiratory insufficiency were the most important risk factors.

CONCLUSION

In our study, prognostic models built on less-granular but readily-available registry data had similar performance to models built on high-granular EHR data and showed similar transportability to a prospective COVID-19 population. Future research is needed to verify whether this finding can be confirmed for upcoming waves.

Development and validation of a dynamic 48-hour in-hospital mortality risk stratification for COVID-19 in a UK teaching hospital: a retrospective cohort study

OBJECTIVES

To develop a disease stratification model for COVID-19 that updates according to changes in a patient's condition while in hospital to facilitate patient management and resource allocation.

DESIGN

In this retrospective cohort study, we adopted a landmarking approach to dynamic prediction of all-cause in-hospital mortality over the next 48 hours. We accounted for informative predictor missingness and selected predictors using penalised regression.

SETTING

All data used in this study were obtained from a single UK teaching hospital.

PARTICIPANTS

We developed the model using 473 consecutive patients with COVID-19 presenting to a UK hospital between 1 March 2020 and 12 September 2020; and temporally validated using data on 1119 patients presenting between 13 September 2020 and 17 March 2021.

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcome is all-cause in-hospital mortality within 48 hours of the prediction time. We accounted for the competing risks of discharge from hospital alive and transfer to a tertiary intensive care unit for extracorporeal membrane oxygenation.

RESULTS

Our final model includes age, Clinical Frailty Scale score, heart rate, respiratory rate, oxygen saturation/fractional inspired oxygen ratio, white cell count, presence of acidosis (pH <7.35) and interleukin-6. Internal validation achieved an area under the receiver operating characteristic (AUROC) of 0.90 (95% CI 0.87 to 0.93) and temporal validation gave an AUROC of 0.86 (95% CI 0.83 to 0.88).

CONCLUSIONS

Our model incorporates both static risk factors (eg, age) and evolving clinical and laboratory data, to provide a dynamic risk prediction model that adapts to both sudden and gradual changes in an individual patient's clinical condition. On successful external validation, the model has the potential to be a powerful clinical risk assessment tool.

TRIAL REGISTRATION

The study is registered as 'researchregistry5464' on the Research Registry (www.researchregistry.com).

Comparison of prognostic scores for inpatients with COVID-19: a retrospective monocentric cohort study

Background

The SARS-CoV-2 pandemic led to a steep increase in hospital and intensive care unit (ICU) admissions for acute respiratory failure worldwide. Early identification of patients at risk of clinical deterioration is crucial in terms of appropriate care delivery and resource allocation. We aimed to evaluate and compare the prognostic performance of Sequential Organ Failure Assessment (SOFA), Quick Sequential Organ Failure Assessment (qSOFA), Confusion, Uraemia, Respiratory Rate, Blood Pressure and Age ≥65 (CURB-65), Respiratory Rate and Oxygenation (ROX) index and Coronavirus Clinical Characterisation Consortium (4C) score to predict death and ICU admission among patients admitted to the hospital for acute COVID-19 infection.

Methods and analysis

Consecutive adult patients admitted to the Geneva University Hospitals during two successive COVID-19 flares in spring and autumn 2020 were included. Discriminative performance of these prediction rules, obtained during the first 24 hours of hospital admission, were computed to predict death or ICU admission. We further exluded patients with therapeutic limitations and reported areas under the curve (AUCs) for 30-day mortality and ICU admission in sensitivity analyses.

Results

A total of 2122 patients were included. 216 patients (10.2%) required ICU admission and 303 (14.3%) died within 30 days post admission. 4C score had the best discriminatory performance to predict 30-day mortality (AUC 0.82, 95% CI 0.80 to 0.85), compared with SOFA (AUC 0.75, 95% CI 0.72 to 0.78), qSOFA (AUC 0.59, 95% CI 0.56 to 0.62), CURB-65 (AUC 0.75, 95% CI 0.72 to 0.78) and ROX index (AUC 0.68, 95% CI 0.65 to 0.72). ROX index had the greatest discriminatory performance (AUC 0.79, 95% CI 0.76 to 0.83) to predict ICU admission compared with 4C score (AUC 0.62, 95% CI 0.59 to 0.66), CURB-65 (AUC 0.60, 95% CI 0.56 to 0.64), SOFA (AUC 0.74, 95% CI 0.71 to 0.77) and qSOFA (AUC 0.59, 95% CI 0.55 to 0.62).

Conclusion

Scores including age and/or comorbidities (4C and CURB-65) have the best discriminatory performance to predict mortality among inpatients with COVID-19, while scores including quantitative assessment of hypoxaemia (SOFA and ROX index) perform best to predict ICU admission. Exclusion of patients with therapeutic limitations improved the discriminatory performance of prognostic scores relying on age and/or comorbidities to predict ICU admission.

ISARIC-COVID-19 dataset: A Prospective, Standardized, Global Dataset of Patients Hospitalized with COVID-19

The International Severe Acute Respiratory and Emerging Infection Consortium (ISARIC) COVID-19 dataset is one of the largest international databases of prospectively collected clinical data on people hospitalized with COVID-19. This dataset was compiled during the COVID-19 pandemic by a network of hospitals that collect data using the ISARIC-World Health Organization Clinical Characterization Protocol and data tools. The database includes data from more than 705,000 patients, collected in more than 60 countries and 1,500 centres worldwide. Patient data are available from acute hospital admissions with COVID-19 and outpatient follow-ups. The data include signs and symptoms, pre-existing comorbidities, vital signs, chronic and acute treatments, complications, dates of hospitalization and discharge, mortality, viral strains, vaccination status, and other data. Here, we present the dataset characteristics, explain its architecture and how to gain access, and provide tools to facilitate its use.

Course of disease and risk factors for hospitalization in outpatients with a SARS-CoV-2 infection

We analyzed symptoms and comorbidities as predictors of hospitalization in 710 outpatients in North-East Germany with PCR-confirmed SARS-CoV-2 infection. During the first 3 days of infection, commonly reported symptoms were fatigue (71.8%), arthralgia/myalgia (56.8%), headache (55.1%), and dry cough (51.8%). Loss of smell (anosmia), loss of taste (ageusia), dyspnea, and productive cough were reported with an onset of 4 days. Anosmia or ageusia were reported by only 18% of the participants at day one, but up to 49% between days 7 and 9. Not all participants who reported ageusia also reported anosmia. Individuals suffering from ageusia without anosmia were at highest risk of hospitalization (OR 6.8, 95% CI 2.5–18.1). They also experienced more commonly dyspnea and nausea (OR of 3.0, 2.9, respectively) suggesting pathophysiological connections between these symptoms. Other symptoms significantly associated with increased risk of hospitalization were dyspnea, vomiting, and fever. Among basic parameters and comorbidities, age > 60 years, COPD, prior stroke, diabetes, kidney and cardiac diseases were also associated with increased risk of hospitalization. In conclusion, due to the delayed onset, ageusia and anosmia may be of limited use in differential diagnosis of SARS-CoV-2. However, differentiation between ageusia and anosmia may be useful for evaluating risk for hospitalization.