External validation of six COVID-19 prognostic models for predicting mortality risk in older populations in a hospital, primary care, and nursing home setting

OBJECTIVES

To systematically evaluate the performance of COVID-19 prognostic models and scores for mortality risk in older populations across three health-care settings: hospitals, primary care, and nursing homes.

STUDY DESIGN AND SETTING

This retrospective external validation study included 14,092 older individuals of ≥70 years of age with a clinical or polymerase chain reaction-confirmed COVID-19 diagnosis from March 2020 to December 2020. The six validation cohorts include three hospital-based (CliniCo, COVID-OLD, COVID-PREDICT), two primary care-based (Julius General Practitioners Network/Academisch network huisartsgeneeskunde/Network of Academic general Practitioners, PHARMO), and one nursing home cohort (YSIS) in the Netherlands. Based on a living systematic review of COVID-19 prediction models using Prediction model Risk Of Bias ASsessment Tool for quality and risk of bias assessment and considering predictor availability in validation cohorts, we selected six prognostic models predicting mortality risk in adults with COVID-19 infection (GAL-COVID-19 mortality, 4C Mortality Score, National Early Warning Score 2-extended model, Xie model, Wang clinical model, and CURB65 score). All six prognostic models were validated in the hospital cohorts and the GAL-COVID-19 mortality model was validated in all three healthcare settings. The primary outcome was in-hospital mortality for hospitals and 28-day mortality for primary care and nursing home settings. Model performance was evaluated in each validation cohort separately in terms of discrimination, calibration, and decision curves. An intercept update was performed in models indicating miscalibration followed by predictive performance re-evaluation.

MAIN OUTCOME MEASURE

In-hospital mortality for hospitals and 28-day mortality for primary care and nursing home setting.

RESULTS

All six prognostic models performed poorly and showed miscalibration in the older population cohorts. In the hospital settings, model performance ranged from calibration-in-the-large -1.45 to 7.46, calibration slopes 0.24-0.81, and C-statistic 0.55-0.71 with 4C Mortality Score performing as the most discriminative and well-calibrated model. Performance across health-care settings was similar for the GAL-COVID-19 model, with a calibration-in-the-large in the range of -2.35 to -0.15 indicating overestimation, calibration slopes of 0.24-0.81 indicating signs of overfitting, and C-statistic of 0.55-0.71.

CONCLUSION

Our results show that most prognostic models for predicting mortality risk performed poorly in the older population with COVID-19, in each health-care setting: hospital, primary care, and nursing home settings. Insights into factors influencing predictive model performance in the older population are needed for pandemic preparedness and reliable prognostication of health-related outcomes in this demographic.

Delirium in older patients with COVID-19: Prevalence, risk factors and clinical outcomes across the first three waves of the pandemic

OBJECTIVES

Delirium is a serious condition, which poses treatment challenges during hospitalisation for COVID-19. Improvements in testing, vaccination and treatment might have changed patient characteristics and outcomes through the pandemic. We evaluated whether the prevalence and risk factors for delirium, and the association of delirium with in-hospital mortality changed through the pandemic.

METHODS

This study was part of the COVID-OLD study in 19 Dutch hospitals including patients ≥70 years in the first (spring 2020), second (autumn 2020) and third wave (autumn 2021). Multivariable logistic regression models were used to study risk factors for delirium, and in-hospital mortality. Differences in effect sizes between waves were studied by including interaction terms between wave and risk factor in logistic regression models.

RESULTS

1540, 884 and 370 patients were included in the first, second and third wave, respectively. Prevalence of delirium in the third wave (12.7%) was significantly lower compared to the first (22.5%) and second wave (23.5%). In multivariable-adjusted analyses, pre-existing memory problems was a consistent risk factor for delirium across waves. Previous delirium was a risk factor for delirium in the first wave (OR 4.02), but not in the second (OR 1.61) and third wave (OR 2.59, p-value interaction-term 0.028). In multivariable-adjusted analyses, delirium was not associated with in-hospital mortality in all waves.

CONCLUSION

Delirium prevalence declined in the third wave, which might be the result of vaccination and improved treatment strategies. Risk factors for delirium remained consistent across waves, although some attenuation was seen in the second wave.

A study protocol of external validation of eight COVID-19 prognostic models for predicting mortality risk in older populations in a hospital, primary care, and nursing home setting

BACKGROUND

The COVID-19 pandemic has a large impact worldwide and is known to particularly affect the older population. This paper outlines the protocol for external validation of prognostic models predicting mortality risk after presentation with COVID-19 in the older population. These prognostic models were originally developed in an adult population and will be validated in an older population (≥ 70 years of age) in three healthcare settings: the hospital setting, the primary care setting, and the nursing home setting.

METHODS

Based on a living systematic review of COVID-19 prediction models, we identified eight prognostic models predicting the risk of mortality in adults with a COVID-19 infection (five COVID-19 specific models: GAL-COVID-19 mortality, 4C Mortality Score, NEWS2 + model, Xie model, and Wang clinical model and three pre-existing prognostic scores: APACHE-II, CURB65, SOFA). These eight models will be validated in six different cohorts of the Dutch older population (three hospital cohorts, two primary care cohorts, and a nursing home cohort). All prognostic models will be validated in a hospital setting while the GAL-COVID-19 mortality model will be validated in hospital, primary care, and nursing home settings. The study will include individuals ≥ 70 years of age with a highly suspected or PCR-confirmed COVID-19 infection from March 2020 to December 2020 (and up to December 2021 in a sensitivity analysis). The predictive performance will be evaluated in terms of discrimination, calibration, and decision curves for each of the prognostic models in each cohort individually. For prognostic models with indications of miscalibration, an intercept update will be performed after which predictive performance will be re-evaluated.

DISCUSSION

Insight into the performance of existing prognostic models in one of the most vulnerable populations clarifies the extent to which tailoring of COVID-19 prognostic models is needed when models are applied to the older population. Such insight will be important for possible future waves of the COVID-19 pandemic or future pandemics.

Imatinib in patients with severe COVID-19: a randomised, double-blind, placebo-controlled, clinical trial

BACKGROUND

The major complication of COVID-19 is hypoxaemic respiratory failure from capillary leak and alveolar oedema. Experimental and early clinical data suggest that the tyrosine-kinase inhibitor imatinib reverses pulmonary capillary leak.

METHODS

This randomised, double-blind, placebo-controlled, clinical trial was done at 13 academic and non-academic teaching hospitals in the Netherlands. Hospitalised patients (aged ≥18 years) with COVID-19, as confirmed by an RT-PCR test for SARS-CoV-2, requiring supplemental oxygen to maintain a peripheral oxygen saturation of greater than 94% were eligible. Patients were excluded if they had severe pre-existing pulmonary disease, had pre-existing heart failure, had undergone active treatment of a haematological or non-haematological malignancy in the previous 12 months, had cytopenia, or were receiving concomitant treatment with medication known to strongly interact with imatinib. Patients were randomly assigned (1:1) to receive either oral imatinib, given as a loading dose of 800 mg on day 0 followed by 400 mg daily on days 1-9, or placebo. Randomisation was done with a computer-based clinical data management platform with variable block sizes (containing two, four, or six patients), stratified by study site. The primary outcome was time to discontinuation of mechanical ventilation and supplemental oxygen for more than 48 consecutive hours, while being alive during a 28-day period. Secondary outcomes included safety, mortality at 28 days, and the need for invasive mechanical ventilation. All efficacy and safety analyses were done in all randomised patients who had received at least one dose of study medication (modified intention-to-treat population). This study is registered with the EU Clinical Trials Register (EudraCT 2020-001236-10).

FINDINGS

Between March 31, 2020, and Jan 4, 2021, 805 patients were screened, of whom 400 were eligible and randomly assigned to the imatinib group (n=204) or the placebo group (n=196). A total of 385 (96%) patients (median age 64 years [IQR 56-73]) received at least one dose of study medication and were included in the modified intention-to-treat population. Time to discontinuation of ventilation and supplemental oxygen for more than 48 h was not significantly different between the two groups (unadjusted hazard ratio [HR] 0·95 [95% CI 0·76-1·20]). At day 28, 15 (8%) of 197 patients had died in the imatinib group compared with 27 (14%) of 188 patients in the placebo group (unadjusted HR 0·51 [0·27-0·95]). After adjusting for baseline imbalances between the two groups (sex, obesity, diabetes, and cardiovascular disease) the HR for mortality was 0·52 (95% CI 0·26-1·05). The HR for mechanical ventilation in the imatinib group compared with the placebo group was 1·07 (0·63-1·80; p=0·81). The median duration of invasive mechanical ventilation was 7 days (IQR 3-13) in the imatinib group compared with 12 days (6-20) in the placebo group (p=0·0080). 91 (46%) of 197 patients in the imatinib group and 82 (44%) of 188 patients in the placebo group had at least one grade 3 or higher adverse event. The safety evaluation revealed no imatinib-associated adverse events.

INTERPRETATION

The study failed to meet its primary outcome, as imatinib did not reduce the time to discontinuation of ventilation and supplemental oxygen for more than 48 consecutive hours in patients with COVID-19 requiring supplemental oxygen. The observed effects on survival (although attenuated after adjustment for baseline imbalances) and duration of mechanical ventilation suggest that imatinib might confer clinical benefit in hospitalised patients with COVID-19, but further studies are required to validate these findings.

FUNDING

Amsterdam Medical Center Foundation, Nederlandse Organisatie voor Wetenschappelijk Onderzoek/ZonMW, and the European Union Innovative Medicines Initiative 2.