External validation of the QCovid risk prediction algorithm for risk of COVID-19 hospitalisation and mortality in adults: national validation cohort study in Scotland

Development and validation of a risk prediction model for hospital admission in COVID-19 patients presenting to primary care

BACKGROUND

There is a paucity of prognostic models for COVID-19 that are usable for in-office patient assessment in general practice (GP).

OBJECTIVES

To develop and validate a risk prediction model for hospital admission with readily available predictors.

METHODS

A retrospective cohort study linking GP records from 8 COVID-19 centres and 55 general practices in the Netherlands to hospital admission records. The development cohort spanned March to June 2020, the validation cohort March to June 2021. The primary outcome was hospital admission within 14 days. We used geographic leave-region-out cross-validation in the development cohort and temporal validation in the validation cohort.

RESULTS

In the development cohort, 4,806 adult patients with COVID-19 consulted their GP (median age 56, 56% female); in the validation cohort 830 patients did (median age 56, 52% female). In the development and validation cohort respectively, 292 (6.1%) and 126 (15.2%) were admitted to the hospital within 14 days, respectively. A logistic regression model based on sex, smoking, symptoms, vital signs and comorbidities predicted hospital admission with a c-index of 0.84 (95% CI 0.83 to 0.86) at geographic cross-validation and 0.79 (95% CI 0.74 to 0.83) at temporal validation, and was reasonably well calibrated (intercept -0.08, 95% CI -0.98 to 0.52, slope 0.89, 95% CI 0.71 to 1.07 at geographic cross-validation and intercept 0.02, 95% CI -0.21 to 0.24, slope 0.82, 95% CI 0.64 to 1.00 at temporal validation).

CONCLUSION

We derived a risk model using readily available variables at GP assessment to predict hospital admission for COVID-19. It performed accurately across regions and waves. Further validation on cohorts with acquired immunity and newer SARS-CoV-2 variants is recommended.

Risk of COVID-19 death in adults who received booster COVID-19 vaccinations in England

The emergence of the COVID-19 vaccination has been critical in changing the course of the COVID-19 pandemic. To ensure protection remains high in vulnerable groups booster vaccinations in the UK have been targeted based on age and clinical vulnerabilities. We undertook a national retrospective cohort study using data from the 2021 Census linked to electronic health records. We fitted cause-specific Cox models to examine the association between health conditions and the risk of COVID-19 death and all-other-cause death for adults aged 50-100-years in England vaccinated with a booster in autumn 2022. Here we show, having learning disabilities or Down Syndrome (hazard ratio=5.07;95% confidence interval=3.69-6.98), pulmonary hypertension or fibrosis (2.88;2.43-3.40), motor neuron disease, multiple sclerosis, myasthenia or Huntington's disease (2.94, 1.82-4.74), cancer of blood and bone marrow (3.11;2.72-3.56), Parkinson's disease (2.74;2.34-3.20), lung or oral cancer (2.57;2.04 to 3.24), dementia (2.64;2.46 to 2.83) or liver cirrhosis (2.65;1.95 to 3.59) was associated with an increased risk of COVID-19 death. Individuals with cancer of the blood or bone marrow, chronic kidney disease, cystic fibrosis, pulmonary hypotension or fibrosis, or rheumatoid arthritis or systemic lupus erythematosus had a significantly higher risk of COVID-19 death relative to other causes of death compared with individuals who did not have diagnoses. Policy makers should continue to priorities vulnerable groups for subsequent COVID-19 booster doses to minimise the risk of COVID-19 death.

External validation of the QCovid 2 and 3 risk prediction algorithms for risk of COVID-19 hospitalisation and mortality in adults: a national cohort study in Scotland

OBJECTIVE

The QCovid 2 and 3 algorithms are risk prediction tools developed during the second wave of the COVID-19 pandemic that can be used to predict the risk of COVID-19 hospitalisation and mortality, taking vaccination status into account. In this study, we assess their performance in Scotland.

METHODS

We used the Early Pandemic Evaluation and Enhanced Surveillance of COVID-19 national data platform consisting of individual-level data for the population of Scotland (5.4 million residents). Primary care data were linked to reverse-transcription PCR virology testing, hospitalisation and mortality data. We assessed the discrimination and calibration of the QCovid 2 and 3 algorithms in predicting COVID-19 hospitalisations and deaths between 8 December 2020 and 15 June 2021.

RESULTS

Our validation dataset comprised 465 058 individuals, aged 19-100. We found the following performance metrics (95% CIs) for QCovid 2 and 3: Harrell's C 0.84 (0.82 to 0.86) for hospitalisation, and 0.92 (0.90 to 0.94) for death, observed-expected ratio of 0.24 for hospitalisation and 0.26 for death (ie, both the number of hospitalisations and the number of deaths were overestimated), and a Brier score of 0.0009 (0.00084 to 0.00096) for hospitalisation and 0.00036 (0.00032 to 0.0004) for death.

CONCLUSIONS

We found good discrimination of the QCovid 2 and 3 algorithms in Scotland, although performance was worse in higher age groups. Both the number of hospitalisations and the number of deaths were overestimated.

Vaccine effectiveness for prevention of covid-19 related hospital admission during pregnancy in England during the alpha and delta variant dominant periods of the SARS-CoV-2 pandemic: population based cohort study

Objective

To estimate vaccine effectiveness for preventing covid-19 related hospital admission in individuals first infected with the SARS-CoV-2 virus during pregnancy compared with those of reproductive age who were not pregnant when first infected with the virus.

Design

Population based cohort study.

Setting

Office for National Statistics Public Health Data Asset linked dataset, providing national linked census and administrative data in England, 8 December 2020 to 31 August 2021.

Participants

815 477 females aged 18-45 years (mean age 30.4 years) who had documented evidence of a first SARS-CoV-2 infection in the NHS Test and Trace or Hospital Episode Statistics data.

Main outcome measures

Hospital admission where covid-19 was recorded as the primary diagnosis. Cox proportional hazards models, adjusted for calendar time of infection, sociodemographic factors, and pre-existing health conditions related to uptake of the covid-19 vaccine and risk of severe covid-19 outcomes, were used to estimate vaccine effectiveness as the complement of the hazard ratio for hospital admission for covid-19.

Results

Compared with pregnant individuals who were not vaccinated, the adjusted rate of hospital admission for covid-19 was 77% (95% confidence interval 70% to 82%) lower for pregnant individuals who had received one dose and 83% (76% to 89%) lower for those who had received two doses of vaccine. These estimates were similar to those found in the non-pregnant group: 79% (77% to 81%) for one dose and 83% (82% to 85%) for two doses of vaccine. Among those who were vaccinated >90 days before infection, having two doses of vaccine was associated with a greater reduction in risk than one dose.

Conclusions

Covid-19 vaccination was associated with reduced rates of hospital admission in pregnant individuals infected with the SARS-CoV-2 virus, and the reduction in risk was similar to that in non-pregnant individuals. Waning of vaccine effectiveness occurred more quickly after one than after two doses of vaccine.

Risk prediction of covid-19 related death or hospital admission in adults testing positive for SARS-CoV-2 infection during the omicron wave in England (QCOVID4): cohort study

OBJECTIVES

To derive and validate risk prediction algorithms (QCOVID4) to estimate the risk of covid-19 related death and hospital admission in people with a positive SARS-CoV-2 test result during the period when the omicron variant of the virus was predominant in England, and to evaluate performance compared with a high risk cohort from NHS Digital.

DESIGN

Cohort study.

SETTING

QResearch database linked to English national data on covid-19 vaccinations, SARS-CoV-2 test results, hospital admissions, and cancer and mortality data, 11 December 2021 to 31 March 2022, with follow-up to 30 June 2022.

PARTICIPANTS

1.3 million adults in the derivation cohort and 0.15 million adults in the validation cohort, aged 18-100 years, with a positive test result for SARS-CoV-2 infection.

MAIN OUTCOME MEASURES

Primary outcome was covid-19 related death and secondary outcome was hospital admission for covid-19. Risk equations with predictor variables were derived from models fitted in the derivation cohort. Performance was evaluated in a separate validation cohort.

RESULTS

Of 1 297 922 people with a positive test result for SARS-CoV-2 infection in the derivation cohort, 18 756 (1.5%) had a covid-19 related hospital admission and 3878 (0.3%) had a covid-19 related death during follow-up. The final QCOVID4 models included age, deprivation score and a range of health and sociodemographic factors, number of covid-19 vaccinations, and previous SARS-CoV-2 infection. The risk of death related to covid-19 was lower among those who had received a covid-19 vaccine, with evidence of a dose-response relation (42% risk reduction associated with one vaccine dose and 92% reduction with four or more doses in men). Previous SARS-CoV-2 infection was associated with a reduction in the risk of covid-19 related death (49% reduction in men). The QCOVID4 algorithm for covid-19 explained 76.0% (95% confidence interval 73.9% to 78.2%) of the variation in time to covid-19 related death in men with a D statistic of 3.65 (3.43 to 3.86) and Harrell's C statistic of 0.970 (0.962 to 0.979). Results were similar for women. QCOVID4 was well calibrated. QCOVID4 was substantially more efficient than the NHS Digital algorithm for correctly identifying patients at high risk of covid-19 related death. Of the 461 covid-19 related deaths in the validation cohort, 333 (72.2%) were in the QCOVID4 high risk group and 95 (20.6%) in the NHS Digital high risk group.

CONCLUSION

The QCOVID4 risk algorithm, modelled from data during the period when the omicron variant of the SARS-CoV-2 virus was predominant in England, now includes vaccination dose and previous SARS-CoV-2 infection, and predicted covid-19 related death among people with a positive test result. QCOVID4 more accurately identified individuals at the highest levels of absolute risk for targeted interventions than the approach adopted by NHS Digital. QCOVID4 performed well and could be used for targeting treatments for covid-19 disease.

An external validation of the QCOVID3 risk prediction algorithm for risk of hospitalisation and death from COVID-19: An observational, prospective cohort study of 1.66m vaccinated adults in Wales, UK

INTRODUCTION

At the start of the COVID-19 pandemic there was an urgent need to identify individuals at highest risk of severe outcomes, such as hospitalisation and death following infection. The QCOVID risk prediction algorithms emerged as key tools in facilitating this which were further developed during the second wave of the COVID-19 pandemic to identify groups of people at highest risk of severe COVID-19 related outcomes following one or two doses of vaccine.

OBJECTIVES

To externally validate the QCOVID3 algorithm based on primary and secondary care records for Wales, UK.

METHODS

We conducted an observational, prospective cohort based on electronic health care records for 1.66m vaccinated adults living in Wales on 8th December 2020, with follow-up until 15th June 2021. Follow-up started from day 14 post vaccination to allow the full effect of the vaccine.

RESULTS

The scores produced by the QCOVID3 risk algorithm showed high levels of discrimination for both COVID-19 related deaths and hospital admissions and good calibration (Harrell C statistic: ≥ 0.828).

CONCLUSION

This validation of the updated QCOVID3 risk algorithms in the adult vaccinated Welsh population has shown that the algorithms are valid for use in the Welsh population, and applicable on a population independent of the original study, which has not been previously reported. This study provides further evidence that the QCOVID algorithms can help inform public health risk management on the ongoing surveillance and intervention to manage COVID-19 related risks.

Accelerated waning of the humoral response to COVID-19 vaccines in obesity

Obesity is associated with an increased risk of severe Coronavirus Disease 2019 (COVID-19) infection and mortality. COVID-19 vaccines reduce the risk of serious COVID-19 outcomes; however, their effectiveness in people with obesity is incompletely understood. We studied the relationship among body mass index (BMI), hospitalization and mortality due to COVID-19 among 3.6 million people in Scotland using the Early Pandemic Evaluation and Enhanced Surveillance of COVID-19 (EAVE II) surveillance platform. We found that vaccinated individuals with severe obesity (BMI > 40 kg/m2) were 76% more likely to experience hospitalization or death from COVID-19 (adjusted rate ratio of 1.76 (95% confidence interval (CI), 1.60-1.94). We also conducted a prospective longitudinal study of a cohort of 28 individuals with severe obesity compared to 41 control individuals with normal BMI (BMI 18.5-24.9 kg/m2). We found that 55% of individuals with severe obesity had unquantifiable titers of neutralizing antibody against authentic severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus compared to 12% of individuals with normal BMI (P = 0.0003) 6 months after their second vaccine dose. Furthermore, we observed that, for individuals with severe obesity, at any given anti-spike and anti-receptor-binding domain (RBD) antibody level, neutralizing capacity was lower than that of individuals with a normal BMI. Neutralizing capacity was restored by a third dose of vaccine but again declined more rapidly in people with severe obesity. We demonstrate that waning of COVID-19 vaccine-induced humoral immunity is accelerated in individuals with severe obesity. As obesity is associated with increased hospitalization and mortality from breakthrough infections, our findings have implications for vaccine prioritization policies.

Ethnic differences in COVID-19 mortality in the second and third waves of the pandemic in England during the vaccine rollout: a retrospective, population-based cohort study

BACKGROUND

Ethnic minority groups in England have been disproportionately affected by the COVID-19 pandemic and have lower vaccination rates than the White British population. We examined whether ethnic differences in COVID-19 mortality in England have continued since the vaccine rollout and to what extent differences in vaccination rates contributed to excess COVID-19 mortality after accounting for other risk factors.

METHODS

We conducted a retrospective, population-based cohort study of 28.8 million adults aged 30-100 years in England. Self-reported ethnicity was obtained from the 2011 Census. The outcome was death involving COVID-19 during the second (8 December 2020 to 12 June 2021) and third wave (13 June 2021 to 1 December 2021). We calculated hazard ratios (HRs) for death involving COVID-19, sequentially adjusting for age, residence type, geographical factors, sociodemographic characteristics, pre-pandemic health, and vaccination status.

RESULTS

Age-adjusted HRs of death involving COVID-19 were elevated for most ethnic minority groups during both waves, particularly for groups with lowest vaccination rates (Bangladeshi, Pakistani, Black African, and Black Caribbean). HRs were attenuated after adjusting for geographical factors, sociodemographic characteristics, and pre-pandemic health. Further adjusting for vaccination status substantially reduced residual HRs for Black African, Black Caribbean, and Pakistani groups in the third wave. Fully adjusted HRs only remained elevated for the Bangladeshi group (men: 2.19 [95% CI 1.72-2.78]; women: 2.12 [1.58-2.86]) and Pakistani men (1.24 [1.06-1.46]).

CONCLUSIONS

Lower COVID-19 vaccination uptake in several ethnic minority groups may drive some of the differences in COVID-19 mortality compared to White British. Public health strategies to increase vaccination uptake in ethnic minority groups would help reduce inequalities in COVID-19 mortality, which have remained substantial since the start of the vaccination campaign.

COVID-19 vaccine effectiveness against symptomatic SARS-CoV-2 infection and severe COVID-19 outcomes from Delta AY.4.2: Cohort and test-negative study of 5.4 million individuals in Scotland

Background

In July 2021, a new variant of SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) in the Delta lineage was detected in the United Kingdom (UK), named AY.4.2 or "Delta plus". By October 2021, the AY.4.2 variant accounted for approximately 10-11% of cases in the UK. AY.4.2 was designated as a variant under investigation by the UK Health and Security Agency on 20 October 2021. This study aimed to investigate vaccine effectiveness (VE) against symptomatic COVID-19 (Coronavirus disease 2019) infection and COVID-19 hospitalisation/death for the AY.4.2 variant.

Methods

We used the Scotland-wide Early Pandemic Evaluation and Enhanced Surveillance (EAVE-II) platform to estimate the VE of the ChAdOx1, BNT162b2, and mRNA-1273 vaccines against symptomatic infection and severe COVID-19 outcomes in adults. The study was conducted from June 8 to October 25, 2021. We used a test-negative design (TND) to estimate VE against reverse transcriptase polymerase chain reaction (RT-PCR) confirmed symptomatic SARS-CoV-2 infection while adjusting for sex, socioeconomic status, number of coexisting conditions, and splines in time and age. We also performed a cohort study using a Cox proportional hazards model to estimate VE against a composite outcome of COVID-19 hospital admission or death, with the same adjustments.

Results

We found an overall VE against symptomatic SARS-CoV-2 infection due to AY.4.2 of 73% (95% confidence interval (CI) = 62-81) for >14 days post-second vaccine dose. Good protection against AY.4.2 symptomatic infection was observed for BNT162b2, ChAdOx1, and mRNA-1273. In unvaccinated individuals, the hazard ratio (HR) for COVID-19 hospital admission or death from AY.4.2 among community detected cases was 1.77 (95% CI = 1.02-3.07) relative to unvaccinated individuals who were infected with Delta, after adjusting for multiple potential confounders. VE against AY.4.2 COVID-19 admissions or deaths was 87% (95% CI = 74-93) >28 days post-second vaccination relative to unvaccinated.

Conclusions

We found that AY.4.2 was associated with an increased risk of COVID-19 hospitalisations or deaths in unvaccinated individuals compared with Delta and that vaccination provided substantial protection against symptomatic SARS-CoV-2 and severe COVID-19 outcomes following Delta AY.4.2 infection. High levels of vaccine uptake and protection offered by existing vaccines, as well as the rapid emergence of the Omicron variant may have contributed to the AY.4.2 variant never progressing to a variant of concern.

Risk of COVID-19 hospital admission among children aged 5-17 years with asthma in Scotland: a national incident cohort study

BACKGROUND

There is an urgent need to inform policy deliberations about whether children with asthma should be vaccinated against SARS-CoV-2 and, if so, which subset of children with asthma should be prioritised. We were asked by the UK's Joint Commission on Vaccination and Immunisation to undertake an urgent analysis to identify which children with asthma were at increased risk of serious COVID-19 outcomes.

METHODS

This national incident cohort study was done in all children in Scotland aged 5-17 years who were included in the linked dataset of Early Pandemic Evaluation and Enhanced Surveillance of COVID-19 (EAVE II). We used data from EAVE II to investigate the risk of COVID-19 hospitalisation among children with markers of uncontrolled asthma defined by either previous asthma hospital admission or oral corticosteroid prescription in the previous 2 years. A Cox proportional hazard model was used to derive hazard ratios (HRs) and 95% CIs for the association between asthma and COVID-19 hospital admission, stratified by markers of asthma control (previous asthma hospital admission and number of previous prescriptions for oral corticosteroids within 2 years of the study start date). Analyses were adjusted for age, sex, socioeconomic status, comorbidity, and previous hospital admission.

FINDINGS

Between March 1, 2020, and July 27, 2021, 752 867 children were included in the EAVE II dataset, 63 463 (8·4%) of whom had clinician-diagnosed-and-recorded asthma. Of these, 4339 (6·8%) had RT-PCR confirmed SARS-CoV-2 infection. In those with confirmed infection, 67 (1·5%) were admitted to hospital with COVID-19. Among the 689 404 children without asthma, 40 231 (5·8%) had confirmed SARS-CoV-2 infections, of whom 382 (0·9%) were admitted to hospital with COVID-19. The rate of COVID-19 hospital admission was higher in children with poorly controlled asthma than in those with well controlled asthma or without asthma. When using previous hospital admission for asthma as the marker of uncontrolled asthma, the adjusted HR was 6·40 (95% CI 3·27-12·53) for those with poorly controlled asthma and 1·36 (1·02-1·80) for those with well controlled asthma, compared with those with no asthma. When using oral corticosteroid prescriptions as the marker of uncontrolled asthma, the adjusted HR was 3·38 (1·84-6·21) for those with three or more prescribed courses of corticosteroids, 3·53 (1·87-6·67) for those with two prescribed courses of corticosteroids, 1·52 (0·90-2·57) for those with one prescribed course of corticosteroids, and 1·34 (0·98-1·82) for those with no prescribed course, compared with those with no asthma.

INTERPRETATION

School-aged children with asthma with previous recent hospital admission or two or more courses of oral corticosteroids are at markedly increased risk of COVID-19 hospital admission and should be considered a priority for vaccinations. This would translate into 9124 children across Scotland and an estimated 109 448 children across the UK.

FUNDING

UK Research and Innovation (Medical Research Council), Research and Innovation Industrial Strategy Challenge Fund, Health Data Research UK, and Scottish Government.

BNT162b2 and ChAdOx1 nCoV-19 vaccinations, incidence of SARS-CoV-2 infections and COVID-19 hospitalisations in Scotland in the Delta era

Background

The emergence of the B.1.617.2 Delta variant of concern was associated with increasing numbers of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections and COVID-19 hospital admissions. We aim to study national population level SARS-CoV-2 infections and COVID-19 associated hospitalisations by vaccination status to provide insight into the association of vaccination on temporal trends during the time in which the SARS-CoV-2 Delta variant became dominant in Scotland.

Methods

We used the Scotland-wide Early Pandemic Evaluation and Enhanced Surveillance (EAVE II) platform, covering the period when Delta was pervasive (May 01 to October 23, 2021). We performed a cohort analysis of every vaccine-eligible individual aged 20 or over from across Scotland. We determined the vaccination coverage, SARS-CoV-2 incidence rate and COVID-19 associated hospitalisations incidence rate. We then stratified those rates by age group, vaccination status (defined as "unvaccinated", "partially vaccinated" (1 dose), or "fully vaccinated" (2 doses)), vaccine type (BNT162b2 or ChAdOx1 nCoV-19), and coexisting conditions known to be associated with severe COVID-19 outcomes.

Results

During the follow-up of 4 183 022 individuals, there were 407 405 SARS-CoV-2 positive cases with 10 441 (2.6%) associated with a hospital admission. Those vaccinated with two doses (defined as fully vaccinated in the current study) of either vaccine had lower incidence rates of SARS-CoV-2 infections and much lower incidence rates of COVID-19 associated hospitalisations than those unvaccinated in the Delta era in Scotland. Younger age groups were substantially more likely to get infected. In contrast, older age groups were much more likely to be hospitalised. The incidence rates stratified by coexisting conditions were broadly comparable with the overall age group patterns.

Conclusions

This study suggests that national population level vaccination was associated with a reduction in SARS-CoV-2 infections and COVID-19 associated hospitalisation in Scotland throughout the Delta era.

Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal

OBJECTIVE

To review and appraise the validity and usefulness of published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at increased risk of covid-19 infection or being admitted to hospital with the disease.

DESIGN

Living systematic review and critical appraisal by the COVID-PRECISE (Precise Risk Estimation to optimise covid-19 Care for Infected or Suspected patients in diverse sEttings) group.

DATA SOURCES

PubMed and Embase through Ovid, up to 1 July 2020, supplemented with arXiv, medRxiv, and bioRxiv up to 5 May 2020.

STUDY SELECTION

Studies that developed or validated a multivariable covid-19 related prediction model.

DATA EXTRACTION

At least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool).

RESULTS

37 421 titles were screened, and 169 studies describing 232 prediction models were included. The review identified seven models for identifying people at risk in the general population; 118 diagnostic models for detecting covid-19 (75 were based on medical imaging, 10 to diagnose disease severity); and 107 prognostic models for predicting mortality risk, progression to severe disease, intensive care unit admission, ventilation, intubation, or length of hospital stay. The most frequent types of predictors included in the covid-19 prediction models are vital signs, age, comorbidities, and image features. Flu-like symptoms are frequently predictive in diagnostic models, while sex, C reactive protein, and lymphocyte counts are frequent prognostic factors. Reported C index estimates from the strongest form of validation available per model ranged from 0.71 to 0.99 in prediction models for the general population, from 0.65 to more than 0.99 in diagnostic models, and from 0.54 to 0.99 in prognostic models. All models were rated at high or unclear risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, high risk of model overfitting, and unclear reporting. Many models did not include a description of the target population (n=27, 12%) or care setting (n=75, 32%), and only 11 (5%) were externally validated by a calibration plot. The Jehi diagnostic model and the 4C mortality score were identified as promising models.

CONCLUSION

Prediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that almost all pubished prediction models are poorly reported, and at high risk of bias such that their reported predictive performance is probably optimistic. However, we have identified two (one diagnostic and one prognostic) promising models that should soon be validated in multiple cohorts, preferably through collaborative efforts and data sharing to also allow an investigation of the stability and heterogeneity in their performance across populations and settings. Details on all reviewed models are publicly available at https://www.covprecise.org/. Methodological guidance as provided in this paper should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, prediction model authors should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline.

SYSTEMATIC REVIEW REGISTRATION

Protocol https://osf.io/ehc47/, registration https://osf.io/wy245.

READERS' NOTE

This article is a living systematic review that will be updated to reflect emerging evidence. Updates may occur for up to two years from the date of original publication. This version is update 3 of the original article published on 7 April 2020 (BMJ 2020;369:m1328). Previous updates can be found as data supplements (https://www.bmj.com/content/369/bmj.m1328/related#datasupp). When citing this paper please consider adding the update number and date of access for clarity.