Current Challenges With the Use of Test-Negative Designs for Modeling COVID-19 Vaccination and Outcomes

California's COVID-19 Vaccine Equity Policy: Cases, Hospitalizations, And Deaths Averted In Affected Communities

In March 2021, California implemented a vaccine equity policy that prioritized COVID-19 vaccine allocation to communities identified as least advantaged by an area-based socioeconomic measure, the Healthy Places Index. We conducted quasi-experimental and counterfactual analyses to estimate the effect of this policy on COVID-19 vaccination, case, hospitalization, and death rates. Among prioritized communities, vaccination rates increased 28.4 percent after policy implementation. Furthermore, an estimated 160,892 COVID-19 cases, 10,248 hospitalizations, and 679 deaths in the least-advantaged communities were averted by the policy. Despite these improvements, the share of COVID-19 cases, hospitalizations, and deaths in prioritized communities remained elevated. These estimates were robust in sensitivity analyses that tested exchangeability between prioritized communities and those not prioritized by the policy; model specifications; and potential temporal confounders, including prior infections. Correcting for disparities by strategically allocating limited resources to the least-advantaged or most-affected communities can reduce the impacts of COVID-19 and other diseases but might not eliminate health disparities.

Effectiveness of a bivalent mRNA vaccine dose against symptomatic SARS-CoV-2 infection among U.S. Healthcare personnel, September 2022-May 2023

BACKGROUND

Bivalent mRNA vaccines were recommended since September 2022. However, coverage with a recent vaccine dose has been limited, and there are few robust estimates of bivalent VE against symptomatic SARS-CoV-2 infection (COVID-19). We estimated VE of a bivalent mRNA vaccine dose against COVID-19 among eligible U.S. healthcare personnel who had previously received monovalent mRNA vaccine doses.

METHODS

We conducted a case-control study in 22 U.S. states, and enrolled healthcare personnel with COVID-19 (case-participants) or without COVID-19 (control-participants) during September 2022-May 2023. Participants were considered eligible for a bivalent mRNA dose if they had received 2-4 monovalent (ancestral-strain) mRNA vaccine doses, and were ≥67 days after the most recent vaccine dose. We estimated VE of a bivalent mRNA dose using conditional logistic regression, accounting for matching by region and four-week calendar period. We adjusted estimates for age group, sex, race and ethnicity, educational level, underlying health conditions, community COVID-19 exposure, prior SARS-CoV-2 infection, and days since the last monovalent mRNA dose.

RESULTS

Among 3,647 healthcare personnel, 1,528 were included as case-participants and 2,119 as control-participants. Participants received their last monovalent mRNA dose a median of 404 days previously; 1,234 (33.8%) also received a bivalent mRNA dose a median of 93 days previously. Overall, VE of a bivalent dose was 34.1% (95% CI, 22.6%-43.9%) against COVID-19 and was similar by product, days since last monovalent dose, number of prior doses, age group, and presence of underlying health conditions. However, VE declined from 54.8% (95% CI, 40.7%-65.6%) after 7-59 days to 21.6% (95% CI 5.6%-34.9%) after ≥60 days.

CONCLUSIONS

Bivalent mRNA COVID-19 vaccines initially conferred approximately 55% protection against COVID-19 among U.S. healthcare personnel. However, protection waned after two months. These findings indicate moderate initial protection against symptomatic SARS-CoV-2 infection by remaining up-to-date with COVID-19 vaccines.

ution of the data and methods in real-world COVID-19 vaccine effectiveness studies on mortality: a oping eview protocol

BACKGROUND

Early evidence on COVID-19 vaccine efficacy came from randomised trials. Many important questions subsequently about vaccine effectiveness (VE) have been addressed using real-world studies (RWS) and have informed most vaccination policies globally. As the questions about VE have evolved during the pandemic so have data, study design, and analytical choices. This scoping review aims to characterise this evolution and provide insights for future pandemic planning-specifically, what kinds of questions are asked at different stages of a pandemic, and what data infrastructure and methods are used?

METHODS AND ANALYSIS

We will identify relevant studies in the Johns Hopkins Bloomberg School of Public Health VIEW-hub database, which curates both published and preprint VE RWS identified from PubMed, Embase, Scopus, Web of Science, the WHO COVID Database, MMWR, Eurosurveillance, medRxiv, bioRxiv, SSRN, Europe PMC, Research Square, Knowledge Hub, and Google. We will include RWS of COVID-19 VE that reported COVID-19-specific or all-cause mortality (coded as 'death' in the 'effectiveness studies' data set).Information on study characteristics; study context; data sources; design and analytic methods that address confounding will be extracted by single reviewer and checked for accuracy and discussed in a small group setting by methodological and analytic experts. A timeline mapping approach will be used to capture the evolution of this body of literature.By describing the evolution of RWS of VE through the COVID-19 pandemic, we will help identify options for VE studies and inform policy makers on the minimal data and analytic infrastructure needed to support rapid RWS of VE in future pandemics and of healthcare strategies more broadly.

ETHICS AND DISSEMINATION

As data is in the public domain, ethical approval is not required. Findings of this study will be disseminated through peer-reviewed publications, conference presentations, and working-papers to policy makers.

REGISTRATION

https://doi.org/10.17605/OSF.IO/ZHDKR.

Effectiveness of Immunization Products Against Medically Attended Respiratory Syncytial Virus Infection: Generic Protocol for a Test-Negative Case-Control Study

Monitoring the real-life effectiveness of respiratory syncytial virus (RSV) products is of major public health importance. This generic protocol for a test-negative design study aims to address currently envisioned approaches for RSV prevention (monoclonal antibodies and vaccines) to study effectiveness of these products among target groups: children, older adults, and pregnant women. The generic protocol approach was chosen to allow for flexibility in adapting the protocol to a specific setting. This protocol includes severe acute respiratory infection (SARI) and acute respiratory infection (ARI), both due to RSV, as end points. These end points can be applied to studies in hospitals, primarily targeting patients with more severe disease, but also to studies in general practitioner clinics targeting ARI.

Utilizing a university testing program to estimate relative effectiveness of monovalent COVID-19 mRNA booster vaccine versus two-dose primary series against symptomatic SARS-CoV-2 infection

Vaccine effectiveness (VE) studies utilizing the test-negative design are typically conducted in clinical settings, rather than community populations, leading to bias in VE estimates against mild disease and limited information on VE in healthy young adults. In a community-based university population, we utilized data from a large SARS-CoV-2 testing program to estimate relative VE of COVID-19 mRNA vaccine primary series and monovalent booster dose versus primary series only against symptomatic SARS-CoV-2 infection from September 2021 to July 2022. We used the test-negative design and logistic regression implemented via generalized estimating equations adjusted for age, calendar time, prior SARS-CoV-2 infection, and testing frequency (proxy for test-seeking behavior) to estimate relative VE. Analyses included 2,218 test-positive cases (59 % received monovalent booster dose) and 9,615 test-negative controls (62 %) from 9,066 individuals, with median age of 21 years, mostly students (71 %), White (56 %) or Asian (28 %), and with few comorbidities (3 %). More cases (23 %) than controls (6 %) had COVID-19-like illness. Estimated adjusted relative VE of primary series and monovalent booster dose versus primary series only against symptomatic SARS-CoV-2 infection was 40 % (95 % CI: 33-47 %) during the overall analysis period and 46 % (39-52 %) during the period of Omicron circulation. Relative VE was greater for those without versus those with prior SARS-CoV-2 infection (41 %, 34-48 % versus 33 %, 9 %-52 %, P < 0.001). Relative VE was also greater in the six months after receiving a booster dose (41 %, 33-47 %) compared to more than six months (27 %, 8-42 %), but this difference was not statistically significant (P = 0.06). In this relatively young and healthy adult population, an mRNA monovalent booster dose provided increased protection against symptomatic SARS-CoV-2 infection, overall and with the Omicron variant. University testing programs may be utilized for estimating VE in healthy young adults, a population that is not well-represented by routine VE studies.

Effect of second booster vaccinations and prior infection against SARS-CoV-2 in the UK SIREN healthcare worker cohort

Background

The protection of fourth dose mRNA vaccination against SARS-CoV-2 is relevant to current global policy decisions regarding ongoing booster roll-out. We aimed to estimate the effect of fourth dose vaccination, prior infection, and duration of PCR positivity in a highly-vaccinated and largely prior-COVID-19 infected cohort of UK healthcare workers.

Methods

Participants underwent fortnightly PCR and regular antibody testing for SARS-CoV-2 and completed symptoms questionnaires. A multi-state model was used to estimate vaccine effectiveness (VE) against infection from a fourth dose compared to a waned third dose, with protection from prior infection and duration of PCR positivity jointly estimated.

Findings

1298 infections were detected among 9560 individuals under active follow-up between September 2022 and March 2023. Compared to a waned third dose, fourth dose VE was 13.1% (95% CI 0.9 to 23.8) overall; 24.0% (95% CI 8.5 to 36.8) in the first 2 months post-vaccination, reducing to 10.3% (95% CI -11.4 to 27.8) and 1.7% (95% CI -17.0 to 17.4) at 2-4 and 4-6 months, respectively. Relative to an infection >2 years ago and controlling for vaccination, 63.6% (95% CI 46.9 to 75.0) and 29.1% (95% CI 3.8 to 43.1) greater protection against infection was estimated for an infection within the past 0-6, and 6-12 months, respectively. A fourth dose was associated with greater protection against asymptomatic infection than symptomatic infection, whilst prior infection independently provided more protection against symptomatic infection, particularly if the infection had occurred within the previous 6 months. Duration of PCR positivity was significantly lower for asymptomatic compared to symptomatic infection.

Interpretation

Despite rapid waning of protection, vaccine boosters remain an important tool in responding to the dynamic COVID-19 landscape; boosting population immunity in advance of periods of anticipated pressure, such as surging infection rates or emerging variants of concern.

Funding

UK Health Security Agency, Medical Research Council, NIHR HPRU Oxford, Bristol, and others.

Design and analysis heterogeneity in observational studies of COVID-19 booster effectiveness: A review and case study

We investigated the design and analysis of observational booster vaccine effectiveness (VE) studies by performing a scoping review of booster VE literature with a focus on study design and analytic choices. We then applied 20 different approaches, including those found in the literature, to a single dataset from Michigan Medicine. We identified 80 studies in our review, including over 150 million observations in total. We found that while protection against infection is variable and dependent on several factors including the study population and time period, both monovalent boosters and particularly the bivalent booster offer strong protection against severe COVID-19. In addition, VE analyses with a severe disease outcome (hospitalization, intensive care unit admission, or death) appear to be more robust to design and analytic choices than an infection endpoint. In terms of design choices, we found that test-negative designs and their variants may offer advantages in statistical efficiency compared to cohort designs.

Building the U.S. COVID-19 vaccine effectiveness program: Past successes and future directions

COVID-19 vaccines were originally authorized in the United States in December 2020 on the basis of safety, immunogenicity, and clinical efficacy data from randomized controlled trials (RCTs). However, real-world vaccine effectiveness (VE) data are necessary to provide information on how the vaccines work in populations not included in the RCTs (e.g., nursing home residents), against new SARS-CoV-2 variants, with increasing time since vaccination, and in populations with increasing levels of prior infection. The goal of CDC's COVID-19 VE program is to provide timely and robust data to support ongoing policy decisions and implementation of vaccination and includes VE platforms to study the spectrum of illness, from infection to critical illness. Challenges to estimating VE include accurate ascertainment of vaccination history, outcome status, changing rates of prior infection, emergence of new variants, and appropriate interpretation of absolute and relative VE measures. CDC COVID-19 VE platforms have played a pivotal role in numerous vaccine policy decisions since 2021 and will continue to play a key role in future decisions as the vaccine program moves from an emergency response to a routine schedule.

Effectiveness of BNT162b2 BA.4/5 bivalent mRNA vaccine against a range of COVID-19 outcomes in a large health system in the USA: a test-negative case-control study

BACKGROUND

XBB-related omicron sublineages have recently replaced BA.4/5 as the predominant omicron sublineages in the USA and other regions globally. Despite preliminary signs of immune evasion of XBB sublineages, few data exist describing the real-world effectiveness of bivalent COVID-19 vaccines, especially against XBB-related illness. We aimed to investigate the effectiveness of the Pfizer--BioNTech BNT162b2 BA.4/5 bivalent vaccine against both BA.4/5-related and XBB-related disease in adults aged 18 years or older.

METHODS

In this test-negative case-control study, we estimated the effectiveness of the BNT162b2 BA.4/5 bivalent vaccine using data from electronic health records of Kaiser Permanente Southern California health system members aged 18 years or older who received at least two doses of the wild-type COVID-19 mRNA vaccines. Participants sought care for acute respiratory infection between Aug 31, 2022, and April 15, 2023, and were tested for SARS-CoV-2 via PCR tests. Relative vaccine effectiveness (≥2 doses of wild-type mRNA vaccine plus a BNT162b2 BA.4/5 bivalent booster vs ≥2 doses of a wild-type mRNA vaccine alone) and absolute vaccine effectiveness (vs unvaccinated individuals) was estimated against critical illness related to acute respiratory infection (intensive care unit [ICU] admission, mechanical ventilation, or inpatient death), hospital admission, emergency department or urgent care visits, and in-person outpatient encounters with odds ratios from logistic regression models adjusted for demographic and clinical factors. We stratified vaccine effectiveness estimates for hospital admission, emergency department or urgent care visits, and outpatient encounters by omicron sublineage (ie, likely BA.4/5-related vs likely XBB-related), time since bivalent booster receipt, age group, number of wild-type doses received, and immunocompromised status. This study is registered with ClinicalTrials.gov (NCT04848584).

FINDINGS

Analyses were conducted for 123 419 encounters (24 246 COVID-19 cases and 99 173 test-negative controls), including 4131 episode of critical illness (a subset of hospital admissions), 14 529 hospital admissions, 63 566 emergency department or urgent care visits, and 45 324 outpatient visits. 20 555 infections were BA.4/5 related and 3691 were XBB related. In adjusted analyses, relative vaccine effectiveness for those who received the BNT162b2 BA.4/5 bivalent booster compared with those who received at least two doses of a wild-type mRNA vaccine alone was an additional 50% (95% CI 23-68) against critical illness, an additional 39% (28-49) against hospital admission, an additional 35% (30-40) against emergency department or urgent care visits, and an additional 28% (22-33) against outpatient encounters. Waning of the bivalent booster from 0-3 months to 4-7 months after vaccination was evident for outpatient outcomes but was not detected for critical illness, hospital admission, and emergency department or urgent care outcomes. The relative effectiveness of the BNT162b2 BA.4/5 bivalent booster for XBB-related infections compared with BA.4/5-related infections was 56% (95% CI 12-78) versus 40% (27-50) for hospital admission; 34% (21-45) versus 36% (30-41) against emergency department or urgent care visits; and 29% (19-38) versus 27% (20-33) for outpatient encounters.

INTERPRETATION

By mid-April, 2023, individuals previously vaccinated only with wild-type vaccines had little protection against COVID-19-including hospital admission. A BNT162b2 BA.4/5 bivalent booster restored protection against a range of COVID-19 outcomes, including against XBB-related sublineages, with the most substantial protection observed against hospital admission and critical illness.

FUNDING

Pfizer.

Design and Analysis Heterogeneity in Observational Studies of COVID-19 Booster Effectiveness: A Review and Case Study

Background

Observational vaccine effectiveness (VE) studies based on real-world data are a crucial supplement to initial randomized clinical trials of Coronavirus Disease 2019 (COVID-19) vaccines. However, there exists substantial heterogeneity in study designs and statistical methods for estimating VE. The impact of such heterogeneity on VE estimates is not clear.

Methods

We conducted a two-step literature review of booster VE: a literature search for first or second monovalent boosters on January 1, 2023, and a rapid search for bivalent boosters on March 28, 2023. For each study identified, study design, methods, and VE estimates for infection, hospitalization, and/or death were extracted and summarized via forest plots. We then applied methods identified in the literature to a single dataset from Michigan Medicine (MM), providing a comparison of the impact of different statistical methodologies on the same dataset.

Results

We identified 53 studies estimating VE of the first booster, 16 for the second booster. Of these studies, 2 were case-control, 17 were test-negative, and 50 were cohort studies. Together, they included nearly 130 million people worldwide. VE for all outcomes was very high (around 90%) in earlier studies (i.e., in 2021), but became attenuated and more heterogeneous over time (around 40%-50% for infection, 60%-90% for hospitalization, and 50%-90% for death). VE compared to the previous dose was lower for the second booster (10-30% for infection, 30-60% against hospitalization, and 50-90% against death). We also identified 11 bivalent booster studies including over 20 million people. Early studies of the bivalent booster showed increased effectiveness compared to the monovalent booster (VE around 50-80% for hospitalization and death).Our primary analysis with MM data using a cohort design included 186,495 individuals overall (including 153,811 boosted and 32,684 with only a primary series vaccination), and a secondary test-negative design included 65,992 individuals tested for SARS-CoV-2. When different statistical designs and methods were applied to MM data, VE estimates for hospitalization and death were robust to analytic choices, with test-negative designs leading to narrower confidence intervals. Adjusting either for the propensity of getting boosted or directly adjusting for covariates reduced the heterogeneity across VE estimates for the infection outcome.

Conclusion

While the advantage of the second monovalent booster is not obvious from the literature review, the first monovalent booster and the bivalent booster appear to offer strong protection against severe COVID-19. Based on both the literature view and data analysis, VE analyses with a severe disease outcome (hospitalization, ICU admission, or death) appear to be more robust to design and analytic choices than an infection endpoint. Test-negative designs can extend to severe disease outcomes and may offer advantages in statistical efficiency when used properly.