A systematic review of quantitative bias analysis applied to epidemiological research

The economic rationale for cell-based influenza vaccines in children and adults: A review of cost-effectiveness analyses

Seasonal influenza significantly affects both health and economic costs in children and adults. This narrative review summarizes published cost-effectiveness analyses (CEAs) of cell-based influenza vaccines in children and adults <65 years of age, critically assesses the assumptions and approaches used in these analyses, and considers the role of cell-based influenza vaccines for children and adults. CEAs from multiple countries demonstrated the cost-effectiveness of cell-based quadrivalent influenza vaccines (QIVc) compared with egg-based trivalent/quadrivalent influenza vaccines (TIVe/QIVe). CEA findings were consistent across models relying on different relative vaccine effectiveness (rVE) estimate inputs, with the rVE of QIVc versus QIVe ranging from 8.1% to 36.2% in favor of QIVc. Across multiple scenarios and types of analyses, QIVc was consistently cost-effective compared with QIVe, including in children and adults across different regions of the world.

Estimating the impact of bias in causal epidemiological studies: the case of health outcomes following assisted reproduction

Researchers interested in causal questions must deal with two sources of error: random error (random deviation from the true mean value of a distribution), and bias (systematic deviance from the true mean value due to extraneous factors). For some causal questions, randomization is not feasible, and observational studies are necessary. Bias poses a substantial threat to the validity of observational research and can have important consequences for health policy developed from the findings. The current piece describes bias and its sources, outlines proposed methods to estimate its impacts in an observational study, and demonstrates how these methods may be used to inform debate on the causal relationship between medically assisted reproduction (MAR) and health outcomes, using cancer as an example. In doing so, we aim to enlighten researchers who work with observational data, especially regarding the health effects of MAR and infertility, on the pitfalls of bias, and how to address them. We hope that, in combination with the provided example, we can convince readers that estimating the impact of bias in causal epidemiologic research is not only important but necessary to inform the development of robust health policy and clinical practice recommendations.

Investigation of the structure and magnitude of time-varying uncontrolled confounding in simulated cohort data analyzed using g-computation

BACKGROUND

When estimating the effect of time-varying exposures on longer-term outcomes, the assumption of conditional exchangeability or no uncontrolled confounding extends beyond baseline confounding to include time-varying confounding. We illustrate the structures and magnitude of uncontrolled time-varying confounding in exposure effect estimates obtained from g-computation when sequential conditional exchangeability is violated.

METHODS

We used directed acyclic graphs (DAGs) to depict time-varying uncontrolled confounding. We performed simulations and used g-computation to quantify the effects of each time-varying exposure for each DAG type. Models adjusting all time-varying confounders were considered the true (bias-adjusted) estimate. The exclusion of time-varying uncontrolled confounders represented the biased effect estimate and an unmet 'no uncontrolled confounding' assumption. True and biased estimates were compared across DAGs, with different magnitudes of uncontrolled confounding.

RESULTS

Time-varying uncontrolled confounding can present in several scenarios, including relationships into subsequently measured exposure(s), outcome, unmeasured confounder(s) and other measured confounder(s). In simulations, effect estimates obtained from g-computation were more biased in DAGs when the uncontrolled confounders were directly related to the outcome. Complex DAGs that included relationships between uncontrolled confounders and other variables and relationships where exposures caused uncontrolled confounders at the next time point resulted in the most biased effect estimates. In these complex DAGs, excluding uncontrolled confounders affected the multiple effect estimates.

CONCLUSIONS

Time-varying uncontrolled confounding has the potential to substantially impact observed effect estimates. Given the importance of longitudinal studies in advising public health, the impact of time-varying uncontrolled confounding warrants more recognition and evaluation using quantitative bias analysis.

Bias analyses to investigate the impact of differential participation: Application to a birth defects case-control study

BACKGROUND

Certain associations observed in the National Birth Defects Prevention Study (NBDPS) contrasted with other research or were from areas with mixed findings, including no decrease in odds of spina bifida with periconceptional folic acid supplementation, moderately increased cleft palate odds with ondansetron use and reduced hypospadias odds with maternal smoking.

OBJECTIVES

To investigate the plausibility and extent of differential participation to produce effect estimates observed in NBDPS.

METHODS

We searched the literature for factors related to these exposures and participation and conducted deterministic quantitative bias analyses. We estimated case-control participation and expected exposure prevalence based on internal and external reports, respectively. For the folic acid-spina bifida and ondansetron-cleft palate analyses, we hypothesized the true odds ratio (OR) based on prior studies and quantified the degree of exposure over- (or under-) representation to produce the crude OR (cOR) in NBDPS. For the smoking-hypospadias analysis, we estimated the extent of selection bias needed to nullify the association as well as the maximum potential harmful OR.

RESULTS

Under our assumptions (participation, exposure prevalence, true OR), there was overrepresentation of folic acid use and underrepresentation of ondansetron use and smoking among participants. Folic acid-exposed spina bifida cases would need to have been ≥1.2× more likely to participate than exposed controls to yield the observed null cOR. Ondansetron-exposed cleft palate cases would need to have been 1.6× more likely to participate than exposed controls if the true OR is null. Smoking-exposed hypospadias cases would need to have been ≥1.2 times less likely to participate than exposed controls for the association to falsely appear protective (upper bound of selection bias adjusted smoking-hypospadias OR = 2.02).

CONCLUSIONS

Differential participation could partly explain certain associations observed in NBDPS, but questions remain about why. Potential impacts of other systematic errors (e.g. exposure misclassification) could be informed by additional research.

Quantitative Assessment of Systematic Bias: A Guide for Researchers

Observational research provides valuable opportunities to advance oral health science but is limited by vulnerabilities to systematic bias, including unmeasured confounding, errors in variable measurement, or bias in the creation of study populations and/or analytic samples. The potential influence of systematic biases on observed results is often only briefly mentioned among the discussion of limitations of a given study, despite existing methods that support detailed assessments of their potential effects. Quantitative bias analysis is a set of methodological techniques that, when applied to observational data, can provide important context to aid in the interpretation and integration of observational research findings into the broader body of oral health research. Specifically, these methods were developed to provide quantitative estimates of the potential magnitude and direction of the influence of systematic biases on observed results. We aim to encourage and facilitate the broad adoption of quantitative bias analyses into observational oral health research. To this end, we provide an overview of quantitative bias analysis techniques, including a step-by-step implementation guide. We also provide a detailed appendix that guides readers through an applied example using real data obtained from a prospective observational cohort study of preconception periodontitis in relation to time to pregnancy. Quantitative bias analysis methods are available to all investigators. When appropriately applied to observational studies, findings from such studies can have a greater impact in the broader research context.

Performance of French medico-administrative databases in epidemiology of infectious diseases: a scoping review

The development of medico-administrative databases over the last few decades has led to an evolution and to a significant production of epidemiological studies on infectious diseases based on retrospective medical data and consumption of care. This new form of epidemiological research faces numerous methodological challenges, among which the assessment of the validity of targeting algorithm. We conducted a scoping review of studies that undertook an estimation of the completeness and validity of French medico-administrative databases for infectious disease epidemiological research. Nineteen validation studies and nine capture-recapture studies were identified. These studies covered 20 infectious diseases and were mostly based on the evaluation of hospital claimed data. The evaluation of their methodological qualities highlighted the difficulties associated with these types of research, particularly those linked to the assessment of their underlying hypotheses. We recall several recommendations relating to the problems addressed, which should contribute to the quality of future evaluation studies based on medico-administrative data and consequently to the quality of the epidemiological indicators produced from these information systems.

Quantitative bias analysis of prevalence under misclassification: evaluation indicators, calculation method and case analysis

Prevalence estimates are fundamental to epidemiological studies. Although they are highly vulnerable to misclassification bias, the risk of bias assessment of prevalence estimates is often neglected. Quantitative bias analysis (QBA) can effectively estimate misclassification bias in epidemiological studies; however, relatively few applications are identified. One reason for its low usage is the lack of knowledge and tools for these methods among researchers. To expand existing evaluation methods, based on the QBA principles, three indicators are proposed. One is the relative bias that quantifies the bias direction through its signs and the bias magnitude through its quantity. The second is the critical point of positive test proportion in case of a misclassification bias that is equal to zero. The third is the bound of positive test proportion equal to adjusted prevalence at misclassification bias level α. These indicators express the magnitude, direction and uncertainty of the misclassification bias of prevalence estimates, respectively. Using these indicators, it was found that slight oscillations of the positive test proportion within a certain range can lead to substantial increases in the misclassification bias. Hence, researchers should account for misclassification error analytically when interpreting the significance of adjusted prevalence for epidemiological decision making. This highlights the importance of applying QBA to these analyses. In this article, we have used three real-world cases to illustrate the characteristics and calculation methods of presented indicators. To facilitate application, an Excel-based calculation tool is provided.

Accounting for Misclassification and Selection Bias in Estimating Effectiveness of Self-managed Medication Abortion

BACKGROUND

Studies on the effectiveness of self-managed medication abortion may suffer from misclassification and selection bias due to self-reported outcomes and loss of follow-up. Monte Carlo sensitivity analysis can estimate self-managed abortion effectiveness accounting for these potential biases.

METHODS

We conducted a Monte Carlo sensitivity analysis based on data from the Studying Accompaniment model Feasibility and Effectiveness Study (the SAFE Study), to generate bias-adjusted estimates of the effectiveness of self-managed abortion with accompaniment group support. Between July 2019 and April 2020, we enrolled a total of 1051 callers who contacted accompaniment groups in Argentina and Nigeria for self-managed abortion information; 961 took abortion medications and completed at least one follow-up. Using these data, we calculated measures of effectiveness adjusted for ineligibility, misclassification, and selection bias across 50,000 simulations with bias parameters drawn from pre-specified Beta distributions in R.

RESULTS

After accounting for the potential influence of various sources of bias, bias-adjusted estimates of effectiveness were similar to observed estimates, conditional on chosen bias parameters: 92.68% (95% simulation interval: 87.80%, 95.74%) for mifepristone in combination with misoprostol (versus 93.7% in the observed data) and 98.47% (95% simulation interval: 96.79%, 99.39%) for misoprostol alone (versus 99.3% in the observed data).

CONCLUSIONS

After adjustment for multiple potential sources of bias, estimates of self-managed medication abortion effectiveness remain high. Monte Carlo sensitivity analysis may be useful in studies measuring an epidemiologic proportion (i.e., effectiveness, prevalence, cumulative incidence) while accounting for possible selection or misclassification bias.

Causal Modeling to Mitigate Selection Bias and Unmeasured Confounding in Internet-Based Epidemiology of COVID-19: Model Development and Validation

BACKGROUND

Selection bias and unmeasured confounding are fundamental problems in epidemiology that threaten study internal and external validity. These phenomena are particularly dangerous in internet-based public health surveillance, where traditional mitigation and adjustment methods are inapplicable, unavailable, or out of date. Recent theoretical advances in causal modeling can mitigate these threats, but these innovations have not been widely deployed in the epidemiological community.

OBJECTIVE

The purpose of our paper is to demonstrate the practical utility of causal modeling to both detect unmeasured confounding and selection bias and guide model selection to minimize bias. We implemented this approach in an applied epidemiological study of the COVID-19 cumulative infection rate in the New York City (NYC) spring 2020 epidemic.

METHODS

We collected primary data from Qualtrics surveys of Amazon Mechanical Turk (MTurk) crowd workers residing in New Jersey and New York State across 2 sampling periods: April 11-14 and May 8-11, 2020. The surveys queried the subjects on household health status and demographic characteristics. We constructed a set of possible causal models of household infection and survey selection mechanisms and ranked them by compatibility with the collected survey data. The most compatible causal model was then used to estimate the cumulative infection rate in each survey period.

RESULTS

There were 527 and 513 responses collected for the 2 periods, respectively. Response demographics were highly skewed toward a younger age in both survey periods. Despite the extremely strong relationship between age and COVID-19 symptoms, we recovered minimally biased estimates of the cumulative infection rate using only primary data and the most compatible causal model, with a relative bias of +3.8% and -1.9% from the reported cumulative infection rate for the first and second survey periods, respectively.

CONCLUSIONS

We successfully recovered accurate estimates of the cumulative infection rate from an internet-based crowdsourced sample despite considerable selection bias and unmeasured confounding in the primary data. This implementation demonstrates how simple applications of structural causal modeling can be effectively used to determine falsifiable model conditions, detect selection bias and confounding factors, and minimize estimate bias through model selection in a novel epidemiological context. As the disease and social dynamics of COVID-19 continue to evolve, public health surveillance protocols must continue to adapt; the emergence of Omicron variants and shift to at-home testing as recent challenges. Rigorous and transparent methods to develop, deploy, and diagnosis adapted surveillance protocols will be critical to their success.

Quantifying the influence of bias in reproductive and perinatal epidemiology through simulation

PURPOSE

The application of simulated data in epidemiological studies enables the illustration and quantification of the magnitude of various types of bias commonly found in observational studies. This was a review of the application of simulation methods to the quantification of bias in reproductive and perinatal epidemiology and an assessment of value gained.

METHODS

A search of published studies available in English was conducted in August 2020 using PubMed, Medline, Embase, CINAHL, and Scopus. A gray literature search of Google and Google Scholar, and a hand search using the reference lists of included studies was undertaken.

RESULTS

Thirty-nine papers were included in this study, covering information (n = 14), selection (n = 14), confounding (n = 9), protection (n = 1), and attenuation bias (n = 1). The methods of simulating data and reporting of results varied, with more recent studies including causal diagrams. Few studies included code for replication.

CONCLUSIONS

Although there has been an increasing application of simulation in reproductive and perinatal epidemiology since 2015, overall this remains an underexplored area. Further efforts are required to increase knowledge of how the application of simulation can quantify the influence of bias, including improved design, analysis and reporting. This will improve causal interpretation in reproductive and perinatal studies.