Limitations and Misinterpretations of E-Values for Sensitivity Analyses of Observational Studies

Breaking Down Bias: A Methodological Primer on Identifying, Evaluating, and Mitigating Bias in Cardiovascular Research

Systematic error, often referred to as bias is an inherent challenge in observational cardiovascular research, and has the potential to profoundly influence the design, conduct, and interpretation of study results. If not carefully considered and managed, bias can lead to spurious results, which can misinform clinical practice or public health initiatives and compromise patient outcomes. This methodological primer offers a concise introduction to identifying, evaluating, and mitigating bias in observational cardiovascular research studies that examine the causal association between an exposure (or treatment) and an outcome. Using high-profile examples from the cardiovascular literature, this review provides a theoretical overview of 3 main types of bias-selection bias, information bias, and confounding-and discusses the implications of specialized types of biases commonly encountered in longitudinal cardiovascular research studies, namely, competing risks, immortal time bias, and confounding by indication. Furthermore, strategies and tools that can be used to minimize and assess the influence of bias are highlighted, with a specific focus on using the target trial framework, directed acyclic graphs, quantitative bias analysis, and formal risk of bias assessments. This review aims to assist researchers and health care professionals in designing observational studies and selecting appropriate methodologies to reduce bias, ultimately enhancing the estimation of causal associations in cardiovascular research.

Analysis of demographic variation and childhood correlates of financial well-being across 22 countries

Using nationally representative data from 202,898 participants in the Global Flourishing Study, this work examines factors associated with financial well-being across 22 countries. We investigate how demographic factors-including age, gender, marital status, employment status, education, religious service attendance and immigration status-are correlated with financial well-being (as assessed through four dimensions). Additionally, we analyse associations between recalled early-life conditions, such as parental marital status and childhood health, with financial well-being in adulthood. Our findings reveal cross-national differences in levels of financial well-being and its demographic correlates. Early-life conditions were also consistently associated with adult financial well-being, although these associations varied substantially across countries. These results suggest that understanding financial well-being should encompass both current sociodemographic factors and early-life experiences within the unique cultural and socioeconomic contexts of different populations.

Reply to 'The causal relationship between long-term exposure to ambient fine particulate matter and cognitive performance: Insights from Mendelian randomization'

Li et al. [1] have commented on our recent paper investigating the association between exposure to fine particulate matter (PM2.5) constituents and the risk of cognitive impairment [2]. They provided a Mendelian randomization (MR) analysis using large-scale genome-wide association study (GWAS) datasets from the European population, confirming a causal relationship between PM2.5 exposure and cognitive performance. In our reply, we employed three causal inference models, including a generalized propensity score (GPS) adjusted Cox model, an inverse-probability weights (IPW) weighted Cox model, and a trimmed IPW-weighted Cox model, to confirm the relationship of PM2.5 and cognitive impairment in our study cohort.

The Inflation Reduction Act: An Opportunity to Accelerate Confidence in Real-World Evidence in the United States

OBJECTIVES

The Inflation Reduction Act (IRA), enacted in 2022, brings substantial reforms to the US healthcare system, particularly regarding Medicare. A key aspect includes the introduction of Medicare price negotiation. The objective of this commentary is to explore the implications of the IRA for US pharmaceutical companies, with a specific focus on the role of real-world evidence (RWE) in the context of Medicare reforms.

METHODS

This commentary uses a qualitative analysis of the IRA's provisions related to healthcare and pharmaceutical regulation, focusing on how these reforms change the evidence requirements for pharmaceutical companies. It discusses the methodological aspects of generating and using RWE, including techniques such as target trial emulation and quantitative bias analysis methods to address biases inherent in RWE.

RESULTS

This commentary highlights that the IRA introduces a unique approach to value assessment in the United States by evaluating drug value several years after launch, as opposed to at launch, similar to health technology assessments in other regions. It underscores the central role of RWE in comparing drug effectiveness across diverse clinical scenarios to improve the accuracy of real-world data comparisons. Furthermore, this article identifies key methodologies for managing the inherent biases in RWE, which are crucial for generating credible evidence for IRA price negotiations.

CONCLUSIONS

This article underscores the importance of these methodologies in ensuring credible evidence for IRA price negotiations. It advocates for an integrated approach in evidence generation, positioning RWE as pivotal for informed pricing discussions in the US healthcare landscape.

Quantitative bias analysis for external control arms using real-world data in clinical trials: a primer for clinical researchers

Development of medicines in rare oncologic patient populations are growing, but well-powered randomized controlled trials are typically extremely challenging or unethical to conduct in such settings. External control arms using real-world data are increasingly used to supplement clinical trial evidence where no or little control arm data exists. The construction of an external control arm should always aim to match the population, treatment settings and outcome measurements of the corresponding treatment arm. Yet, external real-world data is typically fraught with limitations including missing data, measurement error and the potential for unmeasured confounding given a nonrandomized comparison. Quantitative bias analysis (QBA) comprises a collection of approaches for modelling the magnitude of systematic errors in data which cannot be addressed with conventional statistical adjustment. Their applications can range from simple deterministic equations to complex hierarchical models. QBA applied to external control arm represent an opportunity for evaluating the validity of the corresponding comparative efficacy estimates. We provide a brief overview of available QBA approaches and explore their application in practice. Using a motivating example of a comparison between pralsetinib single-arm trial data versus pembrolizumab alone or combined with chemotherapy real-world data for RET fusion-positive advanced non-small cell lung cancer (aNSCLC) patients (1-2% among all NSCLC), we illustrate how QBA can be applied to external control arms. We illustrate how QBA is used to ascertain robustness of results despite a large proportion of missing data on baseline ECOG performance status and suspicion of unknown confounding. The robustness of findings is illustrated by showing that no meaningful change to the comparative effect was observed across several 'tipping-point' scenario analyses, and by showing that suspicion of unknown confounding was ruled out by use of E-values. Full R code is also provided.

Sensitivity analysis for the interactive effects of internal bias and publication bias in meta-analyses

Meta-analyses can be compromised by studies' internal biases (e.g., confounding in nonrandomized studies) as well as publication bias. These biases often operate nonadditively: publication bias that favors significant, positive results selects indirectly for studies with more internal bias. We propose sensitivity analyses that address two questions: (1) "For a given severity of internal bias across studies and of publication bias, how much could the results change?"; and (2) "For a given severity of publication bias, how severe would internal bias have to be, hypothetically, to attenuate the results to the null or by a given amount?" These methods consider the average internal bias across studies, obviating specifying the bias in each study individually. The analyst can assume that internal bias affects all studies, or alternatively that it only affects a known subset (e.g., nonrandomized studies). The internal bias can be of unknown origin or, for certain types of bias in causal estimates, can be bounded analytically. The analyst can specify the severity of publication bias or, alternatively, consider a "worst-case" form of publication bias. Robust estimation methods accommodate non-normal effects, small meta-analyses, and clustered estimates. As we illustrate by re-analyzing published meta-analyses, the methods can provide insights that are not captured by simply considering each bias in turn. An R package implementing the methods is available (multibiasmeta).

The prognostic value of Eastern Cooperative Oncology Group performance status on overall survival among patients with metastatic prostate cancer: a systematic review and meta-analysis

Background

There is heterogeneity in the literature regarding the strength of association between Eastern Cooperative Oncology Group performance status (ECOG PS) and mortality. We conducted a systematic review and meta-analysis of studies reporting the prognostic value of ECOG PS on overall survival (OS) in metastatic prostate cancer (mPC).

Methods

PubMed was searched from inception to March 21, 2022. A meta-analysis pooling the effect of ECOG PS categories (≥2 vs. <2, 2 vs. <2, and ≥1 vs. <1) on OS was performed separately for studies including patients with metastatic castration-resistant prostate cancer (mCRPC) and metastatic castration-sensitive prostate cancer (mCSPC) using a random-effects model. Analyses were stratified by prior chemotherapy and study type.

Results

Overall, 75 studies, comprising 32,298 patients, were included. Most studies (72/75) included patients with mCRPC. Higher ECOG PS was associated with a significant increase in mortality risk, with the highest estimate observed among patients with mCRPC with an ECOG PS of ≥2 versus <2 (hazard ratio [HR]: 2.10, 95% confidence interval [CI]: 1.87-2.37). When stratifying by study type, there was a higher risk estimate of mortality among patients with mCRPC with an ECOG PS of ≥1 versus <1 in real-world data studies (HR: 1.98, 95% CI: 1.72-2.26) compared with clinical trials (HR: 1.32, 95% CI: 1.13-1.54; p < 0.001). There were no significant differences in the HR of OS stratified by previous chemotherapy.

Conclusion

ECOG PS was a significant predictor of OS regardless of category, previous chemotherapy, and mPC population. Additional studies are needed to better characterize the effect of ECOG PS on OS in mCSPC.

Mediation analysis quantifying the magnitude of stillbirth risk attributable to small for gestational age infants

BACKGROUND

Many risk factors for stillbirth are linked to placental dysfunction, which leads to suboptimal intrauterine growth and small for gestational age infants. Such infants also have an increased risk for stillbirth.

OBJECTIVE

This study aimed to investigate the effect of known causal risk factors for stillbirth, and to identify those that have a large proportion of their risk mediated through small for gestational age birth.

STUDY DESIGN

This retrospective cohort study used data from all births in the state of Queensland, Australia between 2000 and 2018. The total effects of exposures on the odds of stillbirth were determined using multivariable, clustered logistic regression models. Mediation analysis was performed using a counterfactual approach to determine the indirect effect and percentage of effect mediated through small for gestational age. For risk factors significantly mediated through small for gestational age, the relative risks of stillbirth were compared between small for gestational age and appropriate for gestational age infants. We also investigated the proportion of risk mediated via small for gestational age for late stillbirths (≥28 weeks).

RESULTS

The initial data set consisted of 1,105,612 births. After exclusions, the final study cohort constituted 925,053 births. Small for gestational age births occurred in 9.9% (91,859/925,053) of the study cohort. Stillbirths occurred in 0.5% of all births (4234/925,053) and 1.5% of small for gestational age births (1414/91,859). Births at ≥28 weeks occurred in 99.4% (919,650/925,053) of the study cohort and in 98.9% (90,804/91,859) of all small for gestational age births. Of the ≥28-week births, stillbirths occurred in 0.2% (2156/919,650) of all births and 0.8% (677/90,804) of the small for gestational age births. Overall, increased odds of stillbirth were significantly mediated through small for gestational age for age <20 years, low socioeconomic status, Indigenous ethnicity, birth in sub-Saharan and North Africa or the Middle East, smoking, nulliparity, multiple pregnancy, assisted conception, previous stillbirth, preeclampsia, and renal disease. Preeclampsia had the largest proportion mediated through small for gestational age (66.7%), followed by nulliparity (61.6%), smoking (29.4%), North-African or Middle Eastern ethnicity (27.6%), multiple pregnancy (26.3%), low socioeconomic status (25.8%), and Indigenous status (18.7%). Sensitivity analysis showed that for late stillbirths, the portions mediated through small for gestational age remained very similar for many of the risk factors.

CONCLUSION

Although small for gestational age is an important mediator between many pregnancy risk factors and stillbirth, mitigating the risk of small for gestational age is likely to be of value only when it is a major contributor in the pathway to fetal demise.

Periodontitis is associated with multimorbidity in a large dental school population

AIM

To investigate whether and which diseases co-occur with periodontitis (PD) to assess the prevalence of comorbidities and multimorbidity and to identify patterns and profiles of comorbidity and multimorbidity and the influence of demographic and lifestyle factors to identify distinct groups of multimorbid patients.

MATERIALS AND METHODS

A database from the Academic Centre of Dentistry Amsterdam (ACTA) with 37,801 adult individuals containing information about demographic (age, sex, socio-economic position [SEP]) and lifestyle factors (smoking, alcohol use and addictive substance use) and PD and systemic diseases was constructed. PD assessment was based on clinical information by the use of claim codes and systemic diseases data were derived from self-reported medical history. For analyses, univariable and multivariable (adjusted for age, sex, SEP, smoking, alcohol use and addictive substance use) logistic regression analyses and cluster analysis were used.

RESULTS

Individuals with PD more often had one or multiple diseases. The adjusted odds ratio (OR) for PD patients having up to four systemic diseases ranged from 1.46 to 1.20. Co-occurrence of PD with several systemic diseases and a higher prevalence of multimorbidity was found (adjusted OR comorbidity = 1.36; 95% confidence interval (CI): 1.30-1.43; multimorbidity = 1.18; 95% CI: 1.11-1.25). Four clusters existed: cluster 1 was defined as a periodontal and systemically healthy group and cluster 4 as burdened with PD but not containing any systemic diseases. Individuals in cluster 1 were of the lowest age (44.9 [SD: 15.5]) and had the lowest prevalence of the lifestyle factors of smoking (13.6%) and alcohol use (3.9%). Clusters 2 and 3 contained both PD and had several systemic diseases but were different from each other. Cluster 2 contained 34.5% of PD individuals and had mainly respiratory tract, immune system and digestive system diseases. Cluster 3 contained 45.9% of PD individuals and had mainly cardiometabolic diseases. Cluster 2 had the highest prevalence of females (63.1%) and the highest prevalence of smokers (23.8%) and addictive substance users (8.9%). Cluster 3 included individuals of the highest age (63.5 [SD: 11.9]), and had highest prevalence of alcohol users (17.7%) and lowest prevalence of addictive substance users (3.8%).

CONCLUSIONS

This study shows that individuals with PD are more often burdened with comorbidity and multimorbidity. Presence of distinct clusters suggests overlap in pathophysiology between certain types of PD and specific systemic diseases. Therefore, PD can be considered as part of multimorbidity, as one of the systemic diseases co-occurring in certain groups of individuals.

The use of the E-value for sensitivity analysis

Observational research designs enable clinicians to investigate topics for which randomized-controlled trials may be difficult to conduct. However, the lack of randomization in observational studies increases the likelihood of confounders introducing bias to study results. Analytical methods such as propensity score matching and regression analysis are employed to reduce the effects of such confounding, mainly by determining characteristics of patient groups and adjusting for measured confounders. Sensitivity analyses are subsequently applied to elucidate the extent to which study results could still be affected by unmeasured confounding. The E-value is one such approach. By presenting a value that quantifies the strength of unmeasured confounding necessary to negate the observed results, the E-value is a useful heuristic concept for assessing the robustness of observational studies. This article provides an introductory overview of how the E-value can be evaluated and presented in clinical research studies.

The effects of long-term exposure to air pollution on incident mental disorders among patients with prediabetes and diabetes: Findings from a large prospective cohort

BACKGROUND

The association between air pollution and mental disorders has been widely documented in the general population. However, the evidence among susceptible populations, such as individuals with prediabetes or diabetes, is still insufficient.

METHODS

We analyzed data from 48,515 participants with prediabetes and 24,393 participants with diabetes from the UK Biobank. Annual pollution data were collected for fine particulate matter (PM2.5), inhalable particulate matter (PM10), nitrogen dioxide (NO2), and nitrogen dioxides (NOx) during 2006-2021. The exposure to air pollution and temperature for each participant were estimated by the bilinear interpolation approach and time-weighted method based on their geocoded home addresses and time spent at each address. We employed the generalized propensity score model based on the generalized estimating equation and the time-varying covariates Cox model to assess the effects of air pollution.

RESULTS

We observed causal links between air pollutants and mental disorders among both prediabetic and diabetic participants, with stronger effects among those with diabetes than prediabetes. The hazard ratios were 1.18 (1.12, 1.24), 1.15 (1.10, 1.20), 1.18 (1.13, 1.23), and 1.15 (1.11, 1.19) in patients with prediabetes, and 1.21 (1.13, 1.29), 1.17 (1.11, 1.24), 1.19 (1.13, 1.25), and 1.17 (1.12, 1.23) in patients with diabetes per interquartile range elevation in PM2.5, PM10, NO2, and NOx. Furthermore, the effects were more pronounced among people who were older, alcohol drinkers, and living in urban areas.

CONCLUSIONS

Our study indicates the potential causal links between long-term exposure to air pollution and incident mental disorders among those with prediabetes and diabetes. Reducing air pollution levels would significantly benefit this vulnerable population by reducing the incidence of mental disorders.

Causal feature selection using a knowledge graph combining structured knowledge from the biomedical literature and ontologies: A use case studying depression as a risk factor for Alzheimer's disease

BACKGROUND

Causal feature selection is essential for estimating effects from observational data. Identifying confounders is a crucial step in this process. Traditionally, researchers employ content-matter expertise and literature review to identify confounders. Uncontrolled confounding from unidentified confounders threatens validity, conditioning on intermediate variables (mediators) weakens estimates, and conditioning on common effects (colliders) induces bias. Additionally, without special treatment, erroneous conditioning on variables combining roles introduces bias. However, the vast literature is growing exponentially, making it infeasible to assimilate this knowledge. To address these challenges, we introduce a novel knowledge graph (KG) application enabling causal feature selection by combining computable literature-derived knowledge with biomedical ontologies. We present a use case of our approach specifying a causal model for estimating the total causal effect of depression on the risk of developing Alzheimer's disease (AD) from observational data.

METHODS

We extracted computable knowledge from a literature corpus using three machine reading systems and inferred missing knowledge using logical closure operations. Using a KG framework, we mapped the output to target terminologies and combined it with ontology-grounded resources. We translated epidemiological definitions of confounder, collider, and mediator into queries for searching the KG and summarized the roles played by the identified variables. We compared the results with output from a complementary method and published observational studies and examined a selection of confounding and combined role variables in-depth.

RESULTS

Our search identified 128 confounders, including 58 phenotypes, 47 drugs, 35 genes, 23 collider, and 16 mediator phenotypes. However, only 31 of the 58 confounder phenotypes were found to behave exclusively as confounders, while the remaining 27 phenotypes played other roles. Obstructive sleep apnea emerged as a potential novel confounder for depression and AD. Anemia exemplified a variable playing combined roles.

CONCLUSION

Our findings suggest combining machine reading and KG could augment human expertise for causal feature selection. However, the complexity of causal feature selection for depression with AD highlights the need for standardized field-specific databases of causal variables. Further work is needed to optimize KG search and transform the output for human consumption.

Are Observational Studies on Distal Radius Fracture Treatment Robust? An E-value Approach to Analysis

BACKGROUND

Reported complication frequencies after distal radius fracture (DRF) treatment vary widely in the literature and are based mostly on observational evidence. Whether that evidence is sufficiently robust to use in practice is controversial. The E-value is an innovative sensitivity analysis that quantitates the robustness of observational evidence against unmeasured confounders, whereby a greater E-value usually implies more robust evidence and vice versa; with DRF complications, this approach can help guide readers to a more confident interpretation of the available evidence.

QUESTIONS/PURPOSES

In this study, we sought (1) to compare the complication frequencies among different DRF treatment modalities, and (2) to evaluate the robustness of these observational studies using the E-value as an index for unmeasured confounding.

METHODS

We searched PubMed, Embase, and SCOPUS for observational studies on the management of DRFs that were published from January 2001 to July 2021 with the last database search performed on July 31, 2021. All articles that compared different DRF treatment modalities with reported complication frequencies were included to accurately capture the quality of the observational studies in research about DRF. Risk ratios (RRs) of the overall complication and major complication risks were calculated for each subgroup comparison: volar plating versus dorsal plating, casting, external fixation, and percutaneous K-wire fixation. The RRs and their corresponding lower limits of the 95% confidence intervals (CIs) were used to derive the E-values. E-values can have a minimum possible value of 1, which signifies that the treatment-outcome association is not strong and can readily be overturned by unmeasured confounders. By contrast, a large E-value means that the observed treatment-outcome association is robust against unmeasured confounders. We averaged RRs and E-values for the effect estimates and lower limits of CIs across studies in each treatment comparison group. We identified 36 comparative observational studies that met the inclusion criteria. Seven studies compared volar with dorsal plating techniques. Volar plating was also compared with casting (eight studies), external fixation (15 studies), and percutaneous K-wire fixation (six studies).

RESULTS

Total and major complication risks did not differ among different DRF treatments. The mean RRs for total and major complications were 1.2 (95% CI 0.4 to 3.9; p = 0.74) and 1.8 (95% CI 0.4 to 11.4; p = 0.52) for the volar versus dorsal plating group; 1.2 (95% CI 0.3 to 11.2; p = 0.87) and 1.5 (95% CI 0.3 to 14.9; p = 0.74) for the volar plating versus casting group; 0.6 (95% CI 0.2 to 2.2; p = 0.33) and 0.8 (95% CI 0.2 to 6.7; p = 0.86) for the volar plating versus external fixation group; and 0.6 (95% CI 0.2 to 2.6; p = 0.47) and 0.7 (95% CI 0.2 to 4.0; p = 0.67) for the volar plating versus K-wire fixation group. The mean E-values for total and major complication frequencies for the between-group comparison ranged from 3.1 to 5.8; these were relatively large in the context of a known complication risk factor, such as high-energy impact (RR 3.2), suggesting a reasonable level of robustness against unmeasured confounding. However, the E-values for lower limits of CIs remained close to 1, which indicates the observed complication frequencies in these studies were likely to have been influenced by unmeasured confounders.

CONCLUSION

Complication frequencies did not differ among different DRF treatment modalities, but the observed complication frequencies from most comparative observational studies were less robust against potential unmeasured confounders. The E-value method, or another type of sensitivity analysis, should be implemented in observational hand surgery research at the individual-study level to facilitate assessment of robustness against potential unmeasured confounders.

LEVEL OF EVIDENCE

Level III, therapeutic study.

The M-Value: A Simple Sensitivity Analysis for Bias Due to Missing Data in Treatment Effect Estimates

Complete-case analyses can be biased if missing data are not missing completely at random. We propose simple sensitivity analyses that apply to complete-case estimates of treatment effects; these analyses use only simple summary data and obviate specifying the precise mechanism of missingness and making distributional assumptions. Bias arises when treatment effects differ between retained and nonretained participants or, among retained participants, the estimate is biased because conditioning on retention has induced a noncausal path between the treatment and outcome. We thus bound the overall treatment effect on the difference scale by specifying: 1) the unobserved treatment effect among nonretained participants; and 2) the strengths of association that unobserved variables have with the exposure and with the outcome among retained participants ("induced confounding associations"). Working with the former sensitivity parameter subsumes certain existing methods of worst-case imputation while also accommodating less-conservative assumptions (e.g., that the treatment is not detrimental on average even among nonretained participants). As an analog to the E-value for confounding, we propose the M-value, which represents, for a specified treatment effect among nonretained participants, the strength of induced confounding associations required to reduce the treatment effect to the null or to any other value. These methods could help characterize the robustness of complete-case analyses to potential bias due to missing data.

Assessing the Most Vulnerable Subgroup to Type II Diabetes Associated with Statin Usage: Evidence from Electronic Health Record Data

There have been increased concerns that the use of statins, one of the most commonly prescribed drugs for treating coronary artery disease, is potentially associated with the increased risk of new-onset Type II diabetes (T2D). Nevertheless, to date, there is no robust evidence supporting as to whether and what kind of populations are indeed vulnerable for developing T2D after taking statins. In this case study, leveraging the biobank and electronic health record data in the Partner Health System, we introduce a new data analysis pipeline and a novel statistical methodology that address existing limitations by (i) designing a rigorous causal framework that systematically examines the causal effects of statin usage on T2D risk in observational data, (ii) uncovering which patient subgroup is most vulnerable for developing T2D after taking statins, and (iii) assessing the replicability and statistical significance of the most vulnerable subgroup via a bootstrap calibration procedure. Our proposed approach delivers asymptotically sharp confidence intervals and debiased estimate for the treatment effect of the most vulnerable subgroup in the presence of high-dimensional covariates. With our proposed approach, we find that females with high T2D genetic risk are at the highest risk of developing T2D due to statin usage.

Preoperative Anemia and Risk of In-hospital Postoperative Complications in Patients with Hip Fracture

Purpose

To evaluate the impact of preoperative anemia on postoperative complications after hip fracture surgery.

Patients and Methods

We conducted a retrospective study including hip fracture patients at a teaching hospital between 2005 and 2022. We defined preoperative anemia as the last hemoglobin measurement level before surgery < 130 g/L for men and < 120 g/L for women. The primary outcome was a composite of in-hospital major complications, including pneumonia, respiratory failure, gastrointestinal bleeding, urinary tract infection, incision infection, deep venous thrombosis, pulmonary embolism, angina pectoris, arrhythmia, myocardial infarction, heart failure, stroke, and death. Secondary outcomes were cardiovascular events, infection, pneumonia, and death. We used multivariate negative binomial or logistic regression to evaluate the impact of anemia and its severity, categorized as mild (90-130 g/L for men, 90-120 g/L for women) or moderate-to-severe (< 90 g/L for both) anemia on outcomes.

Results

Of the 3540 included patients, 1960 had preoperative anemia. 188 anemic patients experienced 324 major complications, while 63 non-anemic patients had 94 major complications. The risk of major complications was 165.3 (95% CI, 149.5-182.4) and 59.5 (95% CI, 48.9-72.3) per 1000 persons in anemic and non-anemic patients, respectively. Anemic patients were more likely to have major complications than non-anemic patients (adjusted incidence rate ratio (aIRR), 1.87; 95% CI, 1.30-2.72), which was consistent in mild (aIRR, 1.77; 95% CI, 1.22-2.59) and moderate-to-severe (aIRR, 2.97; 95% CI, 1.65-5.38) anemia. Preoperative anemia also increased the risk of cardiovascular events (aIRR, 1.96; 95% CI, 1.29-3.01), infection (aIRR, 1.68; 95% CI, 1.01-2.86), pneumonia (adjusted odds ratio (aOR), 1.91; 95% CI, 1.06-3.57), and death (aOR, 3.17; 95% CI, 1.06-11.89).

Conclusion

Our findings suggest that even mild preoperative anemia is associated with major postoperative complications in hip fracture patients. This finding highlights considering preoperative anemia as a risk factor in surgical decision-making for high-risk patients.

Sensitivity Analyses for Unmeasured Confounders

Purpose of Review

This review expands on sensitivity analyses for unmeasured confounding techniques, demonstrating state-of-the-art methods as well as specifying which should be used under various scenarios, depending on the information about a potential unmeasured confounder available to the researcher.

Recent Findings

Methods to assess how sensitive an observed estimate is to unmeasured confounding have been developed for decades. Recent advancements have allowed for the incorporation of measured confounders in these assessments, updating the methods used to quantify the impact of an unmeasured confounder, whether specified in terms of the magnitude of the effect from a regression standpoint, for example, as a risk ratio, or with respect to the percent of variation in the outcome or exposure explained by the unmeasured confounder. Additionally, single number summaries, such as the E-value or robustness value, have been proposed to allow for ease of computation when less is known about a specific potential unmeasured confounder.

Summary

This paper aimed to provide methods and tools to implement sensitivity to unmeasured confounder analyses appropriate for various research settings depending on what is known or assumed about a potential unmeasured confounder. We have provided mathematical justification, recommendations, as well as R code to ease the implementation of these methods.

Recommendations for the use of propensity score methods in multiple sclerosis research

BACKGROUND

With many disease-modifying therapies currently approved for the management of multiple sclerosis, there is a growing need to evaluate the comparative effectiveness and safety of those therapies from real-world data sources. Propensity score methods have recently gained popularity in multiple sclerosis research to generate real-world evidence. Recent evidence suggests, however, that the conduct and reporting of propensity score analyses are often suboptimal in multiple sclerosis studies.

OBJECTIVES

To provide practical guidance to clinicians and researchers on the use of propensity score methods within the context of multiple sclerosis research.

METHODS

We summarize recommendations on the use of propensity score matching and weighting based on the current methodological literature, and provide examples of good practice.

RESULTS

Step-by-step recommendations are presented, starting with covariate selection and propensity score estimation, followed by guidance on the assessment of covariate balance and implementation of propensity score matching and weighting. Finally, we focus on treatment effect estimation and sensitivity analyses.

CONCLUSION

This comprehensive set of recommendations highlights key elements that require careful attention when using propensity score methods.

Machine learning for improving high-dimensional proxy confounder adjustment in healthcare database studies: an overview of the current literature

Controlling for large numbers of variables that collectively serve as 'proxies' for unmeasured factors can often improve confounding control in pharmacoepidemiologic studies utilizing administrative healthcare databases. There is a growing body of evidence showing that data-driven machine learning algorithms for high-dimensional proxy confounder adjustment can supplement investigator-specified variables to improve confounding control compared to adjustment based on investigator-specified variables alone. Consequently, there has been a recent focus on the development of data-driven methods for high-dimensional proxy confounder adjustment. In this paper, we discuss the considerations underpinning three areas for data-driven high-dimensional proxy confounder adjustment: 1) feature generation-transforming raw data into covariates (or features) to be used for proxy adjustment; 2) covariate prioritization, selection and adjustment; and 3) diagnostic assessment. We survey current approaches and recent advancements within each area, including the most widely used approach to proxy confounder adjustment in healthcare database studies (the high-dimensional propensity score or hdPS). We also discuss limitations of the hdPS and outline recent advancements that incorporate the principles of proxy adjustment with machine learning extensions to improve performance. We further discuss challenges and avenues of future development within each area. This manuscript is endorsed by the International Society for Pharmacoepidemiology (ISPE). This article is protected by copyright. All rights reserved.

Causal effects of cingulate morphology on executive functions in healthy young adults

In this study, we want to explore evidence for the causal relationship between the anatomical descriptors of the cingulate cortex (surface area, mean curvature-corrected thickness, and volume) and the performance of cognitive tasks such as Card Sort, Flanker, List Sort used as instruments to measure the executive functions of flexibility, inhibitory control, and working memory. We have performed this analysis in a cross-sectional sample of 899 healthy young subjects of the Human Connectome Project. To the best of our knowledge, this is the first study using causal inference to explain the relationship between cingulate morphology and the performance of executive tasks in healthy subjects. We have tested the causal model under a counterfactual framework using stabilized inverse probability of treatment weighting and marginal structural models. The results showed that the posterior cingulate surface area has a positive causal effect on inhibition (Flanker task) and cognitive flexibility (Card Sort). A unit increase (+1 mm² ) in the posterior cingulate surface area will cause a 0.008% and 0.009% increase from the National Institute of Health (NIH) normative mean in Flankers (p-value <0.001), and Card Sort (p-value 0.005), respectively. Furthermore, a unit increase (+1 mm² ) in the anterior cingulate surface area will cause a 0.004% (p-value <0.001) and 0.005% (p-value 0.001) increase from the NIH normative mean in Flankers and Card Sort. In contrast, the curvature-corrected-mean thickness only showed an association for anterior cingulate with List Sort (p = 0.034) but no causal effect.

E-values for effect heterogeneity and approximations for causal interaction

Background

Estimates of effect heterogeneity (i.e. the extent to which the causal effect of one exposure varies across strata of a second exposure) can be biased if the exposure–outcome relationship is subject to uncontrolled confounding whose severity differs across strata of the second exposure.

Methods

We propose methods, analogous to the E-value for total effects, that help to assess the sensitivity of effect heterogeneity estimates to possible uncontrolled confounding. These E-value analogues characterize the severity of uncontrolled confounding strengths that would be required, hypothetically, to ‘explain away’ an estimate of multiplicative or additive effect heterogeneity in the sense that appropriately controlling for those confounder(s) would have shifted the effect heterogeneity estimate to the null, or alternatively would have shifted its confidence interval to include the null. One can also consider shifting the estimate or confidence interval to an arbitrary non-null value. All of these E-values can be obtained using the R package EValue.

Results

We illustrate applying the proposed E-value analogues to studies on: (i) effect heterogeneity by sex of the effect of educational attainment on dementia incidence and (ii) effect heterogeneity by age on the effect of obesity on all-cause mortality.

Conclusion

Reporting these proposed E-values could help characterize the robustness of effect heterogeneity estimates to potential uncontrolled confounding.

Leveraging vibration of effects analysis for robust discovery in observational biomedical data science

Hypothesis generation in observational, biomedical data science often starts with computing an association or identifying the statistical relationship between a dependent and an independent variable. However, the outcome of this process depends fundamentally on modeling strategy, with differing strategies generating what can be called "vibration of effects" (VoE). VoE is defined by variation in associations that often lead to contradictory results. Here, we present a computational tool capable of modeling VoE in biomedical data by fitting millions of different models and comparing their output. We execute a VoE analysis on a series of widely reported associations (e.g., carrot intake associated with eyesight) with an extended additional focus on lifestyle exposures (e.g., physical activity) and components of the Framingham Risk Score for cardiovascular health (e.g., blood pressure). We leveraged our tool for potential confounder identification, investigating what adjusting variables are responsible for conflicting models. We propose modeling VoE as a critical step in navigating discovery in observational data, discerning robust associations, and cataloging adjusting variables that impact model output.

The Importance of Making Assumptions in Bias Analysis

Quantitative bias analyses allow researchers to adjust for uncontrolled confounding, given specification of certain bias parameters. When researchers are concerned about unknown confounders, plausible values for these bias parameters will be difficult to specify. Ding and VanderWeele developed bounding factor and E-value approaches that require the user to specify only some of the bias parameters. We describe the mathematical meaning of bounding factors and E-values and the plausibility of these methods in an applied context. We encourage researchers to pay particular attention to the assumption made, when using E-values, that the prevalence of the uncontrolled confounder among the exposed is 100% (or, equivalently, the prevalence of the exposure among those without the confounder is 0%). We contrast methods that attempt to bound biases or effects and alternative approaches such as quantitative bias analysis. We provide an example where failure to make this distinction led to erroneous statements. If the primary concern in an analysis is with known but unmeasured potential confounders, then E-values are not needed and may be misleading. In cases where the concern is with unknown confounders, the E-value assumption of an extreme possible prevalence of the confounder limits its practical utility.

Bias Analysis Gone Bad

Quantitative bias analysis comprises the tools used to estimate the direction, magnitude, and uncertainty from systematic errors affecting epidemiologic research. Despite the availability of methods and tools, and guidance for good practices, few reports of epidemiologic research incorporate quantitative estimates of bias impacts. The lack of familiarity with bias analysis allows for the possibility of misuse, which is likely most often unintentional but could occasionally include intentional efforts to mislead. We identified 3 examples of suboptimal bias analysis, one for each common bias. For each, we describe the original research and its bias analysis, compare the bias analysis with good practices, and describe how the bias analysis and research findings might have been improved. We assert no motive to the suboptimal bias analysis by the original authors. Common shortcomings in the examples were lack of a clear bias model, computed example, and computing code; poor selection of the values assigned to the bias model's parameters; and little effort to understand the range of uncertainty associated with the bias. Until bias analysis becomes more common, community expectations for the presentation, explanation, and interpretation of bias analyses will remain unstable. Attention to good practices should improve quality, avoid errors, and discourage manipulation.

On the use of the E-value for sensitivity analysis in epidemiologic studies

This study illustrates the use of a recently developed sensitivity index, the E-value, helpful in strengthening causal inferences in observational epidemiological studies. The E-value aims to determine the minimum required strength of association between an unmeasured confounder and an exposure/outcome to explain the observed association as non-causal. Such parameter is defined as E - v a l u e = R R + R R R R - 1 , where RR is the risk ratio between the exposure and the outcome. Our work illustrates the E-value using observational data from a recently published study on the relationship between indicators of prenatal care adequacy and the outcome low birthweight. The E-value ranged between 1.45 and 5.63 according to the category and prenatal care index evaluated, showing the highest value for the "no prenatal care" category of the GINDEX index and the minimum value for "intermediate prenatal care" of the APNCU index. For "inappropriate prenatal care" (all indexes), the E-value ranged between 2.76 (GINDEX) and 4.99 (APNCU). These findings indicate that only strong confounder/low birthweight associations (more than 400% increased risk) would be able to fully explain the prenatal care vs. low birthweight association observed. The E-value is a useful, intuitive sensitivity analysis tool that may help strengthening causal inferences in epidemiological observational studies.

The association between silica exposure, silicosis and tuberculosis: a systematic review and meta-analysis

BACKGROUND

While the association between occupational inhalation of silica dust and pulmonary tuberculosis has been known for over a century, there has never been a published systematic review, particularly of experience in the current era of less severe silicosis and treatable tuberculosis. We undertook a systematic review of the evidence for the association between (1) silicosis and pulmonary tuberculosis, and (2) silica exposure and pulmonary tuberculosis controlling for silicosis, and their respective exposure-response gradients.

METHODS

We searched PUBMED and EMBASE, and selected studies according to a priori inclusion criteria. We extracted, summarised and pooled the results of published case-control and cohort studies of silica exposure and/or silicosis and incident active tuberculosis. Study quality was assessed on the Newcastle-Ottawa Scale. Where meta-analysis was possible, effect estimates were pooled using inverse-variance weighted random-effects models. Otherwise narrative and graphic synthesis was undertaken. Confidence regarding overall effect estimates was assessed using the GRADE schema.

RESULTS

Nine studies met the inclusion criteria. Meta-analysis of eight studies of silicosis and tuberculosis yielded a pooled relative risk of 4.01 (95% confidence interval (CI) 2.88, 5.58). Exposure-response gradients were strong with a low silicosis severity threshold for increased risk. Our GRADE assessment was high confidence in a strong association. Meta-analysis of five studies of silica exposure controlling for or excluding silicosis yielded a pooled relative risk of 1.92 (95% CI 1.36, 2.73). Exposure-response gradients were observable in individual studies but not finely stratified enough to infer an exposure threshold. Our GRADE assessment was low confidence in the estimated effect owing to inconsistency and use of proxies for silica exposure.

CONCLUSIONS

The evidence is robust for a strongly elevated risk of tuberculosis with radiological silicosis, with a low disease severity threshold. The effect estimate is more uncertain for silica exposure without radiological silicosis. Research is needed, particularly cohort studies measuring silica exposure in different settings, to characterise the effect more accurately as well as the silica exposure threshold that could be used to prevent excess tuberculosis risk.

Pharmacoepidemiology for nephrologists (part 2): potential biases and how to overcome them

Observational pharmacoepidemiological studies using routinely collected healthcare data are increasingly being used in the field of nephrology to answer questions on the effectiveness and safety of medications. This review discusses a number of biases that may arise in such studies and proposes solutions to minimize them during the design or statistical analysis phase. We first describe designs to handle confounding by indication (e.g. active comparator design) and methods to investigate the influence of unmeasured confounding, such as the E-value, the use of negative control outcomes and control cohorts. We next discuss prevalent user and immortal time biases in pharmacoepidemiology research and how these can be prevented by focussing on incident users and applying either landmarking, using a time-varying exposure, or the cloning, censoring and weighting method. Lastly, we briefly discuss the common issues with missing data and misclassification bias. When these biases are properly accounted for, pharmacoepidemiological observational studies can provide valuable information for clinical practice.

Confounding and effect measure modification in reproductive medicine research

The majority of research within reproductive and gynecologic health, or investigating ART, is observational in design. One of the most critical challenges for observational studies is confounding, while one of the most important for discovery and inference is effect modification. In this commentary, we explain what confounding and effect modification are and why they matter. We present examples illustrating how failing to adjust for a confounder leads to invalid conclusions, as well as examples where adjusting for a factor that is not a confounder also leads to invalid or imprecise conclusions. Careful consideration of which factors may act as confounders or modifiers of the association of interest is critical to conducting sound research, particularly with complex observational studies in reproductive medicine.

Use of E-values for addressing confounding in observational studies-an empirical assessment of the literature

BACKGROUND

E-values are a recently introduced approach to evaluate confounding in observational studies. We aimed to empirically assess the current use of E-values in published literature.

METHODS

We conducted a systematic literature search for all publications, published up till the end of 2018, which cited at least one of two inceptive E-value papers and presented E-values for original data. For these case publications we identified control publications, matched by journal and issue, where the authors had not calculated E-values.

RESULTS

In total, 87 papers presented 516 E-values. Of the 87 papers, 14 concluded that residual confounding likely threatens at least some of the main conclusions. Seven of these 14 named potential uncontrolled confounders. 19 of 87 papers related E-value magnitudes to expected strengths of field-specific confounders. The median E-value was 1.88, 1.82, and 2.02 for the 43, 348, and 125 E-values where confounding was felt likely to affect the results, unlikely to affect the results, or not commented upon, respectively. The 69 case-control publication pairs dealt with effect sizes of similar magnitude. Of 69 control publications, 52 did not comment on unmeasured confounding and 44/69 case publications concluded that confounding was unlikely to affect study conclusions.

CONCLUSIONS

Few papers using E-values conclude that confounding threatens their results, and their E-values overlap in magnitude with those of papers acknowledging susceptibility to confounding. Facile automation in calculating E-values may compound the already poor handling of confounding. E-values should not be a substitute for careful consideration of potential sources of unmeasured confounding. If used, they should be interpreted in the context of expected confounding in specific fields.

Modest Association of Long-Term ACE Inhibitor Treatment With Lung Cancer: The Promise and Pitfalls of Epidemiological Drug-Safety Analyses

Results of the carefully executed Evaluation of Treatment with Angiotensin Converting Enzyme Inhibitors and the Risk of Lung Cancer (ERACER) study, reported in this issue, echo those of several previous observational analyses of the association of long-term angiotensin-converting enzyme (ACE) inhibitor use with incident lung cancer. These epidemiological drug-safety analyses merit cautious interpretation. First, the number needed to harm (NNH) of 6667 reported in ERACER for ACE inhibitors compared with angiotensin-2 receptor blockers (ARBs) after approximately 12 years of follow-up should be balanced against therapeutic benefits. Previously reported meta-analyses of randomized controlled trials (RCTs) over a mean 4.3-year follow-up suggested number needed to treat (NNT) of 67 for all-cause mortality, 116 for cardiovascular mortality, and 86 for a composite of myocardial infarction (MI) and stroke for ACE inhibitors, compared with nonsignificant benefits for ARBs on the mortality outcomes and NNT of 157 for ARBs on the MI/stroke composite. Second, confounding by indication is possible because until 2013, ACE inhibitors, not ARBs, were first-line medications for heart failure, which is associated with incident lung cancer. Third, findings may be compromised by detection bias due to investigation of ACE inhibitor-induced cough, or by residual confounding due to influential factors not measurable in the available data, such as socioeconomic status (SES) or smoking history. The important questions raised by ERACER and similar drug-safety analyses should be addressed in long-term RCTs or in enhanced large-database pharmacoepidemiological analyses, measuring both NNH and NNT and controlling for SES, indication, medication, and dosage.

Post-tuberculosis airway disease: A population-based cohort study of people immigrating to British Columbia, Canada, 1985-2015

BACKGROUND

Current epidemiological evidence of post-TB airway disease is largely cross-sectional and derived from high-TB-incidence settings. We present the first cohort study of post-TB airway disease in a low-TB-incidence setting.

AIMS

(1) analyze the risk of airway disease by respiratory TB, (2) assess potential unmeasured confounding between TB and airway disease, and (3) investigate TB effect measure modification.

METHODS

A population-based cohort study using healthcare claims data for immigrants to British Columbia (BC), Canada, 1985-2015. Airway disease included chronic airway obstruction, asthma, bronchitis, bronchiolitis, and emphysema. Respiratory TB was defined from TB registry data. Cox proportional hazards (PH) regressions were used to analyze time-to-airway disease by respiratory TB. Sensitivity analyses included varying definitions of TB and airway disease. Potential unmeasured confounding by smoking was evaluated by E-value and hybrid least absolute shrinkage and selection operator (LASSO)-high-dimensional propensity score (hdPS).

FINDINGS

In our cohort (N = 1 005 328; n_TB=1141) there were 116 840 incident cases of airway disease during our 30-year study period (10.43 per 1,000 person-years of follow-up), with cumulative incidence of 42·5% among respiratory TB patients compared with 11·6% among non-TB controls. The covariate-adjusted hazard ratio (aHR) for airway disease by respiratory TB was 2·08 (95% CI: 1·91-2·28) with E-value=3·58. The LASSO-hdPS analysis produced aHR=2·26 (95% CI: 2·07-2·47).

INTERPRETATION

A twofold higher risk of airway disease was observed among immigrants diagnosed with respiratory TB, compared with non-TB controls, in a low-TB-incidence setting. Unmeasured confounding is unlikely to explain this relationship. Models of post-TB care are needed.

FUNDING

Canadian Institutes of Health Research.

Overall Survival Among Chemotherapy-Naive Patients With Castration-Resistant Prostate Cancer Under Abiraterone Versus Enzalutamide: A Direct Comparison Based on a 2014-2018 French Population Study (the SPEAR Cohort)

Abiraterone acetate (ABI) and enzalutamide (ENZ) are considered to be clinically relevant comparators among chemotherapy-naive patients with castration-resistant prostate cancer. No clinical trials comparing overall survival with ABI versus ENZ in a head-to-head approach have been published so far. A few observational studies with low power suggested a potential benefit of ENZ. We used the French National Health Data System to compare overall survival of new users of ABI and ENZ among chemotherapy-naive patients with castration-resistant prostate cancer in 2014-2017, followed through 2018 (the SPEAR cohort, a 2014-2018 cohort study). With an intent-to-treat approach, a survival analysis was performed, estimating hazard ratios for overall survival with the inverse probability weighted Cox model method. Among 10,308 new users, 64% were treated with ABI and 36% with ENZ. The crude mortality rate was 25.2 per 100 person-years (95% confidence interval (CI): 24.4, 26.0) for ABI and 23.7 per 100 person-years (95% CI: 22.6, 24.9) for ENZ. In the weighted analysis, ENZ was associated with better overall survival compared with ABI (hazard ratio = 0.90 (95% CI: 0.85, 0.96) with a median overall survival of 31.7 months for ABI and 34.2 months for ENZ). When restricting to 2015-2017 new users, the effect estimate shifted up to a hazard ratio of 0.93 (95% CI: 0.86, 1.01).

Long-term exposure to NO2 and O3 and all-cause and respiratory mortality: A systematic review and meta-analysis

BACKGROUND

WHO has published several volumes of Global Air Quality Guidelines to provide guidance on the health risks associated with exposure to outdoor air pollution. As new scientific evidence is generated, air quality guidelines need to be periodically revised and, where necessary, updated.

OBJECTIVES

The aims of the study were 1) to summarise the available evidence on the effect of long-term exposure to ozone (O3) and nitrogen dioxide (NO2) on mortality; 2) and to assess concentration response functions (CRF), their shape and the minimum level of exposures measured in studies to support WHO's update of the global air quality guidelines.

DATA SOURCES

We conducted a systematic literature search of the Medline, Embase and Web of Science databases following a protocol proposed by WHO and applied Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines for reporting our results.

STUDY ELIGIBILITY CRITERIA

Cohort studies in human populations (including sub-groups at risk) exposed to long-term concentrations of NO2 and O3. Outcomes assessed were all-cause, respiratory, Chronic Obstructive Pulmonary Disease (COPD) and Acute Lower Respiratory Infection (ALRI) mortality.

STUDY APPRAISAL AND SYNTHESIS METHODS

Studies included in the meta-analyses were assessed using a new Risk of Bias instrument developed by a group of experts convened by WHO. Study results are presented in forest plots and quantitative meta-analyses were conducted using random effects models. The certainty of evidence was assessed using a newly developed adaptation of GRADE.

RESULTS

The review identified 2068 studies of which 95 were subject to full-text review with 45 meeting the inclusion criteria. An update in September 2018 identified 159 studies with 1 meeting the inclusion criteria. Of the 46 included studies, 41 reported results for NO2 and 20 for O3. The majority of studies were from the USA and Europe with the remainder from Canada, China and Japan. Forty-two studies reported results for all-cause mortality and 22 for respiratory mortality. Associations for NO2 and mortality were positive; random-effects summary relative risks (RR) were 1.02 (95% CI: 1.01, 1.04), 1.03 (1.00, 1.05), 1.03 (1.01, 1.04) and 1.06 (1.02, 1.10) per 10 μg/m3 for all-cause (24 cohorts), respiratory (15 cohorts), COPD (9 cohorts) and ALRI (5 cohorts) mortality respectively. The review identified high levels of heterogeneity for all causes of death except COPD. A small number of studies investigated the shape of the concentration-response relationship and generally found little evidence to reject the assumption of linearity across the concentration range. Studies of O3 using annual metrics showed the associations with all-cause and respiratory mortality were 0.97 (0.93, 1.02) and 0.99 (0.89, 1.11) per 10 μg/m3 respectively. For studies using peak O3 metrics, the association with all-cause mortality was 1.01 (1.00, 1.02) and for respiratory mortality 1.02 (0.99, 1.05), each per 10 μg/m3. The review identified high levels of heterogeneity. Few studies investigated the shape of the concentration-response relationship. Certainty in the associations (adapted GRADE) with mortality was rated low to moderate for each exposure-outcome pair, except for NO2 and COPD mortality which was rated high.

LIMITATIONS

The substantial heterogeneity for most outcomes in the review requires explanation. The evidence base is limited in terms of the geographical spread of the study populations and, for some outcomes, the small number of independent cohorts for meta-analysis precludes meaningful meta-regression to explore causes of heterogeneity. Relatively few studies assessed specifically the shape of the CRF or multi-pollutant models.

CONCLUSIONS

The short-comings in the existing literature base makes determining the precise nature (magnitude and linearity) of the associations challenging. Certainty of evidence assessments were moderate or low for both NO2 and O3 for all causes of mortality except for NO2 and COPD mortality where the certainty of the evidence was judged as high.

Robustness of the Comparative Observational Evidence Supporting Class I and II Cardiac Surgery Procedures

Background

Current cardiac surgery guidelines give Class I and II recommendations to valve‐sparing root replacement over the Bentall procedure, mitral valve (MV) repair over replacement, and multiple arterial grafting with bilateral internal thoracic artery based on observational evidence. We evaluated the robustness of the observational studies supporting these recommendations using the E value, an index of unmeasured confounding.

Methods and Results

Observational studies cited in the guidelines and in the 3 largest meta‐analyses comparing the procedures were evaluated for statistically significant effect measures. Two E values were calculated: 1 for the effect‐size estimate and 1 for the lower limit of the 95% CI. Thirty‐one observational studies were identified, and E values were computed for 75 effect estimates. The observed effect estimates for improved clinical outcomes with valve‐sparing root replacement versus the Bentall procedure, MV repair versus replacement, and grafting with bilateral internal thoracic artery versus single internal thoracic artery could be explained by an unmeasured confounder that was associated with both the treatment and outcome by a risk ratio of more than 16.77, 4.32, and 3.14, respectively. For MV repair versus replacement and grafting with bilateral internal thoracic artery versus single internal thoracic artery, the average E values were lower than the effect sizes of the other measured confounders in 33.3% and 60.9% of the studies, respectively. For valve‐sparing root replacement versus the Bentall procedure, no study reported effect sizes for associations of other covariates with outcomes.

Conclusions

The E values for observational evidence supporting the use of valve‐sparing root replacement, MV repair, and grafting with bilateral internal thoracic artery over the Bentall procedure, MV replacement, and grafting with single internal thoracic artery are relatively low. This suggests that small‐to‐moderate unmeasured confounding could explain most of the observed associations for these procedures.