Effect heterogeneity and variable selection for standardizing causal effects to a target population

Bias due to non-consent in assisted reproductive treatment cohort studies: consent for disclosure to non-contact research in the Human Fertilisation and Embryology Authority register

STUDY QUESTION

Is patient consent to research associated with the distribution of population characteristics and study outcomes in ART cohort studies?

SUMMARY ANSWER

The distribution of population characteristics in the patient consent subset differs from that in the non-consent subset and is not fully representative of the general ART population; thus, study results of population subsets requiring patient consent may be subject to bias.

WHAT IS KNOWN ALREADY

Non-consent in epidemiological studies may bias study results if the consent subset differs systematically from the non-consent subset and is thus not representative of the full study population. ART cohort datasets may be biased if they require patients to consent to use their data. As an example, from September 2009 onwards, ART patients in the UK have been asked for specific 'consent to disclosure of identifying information' (CD) for research studies.

STUDY DESIGN, SIZE, DURATION

This cohort study utilized an anonymized version of the Human Fertilisation and Embryology Authority (HFEA) dataset containing all CD and non-CD autologous ART treatment cycles (n = 819 512) conducted from 2004 to 2018 in the UK. A live birth (LB) subset of 155 986 singletons born during the same period was used to analyse child outcomes. Additionally, an aggregated version of the HFEA dataset was used to explore CD trends by clinic type (National Health Service [NHS], private, or both NHS and privately funded).

PARTICIPANTS/MATERIALS, SETTING, METHODS

The dataset containing all gamete cycles was used to explore factors associated with giving CD and to compare LB outcome trends (number of LBs per yearly treatment cycles started) between CD and non-CD cycles. The LB subset was used to compare the birthweight outcomes (low birthweight (LBW = birthweight < 2500 g or otherwise) and macrosomia (birthweight ≥4000 g or otherwise)) between CD and non-CD cycles. Logistic regression models explored the association between CD and population characteristics and the impact of CD on birthweight outcomes over the calendar years. Each regression model was adjusted for potential confounders: for all models (maternal age, ethnicity, previous IVF cycles, previous pregnancies, previous LBs, causes of infertility (tubal, endometriosis, male factor, ovulatory, unknown), and embryo transfer type and stage); for LB and birthweight models (ICSI, elective single embryo transfer, and ovarian stimulation); and additionally for birthweight models (child sex and gestation).

MAIN RESULTS AND THE ROLE OF CHANCE

During the study period, CD rates increased from 16% at its inception in 2009 to 64% in 2018. Fewer cycles from older patients (40-44 years old) and ethnic minorities (Black and Asian) gave CD. Cycles with previous ART treatments and LBs had lower rates of giving CD. CD was also associated with LB rates (higher in the CD group) and LBW (slightly more prevalent in the non-CD group). CD rates were consistently higher in NHS-only funded clinics than in clinics with partly or fully private funding. It may be possible to adjust for much of the post-2009 bias by weighting by the probability of inclusion derived from supplementary data.

LIMITATIONS, REASONS FOR CAUTION

Important factors not provided or unavailable in the dataset included socio-economic and lifestyle factors. Additionally, the anonymized dataset provided to us had banded/categorized maternal age, gestation, and birthweight variables, possibly limiting our estimates' precision.

WIDER IMPLICATIONS OF THE FINDINGS

This study shows that using only consented data in ART observational cohort studies may result in a sample that differs from the non-consented sample and general ART population. Specifically, our results show differences in the distribution of population characteristics, LB, and LBW outcomes between CD and non-CD groups in the UK HFEA ART register dataset. Careful attention is therefore required when analysing and interpreting these and similar cohort data; failure to consider the impact of consent will likely produce misleading results. In the HFEA register, this applies to research studies using CD data (including bespoke data requests and linkage studies) after the introduction of CD in October 2009. A potential solution weighting by the probability of consent is briefly introduced.

STUDY FUNDING/COMPETING INTEREST(S)

This study was funded by the EU H2020 Marie Sklodowska-Curie Innovative Training Networks (ITN) grant Dohartnet (H2020-MSCA-ITN-2018-812660). The authors have no competing interests to declare.

TRIAL REGISTRATION NUMBER

N/A.

Generalizing and Transporting Causal Inferences from Randomized Trials in the Presence of Trial Engagement Effects

Trial engagement effects are effects of trial participation on the outcome that are not mediated by treatment assignment. Most work on extending (generalizing or transporting) causal inferences from a randomized trial to a target population has, explicitly or implicitly, assumed that trial engagement effects are absent, allowing evidence about the effects of the treatments examined in trials to be applied to nonexperimental settings. Here, we define novel causal estimands and present identification results for generalizability and transportability analyses in the presence of trial engagement effects. Our approach allows for trial engagement effects under assumptions of no causal interaction between trial participation and treatment assignment on the absolute or relative scales. We show that under these assumptions, even in the presence of trial engagement effects, the trial data can be combined with covariate data from the target population to identify average treatment effects in the context of usual care as implemented in the target population (i.e., outside the experimental setting). The identifying observed data functionals under these no-interaction assumptions are the same as those obtained under the stronger identifiability conditions that have been invoked in prior work. Therefore, our results suggest a new interpretation for previously proposed generalizability and transportability estimators; this interpretation may be useful in analyses under causal structures where background knowledge suggests that trial engagement effects are present but interactions between trial participation and treatment are negligible.

When, why and how are estimated effects transported between populations? A scoping review of studies applying transportability methods

Transportability methods can improve the external validity of estimated effects by accounting for effect heterogeneity due to differently distributed covariates between populations. This scoping review aims to provide an overview of when, why and how transportability methods have been applied. We systematically searched MEDLINE (Ovid), Embase, Web of Science, EconLit and Google Scholar for studies published between 2010 and December 18, 2024. Studies using transportability methods in a numerical application for at least partly non-overlapping source and target populations were included. We identified 3432 unique studies and included 64 studies applying transportability methods. Over two thirds of the included studies (44/64) introduced new methods. Less than one third of the included studies (20/64) were pure applications of transportability methods. Most applied studies (17/20) transported effect estimates from randomized controlled trials. Effects were transported to target populations with either complete (9/20) or no (9/20) treatment and outcome data or both (2/20). The most frequent aims of applied studies were to transport estimated effects to new populations (10/20) and to assess effect heterogeneity explainable by measured covariates (8/20). How transportability methods were applied varied widely between studies, for instance in the covariate selection approach and sensitivity analysis. Methodological studies with a transportability application presented new transportability estimators for randomized data (5/44), specific transportability applications (e.g., meta-analysis, mediation analysis; 21/44) and other methodological aspects (e.g., covariate selection, missing data handling; 18/44). Transportability methods are a useful tool for knowledge transfer between populations. More applications of transportability methods and guidance for their use are desirable.

Learning about treatment effects in a new target population under transportability assumptions for relative effect measures

Investigators often believe that relative effect measures conditional on covariates, such as risk ratios and mean ratios, are "transportable" across populations. Here, we examine the identification of causal effects in a target population using an assumption that conditional relative effect measures are transportable from a trial to the target population. We show that transportability for relative effect measures is largely incompatible with transportability for difference effect measures, unless the treatment has no effect on average or one is willing to make even stronger transportability assumptions that imply the transportability of both relative and difference effect measures. We then describe how marginal (population-averaged) causal estimands in a target population can be identified under the assumption of transportability of relative effect measures, when we are interested in the effectiveness of a new experimental treatment in a target population where the only treatment in use is the control treatment evaluated in the trial. We extend these results to consider cases where the control treatment evaluated in the trial is only one of the treatments in use in the target population, under an additional partial exchangeability assumption in the target population (i.e., an assumption of no unmeasured confounding in the target population with respect to potential outcomes under the control treatment in the trial). We also develop identification results that allow for the covariates needed for transportability of relative effect measures to be only a small subset of the covariates needed to control confounding in the target population. Last, we propose estimators that can be easily implemented in standard statistical software and illustrate their use using data from a comprehensive cohort study of stable ischemic heart disease.

Selection Bias in Health Research: Quantifying, Eliminating, or Exacerbating Health Disparities?

Purpose of review

To summarize recent literature on selection bias in disparities research addressing either descriptive or causal questions, with examples from dementia research.

Recent findings

Defining a clear estimand, including the target population, is essential to assess whether generalizability bias or collider-stratification bias are threats to inferences. Selection bias in disparities research can result from sampling strategies, differential inclusion pipelines, loss to follow-up, and competing events. If competing events occur, several potentially relevant estimands can be estimated under different assumptions, with different interpretations. The apparent magnitude of a disparity can differ substantially based on the chosen estimand. Both randomized and observational studies may misrepresent health disparities or heterogeneity in treatment effects if they are not based on a known sampling scheme.

Conclusion

Researchers have recently made substantial progress in conceptualization and methods related to selection bias. This progress will improve the relevance of both descriptive and causal health disparities research.

A Causal Framework for Making Individualized Treatment Decisions in Oncology

We discuss how causal diagrams can be used by clinicians to make better individualized treatment decisions. Causal diagrams can distinguish between settings where clinical decisions can rely on a conventional additive regression model fit to data from a historical randomized clinical trial (RCT) to estimate treatment effects and settings where a different approach is needed. This may be because a new patient does not meet the RCT's entry criteria, or a treatment's effect is modified by biomarkers or other variables that act as mediators between treatment and outcome. In some settings, the problem can be addressed simply by including treatment-covariate interaction terms in the statistical regression model used to analyze the RCT dataset. However, if the RCT entry criteria exclude a new patient seen in the clinic, it may be necessary to combine the RCT data with external data from other RCTs, single-arm trials, or preclinical experiments evaluating biological treatment effects. For example, external data may show that treatment effects differ between histological subgroups not recorded in an RCT. A causal diagram may be used to decide whether external observational or experimental data should be obtained and combined with RCT data to compute statistical estimates for making individualized treatment decisions. We use adjuvant treatment of renal cell carcinoma as our motivating example to illustrate how to construct causal diagrams and apply them to guide clinical decisions.

A directed acyclic graph for interactions

BACKGROUND

Directed acyclic graphs (DAGs) are of great help when researchers try to understand the nature of causal relationships and the consequences of conditioning on different variables. One fundamental feature of causal relations that has not been incorporated into the standard DAG framework is interaction, i.e. when the effect of one variable (on a chosen scale) depends on the value that another variable is set to. In this paper, we propose a new type of DAG-the interaction DAG (IDAG), which can be used to understand this phenomenon.

METHODS

The IDAG works like any DAG but instead of including a node for the outcome, it includes a node for a causal effect. We introduce concepts such as confounded interaction and total, direct and indirect interaction, showing that these can be depicted in ways analogous to how similar concepts are depicted in standard DAGs. This also allows for conclusions on which treatment interactions to account for empirically. Moreover, since generalizability can be compromised in the presence of underlying interactions, the framework can be used to illustrate threats to generalizability and to identify variables to account for in order to make results valid for the target population.

CONCLUSIONS

The IDAG allows for a both intuitive and stringent way of illustrating interactions. It helps to distinguish between causal and non-causal mechanisms behind effect variation. Conclusions about how to empirically estimate interactions can be drawn-as well as conclusions about how to achieve generalizability in contexts where interest lies in estimating an overall effect.

Reweighting a Swedish health questionnaire survey using extensive population register and self-reported data for assessing and improving the validity of longitudinal associations

Background

In cohorts with voluntary participation, participants may not be representative of the underlying population, leading to distorted estimates. If the relevant sources of selective participation are observed, it is however possible to restore the representativeness by reweighting the sample to resemble the target population. So far, few studies in epidemiology have applied reweighting based on extensive register data on socio-demographics and disease history, or with self-reported data on health and health-related behaviors.

Methods

We examined selective participation at baseline and the first two follow-ups of the Scania Public Health Cohort (SPHC), a survey conducted in Southern Sweden in 1999/2000 (baseline survey; n = 13,581 participants, 58% participation rate), 2005 (first follow-up, n = 10,471), and 2010 (second follow-up; n = 9,026). Survey participants were reweighted to resemble the underlying population with respect to a broad range of socio-demographic, disease, and health-related characteristics, and we assessed how selective participation impacted the validity of associations between self-reported overall health and dimensions of socio-demographics and health.

Results

Participants in the baseline and follow-up surveys were healthier and more likely to be female, born in Sweden, middle-aged, and have higher socioeconomic status. However, the differences were not very large. In turn, reweighting the samples to match the target population had generally small or moderate impacts on associations. Most examined regression coefficients changed by less than 20%, with virtually no changes in the directions of the effects.

Conclusion

Overall, selective participation with respect to the observed factors was not strong enough to substantially alter the associations with self-assessed health. These results are consistent with an interpretation that SPHC has high validity, perhaps reflective of a relatively high participation rate. Since validity must be determined on a case-by-case basis, however, researchers should apply the same method to other health cohorts to assess and potentially improve the validity.

Generalizing experimental results by leveraging knowledge of mechanisms

We show how experimental results can be generalized across diverse populations by leveraging knowledge of local mechanisms that produce the outcome of interest, only some of which may differ in the target domain. We use structural causal models and a refined version of selection diagrams to represent such knowledge, and to decide whether it entails the invariance of probabilities of causation across populations, which then enables generalization. We further provide: (i) bounds for the target effect when some of these conditions are violated; (ii) new identification results for probabilities of causation and the transported causal effect when trials from multiple source domains are available; as well as (iii) a Bayesian approach for estimating the transported causal effect from finite samples. We illustrate these methods both with simulated data and with a real example that transports the effects of Vitamin A supplementation on childhood mortality across different regions.

Objectives, design and main findings until 2020 from the Rotterdam Study

The Rotterdam Study is an ongoing prospective cohort study that started in 1990 in the city of Rotterdam, The Netherlands. The study aims to unravel etiology, preclinical course, natural history and potential targets for intervention for chronic diseases in mid-life and late-life. The study focuses on cardiovascular, endocrine, hepatic, neurological, ophthalmic, psychiatric, dermatological, otolaryngological, locomotor, and respiratory diseases. As of 2008, 14,926 subjects aged 45 years or over comprise the Rotterdam Study cohort. Since 2016, the cohort is being expanded by persons aged 40 years and over. The findings of the Rotterdam Study have been presented in over 1700 research articles and reports. This article provides an update on the rationale and design of the study. It also presents a summary of the major findings from the preceding 3 years and outlines developments for the coming period.