An empirical evaluation of approximate and exact regression-based causal mediation approaches for a binary outcome and a continuous or a binary mediator for case-control study designs

BACKGROUND

In the causal mediation analysis framework, several parametric regression-based approaches have been introduced in past years for decomposing the total effect of an exposure on a binary outcome into a direct effect and an indirect effect through a target mediator. In this context, a well-known strategy involves specifying a logistic model for the outcome and invoking the rare outcome assumption (ROA) to simplify estimation. Recently, exact estimators for natural direct and indirect effects have been introduced to circumvent the challenges prompted by the ROA. As for the approximate approaches relying on the ROA, these exact approaches cannot be used as is on case-control data where the sampling mechanism depends on the outcome.

METHODS

Considering a continuous or a binary mediator, we empirically compare the approximate and exact approaches using simulated data under various case-control scenarios. An illustration of these approaches on case-control data is provided, where the natural mediation effects of long-term use of oral contraceptives on ovarian cancer, with lifetime number of ovulatory cycles as the mediator, are estimated.

RESULTS

In the simulations, we found few differences between the performances of the approximate and exact approaches when the outcome was rare, both marginally and conditionally on variables. However, the performance of the approximate approaches degraded as the prevalence of the outcome increased in at least one stratum of variables. Differences in behavior were also observed among the approximate approaches. In the data analysis, all studied approaches were in agreement with respect to the natural direct and indirect effects estimates.

CONCLUSIONS

In the case where a violation of the ROA applies or is expected, approximate mediation approaches should be avoided or used with caution, and exact estimators favored.

A simulation study for evaluating the performance of clustering measures in multilevel logistic regression

BACKGROUND

Multilevel logistic regression models are widely used in health sciences research to account for clustering in multilevel data when estimating effects on subject binary outcomes of individual-level and cluster-level covariates. Several measures for quantifying between-cluster heterogeneity have been proposed. This study compared the performance of between-cluster variance based heterogeneity measures (the Intra-class Correlation Coefficient (ICC) and the Median Odds Ratio (MOR)), and cluster-level covariate based heterogeneity measures (the 80% Interval Odds Ratio (IOR-80) and the Sorting Out Index (SOI)).

METHODS

We used several simulation datasets of a two-level logistic regression model to assess the performance of the four clustering measures for a multilevel logistic regression model. We also empirically compared the four measures of cluster variation with an analysis of childhood anemia to investigate the importance of unexplained heterogeneity between communities and community geographic type (rural vs urban) effect in Malawi.

RESULTS

Our findings showed that the estimates of SOI and ICC were generally unbiased with at least 10 clusters and a cluster size of at least 20. On the other hand, estimates of MOR and IOR-80 were less accurate with 50 or fewer clusters regardless of the cluster size. The performance of the four clustering measures improved with increased clusters and cluster size at all cluster variances. In the analysis of childhood anemia, the estimate of the between-community variance was 0.455, and the effect of community geographic type (rural vs urban) had an odds ratio (OR)=1.21 (95% CI: 0.97, 1.52). The resulting estimates of ICC, MOR, IOR-80 and SOI were 0.122 (indicative of low homogeneity of childhood anemia in the same community); 1.898 (indicative of large unexplained heterogeneity); 0.345-3.978 and 56.7% (implying that the between community heterogeneity was more significant in explaining the variations in childhood anemia than the estimated effect of community geographic type (rural vs urban)), respectively.

CONCLUSION

At least 300 clusters with sizes of at least 50 would be adequate to estimate the strength of clustering in multilevel logistic regression with negligible bias. We recommend using the SOI to assess unexplained heterogeneity between clusters when the interest also involves the effect of cluster-level covariates, otherwise, the usual intra-cluster correlation coefficient would suffice in multilevel logistic regression analyses.

Analysis of multicenter clinical trials with very low event rates

INTRODUCTION

In a five-arm randomized clinical trial (RCT) with stratified randomization across 54 sites, we encountered low primary outcome event proportions, resulting in multiple sites with zero events either overall or in one or more study arms. In this paper, we systematically evaluated different statistical methods of accounting for center in settings with low outcome event proportions.

METHODS

We conducted a simulation study and a reanalysis of a completed RCT to compare five popular methods of estimating an odds ratio for multicenter trials with stratified randomization by center: (i) no center adjustment, (ii) random intercept model, (iii) Mantel-Haenszel model, (iv) generalized estimating equation (GEE) with an exchangeable correlation structure, and (v) GEE with small sample correction (GEE-small sample correction). We varied the number of total participants (200, 500, 1000, 5000), number of centers (5, 50, 100), control group outcome percentage (2%, 5%, 10%), true odds ratio (1, > 1), intra-class correlation coefficient (ICC) (0.025, 0.075), and distribution of participants across the centers (balanced, skewed).

RESULTS

Mantel-Haenszel methods generally performed poorly in terms of power and bias and led to the exclusion of participants from the analysis because some centers had no events. Failure to account for center in the analysis generally led to lower power and type I error rates than other methods, particularly with ICC = 0.075. GEE had an inflated type I error rate except in some settings with a large number of centers. GEE-small sample correction maintained the type I error rate at the nominal level but suffered from reduced power and convergence issues in some settings when the number of centers was small. Random intercept models generally performed well in most scenarios, except with a low event rate (i.e., 2% scenario) and small total sample size (n ≤ 500), when all methods had issues.

DISCUSSION

Random intercept models generally performed best across most scenarios. GEE-small sample correction performed well when the number of centers was large. We do not recommend the use of Mantel-Haenszel, GEE, or models that do not account for center. When the expected event rate is low, we suggest that the statistical analysis plan specify an alternative method in the case of non-convergence of the primary method.

Current practice in analysing and reporting binary outcome data-a review of randomised controlled trial reports

BACKGROUND

Randomised controlled trials (RCTs) need to be reported so that their results can be unambiguously and robustly interpreted. Binary outcomes yield unique challenges, as different analytical approaches may produce relative, absolute, or no treatment effects, and results may be particularly sensitive to the assumptions made about missing data. This review of recently published RCTs aimed to identify the methods used to analyse binary primary outcomes, how missing data were handled, and how the results were reported.

METHODS

Systematic review of reports of RCTs published in January 2019 that included a binary primary outcome measure. We identified potentially eligible English language papers on PubMed, without restricting by journal or medical research area. Papers reporting the results from individually randomised, parallel-group RCTs were included.

RESULTS

Two hundred reports of RCTs were included in this review. We found that 64% of the 200 reports used a chi-squared-style test as their primary analytical method. Fifty-five per cent (95% confidence interval 48% to 62%) reported at least one treatment effect measure, and 38% presented only a p value without any treatment effect measure. Missing data were not always adequately described and were most commonly handled using available case analysis (69%) in the 140 studies that reported missing data. Imputation and best/worst-case scenarios were used in 21% of studies. Twelve per cent of articles reported an appropriate sensitivity analysis for missing data.

CONCLUSIONS

The statistical analysis and reporting of treatment effects in reports of randomised trials with a binary primary endpoint requires substantial improvement. Only around half of the studied reports presented a treatment effect measure, hindering the understanding and dissemination of the findings. We also found that published trials often did not clearly describe missing data or sensitivity analyses for these missing data. Practice for secondary endpoints or observational studies may differ.

Approaches for testing noninferiority in two-arm trials for risk ratio and odds ratio

For an existing established drug regimen, active control trials are defacto standard due to ethical reason as well as for clinical equipoise. However, when superiority claim of a new drug against the active control is unlikely to be successful, researchers often address the issue in terms of noninferiority (NI), provided the experimental drug demonstrates the evidence of other benefits beyond efficacy. Such trials aim to demonstrate that an experimental treatment is non-inferior to an existing comparator by not more than a pre-specified margin. The issue of choosing such a margin is complex. In this article, two-arm NI trials with binary outcomes are considered when margin is defined in terms of relative risk or odds ratio. A Frequentist test based on proposed NI margin is developed first. Since two-arm NI trials without placebo arm are dependent upon historical information, in order to make accurate and meaningful interpretation of their results, a Bayesian approach is developed next. Bayesian approach is flexible to incorporate the available information from the historical trial. The operating characteristics of the proposed methods are studied in terms of power and sample size for varying design factors. A clinical trial data is reanalyzed to study the properties of the proposed approach.

A unifying framework for marginalized random intercept models of correlated binary outcomes

We demonstrate that many current approaches for marginal modeling of correlated binary outcomes produce likelihoods that are equivalent to the copula-based models herein. These general copula models of underlying latent threshold random variables yield likelihood-based models for marginal fixed effects estimation and interpretation in the analysis of correlated binary data with exchangeable correlation structures. Moreover, we propose a nomenclature and set of model relationships that substantially elucidates the complex area of marginalized random intercept models for binary data. A diverse collection of didactic mathematical and numerical examples are given to illustrate concepts.

Near/far matching: a study design approach to instrumental variables

Classic instrumental variable techniques involve the use of structural equation modeling or other forms of parameterized modeling. In this paper we use a nonparametric, matching-based instrumental variable methodology that is based on a study design approach. Similar to propensity score matching, though unlike classic instrumental variable approaches, near/far matching is capable of estimating causal effects when the outcome is not continuous. Unlike propensity score matching, though similar to instrumental variable techniques, near/far matching is also capable of estimating causal effects even when unmeasured covariates produce selection bias. We illustrate near/far matching by using Medicare data to compare the effectiveness of carotid arterial stents with cerebral protection versus carotid endarterectomy for the treatment of carotid stenosis.