Conduct and reporting of multivariate network meta-analyses: a scoping review

OBJECTIVES

Combining multivariate and network meta-analysis methods simultaneously in a multivariate network meta-analysis (MVNMA) provides the methodological framework to analyze the largest amount of evidence relevant to decision-makers (i.e., from indirect evidence and correlated outcomes). The objectives of this scoping review were to summarize the characteristics of MVNMAs published in the health sciences literature and map the methodological guidance available for MVNMA.

STUDY DESIGN AND SETTING

We searched MEDLINE, Embase, and the Cumulative Index to Nursing and Allied Health Literature from inception to 28 August 2023, along with citations of included studies, for quantitative evidence syntheses that applied MVNMA and articles addressing MVNMA methods. Pairs of reviewers independently screened potentially eligible studies. Collected data included bibliographic, methodological, and analytical characteristics of included studies. We reported results as total numbers, frequencies, and percentages for categorical variables and medians and interquartile ranges for continuous variables that were not normally distributed.

RESULTS

After screening 1,075 titles and abstracts, and 112 full texts, we included 38 unique studies, of which, 10 were quantitative evidence syntheses that applied MVNMA and 28 were articles addressing MVNMA methods. Among the 10 MVNMAs, the first was published in 2013, four used studies identified from already published systematic reviews, and eight addressed pharmacological interventions, which were the most common interventions. They evaluated interventions for metastatic melanoma, colorectal cancer, prostate cancer, oral hygiene, disruptive behavior disorders, rheumatoid arthritis, narcolepsy, type 2 diabetes, and overactive bladder syndrome. Five MVNMAs analyzed two outcomes simultaneously, and four MVNMAs analyzed three outcomes simultaneously. Among the articles addressing MVNMA methods, the first was published in 2007 and the majority provided methodological frameworks for conducting MVNMAs (26/28, 93%). One study proposed criteria to standardize reporting of MVNMAs and two proposed items relevant to the quality assessment of MVNMAs. Study authors used data from 18 different illnesses to provide illustrative examples within their methodological guidance.

CONCLUSIONS

The application of MVNMA in the health sciences literature is uncommon. Many methodological frameworks are published; however, standardization and specific criteria to guide reporting and quality assessment are lacking. This overview of the current landscape may help inform future conduct of MVNMAs and research on MVNMA methods.

Modeling of correlated cognitive function and functional disability outcomes with bounded and missing data in a longitudinal aging study

Longitudinal studies of correlated cognitive and disability outcomes among older adults are characterized by missing data due to death or loss to follow-up from deteriorating health conditions. The Mini-Mental State Examination (MMSE) score for assessing cognitive function ranges from a minimum of 0 (floor) to a maximum of 30 (ceiling). To study the risk factors of cognitive function and functional disability, we propose a shared parameter model to handle missingness, correlation between outcomes, and the floor and ceiling effects of the MMSE measurements. The shared random effects in the proposed model handle missingness (either missing at random or missing not at random) and correlation between these outcomes, while the Tobit distribution handles the floor and ceiling effects of the MMSE measurements. We used data from the Chinese Longitudinal Healthy Longevity Survey (CLHLS) and a simulation study. By ignoring the MMSE floor and ceiling effects in the analyses of the CLHLS, the association of systolic blood pressure with cognitive function was not significant and the association of age with cognitive function was lower by 16.6% (from -6.237 to -5.201). By ignoring the MMSE floor and ceiling effects in the simulation study, the relative bias in the estimated association of female gender with cognitive function was 43 times higher (from -0.01 to -0.44). The estimated associations obtained with data missing at random were smaller than those with data missing not at random, demonstrating how the missing data mechanism affects the analytic results. Our work underscores the importance of proper model specification in longitudinal analysis of correlated outcomes subject to missingness and bounded values.

Joint modeling of concurrent binary outcomes in a longitudinal observational study using inverse probability of treatment weighting for treatment effect estimation

PURPOSE

Correlated healthcare utilization outcomes may be encoded as binary outcomes in epidemiologic studies. We demonstrate how to account for correlation between concurrent binary outcomes and confounding by person characteristics when estimating a treatment effect in observational studies.

METHODS

We present a joint shared-parameter model, weighted by inverse probability of treatment weights (IPTW) to account for confounding. The model is evaluated in a simulation study that emulates the Medical Expenditure Panel Survey data and compared with a covariate-adjusted joint model and with separate outcome models (IPTW weighted and covariate adjusted).

RESULTS

For the IPTW-weighted joint model, relative bias in the estimated treatment effect on outcome 1 ranged from -0.057 to -0.033 and outcome 2 from -0.077 to -0.043. For the covariate-adjusted joint model, relative bias ranged from -0.010 to -0.083 for outcome 1 and from -0.087 to -0.110 for outcome 2. The covariate-adjusted joint model estimated the effect more closely than the covariate-adjusted separate model. The IPTW-weighted joint model estimated the effect more closely for outcome 1.

CONCLUSIONS

The IPTW-weighted joint model handles correlation between binary outcomes, adjusts for confounding, and estimates the treatment effect accurately in observational studies. We illustrate the contribution of person-specific effects in estimating personalized risk.

Beyond the Primary Endpoint Paradigm: A Test of Intervention Effect in HIV Behavioral Intervention Trials with Numerous Correlated Outcomes

Behavioral interventions are increasingly based on holistic approaches to health with an understanding that health-related behaviors are linked. A motivating example is provided by the Philani study, an intervention trial conducted to improve the health of South African mothers and their children. Inter-related health problems around maternal alcohol use, malnutrition, and HIV were addressed; multiple endpoints were targeted. The traditional hypothesis testing paradigm that tests significance on one primary outcome did not suffice. Past multiple endpoint studies have utilized a sign test on the number of estimated differences between treatment and control that favor the intervention. However, in order to preserve type 1 error, one must account for correlations among the outcomes. We propose an alternative approach that counts the number of significant treatment-control differences. Monte Carlo simulation is used to adjust for correlation, providing updated critical values and p values. Our method is implemented through an R package and applied to the Philani data to test the intervention's overall effect.

A model for meta-analysis of correlated binary outcomes: The case of split-body interventions

In several areas of clinical research, it is common for trials to assign different sites of the participants’ bodies to different interventions. For example, a randomized controlled trial comparing surgical techniques for correcting myopia may randomize each eye of a participant to a different operation. Under such bilateral (‘split-body’) interventions, the observations from each participant are correlated. It is challenging to account for these correlations at the meta-analysis level, especially when the outcome is rare. Here, we present a meta-analysis model based on the bivariate binomial distribution. Our model can synthesize studies on patients who received one intervention at one body site, patients who received two interventions at different sites or a mixture of these two groups. The model can analyse studies with zero events in one or both treatment arms and can handle the case of incomplete data reporting. We use simulations to assess the performance of our model and to compare it with the bivariate beta-binomial model. In the case of bilateral interventions, our model performed well and outperformed the bivariate beta-binomial model in all scenarios explored. We illustrate our methods using two previously published meta-analyses from the fields of orthopaedics and ophthalmology. We conclude that our model constitutes a useful new tool for the meta-analysis of binary outcomes in the presence of split-body interventions.

Accounting for selection and correlation in the analysis of two-stage genome-wide association studies

The problem of selection bias has long been recognized in the analysis of two-stage trials, where promising candidates are selected in stage 1 for confirmatory analysis in stage 2. To efficiently correct for bias, uniformly minimum variance conditionally unbiased estimators (UMVCUEs) have been proposed for a wide variety of trial settings, but where the population parameter estimates are assumed to be independent. We relax this assumption and derive the UMVCUE in the multivariate normal setting with an arbitrary known covariance structure. One area of application is the estimation of odds ratios (ORs) when combining a genome-wide scan with a replication study. Our framework explicitly accounts for correlated single nucleotide polymorphisms, as might occur due to linkage disequilibrium. We illustrate our approach on the measurement of the association between 11 genetic variants and the risk of Crohn's disease, as reported in Parkes and others (2007. Sequence variants in the autophagy gene IRGM and multiple other replicating loci contribute to Crohn's disease susceptibility. Nat. Gen. 39: (7), 830-832.), and show that the estimated ORs can vary substantially if both selection and correlation are taken into account.

Misspecification of the covariance structure in generalized linear mixed models

When fitting marginal models to correlated outcomes, the so-called sandwich variance is commonly used. However, this is not the case when fitting mixed models. Using two data sets, we illustrate the problems that can be encountered. We show that the differences or the ratios between the naive and sandwich standard deviations of the fixed effects estimators provide convenient means of assessing the fit of the model, as both are consistent when the covariance structure is correctly specified, but only the latter is when that structure is misspecified. When the number of statistical units is not too small, the sandwich formula correctly estimates the variance of the fixed effects estimator even if the random effects are misspecified, and it can be used in a diagnostic tool for assessing the misspecification of the random effects. A simple comparison with the naive variance is sufficient and we propose considering a ratio of the naive and sandwich standard deviation out of the [3/4; 4/3] interval as signaling a risk of erroneous inference due to a model misspecification. We strongly advocate broader use of the sandwich variance for statistical inference about the fixed effects in mixed models.