The role of secondary outcomes in multivariate meta-analysis

The power of integrating data: advancing pain research using meta-analysis

Publications related to pain research have increased significantly in recent years. The abundance of new evidence creates challenges staying up to date with the latest information. A comprehensive understanding of the literature is important for both clinicians and investigators involved in pain research. One commonly used method to combine and analyse data in health care research is meta-analysis. The primary aim of a meta-analysis is to quantitatively synthesise the results of multiple studies focused on the same research question. Meta-analysis is a powerful tool that can be used to advance pain research. However, there are inherent challenges when combining data from multiple sources. There are also numerous models and statistical considerations when undertaking a meta-analysis. This review aims to discuss the planning and preparation for completing a meta-analysis, review commonly used meta-analysis models, and evaluate the clinical implications of meta-analysis in pain research.

Multivariate meta-analysis of multiple outcomes: characteristics and predictors of borrowing of strength from Cochrane reviews

OBJECTIVES

Multivariate meta-analysis allows the joint synthesis of multiple outcomes accounting for their correlation. This enables borrowing of strength (BoS) across outcomes, which may lead to greater efficiency and even different conclusions compared to separate univariate meta-analyses. However, multivariate meta-analysis is complex to apply, so guidance is needed to flag (in advance of analysis) when the approach is most useful.

STUDY DESIGN AND SETTING

We use 43 Cochrane intervention reviews to empirically investigate the characteristics of meta-analysis datasets that are associated with a larger BoS statistic (from 0 to 100%) when applying a bivariate meta-analysis of binary outcomes.

RESULTS

Four characteristics were identified as strongly associated with BoS: the total number of studies, the number of studies with the outcome of interest, the percentage of studies missing the outcome of interest, and the largest absolute within-study correlation. Using these characteristics, we then develop a model for predicting BoS in a new dataset, which is shown to have good performance (an adjusted R² of 50%). Applied examples are used to illustrate the use of the BoS prediction model.

CONCLUSIONS

Cochrane reviewers mainly use univariate meta-analysis methods, but the identified characteristics associated with BoS and our subsequent prediction model for BoS help to flag when a multivariate meta-analysis may also be beneficial in Cochrane reviews with multiple binary outcomes. Extension to non-Cochrane reviews and other outcome types is still required.

Resilience Interventions Conducted in Western and Eastern Countries-A Systematic Review

Previous research has demonstrated the efficacy of psychological interventions to foster resilience. However, little is known about whether the cultural context in which resilience interventions are implemented affects their efficacy on mental health. Studies performed in Western (k = 175) and Eastern countries (k = 46) regarding different aspects of interventions (setting, mode of delivery, target population, underlying theoretical approach, duration, control group design) and their efficacy on resilience, anxiety, depressive symptoms, quality of life, perceived stress, and social support were compared. Interventions in Eastern countries were longer in duration and tended to be more often conducted in group settings with a focus on family caregivers. We found evidence for larger effect sizes of resilience interventions in Eastern countries for improving resilience (standardized mean difference [SMD] = 0.48, 95% confidence interval [CI] 0.28 to 0.67; p < 0.0001; 43 studies; 6248 participants; I2 = 97.4%). Intercultural differences should receive more attention in resilience intervention research. Future studies could directly compare interventions in different cultural contexts to explain possible underlying causes for differences in their efficacy on mental health outcomes.

mHealth Interventions to Reduce Physical Inactivity and Sedentary Behavior in Children and Adolescents: Systematic Review and Meta-analysis of Randomized Controlled Trials

Background

Children and adolescents increasingly do not meet physical activity (PA) recommendations. Hence, insufficient PA (IPA) and sedentary behavior (SB) among children and adolescents are relevant behavior change domains for using individualized mobile health (mHealth) interventions.

Objective

This review and meta-analysis investigated the effectiveness of mHealth interventions on IPA and SB, with a special focus on the age and level of individualization.

Methods

PubMed, Scopus, Web of Science, SPORTDiscus, and Cochrane Library were searched for randomized controlled trials published between January 2000 and March 2021. mHealth interventions for primary prevention in children and adolescents addressing behavior change related to IPA and SB were included. Included studies were compared for content characteristics and methodological quality and summarized narratively. In addition, a meta-analysis with a subsequent exploratory meta-regression examining the moderating effects of age and individualization on overall effectiveness was performed.

Results

On the basis of the inclusion criteria, 1.3% (11/828) of the preliminary identified studies were included in the qualitative synthesis, and 1.2% (10/828) were included in the meta-analysis. Trials included a total of 1515 participants (mean age (11.69, SD 0.788 years; 65% male and 35% female) self-reported (3/11, 27%) or device-measured (8/11, 73%) health data on the duration of SB and IPA for an average of 9.3 (SD 5.6) weeks. Studies with high levels of individualization significantly decreased insufficient PA levels (Cohen d=0.33; 95% CI 0.08-0.58; Z=2.55; P=.01), whereas those with low levels of individualization (Cohen d=−0.06; 95% CI −0.32 to 0.20; Z=0.48; P=.63) or targeting SB (Cohen d=−0.11; 95% CI −0.01 to 0.23; Z=1.73; P=.08) indicated no overall significant effect. The heterogeneity of the studies was moderate to low, and significant subgroup differences were found between trials with high and low levels of individualization (χ21=4.0; P=.04; I2=75.2%). Age as a moderator variable showed a small effect; however, the results were not significant, which might have been because of being underpowered.

Conclusions

Evidence suggests that mHealth interventions for children and adolescents can foster moderate reductions in IPA but not SB. Moreover, individualized mHealth interventions to reduce IPA seem to be more effective for adolescents than for children. Although, to date, only a few mHealth studies have addressed inactive and sedentary young people, and their quality of evidence is moderate, these findings indicate the relevance of individualization on the one hand and the difficulties in reducing SB using mHealth interventions on the other.

Trial Registration

PROSPERO CRD42020209417; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=209417

Copula-Based Estimation Methods for a Common Mean Vector for Bivariate Meta-Analyses

Traditional bivariate meta-analyses adopt the bivariate normal model. As the bivariate normal distribution produces symmetric dependence, it is not flexible enough to describe the true dependence structure of real meta-analyses. As an alternative to the bivariate normal model, recent papers have adopted “copula” models for bivariate meta-analyses. Copulas consist of both symmetric copulas (e.g., the normal copula) and asymmetric copulas (e.g., the Clayton copula). While copula models are promising, there are only a few studies on copula-based bivariate meta-analysis. Therefore, the goal of this article is to fully develop the methodologies and theories of the copula-based bivariate meta-analysis, specifically for estimating the common mean vector. This work is regarded as a generalization of our previous methodological/theoretical studies under the FGM copula to a broad class of copulas. In addition, we develop a new R package, “CommonMean.Copula”, to implement the proposed methods. Simulations are performed to check the proposed methods. Two real dataset are analyzed for illustration, demonstrating the insufficiency of the bivariate normal model.

meta.shrinkage: An R Package for Meta-Analyses for Simultaneously Estimating Individual Means

Meta-analysis is an indispensable tool for synthesizing statistical results obtained from individual studies. Recently, non-Bayesian estimators for individual means were proposed by applying three methods: the James–Stein (JS) shrinkage estimator, isotonic regression estimator, and pretest (PT) estimator. In order to make these methods available to users, we develop a new R package meta.shrinkage. Our package can compute seven estimators (named JS, JS+, RML, RJS, RJS+, PT, and GPT). We introduce this R package along with the usage of the R functions and the “average-min-max” steps for the pool-adjacent violators algorithm. We conduct Monte Carlo simulations to validate the proposed R package to ensure that the package can work properly in a variety of scenarios. We also analyze a data example to show the ability of the R package.

A Meta-Analysis for Simultaneously Estimating Individual Means with Shrinkage, Isotonic Regression and Pretests

Meta-analyses combine the estimators of individual means to estimate the common mean of a population. However, the common mean could be undefined or uninformative in some scenarios where individual means are “ordered” or “sparse”. Hence, assessments of individual means become relevant, rather than the common mean. In this article, we propose simultaneous estimation of individual means using the James–Stein shrinkage estimators, which improve upon individual studies’ estimators. We also propose isotonic regression estimators for ordered means, and pretest estimators for sparse means. We provide theoretical explanations and simulation results demonstrating the superiority of the proposed estimators over the individual studies’ estimators. The proposed methods are illustrated by two datasets: one comes from gastric cancer patients and the other from COVID-19 patients.

Multivariate meta-analysis of critical care meta-analyses: a meta-epidemiological study

Background

Meta-analyses typically consider multiple outcomes and report univariate effect sizes considered as independent. Multivariate meta-analysis (MVMA) incorporates outcome correlation and synthesises direct evidence and related outcome estimates within a single analysis. In a series of meta-analyses from the critically ill literature, the current study contrasts multiple univariate effect estimates and their precision with those derived from MVMA.

Methods

A previous meta-epidemiological study was used to identify meta-analyses with either one or two secondary outcomes providing sufficient detail to structure bivariate or tri-variate MVMA, with mortality as primary outcome. Analysis was performed using a random effects model for both odds ratio (OR) and risk ratio (RR); borrowing of strength (BoS) between multivariate outcome estimates was reported. Estimate comparisons, β coefficients, standard errors (SE) and confidence interval (CI) width, univariate versus multivariate, were performed using Lin’s concordance correlation coefficient (CCC).

Results

In bivariate meta-analyses, for OR (n = 49) and RR (n = 48), there was substantial concordance (≥ 0.69) between estimates; but this was less so for tri-variate meta-analyses for both OR (n = 25; ≥ 0.38) and RR (≥ -0.10; n = 22). A variable change in the multivariate precision of primary mortality outcome estimates compared with univariate was present for both bivariate and tri-variate meta-analyses and for metrics. For second outcomes, precision tended to decrease and CI width increase for bivariate meta-analyses, but was variable in the tri-variate. For third outcomes, precision increased and CI width decreased. In bivariate meta-analyses, OR coefficient significance reversal, univariate versus MVMA, occurred once for mortality and 6 cases for second outcomes. RR coefficient significance reversal occurred in 4 cases; 2 were discordant with OR. For tri-variate OR meta-analyses reversal of coefficient estimate significance occurred in two cases for mortality, nine cases for second and 7 cases for third outcomes. In RR meta-analyses significance reversals occurred for mortality in 2 cases, 6 cases for second and 3 cases for third; there were 7 discordances with OR. BoS was greater in trivariate MVMAs compared with bivariate and for OR versus RR.

Conclusions

MVMA would appear to be the preferred solution to multiple univariate analyses; parameter significance changes may occur. Analytic metric appears to be a determinant.

Classifying information-sharing methods

BACKGROUND

Sparse relative effectiveness evidence is a frequent problem in Health Technology Assessment (HTA). Where evidence directly pertaining to the decision problem is sparse, it may be feasible to expand the evidence-base to include studies that relate to the decision problem only indirectly: for instance, when there is no evidence on a comparator, evidence on other treatments of the same molecular class could be used; similarly, a decision on children may borrow-strength from evidence on adults. Usually, in HTA, such indirect evidence is either included by ignoring any differences ('lumping') or not included at all ('splitting'). However, a range of more sophisticated methods exists, primarily in the biostatistics literature. The objective of this study is to identify and classify the breadth of the available information-sharing methods.

METHODS

Forwards and backwards citation-mining techniques were used on a set of seminal papers on the topic of information-sharing. Papers were included if they specified (network) meta-analytic methods for combining information from distinct populations, interventions, outcomes or study-designs.

RESULTS

Overall, 89 papers were included. A plethora of evidence synthesis methods have been used for information-sharing. Most papers (n=79) described methods that shared information on relative treatment effects. Amongst these, there was a strong emphasis on methods for information-sharing across multiple outcomes (n=42) and treatments (n=25), with fewer papers focusing on study-designs (n=23) or populations (n=8). We categorise and discuss the methods under four 'core' relationships of information-sharing: functional, exchangeability-based, prior-based and multivariate relationships, and explain the assumptions made within each of these core approaches.

CONCLUSIONS

This study highlights the range of information-sharing methods available. These methods often impose more moderate assumptions than lumping or splitting. Hence, the degree of information-sharing that they impose could potentially be considered more appropriate. Our identification of four 'core' methods of information-sharing allows for an improved understanding of the assumptions underpinning the different methods. Further research is required to understand how the methods differ in terms of the strength of sharing they impose and the implications of this for health care decisions.

The statistical importance of a study for a network meta-analysis estimate

Background

In pairwise meta-analysis, the contribution of each study to the pooled estimate is given by its weight, which is based on the inverse variance of the estimate from that study. For network meta-analysis (NMA), the contribution of direct (and indirect) evidence is easily obtained from the diagonal elements of a hat matrix. It is, however, not fully clear how to generalize this to the percentage contribution of each study to a NMA estimate.

Methods

We define the importance of each study for a NMA estimate by the reduction of the estimate’s variance when adding the given study to the others. An equivalent interpretation is the relative loss in precision when the study is left out. Importances are values between 0 and 1. An importance of 1 means that the study is an essential link of the pathway in the network connecting one of the treatments with another.

Results

Importances can be defined for two-stage and one-stage NMA. These numbers in general do not add to one and thus cannot be interpreted as ‘percentage contributions’. After briefly discussing other available approaches, we question whether it is possible to obtain unique percentage contributions for NMA.

Conclusions

Importances generalize the concept of weights in pairwise meta-analysis in a natural way. Moreover, they are uniquely defined, easily calculated, and have an intuitive interpretation. We give some real examples for illustration.

Empirical comparison of univariate and multivariate meta-analyses in Cochrane Pregnancy and Childbirth reviews with multiple binary outcomes

BACKGROUND

Multivariate meta-analysis (MVMA) jointly synthesizes effects for multiple correlated outcomes. The MVMA model is potentially more difficult and time-consuming to apply than univariate models, so if its use makes little difference to parameter estimates, it could be argued that it is redundant.

METHODS

We assessed the applicability and impact of MVMA in Cochrane Pregnancy and Childbirth (CPCB) systematic reviews. We applied MVMA to CPCB reviews published between 2011 and 2013 with two or more binary outcomes with at least three studies and compared findings with results of univariate meta-analyses. Univariate random effects meta-analysis models were fitted using restricted maximum likelihood estimation (REML).

RESULTS

Eighty CPCB reviews were published. MVMA could not be applied in 70 of these reviews. MVMA was not feasible in three of the remaining 10 reviews because the appropriate models failed to converge. Estimates from MVMA agreed with those of univariate analyses in most of the other seven reviews. Statistical significance changed in two reviews: In one, this was due to a very small change in P value; in the other, the MVMA result for one outcome suggested that previous univariate results may be vulnerable to small-study effects and that the certainty of clinical conclusions needs consideration.

CONCLUSIONS

MVMA methods can be applied only in a minority of reviews of interventions in pregnancy and childbirth and can be difficult to apply because of missing correlations or lack of convergence. Nevertheless, clinical and/or statistical conclusions from MVMA may occasionally differ from those from univariate analyses.

Borrowing of strength from indirect evidence in 40 network meta-analyses

OBJECTIVES

Network meta-analysis (NMA) is increasingly being used to synthesize direct and indirect evidence and help decision makers simultaneously compare multiple treatments. We empirically evaluate the incremental gain in precision achieved by incorporating indirect evidence in NMAs.

STUDY DESIGN AND SETTING

We performed both network and pairwise meta-analyses on 40 published data sets of multiple-treatment comparisons. Their results were compared using the recently proposed borrowing of strength (BoS) statistic, which quantifies the percentage reduction in the uncertainty of the effect estimate when adding indirect evidence to an NMA.

RESULTS

We analyzed 915 possible treatment comparisons, from which 484 (53%) had no direct evidence (BoS = 100%). In 181 comparisons with only one study contributing direct evidence, NMAs resulted in reduced precision (BoS < 0) and no appreciable improvements in precision (0 < BoS < 30%) for 104 (57.5%) and 23 (12.7%) comparisons, respectively. In 250 comparisons with at least two studies contributing direct evidence, NMAs provided increased precision with BoS ≥ 30% for 166 (66.4%) comparisons.

CONCLUSION

Although NMAs have the potential to provide more precise results than those only based on direct evidence, the incremental gain may reliably occur only when at least two head-to-head studies are available and treatments are well connected. Researchers should routinely report and compare the results from both network and pairwise meta-analyses.