Absolute or relative effects? Arm-based synthesis of trial data

Meta-analysis of the implied distribution of callous-unemotional traits across sampling methods and informant

Callous-unemotional (CU) traits have been measured in a variety of sample-types (e.g., community or forensic) and from the perspective of different informants (e.g., self-report or parent-report) using the inventory of callous-unemotional traits total score (ICU-T). Although the positive association between CU traits and antisocial behavior is uncontroversial, the degree to which sample-types are different from each other has received little attention despite such knowledge being important for generalization and interpretation of research findings. To address this gap in the literature, we estimated the implied distribution of the ICU-T across sample-types, informants, and their interaction using meta-analytic models of sample means and variances. In unconditional models, we found that sample-type significantly moderated mean ICU-T scores but not variance, while informant significantly moderated the variance of ICU-T scores but not means. There was also a significant interaction between sample-type and informant. Mean parent-reported ICU-T scores were significantly lower than self-reported scores in community samples, but not significantly different in samples with elevated levels of antisocial behavior. Implications of our findings include improved research efficiency, the need for different ICU-T norms across informants, and greater understanding of informant biases.

Network meta analysis to predict the efficacy of an approved treatment in a new indication

Drug repurposing refers to the process of discovering new therapeutic uses for existing medicines. Compared to traditional drug discovery, drug repurposing is attractive for its speed, cost, and reduced risk of failure. However, existing approaches for drug repurposing involve complex, computationally-intensive analytical methods that are not widely used in practice. Instead, repurposing decisions are often based on subjective judgments from limited empirical evidence. In this article, we develop a novel Bayesian network meta-analysis (NMA) framework that can predict the efficacy of an approved treatment in a new indication and thereby identify candidate treatments for repurposing. We obtain predictions using two main steps: first, we use standard NMA modeling to estimate average relative effects from a network comprised of treatments studied in both indications in addition to one treatment studied in only one indication. Then, we model the correlation between relative effects using various strategies that differ in how they model treatments across indications and within the same drug class. We evaluate the predictive performance of each model using a simulation study and find that the model minimizing root mean squared error of the posterior median for the candidate treatment depends on the amount of available data, the level of correlation between indications, and whether treatment effects differ, on average, by drug class. We conclude by discussing an illustrative example in psoriasis and psoriatic arthritis and find that the candidate treatment has a high probability of success in a future trial.

Evidence Synthesis for Complex Interventions Using Meta-Regression Models

A goal of evidence synthesis for trials of complex interventions is to inform the design or implementation of novel versions of complex interventions by predicting expected outcomes with each intervention version. Conventional aggregate data meta-analyses of studies comparing complex interventions have limited ability to provide such information. We argue that evidence synthesis for trials of complex interventions should forgo aspirations of estimating causal effects and instead model the response surface of study results to 1) summarize the available evidence and 2) predict the average outcomes of future studies or in new settings. We illustrate this modeling approach using data from a systematic review of diabetes quality improvement (QI) interventions involving at least 1 of 12 QI strategy components. We specify a series of meta-regression models to assess the association of specific components with the posttreatment outcome mean and compare the results to conventional meta-analysis approaches. Compared with conventional approaches, modeling the response surface of study results can better reflect the associations between intervention components and study characteristics with the posttreatment outcome mean. Modeling study results using a response surface approach offers a useful and feasible goal for evidence synthesis of complex interventions that rely on aggregate data.

Twenty years of network meta-analysis: Continuing controversies and recent developments

Network meta-analysis (NMA) is an extension of pairwise meta-analysis (PMA) which combines evidence from trials on multiple treatments in connected networks. NMA delivers internally consistent estimates of relative treatment efficacy, needed for rational decision making. Over its first 20 years NMA's use has grown exponentially, with applications in both health technology assessment (HTA), primarily re-imbursement decisions and clinical guideline development, and clinical research publications. This has been a period of transition in meta-analysis, first from its roots in educational and social psychology, where large heterogeneous datasets could be explored to find effect modifiers, to smaller pairwise meta-analyses in clinical medicine on average with less than six studies. This has been followed by narrowly-focused estimation of the effects of specific treatments at specific doses in specific populations in sparse networks, where direct comparisons are unavailable or informed by only one or two studies. NMA is a powerful and well-established technique but, in spite of the exponential increase in applications, doubts about the reliability and validity of NMA persist. Here we outline the continuing controversies, and review some recent developments. We suggest that heterogeneity should be minimized, as it poses a threat to the reliability of NMA which has not been fully appreciated, perhaps because it has not been seen as a problem in PMA. More research is needed on the extent of heterogeneity and inconsistency in datasets used for decision making, on formal methods for making recommendations based on NMA, and on the further development of multi-level network meta-regression.

Rare events meta-analysis using the Bayesian beta-binomial model

In meta-analyses of rare events, it can be challenging to obtain a reliable estimate of the pooled effect, in particular when the meta-analysis is based on a small number of studies. Recent simulation studies have shown that the beta-binomial model is a promising candidate in this situation, but have thus far only investigated its performance in a frequentist framework. In this study, we aim to make the beta-binomial model for meta-analysis of rare events amenable to Bayesian inference by proposing prior distributions for the effect parameter and investigating the models' robustness to different specifications of priors for the scale parameter. To evaluate the performance of Bayesian beta-binomial models with different priors, we conducted a simulation study with two different data generating models in which we varied the size of the pooled effect, the degree of heterogeneity, the baseline probability, and the sample size. Our results show that while some caution must be exercised when using the Bayesian beta-binomial in meta-analyses with extremely sparse data, the use of a weakly informative prior for the effect parameter is beneficial in terms of mean bias, mean squared error, and coverage. For the scale parameter, half-normal and exponential distributions are identified as candidate priors in meta-analysis of rare events using the Bayesian beta-binomial model.

Optimal dose and type of physical activity to improve functional capacity and minimise adverse events in acutely hospitalised older adults: a systematic review with dose-response network meta-analysis of randomised controlled trials

OBJECTIVE

To identify the optimal dose and type of physical activity to improve functional capacity and reduce adverse events in acutely hospitalised older adults.

DESIGN

Systematic review and Bayesian model-based network meta-analysis.

DATA SOURCES

Four databases were searched from inception to 20 June 2022.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Randomised controlled trials that assessed the effectiveness of a physical activity-based intervention on at least one functional outcome in people aged ≥50 years hospitalised due to an acute medical condition were included. Pooled effect estimates (ie, standardised mean differences for functional capacity and the ratio of means for adverse events) were calculated using random treatment effects network meta-analysis models.

RESULTS

Nineteen studies (3842 participants) met the inclusion criteria. Approximately 100 Metabolic Equivalents of Task per day (METs-min/day) (~40 min/day of light effort or ~25 min/day of moderate effort activities) was the minimal dose to improve the functional capacity of acute hospitalised older adults (standardised mean difference (SMD)=0.28, 95% credible interval (CrI) 0.01 to 0.55). The optimal dose was estimated at 159 METs-min/day (~70 min/day of light effort or ~40 min/day of moderate effort activities; SMD=0.41, 95% CrI 0.08 to 0.72). Ambulation was deemed the most efficient intervention, and the optimal dose was reached at 143 METs-min/day (~50 min/day of slow-paced walking; SMD=0.76, 95% CrI 0.35 to 1.16), showing a high evidential power (87.68%). The minimal effective ambulation dose was estimated at 74 METs-min/day (~25 min/day of slow-paced walking; SMD=0.25, 95% CrI 0.01 to 0.41). Physical activity interventions resulted in a decrease in the rate of adverse events compared with usual care at discharge (ratio of means=0.96, 95% CrI 0.95 to 0.97; median time 7 days).

CONCLUSIONS

This meta-analysis yielded low to moderate evidence supporting the use of in-hospital supervised physical activity programmes in acutely hospitalised older adults. As little as ~25 min/day of slow-paced walking is sufficient to improve functional capacity and minimise adverse events in this population.

TRIAL REGISTRATION NUMBER

PROSPERO CRD42021271999.

A Bayesian model for combining standardized mean differences and odds ratios in the same meta-analysis

In meta-analysis practice, researchers frequently face studies that report the same outcome differently, such as a continuous variable (e.g., scores for rating depression) or a binary variable (e.g., counts of patients with depression dichotomized by certain latent and unreported depression scores). For combining these two types of studies in the same analysis, a simple conversion method has been widely used to handle standardized mean differences (SMDs) and odds ratios (ORs). This conventional method uses a linear function connecting the SMD and log OR; it assumes logistic distributions for (latent) continuous measures. However, the normality assumption is more commonly used for continuous measures, and the conventional method may be inaccurate when effect sizes are large or cutoff values for dichotomizing binary events are extreme (leading to rare events). This article proposes a Bayesian hierarchical model to synthesize SMDs and ORs without using the conventional conversion method. This model assumes exact likelihoods for continuous and binary outcome measures, which account for full uncertainties in the synthesized results. We performed simulation studies to compare the performance of the conventional and Bayesian methods in various settings. The Bayesian method generally produced less biased results with smaller mean squared errors and higher coverage probabilities than the conventional method in most cases. Nevertheless, this superior performance depended on the normality assumption for continuous measures; the Bayesian method could lead to nonignorable biases for non-normal data. In addition, we used two case studies to illustrate the proposed Bayesian method in real-world settings.

Accounting for post-randomization variables in meta-analysis: A joint meta-regression approach

Meta-regression is widely used in systematic reviews to investigate sources of heterogeneity and the association of study-level covariates with treatment effectiveness. Existing meta-regression approaches are successful in adjusting for baseline covariates, which include real study-level covariates (e.g., publication year) that are invariant within a study and aggregated baseline covariates (e.g., mean age) that differ for each participant but are measured before randomization within a study. However, these methods have several limitations in adjusting for post-randomization variables. Although post-randomization variables share a handful of similarities with baseline covariates, they differ in several aspects. First, baseline covariates can be aggregated at the study level presumably because they are assumed to be balanced by the randomization, while post-randomization variables are not balanced across arms within a study and are commonly aggregated at the arm level. Second, post-randomization variables may interact dynamically with the primary outcome. Third, unlike baseline covariates, post-randomization variables are themselves often important outcomes under investigation. In light of these differences, we propose a Bayesian joint meta-regression approach adjusting for post-randomization variables. The proposed method simultaneously estimates the treatment effect on the primary outcome and on the post-randomization variables. It takes into consideration both between- and within-study variability in post-randomization variables. Studies with missing data in either the primary outcome or the post-randomization variables are included in the joint model to improve estimation. Our method is evaluated by simulations and a real meta-analysis of major depression disorder treatments.

Bias propagation in network meta-analysis models

Network meta-analysis combines direct and indirect evidence to compare multiple treatments. As direct evidence for one treatment contrast may be indirect evidence for other treatment contrasts, biases in the direct evidence for one treatment contrast may affect not only the estimate for this particular treatment contrast but also estimates of other treatment contrasts. Because network structure determines how direct and indirect evidence are combined and weighted, the impact of biased evidence will be determined by the network geometry. Thus, this study's aim was to investigate how the impact of biased evidence spreads across the whole network and how the propagation of bias is influenced by the network structure. In addition to the popular Lu & Ades model, we also investigate bias propagation in the baseline model and arm-based model to compare the effects of bias in the different models. We undertook extensive simulations under different scenarios to explore how the impact of bias may be affected by the location of the bias, network geometry and the statistical model. Our results showed that the structure of a network has an important impact on how the bias spreads across the network, and this is especially true for the Lu & Ades model. The impact of bias is more likely to be diluted by other unbiased evidence in a well-connected network. We also used a real network meta-analysis to demonstrate how to use the new knowledge about bias propagation to explain questionable results from the original analysis.

Using the bayesmeta R package for Bayesian random-effects meta-regression

BACKGROUND

Random-effects meta-analysis within a hierarchical normal modeling framework is commonly implemented in a wide range of evidence synthesis applications. More general problems may even be tackled when considering meta-regression approaches that in addition allow for the inclusion of study-level covariables.

METHODS

We describe the Bayesian meta-regression implementation provided in the bayesmetaR package including the choice of priors, and we illustrate its practical use.

RESULTS

A wide range of example applications are given, such as binary and continuous covariables, subgroup analysis, indirect comparisons, and model selection. Example R code is provided.

CONCLUSIONS

The bayesmeta package provides a flexible implementation. Due to the avoidance of MCMC methods, computations are fast and reproducible, facilitating quick sensitivity checks or large-scale simulation studies.

Bayesian modelling strategies for borrowing of information in randomised basket trials

Basket trials are an innovative precision medicine clinical trial design evaluating a single targeted therapy across multiple diseases that share a common characteristic. To date, most basket trials have been conducted in early-phase oncology settings, for which several Bayesian methods permitting information sharing across subtrials have been proposed. With the increasing interest of implementing randomised basket trials, information borrowing could be exploited in two ways; considering the commensurability of either the treatment effects or the outcomes specific to each of the treatment groups between the subtrials. In this article, we extend a previous analysis model based on distributional discrepancy for borrowing over the subtrial treatment effects ('treatment effect borrowing', TEB) to borrowing over the subtrial groupwise responses ('treatment response borrowing', TRB). Simulation results demonstrate that both modelling strategies provide substantial gains over an approach with no borrowing. TRB outperforms TEB especially when subtrial sample sizes are small on all operational characteristics, while the latter has considerable gains in performance over TRB when subtrial sample sizes are large, or the treatment effects and groupwise mean responses are noticeably heterogeneous across subtrials. Further, we notice that TRB, and TEB can potentially lead to different conclusions in the analysis of real data.

Network Meta-analysis on Disconnected Evidence Networks When Only Aggregate Data Are Available: Modified Methods to Include Disconnected Trials and Single-Arm Studies while Minimizing Bias

BACKGROUND

Network meta-analysis (NMA) requires a connected network of randomized controlled trials (RCTs) and cannot include single-arm studies. Regulators or academics often have only aggregate data. Two aggregate data methods for analyzing disconnected networks are random effects on baseline and aggregate-level matching (ALM). ALM has been used only for single-arm studies, and both methods may bias effect estimates.

METHODS

We modified random effects on baseline to separate RCTs connected to and disconnected from the reference and any single-arm studies, minimizing the introduction of bias. We term our modified method reference prediction. We similarly modified ALM and extended it to include RCTs disconnected from the reference. We tested these methods using constructed data and a simulation study.

RESULTS

In simulations, bias for connected treatments for ALM ranged from -0.0158 to 0.051 and for reference prediction from -0.0107 to 0.083. These were low compared with the true mean effect of 0.5. Coverage ranged from 0.92 to 1.00. In disconnected treatments, bias of ALM ranged from -0.16 to 0.392 and of reference prediction from -0.102 to 0.40, whereas coverage of ALM ranged from 0.30 to 0.82 and of reference prediction from 0.64 to 0.94. Under fixed study effects for disconnected evidence, bias was similar, but coverage was 0.81 to 1.00 for reference prediction and 0.18 to 0.76 for ALM. Trends of similar bias but greater coverage for reference prediction with random study effects were repeated in constructed data.

CONCLUSIONS

Both methods with random study effects seem to minimize bias in treatment connected to the reference. They can estimate treatment effects for disconnected treatments but may be biased. Reference prediction has greater coverage and may be recommended overall.

HIGHLIGHTS

Two methods were modified for network meta-analysis on disconnected networks and for including single-arm observational or interventional studies in network meta-analysis using only aggregate data and for minimizing the bias of effect estimates for treatments only in trials connected to the reference.Reference prediction was developed as a modification of random effects on baseline that keeps analyses of trials connected to the reference separately from those disconnected from the reference and from single-arm studies. The method was further modified to account for correlation in trials with more than 2 arms and, under random study effects, to estimate variance in heterogeneity separately in connected and disconnected evidence.Aggregate-level matching was extended to include trials disconnected from the reference, rather than only single-arm studies. The method was further modified to separately estimate treatment effects and heterogeneity variance in the connected and disconnected evidence and to account for the correlation between arms in trials with more than 2 arms.Performance was assessed using a constructed data example and simulation study.The methods were found to have similar, and sometimes low, bias when estimating the relative effects for disconnected treatments, but reference prediction with random study effects had the greatest coverage.The use of reference prediction with random study effects for disconnected networks is recommended if no individual patient data or alternative real-world evidence is available.

Bayesian Models for N-of-1 Trials

We describe Bayesian models for data from N-of-1 trials, reviewing both the basics of Bayesian inference and applications to data from single trials and collections of trials sharing the same research questions and data structures. Bayesian inference is natural for drawing inferences from N-of-1 trials because it can incorporate external and subjective information to supplement trial data as well as give straightforward interpretations of posterior probabilities as an individual's state of knowledge about their own condition after their trial. Bayesian models are also easily augmented to incorporate specific characteristics of N-of-1 data such as trend, carryover, and autocorrelation and offer flexibility of implementation. Combining data from multiple N-of-1 trials using Bayesian multilevel models leads naturally to inferences about population and subgroup parameters such as average treatment effects and treatment effect heterogeneity and to improved inferences about individual parameters. Data from a trial comparing different diets for treating children with inflammatory bowel disease are used to illustrate the models and inferences that may be drawn. The analysis shows that certain diets were better on average at reducing pain, but that benefits were restricted to a subset of patients and that withdrawal from the study was a good marker for lack of benefit.

Bayesian network meta-analysis methods for combining individual participant data and aggregate data from single arm trials and randomised controlled trials

BACKGROUND

Increasingly in network meta-analysis (NMA), there is a need to incorporate non-randomised evidence to estimate relative treatment effects, and in particular in cases with limited randomised evidence, sometimes resulting in disconnected networks of treatments. When combining different sources of data, complex NMA methods are required to address issues associated with participant selection bias, incorporating single-arm trials (SATs), and synthesising a mixture of individual participant data (IPD) and aggregate data (AD). We develop NMA methods which synthesise data from SATs and randomised controlled trials (RCTs), using a mixture of IPD and AD, for a dichotomous outcome.

METHODS

We propose methods under both contrast-based (CB) and arm-based (AB) parametrisations, and extend the methods to allow for both within- and across-trial adjustments for covariate effects. To illustrate the methods, we use an applied example investigating the effectiveness of biologic disease-modifying anti-rheumatic drugs for rheumatoid arthritis (RA). We applied the methods to a dataset obtained from a literature review consisting of 14 RCTs and an artificial dataset consisting of IPD from two SATs and AD from 12 RCTs, where the artificial dataset was created by removing the control arms from the only two trials assessing tocilizumab in the original dataset.

RESULTS

Without adjustment for covariates, the CB method with independent baseline response parameters (CBunadjInd) underestimated the effectiveness of tocilizumab when applied to the artificial dataset compared to the original dataset, albeit with significant overlap in posterior distributions for treatment effect parameters. The CB method with exchangeable baseline response parameters produced effectiveness estimates in agreement with CBunadjInd, when the predicted baseline response estimates were similar to the observed baseline response. After adjustment for RA duration, there was a reduction in across-trial heterogeneity in baseline response but little change in treatment effect estimates.

CONCLUSIONS

Our findings suggest incorporating SATs in NMA may be useful in some situations where a treatment is disconnected from a network of comparator treatments, due to a lack of comparative evidence, to estimate relative treatment effects. The reliability of effect estimates based on data from SATs may depend on adjustment for covariate effects, although further research is required to understand this in more detail.

Theory and practice of Bayesian and frequentist frameworks for network meta-analysis

Network meta-analysis (NMA) is an increasingly popular statistical method of synthesising evidence to assess the comparative benefits and harms of multiple treatments in a single analysis. Several automated software packages facilitate conducting NMA using either of two alternative approaches, Bayesian or frequentist frameworks. Researchers must choose a framework for conducting NMA (Bayesian or frequentist) and select appropriate model(s), and those conducting NMA need to understand the assumptions and limitations of different approaches. Bayesian models are more frequently used and can be more flexible but require checking additional assumptions and greater statistical expertise that are often ignored. The present paper describes the important theoretical aspects of Bayesian and frequentist models for NMA and the applications and considerations of contrast-synthesis and arm-synthesis NMAs. In addition, we present evidence from a limited number of simulation and empirical studies that compared different frequentist and Bayesian models and provide an overview of available automated software packages to perform NMA. We will conclude that when analysts choose appropriate models, there are seldom important differences in the results of Bayesian and frequentist approaches and that network meta-analysts should therefore focus on model features rather than the statistical framework.

How to observe the principle of concurrent control in an arm-based meta-analysis using SAS procedures GLIMMIX and BGLIMM

Network meta-analysis is a popular method to synthesize the information obtained in a systematic review of studies (e.g. randomized clinical trials) involving subsets of multiple treatments of interest. The dominant method of analysis employs within-study information on treatment contrasts and integrates this over a network of studies. One advantage of this approach is that all inference is protected by within-study randomization. By contrast, arm-based analyses have been criticized in the past because they may also recover inter-study information when studies are modelled as random, which is the dominant practice, hence violating the principle of concurrent control, requiring treated individuals to only be compared directly with randomized controls. This issue arises regardless of whether analysis is implemented within a frequentist or a Bayesian framework. Here, we argue that recovery of inter-study information can be prevented in an arm-based analysis by adding a fixed study main effect. This simple device means that it is possible to honour the principle of concurrent control in a two-way analysis-of-variance approach that is very easy to implement using generalized linear mixed model procedures and hence may be particularly welcome to those not well versed in the more intricate coding required for a contrast-based analysis. This article is protected by copyright. All rights reserved.

Contrast-Based and Arm-Based Models for Network Meta-Analysis

Network meta-analysis is used to synthesize evidence from a network of treatments. The models used in a network meta-analysis are more complex than those used for pairwise meta-analysis. Two types of models are available to undertake a network meta-analysis: contrast-based and arm-based models. Contrast-based models have been used in most published network meta-analyses. Arm-based models offer greater flexibility and handle treatments symmetrically, but risk compromising randomization. In this chapter, we (1) present the contrast-based and arm-based statistical models; (2) describe the theoretical differences between the models (noting when the estimates from the models are expected to diverge); (3) summarize the evidence comparing the two models from simulation studies and empirical investigations; and (4) provide a worked example applying the two models to a network using the R software package.

BRIDGING RANDOMIZED CONTROLLED TRIALS AND SINGLE-ARM TRIALS USING COMMENSURATE PRIORS IN ARM-BASED NETWORK META-ANALYSIS

Network meta-analysis (NMA) is a powerful tool to compare multiple treatments directly and indirectly by combining and contrasting multiple independent clinical trials. Because many NMAs collect only a few eligible randomized controlled trials (RCTs), there is an urgent need to synthesize different sources of information, e.g., from both RCTs and single-arm trials. However, single-arm trials and RCTs may have different populations and quality, so that assuming they are exchangeable may be inappropriate. This article presents a novel method using a commensurate prior on variance (CPV) to borrow variance (rather than mean) information from single-arm trials in an arm-based (AB) Bayesian NMA. We illustrate the advantages of this CPV method by reanalyzing an NMA of immune checkpoint inhibitors in cancer patients. Comprehensive simulations investigate the impact on statistical inference of including single-arm trials. The simulation results show that the CPV method provides efficient and robust estimation even when the two sources of information are moderately inconsistent.

Bayesian meta-analysis using SAS PROC BGLIMM

Meta-analysis is commonly used to compare two treatments. Network meta-analysis (NMA) is a powerful extension for comparing and contrasting multiple treatments simultaneously in a systematic review of multiple clinical trials. Although the practical utility of meta-analysis is apparent, it is not always straightforward to implement, especially for those interested in a Bayesian approach. This paper demonstrates that the recently-developed SAS procedure BGLIMM provides an intuitive and computationally efficient means for conducting Bayesian meta-analysis in SAS, using a worked example of a smoking cessation NMA data set. BGLIMM gives practitioners an effective and simple way to implement Bayesian meta-analysis (pairwise and network, either contrast-based or arm-based) without requiring significant background in coding or statistical modeling. Those familiar with generalized linear mixed models, and especially the SAS procedure GLIMMIX, will find this tutorial a useful introduction to Bayesian meta-analysis in SAS.

Comparing direct acting antivirals for hepatitis C using observational data - Why and how?

The World Health Organisation's goal of hepatitis C virus (HCV) elimination by 2030 will require lower drug prices. Estimates of comparative efficacy promote competition between pharmaceutical companies but direct acting antivirals have been approved for the treatment of HCV without comparative trials. We emulated a randomized trial to answer the question of whether easy to treat patients with genotype 1 HCV could be treated with sofosbuvir/ledipasvir (SOF/LDV) rather than sofosbuvir/velpatasvir (SOF/VEL). Patients without comorbidities or end stage liver disease were selected from the British Colombia Hepatitis Testers Cohort. To create a conceptual trial, we matched each patient starting SOF/VEL (a ‘case’) to the patient starting SOF/LDV with the closest propensity score (a ‘control’). We estimated the probability of treatment failure under a Bayesian logistic model with a random effect for each case‐control set and used that model to give an estimate of a risk difference for the conceptual trial. Treatment failure was recorded for 27 of 825 (3%) cases and for 29 of 602 (5%) matched controls. Estimates from our model were treatment success rates of 97% (95% credible interval, CrI, 95%‐98%) for treatment with SOF/VEL, 95% (95% CrI 93%‐97%) for treatment with SOF/LDV and a risk difference between treatments of 2% (95% CrI 0%‐4%). This risk difference is evidence that SOF/LDV is not inferior to SOF/VEL for easy to treat patients with genotype 1 HCV. The approach is a template for comparing drugs when there are no data from comparative trials.

Incorporating single-arm studies in meta-analysis of randomised controlled trials: a simulation study

Background

Use of real world data (RWD) from non-randomised studies (e.g. single-arm studies) is increasingly being explored to overcome issues associated with data from randomised controlled trials (RCTs). We aimed to compare methods for pairwise meta-analysis of RCTs and single-arm studies using aggregate data, via a simulation study and application to an illustrative example.

Methods

We considered contrast-based methods proposed by Begg & Pilote (1991) and arm-based methods by Zhang et al (2019). We performed a simulation study with scenarios varying (i) the proportion of RCTs and single-arm studies in the synthesis (ii) the magnitude of bias, and (iii) between-study heterogeneity. We also applied methods to data from a published health technology assessment (HTA), including three RCTs and 11 single-arm studies.

Results

Our simulation study showed that the hierarchical power and commensurate prior methods by Zhang et al provided a consistent reduction in uncertainty, whilst maintaining over-coverage and small error in scenarios where there was limited RCT data, bias and differences in between-study heterogeneity between the two sets of data. The contrast-based methods provided a reduction in uncertainty, but performed worse in terms of coverage and error, unless there was no marked difference in heterogeneity between the two sets of data.

Conclusions

The hierarchical power and commensurate prior methods provide the most robust approach to synthesising aggregate data from RCTs and single-arm studies, balancing the need to account for bias and differences in between-study heterogeneity, whilst reducing uncertainty in estimates. This work was restricted to considering a pairwise meta-analysis using aggregate data.

Supplementary Information

The online version contains supplementary material available at (10.1186/s12874-021-01301-1).

Bayesian Methods for Meta-Analyses of Binary Outcomes: Implementations, Examples, and Impact of Priors

Bayesian methods are an important set of tools for performing meta-analyses. They avoid some potentially unrealistic assumptions that are required by conventional frequentist methods. More importantly, meta-analysts can incorporate prior information from many sources, including experts’ opinions and prior meta-analyses. Nevertheless, Bayesian methods are used less frequently than conventional frequentist methods, primarily because of the need for nontrivial statistical coding, while frequentist approaches can be implemented via many user-friendly software packages. This article aims at providing a practical review of implementations for Bayesian meta-analyses with various prior distributions. We present Bayesian methods for meta-analyses with the focus on odds ratio for binary outcomes. We summarize various commonly used prior distribution choices for the between-studies heterogeneity variance, a critical parameter in meta-analyses. They include the inverse-gamma, uniform, and half-normal distributions, as well as evidence-based informative log-normal priors. Five real-world examples are presented to illustrate their performance. We provide all of the statistical code for future use by practitioners. Under certain circumstances, Bayesian methods can produce markedly different results from those by frequentist methods, including a change in decision on statistical significance. When data information is limited, the choice of priors may have a large impact on meta-analytic results, in which case sensitivity analyses are recommended. Moreover, the algorithm for implementing Bayesian analyses may not converge for extremely sparse data; caution is needed in interpreting respective results. As such, convergence should be routinely examined. When select statistical assumptions that are made by conventional frequentist methods are violated, Bayesian methods provide a reliable alternative to perform a meta-analysis.

On weakly informative prior distributions for the heterogeneity parameter in Bayesian random-effects meta-analysis

The normal-normal hierarchical model (NNHM) constitutes a simple and widely used framework for meta-analysis. In the common case of only few studies contributing to the meta-analysis, standard approaches to inference tend to perform poorly, and Bayesian meta-analysis has been suggested as a potential solution. The Bayesian approach, however, requires the sensible specification of prior distributions. While noninformative priors are commonly used for the overall mean effect, the use of weakly informative priors has been suggested for the heterogeneity parameter, in particular in the setting of (very) few studies. To date, however, a consensus on how to generally specify a weakly informative heterogeneity prior is lacking. Here we investigate the problem more closely and provide some guidance on prior specification.

Joining the Dots: Linking Disconnected Networks of Evidence Using Dose-Response Model-Based Network Meta-Analysis

BACKGROUND

Network meta-analysis (NMA) synthesizes direct and indirect evidence on multiple treatments to estimate their relative effectiveness. However, comparisons between disconnected treatments are not possible without making strong assumptions. When studies including multiple doses of the same drug are available, model-based NMA (MBNMA) presents a novel solution to this problem by modeling a parametric dose-response relationship within an NMA framework. In this article, we illustrate several scenarios in which dose-response MBNMA can connect and strengthen evidence networks.

METHODS

We created illustrative data sets by removing studies or treatments from an NMA of triptans for migraine relief. We fitted MBNMA models with different dose-response relationships. For connected networks, we compared MBNMA estimates with NMA estimates. For disconnected networks, we compared MBNMA estimates with NMA estimates from an "augmented" network connected by adding studies or treatments back into the data set.

RESULTS

In connected networks, relative effect estimates from MBNMA were more precise than those from NMA models (ratio of posterior SDs NMA v. MBNMA: median = 1.13; range = 1.04-1.68). In disconnected networks, MBNMA provided estimates for all treatments where NMA could not and were consistent with NMA estimates from augmented networks for 15 of 18 data sets. In the remaining 3 of 18 data sets, a more complex dose-response relationship was required than could be fitted with the available evidence.

CONCLUSIONS

Where information on multiple doses is available, MBNMA can connect disconnected networks and increase precision while making less strong assumptions than alternative approaches. MBNMA relies on correct specification of the dose-response relationship, which requires sufficient data at different doses to allow reliable estimation. We recommend that systematic reviews for NMA search for and include evidence (including phase II trials) on multiple doses of agents where available.

A unified method for improved inference in random effects meta-analysis

Random effects meta-analyses have been widely applied in evidence synthesis for various types of medical studies. However, standard inference methods (e.g. restricted maximum likelihood estimation) usually underestimate statistical errors and possibly provide highly overconfident results under realistic situations; for instance, coverage probabilities of confidence intervals can be substantially below the nominal level. The main reason is that these inference methods rely on large sample approximations even though the number of synthesized studies is usually small or moderate in practice. In this article, we solve this problem using a unified inference method based on Monte Carlo conditioning for broad application to random effects meta-analysis. The developed method provides improved confidence intervals with coverage probabilities that are closer to the nominal level than standard methods. As specific applications, we provide new inference procedures for three types of meta-analysis: conventional univariate meta-analysis for pairwise treatment comparisons, meta-analysis of diagnostic test accuracy, and multiple treatment comparisons via network meta-analysis. We also illustrate the practical effectiveness of these methods via real data applications and simulation studies.

Application of network meta-analysis in the field of physical activity and health promotion

Continued advancement in the field of physical activity and health promotion relies heavily on the synthesis of rigorous scientific evidence. As such, systematic reviews and meta-analyses of randomized controlled trials have led to a better understanding of which intervention strategies are superior (i.e., produce the greatest effects) in physical activity-based health behavior change interventions. Indeed, standard meta-analytic approaches have allowed researchers in the field to synthesize relevant experimental evidence using pairwise procedures that produce reliable estimates of the homogeneity, magnitude, and potential biases in the observed effects. However, pairwise meta-analytic procedures are only capable to discerning differences in effects between a select intervention strategy and a select comparison or control condition. In order to maximize the impact of physical activity interventions on health-related outcomes, it is necessary to establish evidence concerning the comparative efficacy of all relevant physical activity intervention strategies. The development of network meta-analysis (NMA)-most commonly used in medical-based clinical trials-has allowed for the quantification of indirect comparisons, even in the absence of direct, head-to-head trials. Thus, it stands to reason that NMA can be applied in physical activity and health promotion research to identify the best intervention strategies. Given that this analysis technique is novel and largely unexplored in the field of physical activity and health promotion, care must be taken in its application to ensure reliable estimates and discernment of the effect sizes among interventions. Therefore, the purpose of this review is to comment on the potential application and importance of NMA in the field of physical activity and health promotion, describe how to properly and effectively apply this technique, and suggest important considerations for its appropriate application in this field. In this paper, overviews of the foundations of NMA and commonly used approaches for conducting NMA are provided, followed by assumptions related to NMA, opportunities and challenges in NMA, and a step-by-step example of developing and conducting an NMA.

Exploiting relationships between outcomes in Bayesian multivariate network meta-analysis with an application to relapsing-remitting multiple sclerosis

In multivariate network meta-analysis (NMA), the piecemeal nature of the evidence base means that there may be treatment-outcome combinations for which no data is available. Most existing multivariate evidence synthesis models are either unable to estimate the missing treatment-outcome combinations, or can only do so under particularly strong assumptions, such as perfect between-study correlations between outcomes or constant effect size across outcomes. Many existing implementations are also limited to two treatments or two outcomes, or rely on model specification that is heavily tailored to the dimensions of the dataset. We present a Bayesian multivariate NMA model that estimates the missing treatment-outcome combinations via mappings between the population mean effects, while allowing the study-specific effects to be imperfectly correlated. The method is designed for aggregate-level data (rather than individual patient data) and is likely to be useful when modeling multiple sparsely reported outcomes, or when varying definitions of the same underlying outcome are adopted by different studies. We implement the model via a novel decomposition of the treatment effect variance, which can be specified efficiently for an arbitrary dataset given some basic assumptions regarding the correlation structure. The method is illustrated using data concerning the efficacy and liver-related safety of eight active treatments for relapsing-remitting multiple sclerosis. The results indicate that fingolimod and interferon beta-1b are the most efficacious treatments but also have some of the worst effects on liver safety. Dimethyl fumarate and glatiramer acetate perform reasonably on all of the efficacy and safety outcomes in the model.

The impact of covariance priors on arm-based Bayesian network meta-analyses with binary outcomes

Bayesian analyses with the arm-based (AB) network meta-analysis (NMA) model require researchers to specify a prior distribution for the covariance matrix of the treatment-specific event rates in a transformed scale, for example, the treatment-specific log-odds when a logit transformation is used. The commonly used conjugate prior for the covariance matrix, the inverse-Wishart (IW) distribution, has several limitations. For example, although the IW distribution is often described as noninformative or weakly informative, it may in fact provide strong information when some variance components are small (eg, when the standard deviation of study-specific log-odds of a treatment is smaller than 1/2), as is common in NMAs with binary outcomes. In addition, the IW prior generally leads to underestimation of correlations between treatment-specific log-odds, which are critical for borrowing strength across treatment arms to estimate treatment effects efficiently and to reduce potential bias. Alternatively, several separation strategies (ie, separate priors on variances and correlations) can be considered. To study the IW prior's impact on NMA results and compare it with separation strategies, we did simulation studies under different missing-treatment mechanisms. A separation strategy with appropriate priors for the correlation matrix and variances performs better than the IW prior, and should be recommended as the default vague prior in the AB NMA approach. Finally, we reanalyzed three case studies and illustrated the importance, when performing AB-NMA, of sensitivity analyses with different prior specifications on variances.

Degree irregularity and rank probability bias in network meta-analysis

Network meta-analysis (NMA) is a statistical technique for the comparison of treatment options. Outcomes of Bayesian NMA include estimates of treatment effects, and the probabilities that each treatment is ranked best, second best and so on. How exactly network topology affects the accuracy and precision of these outcomes is not fully understood. Here we carry out a simulation study and find that disparity in the number of trials involving different treatments leads to a systematic bias in estimated rank probabilities. This bias is associated with an increased variation in the precision of treatment effect estimates. Using ideas from the theory of complex networks, we define a measure of "degree irregularity" to quantify asymmetry in the number of studies involving each treatment. Our simulations indicate that more regular networks have more precise treatment effect estimates and smaller bias of rank probabilities. Conversely, these topological effects are not observed for the accuracy of treatment effect estimates. This reinforces the importance of taking into account multiple measures, rather than making decisions based on a single metric. We also find that degree regularity is a better indicator for the accuracy and precision of parameter estimates in NMA than both the total number of studies in a network and the disparity in the number of trials per comparison. These results have implications for planning future trials. We demonstrate that choosing trials which reduce the network's irregularity can improve the precision and accuracy of parameter estimates from NMA.

A variance shrinkage method improves arm-based Bayesian network meta-analysis

Network meta-analysis is a commonly used tool to combine direct and indirect evidence in systematic reviews of multiple treatments to improve estimation compared to traditional pairwise meta-analysis. Unlike the contrast-based network meta-analysis approach, which focuses on estimating relative effects such as odds ratios, the arm-based network meta-analysis approach can estimate absolute risks and other effects, which are arguably more informative in medicine and public health. However, the number of clinical studies involving each treatment is often small in a network meta-analysis, leading to unstable treatment-specific variance estimates in the arm-based network meta-analysis approach when using non- or weakly informative priors under an unequal variance assumption. Additional assumptions, such as equal (i.e. homogeneous) variances for all treatments, may be used to remedy this problem, but such assumptions may be inappropriately strong. This article introduces a variance shrinkage method for an arm-based network meta-analysis. Specifically, we assume different treatment variances share a common prior with unknown hyperparameters. This assumption is weaker than the homogeneous variance assumption and improves estimation by shrinking the variances in a data-dependent way. We illustrate the advantages of the variance shrinkage method by reanalyzing a network meta-analysis of organized inpatient care interventions for stroke. Finally, comprehensive simulations investigate the impact of different variance assumptions on statistical inference, and simulation results show that the variance shrinkage method provides better estimation for log odds ratios and absolute risks.

Illustrating the importance of meta-analysing variances alongside means in ecology and evolution

Meta-analysis is increasingly used in biology to both quantitatively summarize available evidence for specific questions and generate new hypotheses. Although this powerful tool has mostly been deployed to study mean effects, there is untapped potential to study effects on (trait) variance. Here, we use a recently published data set as a case study to demonstrate how meta-analysis of variance can be used to provide insights into biological processes. This data set included 704 effect sizes from 89 studies, covering 56 animal species, and was originally used to test developmental stress effects on a range of traits. We found that developmental stress not only negatively affects mean trait values, but also increases trait variance, mostly in reproduction, showcasing how meta-analysis of variance can reveal previously overlooked effects. Furthermore, we show how meta-analysis of variance can be used as a tool to help meta-analysts make informed methodological decisions, even when the primary focus is on mean effects. We provide all data and comprehensive R scripts with detailed explanations to make it easier for researchers to conduct this type of analysis. We encourage meta-analysts in all disciplines to move beyond the world of means and start unravelling secrets of the world of variance.

Nonproportional Hazards in Network Meta-Analysis: Efficient Strategies for Model Building and Analysis

OBJECTIVES

To develop efficient approaches for fitting network meta-analysis (NMA) models with time-varying hazard ratios (such as fractional polynomials and piecewise constant models) to allow practitioners to investigate a broad range of models rapidly and to achieve a more robust and comprehensive model selection strategy.

METHODS

We reformulated the fractional polynomial and piecewise constant NMA models using analysis of variance-like parameterization. With this approach, both models are expressed as generalized linear models (GLMs) with time-varying covariates. Such models can be fitted efficiently with standard frequentist techniques. We applied our approach to the example data from the study by Jansen et al, in which fractional polynomial NMA models were introduced.

RESULTS

Fitting frequentist fixed-effect NMAs for a large initial set of candidate models took less than 1 second with standard GLM routines. This allowed for model selection from a large range of hazard ratio structures by comparing a set of criteria including Akaike information criterion/Bayesian information criterion, visual inspection of goodness-of-fit, and long-term extrapolations. The "best" models were then refitted in a Bayesian framework. Estimates agreed very closely.

CONCLUSIONS

NMA models with time-varying hazard ratios can be explored efficiently with a stepwise approach. A frequentist fixed-effect framework enables rapid exploration of different models. The best model can then be assessed further in a Bayesian framework to capture and propagate uncertainty for decision-making.

Component network meta-analysis compared to a matching method in a disconnected network: A case study

Network meta-analysis is a method to combine evidence from randomized controlled trials (RCTs) that compare a number of different interventions for a given clinical condition. Usually, this requires a connected network. A possible approach to link a disconnected network is to add evidence from nonrandomized comparisons, using propensity score or matching-adjusted indirect comparisons methods. However, nonrandomized comparisons may be associated with an unclear risk of bias. Schmitz et al. used single-arm observational studies for bridging the gap between two disconnected networks of treatments for multiple myeloma. We present a reanalysis of these data using component network meta-analysis (CNMA) models entirely based on RCTs, utilizing the fact that many of the treatments consisted of common treatment components occurring in both networks. We discuss forward and backward strategies for selecting appropriate CNMA models and compare the results to those obtained by Schmitz et al. using their matching method. CNMA models provided a good fit to the data and led to treatment rankings that were similar, though not fully equal to that obtained by Schmitz et al. We conclude that researchers encountering a disconnected network with treatments in different subnets having common components should consider a CNMA model. Such models, exclusively based on evidence from RCTs, are a promising alternative to matching approaches that require additional evidence from observational studies. CNMA models are implemented in the R package netmeta.

A Bayesian approach to discrete multiple outcome network meta-analysis

In this paper we suggest a new Bayesian approach to network meta-analysis for the case of discrete multiple outcomes. The joint distribution of the discrete outcomes is modeled through a Gaussian copula with binomial marginals. The remaining elements of the hierarchial random effects model are specified in a standard way, with the logit of the success probabilities given by the sum of a baseline log-odds and random effects comparing the log-odds of each treatment against the reference and having a Gaussian distribution centered at the vector of pooled effects. An adaptive Markov Chain Monte Carlo algorithm is devised for running posterior inference. The model is applied to two datasets from Cochrane reviews, already analysed in two papers so to assess and compare its performance. We implemented the model in a freely available R package called netcopula.

Bayesian Network Meta-analysis of Multiple Outcomes in Dental Research

OBJECTIVES

Dental research typically targets multiple outcomes. Interdental cleaning devices such as interdental brushes (IB) and water jet devices (WJ) share a sizable portion of the medical device market. However, recommendations for device selection are limited by the conflicting evidence from multiple outcomes in available studies and the lack of an appropriate synthesis approach to summarize evidences taken from multiple outcomes. In particular, both pairwise meta-analyses and single-outcome network meta-analyses can give discordant results. The purpose of this multioutcome, Bayesian network meta-analysis is to introduce this innovative method to the dental research community using data from interdental cleaning device studies for illustrative purposes.

METHODS

We reanalyzed a network meta-analysis of interproximal oral hygiene methods in the reduction of clinical indices of inflammation, which included 22 trials assessing 10 interproximal oral hygiene aids. We focused on the primary outcome of gingival inflammation, which was measured by 2 correlated outcome variables, the Gingival Index (GI) and bleeding on probing (BOP).

RESULTS

In our previous single-outcome analysis, we concluded that IB and WJ rank high for reducing gingival inflammation while toothpick and flossing rank last. In this multioutcome Bayesian network meta-analysis with equal weight on GI and BOP, the surface under the cumulative ranking curve was 0.87 for WJ and 0.85 for IB. WJ and IB remained ranked as the 2 best devices across different sets of weightings for the GI and BOP.

CONCLUSION

In conclusion, multioutcome Bayesian network meta-analysis naturally takes the correlations among multiple outcomes into account, which in turn can provide more comprehensive evidence.

A comparison of arm-based and contrast-based models for network meta-analysis

Differences between arm-based (AB) and contrast-based (CB) models for network meta-analysis (NMA) are controversial. We compare the CB model of Lu and Ades (2006), the AB model of Hong et al(2016), and two intermediate models, using hypothetical data and a selected real data set. Differences between models arise primarily from study intercepts being fixed effects in the Lu-Ades model but random effects in the Hong model, and we identify four key difference. (1) If study intercepts are fixed effects then only within-study information is used, but if they are random effects then between-study information is also used and can cause important bias. (2) Models with random study intercepts are suitable for deriving a wider range of estimands, eg, the marginal risk difference, when underlying risk is derived from the NMA data; but underlying risk is usually best derived from external data, and then models with fixed intercepts are equally good. (3) The Hong model allows treatment effects to be related to study intercepts, but the Lu-Ades model does not. (4) The Hong model is valid under a more relaxed missing data assumption, that arms (rather than contrasts) are missing at random, but this does not appear to reduce bias. We also describe an AB model with fixed study intercepts and a CB model with random study intercepts. We conclude that both AB and CB models are suitable for the analysis of NMA data, but using random study intercepts requires a strong rationale such as relating treatment effects to study intercepts.

Integrated evaluation of targeted and non-targeted therapies in a network meta-analysis

Individualized therapies for patients with biomarkers are moving more and more into the focus of research interest when developing new treatments. Hereby, the term individualized (or targeted) therapy denotes a treatment specifically developed for biomarker-positive patients. A network meta-analysis model for a binary endpoint combining the evidence for a targeted therapy from individual patient data with the evidence for a non-targeted therapy from aggregate data is presented and investigated. The biomarker status of the patients is either available at patient-level in individual patient data or at study-level in aggregate data. Both types of biomarker information have to be included. The evidence synthesis model follows a Bayesian approach and applies a meta-regression to the studies with aggregate data. In a simulation study, we address three treatment arms, one of them investigating a targeted therapy. The bias and the root-mean-square error of the treatment effect estimate for the subgroup of biomarker-positive patients based on studies with aggregate data are investigated. Thereby, the meta-regression approach is compared to approaches applying alternative solutions. The regression approach has a surprisingly small bias even in the presence of few studies. By contrast, the root-mean-square error is relatively greater. An illustrative example is provided demonstrating implementation of the presented network meta-analysis model in a clinical setting.

Bayesian inference for network meta-regression using multivariate random effects with applications to cholesterol lowering drugs

Low-density lipoprotein cholesterol (LDL-C) has been identified as a causative factor for atherosclerosis and related coronary heart disease, and as the main target for cholesterol- and lipid-lowering therapy. Statin drugs inhibit cholesterol synthesis in the liver and are typically the first line of therapy to lower elevated levels of LDL-C. On the other hand, a different drug, Ezetimibe, inhibits the absorption of cholesterol by the small intestine and provides a different mechanism of action. Many clinical trials have been carried out on safety and efficacy evaluation of cholesterol lowering drugs. To synthesize the results from different clinical trials, we examine treatment level (aggregate) network meta-data from 29 double-blind, randomized, active, or placebo-controlled statins +/$-$ Ezetimibe clinical trials on adult treatment-naïve patients with primary hypercholesterolemia. In this article, we propose a new approach to carry out Bayesian inference for arm-based network meta-regression. Specifically, we develop a new strategy of grouping the variances of random effects, in which we first formulate possible sets of the groups of the treatments based on their clinical mechanisms of action and then use Bayesian model comparison criteria to select the best set of groups. The proposed approach is especially useful when some treatment arms are involved in only a single trial. In addition, a Markov chain Monte Carlo sampling algorithm is developed to carry out the posterior computations. In particular, the correlation matrix is generated from its full conditional distribution via partial correlations. The proposed methodology is further applied to analyze the network meta-data from 29 trials with 11 treatment arms.

Modelling time-course relationships with multiple treatments: Model-based network meta-analysis for continuous summary outcomes

BACKGROUND

Model-based meta-analysis (MBMA) is increasingly used to inform drug-development decisions by synthesising results from multiple studies to estimate treatment, dose-response, and time-course characteristics. Network meta-analysis (NMA) is used in Health Technology Appraisals for simultaneously comparing effects of multiple treatments, to inform reimbursement decisions. Recently, a framework for dose-response model-based network meta-analysis (MBNMA) has been proposed that combines, often nonlinear, MBMA modelling with the statistically robust properties of NMA. Here, we aim to extend this framework to time-course models.

METHODS

We propose a Bayesian time-course MBNMA modelling framework for continuous summary outcomes that allows for nonlinear modelling of multiparameter time-course functions, accounts for residual correlation between observations, preserves randomisation by modelling relative effects, and allows for testing of inconsistency between direct and indirect evidence on the time-course parameters. We demonstrate our modelling framework using an illustrative dataset of 23 trials investigating treatments for pain in osteoarthritis.

RESULTS

Of the time-course functions that we explored, the E_max model gave the best fit to the data and has biological plausibility. Some simplifying assumptions were needed to identify the ET₅₀ , due to few observations at early follow-up times. Treatment estimates were robust to the inclusion of correlations in the likelihood.

CONCLUSIONS

Time-course MBNMA provides a statistically robust framework for synthesising evidence on multiple treatments at multiple time points. The use of placebo-controlled studies in drug-development means there is limited potential for inconsistency. The methods can inform drug-development decisions and provide the rigour needed in the reimbursement decision-making process.

A Bayesian hierarchical model for network meta-analysis of multiple diagnostic tests

To compare the accuracy of multiple diagnostic tests in a single study, three designs are commonly used (i) the multiple test comparison design; (ii) the randomized design, and (iii) the non-comparative design. Existing meta-analysis methods of diagnostic tests (MA-DT) have been focused on evaluating the performance of a single test by comparing it with a reference test. The increasing number of available diagnostic instruments for a disease condition and the different study designs being used have generated the need to develop efficient and flexible meta-analysis framework to combine all designs for simultaneous inference. In this article, we develop a missing data framework and a Bayesian hierarchical model for network MA-DT (NMA-DT) and offer important promises over traditional MA-DT: (i) It combines studies using all three designs; (ii) It pools both studies with or without a gold standard; (iii) it combines studies with different sets of candidate tests; and (iv) it accounts for heterogeneity across studies and complex correlation structure among multiple tests. We illustrate our method through a case study: network meta-analysis of deep vein thrombosis tests.

Incorporating single-arm evidence into a network meta-analysis using aggregate level matching: Assessing the impact

Increasingly, single-armed evidence is included in health technology assessment submissions when companies are seeking reimbursement for new drugs. While it is recognized that randomized controlled trials provide a higher standard of evidence, these are not available for many new agents that have been granted licenses in recent years. Therefore, it is important to examine whether alternative strategies for assessing this evidence may be used. In this work, we examine approaches to incorporating single-armed evidence formally in the evaluation process. We consider matching aggregate level covariates to comparator arms or trials and including this evidence in a network meta-analysis. We consider two methods of matching: (i) we include the chosen matched arm in the data set itself as a comparator for the single-arm trial; (ii) we use the baseline odds of an event in a chosen matched trial to use as a plug-in estimator for the single-arm trial. We illustrate that the synthesis of evidence resulting from such a setup is sensitive to the between-study variability, formulation of the prior for the between-design effect, weight given to the single-arm evidence, and extent of the bias in single-armed evidence. We provide a flowchart for the process involved in such a synthesis and highlight additional sensitivity analyses that should be carried out. This work was motivated by a hepatitis C data set, where many agents have only been examined in single-arm studies. We present the results of our methods applied to this data set.

Complications Related to the Initial Trocar Insertion of 3 Different Techniques: A Systematic Review and Meta-analysis

This systematic review aimed to investigate complications related to initial trocar insertion among 3 different laparoscopic techniques: Veress needle (VN) entry, direct trocar entry (DTE), and open entry (OE). A literature search was completed, and complications were assessed. Major vessel injury, gastrointestinal injury, and solid organ injury were defined as major complications. Minor complications were defined as subcutaneous emphysema, extraperitoneal insufflation, omental emphysema, trocar site bleeding, and trocar site infection. Arm-based network meta-analyses were performed to identify the differences in complications among the 3 techniques. Seventeen studies were included in the quantitative analysis. DTE resulted in fewer major complications when compared with VN entry although the difference was not significant (p = .23) as well as significantly fewer minor complications (p < .001). There were no significant differences in minor complications when comparing OE and DTE (p = .74). Fewer major complications were observed with OE compared with VN entry although the difference was not significant (p = .31). There were significantly fewer minor complications for patients who underwent OE (p = .01). DTE patients experienced the least number of minor complications followed by VN entry and OE. In conclusion, major complications are extremely rare, and all 3 insertion methods can be performed without mortality.

Multivariate network meta-analysis to mitigate the effects of outcome reporting bias

Outcome reporting bias (ORB) is recognized as a threat to the validity of both pairwise and network meta-analysis (NMA). In recent years, multivariate meta-analytic methods have been proposed to reduce the impact of ORB in the pairwise setting. These methods have shown that multivariate meta-analysis can reduce bias and increase efficiency of pooled effect sizes. However, it is unknown whether multivariate NMA (MNMA) can similarly reduce the impact of ORB. Additionally, it is quite challenging to implement MNMA due to the fact that correlation between treatments and outcomes must be modeled; thus, the dimension of the covariance matrix and number of components to estimate grows quickly with the number of treatments and number of outcomes. To determine whether MNMA can reduce the effects of ORB on pooled treatment effect sizes, we present an extensive simulation study of Bayesian MNMA. Via simulation studies, we show that MNMA reduces the bias of pooled effect sizes under a variety of outcome missingness scenarios, including missing at random and missing not at random. Further, MNMA improves the precision of estimates, producing narrower credible intervals. We demonstrate the applicability of the approach via application of MNMA to a multi-treatment systematic review of randomized controlled trials of anti-depressants for the treatment of depression in older adults.

Mapping the characteristics of network meta-analyses on drug therapy: A systematic review

BACKGROUND

Network meta-analysis (NMA) is a new tool developed to overcome some limitations of pairwise meta-analyses. NMAs provide evidence on more than two comparators simultaneously. This study aimed to map the characteristics of the published NMAs on drug therapy comparisons.

METHODS

A systematic review of NMAs comparing pharmacological interventions was performed. Searches in Medline (PubMed) and Scopus along with manual searches were conducted. The main characteristics of NMAs were systematically collected: publication metadata, criteria for drug inclusion, statistical methods used, and elements reported. A methodological quality score with 25 key elements was created and applied to the included NMAs. To identify potential trends, the median of the publication year distribution was used as a cut-off.

RESULTS

The study identified 365 NMAs published from 2003 to 2016 in more than 30 countries. Randomised controlled trials were the primary source of data, with only 5% including observational studies, and 230 NMAs used a placebo as a comparator. Less than 15% of NMAs were registered in PROSPERO or a similar system. One third of studies followed PRISMA and less than 9% Cochrane recommendations. Around 30% presented full-search strategies of the systematic review, and 146 NMAs stated the selection criteria for drug inclusion. Over 75% of NMAs presented network plots, but only half described their geometry. Statistical parameters (model fit, inconsistency, convergence) were properly reported by one third of NMAs. Although 216 studies exhibited supplemental material, no data set of primary studies was available. The methodological quality score (mean 13·9; SD 3·8) presented a slightly positive trend over the years.

CONCLUSION

The map of the published NMAs emphasises the potential of this tool to gather evidence in healthcare, but it also identified some weaknesses, especially in the report, which limits its transparency and reproducibility.