When does the placebo effect have an impact on network meta-analysis results?

The placebo effect is the 'effect of the simulation of treatment that occurs due to a participant's belief or expectation that a treatment is effective'. Although the effect might be of little importance for some conditions, it can have a great role in others, mostly when the evaluated symptoms are subjective. Several characteristics that include informed consent, number of arms in a study, the occurrence of adverse events and quality of blinding may influence response to placebo and possibly bias the results of randomised controlled trials. Such a bias is inherited in systematic reviews of evidence and their quantitative components, pairwise meta-analysis (when two treatments are compared) and network meta-analysis (when more than two treatments are compared). In this paper, we aim to provide red flags as to when a placebo effect is likely to bias pairwise and network meta-analysis treatment effects. The classic paradigm has been that placebo-controlled randomised trials are focused on estimating the treatment effect. However, the magnitude of placebo effect itself may also in some instances be of interest and has also lately received attention. We use component network meta-analysis to estimate placebo effects. We apply these methods to a published network meta-analysis, examining the relative effectiveness of four psychotherapies and four control treatments for depression in 123 studies.

Evaluating agreement between evidence from randomised controlled trials and cohort studies in nutrition: a meta-research replication study

This meta-research study aims to evaluate the agreement of effect estimates between bodies of evidence (BoE) from RCTs and cohort studies included in the same nutrition evidence synthesis, to identify factors associated with disagreement, and to replicate the findings of a previous study. We searched Medline, Epistemonikos and the Cochrane Database of Systematic Reviews for nutrition systematic reviews that included both RCTs and cohort studies for the same patient-relevant outcome or intermediate-disease marker. We rated similarity of PI/ECO (population, intervention/exposure, comparison, outcome) between BoE from RCTs and cohort studies. Agreement of effect estimates across BoE was analysed by pooling ratio of risk ratios (RRR) for binary outcomes and difference of standardised mean differences (DSMD) for continuous outcomes. We performed subgroup and sensitivity analyses to explore determinants associated with disagreements. We included 82 BoE-pairs from 51 systematic reviews. For binary outcomes, the RRR was 1.04 (95% confidence interval (CI) 0.99 to 1.10, I2 = 59%, τ2 = 0.02, prediction interval (PI) 0.77 to 1.41). For continuous outcomes, the pooled DSMD was  - 0.09 (95% CI  - 0.26 to 0.09, PI  - 0.55 to 0.38). Subgroup analyses yielded that differences in type of intake/exposure were drivers towards disagreement. We replicated the findings of a previous study, where on average RCTs and cohort studies had similar effect estimates. Disagreement and wide prediction intervals were mainly driven by PI/ECO-dissimilarities. More research is needed to explore other potentially influencing factors (e.g. risk of bias) on the disagreement between effect estimates of both BoE.Trial registration: CRD42021278908.

Long-term efficacy of antipsychotic drugs in initially acutely ill adults with schizophrenia: systematic review and network meta-analysis

Most acute phase antipsychotic drug trials in schizophrenia last only a few weeks, but patients must usually take these drugs much longer. We examined the long-term efficacy of antipsychotic drugs in acutely ill patients using network meta-analysis. We searched the Cochrane Schizophrenia Group register up to March 6, 2022 for randomized, blinded trials of at least 6-month duration on all second-generation and 18 first-generation antipsychotics. The primary outcome was change in overall symptoms of schizophrenia; secondary outcomes were all-cause discontinuation; change in positive, negative and depressive symptoms; quality of life, social functioning, weight gain, antiparkinson medication use, akathisia, serum prolactin level, QTc prolongation, and sedation. Confidence in the results was assessed by the CINeMA (Confidence in Network Meta-Analysis) framework. We included 45 studies with 11,238 participants. In terms of overall symptoms, olanzapine was on average more efficacious than ziprasidone (standardized mean difference, SMD=0.37, 95% CI: 0.26-0.49), asenapine (SMD=0.33, 95% CI: 0.21-0.45), iloperidone (SMD=0.32, 95% CI: 0.15-0.49), paliperidone (SMD=0.28, 95% CI: 0.11-0.44), haloperidol (SMD=0.27, 95% CI: 0.14-0.39), quetiapine (SMD=0.25, 95% CI: 0.12-0.38), aripiprazole (SMD=0.16, 95% CI: 0.04-0.28) and risperidone (SMD=0.12, 95% CI: 0.03-0.21). The 95% CIs for olanzapine versus aripiprazole and risperidone included the possibility of trivial effects. The differences between olanzapine and lurasidone, amisulpride, perphenazine, clozapine and zotepine were either small or uncertain. These results were robust in sensitivity analyses and in line with other efficacy outcomes and all-cause discontinuation. Concerning weight gain, the impact of olanzapine was higher than all other antipsychotics, with a mean difference ranging from -4.58 kg (95% CI: -5.33 to -3.83) compared to ziprasidone to -2.30 kg (95% CI: -3.35 to -1.25) compared to amisulpride. Our data suggest that olanzapine is more efficacious than a number of other antipsychotic drugs in the longer term, but its efficacy must be weighed against its side effect profile.

Considering multiple outcomes with different weights informed the hierarchy of interventions in network meta-analysis

OBJECTIVES

Ranking metrics in network meta-analysis (NMA) are computed separately for each outcome. Our aim is to 1) present graphical ways to group competing interventions considering multiple outcomes and 2) use conjoint analysis for placing weights on the various outcomes based on the stakeholders' preferences.

STUDY DESIGN AND SETTING

We used multidimensional scaling (MDS) and hierarchical tree clustering to visualize the extent of similarity of interventions in terms of the relative effects they produce through a random effect NMA. We reanalyzed a published network of 212 psychosis trials taking three outcomes into account as follows: reduction in symptoms of schizophrenia, all-cause treatment discontinuation, and weight gain.

RESULTS

Conjoint analysis provides a mathematical method to transform judgements into weights that can be subsequently used to visually represent interventions on a two-dimensional plane or through a dendrogram. These plots provide insightful information about the clustering of interventions.

CONCLUSION

Grouping interventions can help decision makers not only to identify the optimal ones in terms of benefit-risk balance but also choose one from the best cluster based on other grounds, such as cost, implementation etc. Placing weights on outcomes allows considering patient profile or preferences.

Introducing the Treatment Hierarchy Question in Network Meta-Analysis

Comparative effectiveness research using network meta-analysis can present a hierarchy of competing treatments, from the most to the least preferable option. However, in published reviews, the research question associated with the hierarchy of multiple interventions is typically not clearly defined. Here we introduce the novel notion of a treatment hierarchy question that describes the criterion for choosing a specific treatment over one or more competing alternatives. For example, stakeholders might ask which treatment is most likely to improve mean survival by at least 2 years, or which treatment is associated with the longest mean survival. We discuss the most commonly used ranking metrics (quantities that compare the estimated treatment-specific effects), how the ranking metrics produce a treatment hierarchy, and the type of treatment hierarchy question that each ranking metric can answer. We show that the ranking metrics encompass the uncertainty in the estimation of the treatment effects in different ways, which results in different treatment hierarchies. When using network meta-analyses that aim to rank treatments, investigators should state the treatment hierarchy question they aim to address and employ the appropriate ranking metric to answer it. Following this new proposal will avoid some controversies that have arisen in comparative effectiveness research.

Network meta-analysis and random walks

Network meta-analysis (NMA) is a central tool for evidence synthesis in clinical research. The results of an NMA depend critically on the quality of evidence being pooled. In assessing the validity of an NMA, it is therefore important to know the proportion contributions of each direct treatment comparison to each network treatment effect. The construction of proportion contributions is based on the observation that each row of the hat matrix represents a so-called "evidence flow network" for each treatment comparison. However, the existing algorithm used to calculate these values is associated with ambiguity according to the selection of paths. In this article, we present a novel analogy between NMA and random walks. We use this analogy to derive closed-form expressions for the proportion contributions. A random walk on a graph is a stochastic process that describes a succession of random "hops" between vertices which are connected by an edge. The weight of an edge relates to the probability that the walker moves along that edge. We use the graph representation of NMA to construct the transition matrix for a random walk on the network of evidence. We show that the net number of times a walker crosses each edge of the network is related to the evidence flow network. By then defining a random walk on the directed evidence flow network, we derive analytically the matrix of proportion contributions. The random-walk approach has none of the associated ambiguity of the existing algorithm.

Answering complex hierarchy questions in network meta-analysis

BACKGROUND

Network meta-analysis estimates all relative effects between competing treatments and can produce a treatment hierarchy from the most to the least desirable option according to a health outcome. While about half of the published network meta-analyses present such a hierarchy, it is rarely the case that it is related to a clinically relevant decision question.

METHODS

We first define treatment hierarchy and treatment ranking in a network meta-analysis and suggest a simulation method to estimate the probability of each possible hierarchy to occur. We then propose a stepwise approach to express clinically relevant decision questions as hierarchy questions and quantify the uncertainty of the criteria that constitute them. The steps of the approach are summarized as follows: a) a question of clinical relevance is defined, b) the hierarchies that satisfy the defined question are collected and c) the frequencies of the respective hierarchies are added; the resulted sum expresses the certainty of the defined set of criteria to hold. We then show how the frequencies of all possible hierarchies relate to common ranking metrics.

RESULTS

We exemplify the method and its implementation using two networks. The first is a network of four treatments for chronic obstructive pulmonary disease where the most probable hierarchy has a frequency of 28%. The second is a network of 18 antidepressants, among which Vortioxetine, Bupropion and Escitalopram occupy the first three ranks with frequency 19%.

CONCLUSIONS

The developed method offers a generalised approach of producing treatment hierarchies in network meta-analysis, which moves towards attaching treatment ranking to a clear decision question, relevant to all or a subset of competing treatments.

Do reporting guidelines have an impact? Empirical assessment of changes in reporting before and after the PRISMA extension statement for network meta-analysis

BACKGROUND

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension statement for network meta-analysis (NMA) published in 2015 promotes comprehensive reporting in published systematic reviews with NMA. PRISMA-NMA includes 32 items: 27 core items as indicated in the 2009 PRISMA Statement and five items specific to the reporting of NMAs. Although NMA reporting is improving, it is unclear whether PRISMA-NMA has accelerated this improvement. We aimed to investigate the impact of PRISMA-NMA and highlight key items that require attention and improvement.

METHODS

We updated our previous collection of NMAs with articles published between April 2015 and July 2018. We assessed the completeness of reporting for each NMA, including main manuscript and online supplements, using the PRISMA-NMA checklist. The PRISMA-NMA checklist originally includes 32 total items (i.e. a 32-point scale original PRISMA-NMA score). We also prepared a modified version of the PRISMA-NMA checklist with 49 items to evaluate separately at a more granular level all multiple-content items (i.e. a 49-point scale modified PRISMA-NMA score). We compared average reporting scores of articles published until and after 2015.

RESULTS

In the 1144 included NMAs the mean modified PRISMA-NMA score was 32.1 (95% CI 31.8-32.4) of a possible 49-excellence-score. For 1-year increase, the mean modified score increased by 0.96 (95% CI 0.32 to 1.59) for 389 NMAs published until 2015 and by 0.53 (95% CI 0.02 to 1.04) for 755 NMAs published after 2015. The mean modified PRISMA-NMA score for NMAs published after 2015 was higher by 0.81 (95% CI 0.23 to 1.39) compared to before 2015 when adjusting for journal impact factor, type of review, funding, and treatment category. Description of summary effect sizes to be used, presentation of individual study data, sources of funding for the systematic review, and role of funders dropped in frequency after 2015 by 6-16%.

CONCLUSIONS

NMAs published after 2015 more frequently reported the five items associated with NMA compared to those published until 2015. However, improvement in reporting after 2015 is compatible with that observed on a yearly basis until 2015, and hence, it could not be attributed solely to the publication of the PRISMA-NMA.

Meta-analysis as a system of springs

Meta-analysis results are usually presented in forest plots, which show the individual study results and the summary effect along with their confidence intervals. In this paper, we propose a system of linear springs as a mechanical analogue of meta-analysis that enables visualization and enhances intuition. The length of a spring corresponds to a study treatment effect and the stiffness of the spring corresponds to its inverse variance. To synthesize study springs we use two main operations: connection in parallel and connection in series. We show the equivalence between meta-analysis and linear springs for fixed effect and random effects pairwise meta-analysis and we also derive indirect treatment effects. We use examples to illustrate the different meta-analytical schemes using the corresponding system of springs. The proposed visualization can serve as an educational tool, especially useful for researchers with no statistical background. The analogy between meta-analysis and springs facilitates intuition for notions such as heterogeneity and the differences between fixed and random effects meta-analysis.

Antidepressant treatment in patients following acute coronary syndromes: a systematic review and Bayesian meta-analysis

AIMS

The aim of this study is to investigate the effect of antidepressant therapy on mortality and cardiovascular outcomes in patients with acute coronary syndrome (ACS).

METHODS AND RESULTS

We systematically searched PubMed, Embase, and the Cochrane Central Register of Controlled Trials and performed a Bayesian random-effects meta-analysis of randomized controlled trials that investigated antidepressant pharmacotherapy in patients following ACS. The primary outcome was all-cause mortality. Secondary outcomes were repeat hospitalizations and recurrent myocardial infarctions (MIs). Ten randomized controlled trials with a total of 1935 patients qualified for inclusion. Selective serotonin reuptake inhibitors were investigated in six, bupropion in three, and mirtazapine in one trial. Placebo was used as control in eight trials. There was no difference in all-cause mortality [odds ratio (OR) 0.97, 95% credible interval (CrI) 0.66-1.42] and recurrent MI (OR 0.64, 95% CrI 0.40-1.02) between patients receiving antidepressants compared with controls, whereas antidepressant therapy was associated with less repeat hospitalizations (OR 0.62, 95% CrI 0.40-0.94). In patients with ACS and concomitant depression, antidepressants reduced the odds of recurrent MI compared with usual care/placebo (OR 0.45, 95% CrI 0.25-0.81). Extended funnel plots suggest robustness of the observations.

CONCLUSIONS

Antidepressants in patients following ACS have no effect on mortality but reduce repeat hospitalizations; in patients with depression, there is a reduced risk of recurrent MI with antidepressant therapy.

Network meta-analysis results against a fictional treatment of average performance: Treatment effects and ranking metric

BACKGROUND

Network meta-analysis (NMA) produces complex outputs as many comparisons between interventions are of interest. The estimated relative treatment effects are usually displayed in a forest plot or in a league table and several ranking metrics are calculated and presented.

METHODS

In this article, we estimate relative treatment effects of each competing treatment against a fictional treatment of average performance using the "deviation from the means" coding that has been used to parametrize categorical covariates in regression models. We then use this alternative parametrization of the NMA model to present a ranking metric (PreTA: Preferable Than Average) interpreted as the probability that a treatment is better than a fictional treatment of average performance.

RESULTS

We illustrate the alternative parametrization of the NMA model using two networks of interventions, a network of 18 antidepressants for acute depression and a network of four interventions for heavy menstrual bleeding. We also use these two networks to highlight differences among PreTA and existing ranking metrics. We further examine the agreement between PreTA and existing ranking metrics in 232 networks of interventions and conclude that their agreement depends on the precision with which relative effects are estimated.

CONCLUSIONS

A forest plot with NMA relative treatment effects using "deviation from means" coding could complement presentation of NMA results in large networks and in absence of an obvious reference treatment. PreTA is a viable alternative to existing probabilistic ranking metrics that naturally incorporates uncertainty.

Comparative Efficacy and Tolerability of 32 Oral Antipsychotics for the Acute Treatment of Adults With Multi-Episode Schizophrenia: A Systematic Review and Network Meta-Analysis

(Reprinted with permission from the Lancet 2019; 394: 939-51).

Evaluation of Cumulative Meta-analysis of Rare Events as a Tool for Clinical Trials Safety Monitoring

This meta-analysis evaluates the use of cumulative meta-analysis of rare events as a tool for safety monitoring of clinical trials using the example of coronary bioresorbable vascular scaffold–associated thrombosis.

Agreement between ranking metrics in network meta-analysis: an empirical study

OBJECTIVE

To empirically explore the level of agreement of the treatment hierarchies from different ranking metrics in network meta-analysis (NMA) and to investigate how network characteristics influence the agreement.

DESIGN

Empirical evaluation from re-analysis of NMA.

DATA

232 networks of four or more interventions from randomised controlled trials, published between 1999 and 2015.

METHODS

We calculated treatment hierarchies from several ranking metrics: relative treatment effects, probability of producing the best value [Formula: see text] and the surface under the cumulative ranking curve (SUCRA). We estimated the level of agreement between the treatment hierarchies using different measures: Kendall's τ and Spearman's ρ correlation; and the Yilmaz [Formula: see text] and Average Overlap, to give more weight to the top of the rankings. Finally, we assessed how the amount of the information present in a network affects the agreement between treatment hierarchies, using the average variance, the relative range of variance and the total sample size over the number of interventions of a network.

RESULTS

Overall, the pairwise agreement was high for all treatment hierarchies obtained by the different ranking metrics. The highest agreement was observed between SUCRA and the relative treatment effect for both correlation and top-weighted measures whose medians were all equal to 1. The agreement between rankings decreased for networks with less precise estimates and the hierarchies obtained from [Formula: see text] appeared to be the most sensitive to large differences in the variance estimates. However, such large differences were rare.

CONCLUSIONS

Different ranking metrics address different treatment hierarchy problems, however they produced similar rankings in the published networks. Researchers reporting NMA results can use the ranking metric they prefer, unless there are imprecise estimates or large imbalances in the variance estimates. In this case treatment hierarchies based on both probabilistic and non-probabilistic ranking metrics should be presented.

The statistical importance of a study for a network meta-analysis estimate

Background

In pairwise meta-analysis, the contribution of each study to the pooled estimate is given by its weight, which is based on the inverse variance of the estimate from that study. For network meta-analysis (NMA), the contribution of direct (and indirect) evidence is easily obtained from the diagonal elements of a hat matrix. It is, however, not fully clear how to generalize this to the percentage contribution of each study to a NMA estimate.

Methods

We define the importance of each study for a NMA estimate by the reduction of the estimate’s variance when adding the given study to the others. An equivalent interpretation is the relative loss in precision when the study is left out. Importances are values between 0 and 1. An importance of 1 means that the study is an essential link of the pathway in the network connecting one of the treatments with another.

Results

Importances can be defined for two-stage and one-stage NMA. These numbers in general do not add to one and thus cannot be interpreted as ‘percentage contributions’. After briefly discussing other available approaches, we question whether it is possible to obtain unique percentage contributions for NMA.

Conclusions

Importances generalize the concept of weights in pairwise meta-analysis in a natural way. Moreover, they are uniquely defined, easily calculated, and have an intuitive interpretation. We give some real examples for illustration.

CINeMA: An approach for assessing confidence in the results of a network meta-analysis

BACKGROUND

The evaluation of the credibility of results from a meta-analysis has become an important part of the evidence synthesis process. We present a methodological framework to evaluate confidence in the results from network meta-analyses, Confidence in Network Meta-Analysis (CINeMA), when multiple interventions are compared.

METHODOLOGY

CINeMA considers 6 domains: (i) within-study bias, (ii) reporting bias, (iii) indirectness, (iv) imprecision, (v) heterogeneity, and (vi) incoherence. Key to judgments about within-study bias and indirectness is the percentage contribution matrix, which shows how much information each study contributes to the results from network meta-analysis. The contribution matrix can easily be computed using a freely available web application. In evaluating imprecision, heterogeneity, and incoherence, we consider the impact of these components of variability in forming clinical decisions.

CONCLUSIONS

Via 3 examples, we show that CINeMA improves transparency and avoids the selective use of evidence when forming judgments, thus limiting subjectivity in the process. CINeMA is easy to apply even in large and complicated networks.

Comparison of dietary macronutrient patterns of 14 popular named dietary programmes for weight and cardiovascular risk factor reduction in adults: systematic review and network meta-analysis of randomised trials

OBJECTIVE

To determine the relative effectiveness of dietary macronutrient patterns and popular named diet programmes for weight loss and cardiovascular risk factor improvement among adults who are overweight or obese.

DESIGN

Systematic review and network meta-analysis of randomised trials.

DATA SOURCES

Medline, Embase, CINAHL, AMED, and CENTRAL from database inception until September 2018, reference lists of eligible trials, and related reviews.

STUDY SELECTION

Randomised trials that enrolled adults (≥18 years) who were overweight (body mass index 25-29) or obese (≥30) to a popular named diet or an alternative diet.

OUTCOMES AND MEASURES

Change in body weight, low density lipoprotein (LDL) cholesterol, high density lipoprotein (HDL) cholesterol, systolic blood pressure, diastolic blood pressure, and C reactive protein at the six and 12 month follow-up.

REVIEW METHODS

Two reviewers independently extracted data on study participants, interventions, and outcomes and assessed risk of bias, and the certainty of evidence using the GRADE (grading of recommendations, assessment, development, and evaluation) approach. A bayesian framework informed a series of random effects network meta-analyses to estimate the relative effectiveness of the diets.

RESULTS

121 eligible trials with 21 942 patients were included and reported on 14 named diets and three control diets. Compared with usual diet, low carbohydrate and low fat diets had a similar effect at six months on weight loss (4.63 v 4.37 kg, both moderate certainty) and reduction in systolic blood pressure (5.14 mm Hg, moderate certainty v 5.05 mm Hg, low certainty) and diastolic blood pressure (3.21 v 2.85 mm Hg, both low certainty). Moderate macronutrient diets resulted in slightly less weight loss and blood pressure reductions. Low carbohydrate diets had less effect than low fat diets and moderate macronutrient diets on reduction in LDL cholesterol (1.01 mg/dL, low certainty v 7.08 mg/dL, moderate certainty v 5.22 mg/dL, moderate certainty, respectively) but an increase in HDL cholesterol (2.31 mg/dL, low certainty), whereas low fat (-1.88 mg/dL, moderate certainty) and moderate macronutrient (-0.89 mg/dL, moderate certainty) did not. Among popular named diets, those with the largest effect on weight reduction and blood pressure in comparison with usual diet were Atkins (weight 5.5 kg, systolic blood pressure 5.1 mm Hg, diastolic blood pressure 3.3 mm Hg), DASH (3.6 kg, 4.7 mm Hg, 2.9 mm Hg, respectively), and Zone (4.1 kg, 3.5 mm Hg, 2.3 mm Hg, respectively) at six months (all moderate certainty). No diets significantly improved levels of HDL cholesterol or C reactive protein at six months. Overall, weight loss diminished at 12 months among all macronutrient patterns and popular named diets, while the benefits for cardiovascular risk factors of all interventions, except the Mediterranean diet, essentially disappeared.

CONCLUSIONS

Moderate certainty evidence shows that most macronutrient diets, over six months, result in modest weight loss and substantial improvements in cardiovascular risk factors, particularly blood pressure. At 12 months the effects on weight reduction and improvements in cardiovascular risk factors largely disappear.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42015027929.

Extensions of the probabilistic ranking metrics of competing treatments in network meta-analysis to reflect clinically important relative differences on many outcomes

One of the key features of network meta-analysis is ranking of interventions according to outcomes of interest. Ranking metrics are prone to misinterpretation because of two limitations associated with the current ranking methods. First, differences in relative treatment effects might not be clinically important and this is not reflected in the ranking metrics. Second, there are no established methods to include several health outcomes in the ranking assessments. To address these two issues, we extended the P-score method to allow for multiple outcomes and modified it to measure the mean extent of certainty that a treatment is better than the competing treatments by a certain amount, for example, the minimum clinical important difference. We suggest to present the tradeoff between beneficial and harmful outcomes allowing stakeholders to consider how much adverse effect they are willing to tolerate for specific gains in efficacy. We used a published network of 212 trials comparing 15 antipsychotics and placebo using a random effects network meta-analysis model, focusing on three outcomes; reduction in symptoms of schizophrenia in a standardized scale, all-cause discontinuation, and weight gain.

CINeMA: Software for semiautomated assessment of the confidence in the results of network meta-analysis

Network meta-analysis (NMA) compares several interventions that are linked in a network of comparative studies and estimates the relative treatment effects between all treatments, using both direct and indirect evidence. NMA is increasingly used for decision making in health care, however, a user-friendly system to evaluate the confidence that can be placed in the results of NMA is currently lacking. This paper is a tutorial describing the Confidence In Network Meta-Analysis (CINeMA) web application, which is based on the framework developed by Salanti et al (2014, PLOS One, 9, e99682) and refined by Nikolakopoulou et al (2019, bioRxiv). Six domains that affect the level of confidence in the NMA results are considered: (a) within-study bias, (b) reporting bias, (c) indirectness, (d) imprecision, (e) heterogeneity, and (f) incoherence. CINeMA is freely available and open-source and no login is required. In the configuration step users upload their data, produce network plots and define the analysis and effect measure. The dataset should include assessments of study-level risk of bias and judgments on indirectness. CINeMA calls the netmeta routine in R to estimate relative effects and heterogeneity. Users are then guided through a systematic evaluation of the six domains. In this way reviewers assess the level of concerns for each relative treatment effect from NMA as giving rise to "no concerns," "some concerns," or "major concerns" in each of the six domains, which are graphically summarized on the report page for all effect estimates. Finally, judgments across the domains are summarized into a single confidence rating ("high," "moderate," "low," or "very low"). In conclusion, the user-friendly web-based CINeMA platform provides a transparent framework to evaluate evidence from systematic reviews with multiple interventions.

Comparative efficacy and tolerability of 32 oral antipsychotics for the acute treatment of adults with multi-episode schizophrenia: a systematic review and network meta-analysis

BACKGROUND

Schizophrenia is one of the most common, burdensome, and costly psychiatric disorders in adults worldwide. Antipsychotic drugs are its treatment of choice, but there is controversy about which agent should be used. We aimed to compare and rank antipsychotics by quantifying information from randomised controlled trials.

METHODS

We did a network meta-analysis of placebo-controlled and head-to-head randomised controlled trials and compared 32 antipsychotics. We searched Embase, MEDLINE, PsycINFO, PubMed, BIOSIS, Cochrane Central Register of Controlled Trials (CENTRAL), WHO International Clinical Trials Registry Platform, and ClinicalTrials.gov from database inception to Jan 8, 2019. Two authors independently selected studies and extracted data. We included randomised controlled trials in adults with acute symptoms of schizophrenia or related disorders. We excluded studies in patients with treatment resistance, first episode, predominant negative or depressive symptoms, concomitant medical illnesses, and relapse-prevention studies. Our primary outcome was change in overall symptoms measured with standardised rating scales. We also extracted data for eight efficacy and eight safety outcomes. Differences in the findings of the studies were explored in metaregressions and sensitivity analyses. Effect size measures were standardised mean differences, mean differences, or risk ratios with 95% credible intervals (CrIs). Confidence in the evidence was assessed using CINeMA (Confidence in Network Meta-Analysis). The study protocol is registered with PROSPERO, number CRD42014014919.

FINDINGS

We identified 54 417 citations and included 402 studies with data for 53 463 participants. Effect size estimates suggested all antipsychotics reduced overall symptoms more than placebo (although not statistically significant for six drugs), with standardised mean differences ranging from -0·89 (95% CrI -1·08 to -0·71) for clozapine to -0·03 (-0·59 to 0·52) for levomepromazine (40 815 participants). Standardised mean differences compared with placebo for reduction of positive symptoms (31 179 participants) varied from -0·69 (95% CrI -0·86 to -0·52) for amisulpride to -0·17 (-0·31 to -0·04) for brexpiprazole, for negative symptoms (32 015 participants) from -0·62 (-0·84 to -0·39; clozapine) to -0·10 (-0·45 to 0·25; flupentixol), for depressive symptoms (19 683 participants) from -0·90 (-1·36 to -0·44; sulpiride) to 0·04 (-0·39 to 0·47; flupentixol). Risk ratios compared with placebo for all-cause discontinuation (42 672 participants) ranged from 0·52 (0·12 to 0·95; clopenthixol) to 1·15 (0·36 to 1·47; pimozide), for sedation (30 770 participants) from 0·92 (0·17 to 2·03; pimozide) to 10·20 (4·72 to 29·41; zuclopenthixol), for use of antiparkinson medication (24 911 participants) from 0·46 (0·19 to 0·88; clozapine) to 6·14 (4·81 to 6·55; pimozide). Mean differences compared to placebo for weight gain (28 317 participants) ranged from -0·16 kg (-0·73 to 0·40; ziprasidone) to 3·21 kg (2·10 to 4·31; zotepine), for prolactin elevation (21 569 participants) from -77·05 ng/mL (-120·23 to -33·54; clozapine) to 48·51 ng/mL (43·52 to 53·51; paliperidone) and for QTc prolongation (15 467 participants) from -2·21 ms (-4·54 to 0·15; lurasidone) to 23·90 ms (20·56 to 27·33; sertindole). Conclusions for the primary outcome did not substantially change after adjusting for possible effect moderators or in sensitivity analyses (eg, when excluding placebo-controlled studies). The confidence in evidence was often low or very low.

INTERPRETATION

There are some efficacy differences between antipsychotics, but most of them are gradual rather than discrete. Differences in side-effects are more marked. These findings will aid clinicians in balancing risks versus benefits of those drugs available in their countries. They should consider the importance of each outcome, the patients' medical problems, and preferences.

FUNDING

German Ministry of Education and Research and National Institute for Health Research.

Synthesizing existing evidence to design future trials: survey of methodologists from European institutions

Background

‘Conditional trial design’ is a framework for efficiently planning new clinical trials based on a network of relevant existing trials. The framework considers whether new trials are required and how the existing evidence can be used to answer the research question and plan future research. The potential of this approach has not been fully realized.

Methods

We conducted an online survey among trial statisticians, methodologists, and users of evidence synthesis research using referral sampling to capture opinions about the conditional trial design framework and current practices among clinical researchers. The questions included in the survey were related to the decision of whether a meta-analysis answers the research question, the optimal way to synthesize available evidence, which relates to the acceptability of network meta-analysis, and the use of evidence synthesis in the planning of new studies.

Results

In total, 76 researchers completed the survey. Two out of three survey participants (65%) were willing to possibly or definitely consider using evidence synthesis to design a future clinical trial and around half of the participants would give priority to such a trial design. The median rating of the frequency of using such a trial design was 0.41 on a scale from 0 (never) to 1 (always). Major barriers to adopting conditional trial design include the current regulatory paradigm and the policies of funding agencies and sponsors.

Conclusions

Participants reported moderate interest in using evidence synthesis methods in the design of future trials. They indicated that a major paradigm shift is required before the use of network meta-analysis is regularly employed in the design of trials.

Electronic supplementary material

The online version of this article (10.1186/s13063-019-3449-6) contains supplementary material, which is available to authorized users.

Transcatheter aortic valve implantation vs. surgical aortic valve replacement for treatment of symptomatic severe aortic stenosis: an updated meta-analysis

Aims 

Owing to new evidence from randomized controlled trials (RCTs) in low-risk patients with severe aortic stenosis, we compared the collective safety and efficacy of transcatheter aortic valve implantation (TAVI) vs. surgical aortic valve replacement (SAVR) across the entire spectrum of surgical risk patients.

Methods and results 

The meta-analysis is registered with PROSPERO (CRD42016037273). We identified RCTs comparing TAVI with SAVR in patients with severe aortic stenosis reporting at different follow-up periods. We extracted trial, patient, intervention, and outcome characteristics following predefined criteria. The primary outcome was all-cause mortality up to 2 years for the main analysis. Seven trials that randomly assigned 8020 participants to TAVI (4014 patients) and SAVR (4006 patients) were included. The combined mean STS score in the TAVI arm was 9.4%, 5.1%, and 2.0% for high-, intermediate-, and low surgical risk trials, respectively. Transcatheter aortic valve implantation was associated with a significant reduction of all-cause mortality compared to SAVR {hazard ratio [HR] 0.88 [95% confidence interval (CI) 0.78–0.99], P = 0.030}; an effect that was consistent across the entire spectrum of surgical risk (P-for-interaction = 0.410) and irrespective of type of transcatheter heart valve (THV) system (P-for-interaction = 0.674). Transcatheter aortic valve implantation resulted in lower risk of strokes [HR 0.81 (95% CI 0.68–0.98), P = 0.028]. Surgical aortic valve replacement was associated with a lower risk of major vascular complications [HR 1.99 (95% CI 1.34–2.93), P = 0.001] and permanent pacemaker implantations [HR 2.27 (95% CI 1.47–3.64), P < 0.001] compared to TAVI.

Conclusion 

Compared with SAVR, TAVI is associated with reduction in all-cause mortality and stroke up to 2 years irrespective of baseline surgical risk and type of THV system.

How Many Patients With Schizophrenia Do Not Respond to Antipsychotic Drugs in the Short Term? An Analysis Based on Individual Patient Data From Randomized Controlled Trials

OBJECTIVE

An important clinical question is how many patients with acute schizophrenia do not respond to antipsychotics despite being treated for adequate time and with an effective dose. However, up to date, the exact extent of the phenomenon remains unclear.

METHODS

We calculated the nonresponse and nonremission percentages using individual patient data from 16 randomized controlled trials (RCTs). Six thousand two hundred twenty-one patients were assigned to one antipsychotic (amisulpride, flupenthixol, haloperidol, olanzapine, quetiapine, risperidone, or ziprasidone) at an adequate dose; the response was assessed at 4-6 weeks. As various definitions of nonresponse have been used in the literature, we applied 4 different cut-offs covering the whole range of percent Positive and Negative Syndrome Scale (PANSS)/Brief Psychiatric Rating Scale (BPRS) reduction (≤0%, <25%, <50%, <75%).For symptomatic remission, we used the definition proposed by Andreasen without employing the time criterion.

RESULTS

The overall nonresponse for the cut-off of ≤0% PANSS/BPRS reduction was 19.8% (18.8%-20.8%); for the cut-off of <25% reduction it was 43% (41.7%-44.3%); for the cut-off of <50% reduction it was 66.5% (65.3%-67.8%); and for the cut-off of <75% reduction it was 87% (86%-87.9%). The overall percentage of no symptomatic remission was 66.9% (65.7%-68.1%). Earlier onset of illness, lower baseline severity and the antipsychotic used were significantly associated with higher nonresponse percentages.

CONCLUSIONS

Nonresponse and nonremission percentages were notably high. Nevertheless, the patients in our analysis could represent a negative selection since they came from short-term RCTs and could have been treated before study inclusion; thus, further response may not have been observed. Observational studies on this important question are needed.

Antipsychotic drugs for patients with schizophrenia and predominant or prominent negative symptoms: a systematic review and meta-analysis

BACKGROUND

Negative symptoms are the core of schizophrenia, but whether antipsychotics are efficacious for their treatment is unclear. Moreover, there is debate whether patients in relevant trials should have predominant negative symptoms or whether prominent negative symptoms are also acceptable.

METHODS

We systematically reviewed randomised, blinded antipsychotic drug trials in patients with schizophrenia and either predominant or prominent negative symptoms (last search Dec 12, 2017). Separate pairwise meta-analyses were conducted in these two populations. The primary outcome was negative symptoms. Depressive, symptoms, positive symptoms, and extrapyramidal side-effects were analysed as causes of secondary negative symptoms.

FINDINGS

We included 21 randomized-controlled trials with 3451 participants which revealed the following significant differences in the primary outcome: in patients with predominant negative symptoms amisulpride was superior to placebo (N = 4; n = 590, SMD 0.47, CI 0.23, 0.71), olanzapine was superior to haloperidol in a small trial (n = 35) and cariprazine outperformed risperidone (N = 1, n = 456, SMD - 0.29, CI - 0.48, - 0.11). In patients with prominent negative symptoms, olanzapine and quetiapine were superior to risperidone in single trials. Overall, studies in prominent negative symptoms were potentially more confounded by improvements of secondary negative symptoms.

INTERPRETATION

Amisulpride is the only antipsychotic that outperformed placebo in the treatment of predominant negative symptoms, but there was a parallel reduction of depression. Cariprazine was better than risperidone in a large trial that was well-controlled for secondary negative symptoms, but the trial was sponsored by its manufacturer. Future trials should apply scientifically developed definitions such as the deficit syndrome and the persistent negative symptoms concept.

Comparative efficacy and acceptability of pharmacological treatments for post-traumatic stress disorder in adults: a network meta-analysis

BACKGROUND

Guidelines about post-traumatic stress disorder (PTSD) recommend broad categories of drugs, but uncertainty remains about what pharmacological treatment to select among all available compounds.

METHODS

Cochrane Central Register of Controlled Trials register, MEDLINE, PsycINFO, National PTSD Center Pilots database, PubMed, trial registries, and databases of pharmaceutical companies were searched until February 2016 for double-blind randomised trials comparing any pharmacological intervention or placebo as oral therapy in adults with PTSD. Initially, we performed standard pairwise meta-analyses using a random effects model. We then carried out a network meta-analysis. The main outcome measures were mean change on a standardised scale and all-cause dropout rate. Acute treatment was defined as 8-week follow up.

RESULTS

Desipramine, fluoxetine, paroxetine, phenelzine, risperidone, sertraline, and venlafaxine were more effective than placebo; phenelzine was better than many other active treatments and was the only drug, which was significantly better than placebo in terms of dropouts (odds ratio 7.50, 95% CI 1.72-32.80). Mirtazapine yielded a relatively high rank for efficacy, but the respective value for acceptability was not among the best treatments. Divalproex had overall the worst ranking.

CONCLUSIONS

The efficacy and acceptability hierarchies generated by our study were robust against many sources of bias. The differences between drugs and placebo were small, with the only exception of phenelzine. Considering the small amount of available data, these results are probably not robust enough to suggest phenelzine as a drug of choice. However, findings from this review reinforce the idea that phenelzine should be prioritised in future trials in PTSD.

Client feedback in psychological therapy for children and adolescents with mental health problems

BACKGROUND

Childhood and adolescent mental health problems are a serious and growing concern worldwide. Research suggests that psychotherapy can have a significant and positive impact on children and adolescents with mental health problems, such as anxiety disorders, depression and conduct disorders. Client feedback tools serve as a method of monitoring clients' progress and providing feedback from clients to therapists during the therapeutic process. These tools may help to enhance clinicians' decision-making by allowing them to adapt their treatment plans as the therapy progresses, resulting in a reduction of treatment failures. Research has shown that client feedback tools have a positive effect on adults' psychotherapy. This review addresses whether feedback tools in child and adolescent therapy could help therapists to better treat their young clients.

OBJECTIVES

To assess the effects of client feedback in psychological therapy on child and adolescent mental health outcomes.

SEARCH METHODS

We searched the Cochrane Common Mental Disorders Controlled Trials Register (CCMDCTR, Studies and References), the Cochrane Central Register of Controlled Trials (CENTRAL), Ovid MEDLINE (1946-), Embase (1974-) and PsycINFO (1967-) to 3 April 2018. We did not apply any restriction on date, language or publication status to the search.

SELECTION CRITERIA

We included randomised controlled trials (RCTs) that compared client feedback to no client feedback in psychological therapies for children and adolescents.

DATA COLLECTION AND ANALYSIS

Two review authors independently assessed references for inclusion eligibility and extracted outcome, risk of bias and study characteristics data into customised forms. We contacted study authors to obtain missing data. We analysed dichotomous data using risk ratios (RRs) and calculated their 95% confidence intervals (CIs). For continuous data, we calculated mean differences (MDs), or standardised mean differences (SMDs) if different scales were used to measure the same outcome. We used a random-effects model for all analyses.

MAIN RESULTS

We included six published RCTs, conducted in the USA (5 RCTs) and Israel (1 RCT), with 1097 children and adolescents (11 to 18 years old), in the review.We are very uncertain about the effect of client feedback on improvement of symptoms, as reported by youth in the short term because we considered evidence to be of very low-certainty due to high risk of bias and very serious inconsistency in the effect estimates from the different studies. Similarly, we are very uncertain about the effect of client feedback on treatment acceptability, due to high risk of bias, imprecision in the results, and indirectness of measuring the outcome (RR 1.08, 95% CI 0.73 to 1.61; 2 studies, 237 participants; very low-certainty).Overall, most studies reported and carried out randomisation and allocation concealment adequately. None of the studies were blinded or attempted to blind participants and personnel and were at high risk of performance bias, and only one study had blind outcome assessors. All of the studies were at high or unclear risk of attrition bias mainly due to poor, non-transparent reporting of participants' flow through the studies.

AUTHORS' CONCLUSIONS

Due to the paucity of high-quality data and considerable inconsistency in results from different studies, there is currently insufficient evidence to reach any firm conclusions regarding the role of client feedback in psychological therapies for children and adolescents with mental health problems, and further research on this important topic is needed.Future studies should avoid risks of performance, detection and attrition biases, as seen in the studies included in this review. Studies from countries other than the USA are needed, as well as studies including children younger than 10 years.

Planning a future randomized clinical trial based on a network of relevant past trials

Background

The important role of network meta-analysis of randomized clinical trials in health technology assessment and guideline development is increasingly recognized. This approach has the potential to obtain conclusive results earlier than with new standalone trials or conventional, pairwise meta-analyses.

Methods

Network meta-analyses can also be used to plan future trials. We introduce a four-step framework that aims to identify the optimal design for a new trial that will update the existing evidence while minimizing the required sample size. The new trial designed within this framework does not need to include all competing interventions and comparisons of interest and can contribute direct and indirect evidence to the updated network meta-analysis. We present the method by virtually planning a new trial to compare biologics in rheumatoid arthritis and a new trial to compare two drugs for relapsing-remitting multiple sclerosis.

Results

A trial design based on updating the evidence from a network meta-analysis of relevant previous trials may require a considerably smaller sample size to reach the same conclusion compared with a trial designed and analyzed in isolation. Challenges of the approach include the complexity of the methodology and the need for a coherent network meta-analysis of previous trials with little heterogeneity.

Conclusions

When used judiciously, conditional trial design could significantly reduce the required resources for a new study and prevent experimentation with an unnecessarily large number of participants.

Electronic supplementary material

The online version of this article (10.1186/s13063-018-2740-2) contains supplementary material, which is available to authorized users.

Systematic review of interventions for treating or preventing antipsychotic-induced tardive dyskinesia

BACKGROUND

Antipsychotic medication can cause tardive dyskinesia (TD) - late-onset, involuntary, repetitive movements, often involving the face and tongue. TD occurs in > 20% of adults taking antipsychotic medication (first-generation antipsychotics for > 3 months), with this proportion increasing by 5% per year among those who continue to use these drugs. The incidence of TD among those taking newer antipsychotics is not different from the rate in people who have used older-generation drugs in moderate doses. Studies of TD have previously been found to be limited, with no treatment approach shown to be effective.

OBJECTIVES

To summarise the clinical effectiveness and safety of treatments for TD by updating past Cochrane reviews with new evidence and improved methods; to undertake public consultation to gauge the importance of the topic for people living with TD/the risk of TD; and to make available all data from relevant trials.

DATA SOURCES

All relevant randomised controlled trials (RCTs) and observational studies.

REVIEW METHODS

Cochrane review methods, network meta-analysis (NMA).

DESIGN

Systematic reviews, patient and public involvement consultation and NMA.

SETTING

Any setting, inpatient or outpatient.

PARTICIPANTS

For systematic reviews, adults with TD who have been taking a stable antipsychotic drug dose for > 3 months.

INTERVENTIONS

Any, with emphasis on those relevant to UK NHS practice.

MAIN OUTCOME MEASURES

Any measure of TD, global assessments and adverse effects/events.

RESULTS

We included 112 studies (nine Cochrane reviews). Overall, risk of bias showed little sign of improvement over two decades. Taking the outcome of 'TD symptoms improved to a clinically important extent', we identified two trials investigating reduction of antipsychotic dose [n = 17, risk ratio (RR) 0.42, 95% confidence interval (CI) 0.17 to 1.04; very low quality]. Switching was investigated twice in trials that could not be combined (switching to risperidone vs. antipsychotic withdrawal: one RCT, n = 42, RR 0.45, 95% CI 0.23 to 0.89; low quality; switching to quetiapine vs. haloperidol: one RCT, n = 45, RR 0.80, 95% CI 0.52 to 1.22; low quality). In addition to RCTs, six observational studies compared antipsychotic discontinuation with decreased or increased dosage, and there was no clear evidence that any of these strategies had a beneficial effect on TD symptoms (very low-quality evidence). We evaluated the addition to standard antipsychotic care of several treatments, but not anticholinergic treatments, for which we identified no trials. We found no clear effect of the addition of either benzodiazepines (two RCTs, n = 32, RR 1.12, 95% CI 0.6 to 2.09; very low quality) or vitamin E (six RCTs, n = 264, RR 0.95, 95% CI 0.89 to 1.01; low quality). Buspirone as an adjunctive treatment did have some effect in one small study (n = 42, RR 0.53, 95% CI 0.33 to 0.84; low quality), as did hypnosis and relaxation (one RCT, n = 15, RR 0.45, 95% CI 0.21 to 0.94; very low quality). We identified no studies focusing on TD in people with dementia. The NMA model found indirect estimates to be imprecise and failed to produce useful summaries on relative effects of interventions or interpretable results for decision-making. Consultation with people with/at risk of TD highlighted that management of TD remains a concern, and found that people are deeply disappointed at the length of time it has taken researchers to address the issue.

LIMITATIONS

Most studies remain small and poorly reported.

CONCLUSIONS

Clinicians, policy-makers and people with/at risk of TD are little better informed than they were decades ago. Underpowered trials of limited quality repeatedly fail to provide answers.

FUTURE WORK

TD reviews have data from current trials extracted, tabulated and traceable to source. The NMA highlights one context in which support for this technique is ill advised. All relevant trials, even if not primarily addressing the issue of TD, should report appropriate binary outcomes on groups of people with this problem. Randomised trials of treatments for people with established TD are indicated. These should be large (> 800 participants), necessitating accrual through accurate local/national registers, including an intervention with acceptable treatments and recording outcomes used in clinical practice.

STUDY REGISTRATION

This study is registered as PROSPERO CRD4201502045.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

Estimating the contribution of studies in network meta-analysis: paths, flows and streams

In network meta-analysis, it is important to assess the influence of the limitations or other characteristics of individual studies on the estimates obtained from the network. The proportion contribution matrix, which shows how much each direct treatment effect contributes to each treatment effect estimate from network meta-analysis, is crucial in this context. We use ideas from graph theory to derive the proportion that is contributed by each direct treatment effect. We start with the 'projection' matrix in a two-step network meta-analysis model, called the H matrix, which is analogous to the hat matrix in a linear regression model. We develop a method to translate H entries to proportion contributions based on the observation that the rows of H  can be interpreted as flow networks, where a stream is defined as the composition of a path and its associated flow. We present an algorithm that identifies the flow of evidence in each path and decomposes it into direct comparisons. To illustrate the methodology, we use two published networks of interventions. The first compares no treatment, quinolone antibiotics, non-quinolone antibiotics and antiseptics for underlying eardrum perforations and the second compares 14 antimanic drugs. We believe that this approach is a useful and novel addition to network meta-analysis methodology, which allows the consistent derivation of the proportion contributions of direct evidence from individual studies to network treatment effects.

Estimating the contribution of studies in network meta-analysis: paths, flows and streams

In network meta-analysis, it is important to assess the influence of the limitations or other characteristics of individual studies on the estimates obtained from the network. The proportion contribution matrix, which shows how much each direct treatment effect contributes to each treatment effect estimate from network meta-analysis, is crucial in this context. We use ideas from graph theory to derive the proportion that is contributed by each direct treatment effect. We start with the ‘projection’ matrix in a two-step network meta-analysis model, called the H matrix, which is analogous to the hat matrix in a linear regression model. We develop a method to translate H entries to proportion contributions based on the observation that the rows of H can be interpreted as flow networks, where a stream is defined as the composition of a path and its associated flow. We present an algorithm that identifies the flow of evidence in each path and decomposes it into direct comparisons. To illustrate the methodology, we use two published networks of interventions. The first compares no treatment, quinolone antibiotics, non-quinolone antibiotics and antiseptics for underlying eardrum perforations and the second compares 14 antimanic drugs. We believe that this approach is a useful and novel addition to network meta-analysis methodology, which allows the consistent derivation of the proportion contributions of direct evidence from individual studies to network treatment effects.

Living network meta-analysis compared with pairwise meta-analysis in comparative effectiveness research: empirical study

OBJECTIVE

To examine whether the continuous updating of networks of prospectively planned randomised controlled trials (RCTs) ("living" network meta-analysis) provides strong evidence against the null hypothesis in comparative effectiveness of medical interventions earlier than the updating of conventional, pairwise meta-analysis.

DESIGN

Empirical study of the accumulating evidence about the comparative effectiveness of clinical interventions.

DATA SOURCES

Database of network meta-analyses of RCTs identified through searches of Medline, Embase, and the Cochrane Database of Systematic Reviews until 14 April 2015.

ELIGIBILITY CRITERIA FOR STUDY SELECTION

Network meta-analyses published after January 2012 that compared at least five treatments and included at least 20 RCTs. Clinical experts were asked to identify in each network the treatment comparison of greatest clinical interest. Comparisons were excluded for which direct and indirect evidence disagreed, based on side, or node, splitting test (P<0.10).

OUTCOMES AND ANALYSIS

Cumulative pairwise and network meta-analyses were performed for each selected comparison. Monitoring boundaries of statistical significance were constructed and the evidence against the null hypothesis was considered to be strong when the monitoring boundaries were crossed. A significance level was defined as α=5%, power of 90% (β=10%), and an anticipated treatment effect to detect equal to the final estimate from the network meta-analysis. The frequency and time to strong evidence was compared against the null hypothesis between pairwise and network meta-analyses.

RESULTS

49 comparisons of interest from 44 networks were included; most (n=39, 80%) were between active drugs, mainly from the specialties of cardiology, endocrinology, psychiatry, and rheumatology. 29 comparisons were informed by both direct and indirect evidence (59%), 13 by indirect evidence (27%), and 7 by direct evidence (14%). Both network and pairwise meta-analysis provided strong evidence against the null hypothesis for seven comparisons, but for an additional 10 comparisons only network meta-analysis provided strong evidence against the null hypothesis (P=0.002). The median time to strong evidence against the null hypothesis was 19 years with living network meta-analysis and 23 years with living pairwise meta-analysis (hazard ratio 2.78, 95% confidence interval 1.00 to 7.72, P=0.05). Studies directly comparing the treatments of interest continued to be published for eight comparisons after strong evidence had become evident in network meta-analysis.

CONCLUSIONS

In comparative effectiveness research, prospectively planned living network meta-analyses produced strong evidence against the null hypothesis more often and earlier than conventional, pairwise meta-analyses.

Outcomes of non-invasive diagnostic modalities for the detection of coronary artery disease: network meta-analysis of diagnostic randomised controlled trials

OBJECTIVE

To evaluate differences in downstream testing, coronary revascularisation, and clinical outcomes following non-invasive diagnostic modalities used to detect coronary artery disease.

DESIGN

Systematic review and network meta-analysis.

DATA SOURCES

Medline, Medline in process, Embase, Cochrane Library for clinical trials, PubMed, Web of Science, SCOPUS, WHO International Clinical Trials Registry Platform, and Clinicaltrials.gov.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Diagnostic randomised controlled trials comparing non-invasive diagnostic modalities in patients presenting with symptoms suggestive of low risk acute coronary syndrome or stable coronary artery disease.

DATA SYNTHESIS

A random effects network meta-analysis synthesised available evidence from trials evaluating the effect of non-invasive diagnostic modalities on downstream testing and patient oriented outcomes in patients with suspected coronary artery disease. Modalities included exercise electrocardiograms, stress echocardiography, single photon emission computed tomography-myocardial perfusion imaging, real time myocardial contrast echocardiography, coronary computed tomographic angiography, and cardiovascular magnetic resonance. Unpublished outcome data were obtained from 11 trials.

RESULTS

18 trials of patients with low risk acute coronary syndrome (n=11 329) and 12 trials of those with suspected stable coronary artery disease (n=22 062) were included. Among patients with low risk acute coronary syndrome, stress echocardiography, cardiovascular magnetic resonance, and exercise electrocardiograms resulted in fewer invasive referrals for coronary angiography than coronary computed tomographic angiography (odds ratio 0.28 (95% confidence interval 0.14 to 0.57), 0.32 (0.15 to 0.71), and 0.53 (0.28 to 1.00), respectively). There was no effect on the subsequent risk of myocardial infarction, but estimates were imprecise. Heterogeneity and inconsistency were low. In patients with suspected stable coronary artery disease, an initial diagnostic strategy of stress echocardiography or single photon emission computed tomography-myocardial perfusion imaging resulted in fewer downstream tests than coronary computed tomographic angiography (0.24 (0.08 to 0.74) and 0.57 (0.37 to 0.87), respectively). However, exercise electrocardiograms yielded the highest downstream testing rate. Estimates for death and myocardial infarction were imprecise without clear discrimination between strategies.

CONCLUSIONS

For patients with low risk acute coronary syndrome, an initial diagnostic strategy of stress echocardiography or cardiovascular magnetic resonance is associated with fewer referrals for invasive coronary angiography and revascularisation procedures than non-invasive anatomical testing, without apparent impact on the future risk of myocardial infarction. For suspected stable coronary artery disease, there was no clear discrimination between diagnostic strategies regarding the subsequent need for invasive coronary angiography, and differences in the risk of myocardial infarction cannot be ruled out.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO registry no CRD42016049442.

A model for meta-analysis of correlated binary outcomes: The case of split-body interventions

In several areas of clinical research, it is common for trials to assign different sites of the participants’ bodies to different interventions. For example, a randomized controlled trial comparing surgical techniques for correcting myopia may randomize each eye of a participant to a different operation. Under such bilateral (‘split-body’) interventions, the observations from each participant are correlated. It is challenging to account for these correlations at the meta-analysis level, especially when the outcome is rare. Here, we present a meta-analysis model based on the bivariate binomial distribution. Our model can synthesize studies on patients who received one intervention at one body site, patients who received two interventions at different sites or a mixture of these two groups. The model can analyse studies with zero events in one or both treatment arms and can handle the case of incomplete data reporting. We use simulations to assess the performance of our model and to compare it with the bivariate beta-binomial model. In the case of bilateral interventions, our model performed well and outperformed the bivariate beta-binomial model in all scenarios explored. We illustrate our methods using two previously published meta-analyses from the fields of orthopaedics and ophthalmology. We conclude that our model constitutes a useful new tool for the meta-analysis of binary outcomes in the presence of split-body interventions.

An investigation of the impact of using different methods for network meta-analysis: a protocol for an empirical evaluation

BACKGROUND

Network meta-analysis, a method to synthesise evidence from multiple treatments, has increased in popularity in the past decade. Two broad approaches are available to synthesise data across networks, namely, arm- and contrast-synthesis models, with a range of models that can be fitted within each. There has been recent debate about the validity of the arm-synthesis models, but to date, there has been limited empirical evaluation comparing results using the methods applied to a large number of networks. We aim to address this gap through the re-analysis of a large cohort of published networks of interventions using a range of network meta-analysis methods.

METHODS

We will include a subset of networks from a database of network meta-analyses of randomised trials that have been identified and curated from the published literature. The subset of networks will include those where the primary outcome is binary, the number of events and participants are reported for each direct comparison, and there is no evidence of inconsistency in the network. We will re-analyse the networks using three contrast-synthesis methods and two arm-synthesis methods. We will compare the estimated treatment effects, their standard errors, treatment hierarchy based on the surface under the cumulative ranking (SUCRA) curve, the SUCRA value, and the between-trial heterogeneity variance across the network meta-analysis methods. We will investigate whether differences in the results are affected by network characteristics and baseline risk.

DISCUSSION

The results of this study will inform whether, in practice, the choice of network meta-analysis method matters, and if it does, in what situations differences in the results between methods might arise. The results from this research might also inform future simulation studies.

Characteristics and knowledge synthesis approach for 456 network meta-analyses: a scoping review

BACKGROUND

Network meta-analysis (NMA) has become a popular method to compare more than two treatments. This scoping review aimed to explore the characteristics and methodological quality of knowledge synthesis approaches underlying the NMA process. We also aimed to assess the statistical methods applied using the Analysis subdomain of the ISPOR checklist.

METHODS

Comprehensive literature searches were conducted in MEDLINE, PubMed, EMBASE, and Cochrane Database of Systematic Reviews from inception until April 14, 2015. References of relevant reviews were scanned. Eligible studies compared at least four different interventions from randomised controlled trials with an appropriate NMA approach. Two reviewers independently performed study selection and data abstraction of included articles. All discrepancies between reviewers were resolved by a third reviewer. Data analysis involved quantitative (frequencies) and qualitative (content analysis) methods. Quality was evaluated using the AMSTAR tool for the conduct of knowledge synthesis and the ISPOR tool for statistical analysis.

RESULTS

After screening 3538 citations and 877 full-text papers, 456 NMAs were included. These were published between 1997 and 2015, with 95% published after 2006. Most were conducted in Europe (51%) or North America (31%), and approximately one-third reported public sources of funding. Overall, 84% searched two or more electronic databases, 62% searched for grey literature, 58% performed duplicate study selection and data abstraction (independently), and 62% assessed risk of bias. Seventy-eight (17%) NMAs relied on previously conducted systematic reviews to obtain studies for inclusion in their NMA. Based on the AMSTAR tool, almost half of the NMAs incorporated quality appraisal results to formulate conclusions, 36% assessed publication bias, and 16% reported the source of funding. Based on the ISPOR tool, half of the NMAs did not report if an assessment for consistency was conducted or whether they accounted for inconsistency when present. Only 13% reported heterogeneity assumptions for the random-effects model.

CONCLUSIONS

The knowledge synthesis methods and analytical process for NMAs are poorly reported and need improvement.

Continuously updated network meta-analysis and statistical monitoring for timely decision-making

Pairwise and network meta-analysis (NMA) are traditionally used retrospectively to assess existing evidence. However, the current evidence often undergoes several updates as new studies become available. In each update recommendations about the conclusiveness of the evidence and the need of future studies need to be made. In the context of prospective meta-analysis future studies are planned as part of the accumulation of the evidence. In this setting, multiple testing issues need to be taken into account when the meta-analysis results are interpreted. We extend ideas of sequential monitoring of meta-analysis to provide a methodological framework for updating NMAs. Based on the z-score for each network estimate (the ratio of effect size to its standard error) and the respective information gained after each study enters NMA we construct efficacy and futility stopping boundaries. A NMA treatment effect is considered conclusive when it crosses an appended stopping boundary. The methods are illustrated using a recently published NMA where we show that evidence about a particular comparison can become conclusive via indirect evidence even if no further trials address this comparison.

Efficacy, Acceptability, and Tolerability of Antipsychotics in Treatment-Resistant Schizophrenia: A Network Meta-analysis

IMPORTANCE

In treatment-resistant schizophrenia, clozapine is considered the standard treatment. However, clozapine use has restrictions owing to its many adverse effects. Moreover, an increasing number of randomized clinical trials (RCTs) of other antipsychotics have been published.

OBJECTIVE

To integrate all the randomized evidence from the available antipsychotics used for treatment-resistant schizophrenia by performing a network meta-analysis.

DATA SOURCES

MEDLINE, EMBASE, Biosis, PsycINFO, PubMed, Cochrane Central Register of Controlled Trials, World Health Organization International Trial Registry, and clinicaltrials.gov were searched up to June 30, 2014.

STUDY SELECTION

At least 2 independent reviewers selected published and unpublished single- and double-blind RCTs in treatment-resistant schizophrenia (any study-defined criterion) that compared any antipsychotic (at any dose and in any form of administration) with another antipsychotic or placebo.

DATA EXTRACTION AND SYNTHESIS

At least 2 independent reviewers extracted all data into standard forms and assessed the quality of all included trials with the Cochrane Collaboration's risk-of-bias tool. Data were pooled using a random-effects model in a Bayesian setting.

MAIN OUTCOMES AND MEASURES

The primary outcome was efficacy as measured by overall change in symptoms of schizophrenia. Secondary outcomes included change in positive and negative symptoms of schizophrenia, categorical response to treatment, dropouts for any reason and for inefficacy of treatment, and important adverse events.

RESULTS

Forty blinded RCTs with 5172 unique participants (71.5% men; mean [SD] age, 38.8 [3.7] years) were included in the analysis. Few significant differences were found in all outcomes. In the primary outcome (reported as standardized mean difference; 95% credible interval), olanzapine was more effective than quetiapine (-0.29; -0.56 to -0.02), haloperidol (-0. 29; -0.44 to -0.13), and sertindole (-0.46; -0.80 to -0.06); clozapine was more effective than haloperidol (-0.22; -0.38 to -0.07) and sertindole (-0.40; -0.74 to -0.04); and risperidone was more effective than sertindole (-0.32; -0.63 to -0.01). A pattern of superiority for olanzapine, clozapine, and risperidone was seen in other efficacy outcomes, but results were not consistent and effect sizes were usually small. In addition, relatively few RCTs were available for antipsychotics other than clozapine, haloperidol, olanzapine, and risperidone. The most surprising finding was that clozapine was not significantly better than most other drugs.

CONCLUSIONS AND RELEVANCE

Insufficient evidence exists on which antipsychotic is more efficacious for patients with treatment-resistant schizophrenia, and blinded RCTs-in contrast to unblinded, randomized effectiveness studies-provide little evidence of the superiority of clozapine compared with other second-generation antipsychotics. Future clozapine studies with high doses and patients with extremely treatment-refractory schizophrenia might be most promising to change the current evidence.

Endoscopic and Open Release Similarly Safe for the Treatment of Carpal Tunnel Syndrome. A Systematic Review and Meta-Analysis

Background

The Endoscopic Release of Carpal Tunnel Syndrome (ECTR) is a minimal invasive approach for the treatment of Carpal Tunnel Syndrome. There is scepticism regarding the safety of this technique, based on the assumption that this is a rather “blind” procedure and on the high number of severe complications that have been reported in the literature.

Purpose

To evaluate whether there is evidence supporting a higher risk after ECTR in comparison to the conventional open release.

Methods

We searched MEDLINE (January 1966 to November 2013), EMBASE (January 1980 to November 2013), the Cochrane Neuromuscular Disease Group Specialized Register (November 2013) and CENTRAL (2013, issue 11 in The Cochrane Library). We hand-searched reference lists of included studies. We included all randomized or quasi-randomized controlled trials (e.g. study using alternation, date of birth, or case record number) that compare any ECTR with any OCTR technique. Safety was assessed by the incidence of major, minor and total number of complications, recurrences, and re-operations.The total time needed before return to work or to return to daily activities was also assessed. We synthesized data using a random-effects meta-analysis in STATA. We conducted a sensitivity analysis for rare events using binomial likelihood. We judged the conclusiveness of meta-analysis calculating the conditional power of meta-analysis.

Conclusions

ECTR is associated with less time off work or with daily activities. The assessment of major complications, reoperations and recurrence of symptoms does not favor either of the interventions. There is an uncertain advantage of ECTR with respect to total minor complications (more transient paresthesia but fewer skin-related complications). Future studies are unlikely to alter these findings because of the rarity of the outcome. The effect of a learning curve might be responsible for reduced recurrences and reoperations with ECTR in studies that are more recent, although formal statistical analysis failed to provide evidence for such an association. Level of evidence: I.

Planning future studies based on the precision of network meta-analysis results

When there are multiple competing interventions for a healthcare problem, the design of new studies could be based on the entire network of evidence as reflected in a network meta-analysis. There is a practical need to answer how many (if any) studies are needed, of which design (i.e., which treatments to compare), and with what sample size to infer conclusively about the relative treatment effects of a set of target or all competing treatments and their relative ranking. We consider the precision in the results obtained from network meta-analysis: the precision of the joint distribution of the estimated basic parameters of the model and the precision in the treatment ranking. We quantify the precision in the estimated effects by considering their variance-covariance matrix and estimate the precision in ranking by quantifying the dissimilarity of the density functions of summary effect estimates. Then, based on a desirable improvement in precision, we calculate the required sample size for each possible study design and number of study arms, and we present visual tools that can help trialists select the optimal study design. We use a published network of interventions for the treatment of hepatocellular carcinoma to illustrate the suggested methodology. The presented methodology can aid investigators making informed and evidence-based decisions about planning new studies.

How to interpret meta-analysis models: fixed effect and random effects meta-analyses

This section of the journal is aimed at providing the essential information readers should know about the topics that are addressed in the 'Statistics in practice' paper published in the same issue of the journal. This stand-alone section has to be seen as an articulated summary of the main notions clinicians have to know about some basic concepts in statistics, which may be useful for their evidence-based practice. After going through these notes, readers are encouraged to read the 'Statistics in practice' articles. Of course, we welcome any feedback from you (via email or Twitter) about this! The EBMH Editors Relative treatment effects studied in trials are typically measured using an effect size. The observed effect sizes are synthesised to obtain a summary treatment effect via meta-analysis.

Using conditional power of network meta-analysis (NMA) to inform the design of future clinical trials

Clinical trials are typically designed with an aim to reach sufficient power to test a hypothesis about relative effectiveness of two or more interventions. Their role in informing evidence-based decision-making demands, however, that they are considered in the context of the existing evidence. Consequently, their planning can be informed by characteristics of relevant systematic reviews and meta-analyses. In the presence of multiple competing interventions the evidence base has the form of a network of trials, which provides information not only about the required sample size but also about the interventions that should be compared in a future trial. In this paper we present a methodology to evaluate the impact of new studies, their information size, the comparisons involved, and the anticipated heterogeneity on the conditional power (CP) of the updated network meta-analysis. The methods presented are an extension of the idea of CP initially suggested for a pairwise meta-analysis and we show how to estimate the required sample size using various combinations of direct and indirect evidence in future trials. We apply the methods to two previously published networks and we show that CP for a treatment comparison is dependent on the magnitude of heterogeneity and the ratio of direct to indirect information in existing and future trials for that comparison. Our methodology can help investigators calculate the required sample size under different assumptions about heterogeneity and make decisions about the number and design of future studies (set of treatments compared).