Using conditional power of network meta-analysis (NMA) to inform the design of future clinical trials

OssaNMA: An R package for using information from network meta-analyses to optimize the power and sample allocation of a new two-arm trial

Randomized clinical trials (RCTs) are designed for measuring the effectiveness of the treatments and testing a hypothesis regarding the relative effect between two or more treatments. Trial designers are often interested in maximizing power when the total sample size is fixed or minimizing the required total sample size to reach a pre-specified power. One approach to maximizing power proposed by previous researchers is to leverage prior evidence using meta-analysis (NMA) to inform the sample size determination of a new trial. For example, researchers may be interested in designing a two-arm trial comparing treatments A and B which are already in the existing trial network but do not have any direct comparison. The researchers' intention is to incorporate the result into an existing network for meta-analysis. Here we develop formulas to address these options and use simulations to validate our formula and evaluate the performance of different analysis methods in terms of power. We also implement our proposed method into the R package OssaNMA and publish an R Shiny app for the convenience of the application. The goal of the package is to enable researchers to readily adopt the proposed approach which can improve the power of an RCT and is therefore resource-saving. In the R Shiny app, We also provide the option to include the cost of each treatment which would enable researchers to compare the total treatment cost associated with each design and analysis approach. Further, we explore the effect of allocation to treatment group on study power when the a priori plan is to incorporate the new trial result into an existing network for meta-analysis.

Optimal trial design selection: a comparative analysis between two-arm and three-arm trials incorporating network meta-analysis for evaluating a new treatment

BACKGROUND

Planning the design of a new trial comparing two treatments already in a network of trials with an a-priori plan to estimate the effect size using a network meta-analysis increases power or reduces the sample size requirements. However, when the comparison of interest is between a treatment already in the existing network (old treatment) and a treatment that hasn't been studied previously (new treatment), the impact of leveraging information from the existing network to inform trial design has not been extensively investigated. We aim to identify the most powerful trial design for a comparison of interest between an old treatment A and a new treatment Z, given a fixed total sample size. We consider three possible designs: a two-arm trial between A and Z ('direct two-arm'), a two-arm trial between another old treatment B and Z ('indirect two-arm'), and a three-arm trial among A, B, and Z.

METHODS

We compare the standard error of the estimated effect size between treatments A and Z for each of the three trial designs using formulas. For continuous outcomes, the direct two-arm trial always has the largest power, while for a binary outcome, the minimum variances among the three trial designs are conclusive only when [Formula: see text]. Simulation studies are conducted to demonstrate the potential for the indirect two-arm and three-arm trials to outperform the direct two-arm trial in terms of power under the condition of [Formula: see text].

RESULTS

Based on the simulation results, we observe that the indirect two-arm and three-arm trials have the potential to be more powerful than a direct two-arm trial only when [Formula: see text]. This power advantage is influenced by various factors, including the risk of the three treatments, the total sample size, and the standard error of the estimated effect size from the existing network meta-analysis.

CONCLUSIONS

The standard two-arm trial design between two treatments in the comparison of interest may not always be the most powerful design. Utilizing information from the existing network meta-analysis, incorporating an additional old treatment into the trial design through an indirect two-arm trial or a three-arm trial can increase power.

Connecting a disconnected trial network with a new trial: optimizing the estimation of a comparative effect in a network meta-analysis

BACKGROUND

In network meta-analysis, estimation of a comparative effect can be performed for treatments that are connected either directly or indirectly. However, disconnected trial networks may arise, which poses a challenge to comparing all available treatments of interest. Several modeling approaches attempt to compare treatments from disconnected networks but not without strong assumptions and limitations. Conducting a new trial to connect a disconnected network can enable calculation of all treatment comparisons and help researchers maximize the value of the existing networks. Here, we develop an approach to finding the best connecting trial given a specific comparison of interest.

METHODS

We present formulas to quantify the variation in the estimation of a particular comparative effect of interest for any possible connecting two-arm trial. We propose a procedure to identify the optimal connecting trial that minimizes this variation in effect estimation.

RESULTS

We show that connecting two treatments indirectly might be preferred to direct connection through a new trial, by leveraging information from the existing disconnected networks. Using a real network of studies on the use of vaccines in the treatment of bovine respiratory disease (BRD), we illustrate a procedure to identify the best connecting trial and confirm our findings via simulation.

CONCLUSION

Researchers wishing to conduct a connecting two-arm study can use the procedure provided here to identify the best connecting trial. The choice of trial that minimizes the variance of a comparison of interest is network dependent and it is possible that connecting treatments indirectly may be preferred to direct connection.

Comparative effectiveness and safety of analgesic medicines for adults with acute non-specific low back pain: systematic review and network meta-analysis

OBJECTIVE

To evaluate the comparative effectiveness and safety of analgesic medicines for acute non-specific low back pain.

DESIGN

Systematic review and network meta-analysis.

DATA SOURCES

Medline, PubMed, Embase, CINAHL, CENTRAL, ClinicalTrials.gov, clinicialtrialsregister.eu, and World Health Organization's International Clinical Trials Registry Platform from database inception to 20 February 2022.

ELIGIBILITY CRITERIA FOR STUDY SELECTION

Randomised controlled trials of analgesic medicines (eg, non-steroidal anti-inflammatory drugs, paracetamol, opioids, anti-convulsant drugs, skeletal muscle relaxants, or corticosteroids) compared with another analgesic medicine, placebo, or no treatment. Adults (≥18 years) who reported acute non-specific low back pain (for less than six weeks).

DATA EXTRACTION AND SYNTHESIS

Primary outcomes were low back pain intensity (0-100 scale) at end of treatment and safety (number of participants who reported any adverse event during treatment). Secondary outcomes were low back specific function, serious adverse events, and discontinuation from treatment. Two reviewers independently identified studies, extracted data, and assessed risk of bias. A random effects network meta-analysis was done and confidence was evaluated by the Confidence in Network Meta-Analysis method.

RESULTS

98 randomised controlled trials (15 134 participants, 49% women) included 69 different medicines or combinations. Low or very low confidence was noted in evidence for reduced pain intensity after treatment with tolperisone (mean difference -26.1 (95% confidence intervals -34.0 to -18.2)), aceclofenac plus tizanidine (-26.1 (-38.5 to -13.6)), pregabalin (-24.7 (-34.6 to -14.7)), and 14 other medicines compared with placebo. Low or very low confidence was noted for no difference between the effects of several of these medicines. Increased adverse events had moderate to very low confidence with tramadol (risk ratio 2.6 (95% confidence interval 1.5 to 4.5)), paracetamol plus sustained release tramadol (2.4 (1.5 to 3.8)), baclofen (2.3 (1.5 to 3.4)), and paracetamol plus tramadol (2.1 (1.3 to 3.4)) compared with placebo. These medicines could increase the risk of adverse events compared with other medicines with moderate to low confidence. Moderate to low confidence was also noted for secondary outcomes and secondary analysis of medicine classes.

CONCLUSIONS

The comparative effectiveness and safety of analgesic medicines for acute non-specific low back pain are uncertain. Until higher quality randomised controlled trials of head-to-head comparisons are published, clinicians and patients are recommended to take a cautious approach to manage acute non-specific low back pain with analgesic medicines.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42019145257.

When we shouldn't borrow information from an existing network of trials for planning a new trial

Introduction: To achieve higher power or increased precision for a new trial, methods based on updating network meta-analysis (NMA) have been proposed by researchers. However, this approach could potentially lead to misinterpreted results and misstated conclusions. This work aims to investigate the potential inflation of type I error risk when a new trial is conducted only when, based on a p-value of the comparison in the existing network, a "promising" difference between two treatments is noticed. Methods: We use simulations to evaluate the scenarios of interest. In particular, a new trial is to be conducted independently or depending on the results from previous NMA in various scenarios. Three analysis methods are applied to each simulation scenario: with the existing network, sequential analysis and without the existing network. Results: For the scenario that the new trial will be conducted only when a promising finding (p-value <5%) is indicated by the existing network, the type I error risk increased dramatically (38.5% in our example data) when analyzed with the existing network and sequential analysis. The type I error is controlled at 5% when analyzing the new trial without the existing network. Conclusion: If the intention is to combine a trial result with an existing network of evidence, or if it is expected that the trial will eventually be included in a network meta-analysis, then the decision that a new trial is performed should not depend on a statistically "promising" finding indicated by the existing network.

Using information from network meta-analyses to optimize the power and sample allocation of a subsequent trial with a new treatment

BACKGROUND

A critical step in trial design is determining the sample size and sample allocation to ensure the proposed study has sufficient power to test the hypothesis of interest: superiority, equivalence, or non-inferiority. When data are available from prior trials and leveraged with the new trial to answer the scientific questions, the value of society's investment in prior research is increased. When prior information is available, the trial design including the sample size and allocation should be adapted accordingly, yet the current approach to trial design does not utilize such information. Ensuring we maximize the value of prior research is essential as there are always constraints on resources, either physical or financial, and designing a trial with adequate power can be a challenge.

METHODS

We propose an approach to increasing the power of a new trial by incorporating evidence from a network meta-analysis into the new trial design and analysis. We illustrate the methodology through an example network meta-analysis, where the goal is to identify the optimal allocation ratio for the new three-arm trial, which involves the reference treatment, the new treatment, and the negative control. The primary goal of the new trial is to show that the new treatment is non-inferior to the reference treatment. It may also be of interest to know if the new treatment is superior to the negative control. We propose an optimal treatment allocation strategy which is derived from minimizing the standard error of the log odds ratio estimate of the comparison of interest. We conducted a simulation study to assess the proposed methods to design a new trial while borrowing information from the existing network meta-analysis and compare it to even allocation methods.

RESULTS

Using mathematical derivation and simulations, we document that our proposed approach can borrow information from a network meta-analysis to modify the treatment allocation ratio and increase the power of the new trial given a fixed total sample size or to reduce the total sample size needed to reach a desired power.

CONCLUSIONS

When prior evidence about the hypotheses of interest is available, the traditional equal allocation strategy is not the most powerful approach anymore. Our proposed methodology can improve the power of trial design, reduce the cost of trials, and maximize the utility of prior investments in research.

Evidence Base for Treatment of Infectious Bovine Keratoconjunctivitis

In this article, the evidence base for treating infectious bovine keratoconjunctivitis (IBK) is discussed. First, we summarize the available evidence for antibiotic treatments registered in North America. We then discuss the evidence base for nonantibiotic alternatives. We do not discuss antibiotic treatments that do not use registered protocols; such information is available in another review. Finally, we discuss how the research community could generate more evidence for effective treatments and the comparative efficacy information to help veterinarians and producers decide between treatment options.

Effects of nutrition intervention strategies in the primary prevention of overweight and obesity in school settings: a protocol for a systematic review and network meta-analysis

BACKGROUND

Overweight and obesity in children and adolescents are major public health challenges affecting quality of life and representing important risk factors in the development of non-communicable diseases. School environments provide great possibilities for the primary prevention of overweight and obesity and different school-based nutrition interventions are available. However, existing research on school-based nutrition interventions has important limitations and no network meta-analysis (NMA) has been performed yet to compare all available interventions. Therefore, the present research project aims to investigate the impact of different nutrition interventions in the school setting by comparing and ranking them using NMA methodology.

METHODS/DESIGN

A systematic literature search will be performed in 11 electronic databases (PubMed, the Cochrane Library, Web of Science, ERIC, PsycINFO, CAB Abstracts, Campbell Library, BiblioMap EPPI, Australian Education Index, Joanna Briggs Institute Evidence-Based Practice Database and Practice-based Evidence in Nutrition Database). Parallel or cluster randomized controlled trials (RCTs) meeting the following criteria will be included: (1) generally healthy school students aged 4-18 years, (2) school-based intervention with ≥ 1 nutrition component, and (3) assessed anthropometric (overweight/obesity risk, body weight change, weight Z-score, [standardized] body mass index, body fat, waist circumference) and/or diet-quality measures (daily intake of fruits and vegetables, fat, and sugar-sweetened beverages). Random effects pairwise and NMA will be performed for these outcomes and surface under the cumulative ranking curve (SUCRA) estimated (P-score). Where possible, component NMA (CNMA) will be used additionally. Subgroup analyses are carried out for intervention duration, gender, age of school students, socioeconomic status, and geographical location, and sensitivity analyses by excluding high risk of bias RCTs.

DISCUSSION

This systematic review and NMA will be the first to both directly and indirectly compare and rank different school-based nutrition interventions for the primary prevention of overweight and obesity in childhood and adolescence. Our analyses will provide important insights about the effects of the different interventions and show which are the most promising. The results of our study can help inform the design of new studies and will be of value to anyone interested in developing successful, evidence-based nutrition interventions in school settings.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO: CRD42020220451 .

Conditional power of antidepressant network meta-analysis

BACKGROUND

Conditional power of network meta-analysis (NMA) can support the planning of randomized controlled trials (RCTs) assessing medical interventions. Conditional power is the probability that updating existing inconclusive evidence in NMA with additional trial(s) will result in conclusive evidence, given assumptions regarding trial design, anticipated effect sizes, or event probabilities.

METHODS

The present work aimed to estimate conditional power for potential future trials on antidepressant treatments. Existing evidence was based on a published network of 502 RCTs conducted between 1979-2018 assessing acute antidepressant treatment in major depressive disorder (MDD). Primary outcomes were efficacy in terms of the symptom change on the Hamilton Depression Scale (HAMD) and tolerability in terms of the dropout rate due to adverse events. The network compares 21 antidepressants consisting of 231 relative treatment comparisons, 164 (efficacy) and 127 (tolerability) of which are currently assumed to have inconclusive evidence.

RESULTS

Required sample sizes to achieve new conclusive evidence with at least 80% conditional power were estimated to range between N = 894 - 4190 (efficacy) and N = 521 - 1246 (tolerability). Otherwise, sample sizes ranging between N = 49 - 485 (efficacy) and N = 40 - 320 (tolerability) may require stopping for futility based on a boundary at 20% conditional power. Optimizing trial designs by considering multiple trials that contribute both direct and indirect evidence, anticipating alternative effect sizes or alternative event probabilities, may increase conditional power but required sample sizes remain high. Antidepressants having the greatest conditional power associated with smallest required sample sizes were identified as those on which current evidence is low, i.e., clomipramine, levomilnacipran, milnacipran, nefazodone, and vilazodone, with respect to both outcomes.

CONCLUSIONS

The present results suggest that conditional power to achieve new conclusive evidence in ongoing or future trials on antidepressant treatments is low. Limiting the use of the presented conditional power analysis are primarily due to the estimated large sample sizes which would be required in future trials as well as due to the well-known small effect sizes in antidepressant treatments. These findings may inform researchers and decision-makers regarding the clinical relevance and justification of research in ongoing or future antidepressant RCTs in MDD.

Comparing the diagnostic performance of radiotracers in recurrent prostate cancer: a systematic review and network meta-analysis

Purpose

Many radiotracers are currently available for the detection of recurrent prostate cancer (rPC), yet many have not been compared head-to-head in comparative imaging studies. There is therefore an unmet need for evidence synthesis to guide evidence-based decisions in the selection of radiotracers. The objective of this study was therefore to assess the detection rate of various radiotracers for the rPC.

Methods

The PUBMED, EMBASE, and the EU and NIH trials databases were searched without date or language restriction for comparative imaging tracers for 13 radiotracers of principal interest. Key search terms included 18F-PSMA-1007, 18F-DCPFyl, 68Ga-PSMA-11, 18F-PSMA-11, 68Ga-PSMA-I&T, 68Ga-THP-PSMA, 64Cu-PSMA-617, 18F-JK-PSMA-7, 18F-Fluciclovine, 18F-FABC, 18F-Choline, 11C-Choline, and 68Ga-RM2. Studies reporting comparative imaging data in humans in rPC were selected. Single armed studies and matched pair analyses were excluded. Twelve studies with eight radiotracers were eligible for inclusion. Two independent reviewers screened all studies (using the PRISMA-NMA statement) for inclusion criteria, extracted data, and assessed risk of bias (using the QUADAS-2 tool). A network meta-analysis was performed using Markov-Chain Monte Carlo Bayesian analysis to obtain estimated detection rate odds ratios for each tracer combination.

Results

A majority of studies were judged to be at risk of publication bias. With the exception of 18F-PSMA-1007, little difference in terms of detection rate was revealed between the three most commonly used PSMA-radiotracers (⁶⁸Ga-PSMA-11, ¹⁸F-PSMA-1007, ¹⁸F-DCFPyl), which in turn showed clear superiority to choline and fluciclovine using the derived network.

Conclusion

Differences in patient-level detection rates were observed between PSMA- and choline-radiotracers. However, there is currently insufficient evidence to favour one of the four routinely used PSMA-radioligands (PSMA-11, PSMA-1007, PSMA-I&T, and DCFPyl) over another owing to the limited evidence base and risk of publication bias revealed by our systematic review. A further limitation was lack of reporting on diagnostic accuracy, which might favour radiotracers with low specificity in an analysis restricted only to detection rate. The NMA derived can be used to inform the design of future clinical trials and highlight areas where current evidence is weak.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00259-021-05210-9.

What exercise prescription is optimal to improve body composition and cardiorespiratory fitness in adults living with obesity? A network meta-analysis

Current international guidelines recommend people living with obesity should be prescribed a minimum of 300 min of moderately intense activity per week for weight loss. However, the most efficacious exercise prescription to improve anthropometry, cardiorespiratory fitness (CRF) and metabolic health in this population remains unknown. Thus, this network meta-analysis was conducted to assess and rank comparative efficacy of different exercise interventions on anthropometry, CRF and other metabolic risk factors. Five electronic databases were searched for randomized controlled trials (RCTs) that compared different exercise modalities to improve anthropometry, CRF and/or metabolic health in adults living with obesity. RCTs were evaluated using the Cochrane risk of bias tool. A random effects network meta-analysis was performed within a frequentist framework. Of the 6663 articles retrieved, 45 studies with a total 3566 participants were included. Results reveal that while any type of exercise intervention is more effective than control, weight loss induced is modest. Interventions that combine high-intensity aerobic and high-load resistance training exert beneficial effects that are superior to any other exercise modality at decreasing abdominal adiposity, improving lean body mass and increasing CRF. Clinicians should consider this evidence when prescribing exercise for adults living with obesity, to ensure optimal effectiveness.

Redundant trials can be prevented, if the EU clinical trial regulation is applied duly

The problem of wasteful clinical trials has been debated relentlessly in the medical community. To a significant extent, it is attributed to redundant trials - studies that are carried out to address questions, which can be answered satisfactorily on the basis of existing knowledge and accessible evidence from prior research. This article presents the first evaluation of the potential of the EU Clinical Trials Regulation 536/2014, which entered into force in 2014 but is expected to become applicable at the end of 2021, to prevent such trials. Having reviewed provisions related to the trial authorisation, we propose how certain regulatory requirements for the assessment of trial applications can and should be interpreted and applied by national research ethics committees and other relevant authorities in order to avoid redundant trials and, most importantly, preclude the unnecessary recruitment of trial participants and their unjustified exposure to health risks.

Degree irregularity and rank probability bias in network meta-analysis

Network meta-analysis (NMA) is a statistical technique for the comparison of treatment options. Outcomes of Bayesian NMA include estimates of treatment effects, and the probabilities that each treatment is ranked best, second best and so on. How exactly network topology affects the accuracy and precision of these outcomes is not fully understood. Here we carry out a simulation study and find that disparity in the number of trials involving different treatments leads to a systematic bias in estimated rank probabilities. This bias is associated with an increased variation in the precision of treatment effect estimates. Using ideas from the theory of complex networks, we define a measure of "degree irregularity" to quantify asymmetry in the number of studies involving each treatment. Our simulations indicate that more regular networks have more precise treatment effect estimates and smaller bias of rank probabilities. Conversely, these topological effects are not observed for the accuracy of treatment effect estimates. This reinforces the importance of taking into account multiple measures, rather than making decisions based on a single metric. We also find that degree regularity is a better indicator for the accuracy and precision of parameter estimates in NMA than both the total number of studies in a network and the disparity in the number of trials per comparison. These results have implications for planning future trials. We demonstrate that choosing trials which reduce the network's irregularity can improve the precision and accuracy of parameter estimates from NMA.

Perspective: Network Meta-analysis Reaches Nutrition Research: Current Status, Scientific Concepts, and Future Directions

Traditional pairwise meta-analysis (PMA) is a very useful method that pools evidence from one study design type if appropriate; its widespread use in nutrition research is an important phenomenon. Recently, a promising method for more advanced evidence-synthesis, called network meta-analysis (NMA), was introduced. NMA is an extension of PMA that enables simultaneous comparison of multiple interventions. NMA combines direct evidence (i.e., trials comparing 2 interventions directly) and indirect evidence (i.e., from a connected route via ≥1 comparators, e.g. placebo) in a network of studies. NMAs have the potential to advance knowledge in the field of nutrition as they provide insights that cannot be obtained by individual 2-arm randomized controlled trials or PMA. Thus, in this perspective paper, we aim to summarize the current (methodologic) status of published NMAs in nutrition research and emphasize advances and strengths in comparison with traditional PMA through specific examples, and highlight potential pitfalls and limitations. NMA is an emerging methodology in the field of nutrition research. A PubMed search identified only 23 nutrition research-related NMAs published since the inception of journals up to January 8, 2019 (61% of them published since 2017), compared with >5000 published PMAs. Moreover, we aim to highlight the scientific concepts and standards through the use of the following NMA example: "Which type of oils/solid fats offers the greatest impact on blood lipids?" In this regard, we discuss intervention definitions, transitivity/similarity, statistical methods, description and visualization of results, inconsistency, ranking, dissemination bias, assessing the certainty of evidence by Grading of Recommendations Assessment, Development and Evaluation, and reporting guidelines. We expect that rigorously conducted NMAs based on high-quality systematic reviews will become the new evidence synthesis benchmark in nutrition research. However, caution is warranted because abuse and misinterpretations of PMA and NMA findings could hamper the scientific field and possibly decision-making regarding public policy.

Synthesizing existing evidence to design future trials: survey of methodologists from European institutions

Background

‘Conditional trial design’ is a framework for efficiently planning new clinical trials based on a network of relevant existing trials. The framework considers whether new trials are required and how the existing evidence can be used to answer the research question and plan future research. The potential of this approach has not been fully realized.

Methods

We conducted an online survey among trial statisticians, methodologists, and users of evidence synthesis research using referral sampling to capture opinions about the conditional trial design framework and current practices among clinical researchers. The questions included in the survey were related to the decision of whether a meta-analysis answers the research question, the optimal way to synthesize available evidence, which relates to the acceptability of network meta-analysis, and the use of evidence synthesis in the planning of new studies.

Results

In total, 76 researchers completed the survey. Two out of three survey participants (65%) were willing to possibly or definitely consider using evidence synthesis to design a future clinical trial and around half of the participants would give priority to such a trial design. The median rating of the frequency of using such a trial design was 0.41 on a scale from 0 (never) to 1 (always). Major barriers to adopting conditional trial design include the current regulatory paradigm and the policies of funding agencies and sponsors.

Conclusions

Participants reported moderate interest in using evidence synthesis methods in the design of future trials. They indicated that a major paradigm shift is required before the use of network meta-analysis is regularly employed in the design of trials.

Electronic supplementary material

The online version of this article (10.1186/s13063-019-3449-6) contains supplementary material, which is available to authorized users.

The inclusion of real world evidence in clinical development planning

BACKGROUND

When designing studies it is common to search the literature to investigate variability estimates to use in sample size calculations. Proprietary data of previously designed trials in a particular indication are also used to obtain estimates of variability. Estimates of treatment effects are typically obtained from randomised controlled clinical trials (RCTs). Based on the observed estimates of treatment effect, variability and the minimum clinical relevant difference to detect, the sample size for a subsequent trial is estimated. However, data from real world evidence (RWE) studies, such as observational studies and other interventional studies in patients in routine clinical practice, are not widely used in a systematic manner when designing studies. In this paper, we propose a framework for inclusion of RWE in planning of a clinical development programme.

METHODS

In our proposed approach, all evidence, from both RCTs and RWE (i.e. from studies in routine clinical practice), available at the time of designing of a new clinical trial is combined in a Bayesian network meta-analysis (NMA). The results can be used to inform the design of the next clinical trial in the programme. The NMA was performed at key milestones, such as at the end of the phase II trial and prior to the design of key phase III studies. To illustrate the methods, we designed an alternative clinical development programme in multiple sclerosis using RWE through clinical trial simulations.

RESULTS

Inclusion of RWE in the NMA and the resulting trial simulations demonstrated that 284 patients per arm were needed to achieve 90% power to detect effects of predetermined size in the TRANSFORMS study. For the FREEDOMS and FREEDOMS II clinical trials, 189 patients per arm were required. Overall there was a reduction in sample size of at least 40% across the three phase III studies, which translated to a time savings of at least 6 months for the undertaking of the fingolimod phase III programme.

CONCLUSION

The use of RWE resulted in a reduced sample size of the pivotal phase III studies, which led to substantial time savings compared to the approach of sample size calculations without RWE.

Planning a future randomized clinical trial based on a network of relevant past trials

Background

The important role of network meta-analysis of randomized clinical trials in health technology assessment and guideline development is increasingly recognized. This approach has the potential to obtain conclusive results earlier than with new standalone trials or conventional, pairwise meta-analyses.

Methods

Network meta-analyses can also be used to plan future trials. We introduce a four-step framework that aims to identify the optimal design for a new trial that will update the existing evidence while minimizing the required sample size. The new trial designed within this framework does not need to include all competing interventions and comparisons of interest and can contribute direct and indirect evidence to the updated network meta-analysis. We present the method by virtually planning a new trial to compare biologics in rheumatoid arthritis and a new trial to compare two drugs for relapsing-remitting multiple sclerosis.

Results

A trial design based on updating the evidence from a network meta-analysis of relevant previous trials may require a considerably smaller sample size to reach the same conclusion compared with a trial designed and analyzed in isolation. Challenges of the approach include the complexity of the methodology and the need for a coherent network meta-analysis of previous trials with little heterogeneity.

Conclusions

When used judiciously, conditional trial design could significantly reduce the required resources for a new study and prevent experimentation with an unnecessarily large number of participants.

Electronic supplementary material

The online version of this article (10.1186/s13063-018-2740-2) contains supplementary material, which is available to authorized users.

Power analysis for random-effects meta-analysis

One of the reasons for the popularity of meta-analysis is the notion that these analyses will possess more power to detect effects than individual studies. This is inevitably the case under a fixed-effect model. However, the inclusion of the between-study variance in the random-effects model, and the need to estimate this parameter, can have unfortunate implications for this power. We develop methods for assessing the power of random-effects meta-analyses, and the average power of the individual studies that contribute to meta-analyses, so that these powers can be compared. In addition to deriving new analytical results and methods, we apply our methods to 1991 meta-analyses taken from the Cochrane Database of Systematic Reviews to retrospectively calculate their powers. We find that, in practice, 5 or more studies are needed to reasonably consistently achieve powers from random-effects meta-analyses that are greater than the studies that contribute to them. Not only is statistical inference under the random-effects model challenging when there are very few studies but also less worthwhile in such cases. The assumption that meta-analysis will result in an increase in power is challenged by our findings.

Prioritizing covariates in the planning of future studies in the meta-analytic framework

Science can be seen as a sequential process where each new study augments evidence to the existing knowledge. To have the best prospects to make an impact in this process, a new study should be designed optimally taking into account the previous studies and other prior information. We propose a formal approach for the covariate prioritization, that is the decision about the covariates to be measured in a new study. The decision criteria can be based on conditional power, change of the p-value, change in lower confidence limit, Kullback-Leibler divergence, Bayes factors, Bayesian false discovery rate or difference between prior and posterior expectation. The criteria can be also used for decisions on the sample size. As an illustration, we consider covariate prioritization based on genome-wide association studies for C-reactive protein levels and make suggestions on the genes to be studied further.

Continuously updated network meta-analysis and statistical monitoring for timely decision-making

Pairwise and network meta-analysis (NMA) are traditionally used retrospectively to assess existing evidence. However, the current evidence often undergoes several updates as new studies become available. In each update recommendations about the conclusiveness of the evidence and the need of future studies need to be made. In the context of prospective meta-analysis future studies are planned as part of the accumulation of the evidence. In this setting, multiple testing issues need to be taken into account when the meta-analysis results are interpreted. We extend ideas of sequential monitoring of meta-analysis to provide a methodological framework for updating NMAs. Based on the z-score for each network estimate (the ratio of effect size to its standard error) and the respective information gained after each study enters NMA we construct efficacy and futility stopping boundaries. A NMA treatment effect is considered conclusive when it crosses an appended stopping boundary. The methods are illustrated using a recently published NMA where we show that evidence about a particular comparison can become conclusive via indirect evidence even if no further trials address this comparison.

Wasted research when systematic reviews fail to provide a complete and up-to-date evidence synthesis: the example of lung cancer

BACKGROUND

Multiple treatments are frequently available for a given condition, and clinicians and patients need a comprehensive, up-to-date synthesis of evidence for all competing treatments. We aimed to quantify the waste of research related to the failure of systematic reviews to provide a complete and up-to-date evidence synthesis over time.

METHODS

We performed a series of systematic overviews and networks of randomized trials assessing the gap between evidence covered by systematic reviews and available trials of second-line treatments for advanced non-small cell lung cancer. We searched the Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, MEDLINE, EMBASE, and other resources sequentially by year from 2009 to March 2, 2015. We sequentially compared the amount of evidence missing from systematic reviews to the randomized evidence available for inclusion each year. We constructed cumulative networks of randomized evidence over time and evaluated the proportion of trials, patients, treatments, and treatment comparisons not covered by systematic reviews on December 31 each year from 2009 to 2015.

RESULTS

We identified 77 trials (28,636 patients) assessing 47 treatments with 54 comparisons and 29 systematic reviews (13 published after 2013). From 2009 to 2015, the evidence covered by existing systematic reviews was consistently incomplete: 45 % to 70 % of trials; 30 % to 58 % of patients; 40 % to 66 % of treatments; and 38 % to 71 % of comparisons were missing. In the cumulative networks of randomized evidence, 10 % to 17 % of treatment comparisons were partially covered by systematic reviews and 55 % to 85 % were partially or not covered.

CONCLUSIONS

We illustrate how systematic reviews of a given condition provide a fragmented, out-of-date panorama of the evidence for all treatments. This waste of research might be reduced by the development of live cumulative network meta-analyses.

Planning future studies based on the precision of network meta-analysis results

When there are multiple competing interventions for a healthcare problem, the design of new studies could be based on the entire network of evidence as reflected in a network meta-analysis. There is a practical need to answer how many (if any) studies are needed, of which design (i.e., which treatments to compare), and with what sample size to infer conclusively about the relative treatment effects of a set of target or all competing treatments and their relative ranking. We consider the precision in the results obtained from network meta-analysis: the precision of the joint distribution of the estimated basic parameters of the model and the precision in the treatment ranking. We quantify the precision in the estimated effects by considering their variance-covariance matrix and estimate the precision in ranking by quantifying the dissimilarity of the density functions of summary effect estimates. Then, based on a desirable improvement in precision, we calculate the required sample size for each possible study design and number of study arms, and we present visual tools that can help trialists select the optimal study design. We use a published network of interventions for the treatment of hepatocellular carcinoma to illustrate the suggested methodology. The presented methodology can aid investigators making informed and evidence-based decisions about planning new studies.