A threshold analysis assessed the credibility of conclusions from network meta-analysis

Twenty years of network meta-analysis: Continuing controversies and recent developments

Network meta-analysis (NMA) is an extension of pairwise meta-analysis (PMA) which combines evidence from trials on multiple treatments in connected networks. NMA delivers internally consistent estimates of relative treatment efficacy, needed for rational decision making. Over its first 20 years NMA's use has grown exponentially, with applications in both health technology assessment (HTA), primarily re-imbursement decisions and clinical guideline development, and clinical research publications. This has been a period of transition in meta-analysis, first from its roots in educational and social psychology, where large heterogeneous datasets could be explored to find effect modifiers, to smaller pairwise meta-analyses in clinical medicine on average with less than six studies. This has been followed by narrowly-focused estimation of the effects of specific treatments at specific doses in specific populations in sparse networks, where direct comparisons are unavailable or informed by only one or two studies. NMA is a powerful and well-established technique but, in spite of the exponential increase in applications, doubts about the reliability and validity of NMA persist. Here we outline the continuing controversies, and review some recent developments. We suggest that heterogeneity should be minimized, as it poses a threat to the reliability of NMA which has not been fully appreciated, perhaps because it has not been seen as a problem in PMA. More research is needed on the extent of heterogeneity and inconsistency in datasets used for decision making, on formal methods for making recommendations based on NMA, and on the further development of multi-level network meta-regression.

A systematic review and network meta-analysis of topical pharmacological, oral pharmacological, physical and combined treatments for acne vulgaris

BACKGROUND

Various treatments for acne vulgaris exist, but little is known about their comparative effectiveness in relation to acne severity.

OBJECTIVES

To identify best treatments for mild-to-moderate and moderate-to-severe acne, as determined by clinician-assessed morphological features.

METHODS

We undertook a systematic review and network meta-analysis of randomized controlled trials (RCTs) assessing topical pharmacological, oral pharmacological, physical and combined treatments for mild-to-moderate and moderate-to-severe acne, published up to May 2020. Outcomes included percentage change in total lesion count from baseline, treatment discontinuation for any reason, and discontinuation owing to side-effects. Risk of bias was assessed using the Cochrane risk-of-bias tool and bias adjustment models. Effects for treatments with ≥ 50 observations each compared with placebo are reported below.

RESULTS

We included 179 RCTs with approximately 35 000 observations across 49 treatment classes. For mild-to-moderate acne, the most effective options for each treatment type were as follows: topical pharmacological - combined retinoid with benzoyl peroxide (BPO) [mean difference 26·16%, 95% credible interval (CrI) 16·75-35·36%]; physical - chemical peels, e.g. salicylic or mandelic acid (39·70%, 95% CrI 12·54-66·78%) and photochemical therapy (combined blue/red light) (35·36%, 95% CrI 17·75-53·08%). Oral pharmacological treatments (e.g. antibiotics, hormonal contraceptives) did not appear to be effective after bias adjustment. BPO and topical retinoids were less well tolerated than placebo. For moderate-to-severe acne, the most effective options for each treatment type were as follows: topical pharmacological - combined retinoid with lincosamide (clindamycin) (44·43%, 95% CrI 29·20-60·02%); oral pharmacological - isotretinoin of total cumulative dose ≥ 120 mg kg^-1 per single course (58·09%, 95% CrI 36·99-79·29%); physical - photodynamic therapy (light therapy enhanced by a photosensitizing chemical) (40·45%, 95% CrI 26·17-54·11%); combined - BPO with topical retinoid and oral tetracycline (43·53%, 95% CrI 29·49-57·70%). Topical retinoids and oral tetracyclines were less well tolerated than placebo. The quality of included RCTs was moderate to very low, with evidence of inconsistency between direct and indirect evidence. Uncertainty in findings was high, in particular for chemical peels, photochemical therapy and photodynamic therapy. However, conclusions were robust to potential bias in the evidence.

CONCLUSIONS

Topical pharmacological treatment combinations, chemical peels and photochemical therapy were most effective for mild-to-moderate acne. Topical pharmacological treatment combinations, oral antibiotics combined with topical pharmacological treatments, oral isotretinoin and photodynamic therapy were most effective for moderate-to-severe acne. Further research is warranted for chemical peels, photochemical therapy and photodynamic therapy for which evidence was more limited. What is already known about this topic? Acne vulgaris is the eighth most common disease globally. Several topical, oral, physical and combined treatments for acne vulgaris exist. Network meta-analysis (NMA) synthesizes direct and indirect evidence and allows simultaneous inference for all treatments forming an evidence network. Previous NMAs have assessed a limited range of treatments for acne vulgaris and have not evaluated effectiveness of treatments for moderate-to-severe acne. What does this study add? For mild-to-moderate acne, topical treatment combinations, chemical peels, and photochemical therapy (combined blue/red light; blue light) are most effective. For moderate-to-severe acne, topical treatment combinations, oral antibiotics combined with topical treatments, oral isotretinoin and photodynamic therapy (light therapy enhanced by a photosensitizing chemical) are most effective. Based on these findings, along with further clinical and cost-effectiveness considerations, National Institute for Health and Care Excellence (NICE) guidance recommends, as first-line treatments, fixed topical treatment combinations for mild-to-moderate acne and fixed topical treatment combinations, or oral tetracyclines combined with topical treatments, for moderate-to-severe acne.

Comparative clinical effectiveness and safety of tobacco cessation pharmacotherapies and electronic cigarettes: a systematic review and network meta-analysis of randomized controlled trials

AIM

To determine how varenicline, bupropion, nicotine replacement therapy (NRT) and electronic cigarettes compare with respect to their clinical effectiveness and safety.

METHOD

Systematic reviews and Bayesian network meta-analyses of randomized controlled trials, in any setting, of varenicline, bupropion, NRT and e-cigarettes (in high, standard and low doses, alone or in combination) in adult smokers and smokeless tobacco users with follow-up duration of 24 weeks or greater (effectiveness) or any duration (safety). Nine databases were searched until 19 February 2019. Primary outcomes were sustained tobacco abstinence and serious adverse events (SAEs). We estimated odds ratios (ORs) and treatment rankings and conducted meta-regression to explore covariates.

RESULTS

We identified 363 trials for effectiveness and 355 for safety. Most monotherapies and combination therapies were more effective than placebo at helping participants to achieve sustained abstinence; the most effective of these, estimated with some imprecision, were varenicline standard [OR = 2.83, 95% credible interval (CrI) = 2.34-3.39] and varenicline standard + NRT standard (OR = 5.75, 95% CrI = 2.27-14.88). Estimates were higher in smokers receiving counselling than in those without and in studies with higher baseline nicotine dependence scores than in those with lower scores. Varenicline standard + NRT standard showed a high probability of being ranked best or second-best. For safety, only bupropion at standard dose increased the odds of experiencing SAEs compared with placebo (OR = 1.27, 95% CrI = 1.04-1.58), and we found no evidence of effect modification.

CONCLUSIONS

Most tobacco cessation monotherapies and combination therapies are more effective than placebo at helping participants to achieve sustained abstinence, with varenicline appearing to be most effective based on current evidence. There does not appear to be strong evidence of associations between most tobacco cessation pharmacotherapies and adverse events; however, the data are limited and there is a need for improved reporting of safety data.

Smoking cessation medicines and e-cigarettes: a systematic review, network meta-analysis and cost-effectiveness analysis

BACKGROUND

Cigarette smoking is one of the leading causes of early death. Varenicline [Champix (UK), Pfizer Europe MA EEIG, Brussels, Belgium; or Chantix (USA), Pfizer Inc., Mission, KS, USA], bupropion (Zyban; GlaxoSmithKline, Brentford, UK) and nicotine replacement therapy are licensed aids for quitting smoking in the UK. Although not licensed, e-cigarettes may also be used in English smoking cessation services. Concerns have been raised about the safety of these medicines and e-cigarettes.

OBJECTIVES

To determine the clinical effectiveness, safety and cost-effectiveness of smoking cessation medicines and e-cigarettes.

DESIGN

Systematic reviews, network meta-analyses and cost-effectiveness analysis informed by the network meta-analysis results.

SETTING

Primary care practices, hospitals, clinics, universities, workplaces, nursing or residential homes.

PARTICIPANTS

Smokers aged ≥ 18 years of all ethnicities using UK-licensed smoking cessation therapies and/or e-cigarettes.

INTERVENTIONS

Varenicline, bupropion and nicotine replacement therapy as monotherapies and in combination treatments at standard, low or high dose, combination nicotine replacement therapy and e-cigarette monotherapies.

MAIN OUTCOME MEASURES

Effectiveness - continuous or sustained abstinence. Safety - serious adverse events, major adverse cardiovascular events and major adverse neuropsychiatric events.

DATA SOURCES

Ten databases, reference lists of relevant research articles and previous reviews. Searches were performed from inception until 16 March 2017 and updated on 19 February 2019.

REVIEW METHODS

Three reviewers screened the search results. Data were extracted and risk of bias was assessed by one reviewer and checked by the other reviewers. Network meta-analyses were conducted for effectiveness and safety outcomes. Cost-effectiveness was evaluated using an amended version of the Benefits of Smoking Cessation on Outcomes model.

RESULTS

Most monotherapies and combination treatments were more effective than placebo at achieving sustained abstinence. Varenicline standard plus nicotine replacement therapy standard (odds ratio 5.75, 95% credible interval 2.27 to 14.90) was ranked first for sustained abstinence, followed by e-cigarette low (odds ratio 3.22, 95% credible interval 0.97 to 12.60), although these estimates have high uncertainty. We found effect modification for counselling and dependence, with a higher proportion of smokers who received counselling achieving sustained abstinence than those who did not receive counselling, and higher odds of sustained abstinence among participants with higher average dependence scores. We found that bupropion standard increased odds of serious adverse events compared with placebo (odds ratio 1.27, 95% credible interval 1.04 to 1.58). There were no differences between interventions in terms of major adverse cardiovascular events. There was evidence of increased odds of major adverse neuropsychiatric events for smokers randomised to varenicline standard compared with those randomised to bupropion standard (odds ratio 1.43, 95% credible interval 1.02 to 2.09). There was a high level of uncertainty about the most cost-effective intervention, although all were cost-effective compared with nicotine replacement therapy low at the £20,000 per quality-adjusted life-year threshold. E-cigarette low appeared to be most cost-effective in the base case, followed by varenicline standard plus nicotine replacement therapy standard. When the impact of major adverse neuropsychiatric events was excluded, varenicline standard plus nicotine replacement therapy standard was most cost-effective, followed by varenicline low plus nicotine replacement therapy standard. When limited to licensed interventions in the UK, nicotine replacement therapy standard was most cost-effective, followed by varenicline standard.

LIMITATIONS

Comparisons between active interventions were informed almost exclusively by indirect evidence. Findings were imprecise because of the small numbers of adverse events identified.

CONCLUSIONS

Combined therapies of medicines are among the most clinically effective, safe and cost-effective treatment options for smokers. Although the combined therapy of nicotine replacement therapy and varenicline at standard doses was the most effective treatment, this is currently unlicensed for use in the UK.

FUTURE WORK

Researchers should examine the use of these treatments alongside counselling and continue investigating the long-term effectiveness and safety of e-cigarettes for smoking cessation compared with active interventions such as nicotine replacement therapy.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42016041302.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 59. See the NIHR Journals Library website for further project information.

Lag times in the publication of network meta-analyses: a survey

OBJECTIVE

We assessed the extent of lag times in the publication and indexing of network meta-analyses (NMAs).

STUDY DESIGN

This was a survey of published NMAs on drug interventions.

SETTING

NMAs indexed in PubMed (searches updated in May 2020).

PRIMARY AND SECONDARY OUTCOME MEASURES

Lag times were measured as the time between the last systematic search and the article submission, acceptance, online publication, indexing and Medical Subject Headings (MeSH) allocation dates. Time-to-event analyses were performed considering independent variables (geographical origin, Journal Impact Factor, Scopus CiteScore, open access status) (SPSS V.24, R/RStudio).

RESULTS

We included 1245 NMAs. The median time from last search to article submission was 6.8 months (204 days (IQR 95-381)), and to publication was 11.6 months. Only 5% of authors updated their search after first submission. There is a very slightly decreasing historical trend of acceptance (rho=-0.087; p=0.010), online publication (rho=-0.080; p=0.008) and indexing (rho=-0.080; p=0.007) lag times. Journal Impact Factor influenced the MeSH allocation process, but not the other lag times. The comparison between open access versus subscription journals confirmed meaningless differences in acceptance, online publication and indexing lag times.

CONCLUSION

Efforts by authors to update their search before submission are needed to reduce evidence production time. Peer reviewers and editors should ensure authors' compliance with NMA standards. The accuracy of these findings depends on the accuracy of the metadata used; as we evaluated only NMA on drug interventions, results may not be generalisable to all types of studies.

A method for assessing robustness of the results of a star-shaped network meta-analysis under the unidentifiable consistency assumption

Background

In a star-shaped network, pairwise comparisons link treatments with a reference treatment (often placebo or standard care), but not with each other. Thus, comparisons between non-reference treatments rely on indirect evidence, and are based on the unidentifiable consistency assumption, limiting the reliability of the results. We suggest a method of performing a sensitivity analysis through data imputation to assess the robustness of results with an unknown degree of inconsistency.

Methods

The method involves imputation of data for randomized controlled trials comparing non-reference treatments, to produce a complete network. The imputed data simulate a situation that would allow mixed treatment comparison, with a statistically acceptable extent of inconsistency. By comparing the agreement between the results obtained from the original star-shaped network meta-analysis and the results after incorporating the imputed data, the robustness of the results of the original star-shaped network meta-analysis can be quantified and assessed. To illustrate this method, we applied it to two real datasets and some simulated datasets.

Results

Applying the method to the star-shaped network formed by discarding all comparisons between non-reference treatments from a real complete network, 33% of the results from the analysis incorporating imputed data under acceptable inconsistency indicated that the treatment ranking would be different from the ranking obtained from the star-shaped network. Through a simulation study, we demonstrated the sensitivity of the results after data imputation for a star-shaped network with different levels of within- and between-study variability. An extended usability of the method was also demonstrated by another example where some head-to-head comparisons were incorporated.

Conclusions

Our method will serve as a practical technique to assess the reliability of results from a star-shaped network meta-analysis under the unverifiable consistency assumption.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01290-1.

Efficacy and acceptability of pharmacological and non-pharmacological interventions for non-specific chronic low back pain: a protocol for a systematic review and network meta-analysis

BACKGROUND

Despite the enormous financial and humanistic burden of chronic low back pain (CLBP), there is little consensus on what constitutes the best treatment options from a multitude of competing interventions. The objective of this network meta-analysis (NMA) is to determine the relative efficacy and acceptability of primary care treatments for non-specific CLBP, with the overarching aim of providing a comprehensive evidence base for informing treatment decisions.

METHODS

We will perform a systematic search to identify randomised controlled trials of interventions endorsed in primary care guidelines for the treatment of non-specific CLBP in adults. Information sources searched will include major bibliographic databases (MEDLINE, Embase, CENTRAL, CINAHL, PsycINFO and LILACS) and clinical trial registries. Our primary outcomes will be patient-reported pain ratings and treatment acceptability (all-cause discontinuation), and secondary outcomes will be functional ability, quality of life and patient/physician ratings of overall improvement. A hierarchical Bayesian class-based NMA will be performed to determine the relative effects of different classes of pharmacological (NSAIDs, opioids, paracetamol, anti-depressants, muscle relaxants) and non-pharmacological (exercise, patient education, manual therapies, psychological therapy, multidisciplinary approaches, massage, acupuncture, mindfulness) interventions and individual treatments within a class (e.g. NSAIDs: diclofenac, ibuprofen, naproxen). We will conduct risk of bias assessments and threshold analysis to assess the robustness of the findings to potential bias. We will compute the effect of different interventions relative to placebo/no treatment for both short- and long-term efficacy and acceptability.

DISCUSSION

While many factors are important in selecting an appropriate intervention for an individual patient, evidence for the analgesic effects and acceptability of a treatment are key factors in guiding this selection. Thus, this NMA will provide an important source of evidence to inform treatment decisions and future clinical guidelines.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO registry number: CRD42019138115.

Research Review: Psychological and psychosocial treatments for children and young people with post-traumatic stress disorder: a network meta-analysis

BACKGROUND

Post-traumatic stress disorder (PTSD) is a potentially chronic and disabling disorder that affects a significant minority of youth exposed to trauma. Previous studies have concluded that trauma-focused cognitive behavioural therapy (TF-CBT) is an effective treatment for PTSD in youth, but the relative strengths of different psychological therapies are poorly understood.

METHODS

We undertook a systematic review and network meta-analyses of psychological and psychosocial interventions for children and young people with PTSD. Outcomes included PTSD symptom change scores post-treatment and at 1-4-month follow-up, and remission post-treatment.

RESULTS

We included 32 trials of 17 interventions and 2,260 participants. Overall, the evidence was of moderate-to-low quality. No inconsistency was detected between direct and indirect evidence. Individual forms of TF-CBT showed consistently large effects in reducing PTSD symptoms post-treatment compared with waitlist. The order of interventions by descending magnitude of effect versus waitlist was as follows: cognitive therapy for PTSD (SMD -2.94, 95%CrI -3.94 to -1.95), combined somatic/cognitive therapies, child-parent psychotherapy, combined TF-CBT/parent training, meditation, narrative exposure, exposure/prolonged exposure, play therapy, Cohen TF-CBT/cognitive processing therapy (CPT), eye movement desensitisation and reprocessing (EMDR), parent training, group TF-CBT, supportive counselling and family therapy (SMD -0.37, 95%CrI -1.60 to 0.84). Results for parent training, supportive counselling and family therapy were inconclusive. Cohen TF-CBT/CPT, group TF-CBT and supportive counselling had the largest evidence base. Results regarding changes in PTSD symptoms at follow-up and remission post-treatment were uncertain due to limited evidence.

CONCLUSIONS

Trauma-focused cognitive behavioural therapy, in particular individual forms, appears to be most effective in the management of PTSD in youth. EMDR is effective but to a lesser extent. Supportive counselling does not appear to be effective. Results suggest a large positive effect for emotional freedom technique, child-parent psychotherapy, combined TF-CBT/parent training, and meditation, but further research is needed to confirm these findings as they were based on very limited evidence.

Combining threshold analysis and GRADE to assess sensitivity to bias in antidepressant treatment recommendations adjusted for depression severity

Threshold analysis has recently been proposed to be used in combination with the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) in order to assess the sensitivity to plausible bias of treatment recommendations derived from Bayesian network meta-analysis (NMA). Here, it was aimed to apply the combination of threshold analysis and GRADE to judge quantitative and qualitative information on risk of bias in antidepressant treatment recommendations. The analysis was based on the data set provided by Cipriani et al. (The Lancet 2018) comparing 21 antidepressants in adult major depressive disorder (MDD). Primary outcomes were efficacy (response rate) and acceptability (dropout rate) adjusted for the covariate depression severity. The combined approach suggested sensitivity to plausible bias to be largest for antidepressant recommendations top ranked by Cipriani et al., that is, amitriptyline, duloxetine, paroxetine, and venlafaxine in terms of efficacy and agomelatine, escitalopram, paroxetine, and venlafaxine in terms of acceptability. Covariate ranges within which recommendations were most sensitive to plausible bias were very severe depression in terms of efficacy (smallest threshold, ie, the largest sensitivity, around 39 Hamilton Depression Rating Scale [HDRS]) and moderate depression in terms of acceptability (smallest thresholds around 16 and 35 HDRS). This indicates that treatment recommendations within these ranges may likely change if plausible bias adjustments take place. The present findings may support decision makers in judging the sensitivity to plausible bias of current antidepressant treatment recommendations to accurately guide treatment decisions in MDD depending on depression severity.

Interventions to prevent spontaneous preterm birth in high‐risk women with singleton pregnancy: a systematic review and network meta‐analysis

This is a protocol for a Cochrane Review (Intervention). The objectives are as follows:

To compare the efficacy of current, relevant interventions to prevent preterm birth in women with singleton pregnancy and high individual risk of spontaneous preterm birth. We will consider interventions for women with a history of spontaneous preterm birth or short cervical length and women with asymptomatic vaginal infections.

Empirical evaluation of SUCRA-based treatment ranks in network meta-analysis: quantifying robustness using Cohen's kappa

OBJECTIVE

To provide a framework for quantifying the robustness of treatment ranks based on Surface Under the Cumulative RAnking curve (SUCRA) in network meta-analysis (NMA) and investigating potential factors associated with lack of robustness.

METHODS

We propose the use of Cohen's kappa to quantify the agreement between SUCRA-based treatment ranks estimated through NMA of a complete data set and a subset of it. We illustrate our approach using five published NMA data sets, where robustness was assessed by removing studies one at a time.

RESULTS

Overall, SUCRA-based treatment ranks were robust to individual studies in the five data sets we considered. We observed more incidences of disagreement between ranks in the networks with larger numbers of treatments. Most treatments moved only one or two ranks up or down. The lowest quadratic weighted kappa estimate observed across all networks was in the network with the smallest number of treatments (4), where weighted kappa=40%. In the network with the largest number of treatments (12), the lowest observed quadratic weighted kappa=89%, reflecting a small shift in this network's treatment ranks overall. Preliminary observations suggest that a study's size, the number of studies making a treatment comparison, and the agreement of a study's estimated treatment effect(s) with those estimated by other studies making the same comparison(s) may explain the overall robustness of treatment ranks to studies.

CONCLUSIONS

Investigating robustness or sensitivity in an NMA may reveal outlying rank changes that are clinically or policy-relevant. Cohen's kappa is a useful measure that permits investigation into study characteristics that may explain varying sensitivity to individual studies. However, this study presents a framework as a proof of concept and further investigation is required to identify potential factors associated with the robustness of treatment ranks using more extensive empirical evaluations.

Threshold Analysis as an Alternative to GRADE for Assessing Confidence in Guideline Recommendations Based on Network Meta-analyses

Guideline development requires the synthesis of evidence on several treatments of interest, typically by using network meta-analysis (NMA). Because treatment effects may be estimated imprecisely or be based on evidence lacking internal or external validity, guideline developers must assess the robustness of recommendations made on the basis of the NMA to potential limitations in the evidence. Such limitations arise because the observed estimates differ from the true effects of interest, for example, because of study biases, sampling variation, or issues of relevance. The widely used GRADE (Grading of Recommendations Assessment, Development and Evaluation) framework aims to assess the quality of evidence supporting a recommendation by using a structured series of qualitative judgments. This article argues that GRADE approaches proposed for NMA are insufficient for the purposes of guideline development, because the influence of the evidence on the final recommendation is not taken into account. It outlines threshold analysis as an alternative approach, demonstrating the method with 2 examples of clinical guidelines from the National Institute for Health and Care Excellence (NICE) in the United Kingdom. Threshold analysis quantifies precisely how much the evidence could change (for any reason, such as potential biases, or simply sampling variation) before the recommendation changes, and what the revised recommendation would be. If it is judged that the evidence could not plausibly change by more than this amount, then the recommendation is considered robust; otherwise, it is sensitive to plausible changes in the evidence. In this manner, threshold analysis directly informs decision makers and guideline developers of the robustness of treatment recommendations.

Which treatment is most effective for patients with patellofemoral pain? A protocol for a living systematic review including network meta-analysis

INTRODUCTION

Patellofemoral pain (PFP) affects 1 in every 14 adults. Many treatments for PFP have been evaluated, but the comparative effectiveness of all available treatments has never been examined. Network meta-analysis is the only design to study the comparative effectiveness of all available treatments in one synthesis. This protocol describes the methods for a systematic review including network meta-analysis to assess which treatment is most likely to be effective for patients with PFP.

METHODS AND ANALYSIS

The primary outcome measures of this network meta-analysis are the global rating of change scale at 6-12 weeks, 13-52 weeks and >52 weeks. The secondary outcome measures are patient-rated pain scales at 6-12 weeks, 13-52 weeks and >52 weeks. Completed published and unpublished randomised controlled trials with full-text reports are eligible for inclusion. We will search Embase, PubMed (including MEDLINE), CENTRAL, Scopus, Web of Science, and CINAHL, SPORTDiscus, OpenGrey, WorldCat, conference Proceedings and multiple trial registers for relevant reports. Two researchers will appraise the study eligibility and perform data extraction. Risk of bias will be assessed with the Cochrane Risk of Bias Tool V.2.0.Bayesian network meta-analyses will be constructed for global rating of change scale and patient-rated pain. Consistency between direct and indirect comparisons will be assessed. Between study variability will be explored, and a threshold analysis for the credibility of the network meta-analyses' conclusions will be performed.

ETHICS AND DISSEMINATION

Ethical approval is not required, as this study will be based on published data. The study commenced at 1 February 2018, and its expected completion date is 15 January 2019. Full publication of the work will be sought in an international peer-reviewed journal, as well as translational articles to disseminate the work to clinical practitioners.

PROSPERO REGISTRATION NUMBER

CRD42018079502.

Sensitivity of treatment recommendations to bias in network meta-analysis

Network meta-analysis (NMA) pools evidence on multiple treatments to estimate relative treatment effects. Included studies are typically assessed for risk of bias; however, this provides no indication of the impact of potential bias on a decision based on the NMA. We propose methods to derive bias adjustment thresholds which measure the smallest changes to the data that result in a change of treatment decision. The methods use efficient matrix operations and can be applied to explore the consequences of bias in individual studies or aggregate treatment contrasts, in both fixed and random-effects NMA models. Complex models with multiple types of data input are handled by using an approximation to the hypothetical aggregate likelihood. The methods are illustrated with a simple NMA of thrombolytic treatments and a more complex example comparing social anxiety interventions. An accompanying R package is provided.

Immunomodulatory drugs and the risk of serious infection in multiple myeloma: systematic review and meta-analysis of randomized and observational studies

The effect of immunomodulatory drugs (IMiDs) on serious infection remains uncertain. We therefore conducted a systematic review and meta-analysis to assess the possible impact of IMiDs on serious infection in patients with multiple myeloma (MM). We searched randomized controlled trials (RCTs) and observational studies from databases that addressed the effect of IMiDs on serious infection in patients with MM. We pooled data from RCTs and observational studies separately and used the GRADE approach to rate the quality of evidence. Rates in patients with individual IMiDs at different treatment status ranged from 7.00 to 23.00%. The use of thalidomide- or lenalidomide-based regimen induction therapy for autologous stem cell transplantation (ASCT)-ineligible patients suggests increase in serious infection (RR = 1.59, 95% CI 1.31-1.93, p < 0.01). Compared to conventional therapy, IMiDs' induction in ASCT-eligible patients significantly decreases the risk of serious infection (RR = 0.82, 95% CI 0.72-0.94, p < 0.01). Lenalidomide-based therapy was associated with a significant increase in risk of serious infection in patients treated compared with conventional therapy (RR = 2.45, 95% CI 1.57-3.83, p < 0.01). The current evidence suggests that patients with MM treated with IMiDs are at a high risk of serious infection.