Characteristics and knowledge synthesis approach for 456 network meta-analyses: a scoping review

Conduct and reporting of multivariate network meta-analyses: a scoping review

OBJECTIVES

Combining multivariate and network meta-analysis methods simultaneously in a multivariate network meta-analysis (MVNMA) provides the methodological framework to analyze the largest amount of evidence relevant to decision-makers (i.e., from indirect evidence and correlated outcomes). The objectives of this scoping review were to summarize the characteristics of MVNMAs published in the health sciences literature and map the methodological guidance available for MVNMA.

STUDY DESIGN AND SETTING

We searched MEDLINE, Embase, and the Cumulative Index to Nursing and Allied Health Literature from inception to 28 August 2023, along with citations of included studies, for quantitative evidence syntheses that applied MVNMA and articles addressing MVNMA methods. Pairs of reviewers independently screened potentially eligible studies. Collected data included bibliographic, methodological, and analytical characteristics of included studies. We reported results as total numbers, frequencies, and percentages for categorical variables and medians and interquartile ranges for continuous variables that were not normally distributed.

RESULTS

After screening 1,075 titles and abstracts, and 112 full texts, we included 38 unique studies, of which, 10 were quantitative evidence syntheses that applied MVNMA and 28 were articles addressing MVNMA methods. Among the 10 MVNMAs, the first was published in 2013, four used studies identified from already published systematic reviews, and eight addressed pharmacological interventions, which were the most common interventions. They evaluated interventions for metastatic melanoma, colorectal cancer, prostate cancer, oral hygiene, disruptive behavior disorders, rheumatoid arthritis, narcolepsy, type 2 diabetes, and overactive bladder syndrome. Five MVNMAs analyzed two outcomes simultaneously, and four MVNMAs analyzed three outcomes simultaneously. Among the articles addressing MVNMA methods, the first was published in 2007 and the majority provided methodological frameworks for conducting MVNMAs (26/28, 93%). One study proposed criteria to standardize reporting of MVNMAs and two proposed items relevant to the quality assessment of MVNMAs. Study authors used data from 18 different illnesses to provide illustrative examples within their methodological guidance.

CONCLUSIONS

The application of MVNMA in the health sciences literature is uncommon. Many methodological frameworks are published; however, standardization and specific criteria to guide reporting and quality assessment are lacking. This overview of the current landscape may help inform future conduct of MVNMAs and research on MVNMA methods.

Grading of recommendations assessment, development, and evaluation concept article 5: addressing intransitivity in a network meta-analysis

OBJECTIVES

This article describes considerations for addressing intransitivity when assessing the certainty of the evidence from network meta-analysis (NMA) using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach. Intransitivity is induced by effect modification, that is, when the magnitude of the effect between an intervention and outcome differs depending on the level of another factor.

STUDY DESIGN AND SETTING

To develop this GRADE concept paper, the lead authors conducted iterative discussions, computer simulations, and presentations to the GRADE project group and at GRADE working group meetings. The GRADE Working Group formally approved the article in July 2022.

RESULTS

NMA authors can have a higher or a lower threshold to rate down the certainty of the evidence due to intransitivity, which depends on the extent of their concerns regarding the trustworthiness of indirect comparisons, and their view of the relative problems with rating down excessively or insufficiently. NMA authors should consider three main factors when addressing intransitivity: the credibility of effect modification, the strength of the effect modification, and the distribution of effect modifiers across the direct comparisons. To avoid double counting limitations of the evidence, authors should consider the relationship between intransitivity and other GRADE domains.

CONCLUSION

NMA authors face theoretic and pragmatic challenges and in most situations need to assess intransitivity without the availability of empirical data. Thus, explicitness regarding perspective is crucial.

Network meta-analysis in psychology and educational sciences: A systematic review of their characteristics

Network meta-analysis (NMA) allows the combination of evidence on the effectiveness of several interventions. NMA has mainly been applied in the medical science field, whereas in the domain of psychology and educational sciences its use is less frequent. Consequently, systematic reviews that describe the characteristics of published NMAs are limited to the field of medicine, and nothing is known about the characteristics of NMAs published in the psychology and educational sciences field. However, this information is still relevant for the design of future simulation studies and for detecting good and bad research practices. Thus, this study describes the features of the meta-analytic datasets of NMAs published in the field of psychology and educational sciences, as well as their methodological characteristics, and compares them to those observed in the medical domain. Results show that the number of studies included is larger in NMAs from psychology and educational sciences, the most commonly used effect size is the standardized mean difference (unlike the odds ratio in medicine), the sample size is smaller, more intervention groups are included, and inconsistent effects are observed more often. These results can be used in future simulation studies to generate realistic datasets. Finally, we warn about the poor quality of reporting of some technical aspects of the NMA, such as the statistical model used.

Mapping the characteristics, methodological quality and standards of reporting of network meta-analyses on antithrombotic therapies: An overview

BACKGROUND

Although a large number of network meta-analyses (NMAs) in the field of cardiology are available, little is known about their methodological quality. We aimed to map the characteristics and critically appraised the standards of conduct and evidence reporting of NMAs assessing antithrombotic therapies for the treatment or prophylaxis of heart diseases and cardiac surgical procedures.

METHODS

We systematically searched PubMed and Scopus to identify NMAs comparing the clinical effects of antithrombotic therapies. Overall characteristics of the NMAs were extracted and their reporting quality and methodological quality were evaluated using the PRISMA-NMA checklist and AMSTAR-2, respectively.

RESULTS

We found 86 NMAs published between 2007 and 2022. Comparisons among direct-acting oral anticoagulants were available in 61 (71%) NMAs. Although around 75% of NMAs stated that they followed international guidelines for conduct and reporting, only one third provided a protocol/register. Complete search strategies and publication bias assessment were lacking in around 53% and 59% of studies, respectively. Most NMAs (n = 77, 90%) provided supplemental material; however, only 5 (6%) made the complete raw data available. Network diagrams were depicted in most studies (n = 67, 78%), yet network geometry was described in only 11 (12.8%) of them. Mean adherence to the PRISMA-NMA checklist was 65.1 ± 16.5%. AMSTAR-2 assessment showed 88% of the NMAs had critically low methodological quality.

CONCLUSION

Although there is a wide diffusion of NMA-type studies on antithrombotics for heart diseases, their methodological and reporting quality remains suboptimal. This may reflect fragile clinical practices due to misleading conclusions from critically low-quality NMAs.

Does type of funding affect reporting in network meta-analysis? A scoping review of network meta-analyses

BACKGROUND

Evidence has shown that private industry-sponsored randomized controlled trials (RCTs) and meta-analyses are more likely to report intervention-favourable results compared with other sources of funding. However, this has not been assessed in network meta-analyses (NMAs).

OBJECTIVES

To (a) explore the recommendation rate of industry-sponsored NMAs on their company's intervention, and (b) assess reporting in NMAs of pharmacologic interventions according to their funding type.

METHODS

Design: Scoping review of published NMAs with RCTs.

INFORMATION SOURCES

We used a pre-existing NMA database including 1,144 articles from MEDLINE, EMBASE and Cochrane Database of Systematic Reviews, published between January 2013 and July 2018.

STUDY SELECTION

NMAs with transparent funding information and comparing pharmacologic interventions with/without placebo.

SYNTHESIS

We captured whether NMAs recommended their own or another company's intervention, classified NMAs according to their primary outcome findings (i.e., statistical significance and direction of effect), and according to the overall reported conclusion. We assessed reporting using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension to NMA (PRISMA-NMA) 32-item checklist. We matched and compared industry with non-industry NMAs having the same research question, disease, primary outcome, and pharmacologic intervention against placebo/control.

RESULTS

We retrieved 658 NMAs, which reported a median of 23 items in the PRISMA-NMA checklist (interquartile range [IQR]: 21-26). NMAs were categorized as 314 publicly-sponsored (PRISMA-NMA median 24.5, IQR 22-27), 208 non-sponsored (PRISMA-NMA median 23, IQR 20-25), and 136 industry/mixed-sponsored NMAs (PRISMA-NMA median 21, IQR 19-24). Most industry-sponsored NMAs recommended their own manufactured drug (92%), suggested a statistically significant positive treatment-effect for their drug (82%), and reported an overall positive conclusion (92%). Our matched NMAs (25 industry vs 25 non-industry) indicated that industry-sponsored NMAs had favourable conclusions more often (100% vs 80%) and were associated with larger (but not statistically significantly different) efficacy effect sizes (in 61% of NMAs) compared with non-industry-sponsored NMAs.

CONCLUSIONS

Differences in completeness of reporting and author characteristics were apparent among NMAs with different types of funding. Publicly-sponsored NMAs had the best reporting and published their findings in higher impact-factor journals. Knowledge users should be mindful of this potential funding bias in NMAs.

How to Critically Appraise and Interpret Systematic Reviews and Meta-Analyses of Diagnostic Accuracy: A User Guide

Systematic reviews of diagnostic accuracy studies can provide the best available evidence to inform decisions regarding the use of a diagnostic test. In this guide, the authors provide a practical approach for clinicians to appraise diagnostic accuracy systematic reviews and apply their results to patient care. The first step is to identify an appropriate systematic review with a research question matching the clinical scenario. The user should evaluate the rigor of the review methods to evaluate its credibility (Did the review use clearly defined eligibility criteria, a comprehensive search strategy, structured data collection, risk of bias and applicability appraisal, and appropriate meta-analysis methods?). If the review is credible, the next step is to decide whether the diagnostic performance is adequate for clinical use (Do sensitivity and specificity estimates exceed the threshold that makes them useful in clinical practice? Are these estimates sufficiently precise? Is variability in the estimates of diagnostic accuracy across studies explained?). Diagnostic accuracy systematic reviews that are judged to be credible and provide diagnostic accuracy estimates with sufficient certainty and relevance are the most useful to inform patient care. This review discusses comparative, noncomparative, and emerging approaches to systematic reviews of diagnostic accuracy using a clinical scenario and examples based on recent publications.

Retrieval barriers in individual participant data reviews with network meta-analysis

OBJECTIVES

Individual participant data (IPD) from randomised controlled trials (RCTs) can be used in network meta-analysis (NMA) to underpin patient care and are the best analyses to support the development of guidelines about the use of healthcare interventions for a specific condition. However, barriers to IPD retrieval pose a major threat. The aim of this study was to present barriers we encountered during retrieval of IPD from RCTs in two published systematic reviews with IPD-NMA.

METHODS

We evaluated retrieval of IPD from RCTs for IPD-NMA in Alzheimer's dementia and type 1 diabetes. We requested IPD from authors, industry sponsors and data repositories, and recorded IPD retrieval, reasons for IPD unavailability, and retrieval challenges.

RESULTS

In total, we identified 108 RCTs: 78 industry sponsored, 11 publicly sponsored and 19 with no funding information. After failing to obtain IPD from any trial authors, we requested it from industry sponsors. Seven of the 17 industry sponsors shared IPD for 12 950 participants (59%) through proprietary-specific data sharing platforms from 26 RCTs (33%). We found that lack of RCT identifiers (eg, National Clinical Trial number) and unclear data ownership were major challenges in IPD retrieval. Incomplete information in retrieved datasets was another important problem that led to exclusion of RCTs from the NMA. There were also practical challenges in obtaining IPD from or analysing it within platforms, and additional costs were incurred in accessing IPD this way.

CONCLUSIONS

We found no clear evidence of retrieval bias (where IPD availability was linked to trial findings) in either IPD-NMA, but because retrieval bias could impact NMA findings, subsequent decision-making and guideline development, this should be considered when assessing risk of bias in IPD syntheses.

Auditing the research practices and statistical analyses of the group-level temporal network approach to psychological constructs: A systematic scoping review

Network analyses have become increasingly common within the field of psychology, and temporal network analyses in particular are quickly gaining traction, with many of the initial articles earning substantial interest. However, substantial heterogeneity exists within the study designs and methodology, rendering it difficult to form a comprehensive view of its application in psychology research. Since the field is quickly growing and since there have been many study-to-study variations in terms of choices made by researchers when collecting, processing, and analyzing data, we saw the need to audit this field and formulate a comprehensive view of current temporal network analyses. To systematically chart researchers' practices when conducting temporal network analyses, we reviewed articles conducting temporal network analyses on psychological variables (published until March 2021) in the framework of a scoping review. We identified 43 articles and present the detailed results of how researchers are currently conducting temporal network analyses. A commonality across results concerns the wide variety of data collection and analytical practices, along with a lack of consistency between articles about what is reported. We use these results, along with relevant literature from the fields of ecological momentary assessment and network analysis, to formulate recommendations on what type of data is suited for temporal network analyses as well as optimal methods to preprocess and analyze data. As the field is new, we also discuss key future steps to help usher the field's progress forward and offer a reporting checklist to help researchers navigate conducting and reporting temporal network analyses.

Laser-based refractive surgery techniques to treat myopia in adults. An overview of systematic reviews and meta-analyses

Systematic reviews (SRs) and meta-analyses (MAs) are of great importance for basing clinical decisions. However, misleading interpretations may result when informed decisions rest on biased review papers with methodological issues. To evaluate which treatment is optimal, an overview was made of SRs and MAs to establish the quality and certainty of meta-evidence published on the efficacy of laser-based refractive surgery techniques for treating myopia in adults. A search was made in five databases and was updated using Really Simple Syndication (RSS) feed appliances up to April 2021; SRs with or without MAs were included. Methodological quality was appraised using the AMSTAR-2 tool. The best available reviews were summarized using the GRADE approach. The corrected covered area (CCA) was used to determine the degree of over-representation of publications. The risk of bias of the primary studies was disclosed visually. Thirty-six studies published between 2003 and 2021 were included. Twenty SRs (56%) were conducted in China. The most studied comparisons were SMILE versus FS-LASIK (19%) and FS-LASIK versus MM-LASIK (11%). Of the 251 overlapping index publications, 165 were unique (CCA = 0.015%), representing a negligible risk of skewed reporting. The AMSTAR-2 tool showed most SRs to have critically low or low quality. Nine reviews presented moderate quality. The GRADE approach of the 41 a priori outcomes evidenced critically low and low certainty of evidence. Only the spherical equivalent refraction changes at 12 months between LASEK and PRK showed moderate certainty of evidence, favouring PRK (mean difference 0.06, 95%CI [-0.02 to 0.14], I²  = 0%; p ≥ 0.05). Index trials among less biased reviews are prone to selection, performance and reporting bias. The appraised techniques exhibit comparable results in terms of efficacy. There is moderate certainty of evidence in favour of the use of PRK over LASEK in terms of the spherical equivalent refraction error changes at 1 year of follow-up. Most appraised SRs presented methodological flaws in critical domains, resulting in a low to critically low certainty of evidence after GRADE appraisal. Therefore, investigators need to study and compare the different laser-based refractive techniques to provide better evidence-based medicine. Further well-designed, high-quality clinical trials and SRs are needed to reappraise the current findings.

Evaluation of the Reporting Standard Guidelines of Network Meta-Analyses in Physical Therapy: A Systematic Review

The concept of network meta-analyses (NMA) has been introduced to the field of physical therapy. However, the reporting standard guidelines of these studies have not been evaluated. In this systematic review, we included all published NMA physical therapy studies that compared the clinical efficacy of three or more interventions to evaluate whether NMAs in physical therapy exhibit adequate reporting recommendations. PubMed, EMBASE, Web of Science, and the Cochrane Library were searched up to 30 June 2022. Among the 252 identified articles, 19 NMAs including 805 randomized controlled trials were included. We applied both preferred reporting items for systematic reviews and meta-analysis (PRISMA) and PRISMA-NMA checklists, which are 27- and 32-item reporting standard guidelines assessment tools, respectively. Protocol registrations (68.4%), risk of bias across studies (63.2%), additional analysis (57.9%), and funding (31.6%) were problematic items considering the PRISMA guidelines. Four studies reported all five new NMA-reporting items, and 15 (78.9%) did not address items S1-5 from the PRISMA-NMA guidelines. The median score (interquartile range) of the reporting standard guidelines was 27.0 (25.8-28.0). The identified shortcomings of published NMAs should be addressed while training researchers, and they should be encouraged to apply PRISMA-NMA, as a recognized tool for assessing NMA reporting guidelines is required.

Introducing the Treatment Hierarchy Question in Network Meta-Analysis

Comparative effectiveness research using network meta-analysis can present a hierarchy of competing treatments, from the most to the least preferable option. However, in published reviews, the research question associated with the hierarchy of multiple interventions is typically not clearly defined. Here we introduce the novel notion of a treatment hierarchy question that describes the criterion for choosing a specific treatment over one or more competing alternatives. For example, stakeholders might ask which treatment is most likely to improve mean survival by at least 2 years, or which treatment is associated with the longest mean survival. We discuss the most commonly used ranking metrics (quantities that compare the estimated treatment-specific effects), how the ranking metrics produce a treatment hierarchy, and the type of treatment hierarchy question that each ranking metric can answer. We show that the ranking metrics encompass the uncertainty in the estimation of the treatment effects in different ways, which results in different treatment hierarchies. When using network meta-analyses that aim to rank treatments, investigators should state the treatment hierarchy question they aim to address and employ the appropriate ranking metric to answer it. Following this new proposal will avoid some controversies that have arisen in comparative effectiveness research.

Diagnostic test accuracy network meta-analysis methods: A scoping review and empirical assessment

OBJECTIVE

To a) identify methodological and application papers reporting a model developed specifically for diagnostic test accuracy network meta-analysis (DTA-NMA) or a hierarchical meta-regression method for comparing at least three index tests; b) review and summarize the characteristics of the methods and the application papers; and c) compare DTA-NMA and hierarchical meta-regression methods empirically.

STUDY DESIGN AND SETTINGS

We performed a scoping review and searched major databases until 3^rd March 2021. We assessed the characteristics of the identified methods, conducted a descriptive analysis of characteristics of the application articles, and applied the DTA-NMA and meta-regression methods to the available data.

RESULTS

We included 49 articles, of which 9 were methodological (describing 11 DTA-NMA methods) and 40 were application papers (data available for 32 DTA-NMAs). Our results showed a steep increase in recent years in DTA-NMA publications. DTA-NMA models may lead to different results. Although sensitivity estimates were comparable between meta-regression and DTA-NMA models, specificity estimates were higher in meta-regression.

CONCLUSIONS

The choice of a DTA-NMA model will depend on the available data, including the use of different thresholds for test positivity, different study designs, and software familiarity. Selection between the methods may impact on the NMA results, especially for specificity.

A Revised Framework to Evaluate the Consistency Assumption Globally in a Network of Interventions

Background

The unrelated mean effects (UME) model has been proposed for evaluating the consistency assumption globally in the network of interventions. However, the UME model does not accommodate multiarm trials properly and omits comparisons between nonbaseline interventions in the multiarm trials not investigated in 2-arm trials.

Methods

We proposed a refinement of the UME model that tackles the limitations mentioned above. We also accompanied the scatterplots on the posterior mean deviance contributions of the trial arms under the network meta-analysis (NMA) and UME models with Bland-Altman plots to detect outlying trials contributing to poor model fit. We applied the refined and original UME models to 2 networks with multiarm trials.

Results

The original UME model omitted more than 20% of the observed comparisons in both networks. The thorough inspection of the individual data points’ deviance contribution using complementary plots in conjunction with the measures of model fit and the estimated between-trial variance indicated that the refined and original UME models revealed possible inconsistency in both examples.

Conclusions

The refined UME model allows proper accommodation of the multiarm trials and visualization of all observed evidence in complex networks of interventions. Furthermore, considering several complementary plots to investigate deviance helps draw informed conclusions on the possibility of global inconsistency in the network.

Highlights

Prevalence of evidence of inconsistency and its association with network structural characteristics in 201 published networks of interventions

BACKGROUND

Network meta-analysis (NMA) has attracted growing interest in evidence-based medicine. Consistency between different sources of evidence is fundamental to the reliability of the NMA results. The purpose of the present study was to estimate the prevalence of evidence of inconsistency and describe its association with different NMA characteristics.

METHODS

We updated our collection of NMAs with articles published up to July 2018. We included networks with randomised clinical trials, at least four treatment nodes, at least one closed loop, a dichotomous primary outcome, and available arm-level data. We assessed consistency using the design-by-treatment interaction (DBT) model and testing all the inconsistency parameters globally through the Wald-type chi-squared test statistic. We estimated the prevalence of evidence of inconsistency and its association with different network characteristics (e.g., number of studies, interventions, intervention comparisons, loops). We evaluated the influence of the network characteristics on the DBT p-value via a multivariable regression analysis and the estimated Pearson correlation coefficients. We also evaluated heterogeneity in NMA (consistency) and DBT (inconsistency) random-effects models.

RESULTS

We included 201 published NMAs. The p-value of the design-by-treatment interaction (DBT) model was lower than 0.05 in 14% of the networks and lower than 0.10 in 20% of the networks. Networks including many studies and comparing few interventions were more likely to have small DBT p-values (less than 0.10), which is probably because they yielded more precise estimates and power to detect differences between designs was higher. In the presence of inconsistency (DBT p-value lower than 0.10), the consistency model displayed higher heterogeneity than the DBT model.

CONCLUSIONS

Our findings show that inconsistency was more frequent than what would be expected by chance, suggesting that researchers should devote more resources to exploring how to mitigate inconsistency. The results of this study highlight the need to develop strategies to detect inconsistency (because of the relatively high prevalence of evidence of inconsistency in published networks), and particularly in cases where the existing tests have low power.

Reporting of Cochrane systematic review protocols with network meta-analyses-A scoping review

Publishing systematic review protocols is a fundamental part of systematic reviews to ensure transparency and reproducibility. In this scoping review, we aimed to evaluate reporting of Cochrane systematic review protocols with network meta-analyses (NMA). We searched all Cochrane NMA protocols published in 2018 and 2019, and assessed the characteristics and reporting of methodologies relevant to NMA. We reported frequencies for each reporting item. Forty-five protocols were assessed, including two for overviews and 43 for intervention reviews. Thirty-three (73%) were labelled as NMA protocols in the title. Forty-two (95%) justified the need of an NMA and 40 (89%) used appropriate search strategies to identify potential eligible studies. About half (24, 53%) considered the transitivity assumption when reporting inclusion criteria and 35 (78%) specified potential effect modifiers. Forty-three (96%) reported statistical software for NMA, 25 (56%) reported NMA model choice, 32 (71%) reported framework choice and 32 (71%) reported assumption about heterogeneity variances. Protocols varied in whether they reported methods for relative ranking (35, 78%), statistical inconsistency (40, 89%), reporting bias (44, 98%) and sources of heterogeneity (39, 87%). In conclusion, Cochrane NMA protocols reported multiple NMA-specific items well, but could be further improved, especially regarding transitivity assumptions. Our recommendations for NMA protocol reporting based on this scoping review could assist authors, reviewers, and editors to improve NMA protocols.

Modeling Multicomponent Interventions in Network Meta-Analysis

There is a rapid increase in trials assessing healthcare interventions consisting of a combination of drugs (polytherapies) or multiple components. In the latter type of interventions (also known as complex interventions), the aspect of complexity is of paramount importance. For example, nonpharmacological interventions, such as psychological interventions or self-management interventions, usually share common components that relate to the nature of intervention, who delivers it, or where and how. In a network of trials, there is often the need to identify the most effective (or safest) component and/or combination of components. Four key meta-analytical approaches have been presented in the literature to handle complex interventions. These include (a) the single-effect model, (b) the full interaction model, (c) the additive main effects model, and (d) the two-way interaction model. In this chapter, we present and discuss the advantages and limitations of these approaches. We illustrate these methods using a network that assesses the relative effects of self-management interventions on waist size in patients with type 2 diabetes.

Do reporting guidelines have an impact? Empirical assessment of changes in reporting before and after the PRISMA extension statement for network meta-analysis

BACKGROUND

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension statement for network meta-analysis (NMA) published in 2015 promotes comprehensive reporting in published systematic reviews with NMA. PRISMA-NMA includes 32 items: 27 core items as indicated in the 2009 PRISMA Statement and five items specific to the reporting of NMAs. Although NMA reporting is improving, it is unclear whether PRISMA-NMA has accelerated this improvement. We aimed to investigate the impact of PRISMA-NMA and highlight key items that require attention and improvement.

METHODS

We updated our previous collection of NMAs with articles published between April 2015 and July 2018. We assessed the completeness of reporting for each NMA, including main manuscript and online supplements, using the PRISMA-NMA checklist. The PRISMA-NMA checklist originally includes 32 total items (i.e. a 32-point scale original PRISMA-NMA score). We also prepared a modified version of the PRISMA-NMA checklist with 49 items to evaluate separately at a more granular level all multiple-content items (i.e. a 49-point scale modified PRISMA-NMA score). We compared average reporting scores of articles published until and after 2015.

RESULTS

In the 1144 included NMAs the mean modified PRISMA-NMA score was 32.1 (95% CI 31.8-32.4) of a possible 49-excellence-score. For 1-year increase, the mean modified score increased by 0.96 (95% CI 0.32 to 1.59) for 389 NMAs published until 2015 and by 0.53 (95% CI 0.02 to 1.04) for 755 NMAs published after 2015. The mean modified PRISMA-NMA score for NMAs published after 2015 was higher by 0.81 (95% CI 0.23 to 1.39) compared to before 2015 when adjusting for journal impact factor, type of review, funding, and treatment category. Description of summary effect sizes to be used, presentation of individual study data, sources of funding for the systematic review, and role of funders dropped in frequency after 2015 by 6-16%.

CONCLUSIONS

NMAs published after 2015 more frequently reported the five items associated with NMA compared to those published until 2015. However, improvement in reporting after 2015 is compatible with that observed on a yearly basis until 2015, and hence, it could not be attributed solely to the publication of the PRISMA-NMA.

Critical evaluation of reporting quality of network meta-analyses assessing the effectiveness of acupuncture

BACKGROUND

and purpose: Network meta-analyses (NMAs) comparing the effectiveness of multiple acupuncture have been published but the key concepts underlying NMAs have not been properly reported. This critical evaluation aims to assess the completeness of reporting for NMAs of acupuncture to enhance the validity of findings.

MATERIALS AND METHODS

Five databases were searched. The characteristics and reporting quality based on the Preferred Reporting Items for Systematic Reviews and Meta-analyses Extension Statement for reporting of NMA (PRISMA-NMA) were evaluated. NMAs were categorized into two groups according to the publication date, reporting guideline, and intervention. The group differences and correlation coefficients were calculated.

RESULTS

Forty-two NMAs of acupuncture were evaluated. The reporting quality for new items in the key concepts of NMAs was low (median 70.0 % (interquartile range 60.0-80.0)). While the issue of consistency was well reported, the assumption of transitivity and network geometry showed poor reporting. Seventeen studies that followed the PRISMA-NMA guideline showed a higher reporting rate for essential concepts of NMA. The recency of publication did not guarantee clear reporting.

CONCLUSION

The reporting quality of NMAs of acupuncture was low. The researchers should follow the guidelines on the reporting of NMAs.

A data science approach to drug safety: Semantic and visual mining of adverse drug events from clinical trials of pain treatments

Clinical trials are the basis of Evidence-Based Medicine. Trial results are reviewed by experts and consensus panels for producing meta-analyses and clinical practice guidelines. However, reviewing these results is a long and tedious task, hence the meta-analyses and guidelines are not updated each time a new trial is published. Moreover, the independence of experts may be difficult to appraise. On the contrary, in many other domains, including medical risk analysis, the advent of data science, big data and visual analytics allowed moving from expert-based to fact-based knowledge. Since 12 years, many trial results are publicly available online in trial registries. Nevertheless, data science methods have not yet been applied widely to trial data. In this paper, we present a platform for analyzing the safety events reported during clinical trials and published in trial registries. This platform is based on an ontological model including 582 trials on pain treatments, and uses semantic web technologies for querying this dataset at various levels of granularity. It also relies on a 26-dimensional flower glyph for the visualization of the Adverse Drug Events (ADE) rates in 13 categories and 2 levels of seriousness. We illustrate the interest of this platform through several use cases and we were able to find back conclusions that were initially found during meta-analyses. The platform was presented to four experts in drug safety, and is publicly available online, with the ontology of pain treatment ADE.

Comparative efficacy and safety of drug treatment for premature ejaculation: A systemic review and Bayesian network meta-analysis

To assess the comparative efficacy and safety of drug treatments for premature ejaculation. A systemic review and Bayesian network meta-analysis were executed on randomised controlled trials of drug interventions for premature ejaculation. Intravaginal ejaculation latency time and related adverse effects were outcome measures. A total of 44 RCTs with 11,008 patients were included in our NMA. In therapy <8 weeks, the ranking of drug efficacy was topical creams >selective serotonin reuptake inhibitor (SSRI)+ phosphodiesterase 5 inhibitor (PDE5i) > PDE5i > sertraline > clomipramine > paroxetine > dapoxetine 60 milligram (mg) > dapoxetine 30 mg > fluoxetine>citalopram > duloxetine>placebo. In therapy ≥ 8 weeks, the ranking of drug efficacy was SSRI + PDE5i > topical creams > paroxetine > tramadol > PDE5i > fluoxetine > dapoxetine 60 mg > dapoxetine 30 mg > clomipramine>citalopram > placebo. For total adverse events, clomipramine, dapoxetine 30 mg, dapoxetine 60 mg, paroxetine, PDE5i, SSRI + PDE5i and tramadol had a higher risk than placebo. In conclusion, in ≥8 weeks of therapy, the drug combination of SSRI + PDE5i was the most effective PE therapy. In <8 weeks of therapy, the efficacy of local anaesthetics was best. All drug treatments were ranked better than placebo. In general, drugs with better effects had more obvious side effects.

Safety of pharmacologic interventions for neuropsychiatric symptoms in dementia: a systematic review and network meta-analysis

Background

Prescribing trends suggest that pharmacologic alternatives to antipsychotics are gaining in popularity, but randomized trial (RCT) data of their comparative safety is scarce. Our objective was to describe the comparative safety of pharmacologic interventions for treating neuropsychiatric symptoms in dementia.

Methods

We searched MEDLINE, EMBASE, CENTRAL, CINAHL, and PsycINFO, from inception to May 28, 2019, for studies of pharmacologic interventions used to treat neuropsychiatric symptoms in dementia. Dementia care partners selected fracture risk as our primary outcome. Pairs of reviewers, working independently, conducted all study screening, data abstraction, and risk of bias appraisal. We conducted Bayesian random-effects network meta-analyses (NMAs) using data from RCTs to derive odds ratios (ORs). In secondary analyses, we conducted frequentist random-effects NMAs using data from RCTs and Bayesian three-level hierarchical random-effects NMAs incorporating data from RCTs and non-randomized studies.

Results

Our systematic review included 209 randomized and non-randomized studies (889,378 persons with dementia). In NMAs of data from randomized trials, there were no increased odds of fracture associated with any intervention in primary analyses; however, data were sparse. We found increased odds of cerebrovascular events associated with antipsychotics (odds ratio [OR] 2.12, 95% credible interval [CrI] 1.29 to 3.62; number needed to harm [NNH] = 99) and increased odds of falls associated with dextromethorphan-quinidine (OR 4.16, 95% CrI 1.47 to 14.22; NNH = 55) compared to placebo in persons with dementia. In a subgroup of persons with Alzheimer disease, antipsychotics were associated with increased odds of fracture compared to anticonvulsants (OR 54.1, 95% CrI 1.15 to 38,300; NNH = 18). In older persons (mean age ≥ 80 years) with dementia, anticonvulsants were associated with increased odds of death compared to placebo (OR 8.36, 95% CrI 1.17 to 203.4; NNH = 35) and antipsychotics were associated with increased odds of death compared to antidepressants (OR 5.28, 95% CrI 1.06 to 3.51; NNH = 47).

Conclusion

Although antipsychotics were associated with greater harm than antidepressants and anticonvulsants in subgroups of persons with dementia, medications used in lieu of antipsychotics for treating neuropsychiatric symptoms in dementia, such as anticonvulsants and dextromethorphan-quinidine, were also associated with harm. Decision-making concerning treatments prescribed in lieu of antipsychotics should include potential harms.

PROSPERO registration

CRD42017050130.

Statistical analyses and quality of individual participant data network meta-analyses were suboptimal: a cross-sectional study

BACKGROUND

Network meta-analyses using individual participant data (IPD-NMAs) have been increasingly used to compare the effects of multiple interventions. Although there have been many studies on statistical methods for IPD-NMAs, it is unclear whether there are statistical defects in published IPD-NMAs and whether the reporting of statistical analyses has improved. This study aimed to investigate statistical methods used and assess the reporting and methodological quality of IPD-NMAs.

METHODS

We searched four bibliographic databases to identify published IPD-NMAs. The methodological quality was assessed using AMSTAR-2 and reporting quality assessed based on PRISMA-IPD and PRISMA-NMA. We performed stratified analyses and correlation analyses to explore the factors that might affect quality.

RESULTS

We identified 21 IPD-NMAs. Only 23.8% of the included IPD-NMAs reported statistical techniques used for missing participant data, 42.9% assessed the consistency, and none assessed the transitivity. None of the included IPD-NMAs reported sources of funding for trials included, only 9.5% stated pre-registration of protocols, and 28.6% assessed the risk of bias in individual studies. For reporting quality, compliance rates were lower than 50.0% for more than half of the items. Less than 15.0% of the IPD-NMAs reported data integrity, presented the network geometry, or clarified risk of bias across studies. IPD-NMAs with statistical or epidemiological authors often better assessed the inconsistency (P = 0.017). IPD-NMAs with a priori protocol were associated with higher reporting quality in terms of search (P = 0.046), data collection process (P = 0.031), and syntheses of results (P = 0.006).

CONCLUSIONS

The reporting of statistical methods and compliance rates of methodological and reporting items of IPD-NMAs were suboptimal. Authors of future IPD-NMAs should address the identified flaws and strictly adhere to methodological and reporting guidelines.

A scoping review of network meta-analyses assessing the efficacy and safety of complementary and alternative medicine interventions

BACKGROUND

Network meta-analysis (NMA) has rapidly grown in use during the past decade for the comparison of healthcare interventions. While its general use in the comparison of conventional medicines has been studied previously, to our awareness, its use to assess complementary and alternative medicines (CAM) has not been studied. A scoping review of the literature was performed to identify systematic reviews incorporating NMAs involving one or more CAM interventions.

METHODS

An information specialist executed a multi-database search (e.g., MEDLINE, Embase, Cochrane), and two reviewers performed study selection and data collection. Information on publication characteristics, diseases studied, interventions compared, reporting transparency, outcomes assessed, and other parameters were extracted from each review.

RESULTS

A total of 89 SR/NMAs were included. The largest number of NMAs was conducted in China (39.3%), followed by the United Kingdom (12.4%) and the United States (9.0%). Reviews were published between 2010 and 2018, with the majority published between 2015 and 2018. More than 90 different CAM therapies appeared at least once, and the median number per NMA was 2 (IQR 1-4); 20.2% of reviews consisted of only CAM therapies. Dietary supplements (51.1%) and vitamins and minerals (42.2%) were the most commonly studied therapies, followed by electrical stimulation (31.1%), herbal medicines (24.4%), and acupuncture and related treatments (22.2%). A diverse set of conditions was identified, the most common being various forms of cancer (11.1%), osteoarthritis of the hip/knee (7.8%), and depression (5.9%). Most reviews adequately addressed a majority of the PRISMA NMA extension items; however, there were limitations in indication of an existing review protocol, exploration of network geometry, and exploration of risk of bias across studies, such as publication bias.

CONCLUSION

The use of NMA to assess the effectiveness of CAM interventions is growing rapidly. Efforts to identify priority topics for future CAM-related NMAs and to enhance methods for CAM comparisons with conventional medicine are needed. SYSTEMATIC REVIEW REGISTRATION: https://ruor.uottawa.ca/handle/10393/35658.

CINeMA: An approach for assessing confidence in the results of a network meta-analysis

BACKGROUND

The evaluation of the credibility of results from a meta-analysis has become an important part of the evidence synthesis process. We present a methodological framework to evaluate confidence in the results from network meta-analyses, Confidence in Network Meta-Analysis (CINeMA), when multiple interventions are compared.

METHODOLOGY

CINeMA considers 6 domains: (i) within-study bias, (ii) reporting bias, (iii) indirectness, (iv) imprecision, (v) heterogeneity, and (vi) incoherence. Key to judgments about within-study bias and indirectness is the percentage contribution matrix, which shows how much information each study contributes to the results from network meta-analysis. The contribution matrix can easily be computed using a freely available web application. In evaluating imprecision, heterogeneity, and incoherence, we consider the impact of these components of variability in forming clinical decisions.

CONCLUSIONS

Via 3 examples, we show that CINeMA improves transparency and avoids the selective use of evidence when forming judgments, thus limiting subjectivity in the process. CINeMA is easy to apply even in large and complicated networks.

Generating comparative evidence on new drugs and devices before approval

Fewer than half of new drugs have data on their comparative benefits and harms against existing treatment options at the time of regulatory approval in Europe and the USA. Even when active-comparator trials exist, they might not produce meaningful data to inform decisions in clinical practice and health policy. The uncertainty associated with the paucity of well designed active-comparator trials has been compounded by legal and regulatory changes in Europe and the USA that have created a complex mix of expedited programmes aimed at facilitating faster access to new drugs. Comparative evidence generation is even sparser for medical devices. Some have argued that the current process for regulatory approval needs to generate more evidence that is useful for patients, clinicians, and payers in health-care systems. We propose a set of five key principles relevant to the European Medicines Agency, European medical device regulatory agencies, US Food and Drug Administration, as well as payers, that we believe will provide the necessary incentives for pharmaceutical and device companies to generate comparative data on drugs and devices and assure timely availability of evidence that is useful for decision making. First, labelling should routinely inform patients and clinicians whether comparative data exist on new products. Second, regulators should be more selective in their use of programmes that facilitate drug and device approvals on the basis of incomplete benefit and harm data. Third, regulators should encourage the conduct of randomised trials with active comparators. Fourth, regulators should use prospectively designed network meta-analyses based on existing and future randomised trials. Last, payers should use their policy levers and negotiating power to incentivise the generation of comparative evidence on new and existing drugs and devices, for example, by explicitly considering proven added benefit in pricing and payment decisions.

BUGSnet: an R package to facilitate the conduct and reporting of Bayesian network Meta-analyses

Background

Several reviews have noted shortcomings regarding the quality and reporting of network meta-analyses (NMAs). We suspect that this issue may be partially attributable to limitations in current NMA software which do not readily produce all of the output needed to satisfy current guidelines.

Results

To better facilitate the conduct and reporting of NMAs, we have created an R package called “BUGSnet” (Bayesian inference Using Gibbs Sampling to conduct a Network meta-analysis). This R package relies upon Just Another Gibbs Sampler (JAGS) to conduct Bayesian NMA using a generalized linear model. BUGSnet contains a suite of functions that can be used to describe the evidence network, estimate a model and assess the model fit and convergence, assess the presence of heterogeneity and inconsistency, and output the results in a variety of formats including league tables and surface under the cumulative rank curve (SUCRA) plots. We provide a demonstration of the functions contained within BUGSnet by recreating a Bayesian NMA found in the second technical support document composed by the National Institute for Health and Care Excellence Decision Support Unit (NICE-DSU). We have also mapped these functions to checklist items within current reporting and best practice guidelines.

Conclusion

BUGSnet is a new R package that can be used to conduct a Bayesian NMA and produce all of the necessary output needed to satisfy current scientific and regulatory standards. We hope that this software will help to improve the conduct and reporting of NMAs.

How does exercise treatment compare with antihypertensive medications? A network meta-analysis of 391 randomised controlled trials assessing exercise and medication effects on systolic blood pressure

OBJECTIVE

To compare the effect of exercise regimens and medications on systolic blood pressure (SBP).

DATA SOURCES

Medline (via PubMed) and the Cochrane Library.

ELIGIBILITY CRITERIA

Randomised controlled trials (RCTs) of angiotensin-converting enzyme inhibitors (ACE-I), angiotensin-2 receptor blockers (ARBs), β-blockers, calcium channel blockers (CCBs) and diuretics were identified from existing Cochrane reviews. A previously published meta-analysis of exercise interventions was updated to identify recent RCTs that tested the SBP-lowering effects of endurance, dynamic resistance, isometric resistance, and combined endurance and resistance exercise interventions (up to September 2018).

DESIGN

Random-effects network meta-analysis.

OUTCOME

Difference in mean change from baseline SBP between comparator treatments (change from baseline in one group minus that in the other group) and its 95% credible interval (95% CrI), measured in mmHg.

RESULTS

We included a total of 391 RCTs, 197 of which evaluated exercise interventions (10 461 participants) and 194 evaluated antihypertensive medications (29 281 participants). No RCTs compared directly exercise against medications. While all medication trials included hypertensive populations, only 56 exercise trials included hypertensive participants (≥140 mmHg), corresponding to 3508 individuals. In a 10% random sample, risk of bias was higher in exercise RCTs, primarily due to lack of blinding and incomplete outcome data. In analyses that combined all populations, antihypertensive medications achieved higher reductions in baseline SBP compared with exercise interventions (mean difference -3.96 mmHg, 95% CrI -5.02 to -2.91). Compared with control, all types of exercise (including combination of endurance and resistance) and all classes of antihypertensive medications were effective in lowering baseline SBP. Among hypertensive populations, there were no detectable differences in the SBP-lowering effects of ACE-I, ARB, β-blocker and diuretic medications when compared with endurance or dynamic resistance exercise. There was no detectable inconsistency between direct and indirect comparisons. Although there was evidence of small-study effects, this affected both medication and exercise trials.

CONCLUSIONS

The effect of exercise interventions on SBP remains under-studied, especially among hypertensive populations. Our findings confirm modest but consistent reductions in SBP in many studied exercise interventions across all populations but individuals receiving medications generally achieved greater reductions than those following structured exercise regimens. Assuming equally reliable estimates, the SBP-lowering effect of exercise among hypertensive populations appears similar to that of commonly used antihypertensive medications. Generalisability of these findings to real-world clinical settings should be further evaluated.

The number needed to treat in pairwise and network meta-analysis and its graphical representation

OBJECTIVE

The objective of this study was to present ways to graphically represent a number needed to treat (NNT) in (network) meta-analysis (NMA).

STUDY DESIGN AND SETTING

A barrier to using NNT in NMA when an odds ratio (OR) or risk ratio (RR) is used is the determination of a single control event rate (CER). We discuss approaches to calculate a CER, and illustrate six graphical methods for NNT from NMA. We illustrate the graphical approaches using an NMA of cognitive enhancers for Alzheimer's dementia.

RESULTS

The NNT calculation using a relative effect measure, such as OR and RR, requires a CER value, but different CERs, including mean CER across studies, pooled CER in meta-analysis, and expert opinion-based CER may result in different NNTs. An NNT from NMA can be presented in a bar plot, Cates plot, or forest plot for a single outcome, and a bubble plot, scatterplot, or rank-heat plot for ≥2 outcomes. Each plot is associated with different properties and can serve different needs.

CONCLUSION

Caution is needed in NNT interpretation, as considerations such as selection of effect size and CER, and CER assumption across multiple comparisons, may impact NNT and decision-making. The proposed graphs are helpful to interpret NNTs calculated from (network) meta-analyses.

Upfront Surgery versus Neoadjuvant Therapy for Resectable Pancreatic Cancer: Systematic Review and Bayesian Network Meta-analysis

Current treatment recommendations for resectable pancreatic cancer support upfront resection and adjuvant therapy. Randomized controlled trials offering comparison with the emerging neoadjuvant approach are lacking. This review aims to compare both treatment strategies for resectable pancreatic cancer. PubMed, MEDLINE, Embase, Cochrane Database and Cochrane Databases were searched for studies comparing neoadjuvant and surgery-first with adjuvant therapy for resectable pancreatic cancer. A Bayesian network meta-analysis was conducted using the Markov chain Monte Carlo method. Cochrane Collaboration’s risk of bias, ROBINS-I and GRADE tools were used to assess quality and risk of bias of included trials. 9 studies compared neoadjuvant therapy and surgery-first with adjuvant therapy (n = 22,285). Aggregate rate (AR) of R0 resection for neoadjuvant therapy was 0.8008 (0.3636–0.9144) versus 0.7515 (0.2026–0.8611) odds ratio (O.R.) 1.27 (95% CI 0.60–1.96). 1-year survival AR for neoadjuvant therapy was 0.7969 (0.6061–0.9500) versus 0.7481 (0.4848–0.8500) O.R. 1.38 (95% CI 0.69–2.96). 2-year survival AR for neoadjuvant therapy was 0.5178 (0.3000–0.5970) versus 0.5131 (0.2727–0.5346) O.R. 1.26 (95% CI 0.94–1.74). 5-year AR survival for neoadjuvant therapy was 0.2069 (0.0323–0.3300) versus 0.1783 (0.0606–0.2300) O.R. 1.19 (95% CI 0.65–1.73). In conclusion neoadjuvant therapy may offer benefit over surgery-first and adjuvant therapy. However, further randomized controlled trials are needed.

Description of network meta-analysis geometry: A metrics design study

Background

The conduction and report of network meta-analysis (NMA), including the presentation of the network-plot, should be transparent. We aimed to propose metrics adapted from graph theory and social network-analysis literature to numerically describe NMA geometry.

Methods

A previous systematic review of NMAs of pharmacological interventions was performed. Data on the graph’s presentation were collected. Network-plots were reproduced using Gephi 0.9.1. Eleven geometric metrics were tested. The Spearman test for non-parametric correlation analyses and the Bland-Altman and Lin’s Concordance tests were performed (IBM SPSS Statistics 24.0).

Results

From the 477 identified NMAs only 167 graphs could be reproduced because they provided enough information on the plot characteristics. The median nodes and edges were 8 (IQR 6–11) and 10 (IQR 6–16), respectively, with 22 included studies (IQR 13–35). Metrics such as density (median 0.39, ranged 0.07–1.00), median thickness (2.0, IQR 1.0–3.0), percentages of common comparators (median 68%), and strong edges (median 53%) were found to contribute to the description of NMA geometry. Mean thickness, average weighted degree and average path length produced similar results than other metrics, but they can lead to misleading conclusions.

Conclusions

We suggest the incorporation of seven simple metrics to report NMA geometry. Editors and peer-reviews should ensure that guidelines for NMA report are strictly followed before publication.

Conducting and interpreting results of network meta-analyses in type 2 diabetes mellitus: A review of network meta-analyses that include sodium glucose co-transporter 2 inhibitors

AIMS

Network meta-analyses (NMAs) are valuable ways to generate comparative effectiveness data for therapies available to treat type 2 diabetes mellitus (T2DM). This review assesses NMAs that evaluate sodium glucose co-transporter 2 (SGLT2) inhibitors for treatment of T2DM and discusses potential issues in conducting and interpreting NMAs.

METHODS

A systematic literature search was conducted on September 13, 2018 using the search terms "network meta-analysis," "SGLT2," variations of these terms, and individual SGLT2 inhibitor names. Extracted data included NMA objectives, methods, target populations, treatments, study endpoints, length of follow-up, and funding. Differences between NMAs were investigated.

RESULTS

Thirty-five full-length publications met criteria for inclusion. In most NMAs, the target population was defined by therapeutic regimen (e.g., combination with metformin). Follow-up intervals permitted in NMAs varied considerably (range, 4-208 weeks). Twenty-nine NMAs included dapagliflozin, 28 evaluated canagliflozin, and 27 evaluated empagliflozin. Nine NMAs used frequentist methods; 16 used Bayesian methods. Six NMAs were funded by pharmaceutical manufacturers. Heterogeneity across NMAs was seen in scope, time frame, and other aspects of analytic design.

CONCLUSIONS

Although this review indicates that methodological guidelines for reporting NMAs were generally followed, it also emphasizes the need for T2DM-specific guidance requiring clear reporting of NMA scope and objectives to aid appropriate interpretation and use of NMA results.

Comparative efficacy and safety of cognitive enhancers for treating vascular cognitive impairment: systematic review and Bayesian network meta-analysis

Objective

To assess and compare the clinical efficacy and safety of cognitive enhancers (donepezil, galantamine, rivastigmine, and memantine) on cognition, behavior, function, and global status in patients with vascular cognitive impairment.

Data sources

The initial literature search was performed with PubMed, EMBASE, the Cochrane Methodology Register, the Cochrane Central Register of Controlled Trials, and Cumulative Index to Nursing & Allied Health (CINAHL) from inception to January 2018 for studies regarding donepezil, galantamine, rivastigmine, and memantine for treatment of vascular cognitive impairment.

Data selection

Randomized controlled trials on donepezil, galantamine, rivastigmine, and memantine as monotherapy in the treatment of vascular cognitive impairment were included. A Bayesian network meta-analysis was conducted.

Outcome measures

Efficacy was assessed by changes in scores of the Alzheimer's Disease Assessment Scale, cognitive subscale, Mini-Mental State Examination, Neuropsychiatric Inventory scores and Clinician's Interview-Based Impression of Change Scale Plus Caregiver's Input, Activities of Daily Living, the Clinical Dementia Rating scale. Safety was evaluated by mortality, total adverse events (TAEs), serious adverse events (SAEs), nausea, vomiting. diarrhea, or cerebrovascular accidents (CVAs).

Results

After screening 1717 citations, 12 randomized controlled trials were included. Donepezil and rivastigmine (mean difference (e) = -0.77, 95% confidence interval (CI): 0.25-1.32; MD = 1.05, 95% CI: 0.18-1.79) were significantly more effective than placebo in reducing Mini-Mental State Examination scores. Donepezil, galantamine, and memantine (MD = -1.30, 95% CI: -2.27 to -0.42; MD = -1.67, 95% CI: -3.36 to -0.06; MD = -2.27, 95% CI: -3.91 to -0.53) showed superior benefits on the Alzheimer's Disease Assessment Scale-cognitive scores compared with placebo. Memantine (MD = 2.71, 95% CI: 1.05-7.29) improved global status (Clinician's Interview-Based Impression of Change Scale Plus Caregiver's Input) more than the placebo. Safety results revealed that donepezil 10 mg (odds ratio (OR) = 3.04, 95% CI: 1.86-5.41) contributed to higer risk of adverse events than placebo. Galantamine (OR = 5.64, 95% CI: 1.31-26.71) increased the risk of nausea. Rivastigmine (OR = 16.80, 95% CI: 1.78-319.26) increased the risk of vomiting. No agents displayed a significant risk of serious adverse events, mortality, cerebrovascular accidents, or diarrhea.

Conclusion

We found significant efficacy of donepezil, galantamine, and memantine on cognition. Memantine can provide significant efficacy in global status. They are all safe and well tolerated.

Living network meta-analysis was feasible when considering the pace of evidence generation

OBJECTIVES

The aim of the study was to assess the feasibility of living network meta-analysis (NMA) taking into account the pace of evidence generation across different medical areas.

STUDY DESIGN AND SETTING

We performed a systematic review to identify published NMAs. For each NMA, we calculated the cumulative number of new trials. To assess the feasibility of living NMA, we considered different update frequencies (4, 6, and 12 months), then evaluated the number of new trials to be included at each update in the NMA and the workload percentage for an update relative to the initial NMA.

RESULTS

We identified 77 NMAs covering 17 different medical areas; 60 (78%) had fewer than four new trials included per year, on average, and 5 (7%) had more than seven trials. With an update frequency of 4, 6, and 12 months, the median number of new trials to be included in the NMA was 0 (interquartile range, 0-1), 1 (0-2), and 2 (1-4), respectively, with mean of 4%, 5%, and 11% workload per update, respectively.

CONCLUSION

The workload associated with updating a living NMA represents about one-tenth of the initial workload; therefore, living NMA is manageable.

Assessing the methodological and reporting quality of network meta-analyses in Chinese medicine

BACKGROUND

An increasing number of network meta-analyses (NMAs) in traditional Chinese medicine (TCM) have been published recently, but the quality of them was lack of assessment. This study aims to evaluate the methodological and reporting quality of NMAs in TCM.

METHODS

Six electronic databases, including PubMed, the Cochrane Central Register of Controlled Trials (CENTRAL), Embase, China National Knowledge Infrastructure (CNKI), Wanfang and Chinese Biomedical Literature Database (CBM) from inception to January 2018, were searched. NMAs of TCM were included. A measurement tool to assess the methodological quality of systematic reviews (AMSTAR) and the PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-analyses of Health Care Interventions (PRISMA-NMA) were used to assess the methodological and reporting quality of the included NMAs.

RESULTS

A total of 40 NMAs, including 2535 randomized controlled trials (RCTs), were included. They were published between December 2012 and November 2017. The median score and interquartile range of methodological and reporting quality was 7 (6-8) and 22 (19.1-27.1). Serious methodological flaws existed in the following aspects: the status of publication (22.5%), a list of studies provided (0%), assessment of publication bias (37.5%), and conflicts of interest (12.5%). Several items need to be improved in reporting, especially for Protocol and registration (2.5%), Data items (22.5%), Risk of bias across studies (Methods section) (37.5%), Results of individual studies (27.5%), Risk of bias across studies (Results section) (40%), Results of additional analyses (35%), and Funding (15%).

CONCLUSIONS

The methodological and reporting quality of NMAs in TCM is moderate. Identified shortcomings of published NMAs should be taken into consideration in further trainings of authors and editors of NMAs in TCM. Future researchers should be encouraged to apply PRISMA-NMA, and a recognized tool for the assessment of NMA methodology was wanted.

Partially systematic thoughts on the history of systematic reviews

Six years after the launch of Systematic Reviews by Biomed Central, this article is part of the celebration of the journal. It contains personal reflections on the past, present and future of systematic reviews, using examples relevant to the role of systematic reviews in cataloguing and analysing research, assessing quality and planning new studies. The focus is on the most common of the various types of systematic review in health and social care, namely those assessing the effects of interventions.

A systematic survey shows that reporting and handling of missing outcome data in networks of interventions is poor

Background

To provide empirical evidence about prevalence, reporting and handling of missing outcome data in systematic reviews with network meta-analysis and acknowledgement of their impact on the conclusions.

Methods

We conducted a systematic survey including all published systematic reviews of randomized controlled trials comparing at least three interventions from January 1, 2009 until March 31, 2017.

Results

We retrieved 387 systematic reviews with network meta-analysis. Description of missing outcome data was available in 63 reviews. Intention-to-treat analysis was the most prevalent method (71%), followed by missing outcome data investigated as secondary outcome (e.g., acceptability) (40%). Bias due to missing outcome data was evaluated in half the reviews with explicit judgments in 18 (10%) reviews. Only 88 reviews interpreted their results acknowledging the implications of missing outcome data and mostly using the network meta-analysis results on missing outcome data as secondary outcome. We were unable to judge the actual strategy applied to deal with missing outcome data in 65% of the reviews due to insufficient information. Six percent of network meta-analyses were re-analyzed in sensitivity analysis considering missing outcome data, while 4% explicitly justified the strategy for dealing with missing outcome data.

Conclusions

The description and handling of missing outcome data as well as the acknowledgment of their implications for the conclusions from network meta-analysis are deemed underreported.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0576-9) contains supplementary material, which is available to authorized users.

PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation

Scoping reviews, a type of knowledge synthesis, follow a systematic approach to map evidence on a topic and identify main concepts, theories, sources, and knowledge gaps. Although more scoping reviews are being done, their methodological and reporting quality need improvement. This document presents the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) checklist and explanation. The checklist was developed by a 24-member expert panel and 2 research leads following published guidance from the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network. The final checklist contains 20 essential reporting items and 2 optional items. The authors provide a rationale and an example of good reporting for each item. The intent of the PRISMA-ScR is to help readers (including researchers, publishers, commissioners, policymakers, health care providers, guideline developers, and patients or consumers) develop a greater understanding of relevant terminology, core concepts, and key items to report for scoping reviews.

GRADE approach to rate the certainty from a network meta-analysis: avoiding spurious judgments of imprecision in sparse networks

When direct and indirect estimates of treatment effects are coherent, network meta-analysis (NMA) estimates should have increased precision (narrower confidence or credible intervals compared with relying on direct estimates alone), a benefit of NMA. We have, however, observed cases of sparse networks in which combining direct and indirect estimates results in marked widening of the confidence intervals. In many cases, the assumption of common between-study heterogeneity across the network seems to be responsible for this counterintuitive result. Although the assumption of common between-study heterogeneity across paired comparisons may, in many cases, not be appropriate, it is required to ensure the feasibility of estimating NMA treatment effects. This is especially the case in sparse networks, in which data are insufficient to reliably estimate different variances across the network. The result, however, may be spuriously wide confidence intervals for some of the comparisons in the network (and, in the Grading of Recommendations Assessment, Development, and Evaluation approach, inappropriately low ratings of the certainty of the evidence through rating down for serious imprecision). Systematic reviewers should be aware of the problem and plan sensitivity analyses that produce intuitively sensible confidence intervals. These sensitivity analyses may include using informative priors for the between-study heterogeneity parameter in the Bayesian framework and the use of fixed effects models.

Applying GRADE to a network meta-analysis of antidepressants led to more conservative conclusions

OBJECTIVE

To explore the impact of applying the Grading of Recommendations and Assessment, Development, and Evaluation (GRADE) approach to assess the certainty of the evidence in a published network meta-analysis (NMA) of antidepressant therapies.

STUDY DESIGN AND SETTINGS

We applied the GRADE approach to rate the certainty of the evidence for two outcomes, efficacy and acceptability, in each of the 66 paired comparisons within a previously published NMA assessing the relative efficacy and acceptability of 12 new-generation antidepressants.

RESULTS

For the outcome of efficacy, of the 25 comparisons in which the 95% CrI of OR excluded 1, 18 had certainty of evidence rated high or moderate. For the outcome of acceptability, of the 13 comparisons whose 95% CrI excluded 1, 10 had certainty of evidence rated high or moderate. Of the 11 comparisons involving sertraline, the antidepressants that the authors of the NMA suggested to be best, only 3 demonstrated it to be more effective and only 3 showed better tolerance, based on a 95% CrI excluding 1 and a high or moderate rating of certainty.

CONCLUSIONS

In this example, application of GRADE highlighted varying evidence certainty, led to more conservative conclusions, and potentially avoided unwarranted strong inferences based on low certainty evidence.

An innovative and comprehensive technique to evaluate different measures of medication adherence: The network meta-analysis

BACKGROUND

Poor medication adherence is associated with adverse health outcomes and higher costs of care. However, inconsistencies in the assessment of adherence are found in the literature.

OBJECTIVE

To evaluate the effect of different measures of adherence in the comparative effectiveness of complex interventions to enhance patients' adherence to prescribed medications.

METHODS

A systematic review with network meta-analysis was performed. Electronic searches for relevant pairwise meta-analysis including trials of interventions that aimed to improve medication adherence were performed in PubMed. Data extraction was conducted with eligible trials evaluating short-period adherence follow-up (until 3 months) using any measure of adherence: self-report, pill count, or MEMS (medication event monitoring system). To standardize the results obtained with these different measures, an overall composite measure and an objective composite measure were also calculated. Network meta-analyses for each measure of adherence were built. Rank order and surface under the cumulative ranking curve analyses (SUCRA) were performed.

RESULTS

Ninety-one trials were included in the network meta-analyses. The five network meta-analyses demonstrated robustness and reliability. Results obtained for all measures of adherence were similar across them and to both composite measures. For both composite measures, interventions comprising economic + technical components were the best option (90% of probability in SUCRA analysis) with statistical superiority against almost all other interventions and against standard care (odds ratio with 95% credibility interval ranging from 0.09 to 0.25 [0.02, 0.98]).

CONCLUSION

The use of network meta-analysis was reliable to compare different measures of adherence of complex interventions in short-periods follow-up. Analyses with longer follow-up periods are needed to confirm these results. Different measures of adherence produced similar results. The use of composite measures revealed reliable alternatives to establish a broader and more detailed picture of adherence.