Review of mixed treatment comparisons in published systematic reviews shows marked increase since 2009

Service-level barriers to and facilitators of accessibility to treatment for problematic alcohol use: a scoping review

Introduction

Services to treat problematic alcohol use (PAU) should be highly accessible to optimize treatment engagement. We conducted a scoping review to map characteristics of services for the treatment of PAU that have been reported in the literature to be barriers to or facilitators of access to treatment from the perspective of individuals with PAU.

Methods

A protocol was developed a priori, registered, and published. We searched MEDLINE®, Embase, the Cochrane Library, and additional grey literature sources from 2010 to April 2022 to identify primary qualitative research and surveys of adults with current or past PAU requiring treatment that were designed to identify modifiable characteristics of PAU treatment services (including psychosocial and pharmacologic interventions) that were perceived to be barriers to or facilitators of access to treatment. Studies of concurrent PAU and other substance use disorders were excluded. Study selection was performed by multiple review team members. Emergent barriers were coded and mapped to the accessibility dimensions of the Levesque framework of healthcare access, then descriptively summarized.

Results

One-hundred-and-nine included studies reported an extensive array of unique service-level barriers that could act alone or together to prevent treatment accessibility. These included but were not limited to lack of an obvious entry point, complexity of the care pathway, high financial cost, unacceptably long wait times, lack of geographically accessible treatment, inconvenient appointment hours, poor cultural/demographic sensitivity, lack of anonymity/privacy, lack of services to treat concurrent PAU and mental health problems.

Discussion

Barriers generally aligned with recent reviews of the substance use disorder literature. Ranking of barriers may be explored in a future discrete choice experiment of PAU service users. The rich qualitative findings of this review may support the design of new or modification of existing services for people with PAU to improve accessibility.

Systematic Review Registration

Open Science Framework doi: 10.17605/OSF.IO/S849R.

Methodological quality of network meta-analysis in dentistry: a meta-research

This meta-research aimed to provide an overview of the methodological quality and risk of bias of network meta-analyses (NMA) in dentistry. Searches for NMA of randomized clinical trials with clinical outcomes in dentistry were performed in databases up to January 2022. Two reviewers independently screened titles/abstracts, selected full texts, and extracted the data. The adherence to PRISMA-NMA reporting guideline, the AMSTAR-2 methodological quality tool, and the ROBIS risk of bias tool were assessed in the studies. Correlation between the PRISMA-NMA adherence and the AMSTAR-2 and ROBIS results was also investigated. Sixty-two NMA studies were included and presented varied methodological quality. According to AMSTAR-2, half of the NMA presented moderate quality (n = 32; 51.6%). The adherence to PRISMA-NMA also varied. Only 36 studies (58.1%) prospectively registered the protocol. Other issues lacking of reporting were data related were data related to the NMA geometry and the assessment of results consistency, and the evaluation of risk of bias across the studies. ROBIS assessment showed a high risk of bias mainly for domains 1 (study eligibility criteria) and 2 (identification and selection of studies). Correlation coefficients between the PRISMA-NMA adherence and the AMSTAR-2 and ROBIS results showed moderate correlation (rho < 0.6). Overall, NMA studies in dentistry were of moderate quality and at high risk of bias in several domains, especially study selection. Future reviews should be better planned and conducted and have higher compliance with reporting and quality assessment tools.

A Gentle Introduction to Bayesian Network Meta-Analysis Using an Automated R Package

Network meta-analysis is an extension of standard meta-analysis. It allows researchers to build a network of evidence to compare multiple interventions that may have not been compared directly in existing publications. With a Bayesian approach, network meta-analysis can be used to obtain a posterior probability distribution of all the relative treatment effects, which allows for the estimation of relative treatment effects to quantify the uncertainty of parameter estimates, and to rank all the treatments in the network. Ranking treatments using both direct and indirect evidence can provide guidance to policy makers and clinicians for making decisions. The purpose of this paper is to introduce fundamental concepts of Bayesian network meta-analysis (BNMA) to researchers in psychology and social sciences. We discuss several essential concepts of BNMA, including the assumptions of homogeneity and consistency, the fixed and random effects models, prior specification, and model fit evaluation strategies, while pointing out some issues and areas where researchers should use caution in the application of BNMA. Additionally, using an automated R package, we provide a step-by-step demonstration on how to conduct and report the findings of BNMA with a real dataset of psychological interventions extracted from PubMed.

Network meta-analysis in psychology and educational sciences: A systematic review of their characteristics

Network meta-analysis (NMA) allows the combination of evidence on the effectiveness of several interventions. NMA has mainly been applied in the medical science field, whereas in the domain of psychology and educational sciences its use is less frequent. Consequently, systematic reviews that describe the characteristics of published NMAs are limited to the field of medicine, and nothing is known about the characteristics of NMAs published in the psychology and educational sciences field. However, this information is still relevant for the design of future simulation studies and for detecting good and bad research practices. Thus, this study describes the features of the meta-analytic datasets of NMAs published in the field of psychology and educational sciences, as well as their methodological characteristics, and compares them to those observed in the medical domain. Results show that the number of studies included is larger in NMAs from psychology and educational sciences, the most commonly used effect size is the standardized mean difference (unlike the odds ratio in medicine), the sample size is smaller, more intervention groups are included, and inconsistent effects are observed more often. These results can be used in future simulation studies to generate realistic datasets. Finally, we warn about the poor quality of reporting of some technical aspects of the NMA, such as the statistical model used.

Network meta-analysis of the therapeutic effects of various antibiotics on footrot in sheep and cattle

The present network meta-analysis was performed to compare the effects of antibiotics used in treating footrot in some ruminants and to rank these antibiotics based on their efficacy. Data of 14 eligible studies consisting of 5622 affected animals was included in the analysis. A Bayesian method and Markov Chain Monte Carlo (MCMC) simulations were utilized to analyze data. The estimated results were reported in the form of odds ratios (ORs) with 95% credible intervals (CrI). The Surface Under the Cumulative Ranking Curve (SUCRA) was used to rank antibiotics. Network meta-regressions (NMRs) were conducted to examine the influence of sample sizes, treatment duration, route of administration, and species of animals (sheep and cattle) on the overall outcome. The results indicated that gamithromycin impact on curing footrot was superior to other antibiotics and Lincomycin and oxytetracycline were ranked second and third. The difference between the impact of gamithromycin and amoxicillin (OR = 14.76, CrI: 1.07-193.49) and enrofloxacin (OR = 20.21, CrI: 1.57-229.25) on footrot was significant. There was a significant difference between the effect of oxytetracycline and enrofloxacin (OR = 5.24, CrI: 1.14-23.74) on footrot. The NMR performed based on species of animals fitted data better than network meta-analysis, suggesting erythromycin as the best third antibiotic instead of oxytetracycline. Egger's regression test and the shape of the funnel plot showed no publication bias among included studies. In conclusion, gamithromycin was associated with the highest curing rate benefit when used to treat footrot, followed by lincomycin and oxytetracycline/erythromycin. Among all evaluated antibiotics, enrofloxacin showed the lowest effects on footrot.

Mapping the characteristics, methodological quality and standards of reporting of network meta-analyses on antithrombotic therapies: An overview

BACKGROUND

Although a large number of network meta-analyses (NMAs) in the field of cardiology are available, little is known about their methodological quality. We aimed to map the characteristics and critically appraised the standards of conduct and evidence reporting of NMAs assessing antithrombotic therapies for the treatment or prophylaxis of heart diseases and cardiac surgical procedures.

METHODS

We systematically searched PubMed and Scopus to identify NMAs comparing the clinical effects of antithrombotic therapies. Overall characteristics of the NMAs were extracted and their reporting quality and methodological quality were evaluated using the PRISMA-NMA checklist and AMSTAR-2, respectively.

RESULTS

We found 86 NMAs published between 2007 and 2022. Comparisons among direct-acting oral anticoagulants were available in 61 (71%) NMAs. Although around 75% of NMAs stated that they followed international guidelines for conduct and reporting, only one third provided a protocol/register. Complete search strategies and publication bias assessment were lacking in around 53% and 59% of studies, respectively. Most NMAs (n = 77, 90%) provided supplemental material; however, only 5 (6%) made the complete raw data available. Network diagrams were depicted in most studies (n = 67, 78%), yet network geometry was described in only 11 (12.8%) of them. Mean adherence to the PRISMA-NMA checklist was 65.1 ± 16.5%. AMSTAR-2 assessment showed 88% of the NMAs had critically low methodological quality.

CONCLUSION

Although there is a wide diffusion of NMA-type studies on antithrombotics for heart diseases, their methodological and reporting quality remains suboptimal. This may reflect fragile clinical practices due to misleading conclusions from critically low-quality NMAs.

Network meta-analysis of the analgesic effectiveness of regional anaesthesia techniques for anterior cruciate ligament reconstruction

Anterior cruciate ligament reconstruction can cause moderate to severe acute postoperative pain. Despite advances in our understanding of knee innervation, consensus regarding the most effective regional anaesthesia techniques for this surgical population is lacking. This network meta-analysis compared effectiveness of regional anaesthesia techniques used to provide analgesia for anterior cruciate ligament reconstruction. Randomised trials examining regional anaesthesia techniques for analgesia following anterior cruciate ligament reconstruction were sought. The primary outcome was opioid consumption during the first 24 h postoperatively. Secondary outcomes were: rest pain at 0, 6, 12 and 24 h; area under the curve of pain over 24 h; and opioid-related adverse effects and functional recovery. Network meta-analysis was conducted using a frequentist approach. A total of 57 trials (4069 patients) investigating femoral nerve block, sciatic nerve block, adductor canal block, local anaesthetic infiltration, graft-donor site infiltration and systemic analgesia alone (control) were included. For opioid consumption, all regional anaesthesia techniques were superior to systemic analgesia alone, but differences between regional techniques were not significant. Single-injection femoral nerve block combined with sciatic nerve block had the highest p value probability for reducing postoperative opioid consumption and area under the curve for pain severity over 24 h (78% and 90%, respectively). Continuous femoral nerve block had the highest probability (87%) of reducing opioid-related adverse effects, while local infiltration analgesia had the highest probability (88%) of optimising functional recovery. In contrast, systemic analgesia, local infiltration analgesia and adductor canal block were each poor performers across all analgesic outcomes. Regional anaesthesia techniques that target both the femoral and sciatic nerve distributions, namely a combination of single-injection nerve blocks, provide the most consistent analgesic benefits for anterior cruciate ligament reconstruction compared with all other techniques but will most likely impair postoperative function. Importantly, adductor canal block, local infiltration analgesia and systemic analgesia alone each perform poorly for acute pain management following anterior cruciate ligament reconstruction.

Reporting and methodological quality of acupuncture network meta-analyses could be improved: an evidence mapping

BACKGROUND AND OBJECTIVES

To evaluate and map the reporting and methodological quality of network meta-analysis (NMA) on acupuncture.

METHODS

Published acupuncture NMAs were searched through eight databases from inception to February 2022. The reporting and methodological quality of included studies was assessed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Network Meta-Analysis (PRISMA-NMA) statement and the Assessment of Multiple Systematic Reviews 2 (AMSTAR-2) checklist.

RESULTS

A total of 113 NMAs were identified. Most (99, 87.61%) studies were performed in China. Most studies focused on multiple acupuncture techniques (82.30%), and the main studied ailments were pain and poststroke sequelae (20.24%). The median (interquartile range (IQR)) score of the reporting quality was 26.5 (25-28.5). However, poor reporting rates in the protocol and registration (33.63%) and geometry of the network (35.40%), especially for China-based studies, were identified. The methodological quality of only 2 (1.77%) English studies was high. The reporting rate of Chinese studies was below 15% on each of items 4, 7, 10, and 12.

CONCLUSION

The reporting quality of the NMAs was moderate, but the methodological quality was very low. The reporting and methodological quality of future NMAs, especially for Chinese studies, need further improvement.

Predictive P-score for treatment ranking in Bayesian network meta-analysis

Background

Network meta-analysis (NMA) is a widely used tool to compare multiple treatments by synthesizing different sources of evidence. Measures such as the surface under the cumulative ranking curve (SUCRA) and the P-score are increasingly used to quantify treatment ranking. They provide summary scores of treatments among the existing studies in an NMA. Clinicians are frequently interested in applying such evidence from the NMA to decision-making in the future. This prediction process needs to account for the heterogeneity between the existing studies in the NMA and a future study.

Methods

This article introduces the predictive P-score for informing treatment ranking in a future study via Bayesian models. Two NMAs were used to illustrate the proposed measure; the first assessed 4 treatment strategies for smoking cessation, and the second assessed treatments for all-grade treatment-related adverse events. For all treatments in both NMAs, we obtained their conventional frequentist P-scores, Bayesian P-scores, and predictive P-scores.

Results

In the two examples, the Bayesian P-scores were nearly identical to the corresponding frequentist P-scores for most treatments, while noticeable differences existed for some treatments, likely owing to the different assumptions made by the frequentist and Bayesian NMA models. Compared with the P-scores, the predictive P-scores generally had a trend to converge toward a common value of 0.5 due to the heterogeneity. The predictive P-scores’ numerical estimates and the associated plots of posterior distributions provided an intuitive way for clinicians to appraise treatments for new patients in a future study.

Conclusions

The proposed approach adapts the existing frequentist P-score to the Bayesian framework. The predictive P-score can help inform medical decision-making in future studies.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01397-5.

Progress and challenges of network meta-analysis

In the past years, network meta-analysis (NMA) has been widely used among clinicians, guideline makers, and health technology assessment agencies and has played an important role in clinical decision-making and guideline development. To inform further development of NMAs, we conducted a bibliometric analysis to assess the current status of published NMA methodological studies, summarized the methodological progress of seven types of NMAs, and discussed the current challenges of NMAs.

Bayesian meta-analysis using SAS PROC BGLIMM

Meta-analysis is commonly used to compare two treatments. Network meta-analysis (NMA) is a powerful extension for comparing and contrasting multiple treatments simultaneously in a systematic review of multiple clinical trials. Although the practical utility of meta-analysis is apparent, it is not always straightforward to implement, especially for those interested in a Bayesian approach. This paper demonstrates that the recently-developed SAS procedure BGLIMM provides an intuitive and computationally efficient means for conducting Bayesian meta-analysis in SAS, using a worked example of a smoking cessation NMA data set. BGLIMM gives practitioners an effective and simple way to implement Bayesian meta-analysis (pairwise and network, either contrast-based or arm-based) without requiring significant background in coding or statistical modeling. Those familiar with generalized linear mixed models, and especially the SAS procedure GLIMMIX, will find this tutorial a useful introduction to Bayesian meta-analysis in SAS.

The Quality of Methodological and Reporting in Network Meta-Analysis of Acupuncture and Moxibustion: A Cross-Sectional Survey

Background

Acupuncture had long been a primary treatment in the healthcare system of China. In recent years, there were more and more network meta-analyses (NMAs) in the field of acupuncture and moxibustion, but the quality evaluation of NMAs was rare.

Objectives

The goal of this study was to evaluate the methodological and reporting quality of NMAs and summarize the effects of different treatments of acupuncture and moxibustion.

Methods

PubMed, Embase, the Cochrane Library, China National Knowledge Infrastructure Database (CNKI), WanFang Database (WF), Chinese Scientific Journal Database (VIP), and Chinese Biomedical Literature Database (CBM) were searched from inception to January 2020 without any language restriction. In addition, the unpublished studies and the references of initially included literature were also retrieved manually. We included all relevant NMAs treated with acupuncture and moxibustion; other therapies such as traditional Chinese medicine and Western medicine may also be included, but at least three types fall under the category of acupuncture in each NMA. Outcome indicators were not limited. We selected AMSTAR2 and PRISMA-NMA to evaluate the methodological and reporting quality of eligible studies, respectively.

Results

In total, 29 NMAs were included finally, including 12 Chinese references and 17 English references. All eligible studies were published from May 2013 to August 2019. The number of interventions was between 4 and 22. The number of clinical trials included ranged from 10 to 121, with a total of 1098 clinical trials. The NMAs were involved in up to 23 diseases, knee osteoarthritis and primary dysmenorrhea covered with 3 NMAs separately, others focusing on chronic functional constipation, lumbar disc herniation, chronic fatigue syndrome, and the like. The Jadad scale and RoB scale were used as the bias risk assessment tools. Among them, 7 articles adopted the Jadad scale, 22 articles adopted the RoB scale (1 article adopted both the Jadad scale and RoB scale), and only 1 article did not mention the risk assessment tool. The AMSTAR2 methodological evaluation showed that the highest score was 13.5 points and the lowest was 4, with an average of 8.64 and a median of 9.5. According to the quality criteria, only one of them was in high quality, twenty-four were in medium quality, and four were in low quality. The PRISMA-NMA reporting quality evaluation showed that the highest score was 29 points and the lowest was 13.5, with an average of 23.62 and a median of 24.5; severe flaws also existed in some items, especially in "Structured summary," "Protocol and registration," "Search," "Data collection process," "Data items," "Additional analyses," "Risk of bias across studies," and "Results of additional analyses."

Conclusion

The number of NMAs in the field of acupuncture and moxibustion was still in the initial stage. Overall, their methodology and reports were of moderate quality. However, severe flaws also existed in some items. Because the eligible NMAs were limited, the conclusion needed further research to confirm its authenticity and reliability.

Exploiting relationships between outcomes in Bayesian multivariate network meta-analysis with an application to relapsing-remitting multiple sclerosis

In multivariate network meta-analysis (NMA), the piecemeal nature of the evidence base means that there may be treatment-outcome combinations for which no data is available. Most existing multivariate evidence synthesis models are either unable to estimate the missing treatment-outcome combinations, or can only do so under particularly strong assumptions, such as perfect between-study correlations between outcomes or constant effect size across outcomes. Many existing implementations are also limited to two treatments or two outcomes, or rely on model specification that is heavily tailored to the dimensions of the dataset. We present a Bayesian multivariate NMA model that estimates the missing treatment-outcome combinations via mappings between the population mean effects, while allowing the study-specific effects to be imperfectly correlated. The method is designed for aggregate-level data (rather than individual patient data) and is likely to be useful when modeling multiple sparsely reported outcomes, or when varying definitions of the same underlying outcome are adopted by different studies. We implement the model via a novel decomposition of the treatment effect variance, which can be specified efficiently for an arbitrary dataset given some basic assumptions regarding the correlation structure. The method is illustrated using data concerning the efficacy and liver-related safety of eight active treatments for relapsing-remitting multiple sclerosis. The results indicate that fingolimod and interferon beta-1b are the most efficacious treatments but also have some of the worst effects on liver safety. Dimethyl fumarate and glatiramer acetate perform reasonably on all of the efficacy and safety outcomes in the model.

Comparative Effectiveness of Angiotensin II Receptor Blockers in Patients With Hypertension in Japan - Systematic Review and Network Meta-Analysis

Background: Angiotensin II receptor blockers (ARBs) are widely used for the management of hypertension in Japan; however, comparative efficacy data within the ARB drug class remain limited. Methods and Results: This systematic literature review identified randomized controlled trials (RCT) indexed in PubMed and Ichushi in Japanese patients with hypertension receiving ARB monotherapy (azilsartan, candesartan cilexetil, irbesartan, losartan potassium, olmesartan medoxomil, telmisartan, valsartan) in at least 1 arm. Of 763 RCTs identified, 77 met the eligibility criteria; of which, 37 reported mean change in systolic blood pressure (SBP) and diastolic blood pressure (DBP) from baseline in the office setting and were used to construct the network. A fixed-effects model (FEM) showed the effect of each drug vs. the reference, azilsartan. Using the FEM, the mean (95% credible interval) change from baseline in SBP/DBP for candesartan cilexetil, irbesartan, losartan potassium, olmesartan medoxomil, telmisartan, and valsartan was 3.8 (2.9-4.8)/2.6 (2.0-3.1), 4.8 (2.0-7.5)/3.7 (1.8-5.6), 3.0 (0.8-5.1)/1.9 (0.5-3.3), 3.2 (1.2-5.1)/2.7 (1.3-4.1), 3.2 (0.8-5.6)/2.0 (0.3-3.6), and 3.1 (1.1-5.1)/2.4 (1.1-3.8) mmHg, respectively. Conclusions: The results of this meta-analysis provide evidence that azilsartan has a more favorable efficacy profile than the other ARBs in reducing SBP and DBP.

A simulation study to compare different estimation approaches for network meta-analysis and corresponding methods to evaluate the consistency assumption

Background

Network meta-analysis (NMA) is becoming increasingly popular in systematic reviews and health technology assessments. However, there is still ambiguity concerning the properties of the estimation approaches as well as for the methods to evaluate the consistency assumption.

Methods

We conducted a simulation study for networks with up to 5 interventions. We investigated the properties of different methods and give recommendations for practical application. We evaluated the performance of 3 different models for complex networks as well as corresponding global methods to evaluate the consistency assumption. The models are the frequentist graph-theoretical approach netmeta, the Bayesian mixed treatment comparisons (MTC) consistency model, and the MTC consistency model with stepwise removal of studies contributing to inconsistency identified in a leverage plot.

Results

We found that with a high degree of inconsistency none of the evaluated effect estimators produced reliable results, whereas with moderate or no inconsistency the estimator from the MTC consistency model and the netmeta estimator showed acceptable properties. We also saw a dependency on the amount of heterogeneity. Concerning the evaluated methods to evaluate the consistency assumption, none was shown to be suitable.

Conclusions

Based on our results we recommend a pragmatic approach for practical application in NMA. The estimator from the netmeta approach or the estimator from the Bayesian MTC consistency model should be preferred. Since none of the methods to evaluate the consistency assumption showed satisfactory results, users should have a strong focus on the similarity as well as the homogeneity assumption.

A comparison of Bayesian and frequentist methods in random-effects network meta-analysis of binary data

The performance of statistical methods is often evaluated by means of simulation studies. In case of network meta-analysis of binary data, however, simulations are not currently available for many practically relevant settings. We perform a simulation study for sparse networks of trials under between-trial heterogeneity and including multi-arm trials. Results of the evaluation of two popular frequentist methods and a Bayesian approach using two different prior specifications are presented. Methods are evaluated using coverage, width of intervals, bias, and root mean squared error (RMSE). In addition, deviations from the theoretical surface under the cumulative rankings (SUCRAs) or P-scores of the treatments are evaluated. Under low heterogeneity and when a large number of trials informs the contrasts, all methods perform well with respect to the evaluated performance measures. Coverage is observed to be generally higher for the Bayesian than the frequentist methods. The width of credible intervals is larger than those of confidence intervals and is increasing when using a flatter prior for between-trial heterogeneity. Bias was generally small, but increased with heterogeneity, especially in netmeta. In some scenarios, the direction of bias differed between frequentist and Bayesian methods. The RMSE was comparable between methods but larger in indirectly than in directly estimated treatment effects. The deviation of the SUCRAs or P-scores from their theoretical values was mostly comparable over the methods but differed depending on the heterogeneity and the geometry of the investigated network. Multivariate meta-regression or Bayesian estimation using a half-normal prior scaled to 0.5 seems to be promising with respect to the evaluated performance measures in network meta-analysis of sparse networks.

BUGSnet: an R package to facilitate the conduct and reporting of Bayesian network Meta-analyses

Background

Several reviews have noted shortcomings regarding the quality and reporting of network meta-analyses (NMAs). We suspect that this issue may be partially attributable to limitations in current NMA software which do not readily produce all of the output needed to satisfy current guidelines.

Results

To better facilitate the conduct and reporting of NMAs, we have created an R package called “BUGSnet” (Bayesian inference Using Gibbs Sampling to conduct a Network meta-analysis). This R package relies upon Just Another Gibbs Sampler (JAGS) to conduct Bayesian NMA using a generalized linear model. BUGSnet contains a suite of functions that can be used to describe the evidence network, estimate a model and assess the model fit and convergence, assess the presence of heterogeneity and inconsistency, and output the results in a variety of formats including league tables and surface under the cumulative rank curve (SUCRA) plots. We provide a demonstration of the functions contained within BUGSnet by recreating a Bayesian NMA found in the second technical support document composed by the National Institute for Health and Care Excellence Decision Support Unit (NICE-DSU). We have also mapped these functions to checklist items within current reporting and best practice guidelines.

Conclusion

BUGSnet is a new R package that can be used to conduct a Bayesian NMA and produce all of the necessary output needed to satisfy current scientific and regulatory standards. We hope that this software will help to improve the conduct and reporting of NMAs.

Incorporating adjustments for variability in control group response rates in network meta-analysis: a case study of biologics for rheumatoid arthritis

Background

The importance of adjusting for cross-study heterogeneity in control group response rates when conducting network meta-analyses (NMA) was demonstrated using a case study involving a comparison of biologics for the treatment of moderate-to-severe rheumatoid arthritis.

Methods

Bayesian NMAs were conducted for American College of Rheumatology (ACR) 50 treatment response based upon a set of randomized controlled trials (RCTs) identified by a recently completed systematic review of the literature. In addition to the performance of an unadjusted NMA, a model adjusting for cross-study heterogeneity of control group response rates using meta-regression was fit to the data. Model fit was evaluated, and findings from both analyses were compared with regard to clinical interpretations.

Results

ACR 50 response data from a total of 51 RCTs and 16,223 patients were analyzed. Inspection of cross-study variability in control group response rates identified considerable differences between studies. NMA incorporating adjustment for this variability was associated with an average change of 38.1% in the magnitude of the ORs between treatment comparisons, and over 64% of the odds ratio changed by 15% or more. Important changes in the clinical interpretations drawn from treatment comparisons were identified with this improved modeling approach.

Conclusions

In comparing biologics for moderate to severe rheumatoid arthritis, failure to adjust for cross-trial differences in the control arm response rates in NMA can lead to biased estimates of comparative efficacy between treatments.

An empirical comparison of Bayesian modelling strategies for missing binary outcome data in network meta-analysis

Background

A number of strategies have been proposed to handle missing binary outcome data (MOD) in systematic reviews. However, none of these have been evaluated empirically in a series of published systematic reviews.

Methods

Using published systematic reviews with network meta-analysis (NMA) from a wide range of health-related fields, we evaluated comparatively the most frequently described Bayesian modelling strategies for MOD in terms of log odds ratio (log OR), between-trial variance, inconsistency factor (i.e. difference between direct and indirect estimates for a comparison), surface under the cumulative ranking (SUCRA) and rankings. We extended the Bayesian random-effects NMA model to incorporate the informative missingness odds ratio (IMOR) parameter, and applied the node-splitting approach to investigate inconsistency locally. We considered both pattern-mixture and selection models, different structures for prior distribution of log IMOR, and different scenarios for MOD. To illustrate level of agreement between different strategies and scenarios, we used Bland-Altman plots.

Results

Addressing MOD using extreme scenarios and ignoring the uncertainty about the scenarios led to systematically different and more precise log ORs compared to modelling MOD under the missing at random (MAR) assumption. Hierarchical structure of log IMORs led to lower between-trial variance, especially in the case of substantial MOD. Assuming common-within-network or trial-specific log IMORs yielded similar posterior results for all NMA estimates, whereas intervention-specific structure systematically inflated uncertainty around log ORs and SUCRAs. Pattern-mixture model agreed with selection model, particularly under the trial-specific structure; however, selection model systematically reduced precision around log IMORs. Overall, different strategies and scenarios mostly had good agreement in the case of low MOD.

Conclusions

Addressing MOD using extreme scenarios and/or ignoring the uncertainty about the scenarios may negatively affect NMA estimates. Modelling MOD via the IMOR parameter can ensure bias-adjusted estimates and offer valuable insights into missingness mechanisms. The researcher should seek an expert opinion in order to decide on the structure of log IMOR that best aligns to the condition and interventions studied and to define a proper prior distribution for log IMOR. Our findings also apply to pairwise meta-analyses.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0731-y) contains supplementary material, which is available to authorized users.

The uptake of Bayesian methods in biomedical meta-analyses: A scoping review (2005-2016)

AIM

Bayesian statistical methods can allow for more complete and accurate incorporation of evidence in meta-analyses. However, these methods remain under-utilized.

METHODS

A scoping review was conducted to examine the proportion of biomedical meta-analyses that used Bayesian methods in the period 2005-2016. The review also examined the reproducibility of the work, the cited sources, the reasons for it, its success or failure, the type of model and prior distributions, and whether a mixture of Bayesian and frequentist methods were employed.

RESULTS

We found that 1% of meta-analyses are Bayesian and that the reporting and conduct of these were often poor. Data were published in 41% of analyses, and programs to run the analysis in 18%. Network meta-analysis was the most common reason and became increasingly popular in recent years. In the majority of papers, models and distributions were either not reported or explained in such brief and ambiguous terms as to be uninformative.

CONCLUSIONS

More use needs to be made of Bayesian meta-analysis, and reporting needs to be improved. Greater awareness of these methods and access to training in them is essential.

The quality of systematic reviews addressing peripheral nerve repair and reconstruction

While systematic reviews are regarded as the strongest level of medical evidence, inconsistency in the quality and rigor of systematic reviews raises concerns about their use as a tool in guiding quality delivery in evidence-based clinical practice. The objective of this present study was to assess methodological soundness of systematic reviews with a particular focus on peripheral nerve repair and reconstruction. We performed a comprehensive search using PubMed and Scopus to identify all systematic reviews published on peripheral nerve reconstruction in 9 high-impact surgical journals. Two authors independently performed literature searches, screened abstracts, and extracted data. Discrepancies were resolved by discussion and consensus. The quality of systematic reviews was assessed using AMSTAR criteria. Initial search retrieved 184 articles. After screening duplicates, titles, abstracts, and conducting full text reviews, 26 studies met inclusion criteria. Of those, 18 (65%) were published by Plastic Surgery, 7 (27%) by Orthopedic Surgery, and 1 (4%) by Occupational Therapy. The total number of systematic reviews published on peripheral nerves each year has shown an increasing trend from 2004 through 2015. The overall median AMSTAR score was 5, reflecting a "fair" quality. There was no evidence of AMSTAR score improvement over time. Although the number of systematic reviews published on peripheral nerve repair has risen over the last decade, their quality has not exhibited the same increase. This highlights the necessity to increase familiarity with and conform to methodological quality criteria in order to improve the integrity of evidence-based medicine in peripheral nerve repair and reconstruction.

Systematic review and network meta-analysis of clinical outcomes associated with isavuconazole versus relevant comparators for patients with invasive aspergillosis

OBJECTIVES

Voriconazole, amphotericin B (AmB) formulations, and isavuconazole are all included in guideline recommendations for treatment of patients with invasive aspergillosis (IA) but the relative efficacy of isavuconazole versus AmB formulations has not been directly compared. We aimed to estimate the relative efficacy of isavuconazole compared with AmB deoxycholate (AmB-D), liposomal AmB (L-AmB), and voriconazole for the treatment of patients with proven/probable IA.

METHODS

Nine literature databases were screened for randomized controlled trials comparing treatments with any of voriconazole, AmB-D, L-AmB and isavuconazole for treatment of proven/probable IA. Articles meeting the criteria were included in a meta-analysis to determine the efficacy of AmB-D, L-AmB and voriconazole relative to isavuconazole based on all-cause mortality (ACM) and overall response using a fixed-effects model.

RESULTS

Four articles were identified that compared L-AmB with AmB-D (Study 1), standard-dose L-AmB (3-5 mg/kg/day) with high-dose L-AmB (10 mg/kg/day; Study 2), voriconazole with AmB-D (Study 3), and isavuconazole with voriconazole (Study 4). In the network meta-analysis, isavuconazole was statistically superior to AmB-D on both ACM (odds ratio [95% credible intervals] shown as natural log, 1.00 [0.26, 1.74]) and overall response (-1.39 [-2.21, -0.63]). Differences between isavuconazole, and standard-dose L-AmB, high-dose L-AmB and voriconazole were not statistically significant for either ACM (0.18 [-1.17, 1.53], 0.50 [-1.11, 2.13] and 0.32 [-0.19, 0.84], respectively) or overall response (-0.99 [-2.21, 0.29], -0.89 [-2.41, 0.65] and 0.06 [-0.43, 0.57], respectively).

CONCLUSIONS

This data suggests that the efficacy of isavuconazole for treatment of IA is superior to AmB-D and comparable with both L-AmB and voriconazole.

Incorporating external evidence on between-trial heterogeneity in network meta-analysis

In a network meta‐analysis, between‐study heterogeneity variances are often very imprecisely estimated because data are sparse, so standard errors of treatment differences can be highly unstable. External evidence can provide informative prior distributions for heterogeneity and, hence, improve inferences. We explore approaches for specifying informative priors for multiple heterogeneity variances in a network meta‐analysis.

First, we assume equal heterogeneity variances across all pairwise intervention comparisons (approach 1); incorporating an informative prior for the common variance is then straightforward. Models allowing unequal heterogeneity variances are more realistic; however, care must be taken to ensure implied variance‐covariance matrices remain valid. We consider three strategies for specifying informative priors for multiple unequal heterogeneity variances. Initially, we choose different informative priors according to intervention comparison type and assume heterogeneity to be proportional across comparison types and equal within comparison type (approach 2). Next, we allow all heterogeneity variances in the network to differ, while specifying a common informative prior for each. We explore two different approaches to this: placing priors on variances and correlations separately (approach 3) or using an informative inverse Wishart distribution (approach 4).

Our methods are exemplified through application to two network metaanalyses. Appropriate informative priors are obtained from previously published evidence‐based distributions for heterogeneity.

Relevant prior information on between‐study heterogeneity can be incorporated into network meta‐analyses, without needing to assume equal heterogeneity across treatment comparisons. The approaches proposed will be beneficial in sparse data sets and provide more appropriate intervals for treatment differences than those based on imprecise heterogeneity estimates.

Assessing the methodological and reporting quality of network meta-analyses in Chinese medicine

BACKGROUND

An increasing number of network meta-analyses (NMAs) in traditional Chinese medicine (TCM) have been published recently, but the quality of them was lack of assessment. This study aims to evaluate the methodological and reporting quality of NMAs in TCM.

METHODS

Six electronic databases, including PubMed, the Cochrane Central Register of Controlled Trials (CENTRAL), Embase, China National Knowledge Infrastructure (CNKI), Wanfang and Chinese Biomedical Literature Database (CBM) from inception to January 2018, were searched. NMAs of TCM were included. A measurement tool to assess the methodological quality of systematic reviews (AMSTAR) and the PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-analyses of Health Care Interventions (PRISMA-NMA) were used to assess the methodological and reporting quality of the included NMAs.

RESULTS

A total of 40 NMAs, including 2535 randomized controlled trials (RCTs), were included. They were published between December 2012 and November 2017. The median score and interquartile range of methodological and reporting quality was 7 (6-8) and 22 (19.1-27.1). Serious methodological flaws existed in the following aspects: the status of publication (22.5%), a list of studies provided (0%), assessment of publication bias (37.5%), and conflicts of interest (12.5%). Several items need to be improved in reporting, especially for Protocol and registration (2.5%), Data items (22.5%), Risk of bias across studies (Methods section) (37.5%), Results of individual studies (27.5%), Risk of bias across studies (Results section) (40%), Results of additional analyses (35%), and Funding (15%).

CONCLUSIONS

The methodological and reporting quality of NMAs in TCM is moderate. Identified shortcomings of published NMAs should be taken into consideration in further trainings of authors and editors of NMAs in TCM. Future researchers should be encouraged to apply PRISMA-NMA, and a recognized tool for the assessment of NMA methodology was wanted.

Partially systematic thoughts on the history of systematic reviews

Six years after the launch of Systematic Reviews by Biomed Central, this article is part of the celebration of the journal. It contains personal reflections on the past, present and future of systematic reviews, using examples relevant to the role of systematic reviews in cataloguing and analysing research, assessing quality and planning new studies. The focus is on the most common of the various types of systematic review in health and social care, namely those assessing the effects of interventions.

History and publication trends in the diffusion and early uptake of indirect comparison meta-analytic methods to study drugs: animated coauthorship networks over time

OBJECTIVE

To characterise the early diffusion of indirect comparison meta-analytic methods to study drugs.

DESIGN

Systematic literature synthesis.

DATA SOURCES

Cochrane Database of Systematic Reviews, EMBASE, MEDLINE, Scopus and Web of Science.

STUDY SELECTION

English language papers that used indirect comparison meta-analytic methods to study the efficacy or safety of three or more interventions, where at least one was a drug.

DATA EXTRACTION

The number of publications and authors was plotted by year and type: methodological contribution, review or empirical application. Author and methodological details were summarised for empirical applications, and animated coauthorship networks were created to visualise contributors by country and affiliation type (academia, industry, government or other) over time.

RESULTS

We identified 477 papers (74 methodological contributions, 42 reviews and 361 empirical applications) by 1689 distinct authors from 1997 to 2013. Prior to 2002, only three applications were published, with contributions from the USA (n=2) and Canada (n=1). The number of applications gradually increased annually with rapid uptake between 2011 and 2013 (n=254, 71%). Early diffusion occurred primarily in Europe with the first application credited to the UK in 2003. Application spread to other European countries in 2005, and may have been supported by regulatory requirements for drug approval. By the end of 2013, contributions included 49% credited to Europe (22% UK, 27% other), 37% credited to North America (11% Canada, 26% USA) and 14% from other regions.

CONCLUSION

Indirect comparison meta-analytic methods are an important innovation for health research. Although Canada and the USA were the first to apply these methods, Europe led their diffusion. The increase in uptake of these methods may have been facilitated by acceptance by regulatory agencies, which are calling for more comparative drug effect data to assist in drug accessibility and reimbursement decisions.

The Quality of Systematic Reviews in Head and Neck Microsurgery: A Perspective from Plastic Surgery and Otolaryngology

INTRODUCTION

In recent years, there has been a push to publish higher level of evidence studies in medicine, particularly in plastic surgery. Well-conducted systematic reviews are considered the strongest level of evidence in medicine, recently becoming the key process indicators for quality delivery. A varying quality of systematic reviews, however, has led to concerns of their validity in clinical decision-making. We perform a quality analysis of systematic reviews published in head and neck microsurgery by the surgical specialties of plastic surgery and otolaryngology.

MATERIALS AND METHODS

An evaluation of systematic reviews published on microsurgery in 13 high-impact surgical journals was conducted by searching PubMed and Scopus. Two authors independently performed searches, screened for eligibility, and extracted data from included articles. Discrepancies were resolved by discussion and consensus. Assessment of Multiple Systematic Reviews (AMSTAR) criteria were used to assess methodological quality.

RESULTS

The initial database search retrieved 166 articles. After removing duplicates, screening titles and abstracts, 26 articles remained for full text review. Seven did not focus on head and neck microsurgery and were further excluded, leaving 19 systematic reviews for final analysis. Of those, 10 systematic reviews were published by otolaryngology, and 9 were published by plastic surgery. Median AMSTAR score was 8 for otolaryngology, 7 for plastic surgery, and 8 overall, reflecting "fair to good" quality. The number of systematic reviews on head and neck microsurgery markedly increased over time. Of note, both the AMSTAR score and the number of systematic reviews published by plastic surgery have steadily increased from 2014 to 2016, whereas those published by otolaryngology have remained relatively stable since 2010.

CONCLUSIONS

Our review shows a trend toward publishing more systematic reviews. The increasing quantity and quality of systematic reviews published by plastic surgeons indicates recognition in the need for higher levels of evidence in plastic surgery, as well as growing interest and advances in microsurgery. Given these trends, familiarity with quality assessment guidelines, such as AMSTAR, will remain important in providing a basis for building relevant value-based quality measures.

Preferred reporting items for overviews of systematic reviews including harms checklist: a pilot tool to be used for balanced reporting of benefits and harms

OBJECTIVES

An overview of systematic reviews (OoSRs) is a study designed to synthesize multiple evidence from existing systematic reviews on a specific domain. The aim of this paper was to offer a pilot version checklist with Preferred Reporting Items for OoSRs (PRIO-harms) to promote a more balanced reporting of benefits and harms in OoSRs of health care interventions.

STUDY DESIGN AND SETTING

The included items were developed by combining key features from health care OoSRs designs with recommendations from statements of other relevant checklists and pertinent methodological review articles. Two raters independently used the PRIO-harms checklist to assess a sample of 20 OoSRs.

RESULTS

The PRIO-harms tool consists of a 27-item (56 (sub-)items in total) checklist and is accompanied by a five-stage process flow diagram (identification, screening, eligibility, inclusion, and separation of relevant studies). The mean interrater reliability (Gwet's AC1 statistic) between reviewers was 0.90 (95% confidence interval: 0.88, 0.92) indicating a very good agreement.

CONCLUSION

The PRIO-harms tool can be used in every OoSRs that addresses health care interventions. This instrument will assist overview authors to improve completeness and transparency of research reporting with emphasis on harms. However, it might benefit from critical review and further validation from experts and research teams that produce OoSRs.

Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples

Organisations such as the National Institute for Health and Care Excellence require the synthesis of evidence from existing studies to inform their decisions—for example, about the best available treatments with respect to multiple efficacy and safety outcomes. However, relevant studies may not provide direct evidence about all the treatments or outcomes of interest. Multivariate and network meta-analysis methods provide a framework to address this, using correlated or indirect evidence from such studies alongside any direct evidence. In this article, the authors describe the key concepts and assumptions of these methods, outline how correlated and indirect evidence arises, and illustrate the contribution of such evidence in real clinical examples involving multiple outcomes and multiple treatments

Optimum level of inferior mesenteric artery ligation for the left-sided colorectal cancer. Systematic review for high and low ligation continuum

OBJECTIVES

To compares the effectiveness and impact of high inferior mesenteric artery (IMA) versus low IMA ligation on 5-year survival, lymph node yield rates, and peri-operative morbidity and mortality. 

METHODS

The databases of Educational Resources Information Centre (ERIC), the Web of Science, EBSCO and MEDLINE were searched using MeSH terms 'colorectal cancer', 'inferior mesenteric artery', 'high ligation', 'low ligation', 'mesenteric lymph nodes', 'prognosis', and 'survival'. Only clinical studies were selected and review articles and meta-analysis were excluded. In cases of duplicate cohorts, only the latest article was included. Irrelevant articles and the articles on both right and left sided CRC were excluded. The finally selected studies were analysed for the defined end-point outcomes. 

RESULTS

The published data has shown that high IMA ligation improves the yield of harvested lymph node that allows accurate tumor staging and a more reliable estimation of prognosis. High ligation was not found to be positively correlated with increased anastomotic leakage or impaired genito-urinary function. However, high ligation demands advanced surgical expertise and longer operating time. There was no significant difference in 5-year survival rates for both techniques. Some studies have reported fatal complications of high ligation such as proximal bowel necrosis.  

CONCLUSION

Although there is no consensus, this research signals the routine use of high ligation for left-sided CRC. However, the published fatal complications following high ligation and no significant difference in 5-year survival rates demand more studies to establishing a unified protocol.

Comparison of anti-malarial drugs efficacy in the treatment of uncomplicated malaria in African children and adults using network meta-analysis

Background

Artemisinin-based combination therapy (ACT) and novel drug combinations are available and used in African countries to treat uncomplicated malaria. Network meta-analysis methods are rarely and poorly applied for the comparison of their efficacies. This method was applied on a set of randomized controlled trials to illustrate its usefulness.

Methods

A literature review available in Pubmed was conducted in July 2016. Eligible studies, conducted in sub-Saharan Africa, published between 2002 and 2016, focused on randomized controlled trials of at least two artemisinin-based combinations to treat uncomplicated malaria in children and adults. Agglomerate data were: the number of PCR-corrected adequate clinical and parasitological response (ACPR) on day 28, used as the primary endpoint in all interventions, the number of participants and the list of treatments. A Bayesian random effect meta-analysis using a binary outcome was the method to compare the efficacy. Ranking measure was used to obtain a hierarchy of the competing interventions.

Results

In total, 76 articles were included; 13 treatment regimens were involved and tested in 36,001 patients. Using artemether–lumefantrine (AL) as the common comparator for the entire network, 12 relative treatment effects were estimated and indirect comparisons were obtained. Dihydroartemisinin–piperaquine (DHAP) was shown to be more effective than AL (odds ratio [OR] = 1.92; 95% CI 1.30–2.82; 19,163 patients), ASAQ (OR = 1.70; 95% CI 1.10–2.64; 14,433 patients), and amodiaquine–sulfadoxine–pyrimethamine (AQSP): OR = 2.20; 95% CI 1.21–3.96; 8863 patients. Artesunate–amodiaquine (ASAQ) was comparable to AL (OR = 1.11; 95% CI 0.84–1.45; 21,235 patients). No significant difference was found between artesunate and mefloquine (ASMQ) and AL (OR = 1.20; 95% CI = 0.52-2.8; 13,824 participants). According to treatment ranking, among the WHO-recommended ACT medicines, DHAP was shown to be the most efficacious.

Conclusions

Based on the available evidence, this study demonstrated the superiority of DHAP among currently recommended artemisinin-based combinations. The application of the methods described here may be helpful to gain better understanding of treatment efficacy and improve future decisions. However, more data are needed to allow robust conclusions about the results in comparison with novel drugs. Further surveillance of the efficacy of anti-malarial drugs and clinical trials are needed to closely follow the evolution of the epidemiology of drug-resistant malaria in Africa.

Electronic supplementary material

The online version of this article (doi:10.1186/s12936-017-1963-0) contains supplementary material, which is available to authorized users.

An investigation of the impact of using different methods for network meta-analysis: a protocol for an empirical evaluation

BACKGROUND

Network meta-analysis, a method to synthesise evidence from multiple treatments, has increased in popularity in the past decade. Two broad approaches are available to synthesise data across networks, namely, arm- and contrast-synthesis models, with a range of models that can be fitted within each. There has been recent debate about the validity of the arm-synthesis models, but to date, there has been limited empirical evaluation comparing results using the methods applied to a large number of networks. We aim to address this gap through the re-analysis of a large cohort of published networks of interventions using a range of network meta-analysis methods.

METHODS

We will include a subset of networks from a database of network meta-analyses of randomised trials that have been identified and curated from the published literature. The subset of networks will include those where the primary outcome is binary, the number of events and participants are reported for each direct comparison, and there is no evidence of inconsistency in the network. We will re-analyse the networks using three contrast-synthesis methods and two arm-synthesis methods. We will compare the estimated treatment effects, their standard errors, treatment hierarchy based on the surface under the cumulative ranking (SUCRA) curve, the SUCRA value, and the between-trial heterogeneity variance across the network meta-analysis methods. We will investigate whether differences in the results are affected by network characteristics and baseline risk.

DISCUSSION

The results of this study will inform whether, in practice, the choice of network meta-analysis method matters, and if it does, in what situations differences in the results between methods might arise. The results from this research might also inform future simulation studies.

Additional considerations are required when preparing a protocol for a systematic review with multiple interventions

OBJECTIVES

The number of systematic reviews that aim to compare multiple interventions using network meta-analysis is increasing. In this study, we highlight aspects of a standard systematic review protocol that may need modification when multiple interventions are to be compared.

STUDY DESIGN AND SETTING

We take the protocol format suggested by Cochrane for a standard systematic review as our reference and compare the considerations for a pairwise review with those required for a valid comparison of multiple interventions. We suggest new sections for protocols of systematic reviews including network meta-analyses with a focus on how to evaluate their assumptions. We provide example text from published protocols to exemplify the considerations.

CONCLUSION

Standard systematic review protocols for pairwise meta-analyses need extensions to accommodate the increased complexity of network meta-analysis. Our suggested modifications are widely applicable to both Cochrane and non-Cochrane systematic reviews involving network meta-analyses.

Characteristics and knowledge synthesis approach for 456 network meta-analyses: a scoping review

BACKGROUND

Network meta-analysis (NMA) has become a popular method to compare more than two treatments. This scoping review aimed to explore the characteristics and methodological quality of knowledge synthesis approaches underlying the NMA process. We also aimed to assess the statistical methods applied using the Analysis subdomain of the ISPOR checklist.

METHODS

Comprehensive literature searches were conducted in MEDLINE, PubMed, EMBASE, and Cochrane Database of Systematic Reviews from inception until April 14, 2015. References of relevant reviews were scanned. Eligible studies compared at least four different interventions from randomised controlled trials with an appropriate NMA approach. Two reviewers independently performed study selection and data abstraction of included articles. All discrepancies between reviewers were resolved by a third reviewer. Data analysis involved quantitative (frequencies) and qualitative (content analysis) methods. Quality was evaluated using the AMSTAR tool for the conduct of knowledge synthesis and the ISPOR tool for statistical analysis.

RESULTS

After screening 3538 citations and 877 full-text papers, 456 NMAs were included. These were published between 1997 and 2015, with 95% published after 2006. Most were conducted in Europe (51%) or North America (31%), and approximately one-third reported public sources of funding. Overall, 84% searched two or more electronic databases, 62% searched for grey literature, 58% performed duplicate study selection and data abstraction (independently), and 62% assessed risk of bias. Seventy-eight (17%) NMAs relied on previously conducted systematic reviews to obtain studies for inclusion in their NMA. Based on the AMSTAR tool, almost half of the NMAs incorporated quality appraisal results to formulate conclusions, 36% assessed publication bias, and 16% reported the source of funding. Based on the ISPOR tool, half of the NMAs did not report if an assessment for consistency was conducted or whether they accounted for inconsistency when present. Only 13% reported heterogeneity assumptions for the random-effects model.

CONCLUSIONS

The knowledge synthesis methods and analytical process for NMAs are poorly reported and need improvement.

Automated drawing of network plots in network meta-analysis

In systematic reviews based on network meta-analysis, the network structure should be visualized. Network plots often have been drawn by hand using generic graphical software. A typical way of drawing networks, also implemented in statistical software for network meta-analysis, is a circular representation, often with many crossing lines. We use methods from graph theory in order to generate network plots in an automated way. We give a number of requirements for graph drawing and present an algorithm that fits prespecified ideal distances between the nodes representing the treatments. The method was implemented in the function netgraph of the R package netmeta and applied to a number of networks from the literature. We show that graph representations with a small number of crossing lines are often preferable to circular representations.

IMPRoving Outcomes for children exposed to domestic ViolencE (IMPROVE): an evidence synthesis

Background

Exposure to domestic violence and abuse (DVA) during childhood and adolescence increases the risk of negative outcomes across the lifespan.

Objectives

To synthesise evidence on the clinical effectiveness, cost-effectiveness and acceptability of interventions for children exposed to DVA, with the aim of making recommendations for further research.

Design

(1) A systematic review of controlled trials of interventions; (2) a systematic review of qualitative studies of participant and professional experience of interventions; (3) a network meta-analysis (NMA) of controlled trials and cost-effectiveness analysis; (4) an overview of current UK provision of interventions; and (5) consultations with young people, parents, service providers and commissioners.

Settings

North America (11), the Netherlands (1) and Israel (1) for the systematic review of controlled trials of interventions; the USA (4) and the UK (1) for the systematic review of qualitative studies of participant and professional experience of interventions; and the UK for the overview of current UK provision of interventions and consultations with young people, parents, service providers and commissioners.

Participants

A total of 1345 children for the systematic review of controlled trials of interventions; 100 children, 202 parents and 39 professionals for the systematic review of qualitative studies of participant and professional experience of interventions; and 16 young people, six parents and 20 service providers and commissioners for the consultation with young people, parents, service providers and commissioners.

Interventions

Psychotherapeutic, advocacy, parenting skills and advocacy, psychoeducation, psychoeducation and advocacy, guided self-help.

Main outcome measures

Internalising symptoms and externalising behaviour, mood, depression symptoms and diagnosis, post-traumatic stress disorder symptoms and self-esteem for the systematic review of controlled trials of interventions and NMA; views about and experience of interventions for the systematic review of qualitative studies of participant and professional experience of interventions and consultations.

Data sources

MEDLINE, Cumulative Index to Nursing and Allied Health Literature, PsycINFO, EMBASE, Cochrane Central Register of Controlled Trials, Science Citation Index, Applied Social Sciences Index and Abstracts, International Bibliography of the Social Sciences, Social Services Abstracts, Social Care Online, Sociological Abstracts, Social Science Citation Index, World Health Organization trials portal and clinicaltrials.gov.

Review methods

A narrative review; a NMA and incremental cost-effectiveness analysis; and a qualitative synthesis.

Results

The evidence base on targeted interventions was small, with limited settings and types of interventions; children were mostly < 14 years of age, and there was an absence of comparative studies. The interventions evaluated in trials were mostly psychotherapeutic and psychoeducational interventions delivered to the non-abusive parent and child, usually based on the child’s exposure to DVA (not specific clinical or broader social needs). Qualitative studies largely focused on psychoeducational interventions, some of which included the abusive parent. The evidence for clinical effectiveness was as follows: 11 trials reported improvements in behavioural or mental health outcomes, with modest effect sizes but significant heterogeneity and high or unclear risk of bias. Psychoeducational group-based interventions delivered to the child were found to be more effective for improving mental health outcomes than other types of intervention. Interventions delivered to (non-abusive) parents and to children were most likely to be effective for improving behavioural outcomes. However, there is a large degree of uncertainty around comparisons, particularly with regard to mental health outcomes. In terms of evidence of cost-effectiveness, there were no economic studies of interventions. Cost-effectiveness was modelled on the basis of the NMA, estimating differences between types of interventions. The outcomes measured in trials were largely confined to children’s mental health and behavioural symptoms and disorders, although stakeholders’ concepts of success were broader, suggesting that a broader range of outcomes should be measured in trials. Group-based psychoeducational interventions delivered to children and non-abusive parents in parallel were largely acceptable to all stakeholders. There is limited evidence for the acceptability of other types of intervention. In terms of the UK evidence base and service delivery landscape, there were no UK-based trials, few qualitative studies and little widespread service evaluation. Most programmes are group-based psychoeducational interventions. However, the funding crisis in the DVA sector is significantly undermining programme delivery.

Conclusions

The evidence base regarding the acceptability, clinical effectiveness and cost-effectiveness of interventions to improve outcomes for children exposed to DVA is underdeveloped. There is an urgent need for more high-quality studies, particularly trials, that are designed to produce actionable, generalisable findings that can be implemented in real-world settings and that can inform decisions about which interventions to commission and scale. We suggest that there is a need to pause the development of new interventions and to focus on the systematic evaluation of existing programmes. With regard to the UK, we have identified three types of programme that could be justifiably prioritised for further study: psycho-education delivered to mothers and children, or children alone; parent skills training in combination with advocacy: and interventions involving the abusive parent/caregiver. We also suggest that there is need for key stakeholders to come together to explicitly identify and address the structural, practical and cultural barriers that may have hampered the development of the UK evidence base to date.

Future work recommendations

There is a need for well-designed, well-conducted and well-reported UK-based randomised controlled trials with cost-effectiveness analyses and nested qualitative studies. Development of consensus in the field about core outcome data sets is required. There is a need for further exploration of the acceptability and effectiveness of interventions for specific groups of children and young people (i.e. based on ethnicity, age, trauma exposure and clinical profile). There is also a need for an investigation of the context in which interventions are delivered, including organisational setting and the broader community context, and the evaluation of qualities, qualifications and disciplines of personnel delivering interventions. We recommend prioritisation of psychoeducational interventions and parent skills training delivered in combination with advocacy in the next phase of trials, and exploratory trials of interventions that engage both the abusive and the non-abusive parent.

Study registration

This study is registered as PROSPERO CRD42013004348 and PROSPERO CRD420130043489.

Funding

The National Institute for Health Research Public Health Research programme.

Epidemiology Characteristics, Methodological Assessment and Reporting of Statistical Analysis of Network Meta-Analyses in the Field of Cancer

Because of the methodological complexity of network meta-analyses (NMAs), NMAs may be more vulnerable to methodological risks than conventional pair-wise meta-analysis. Our study aims to investigate epidemiology characteristics, conduction of literature search, methodological quality and reporting of statistical analysis process in the field of cancer based on PRISMA extension statement and modified AMSTAR checklist. We identified and included 102 NMAs in the field of cancer. 61 NMAs were conducted using a Bayesian framework. Of them, more than half of NMAs did not report assessment of convergence (60.66%). Inconsistency was assessed in 27.87% of NMAs. Assessment of heterogeneity in traditional meta-analyses was more common (42.62%) than in NMAs (6.56%). Most of NMAs did not report assessment of similarity (86.89%) and did not used GRADE tool to assess quality of evidence (95.08%). 43 NMAs were adjusted indirect comparisons, the methods used were described in 53.49% NMAs. Only 4.65% NMAs described the details of handling of multi group trials and 6.98% described the methods of similarity assessment. The median total AMSTAR-score was 8.00 (IQR: 6.00-8.25). Methodological quality and reporting of statistical analysis did not substantially differ by selected general characteristics. Overall, the quality of NMAs in the field of cancer was generally acceptable.

Statistical Models and Methods for Network Meta-Analysis

Meta-analysis, the methodology for analyzing the results from multiple independent studies, has grown tremendously in popularity over the last four decades. Although most meta-analyses involve a single effect size (summary result, such as a treatment difference) from each study, there are often multiple treatments of interest across the network of studies in the analysis. Multi-treatment (or network) meta-analysis can be used for simultaneously analyzing the results from all the treatments. However, the methodology is considerably more complicated than for the analysis of a single effect size, and there have not been adequate explanations of the approach for agricultural investigations. We review the methods and models for conducting a network meta-analysis based on frequentist statistical principles, and demonstrate the procedures using a published multi-treatment plant pathology data set. A major advantage of network meta-analysis is that correlations of estimated treatment effects are automatically taken into account when an appropriate model is used. Moreover, treatment comparisons may be possible in a network meta-analysis that are not possible in a single study because all treatments of interest may not be included in any given study. We review several models that consider the study effect as either fixed or random, and show how to interpret model-fitting output. We further show how to model the effect of moderator variables (study-level characteristics) on treatment effects, and present one approach to test for the consistency of treatment effects across the network. Online supplemental files give explanations on fitting the network meta-analytical models using SAS.

A scoping review of indirect comparison methods and applications using individual patient data

BACKGROUND

Several indirect comparison methods, including network meta-analyses (NMAs), using individual patient data (IPD) have been developed to synthesize evidence from a network of trials. Although IPD indirect comparisons are published with increasing frequency in health care literature, there is no guidance on selecting the appropriate methodology and on reporting the methods and results.

METHODS

In this paper we examine the methods and reporting of indirect comparison methods using IPD. We searched MEDLINE, Embase, the Cochrane Library, and CINAHL from inception until October 2014. We included published and unpublished studies reporting a method, application, or review of indirect comparisons using IPD and at least three interventions.

RESULTS

We identified 37 papers, including a total of 33 empirical networks. Of these, only 9 (27 %) IPD-NMAs reported the existence of a study protocol, whereas 3 (9 %) studies mentioned that protocols existed without providing a reference. The 33 empirical networks included 24 (73 %) IPD-NMAs and 9 (27 %) matching adjusted indirect comparisons (MAICs). Of the 21 (64 %) networks with at least one closed loop, 19 (90 %) were IPD-NMAs, 13 (68 %) of which evaluated the prerequisite consistency assumption, and only 5 (38 %) of the 13 IPD-NMAs used statistical approaches. The median number of trials included per network was 10 (IQR 4-19) (IPD-NMA: 15 [IQR 8-20]; MAIC: 2 [IQR 3-5]), and the median number of IPD trials included in a network was 3 (IQR 1-9) (IPD-NMA: 6 [IQR 2-11]; MAIC: 2 [IQR 1-2]). Half of the networks (17; 52 %) applied Bayesian hierarchical models (14 one-stage, 1 two-stage, 1 used IPD as an informative prior, 1 unclear-stage), including either IPD alone or with aggregated data (AD). Models for dichotomous and continuous outcomes were available (IPD alone or combined with AD), as were models for time-to-event data (IPD combined with AD).

CONCLUSIONS

One in three indirect comparison methods modeling IPD adjusted results from different trials to estimate effects as if they had come from the same, randomized, population. Key methodological and reporting elements (e.g., evaluation of consistency, existence of study protocol) were often missing from an indirect comparison paper.

Use of network meta-analysis in systematic reviews: a survey of authors

BACKGROUND

The reporting of network meta-analysis in systematic reviews has increased rapidly since 2009. This qualitative study was undertaken to identify authors' perceptions of the use of these methods and of what standards for conduct and reporting should apply.

METHODS

This is a survey of authors of systematic reviews reporting network meta-analysis.

RESULTS

The response rate was 32 % of the authors contacted, with these authors responsible for 34 % of the fully published systematic reviews identified within the period searched. Almost all authors would use the method again. Elements of reporting standards were proposed. Responses revealed some tensions between the view that use of network meta-analysis should be more easily accessible, particularly in the form of software tools, and concern that there is some inappropriate use of the methods, which wider use and greater accessibility could exacerbate.

CONCLUSIONS

Authors demonstrated strong support for adoption of standards for conduct and reporting. The elements of reporting standards proposed are consistent with those included in the 2015 Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension statement. Adoption of standards for conduct and reporting will be a significant step towards clarifying what is appropriate use of the methods and what is not. This should be followed by the development of a critical appraisal tool to support end users of systematic reviews reporting network meta-analysis.

GetReal in network meta-analysis: a review of the methodology

Pairwise meta-analysis is an established statistical tool for synthesizing evidence from multiple trials, but it is informative only about the relative efficacy of two specific interventions. The usefulness of pairwise meta-analysis is thus limited in real-life medical practice, where many competing interventions may be available for a certain condition and studies informing some of the pairwise comparisons may be lacking. This commonly encountered scenario has led to the development of network meta-analysis (NMA). In the last decade, several applications, methodological developments, and empirical studies in NMA have been published, and the area is thriving as its relevance to public health is increasingly recognized. This article presents a review of the relevant literature on NMA methodology aiming to pinpoint the developments that have appeared in the field. Copyright © 2016 John Wiley & Sons, Ltd.

A 'reluctant' critical review: 'Manual for evidence-based clinical practice (2015)'

BACKGROUND

The Users' Guides to the Medical Literature Manual has been a major influence on the teaching and practice of health care globally.

METHODS

The 3rd edition of the multi-authored Manual was reviewed using the principles outlined in Evidence-based Medicine (EBM) texts. One 'clinical scenario' was selected for critical appraisal, as were several chapters; objectivity was enhanced by citing references to support opinions. RESULTS (SUMMARY OF THE APPRAISAL): (1) Strengths: Clinical pearls, too numerous to list.

EXAMPLES

(i) evidence is never enough to drive clinical decision making; (ii) do not rush to adopt new interventions; and (iii) question efficacy data based only on surrogate markers. (2) Weaknesses: The Manual shares shortcomings of textbooks discussed by Straus et al.: (i) references may not be current, important ones may be excluded and citations may be selective; (ii) often, opinion-based; and (iii) delays between revisions. (3) Notable omissions: Little or no discussion of: (i) important segments of the population: those <18 years of age, >65 years of age and those with multimorbidity; (ii) surgical disciplines; (iii) Greenhalgh et al.'s essay on EBM; (iv) alternate views on the hierarchy of evidence; and (vi) critical thinking. (4) Additional issues: (i) Omission of important references on dabigatran (clinical scenario: chapter 13.1); (ii) authors' advice (Chapter 13.3) to 'bypass the discussion section of published research'; and (iii) the advocacy of pre-appraised sources of evidence and network meta-analysis without warnings about limitations, are critiqued.

CONCLUSION

The Manual has several clinical pearls but readers should also be aware of shortcomings.