Meta-analyses with industry involvement are massively published and report no caveats for antidepressants

Incomplete reporting of adverse events in duloxetine trials: a meta-research survey of randomized controlled trials vs placebo

BACKGROUND AND OBJECTIVES

Relying on published data alone might be insufficient for meta-analyses to be reliable and trustworthy since selective outcome reporting is common, especially for adverse events (AEs). We investigated the existence of selective reporting and its potential for bias in a case study exploring AEs of duloxetine in adults.

STUDY DESIGN AND SETTING

We systematically searched all previous meta-analyses/pooled analyses on duloxetine published on PubMed for seven indications approved by the American and European health authorities. We included all randomized controlled trials (RCTs) vs placebo. For each RCT, we extracted the number of serious adverse events (SAEs), AEs, drop-outs (DOs) and drop-outs for safety reasons (DOSRs) using four information sources: published articles, clinical study registries, clinical study reports and data available in meta-analyses/pooled analyses. To assess the range of differences resulting from these four extraction strategies, we performed 4 meta-analyses using random effect models as well as a complete meta-analysis combining all sources.

RESULTS

A total of 70 RCTs (including 24,330 patients) were included. Of those, SAEs were identified for 42 studies (61%) in published articles, 58 (84%) in study reports (8 study reports were not retrieved), 24 (34.7%) in registries, and 21 (30.4%) in meta-analyses/pooled analyses. For 2 (2.9%), 2 (2.9%), 2 (2.9%) and 1 (1.4%) studies, we found respectively no data on SAEs, AEs, DOs, and DOSRs in any sources. Discrepant results across sources were found in 24 (34.5%), 20 (28.5%), 13 (18.6%), and 9 (12.8%) studies, respectively for SAEs, AEs, DOs, and DOSRs. Despite variations in point estimates and their 95% confidence intervals, we did not find different results in the conclusions of meta-analyses depending on the different information sources used, except for DOs, for which no effect was found using results published in registries, in contrast to other information sources.

CONCLUSION

None of the four information sources provided complete retrieval of safety results for duloxetine in adults across various indications. However, we did not find strong evidence that this underreporting leads to different conclusions in meta-analyses. Nonetheless, this finding remains uncertain, as we were unable to obtain complete information for all studies despite extensive searches.

Investigation of bias due to selective inclusion of study effect estimates in meta-analyses of nutrition research

We aimed to explore, in a sample of systematic reviews (SRs) with meta-analyses of the association between food/diet and health-related outcomes, whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available. We randomly selected SRs of food/diet and health-related outcomes published between January 2018 and June 2019. We selected the first presented meta-analysis in each review (index meta-analysis), and extracted from study reports all study effect estimates that were eligible for inclusion in the meta-analysis. We calculated the Potential Bias Index (PBI) to quantify and test for evidence of selective inclusion. The PBI ranges from 0 to 1; values above or below 0.5 suggest selective inclusion of effect estimates more or less favourable to the intervention, respectively. We also compared the index meta-analytic estimate to the median of a randomly constructed distribution of meta-analytic estimates (i.e., the estimate expected when there is no selective inclusion). Thirty-nine SRs with 312 studies were included. The estimated PBI was 0.49 (95% CI 0.42-0.55), suggesting that the selection of study effect estimates from those reported was consistent with a process of random selection. In addition, the index meta-analytic effect estimates were similar, on average, to what we would expect to see in meta-analyses generated when there was no selective inclusion. Despite this, we recommend that systematic reviewers report the methods used to select effect estimates to include in meta-analyses, which can help readers understand the risk of selective inclusion bias in the SRs.

Pharmaceutical Industry Payments to Medical Oncologists in the Netherlands: Trends and Patterns Provided by an Open-Access Transparency Data Set

PURPOSE

Health care expenditure related to oncologic treatments is skyrocketing although many treatments offer marginal, if any, clinical benefit. Financial conflicts of interest (fCOI) resulting from pharmaceutical industry (pharma) payments to physicians is increasingly recognized as a predictive factor for regulatory board approval and guideline incorporation of low-value treatments. We sought to study the extent to which pharma payments to medical oncologists occur in the Netherlands, the amount of money involved, and whether these occur more frequently and are higher for key opinion leaders (KOLs).

METHODS

In our cross-sectional retrospective database study, we used several Dutch open-access databases and extracted data registered between 2019 and 2021.

RESULTS

A cumulative amount of €899,863 was paid to 48.8% of the 408 registered medical oncologists. Over time, there was a marked decline in both the proportion of medical oncologists receiving payments (from 40.4% in 2019 to 19.1% in 2021) and the mean annual value of payments (from €2,962 in 2019 to €2,188 in 2021) with the latter mainly resulting from a decline in hospitality-related transactions. KOLs were more likely to receive industry payments and received a higher median payment value.

DISCUSSION

Our findings should contribute to the increasing awareness in the Netherlands of the potential effects of fCOI.

Transparency in Cognitive Training Meta-analyses: A Meta-review

Meta-analyses often present flexibility regarding their inclusion criteria, outcomes of interest, statistical analyses, and assessments of the primary studies. For this reason, it is necessary to transparently report all the information that could impact the results. In this meta-review, we aimed to assess the transparency of meta-analyses that examined the benefits of cognitive training, given the ongoing controversy that exists in this field. Ninety-seven meta-analytic reviews were included, which examined a wide range of populations with different clinical conditions and ages. Regarding the reporting, information about the search of the studies, screening procedure, or data collection was detailed by most reviews. However, authors usually failed to report other aspects such as the specific meta-analytic parameters, the formula used to compute the effect sizes, or the data from primary studies that were used to compute the effect sizes. Although some of these practices have improved over the years, others remained the same. Moreover, examining the eligibility criteria of the reviews revealed a great heterogeneity in aspects such as the training duration, age cut-offs, or study designs that were considered. Preregistered meta-analyses often specified poorly how they would deal with the multiplicity of data or assess publication bias in their protocols, and some contained non-disclosed deviations in their eligibility criteria or outcomes of interests. The findings shown here, although they do not question the benefits of cognitive training, illustrate important aspects that future reviews must consider.

Undisclosed financial conflicts of interest in DSM-5-TR: cross sectional analysis

OBJECTIVE

To assess the extent and types of financial ties to industry of panel and task force members of the American Psychiatric Association's Diagnostic and Statistical Manual of Mental Disorders, fifth edition, text revision (DSM-5-TR), published in 2022.

DESIGN

Cross sectional analysis.

SETTING

Open Payments database, USA.

PARTICIPANTS

92 physicians based in the US who served as members of either a panel (n=86) or task force (n=6) on the DSM-5-TR with information recorded in the Centers for Medicare and Medicaid Services Open Payments database during 2016-19. This period was chosen to include the year that development of the DSM-5-TR began and the three years preceding, a time consistent with previous research on conflicts of interest and consistent with the American Psychiatric Association's disclosure requirements for the fifth revision (DSM-5) of the manual.

MAIN OUTCOME MEASURES

Type and amount of compensation the panel and task force members of DSM-5-TR received during 2016-19.

RESULTS

After duplicate names had been removed, 168 individuals were identified who served as either panel or task force members of the DSM-5-TR. 92 met the inclusion criteria of being a physician who was based in the US and therefore could be included in Open Payments. Of these 92 individuals, 55 (60%) received payments from industry. Collectively, these panel members received a total of $14.2m (£11.2m; €13m). One third (33.3%) of the task force members had payments reported in Open Payments.

CONCLUSIONS

Conflicts of interest among panel members of DSM-5-TR were prevalent. Because of the enormous influence of diagnostic and treatment guidelines, the standards for participation on a guideline development panel should be high. A rebuttable presumption should exist for the Diagnostic and Statistical Manual of Mental Disorders to prohibit conflicts of interest among its panel and task force members. When no independent individuals with the requisite expertise are available, individuals with associations to industry could consult to the panels, but they should not have decision making authority on revisions or the inclusion of new disorders.

Does type of funding affect reporting in network meta-analysis? A scoping review of network meta-analyses

BACKGROUND

Evidence has shown that private industry-sponsored randomized controlled trials (RCTs) and meta-analyses are more likely to report intervention-favourable results compared with other sources of funding. However, this has not been assessed in network meta-analyses (NMAs).

OBJECTIVES

To (a) explore the recommendation rate of industry-sponsored NMAs on their company's intervention, and (b) assess reporting in NMAs of pharmacologic interventions according to their funding type.

METHODS

Design: Scoping review of published NMAs with RCTs.

INFORMATION SOURCES

We used a pre-existing NMA database including 1,144 articles from MEDLINE, EMBASE and Cochrane Database of Systematic Reviews, published between January 2013 and July 2018.

STUDY SELECTION

NMAs with transparent funding information and comparing pharmacologic interventions with/without placebo.

SYNTHESIS

We captured whether NMAs recommended their own or another company's intervention, classified NMAs according to their primary outcome findings (i.e., statistical significance and direction of effect), and according to the overall reported conclusion. We assessed reporting using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension to NMA (PRISMA-NMA) 32-item checklist. We matched and compared industry with non-industry NMAs having the same research question, disease, primary outcome, and pharmacologic intervention against placebo/control.

RESULTS

We retrieved 658 NMAs, which reported a median of 23 items in the PRISMA-NMA checklist (interquartile range [IQR]: 21-26). NMAs were categorized as 314 publicly-sponsored (PRISMA-NMA median 24.5, IQR 22-27), 208 non-sponsored (PRISMA-NMA median 23, IQR 20-25), and 136 industry/mixed-sponsored NMAs (PRISMA-NMA median 21, IQR 19-24). Most industry-sponsored NMAs recommended their own manufactured drug (92%), suggested a statistically significant positive treatment-effect for their drug (82%), and reported an overall positive conclusion (92%). Our matched NMAs (25 industry vs 25 non-industry) indicated that industry-sponsored NMAs had favourable conclusions more often (100% vs 80%) and were associated with larger (but not statistically significantly different) efficacy effect sizes (in 61% of NMAs) compared with non-industry-sponsored NMAs.

CONCLUSIONS

Differences in completeness of reporting and author characteristics were apparent among NMAs with different types of funding. Publicly-sponsored NMAs had the best reporting and published their findings in higher impact-factor journals. Knowledge users should be mindful of this potential funding bias in NMAs.

Why psychiatry needs an honest dose of gentle medicine

The pharmaceutical industry's influence on psychiatric research and practice has been profound and has resulted in exaggerated claims of the effectiveness of psychotropic medications and an under-reporting of harms. After the regulatory approval of fluoxetine, the pharmaceutical industry began promoting (and continues to promote) a chemical imbalance theory of emotional distress. In the last decade, there has been an increased awareness about the limits of this theory and the risks of psychotropic medications. Nonetheless, the medicalization of distress, the sedimented belief in "magic bullets," and the push to "scale up" mental health treatment have contributed to the meteoric rise in the prescription of psychiatric drugs and of polypharmacy. A major premise of this paper is that the conceptual framework of medical nihilism can help researchers and clinicians understand and address the harms incurred by inflated claims of the efficacy of psychotropic medications. We propose that psychiatry, and the mental health field more generally, adopt a model of 'gentle medicine' with regard to both the diagnosis of and treatment for mental health conditions and focus greater attention on the upstream causes of distress.

Evaluating the relationship between industry sponsorship and conflicts of interest among systematic review authors on treatments for cannabis use disorder

Background: Cannabis is the most commonly used illicit drug worldwide. In addition to potential adverse effects, an estimated 9% consistent cannabis users are likely to become dependent and may develop a cannabis use disorder (CUD). Methods: This cross-sectional study developed a search strategy using Ovid, MEDLINE, and Ovid Embase for systematic reviews and meta-analyses focused on CUD treatment in June 2020. These reviews were evaluated for conflicts of interest (COIs) per previously developed classification scheme. Our primary objectives were to (1) evaluate the presence of disclosed or undisclosed COI of systematic review authors, regarding treatment of CUD; and (2) determine whether overall summary effect estimates, narrative results and conclusions were influenced by the presence of disclosed or undisclosed COIs among systematic review authors. Results: Our systematic search returned 560 articles which 9 systematic reviews were eligible for data extraction. We found 77.8% (7/9) contained at least one author with a COI. From the 51 authors included, 29.4% (15/51) were found to have a COI. Forty-four percent (4/9) were funded, 22.2% (2/9) were not funded, and 33.3% (3/9) had no funding statements. Out of the 7 systematic reviews with one or more authors containing COI, 14.2% (1/7) included results favoring the treatment group and 28.6% (2/7) included conclusions favoring the treatment group. Our results showed no significance between funding source and results (p = 0.429) or conclusions. Additionally, we found no significance between the presence of COIs with the favorability of results (p = 0.56) or conclusions. Conclusion: Multiple studies favored the treatment of cannabis-containing products, even though COIs were found in the majority of the systematic reviews. COIs have the ability to sway results of a study, which can affect clinical decision-making. Stricter guidelines should be enforced among authors displaying COIs in systematic reviews studying CUD treatment.

The need for "gentle medicine" in a post Covid-19 world

As it has historically been the case with many pandemics, the Covid-19 experience will induce many philosophers to reconsider the value of medical practice. This should be a good opportunity to critically scrutinize the way medical research and medical interventions are carried out. For much of its history, medicine has been very inefficient. But, even in its contemporary forms, a review of common protocols in medical research and medical interventions reveal many shortcomings, especially related to methodological flaws, and more importantly, conflicts of interests due to profit incentives. In the face of these problems, we propose a program of "gentle medicine". This term, originally formulated by philosopher Jacob Stegenga, describes a form of medicine in which physicians intervene less than they currently do. As part of this general program, we advance a series of reform recommendations that could be enacted both by medical staff in their everyday practice, but also by public health officials and policymakers.

Improving the transparency of meta-analyses with interactive web applications

Increased transparency in study design and analysis is one proposed solution to the perceived reproducibility crisis facing science. Systematic review and meta-analysis-through which individual studies on a specific association are ascertained, assessed for quality and quantitatively combined-is a critical process for building consensus in medical research. However, the conventional publication model creates static evidence summaries that force the quality assessment criteria and analytical choices of a small number of authors onto all stakeholders, some of whom will have different views on the quality assessment and key features of the analysis. This leads to discordant inferences from meta-analysis results and delayed arrival at consensus. We propose a shift to interactive meta-analysis, through which stakeholders can take control of the evidence synthesis using their own quality criteria and preferred analytic approach-including the option to incorporate prior information on the association in question-to reveal how their summary estimate differs from that reported by the original analysts. We demonstrate this concept using a web-based meta-analysis of the association between genetic variation in a key tamoxifen-metabolising enzyme and breast cancer recurrence in tamoxifen-treated women. We argue that interactive meta-analyses would speed consensus-building to the degree that they reveal invariance of inferences to different study selection and analysis criteria. On the other hand, when inferences are found to differ substantially as a function of these choices, the disparities highlight where future research resources should be invested to resolve lingering sources of disagreement.

High variability in results and methodological quality among overlapping systematic reviews on the same topics in surgery: a meta-epidemiological study

BACKGROUND

Redundant publication of systematic reviews and meta-analyses (SRs/MAs) on the same topic presents an increasing burden for clinicians. The aim of this study was to describe variabilities in effect size and methodological quality of overlapping surgery-related SRs/MAs and to investigate factors associated with their postpublication citations.

METHODS

PubMed/MEDLINE was searched to identify SRs/MAs of RCTs on thoracoabdominal surgeries published in 2015. Previous SRs/MAs on the same topics published within the preceding 5 years (2011-2015) were identified and 5-year citation counts (through to 2020) were evaluated. Discrepancies in pooled effect sizes and their methodological quality using A Measurement Tool to Assess Systematic Reviews (AMSTAR) among overlapping SRs/MAs were assessed. The SR/MA-level factors associated with 5-year citation counts were explored, using a mixed-effects regression model with a random intercept for surgical topics.

RESULTS

A total of 57 surgery-related SRs/MAs (48 topics) published in 2015 were identified, and 146 SRs/MAs had overlapping publications on 29 topics (60.4 per cent of all topics) in the preceding 5 years. There was considerable variability in methodological quality of SRs/MAs and coverage probability for relevant RCTs, resulting in discrepant effect size estimates for the same topic. High quality (AMSTAR score 8-11) was independently associated with higher 5-year citation counts (coefficient = 32.82; 95 per cent c.i. 15.63 to 50.02; P < 0.001).

CONCLUSION

Overlapping SRs/MAs with high variability in results and methodological quality were common in surgery. A high-quality SR/MA score was an independent predictor of more frequent citations. Researchers and journal editors should concentrate their efforts on limiting publications to higher-quality reviews.

The methodological quality was low and conclusions discordant for meta-analyses comparing proximal humerus fracture treatments: a meta-epidemiological study

OBJECTIVE

To investigate the association between methodological quality and reported conclusions of meta-analyses comparing operative with non-operative treatments for proximal humerus fractures.

STUDY DESIGN AND SETTING

Cross-sectional meta-epidemiological study. We searched EMBASE, PubMed, The Cochrane Library, and Web of Science for systematic reviews with meta-analyses comparing non-operative with operative treatments for proximal humerus fractures. Methodological quality was assessed using AMSTAR2 and the reported conclusions were scored for three outcome domains (functional outcome, quality of life, and harm) on a scale from 1 to 6. The Mann-Whitney and Kruskal-Wallis tests were used to investigate the association between methodological quality and reported conclusions.

RESULTS

We included 21 systematic reviews: 19 pairwise meta-analyses and 2 network meta-analyses, although there are only 8 published randomized controlled trials. Most (n = 18) of the meta-analyses were rated as critically low quality, while the remaining 1 was rated as high quality. The conclusions were discordant for all three outcome domains, even for meta-analyses reporting similar inclusion criteria. We could not perform most of the statistical tests due to the predominantly critically low quality.

CONCLUSION

The methodological quality was so predominantly critically low that it was not possible to evaluate the association between methodological quality and reported conclusions.

Suicide risk with selective serotonin reuptake inhibitors and other new-generation antidepressants in adults: a systematic review and meta-analysis of observational studies

BACKGROUND

There is ongoing controversy whether antidepressant use alters suicide risk in adults with depression and other treatment indications.

METHODS

Systematic review of observational studies, searching MEDLINE, PsycINFO, Web of Science, PsycARTICLES and SCOPUS for case-control and cohort studies. We included studies on depression and various indications unspecified (including off-label use) reporting risk of suicide and/or suicide attempt for adult patients using selective serotonin reuptake inhibitors (SSRI) and other new-generation antidepressants relative to non-users. Effects were meta-analytically aggregated with random-effects models, reporting relative risk (RR) estimates with 95% CIs. Publication bias was assessed via funnel-plot asymmetry and trim-and-fill method. Financial conflict of interest (fCOI) was defined present when lead authors' professorship was industry-sponsored, they received industry-payments, or when the study was industry-sponsored.

RESULTS

We included 27 studies, 19 on depression and 8 on various indications unspecified (n=1.45 million subjects). SSRI were not definitely related to suicide risk (suicide and suicide attempt combined) in depression (RR=1.03, 0.70-1.51) and all indications (RR=1.19, 0.88-1.60). Any new-generation antidepressant was associated with higher suicide risk in depression (RR=1.29, 1.06-1.57) and all indications (RR=1.45, 1.23-1.70). Studies with fCOI reported significantly lower risk estimates than studies without fCOI. Funnel-plots were asymmetrical and imputation of missing studies with trim-and-fill method produced considerably higher risk estimates.

CONCLUSIONS

Exposure to new-generation antidepressants is associated with higher suicide risk in adult routine-care patients with depression and other treatment indications. Publication bias and fCOI likely contribute to systematic underestimation of risk in the published literature.

REGISTRATION

Open Science Framework, https://osf.io/eaqwn/.

Should antidepressants be used for major depressive disorder?

BACKGROUND

Major depressive disorder is estimated by the WHO to affect more than 300 million people globally, making depression the leading cause of disability worldwide. Antidepressants are commonly used to treat depression.

OBJECTIVE

The study aimed to provide an update on the evidence on the effects of antidepressants compared with placebo. Should antidepressants be used for adults with major depressive disorder?

STUDY SELECTION

We searched the Cochrane Library, BMJ Best Practice and PubMed up to June 2019 with the search terms 'depression' and 'antidepressants' targeting reviews published in English since 1990.

FINDINGS

Several reviews have assessed the effects of antidepressants compared with placebo for depression. Generally, all the previous reviews show that antidepressants seem to have statistically significant effects on depressive symptoms, but the size of the effect has questionable importance to most patients. Antidepressants seem to have minimal beneficial effects on depressive symptoms and increase the risk of both serious and non-serious adverse events.

CONCLUSIONS

The benefits of antidepressants seem to be minimal and possibly without any importance to the average patient with major depressive disorder. Antidepressants should not be used for adults with major depressive disorder before valid evidence has shown that the potential beneficial effects outweigh the harmful effects.

Redundant meta-analyses are common in genetic epidemiology

OBJECTIVES

The massive growth in the publication of meta-analyses may cause redundancy and wasted efforts. We performed a metaepidemiologic study to evaluate the extent of potential redundancy in published meta-analyses in genetic epidemiology.

STUDY DESIGN AND SETTING

Using a sample of 38 index meta-analyses of genetic associations published in 2010, we retrieved additional meta-analyses that evaluated identical associations (same genetic variant and phenotype) using the Human Genome Epidemiology (HuGE) Navigator and PubMed databases. We analyzed the frequency of potential duplication and examined whether subsequent meta-analyses cited previous meta-analyses on the exact same association.

RESULTS

Based on 38 index meta-analyses, we retrieved a total of 99 duplicate meta-analyses. Only 12 (32%) of the index meta-analyses were unambiguously unique. We found a mean of 2.6 duplicates and a median of 2 duplicates per meta-analysis. In case studies, only 29-54% of previously published meta-analyses were cited by subsequent ones.

CONCLUSION

These results suggest that duplication is common in meta-analyses of genetic associations.

Transparency and Reproducibility of Meta-Analyses in Psychology: A Meta-Review

Systematic review and meta-analysis are possible as viable research techniques only through transparent reporting of primary research; thus, one might expect meta-analysts to demonstrate best practice in their reporting of results and have a high degree of transparency leading to reproducibility of their work. This assumption has yet to be fully tested in the psychological sciences. We therefore aimed to assess the transparency and reproducibility of psychological meta-analyses. We conducted a meta-review by sampling 150 studies from Psychological Bulletin to extract information about each review's transparent and reproducible reporting practices. The results revealed that authors reported on average 55% of criteria and that transparent reporting practices increased over the three decades studied (b = 1.09, SE = 0.24, t = 4.519, p < .001). Review authors consistently reported eligibility criteria, effect-size information, and synthesis techniques. Review authors, however, on average, did not report specific search results, screening and extraction procedures, and most importantly, effect-size and moderator information from each individual study. Far fewer studies provided statistical code required for complete analytical replication. We argue that the field of psychology and research synthesis in general should require review authors to report these elements in a transparent and reproducible manner.

Reporting of financial conflicts of interest by Canadian clinical practice guideline producers: a descriptive study

BACKGROUND

The producers of clinical practice guidelines (CPGs) may not disclose industry funding in their CPGs. We reviewed Canadian national CPGs to examine the existence and disclosure of industry-related organizational funding in the CPGs, financial conflicts of interest of committee members and organizational procedures for managing financial conflicts of interest.

METHODS

For this descriptive study, we searched the asset map of the Strategy for Patient-Oriented Research Evidence Alliance and the CPG Infobase for CPGs published between Jan. 1, 2016, and Nov. 30, 2018. Eligible guidelines had to have a national focus and either a first-line drug recommendation or a screening recommendation leading to drug treatment. One investigator reviewed all CPG titles to exclude those that were clearly ineligible. Two reviewers independently reviewed all remaining guidelines and extracted data. We analyzed the data descriptively.

RESULTS

We included 21 CPGs: 3 from government-sponsored organizations, 9 from disease or condition interest groups and 9 from medical professional societies. None of the 3 government-sponsored organizations reported industry funding, and none of their committee members disclosed financial conflicts of interest. Among the 18 disease or condition interest groups and medical professional societies, 14 (93%) of the 15 that disclosed funding sources on websites (3 did not disclose) reported organizational funding from industry, but none disclosed this information in the CPGs; 12 (86%) of the 14 with conflict-of-interest disclosure statements in the CPG (4 did not include disclosures) had at least 1 committee member with a financial conflict (mean proportion of committee members with a conflict 56%); and for all 8 CPGs with identifiable chairs or cochairs (chairs or cochairs not reported for 10) at least 1 of these people had a financial conflict of interest. None of the guidelines described a plan to manage organizational financial conflicts of interest.

INTERPRETATION

Canadian CPGs are vulnerable to industry influence through funding of producers of guidelines and through the financial conflicts of interest of committee members. The CPG producers that receive industry funding should disclose organizational financial conflicts in the CPGs, should engage independent oversight committees and should restrict voting on recommendations to guideline panelists who have no financial conflicts.

How effective are antidepressants for depression over the long term? A critical review of relapse prevention trials and the issue of withdrawal confounding

The aim of this article is to discuss the validity of relapse prevention trials and the issue of withdrawal confounding in these trials. Recommendations for long-term antidepressant treatment are based almost exclusively on discontinuation trials. In these relapse prevention trials, participants with remitted depression are randomised either to have the antidepressant abruptly discontinued and replaced by inert placebo or to continue active treatment. The drug-placebo difference in relapse rates at the end of the maintenance phase is then interpreted as a prophylactic drug effect. These trials consistently produce remarkable benefits for maintenance treatment. However, the internal validity of this trial protocol is compromised, as research has shown that abruptly stopping antidepressants can cause severe withdrawal reactions that lead to (or manifest as) depression relapses. That is, there is substantial withdrawal confounding in discontinuation trials, which renders their findings uninterpretable. It is not clear to what degree the drug-placebo separation in relapse prevention (discontinuation) trials is due to withdrawal reactions, but various estimations suggest that it is presumably the majority. A review of findings based on other methodologies, including real-world long-term effectiveness trials like STAR*D and various naturalistic cohort studies, do not indicate that antidepressants have considerable prophylactic effects. As absence of evidence does not imply evidence of absence, no definitive conclusions can be drawn from the literature. To enable a thorough risk-benefit evaluation, real-world effectiveness trials should not only focus on relapse prevention, but also assess antidepressants' long-term effects on social functioning and quality of life. Thus far, reliable long-term data on these outcome domains are lacking.

Development, Reporting, and Evaluation of Clinical Practice Guidelines

Clinical practice parameters have been published with greater frequency by professional societies and groups of experts. These publications run the gamut of practice standards, practice guidelines, consensus statements or practice advisories, position statements, and practice alerts. The definitions of these terms have been clarified in an accompanying article. In this article, we present the criteria for high-quality clinical practice parameters and outline a process for developing them, specifically the Delphi method, which is increasingly being used to build consensus among content experts and stakeholders. Several tools for grading the level of evidence and strength of recommendation are offered and compared. The speciousness of categorizing guidelines as evidence-based or consensus-based will be explained. We examine the recommended checklist for reporting and appraise the tools for evaluating a practice guideline. This article is geared toward developers and reviewers of clinical practice guidelines and consensus statements.

Best Practice Guidelines and Essential Methodological Steps to Conduct Rigorous and Systematic Meta-Reviews

BACKGROUND

A growing body of primary study and systematic review literature evaluates interventions and phenomena in applied and health psychology. Reviews of reviews (i.e., meta-reviews) systematically synthesise and utilise this vast and often overwhelming literature; yet, currently there are few practical guidelines for meta-review authors to follow.

OBJECTIVE

The aim of this article is to provide an overview of the best practice guidelines for all research synthesis and to detail additional specific considerations and methodological details for the best practice of conducting a rigorous meta-review.

METHODS

This article provides readers with six systematic and practical steps along with accompanying examples to address with rigor the unique challenges that arise when authors familiar with systematic review methods begin a meta-review: (a) detailing a clear scope, (b) identifying synthesis literature through strategic searches, (c) considering datedness of the literature, (d) addressing overlap among included reviews, (e) choosing and applying review quality tools, and (f) appropriate options for handling the synthesis and reporting of the vast amount of data collected in a meta-review.

CONCLUSIONS

We have curated best practice recommendations and practical tips for conducting a meta-review. We anticipate that assessments of meta-review quality will ultimately formalise best-method guidelines.

Vibration of effects from diverse inclusion/exclusion criteria and analytical choices: 9216 different ways to perform an indirect comparison meta-analysis

BACKGROUND

Different methodological choices such as inclusion/exclusion criteria and analytical models can yield different results and inferences when meta-analyses are performed. We explored the range of such differences, using several methodological choices for indirect comparison meta-analyses to compare nalmefene and naltrexone in the reduction of alcohol consumption as a case study.

METHODS

All double-blind randomized controlled trials (RCTs) comparing nalmefene to naltrexone or one of these compounds to a placebo in the treatment of alcohol dependence or alcohol use disorders were considered. Two reviewers searched for published and unpublished studies in MEDLINE (August 2017), the Cochrane Library, Embase, and ClinicalTrials.gov and contacted pharmaceutical companies, the European Medicines Agency, and the Food and Drug Administration. The indirect comparison meta-analyses were performed according to different inclusion/exclusion criteria (based on medical condition, abstinence of patients before inclusion, gender, somatic and psychiatric comorbidity, psychological support, treatment administered and dose, treatment duration, outcome reported, publication status, and risk of bias) and different analytical models (fixed and random effects). The primary outcome was the vibration of effects (VoE), i.e. the range of different results of the indirect comparison between nalmefene and naltrexone. The presence of a "Janus effect" was investigated, i.e. whether the 1st and 99th percentiles in the distribution of effect sizes were in opposite directions.

RESULTS

Nine nalmefene and 51 naltrexone RCTs were included. No study provided a direct comparison between the drugs. We performed 9216 meta-analyses for the indirect comparison with a median of 16 RCTs (interquartile range = 12-21) included in each meta-analysis. The standardized effect size was negative at the 1st percentile (- 0.29, favouring nalmefene) and positive at the 99th percentile (0.29, favouring naltrexone). A total of 7.1% (425/5961) of the meta-analyses with a negative effect size and 18.9% (616/3255) of those with a positive effect size were statistically significant (p < 0.05).

CONCLUSIONS

The choice of inclusion/exclusion criteria and analytical models for meta-analysis can result in entirely opposite results. VoE evaluations could be performed when overlapping meta-analyses on the same topic yield contradictory result.

TRIAL REGISTRATION

This study was registered on October 19, 2016, in the Open Science Framework (OSF, protocol available at https://osf.io/7bq4y/ ).

Increase value and reduce waste in research on psychological therapies

A seminal Lancet series focused on increasing value and reducing waste in biomedical research, providing a transferrable template to diagnose problems in research. Our goal was to document how some of these sources of waste apply to mental health and particularly psychological treatments research. We synthesize and critically evaluate empirical findings in relation to four major sources: i) defining research priorities; ii) research design, methods and analysis; iii) accessibility of research information; iv) accuracy and usability of research reports. We demonstrate that each source of waste considered is well-represented and amply documented within this field. We describe hype and insufficient consideration of what is known in defining research priorities, persistent risk of bias, particularly due to selective outcome reporting, for psychotherapy trials across mental disorders, intellectual and financial biases, direct and indirect evidence of publication bias, largely inexistent adoption of data sharing, issues of multiplicity and fragmentation of data and findings, and insufficient adoption of reporting guidelines. We expand on a few general solutions, including supporting meta-research, properly testing interventions to increase research quality, placing open science at the center of psychological treatment research and remaining vigilant particularly regarding the strains of research currently prioritized, such as experimental psychopathology.

The commercialization of the biomedical sciences: (mis)understanding bias

The growing commercialization of scientific research has raised important concerns about industry bias. According to some evidence, so-called industry bias can affect the integrity of the science as well as the direction of the research agenda. I argue that conceptualizing industry's influence in scientific research in terms of bias is unhelpful. Insofar as industry sponsorship negatively affects the integrity of the research, it does so through biasing mechanisms that can affect any research independently of the source of funding. Talk about industry bias thus offers no insight into the particular epistemic shortcomings at stake. If the concern is with the negative effects that industry funding can have on the research agenda, conceptualizing this influence as bias obscures the ways in which such impact is problematic and limits our ability to offer solutions that can successfully address the concerns raised by the growing role of private funding in science.

Hiding negative trials by pooling them: a secondary analysis of pooled-trials publication bias in FDA-registered antidepressant trials

BACKGROUND

Previous studies on reporting bias generally examined whether trials were published in stand-alone publications. In this study, we investigated whether pooled-trials publications constitute a specific form of reporting bias. We assessed whether negative trials were more likely to be exclusively published in pooled-trials publications than positive trials and examined the research questions, individual trial results, and conclusions presented in these articles.

METHODS

Data from a cohort of 105 randomized controlled trials of 16 antidepressants were extracted from earlier publications and the corresponding Food and Drug Administration (FDA) reviews. A systematic literature search was conducted to identify pooled-trials publications.

RESULTS

We found 107 pooled-trials publications that reported results of 23 (72%) of 32 trials not published in stand-alone publications. Only two (3.8%) of 54 positive trials were published exclusively in pooled-trials publications, compared with 21 (41.1%) of 51 negative trials (p < 0.001). Thirteen (12%) of 107 publications had as primary aim to present data on the trial's primary research question (drug efficacy compared with placebo). Only four of these publications, reporting on five (22%) trials, presented individual efficacy data for the primary research question. Additionally, only five (5%) of 107 pooled-trials publications had a negative conclusion.

CONCLUSIONS

Compared with positive trials, negative trials of antidepressants for depression were much more likely to be reported exclusively in pooled-trials publications. Pooled-trials publications flood the evidence base with often-redundant articles that, instead of addressing the original primary research question, present (positive) results on secondary questions. Therefore, pooled-trials publications distort the apparent risk-benefit profile of antidepressants.

Antidepressant withdrawal - the tide is finally turning

Withdrawal reactions when coming off antidepressants have long been neglected or minimised. It took almost two decades after the selective serotonin reuptake inhibitors (SSRIs) entered the market for the first systematic review to be published. More reviews have followed, demonstrating that the dominant and long-held view that withdrawal is mostly mild, affects only a small minority and resolves spontaneously within 1-2 weeks, was at odd with the sparse but growing evidence base. What the scientific literature reveals is in close agreement with the thousands of service user testimonies available online in large forums. It suggests that withdrawal reactions are quite common, that they may last from a few weeks to several months or even longer, and that they are often severe. These findings are now increasingly acknowledged by official professional bodies and societies.

Financial conflicts of interest in systematic reviews: associations with results, conclusions, and methodological quality

BACKGROUND

Financial conflicts of interest in systematic reviews (e.g. funding by drug or device companies or authors' collaboration with such companies) may impact on how the reviews are conducted and reported.

OBJECTIVES

To investigate the degree to which financial conflicts of interest related to drug and device companies are associated with results, conclusions, and methodological quality of systematic reviews.

SEARCH METHODS

We searched PubMed, Embase, and the Cochrane Methodology Register for studies published up to November 2016. We also read reference lists of included studies, searched grey literature sources, and Web of Science for studies citing the included studies.

SELECTION CRITERIA

Eligible studies were studies that compared systematic reviews with and without financial conflicts of interest in order to investigate differences in results (estimated treatment effect and frequency of statistically favourable results), frequency of favourable conclusions, or measures of methodological quality of the review (e.g. as evaluated on the Oxman and Guyatt index).

DATA COLLECTION AND ANALYSIS

Two review authors independently determined the eligibility of studies, extracted data, and assessed risk of bias. We synthesised the results of each study relevant to each of our outcomes. For meta-analyses, we used Mantel-Haenszel random-effects models to estimate risk ratios (RR) with 95% confidence intervals (CIs), with RR > 1 indicating that systematic reviews with financial conflicts of interest more frequently had statistically favourable results or favourable conclusions, and had lower methodological quality. When a quantitative synthesis was considered not meaningful, results from individual studies were summarised qualitatively.

MAIN RESULTS

Ten studies with a total of 995 systematic reviews of drug studies and 15 systematic reviews of device studies were included. We assessed two studies as low risk of bias and eight as high risk, primarily because of risk of confounding. The estimated treatment effect was not statistically significantly different for systematic reviews with and without financial conflicts of interest (Z-score: 0.46, P value: 0.64; based on one study of 14 systematic reviews which had a matched design, comparing otherwise similar systematic reviews). We found no statistically significant difference in frequency of statistically favourable results for systematic reviews with and without financial conflicts of interest (RR: 0.84, 95% CI: 0.62 to 1.14; based on one study of 124 systematic reviews). An analysis adjusting for confounding due to methodological quality (i.e. score on the Oxman and Guyatt index) provided a similar result. Systematic reviews with financial conflicts of interest more often had favourable conclusions compared with systematic reviews without (RR: 1.98, 95% CI: 1.26 to 3.11; based on seven studies of 411 systematic reviews). Similar results were found in two studies with a matched design, which therefore had a reduced risk of confounding. Systematic reviews with financial conflicts of interest tended to have lower methodological quality compared with systematic reviews without financial conflicts of interest (RR for 11 dimensions of methodological quality spanned from 1.00 to 1.83). Similar results were found in analyses based on two studies with matched designs.

AUTHORS' CONCLUSIONS

Systematic reviews with financial conflicts of interest more often have favourable conclusions and tend to have lower methodological quality than systematic reviews without financial conflicts of interest. However, it is uncertain whether financial conflicts of interest are associated with the results of systematic reviews. We suggest that patients, clinicians, developers of clinical guidelines, and planners of further research could primarily use systematic reviews without financial conflicts of interest. If only systematic reviews with financial conflicts of interest are available, we suggest that users read the review conclusions with skepticism, critically appraise the methods applied, and interpret the review results with caution.

Systematic reviews and meta-analyses in the health sciences: Best practice methods for research syntheses

RATIONALE

The journal Social Science & Medicine recently adopted the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA; Moher et al., 2009) as guidelines for authors to use when disseminating their systematic reviews (SRs).

APPROACH

After providing a brief history of evidence synthesis, this article describes why reporting standards are important, summarizes the sequential steps involved in conducting SRs and meta-analyses, and outlines additional methodological issues that researchers should address when conducting and reporting results from their SRs.

RESULTS AND CONCLUSIONS

Successful SRs result when teams of reviewers with appropriate expertise use the highest scientific rigor in all steps of the SR process. Thus, SRs that lack foresight are unlikely to prove successful. We advocate that SR teams consider potential moderators (M) when defining their research problem, along with Time, Outcomes, Population, Intervention, Context, and Study design (i.e., TOPICS + M). We also show that, because the PRISMA reporting standards only partially overlap dimensions of methodological quality, it is possible for SRs to satisfy PRISMA standards yet still have poor methodological quality. As well, we discuss limitations of such standards and instruments in the face of the assumptions of the SR process, including meta-analysis spanning the other SR steps, which are highly synergistic: Study search and selection, coding of study characteristics and effects, analysis, interpretation, reporting, and finally, re-analysis and criticism. When a SR targets an important question with the best possible SR methods, its results can become a definitive statement that guides future research and policy decisions for years to come.

The intersections of industry with the health research enterprise

There is increased awareness of the negative impact of large multinational corporations - the 'industry' - on public health. These corporations have established different types of relationships with a number of actors in the field of health research. This Commentary explores the different types of relationships between the industry and the actors of health research, how they intersect with the different research steps, and how these relationships allow the industry to exert influence. The types of relationships discussed consist of funding of research, direct relationships with the actors of research (namely advocacy groups, funding agencies, experts, professional organisations, regulatory agencies and health practitioners), and the influencing research standards. The potentially influenced research steps either precede the research (i.e. the prioritisation of research question), relate to it directly (i.e. its planning, conduct, reporting, dissemination and evaluation), or build on it (i.e. regulatory approval, integration into guidelines and adoption into practice). In conclusion, the industry has successfully fostered relationships with almost every actor of the health research enterprise and is using these relationships to influence the different steps of health research. The degree of influence the industry is having on health research calls for more work on managing the relationships discussed herein.

Protocol registration or development may benefit the design, conduct and reporting of dose-response meta-analysis: empirical evidence from a literature survey

Background

To investigate the prevalence of protocol registration (or development) among published dose-response meta-analyses (DRMAs), and whether DRMAs with a protocol are better than those not.

Methods

Three databases were searched for eligible DRMAs. The modified AMSTAR (14 items) and PRISMA checklists (26 items) were used to assess the methodological and reporting quality, with each item assigned 1 point if it met the requirement or 0 if not. We matched (1,2) DRMAs with registered or published protocol to those not, by region and publication years. The summarized quality score and compliance rate of each item were compared between the two groups. Multivariable regression was employed to see if protocol registration or development was associated with total quality score.

Results

We included 529 DRMAs, with 45 (8.51%) completed protocol registration or development. We observed a higher methodological score for DRMAs with protocol than the matched controls (9.47 versus 8.58, P <  0.01); this embodied in 4 out of 14 items of AMSTAR [e.g., Duplicate data extraction (rate difference, RD = 0.17, 95% CI: 0.04, 0.30; P = 0.01). A higher reporting score (cubic transformed) for DRMAs with protocol than the matched controls was also observed (11,875.00 versus 10,229.53, P <  0.01); which embodied in 6 out of 26 items of PRISMA [e.g. Describe methods for publication bias (RD = 0.08, 95% CI: 0.01, 0.14; P = 0.02)]. Regression analysis suggested positive association between protocol registration or development and total reporting score (P = 0.012) while not for methodological score (P = 0.87).

Conclusions

Only a small proportion of DRMAs completed protocol registration or development, and those with protocol were better reported than those not. Protocol registration or development is highly desirable.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0715-y) contains supplementary material, which is available to authorized users.

The timing and frequency of trial inclusion in systematic reviews of type 2 diabetes drugs was associated with trial characteristics

OBJECTIVE

To determine whether certain trial characteristics are associated with faster or more frequent inclusion in systematic reviews for drug interventions in type 2 diabetes.

STUDY DESIGN AND SETTING

We examined trials included in systematic reviews published between January 1, 2007 and January 1, 2017. Primary outcomes were time between trial publication and first inclusion in a systematic review and frequency of inclusion in systematic reviews over the study period. Multivariable Cox proportional hazards and regression models quantified associations with funding source, number of participants, trial conclusion, and journal impact factor.

RESULTS

Among 668 trials, the median time to inclusion was 76.1 weeks. Time to inclusion was shorter for trials with industry funding (hazard ratio [HR] 1.39; 95% confidence interval [CI] 1.13-1.71), more participants (HR 1.26; 95% CI 1.17-1.36), and published in higher impact factor journals (HR 1.28; 95% CI 1.14-1.45). The median frequency of inclusion was three. Frequency of inclusion was greater for trials with industry funding (relative risk [RR] 2.36; 95% CI 2.11-2.64), more participants (RR 1.51; 95% CI 1.47-1.55), positive conclusions (RR 1.89; 95% CI 1.68-2.13), and published in higher impact factor journals (RR 1.13; 95% CI 1.08-1.18).

CONCLUSION

Certain trial characteristics are associated with faster or more frequent trial inclusion in systematic reviews of type 2 diabetes.

Inadequate diversity of information resources searched in US-affiliated systematic reviews and meta-analyses: 2005-2016

OBJECTIVE

Systematic reviews and meta-analyses (SRMAs) rely upon comprehensive searches into diverse resources that catalog primary studies. However, since what constitutes a comprehensive search is unclear, we examined trends in databases searched from 2005-2016, surrounding the publication of search guidelines in 2013, and associations between resources searched and evidence of publication bias in SRMAs involving human subjects.

STUDY DESIGN

To ensure comparability of included SRMAs over the 12 years in the face of a near 100-fold increase of international SRMAs (mainly genetic studies from China) during this period, we focused on USA-affiliated SRMAs, manually reviewing 100 randomly selected SRMAs from those published in each year. After excluding articles (mainly for inadequate detail or out-of-scope methods), we identified factors associated with the databases searched, used network analysis to see which resources were simultaneously searched, and used logistic regression to link information sources searched with a lower chance of finding publication bias.

RESULTS

Among 817 SRMA articles studied, the common resources used were Medline (95%), EMBASE (44%), and Cochrane (41%). Methods journal SRMAs were most likely to use registries and grey literature resources. We found substantial co-searching of resources with only published materials, and not complemented by searches of registries and the grey literature. The 2013 guideline did not substantially increase searching of registries and grey literature resources to retrieve primary studies for the SRMAs. When used to augment Medline, Scopus (in all SRMAs) and ClinicalTrials.gov (in SRMAs with safety outcomes) were negatively associated with publication bias.

CONCLUSIONS

Even SRMAs that search multiple sources tend to search similar resources. Our study supports searching Scopus and CTG in addition to Medline to reduce the chance of publication bias.

The Neurocognitive Profile of Attention-Deficit/Hyperactivity Disorder: A Review of Meta-Analyses

Objective

Numerous meta-analyses have summarized studies comparing the neurocognitive performance of individuals with attention-deficit/hyperactivity disorder (ADHD) to that of healthy controls.

Method

The present study is a systematic review and quantitative summary of those meta-analyses that aimed to determine the extent to which individuals with ADHD differ cognitively from typically developing controls.

Results

Of 253 standardized mean differences (SMDs) drawn from 34 meta-analyses, 244 (96%) were positive, indicating better neurocognitive performance in the control group than the ADHD group. The mean effect size was .45 (SD = .27). Unweighted means of SMDs for neurocognitive domains ranged from .35 (set shifting) to .54 (working memory). When weighted by the number of studies aggregated, they ranged from .35 (set shifting) to .66 (reaction time variability). Neurocognitive domains with mean effects over .50 included working memory (.54), reaction time variability (.53), response inhibition (.52), intelligence/achievement (.51), and planning/organization (.51). When weighted by number of aggregated studies, the domains with mean effects over .50 were reaction time variability (.66), intelligence/achievement (.60), vigilance (.56), working memory (.54), and response inhibition (.52). Age moderated the relationship between ADHD diagnosis and neurocognitive functioning, with greater between-groups differences among children and adults than among adolescents. Funding also moderated this relationship: meta-analyses that received drug funding found larger effect sizes than those without drug funding.

Conclusions

The evidence suggests that ADHD is associated with substantial deficits across a variety of neurocognitive domains. This is the most in-depth review of the neurocognitive functioning of people with ADHD to date.

The discordance between evidence and health policy in the United States: the science of translational research and the critical role of diverse stakeholders

Background

There is often a discordance between health research evidence and public health policies implemented by the United States federal government. In the process of developing health policy, discordance can arise through subjective and objective factors that are unrelated to the value of the evidence itself, and can inhibit the use of research evidence. We explore two common types of discordance through four illustrative examples and then propose a potential means of addressing discordance.

Discussion

In Discordance 1, public health authorities make recommendations for policy action, yet these are not based on high quality, rigorously synthesised research evidence. In Discordance 2, evidence-based public health recommendations are ignored or discounted in developing United States federal government policy. Both types could lead to serious risks of public health and clinical patient harms.

We suggest that, to mitigate risks associated with these discordances, public health practitioners, health policy-makers, health advocates and other key stakeholders should take the opportunity to learn or expand their knowledge regarding current research methods, as well as improve their skills for appropriately considering the strengths and limitations of research evidence. This could help stakeholders to adopt a more nuanced approach to developing health policy. Stakeholders should also have a more insightful contextual awareness of these discordances and understand their potential harms. In Discordance 1, public health organisations and authorities need to acknowledge their own historical roles in making public health recommendations with insufficient evidence for improving health outcomes. In Discordance 2, policy-makers should recognise the larger impact of their decision-making based on minimal or flawed evidence, including the potential for poor health outcomes at population level and the waste of huge sums. In both types of discordance, stakeholders need to consider the impact of their own unconscious biases in championing evidence that may not be valid or conclusive.

Conclusion

Public health policy needs to provide evidence-based solutions to public health problems, but this is not always done. We discuss some of the factors inhibiting evidence-based decision-making in United States federal government public health policy and suggest ways these could be addressed.

The currency, completeness and quality of systematic reviews of acute management of moderate to severe traumatic brain injury: A comprehensive evidence map

OBJECTIVE

To appraise the currency, completeness and quality of evidence from systematic reviews (SRs) of acute management of moderate to severe traumatic brain injury (TBI).

METHODS

We conducted comprehensive searches to March 2016 for published, English-language SRs and RCTs of acute management of moderate to severe TBI. Systematic reviews and RCTs were grouped under 12 broad intervention categories. For each review, we mapped the included and non-included RCTs, noting the reasons why RCTs were omitted. An SR was judged as 'current' when it included the most recently published RCT we found on their topic, and 'complete' when it included every RCT we found that met its inclusion criteria, taking account of when the review was conducted. Quality was assessed using the AMSTAR checklist (trichotomised into low, moderate and high quality).

FINDINGS

We included 85 SRs and 213 RCTs examining the effectiveness of treatments for acute management of moderate to severe TBI. The most frequently reviewed interventions were hypothermia (n = 17, 14.2%), hypertonic saline and/or mannitol (n = 9, 7.5%) and surgery (n = 8, 6.7%). Of the 80 single-intervention SRs, approximately half (n = 44, 55%) were judged as current and two-thirds (n = 52, 65.0%) as complete. When considering only the most recently published review on each intervention (n = 25), currency increased to 72.0% (n = 18). Less than half of the 85 SRs were judged as high quality (n = 38, 44.7%), and nearly 20% were low quality (n = 16, 18.8%). Only 16 (20.0%) of the single-intervention reviews (and none of the five multi-intervention reviews) were judged as current, complete and high-quality. These included reviews of red blood cell transfusion, hypothermia, management guided by intracranial pressure, pharmacological agents (various) and prehospital intubation. Over three-quarters (n = 167, 78.4%) of the 213 RCTs were included in one or more SR. Of the remainder, 17 (8.0%) RCTs post-dated or were out of scope of existing SRs, and 29 (13.6%) were on interventions that have not been assessed in SRs.

CONCLUSION

A substantial number of SRs in acute management of moderate to severe TBI lack currency, completeness and quality. We have identified both potential evidence gaps and also substantial research waste. Novel review methods, such as Living Systematic Reviews, may ameliorate these shortcomings and enhance utility and reliability of the evidence underpinning clinical care.