Publication Bias and Nonreporting Found in Majority of Systematic Reviews and Meta-analyses in Anesthesiology Journals

Methodologies for network meta-analysis of randomised controlled trials in pain, anaesthesia, and perioperative medicine: a narrative review

Network meta-analysis has emerged as a method for analysing clinical trials, with a large increase in the number of publications over the past decade. Network meta-analysis offers advantages over traditional pairwise meta-analysis, including increased power, the ability to compare treatments not compared in the original trials, and the ability to rank treatments. However, network meta-analyses are inherently more complex than pairwise meta-analyses, requiring additional statistical expertise and assumptions. Many factors can affect the certainty of evidence from pairwise meta-analysis and can often lead to unreliable results. Network meta-analysis is prone to all these issues, although it has the additional assumption of transitivity. Here we review network meta-analyses, problems with their conduct and reporting, and methodological strategies that can be used by those conducting reviews to help improve the reliability of their findings. We provide evidence that violation of the assumption of transitivity is relatively common and inadequately considered in published network meta-analyses. We explain key concepts with clinically relevant examples for those unfamiliar with network meta-analysis to facilitate their appraisal and application of their results to clinical practice.

Risk of bias and problematic trials: characterising the research integrity of trials submitted to Anaesthesia

BACKGROUND

There is some evidence for systematic biases and failures of research integrity in the anaesthesia literature. However, the features of problematic trials and effect of editorial selection on these issues have not been well quantified.

METHODS

We analysed 209 randomised controlled trials submitted to Anaesthesia between 8 March 2019 and 31 March 2020. We evaluated the submitted manuscript, registry data and the results of investigations into the integrity of the trial undertaken at the time of submission. Trials were labelled 'concerning' if failures of research integrity were found, and 'problematic' if identified issues would have warranted retraction if they had been found after publication. We investigated how 'problematic' trials were detected, the distribution of p values and the risk of outcome reporting bias and p-hacking. We also investigated whether there were any factors that differed in problematic trials.

RESULTS

We found that false data was the most common reason for a trial to be labelled as 'concerning', which occurred in 51/62 (82%) cases. We also found that while 195/209 (93%) trials were preregistered, we found adequate registration for only 166/209 (79%) primary outcomes, 100/209 (48%) secondary outcomes and 11/209 (5%) analysis plans. We also found evidence for a step decrease in the frequency of p values > 0.05 compared with p values < 0.05. 'Problematic' trials were all single-centre and appeared to have fewer authors (incident risk ratio (95%CI) 0.8 (0.7-0.9)), but could not otherwise be distinguished reliably from other trials.

CONCLUSIONS

Identification of 'problematic' trials is frequently dependent on individual patient data, which is often unavailable after publication. Additionally, there is evidence of a risk of outcome reporting bias and p-hacking in submitted trials. Implementation of alternative research and editorial practices could reduce the risk of bias and make identification of problematic trials easier.

Evidence-based medicine or statistically manipulated medicine? Are we slaves to the P-value?

First popularized almost a century ago in epidemiologic research by Ronald Fisher and Jerzy Neyman, the P-value has become perhaps the most misunderstood and even misused statistical value or descriptor. Indeed, modern clinical research has now come to be centered around and guided by an arbitrary P-value of <0.05 as a magical threshold for significance, so much so that experimental design, reporting of experimental findings, and interpretation and adoption of such findings have become largely dependent on this "significant" P-value. This has given rise to multiple biases in the overall body of biomedical literature that threatens the very validity of clinical research. Ultimately, a drive toward reporting a "significant" P-value (by various statistical manipulations) risks creating a falsely positive body of science, leading to (i) wasted resources in pursuing fruitless research and (ii) futile or even harmful policies/therapeutic recommendations. This article reviews the history of the P-value, the conceptual basis of P-value in the context of hypothesis testing and challenges in critically appraising clinical evidence vis-à-vis the P-value. This review is aimed at raising awareness of the pitfalls of this rigid observation of the threshold of statistical significance when evaluating clinical trials and to generate discussion regarding whether the scientific body needs a rethink about how we decide clinical significance.

A Cross-Sectional Evaluation of Publication Bias in the Plastic Surgery Literature

BACKGROUND

Publication bias (PB) is the preferential publishing of studies with statistically significant results. PB can skew findings of systematic reviews (SRs) and meta-analyses (MAs), with potential consequences for patient care and health policy. This study aims to determine the extent to which SRs and MAs in the plastic surgery literature evaluate and report PB.

METHODS

This cross-sectional study assessed PB reporting and analysis from plastic surgery studies published between January 1, 2015, and June 19, 2020. Full texts of SRs and MAs were assessed by two reviewers for PB assessment methodology and analysis. Post hoc assessment of studies that did not originally analyze PB was performed using Egger regression, Duval, Tweedie trim-and-fill, and Copas selection models.

RESULTS

There were 549 studies evaluated, of which 531 full texts were included. PB was discussed by 183 studies (34.5%), and formally assessed by 97 studies (18.3%). Among SRs and MAs that formally assessed PB, PB was present in 24 studies (10.7%), not present in 52 (23.1%), and inconclusive in eight (3.6%); 141 studies (62.7%) did not report the results of their PB assessment. Funnel plots were the most common assessment method [ n = 88 (39.1%)], and 60 studies (68.2%) published funnel plots. The post hoc assessment revealed PB in 17 of 20 studies (85.0%).

CONCLUSIONS

PB is inadequately reported and analyzed among studies in the plastic surgery literature. Most studies that assessed PB found PB, as did post hoc analysis of nonreporting studies. Increased assessment and reporting of PB among SRs and MAs would improve the quality of evidence in plastic surgery.

Use and Clinical Relevancy of Pericapsular Nerve Block (PENG) in Total Hip Arthroplasty: A Systematic Review and Meta-analysis

BACKGROUND

Ultrasound-guided pericapsular nerve group (PENG) block is an emerging regional anesthesia technique that may provide analgesia for patients undergoing total hip arthroplasties (THA). There are clinical studies comparing this fascial plane block to other established methods; however, evidence on the actual efficacy of this block for THA continues to evolve.

OBJECTIVE

Available clinical studies conducted over the past 4 years were reviewed to evaluate the analgesic efficacy and effectiveness of PENG block in patients undergoing THAs.

METHODS

A meta-analysis of randomized controlled trials (RCTs) in patients undergoing THA, where PENG block was compared to no block, placebo/sham block (injection with saline), or other analgesic techniques including suprainguinal fascia iliaca block (FIB), or periarticular infiltration (PAI) was performed. Our primary outcome was opioid consumption during the first 24 hours. Secondary outcomes were postoperative rest and dynamic pain scores at 6-12, 24 and 48 hours, block performance time, sensory-motor assessment, quadriceps weakness, the incidence of postoperative falls, first analgesic request, block and opioid-related complications, surgical complications, patient satisfaction scores, postanesthesia care unit length of stay, hospital length of stay, and functional and quality of life outcomes.

RESULTS

We included 12 RCTs with a total of 705 patients. Data showed that PENG block decreased 24-hour oral morphine milligram equivalent consumption by a mean difference (MD) of 3.75 mg (95% CI: -5.96,-1.54; P =0.0009). No statistically significant differences in rest or dynamic pain were found, except for a modest MD reduction in dynamic pain score of 0.55 points (95% CI: -0.98, -0.12; P =0.01), measured 24 hours after surgery in favor of PENG block.

CONCLUSIONS

Our systematic review and meta-analysis suggest that PENG block provides better analgesia, measured as MME use, in the first 24 hours after THA, with no real impact on postoperative VAS scores. Despite statistical significance, the high heterogeneity across RCTs implies that PENG's benefits may not surpass the minimal clinically important difference threshold for us to recommend PENG as best practice in THA.

Development and Publication of Clinical Practice Parameters, Reviews, and Meta-analyses: A Report From the Society of Cardiovascular Anesthesiologists Presidential Task Force

The Society of Cardiovascular Anesthesiologists (SCA) is committed to improving the quality, safety, and value that cardiothoracic anesthesiologists bring to patient care. To fulfill this mission, the SCA supports the creation of peer-reviewed manuscripts that establish standards, produce guidelines, critically analyze the literature, interpret preexisting guidelines, and allow experts to engage in consensus opinion. The aim of this report, commissioned by the SCA President, is to summarize the distinctions among these publications and describe a novel SCA-supported framework that provides guidance to SCA members for the creation of these publications. The ultimate goal is that through a standardized and transparent process, the SCA will facilitate up-to-date education and implementation of best practices by cardiovascular and thoracic anesthesiologists to improve patient safety, quality of care, and outcomes.

A Review of Meta-Analyses in Plastic Surgery: Need for Adequate Assessment of Publication Bias

INTRODUCTION

Publication bias describes a phenomenon in which significant positive results have a higher likelihood of being published compared to negative or nonsignificant results. Publication bias can confound the estimated therapeutic effect in meta-analyses and needs to be adequately assessed in the surgical literature.

METHODS

A review of meta-analyses published in five plastic surgery journals from 2002 to 2022 was conducted. The inclusion criteria for meta-analyses were factors that demonstrated an obligation to assess publication bias, such as interventions with comparable treatment groups and enough power for statistical analysis. Acknowledgment of publication bias risk, quality of bias assessment, methods used in assessment, and individual article factors were analyzed.

RESULTS

318 unique meta-analyses were identified in literature search, and after full-text reviews, 143 met the inclusion criteria for obligation to assess publication bias. 64% of eligible meta-analyses acknowledged the confounding potential of publication bias, and only 46% conducted a formal assessment. Of those who conducted an assessment, 49% used subjective inspection of funnel plots alone, while 47% used any statistical testing in analysis. Overall, only 9/143 (6.3%) assessed publication bias and attempted to correct for its effect. Journals with a higher average impact factor were associated with mention and assessment of publication bias, but more recent publication year and higher number of primary articles analyzed were not.

CONCLUSIONS

This review identified low rates of proper publication bias assessment in meta-analyses published in five major plastic surgery journals. Assessment of publication bias using objective statistical testing is necessary to ensure quality literature within surgical disciplines.

Narrative Review to Meta-Analysis: A Cookbook Approach to Evidence Synthesis in Surgical Research

Evidence synthesis is a complex approach to research that can consolidate the current understanding of a particular topic from various sources. A design hierarchy based upon reliability is described in detail. Methodology is described explicitly to provide readers with a foundation for performing and understanding published evidence synthesis. Resources that detail access to the comprehensive database are presented and explained. Special care is taken to discuss appraisal of studies prior to analysis.

Simulation-Based Analysis of Trial Design in Regional Anesthesia

Background

In regional anesthesia, the efficacy of novel blocks is typically evaluated using randomized controlled trials (RCTs), the findings of which are aggregated in systematic reviews and meta-analyses. Systematic review authors frequently point out the small sample size of RCTs as limiting conclusions from this literature. We sought to determine via statistical simulation if small sample size could be an expected property of RCTs focusing on novel blocks with typical effect sizes.

Methods

We simulated the conduct of a series of RCTs comparing a novel block versus placebo on a single continuous outcome measure. Simulation analysis inputs were obtained from a systematic bibliographic search of meta-analyses. Primary outcomes were the predicted number of large trials (empirically defined as N ≥ 256) and total patient enrollment.

Results

Simulation analysis predicted that a novel block would be tested in 16 RCTs enrolling a median of 970 patients (interquartile range (IQR) across 1000 simulations: 806, 1269), with no large trials. Among possible modifications to trial design, decreasing the statistical significance threshold from p < 0.05 to p < 0.005 was most effective at increasing the total number of patients represented in the final meta-analysis, but was associated with early termination of the trial sequence due to futility in block vs. block comparisons.

Conclusion

Small sample size of regional anesthesia RCTs comparing novel block to placebo is a rational outcome of trial design. Feasibly large trials are unlikely to change conclusions regarding block vs. placebo comparisons.

Publication bias in otorhinolaryngology meta-analyses in 2021

INTRODUCTION

One concern in meta-analyses is the presence of publication bias (PB) which leads to the dissemination of inflated results. In this study, we assessed how much the meta-analyses in the field of otorhinolaryngology in 2021 evaluated the presence of PB.

METHODS

Six of the most influential journals in the field were selected. A search was conducted, and data were extracted from the included studies. In cases where PB was not assessed by the authors, we evaluated the risk of its presence by designing funnel plots and performing statistical tests.

RESULTS

Seventy-five systematic reviews were included. Fifty-one percent of them used at least one method for assessing the risk of PB, with the visual inspection of a funnel plot being the most frequent method used. Twenty-nine percent of the studies reported a high risk of PB presence. We replicated the results of 11 meta-analyses that did not assess the risk of PB and found that 63.6% were at high risk. We also found that a considerable proportion of the systematic reviews that found a high risk of PB did not take it into consideration when making conclusions and discussing their results.

DISCUSSION

Our results indicate that systematic reviews published in some of the most influential journals in the field do not implement enough measures in their search strategies to reduce the risk of PB, nor do they assess the risk of its presence or take the risk of its presence into consideration when inferring their results.

The Influence of Industry Affiliation on Randomized Controlled Trials of Platelet-Rich Plasma for Knee Osteoarthritis

BACKGROUND

Industry funding and corporate sponsorship have played a significant role in the advancement of orthopaedic research and technology. However, this relationship raises concerns for how industry association may bias research findings and influence clinical practice.

PURPOSE

To determine whether industry affiliation plays a role in the outcomes of randomized controlled trials (RCTs) investigating platelet-rich plasma (PRP).

STUDY DESIGN

Meta-analysis; Level of evidence, 2.

METHODS

A search of the PubMed, Cochrane, and MEDLINE databases for RCTs published between 2011 and the present comparing PRP versus hyaluronic acid, corticosteroid, or placebo for the treatment of knee osteoarthritis was performed. To determine industry affiliation, the conflict of interest, funding, and disclosure sections of publications were assessed, and all authors were assessed through the American Academy of Orthopaedic Surgeons disclosure database and the Centers for Medicare & Medicaid Services open payments database. Studies were classified as industry affiliated (IA) or non-industry affiliated (NIA). The outcomes of each study were rated as favorable, analogous, or unfavorable according to predefined criteria.

RESULTS

A total of 37 studies (6 IA and 31 NIA) were available for analysis. Overall, 19 studies (51.4%) reported PRP as favorable compared with other treatment options, while 18 studies (48.6%) showed no significant differences between PRP and other treatment methods. There was no significant difference in qualitative conclusions between the IA and NIA groups, with the IA group having 3 favorable studies and 3 analogous studies and the NIA group having 16 favorable studies and 15 analogous studies (P = .8881). When comparing IA versus NIA studies using 6- and 12-month Western Ontario and McMaster Universities Arthritis Index and International Knee Documentation Committee scores, there were no significant differences in outcomes.

CONCLUSION

The results of this study demonstrated that qualitative conclusions and outcome scores were found to not be associated with industry affiliation. Although the results of this study suggest that there is no influence of industry involvement on RCTs examining PRP, it is still necessary to carefully evaluate pertinent commercial affiliations when reviewing recommendations from studies before adopting new treatment approaches, such as the use of PRP for knee osteoarthritis.

Immunogenicity and safety of inactivated quadrivalent influenza vaccine compared with the trivalent vaccine for influenza infection: an overview of systematic reviews

BACKGROUND

Influenza infection is a highly preventable transmissible viral disease associated with mild upper respiratory symptoms and more severe conditions such as lethal pneumonia. Studies have shown that a broader spectrum influenza vaccine could reduce influenza's burden of disease in low- and middle-income countries. A considerable number of systematic reviews reported that quadrivalent influenza vaccines are considered more effective compared to trivalent vaccines, hence, there is a need for an overview in order to synthesize the current evidence pertaining to the comparison between quadrivalent and trivalent inactivated influenza vaccines.

OBJECTIVE

The aim was to summarize the evidence from systematic reviews that investigated the immunogenicity and safety of the Influenza's inactivated quadrivalent vaccine (QIV) compared to the trivalent vaccine (TIV), in the general population.

METHODS

We searched articles up to December 2022 at: Web of Science, EMBASE, MEDLINE, Cochrane Library, and SCOPUS. The search strategy was conducted following the PICO model. We included systematic reviews comparing the primary outcomes of immunogenicity (seroprotection rate and seroconversion rate) and adverse events using risk ratios. The AMSTAR 2 and ROBIS were used for quality assessments, and GRADE was used for evidence certainty assessments.

FINDINGS

We included five systematic reviews, totalling 47,740 participants. The Quadrivalent Inactivated Influenza Vaccine (QIV) exhibited enhanced immunogenicity in the context of B-lineage mismatch when compared to the Trivalent Inactivated Influenza Vaccine (TIV). While the safety profile of QIV was found to be comparable to that of TIV, the QIV showed a higher incidence of solicited local pain among children and adolescents, as well as an increased frequency of local adverse events within the adult population.

CONCLUSION

Our findings suggest that the QIV provides a superior immunogenicity response compared to the TIV in all age groups evaluated, especially when a lineage mismatch occurred. The safety of QIV was considered similar to the TIV, with no serious or systemic solicited or unsolicited adverse events; tough pain at the injection site was greater for QIV. We recommend caution owing to the high risk of bias in the selection process and no protocol registration.

Risk of bias assessment tool for systematic review and meta-analysis of the gut microbiome

Risk of bias assessment is a critical step of any meta-analysis or systematic review. Given the low sample count of many microbiome studies, especially observational or cohort studies involving human subjects, many microbiome studies have low power. This increases the importance of performing meta-analysis and systematic review for microbiome research in order to enhance the relevance and applicability of microbiome results. This work proposes a method based on the ROBINS-I tool to systematically consider sources of bias in microbiome research seeking to perform meta-analysis or systematic review for microbiome studies.

Booming the bugs: How can marketing help increase consumer acceptance of insect-based food in Western countries?

Edible insects could be a promising answer to the question of how to feed a growing human population with a healthier and more sustainable source of protein. Despite the increasing interest in entomophagy in food science and industry, however, consumer acceptance for insect-based food products remains low in Western countries. This systematic review provides a comprehensive and timely overview of studies relevant for researchers, practitioners and other stakeholders involved in the marketing of these products. Reviewing data from 45 selected studies, we focus on marketing measures tested for their influence on Western consumers' preference, acceptance, willingness to try, eat and/or buy insect-based food products. Organising the findings according to the 4Ps of the marketing mix framework, five main ways of enhancing the appeal and acceptance of insect-based food products are discussed: 1) developing product attributes that align with the specific preferences of target consumers; 2) a more subtle labelling of the presence of insects; 3) using value-added or competition-based pricing strategies; 4) making products consistently available in the marketplace; and 5) promoting products more effectively through advertising, tasting activities, and social influence. The heterogeneity we find among the studies, whether due to differences in the products researched, in sampling countries, or in data-collection methods, indicates key research gaps for future studies to address.

Publication bias in meta-analyses of the therapeutic efficacy of remdesivir interventions for patients with COVID-19

Purpose

No comprehensive statistical assessment of publication bias has been conducted in remdesivir-based intervention research for COVID-19 patients. This study aims to examine all meta-analyses of the efficacy of remdesivir interventions in COVID-19 patients and perform a statistical assessment of publication bias.

Design/methodology/approach

This is an analytic study conducted to assess the impact of publication bias on the results of meta-analyses of remdesivir-based interventions in patients infected with COVID-19. All English full-text meta-analyses published in peer-reviewed journals in 2019–2021 were included. A computerized search of PubMed and Web of Science electronic databases was performed on December 24, 2021. The trim-and-fill method calculated the number of missing studies and the adjusted cumulative effect sizes.

Findings

The final analysis comprised 21 studies with 88 outcomes. The investigation revealed missing studies in 46 outcomes (52%). Seventy-six missing studies were replaced in the outcomes using the trim-and-fill procedure. The adjusted recalculated effect sizes of the 27 outcomes increased by an average of 0.04. In comparison, the adjusted effect size of 18 outcomes fell by an average of 0.036. Only 14 out of 46 outcomes with publication bias were subjected to a gray literature search (30%). To discover related research, no gray literature search was conducted in most outcomes with publication bias (n = 32; 70%). In conclusion, the reported effect estimates regarding the effect of remdesivir in COVID-19 patients are only slightly affected by publication bias and can be considered authentic. Health-care decision-makers in COVID-19 should consider current research results when making clinical decisions.

Research limitations/implications

Most health decisions are based on the effect sizes revealed in meta-analyses. When deciding on remdesivir-based treatment for COVID-19 patients, therefore, the outcomes of this investigation may be of paramount importance to health policymakers, leading to better treatment strategies.

Practical implications

According to the results, no major publication bias and missing studies were detected on average. Therefore, the calculated effect sizes of remdesivir-based interventions on meta-analyses can be used as authentic and unbiased benchmarks by health-care decision-makers in treating patients with COVID-19.

Originality/value

This is the first study to examine the effect of publication bias and gray literature searches on the results of meta-analyses of treatment with COVID-19 (remdesivir).

Publication bias in pharmacogenetics of adverse reaction to antiseizure drugs: An umbrella review and a meta-epidemiological study

Publication bias may lead to a misestimation in the association between pharmacogenetic biomarkers (PGx) and antiseizure drug's adverse effects (AEs). We aimed to assess its prevalence in this field. We searched for systematic reviews assessing PGx of antiseizure drug's AEs. For each unique association between a PGx, a drug and its AE, we used the available odds ratio (ORs) to generate corresponding funnel plots. We estimated the prevalence of publication bias using visual inspections and asymmetry tests. We explored the impact of publication bias using ORs adjusted for potential publication bias. Twenty-two associations were available. Our visual analysis suggested a publication bias in five out twenty-two funnel plots (23% [95%CI: 8; 45]). The Egger's test showed a significant publication bias in one (HLA-B*15:02 and phenytoin-induced Stevens-Johnson syndrome or toxic epidermal necrolysis, p = 0.03) out of nine (11% [95%CI: 0; 48]) and the Begg's test in one (HLA-B*15:02 and carbamazepine-induced serious cutaneous reactions, p = 0.02) out of ten (10% [95%CI: 0; 45]) assessable funnel plots. Adjusting for publication bias may reduce by half the ORs of the pharmacogenetics associations. Publication bias in the pharmacogenetic of antiseizure drug's AEs is not uncommon and may affect the estimation of the effect of such biomarkers. When conducting pharmacogenetic studies, it is critical to publish also the negative one.

Validating GAN-BioBERT: A Methodology for Assessing Reporting Trends in Clinical Trials

Background

The aim of this study was to validate a three-class sentiment classification model for clinical trial abstracts combining adversarial learning and the BioBERT language processing model as a tool to assess trends in biomedical literature in a clearly reproducible manner. We then assessed the model's performance for this application and compared it to previous models used for this task.

Methods

Using 108 expert-annotated clinical trial abstracts and 2,000 unlabeled abstracts this study develops a three-class sentiment classification algorithm for clinical trial abstracts. The model uses a semi-supervised model based on the Bidirectional Encoder Representation from Transformers (BERT) model, a much more advanced and accurate method compared to previously used models based upon traditional machine learning methods. The prediction performance was compared to those previous studies.

Results

The algorithm was found to have a classification accuracy of 91.3%, with a macro F1-Score of 0.92, significantly outperforming previous studies used to classify sentiment in clinical trial literature, while also making the sentiment classification finer grained with greater reproducibility.

Conclusion

We demonstrate an easily applied sentiment classification model for clinical trial abstracts that significantly outperforms previous models with greater reproducibility and applicability to large-scale study of reporting trends.

The future of Asian Cardiovascular Annals: Goals and quality

The positional statement of the Asian Cardiovascular and Thoracic Annals is presented.

An umbrella review of effect size, bias, and power across meta-analyses in emergency medicine

OBJECTIVES

The objective of this study was to conduct an umbrella review of therapeutic studies relevant to emergency medicine, analyzing patterns in effect size, power, and signals of potential bias across an entire field of clinical research.

METHODS

We combined topic- and journal-driven searches of PubMed and Google Scholar for published articles of systematic reviews and meta-analyses (SRMA) relevant to emergency medicine (last search in November 2020). Data were screened and extracted by six investigators. Redundant meta-analyses were removed. Whenever possible for each comparison we extracted one meta-analysis on mortality with the most events and one meta-analysis on a nonmortality outcome with the most studies. From each meta-analysis we extracted all individual study effects; outcomes were converted to odds ratios (ORs) and placed on a common scale where an OR < 1.0 represents a reduction in a harmful outcome with an experimental treatment versus control. Outcomes were analyzed at the level of individual studies and at the level of summary effects across meta-analyses.

RESULTS

A total of 332 articles contained 431 eligible meta-analyses with a total of 3,129 individual study outcomes; of these, 2,593 (83%) were from randomized controlled trials. The median OR across all studies was 0.70. Within each meta-analysis, the earliest study effect on average demonstrated larger benefit compared to the overall summary effect. Only 57 of 431 meta-analyses (13%) both favored the experimental intervention and did not show any signal of small study effects or excess significance, and of those only 12 had at least one study with 80% or higher power to detect an OR of 0.70. Of these, no interventions significantly decreased mortality in well-powered trials. Although the power of studies increased somewhat over time, the majority of studies were underpowered.

CONCLUSIONS

Few interventions studied within SRMAs relevant to emergency medicine seem to have strong and unbiased evidence for improving outcomes. The field would benefit from more optimally powered trials.

Assessing treatment effects and publication bias across different specialties in medicine: a meta-epidemiological study

OBJECTIVES

To assess the prevalence of statistically significant treatment effects, adverse events and small-study effects (when small studies report more extreme results than large studies) and publication bias (over-reporting of statistically significant results) across medical specialties.

DESIGN

Large meta-epidemiological study of treatment effects from the Cochrane Database of Systematic Reviews.

METHODS

We investigated outcomes from 57 162 studies from 1922 to 2019, and overall 98 966 meta-analyses and 5534 large meta-analyses (≥10 studies). Egger's and Harbord's tests to detect small-study effects, limit meta-analysis and Copas selection models to bias-adjust effect estimates and generalised linear mixed models were used to analyse one of the largest collections of evidence in medicine.

RESULTS

Medical specialties showed differences in the prevalence of statistically significant results of efficacy and safety outcomes. Treatment effects from primary studies published in high ranking journals were more likely to be statistically significant (OR=1.52; 95% CI 1.32 to 1.75) while randomised controlled trials were less likely to report a statistically significant effect (OR=0.90; 95% CI 0.86 to 0.94). Altogether 19% (95% CI 18% to 20%) of the large meta-analyses showed evidence for small-study effects, but only 3.9% (95% CI 3.4% to 4.4%) showed evidence for publication bias after further assessment of funnel plots. Adjusting treatment effects resulted in overall less evidence for efficacy.

CONCLUSIONS

These results suggest that reporting of large treatment effects from small studies may cause greater concern than publication bias. Incentives should be created so that studies of the highest quality become more visible than studies that report more extreme results.

Sex/gender differences in the human autistic brains: A systematic review of 20 years of neuroimaging research

Our current understanding of autism is largely based on clinical experiences and research involving male individuals given the male-predominance in prevalence and the under-inclusion of female individuals due to small samples, co-occurring conditions, or simply being missed for diagnosis. There is a significantly biased 'male lens' in this field with autistic females insufficiently understood. We therefore conducted a systematic review to examine how sex and gender modulate brain structure and function in autistic individuals. Findings from the past 20 years are yet to converge on specific brain regions/networks with consistent sex/gender-modulating effects. Despite at least three well-powered studies identifying specific patterns of significant sex/gender-modulation of autism-control differences, many other studies are likely underpowered, suggesting a critical need for future investigation into sex/gender-based heterogeneity with better-powered designs. Future research should also formally investigate the effects of gender, beyond biological sex, which is mostly absent in the current literature. Understanding the roles of sex and gender in the development of autism is an imperative step to extend beyond the 'male lens' in this field.

Quality Assessment of Published Systematic Reviews in High Impact Cardiology Journals: Revisiting the Evidence Pyramid

Objective: Systematic reviews are increasingly used as sources of evidence in clinical cardiology guidelines. In the present study, we aimed to assess the quality of published systematic reviews in high impact cardiology journals.

Methods: We searched PubMed for systematic reviews published between 2010 and 2019 in five general cardiology journals with the highest impact factor (according to Clarivate Analytics 2019). We extracted data on eligibility criteria, methodological characteristics, bias assessments, and sources of funding. Further, we assessed the quality of retrieved reviews using the AMSTAR tool.

Results: A total of 352 systematic reviews were assessed. The AMSTAR quality score was low or critically low in 71% (95% CI: 65.7–75.4) of the assessed reviews. Sixty-four reviews (18.2%, 95% CI: 14.5–22.6) registered/published their protocol. Only 221 reviews (62.8%, 95% CI: 57.6–67.7) reported adherence to the EQUATOR checklists, 208 reviews (58.4%, 95% CI: 53.9–64.1) assessed the risk of bias in the included studies, and 177 reviews (52.3%, 95% CI: 45.1–55.5) assessed the risk of publication bias in their primary outcome analysis. The primary outcome was statistically significant in 274 (79.6%, 95% CI: 75.1–83.6) and had statistical heterogeneity in 167 (48.5%, 95% CI: 43.3–53.8) reviews. The use and sources of external funding was not disclosed in 87 reviews (24.7%, 95% CI: 20.5–29.5). Data analysis showed that the existence of publication bias was significantly associated with statistical heterogeneity of the primary outcome and that complex design, larger sample size, and higher AMSTAR quality score were associated with higher citation metrics.

Conclusion: Our analysis uncovered widespread gaps in conducting and reporting systematic reviews in cardiology. These findings highlight the importance of rigorous editorial and peer review policies in systematic review publishing, as well as education of the investigators and clinicians on the synthesis and interpretation of evidence.

A systematic review on nomophobia prevalence: Surfacing results and standard guidelines for future research

Background

Mobile phones allow us to stay connected with others and provide us a sense of security. We can work, chat with family and friends, take pictures, buy clothes or books, and even control home appliances. They play such a significant role in our lives that we feel anxious without them. In some cases, the relationship between humans and these communication devices have become problematic. Nomophobia (NMP) is the fear of becoming incommunicable, separated from the mobile phone and losing connection to the Internet. Since this social phobia was coined in the first decade of the XXI century, a growing number of studies have studied it and reported the prevalence of this technology-related problem. However, this research activity has generated mixed results regarding how we assess and report nomophobia and who may be at a higher risk of suffering or developing it.

Methods

We conducted a systematic review of 108 studies published in English and Spanish and collected them in Parsifal. We searched for assessment and prevalence data on nomophobia. Also, we looked at gender and age differences to identify risk factors and see if these differences exist and emerge worldwide.

Results

In this study, we find that women and younger individuals suffer more from nomophobia. The disparity in reporting the prevalence of nomophobia is enormous since the percentages of “at-risk” participants go from 13% to 79%, and participants suffering from it are between 6% and 73%, being the score in the range of 45.5 and 93.82. Within the group of nomophobic people, moderate cases vary between 25.7% and 73.3%, and severe cases, between 1% and 87%. Such disparity is due to differences in assessment criteria. Females and young people seem to be more vulnerable to nomophobia although methodological disparity makes it difficult to reach definitive conclusions. We conclude our review by recommending some common guidelines for guiding future research.

Methodologies for systematic reviews with meta-analysis of randomised clinical trials in pain, anaesthesia, and perioperative medicine

Systematic reviews and meta-analyses (SRMAs) are increasing in popularity, but should they be used to inform clinical decision-making in anaesthesia? We present evidence that the certainty of evidence from SRMAs in anaesthesia (and in general) may be unacceptably low because of risks of bias exaggerating treatment effects, unexplained heterogeneity reducing certainty in estimates, random errors, and widespread prevalence of publication bias. We also present the latest methodological advances to help improve the certainty of evidence from SRMAs. The target audience includes both review authors and practising clinicians to help with SRMA appraisal. Issues discussed include minimising risks of bias from included trials, trial sequential analysis to reduce random error, updated methods for presenting effect estimates, and novel publication bias tests for commonly used outcome measures. These methods can help to reduce spurious conclusions on clinical significance, explain statistical heterogeneity, and reduce false positives when evaluating small-study effects. By reducing concerns in these domains of Grading of Recommendations, Assessment, Development and Evaluation, it should help improve the certainty of evidence from SRMAs used for decision-making in anaesthesia, pain, and perioperative medicine.

Is There an Association Between Authors' Conflicts of Interest and Outcomes in Clinical Studies Involving Autologous Chondrocyte Implantation?

BACKGROUND

Autologous chondrocyte implantation (ACI) is an increasingly popular technique for the treatment of articular cartilage defects. Because several companies have financial interests in ACI, it is important to consider possible conflicts of interest when evaluating studies reporting outcomes of ACI.

PURPOSE

To determine whether there is an association between authors' financial conflicts of interest and the outcomes of ACI studies.

STUDY DESIGN

Cross-sectional study.

METHODS

A search of PubMed and MEDLINE databases for "autologous chondrocyte implantation" was performed. Clinical studies published after 2012 through May 15, 2019, and in English were included. Studies were determined to have financial conflicts of interest if any contributing author had relevant conflicts, either self-reported in the published study's disclosures section or reported online in the American Academy of Orthopaedic Surgeons Disclosure database or the Centers for Medicare & Medicaid Services Open Payments database. The outcomes of each study were rated as favorable, equivocal, or unfavorable based on predefined criteria and then tested for association with conflicts of interest through use of the Fisher exact test.

RESULTS

A total of 79 studies met the inclusion criteria. Nearly all studies were of level 3 or 4 evidence. Conflicts of interest were established in 51.90% of studies (n = 41). Conflicts that were not self-reported by the authors were discovered in 18% of studies. The level of evidence was not associated with conflict of interest. No statistically significant difference was found in the rate of favorable outcomes between studies with conflicts (92.68%) and those with no conflicts (81.58%) (P = .126). Publications by US authors were more likely to have financial conflicts of interest (P = .003).

CONCLUSION

Favorable results were reported in a majority of studies involving ACI. No statistical association was found between the frequency of favorable outcomes and the presence of financial conflicts of interest, country of authorship, or level of evidence. There was a trend toward more favorable outcomes in studies with conflicts of interest. Additionally, nearly 20% of publications had possible conflicts found online that were not self-reported. It is critical for orthopaedic surgeons to judiciously evaluate published studies and consider financial conflicts of interest before performing ACI techniques on patients.

Efficacy and safety of repetitive transcranial magnetic stimulation for peripartum depression: A meta-analysis of randomized controlled trials

This study aims to summarize and systematically review the efficacy and safety of repetitive transcranial magnetic stimulation (rTMS) for women with peripartum depression (PPD). Several databases (Wanfang, CNKI, VIP, CBM, PubMed, Embase, Cochrane Library, PsyINFO, Web of Science, and Clinical trial) were searched from inception until April 12, 2020. In total, ten randomized controlled trials (RCTs) met the eligibility criteria and were included in this systematic review. We calculated the combined effect size (standardized mean difference [SMD] and odds ratio [OR]) for the corresponding effects models. The aggregated result of 10 trials indicated a significant benefit of rTMS on PPD, and the aggregated result of remission showed significantly positive effects of Test group VS. Control group. In terms of treatment adverse effects, the aggregated result showed no statistical significance of headache and dry mouth between the two groups. The results of the meta-analysis suggest that rTMS is an effective and safe intervention for PPD. Owing to poor methodological quality among the included studies, high-quality multicenter RCTs are needed to further verify the effects of this treatment.

Characteristics associated with Publication of Randomized Controlled Trials in the Journal of Cardiothoracic and Vascular Anesthesia: A 15-Year Analysis, 2004-2018

Randomized controlled trials (RCTs) provide important data to guide clinical decisions. Publication bias may limit the applicability of RCTs because many clinical investigators prefer to submit and journals more selectively accept studies with positive results. The authors tested the hypothesis that positive RCTs published in the Journal of Cardiothoracic and Vascular Anesthesia were more likely to be associated with factors known to predict publication of positive versus negative RCTs in other journals. This observational study was an internet analysis of all issues of Journal of Cardiothoracic and Vascular Anesthesia from 2004-2018. Each issue was searched to identify human RCTs. The numbers of centers and enrolled patients in each RCT were tabulated. The corresponding author determined the country of origin (United States v international). A trial was "positive" or "negative" based on rejection or confirmation of the null hypothesis, respectively, for the primary outcome variable or the majority of measured outcomes if a primary outcome was not identified. The presence or absence of a hypothesis, randomization methodology, sample size calculation, and blinded research design was recorded. Registration in a public database, Consolidated Statements of Reporting Trials (CONSORT) guideline compliance, and the source of funding also were determined. The number of citations for each RCT was determined by using Google Scholar; the citation rate was calculated as the ratio of the number of total citations and the duration in years since the trial's original publication. A total of 296 RCTs were identified, of which 58.8% reported positive results. Most RCTs were single center, relatively small, and international in origin. Total citations/RCT decreased over time, but citations/year did not. The percentage of RCTs that identified a randomization method, were registered, or followed CONSORT guidelines increased in a time-dependent manner. No differences in any factors associated with publication of RCTs were observed when positive and negative trials were compared. The Journal of Cardiothoracic and Vascular Anesthesia publishes more positive than negative RCTs, but factors that have been previously associated with RCT publication in other journals were similar between groups.

MedEdTrials: Protocol registration for medical education research

This article was migrated. The article was marked as recommended. Publication bias (the tendency for studies with positive results to be more easily published than studies with null or negative results) and outcome switching (the adjustment of study aims to match results) have long been problematic in medical research. Recent legislation, protocol-registration sites, and agreements by medical journals have led to a reduction of these problems in medical research. In medical education specifically, however, there is no such legislation, registration, or agreement. This paper argues for the creation of such a registration site and agreement by medical education journals as a matter of ethical necessity.

An Evaluation of Publication Bias in High-Impact Orthopaedic Literature

Background:

Statistical analysis of systematic reviews allows the results of previous studies to be combined and synthesized to assess the overall health effect of the intervention in question. Systematic reviews can also be used to guide the creation of clinical practice guidelines and are considered to have a high level of evidence. Thus, it is important that their methodological quality is of the highest standard. Publication bias presents 2 problems: (1) studies with significant results may be overrepresented in systematic reviews and meta-analyses (“false positives”) and (2) studies without significant results may not be included in systematic reviews and meta-analyses (“false negatives”) because each study, on its own, was underpowered, meaning that some treatment options that may have clinical benefit will not be adopted.

Methods:

We performed a study to evaluate the techniques used by authors to report and evaluate publication bias in the top 10 orthopaedic journals as well as 3 orthopaedic-related Cochrane groups. Two authors independently screened the titles and abstracts to identify systematic reviews and meta-analyses. We assessed publication bias in the systematic reviews that did not assess publication bias themselves.

Results:

Our final sample included 694 systematic reviews or meta-analyses that met our inclusion criteria. Our review included 502 studies (72%) that focused on clinical outcomes, with the majority of the remaining studies focused on predictive and prognostic accuracy (20%) or diagnostic accuracy (5%). Publication bias was discussed in 295 (42.5%) of the included studies and was assessed in 135 (19.5%). Of the studies that assessed publication bias, 31.9% demonstrated evidence of publication bias. Only 43% and 22% of studies that involved use of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines discussed and assessed publication bias, respectively.

Conclusions:

Publication bias is infrequently discussed and assessed in the high-impact orthopaedic literature. Furthermore, nearly one-third of the studies that assessed for publication bias demonstrated evidence of publication bias. In addition to these shortcomings, fewer than half of these studies involved use of the PRISMA guidelines and yet only one-fourth of the studies assessed for publication bias.

Clinical Relevance:

By understanding the degree to which publication bias is discussed and presented in high-impact orthopaedic literature, changes can be made by journals and researchers alike to improve the overall quality of research produced and reported.

Propofol vs. inhalational agents to maintain general anaesthesia in ambulatory and in-patient surgery: a systematic review and meta-analysis

Background

It is unclear if anaesthesia maintenance with propofol is advantageous or beneficial over inhalational agents. This study is intended to compare the effects of propofol vs. inhalational agents in maintaining general anaesthesia on patient-relevant outcomes and patient satisfaction.

Methods

Studies were identified by electronic database searches in PubMed™, EMBASE™ and the Cochrane™ library between 01/01/1985 and 01/08/2016. Randomized controlled trials (RCTs) of peer-reviewed journals were studied. Of 6688 studies identified, 229 RCTs were included with a total of 20,991 patients. Quality control, assessment of risk of bias, meta-bias, meta-regression and certainty in evidence were performed according to Cochrane. Common estimates were derived from fixed or random-effects models depending on the presence of heterogeneity. Post-operative nausea and vomiting (PONV) was the primary outcome. Post-operative pain, emergence agitation, time to recovery, hospital length of stay, post-anaesthetic shivering and haemodynamic instability were considered key secondary outcomes.

Results

The risk for PONV was lower with propofol than with inhalational agents (relative risk (RR) 0.61 [0.53, 0.69], p < 0.00001). Additionally, pain score after extubation and time in the post-operative anaesthesia care unit (PACU) were reduced with propofol (mean difference (MD) − 0.51 [− 0.81, − 0.20], p = 0.001; MD − 2.91 min [− 5.47, − 0.35], p = 0.03). In turn, time to respiratory recovery and tracheal extubation were longer with propofol than with inhalational agents (MD 0.82 min [0.20, 1.45], p = 0.01; MD 0.70 min [0.03, 1.38], p = 0.04, respectively). Notably, patient satisfaction, as reported by the number of satisfied patients and scores, was higher with propofol (RR 1.06 [1.01, 1.10], p = 0.02; MD 0.13 [0.00, 0.26], p = 0.05). Secondary analyses supported the primary results.

Conclusions

Based on the present meta-analysis there are several advantages of anaesthesia maintenance with propofol over inhalational agents. While these benefits result in an increased patient satisfaction, the clinical and economic relevance of these findings still need to be addressed in adequately powered prospective clinical trials.

Electronic supplementary material

The online version of this article (10.1186/s12871-018-0632-3) contains supplementary material, which is available to authorized users.

Critical appraisal of methodological quality of Systematic Reviews and Meta-analysis in Paediatric Dentistry journals

OBJECTIVE

To systematically assess the methodological quality of Systematic Reviews (SRs) and Meta-Analyses (MA) published in Paediatric Dentistry journals and to analyse the relationship between the authors, journals, country, review topic, and the year of publication to the methodological quality of SRs and MA.

DESIGN

Paediatric Dentistry journals ranked in the top five of the h5 index of Google Scholar Metrics were selected. SRs with MA were searched independently by two reviewers using PubMed and Scopus databases until December 2017. Methodological quality was assessed using A Measurement Tool to Assess Systematic Reviews (AMSTAR) tool. Statistical significance was set at P < 0.05 and Mann-Whitney U test and Kruskal-Wallis test was employed for comparing the AMSTAR score with the journal characteristics.

RESULTS

Finally, 24 SRs with MA were included. The overall AMSTAR score of SRs and MA published in paediatric dentistry journals was 7.08 ± 2.41. No statistically significant differences were found between the country, journal or focus of study to the quality of SRs except the number of authors and the year of publication (P < 0.05).

CONCLUSIONS

The quality of SRs and MA in leading Paediatric Dentistry journals were evaluated with AMSTAR tool and areas where quality could be improved were identified.

High quality of evidence is uncommon in Cochrane systematic reviews in Anaesthesia, Critical Care and Emergency Medicine

BACKGROUND

The association between the quality of evidence in systematic reviews and authors’ conclusions regarding the effectiveness of interventions relevant to anaesthesia has not been examined.

OBJECTIVE

The objectives of this study were: to determine the proportion of systematic reviews in which the authors made a conclusive statement about the effect of an intervention; to describe the quality of evidence derived from outcomes in reviews that used the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) working group system for grading the quality of evidence; and to identify review characteristics associated with conclusiveness.

DESIGN

Cross-sectional analysis of Cochrane systematic reviews from the Anaesthesia, Critical Care and Emergency Review Group was undertaken.

DATA SOURCES

The Cochrane webpage was used to identify reviews for inclusion (http://.ace.cochrane.org/).

ELIGIBILITY CRITERIA

New and updated versions of systematic reviews published up to 17 September 2015 were eligible. Protocols for systematic reviews were excluded.

RESULTS

A total of 159 reviews were included. GRADE was used in 103 reviews (65%). Of these, high-level evidence for the primary outcome was identified in 11 reviews (10%). The main reasons that quality of evidence for the primary outcome was downgraded were risk of bias (n = 44; 43%) and imprecision (n = 36; 35%). Authors of 47% (n = 75) of the total number of reviews made conclusive statements about the effects of interventions. Independent predictors of conclusiveness in the subgroup of reviews with GRADE assessments were quality of evidence for the primary outcome (odds ratio 2.03; 95% confidence interval: [1.18 to 3.52] and an increasing number of studies included in reviews (OR 1.05; 95% CI: [1.01 to 1.09]).

CONCLUSION

It was common for conclusive statements to be made about the effects of interventions despite evidence for the primary outcome being rated less than high quality. Improving methodological quality of trials would have the greatest impact on improving the quality of evidence.

Is publication bias present in gastroenterological research? An analysis of abstracts presented at an annual congress

Background

Publication bias is the tendency of investigators, reviewers, and editors to submit or accept manuscripts for publication based on their direction or strength of findings. In this study, we investigated if publication bias was present in gastroenterological research by evaluating abstracts at Americas Hepato-Pancreato-Biliary Congresses from 2011 to 2013.

Methods

We searched Google, Google Scholar, and PubMed to locate the published reports of research described in these abstracts. If a publication was not found, a second investigator searched to verify nonpublication. If abstract publication status remained undetermined, authors were contacted regarding reasons for nonpublication. For articles reaching publication, the P value, study design, time to publication, citation count, and journals in which the published report appeared were recorded.

Results

Our study found that of 569 abstracts presented, 297 (52.2%) reported a P value. Of these, 254 (85.5%) contained P values supporting statistical significance. The abstracts reporting a statistically significant outcome were twice as likely to reach publication than abstracts with no significant findings (OR 2.10, 95% CI [1.06–4.14]). Overall, 243 (42.7%) abstracts reached publication. The mean time to publication was 14 months and a median time of nine months.

Conclusion

In conclusion, we found evidence for publication bias in gastroenterological research. Abstracts with significant P values had a higher probability of reaching publication. More than half of abstracts presented from 2011 to 2013 failed to reach publication. Readers should take these findings into consideration when reviewing medical literature.

Tools for assessing risk of reporting biases in studies and syntheses of studies: a systematic review

BACKGROUND

Several scales, checklists and domain-based tools for assessing risk of reporting biases exist, but it is unclear how much they vary in content and guidance. We conducted a systematic review of the content and measurement properties of such tools.

METHODS

We searched for potentially relevant articles in Ovid MEDLINE, Ovid Embase, Ovid PsycINFO and Google Scholar from inception to February 2017. One author screened all titles, abstracts and full text articles, and collected data on tool characteristics.

RESULTS

We identified 18 tools that include an assessment of the risk of reporting bias. Tools varied in regard to the type of reporting bias assessed (eg, bias due to selective publication, bias due to selective non-reporting), and the level of assessment (eg, for the study as a whole, a particular result within a study or a particular synthesis of studies). Various criteria are used across tools to designate a synthesis as being at 'high' risk of bias due to selective publication (eg, evidence of funnel plot asymmetry, use of non-comprehensive searches). However, the relative weight assigned to each criterion in the overall judgement is unclear for most of these tools. Tools for assessing risk of bias due to selective non-reporting guide users to assess a study, or an outcome within a study, as 'high' risk of bias if no results are reported for an outcome. However, assessing the corresponding risk of bias in a synthesis that is missing the non-reported outcomes is outside the scope of most of these tools. Inter-rater agreement estimates were available for five tools.

CONCLUSION

There are several limitations of existing tools for assessing risk of reporting biases, in terms of their scope, guidance for reaching risk of bias judgements and measurement properties. Development and evaluation of a new, comprehensive tool could help overcome present limitations.

Flaws in the application and interpretation of statistical analyses in systematic reviews of therapeutic interventions were common: a cross-sectional analysis

OBJECTIVES

The objective of the study was to investigate the application and interpretation of statistical analyses in a cross-section of systematic reviews (SRs) of therapeutic interventions, without restriction by journal, clinical condition, or specialty.

STUDY DESIGN AND SETTING

We evaluated a random sample of SRs assembled previously, which were indexed in MEDLINE® during February 2014, focused on a treatment or prevention question, and reported at least one meta-analysis. The reported statistical methods used in each SR were extracted from articles and online appendices by one author, with a 20% random sample extracted in duplicate.

RESULTS

We evaluated 110 SRs; 78/110 (71%) were non-Cochrane SRs and 55/110 (50%) investigated a pharmacological intervention. The SRs presented a median of 13 (interquartile range: 5-27) meta-analytic effects. When considering the index (primary or first reported) meta-analysis of each SR, just over half (62/110 [56%]) used the random-effects model, but few (5/62 [8%]) interpreted the meta-analytic effect correctly (as the average of the intervention effects across all studies). A statistical test for funnel plot asymmetry was reported in 17/110 (15%) SRs; however, in only 4/17 (24%) did the test include the recommended number of at least 10 studies of varying size. Subgroup analyses accompanied 42/110 (38%) index meta-analyses, but findings were not interpreted with respect to a test for interaction in 29/42 (69%) cases, and the issue of potential confounding in the subgroup analyses was not raised in any SR.

CONCLUSIONS

There is scope for improvement in the application and interpretation of statistical analyses in SRs of therapeutic interventions. The involvement of statisticians on the SR team and establishment of partnerships between researchers with specialist expertise in SR methods and journal editors may help overcome these shortcomings.

A review of publication bias in the gastroenterology literature

In systematic reviews and meta-analyses, publication bias is particularly problematic, given that combining only statistically significant outcomes is likely to overestimate the true effect of an intervention since non-significant findings have been omitted. We examined practices for evaluating publication bias from gastroenterology literature. We performed a PubMed search to identify systematic reviews published in American Journal of Gastroenterology, Gut, and Gastroenterology from 2005 to 2015. Of the 304 found, 215 studies were eligible for inclusion based on relevant study characteristics. There were 190 systematic reviews which used at least one method to evaluate publication bias and/or included ten or more primary studies. There were 115/190 (60.53%) systematic reviews which used at least one method to evaluate publication bias. Most (105/115, 91.27%) qualified reviews used at least one method to evaluate publication bias and 78/115 (67.83%) used a combination of methods. The most common methods were funnel plot (100/115, 86.96%), Egger's regression (67/115, 58.26%), and Begg's (28/115, 24.35%). Of the 115 reviews that performed evaluations, 26 (22.61%) conducted these analyses with fewer than ten primary studies, and a minority (24/115, 20.87%) reached the conclusion that publication bias was present in their work. While methods to assess publication bias were frequently noted among qualified systematic reviews, these methods are limited in value and could be improved by incorporating approaches that assess the degree of publication bias severity.

Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review

BACKGROUND

The PRISMA Statement is a reporting guideline designed to improve transparency of systematic reviews (SRs) and meta-analyses. Seven extensions to the PRISMA Statement have been published to address the reporting of different types or aspects of SRs, and another eight are in development. We performed a scoping review to map the research that has been conducted to evaluate the uptake and impact of the PRISMA Statement and extensions. We also synthesised studies evaluating how well SRs published after the PRISMA Statement was disseminated adhere to its recommendations.

METHODS

We searched for meta-research studies indexed in MEDLINE® from inception to 31 July 2017, which investigated some component of the PRISMA Statement or extensions (e.g. SR adherence to PRISMA, journal endorsement of PRISMA). One author screened all records and classified the types of evidence available in the studies. We pooled data on SR adherence to individual PRISMA items across all SRs in the included studies and across SRs published after 2009 (the year PRISMA was disseminated).

RESULTS

We included 100 meta-research studies. The most common type of evidence available was data on SR adherence to the PRISMA Statement, which has been evaluated in 57 studies that have assessed 6487 SRs. The pooled results of these studies suggest that reporting of many items in the PRISMA Statement is suboptimal, even in the 2382 SRs published after 2009 (where nine items were adhered to by fewer than 67% of SRs). Few meta-research studies have evaluated the adherence of SRs to the PRISMA extensions or strategies to increase adherence to the PRISMA Statement and extensions.

CONCLUSIONS

Many studies have evaluated how well SRs adhere to the PRISMA Statement, and the pooled result of these suggest that reporting of many items is suboptimal. An update of the PRISMA Statement, along with a toolkit of strategies to help journals endorse and implement the updated guideline, may improve the transparency of SRs.

Methodological and Reporting Quality of Systematic Reviews Published in the Highest Ranking Journals in the Field of Pain

BACKGROUND

Systematic reviews (SRs) are important for making clinical recommendations and guidelines. We analyzed methodological and reporting quality of pain-related SRs published in the top-ranking anesthesiology journals.

METHODS

This was a cross-sectional meta-epidemiological study. SRs published from 2005 to 2015 in the first quartile journals within the Journal Citation Reports category Anesthesiology were analyzed based on the Journal Citation Reports impact factor for year 2014. Each SR was assessed by 2 independent authors using Assessment of Multiple Systematic Reviews (AMSTAR) and Preferred Reporting Items of Systematic reviews and Meta-Analyses (PRISMA) tools. Total score (median and interquartile range, IQR) on checklists, temporal trends in total score, correlation in total scores between the 2 checklists, and variability of those results between journals were analyzed.

RESULTS

A total of 446 SRs were included. Median total score of AMSTAR was 6/11 (IQR: 4-7) and of PRISMA 18.5/27 (IQR: 15-22). High compliance (reported in over 90% SRs) was found in only 1 of 11 AMSTAR and 5 of 27 PRISMA items. Low compliance was found for the majority of AMSTAR and PRISMA individual items. Linear regression indicated that there was no improvement in the methodological and reporting quality of SRs before and after the publication of the 2 checklists (AMSTAR: F(1,8) = 0.22; P = .65, PRISMA: F(1,7) = 0.22; P = .47). Total scores of AMSTAR and PRISMA had positive association (R = 0.71; P < .0001).

CONCLUSIONS

Endorsement of PRISMA in instructions for authors was not a guarantee of compliance. Methodological and reporting quality of pain-related SRs should be improved using relevant checklists. This can be remedied by a joint effort of authors, editors, and peer reviewers.