High variability in results and methodological quality among overlapping systematic reviews on the same topics in surgery: a meta-epidemiological study

Investigate the relationship between the retraction reasons and the quality of methodology in non-Cochrane retracted systematic reviews: a systematic review

BACKGROUND

This systematic review aimed to investigate the relationship between retraction status and the methodology quality in the retracted non-Cochrane systematic review.

METHOD

PubMed, Web of Science, and Scopus databases were searched with keywords including systematic review, meta-analysis, and retraction or retracted as a type of publication until September 2023. There were no time or language restrictions. Non-Cochrane medical systematic review studies that were retracted were included in the present study. The data related to the retraction status of the articles were extracted from the retraction notice and Retraction Watch, and the quality of the methodology was evaluated with the AMSTAR-2 checklist by two independent researchers. Data were analyzed in the Excel 2019 and SPSS 21 software.

RESULT

Of the 282 systematic reviews, the corresponding authors of 208 (73.75%) articles were from China. The average interval between publish and retraction of the article was about 23 months and about half of the non-Cochrane systematic reviews were retracted in the last 4 years. The most common reasons for retractions were fake peer reviews and unreliable data, respectively. Editors and publishers were the most retractors or requestors for retractions. More than 86% of the retracted non-Cochrane SRs were published in journals with an impact factor above two and had a critically low quality. Items 7, 9, and 13 among the critical items of the AMSTAR-2 checklist received the lowest scores.

DISCUSSION AND CONCLUSION

There was a significant relationship between the reasons of retraction and the quality of the methodology (P-value < 0.05). Plagiarism software and using the Cope guidelines may decrease the time of retraction. In some countries, strict rules for promoting researchers increase the risk of misconduct. To avoid scientific errors and improve the quality of systematic reviews/meta-analyses (SRs/MAs), it is better to create protocol registration and retraction guidelines in each journal for SRs/MAs.

The sound and surprise: overlapping meta-analyses on the topic of safety and efficacy of PD-1 and PD-L1 inhibitors in the treatment of non-small cell lung cancer

PURPOSE

To analyze the characteristics of overlapping meta-analyses based on randomized controlled trials (RCTs) which reported PD-1/PD-L1 inhibitors in non-small cell cancer (NSCLC).

METHODS

Meta-analyses were identified from English and Chinese databases until January 1, 2022. Differences in characteristics of overlapping meta-analyses that conducted in China and other countries were compared to assess their publication propensity. The corrected covered area (CCA) and coverage of relevant RCTs were analyzed for subtopics according to detailed intervention types. The waste and redundancy of evidence were assessed in the case of PD-1/PD-L1 inhibitor monotherapy for second-line treatment for NSCLC.

RESULTS

Fifty-nine meta-analyses published in English and 17 meta-analyses published in Chinese reporting 26 RCTs were identified. Fifty-three (69.74%) meta-analyses were conducted in China. The overlapping meta-analyses in China were more likely to be from hospitals, supported by government funding, integrate first and second-line therapies. Five of the six subtopics had overlapping meta-analyses according to specific types of interventions. The CCA of overlapping meta-analyses ranged from 33.33 to 63.19%, and the coverage of relevant RCTs ranged from 63.64 to 100%. All the conclusions of overlapping meta-analyses have been consistent in the subtopic of PD-1/PD-L1 inhibitor monotherapy for second-line treatment since 2017.

CONCLUSION

Overlapping meta-analyses of PD-1/PD-L1 inhibitors in NSCLC hints that meta-analyses under this topic probably exist serious redundancy. Future research should focus on prospective registration of protocols for systematic reviews/meta-analyses, scientific designed PICO, and cumulative meta-analysis to reduce redundant and wasted studies. Journals should strengthen the requirement for reviewing previously published evidence in manuscript review.

Replication of systematic reviews: is it to the benefit or detriment of methodological quality?

OBJECTIVES

To perform an overview of the overlap of systematic reviews (SRs) assessing direct oral anticoagulants and characterize these reviews in terms of bias and methodological quality (PROSPERO: CRD42022316273).

STUDY DESIGN AND SETTING

A PubMed-indexed search was performed from inception to January 31, 2022 to identify SRs evaluating direct oral anticoagulants in patients treated for an acute venous thromboembolism. The risk of bias of these SRs was assessed according to the Risk Of Bias In Systematic reviews tool. Redundancy was defined as overlap in terms of the type of population considered, the interventions compared, and the studies included.

RESULTS

A total of 144 SRs were evaluated, of which 26 (18.1%) were classified as original, 87 (60.4%) as conceptual replications, and 31 (21.5%) as excessive replications. The risk of bias was high in 19 (73.1%) of the original SRs, 65 (74.7%) of the conceptual replications, and 21 (67.7%) of the excessive replications. Compared to the original SRs, the overall methodological quality was not improved in either conceptual or excessive replications.

CONCLUSION

A large number of SRs was classified as replications; a fifth constituted excessive replications. The replications showed no improvement in overall methodological quality compared to the original SRs.

Kenward-Roger-type corrections for inference methods of network meta-analysis and meta-regression

Network meta-analysis has been an essential methodology of systematic reviews for comparative effectiveness research. The restricted maximum likelihood (REML) method is one of the current standard inference methods for multivariate, contrast-based meta-analysis models, but recent studies have revealed the resultant confidence intervals of average treatment effect parameters in random-effects models can seriously underestimate statistical errors; that is, the actual coverage probability of a true parameter cannot retain the nominal level (e.g., 95%). In this article, we provided improved inference methods for the network meta-analysis and meta-regression models using higher-order asymptotic approximations based on the approach of Kenward and Roger (Biometrics 1997;53:983-997). We provided two corrected covariance matrix estimators for the REML estimator and improved approximations for its sample distribution using a t-distribution with adequate degrees of freedom. All of the proposed procedures can be implemented using only simple matrix calculations. In simulation studies under various settings, the REML-based Wald-type confidence intervals seriously underestimated the statistical errors, especially in cases of small numbers of trials meta-analyzed. By contrast, the proposed Kenward-Roger-type inference methods consistently showed accurate coverage properties under all the settings considered in our experiments. We also illustrated the effectiveness of the proposed methods through applications to two real network meta-analysis datasets.

Guidelines, guidelines and more guidelines for haemorrhoid treatment: A review to sort the wheat from the chaff

AIM

Guidelines benefit patients and clinicians by distilling evidence into easy-to-read recommendations. The literature around the management of haemorrhoids is immense and guidelines are invaluable to improve treatment integrity and patient outcomes. We identified current haemorrhoid guidelines and assessed them for quality and consistency.

METHODS

A systematic search of the literature from January 2011 to October 2021 was carried out. Guidelines identified were assessed for quality using the AGREE II instrument and for consistency in terms of tabulated treatment recommendations.

RESULTS

During this period nine guidelines were identified worldwide. The general quality was poor with only one guideline considered of high enough quality for use. In general, expert selection criteria for guideline development groups were vaguely defined. There were inconsistencies in the interpretation of the published evidence leading to variation in treatment recommendations.

DISCUSSION

Fewer, higher quality guidelines, with more consistent results, are needed. Particular attention should be given to defining the selection of experts involved.