Methodological quality and redundancy of systematic reviews that compare endarterectomy versus stenting for carotid stenosis

Replication of systematic reviews: is it to the benefit or detriment of methodological quality?

OBJECTIVES

To perform an overview of the overlap of systematic reviews (SRs) assessing direct oral anticoagulants and characterize these reviews in terms of bias and methodological quality (PROSPERO: CRD42022316273).

STUDY DESIGN AND SETTING

A PubMed-indexed search was performed from inception to January 31, 2022 to identify SRs evaluating direct oral anticoagulants in patients treated for an acute venous thromboembolism. The risk of bias of these SRs was assessed according to the Risk Of Bias In Systematic reviews tool. Redundancy was defined as overlap in terms of the type of population considered, the interventions compared, and the studies included.

RESULTS

A total of 144 SRs were evaluated, of which 26 (18.1%) were classified as original, 87 (60.4%) as conceptual replications, and 31 (21.5%) as excessive replications. The risk of bias was high in 19 (73.1%) of the original SRs, 65 (74.7%) of the conceptual replications, and 21 (67.7%) of the excessive replications. Compared to the original SRs, the overall methodological quality was not improved in either conceptual or excessive replications.

CONCLUSION

A large number of SRs was classified as replications; a fifth constituted excessive replications. The replications showed no improvement in overall methodological quality compared to the original SRs.

AMSTAR 2 appraisal of systematic reviews and meta-analyses in the field of heart failure from high-impact journals

BACKGROUND

The Measurement Tool to Assess systematic Reviews (AMSTAR) 2 is a critical appraisal tool for systematic reviews (SRs) and meta-analyses (MAs) of interventions. We aimed to perform the first AMSTAR 2-based quality assessment of heart failure-related studies.

METHODS

Eleven high-impact journals were searched from 2009 to 2019. The included studies were assessed on the basis of 16 domains. Seven domains were deemed critical for high-quality studies. On the basis of the performance in these 16 domains with different weights, overall ratings were generated, and the quality was determined to be "high," "moderate," "low," or "critically low."

RESULTS

Eighty-one heart failure-related SRs with MAs were included. Overall, 79 studies were of "critically low quality" and two were of "low quality." These findings were attributed to insufficiency in the following critical domains: a priori protocols (compliance rate, 5%), complete list of exclusions with justification (5%), risk of bias assessment (69%), meta-analysis methodology (78%), and investigation of publication bias (60%).

CONCLUSIONS

The low ratings for these potential high-quality heart failure-related SRs and MAs challenge the discrimination capacity of AMSTAR 2. In addition to identifying certain areas of insufficiency, these findings indicate the need to justify or modify AMSTAR 2's rating rules.

Efficacy and safety of therapeutic use of cannabis derivatives and their synthetic analogs: Overview of systematic reviews

The debate on the use of cannabinoids for therapeutic purposes is constantly on the rise. This overview aimed to map the evidence on the therapeutic effects of cannabis derivatives and their synthetic analogs. Systematic reviews (SRs) of randomized trials were identified through a comprehensive search in several databases, and their methodological quality were evaluated with AMSTAR-2. The results for main outcomes are presented, prioritizing those from updated and better quality SRs. Finally, 68 SRs, addressing 37 different health conditions, were included. The methodological quality was high for eight SRs. The evidence certainty (GRADE) for the effects of cannabinoids is not high for any of the outcomes identified. Evidence certainty was moderate for the following: (a) cannabidiol appears to be beneficial for quality of life but increases the risk of adverse events in ulcerative colitis; (b) cannabinoids in general appear to have no clinically important benefit for chronic non-oncologic pain, spasticity-related pain in multiple sclerosis, or for acute post-operative pain; (c) cannabinoids in general appear to have a benefit in reducing chemotherapy-related nausea and vomiting. For all other outcomes from remaining comparisons, the evidence certainty was low, very low, or not evaluated, which prevents recommendations for or against their routine use.

Redundant meta-analyses are common in genetic epidemiology

OBJECTIVES

The massive growth in the publication of meta-analyses may cause redundancy and wasted efforts. We performed a metaepidemiologic study to evaluate the extent of potential redundancy in published meta-analyses in genetic epidemiology.

STUDY DESIGN AND SETTING

Using a sample of 38 index meta-analyses of genetic associations published in 2010, we retrieved additional meta-analyses that evaluated identical associations (same genetic variant and phenotype) using the Human Genome Epidemiology (HuGE) Navigator and PubMed databases. We analyzed the frequency of potential duplication and examined whether subsequent meta-analyses cited previous meta-analyses on the exact same association.

RESULTS

Based on 38 index meta-analyses, we retrieved a total of 99 duplicate meta-analyses. Only 12 (32%) of the index meta-analyses were unambiguously unique. We found a mean of 2.6 duplicates and a median of 2 duplicates per meta-analysis. In case studies, only 29-54% of previously published meta-analyses were cited by subsequent ones.

CONCLUSION

These results suggest that duplication is common in meta-analyses of genetic associations.

Outcome differences between carotid artery stenting (CAS) and carotid endarterectomy (CEA) in postoperative ventricular arrhythmia, neurological complications, and in-hospital mortality

Objective: We study whether the carotid artery stenting (CAS) and carotid endarterectomy (CEA) differ from each other in postoperative ventricular arrhythmia, along with neurological complications (perioperative stroke and transient ischemic attack), in-hospital mortality, and estimated medical cost. Methods: This study used data of patients with carotid artery stenosis from the National Inpatient Sample (NIS) database (2011-2014) from the United States of America. Based on the procedure that patients received, individuals were categorized into groups of CAS and CEA. Multilevel analyses were conducted to examine the difference in the following outcomes: postoperative ventricular arrhythmia, neurological complications, in-hospital mortality, and medical costs between CAS and CEA. The patient age, gender, race, Charlson Comorbidity Index, primary payer, emergency department service record, bed size of hospital, region of the hospital, and location of the hospital were adjusted in each model. In addition, preexisting cardiovascular diseases (CVDs) were adjusted for when predicting postoperative ventricular arrhythmia; postoperative CVDs were adjusted for in the model of in-hospital mortality. Results: A total of 127,321 carotid artery stenosis hospitalizations were included in our analyses (n = 17,074 in CAS, n = 110,247 in CEA). Multivariate logistic regressions showed that compared with patients underwent CAS, those with CEA had a lower odds of postoperative ventricular arrhythmia (odds ratio [OR] = 0.81, 95% confidence interval [CI]: [0.66-0.98]), less neurological complications (OR = 0.55, 95% CI: [0.51-0.59] in general; OR = 0.63, 95% CI: [0.57-0.69] in ischemic stroke; OR = 0.26, 95% CI: [0.20-0.32] in hemorrhagic stroke; and OR = 0.58, 95% CI: [0.47-0.71] in transient ischemic attack), and in-hospital mortality (OR = 0.52, 95% CI: [0.42-0.64]). Generalized linear model indicated patients undergoing CEA had lower medical cost (β = -4329.99, 95% CI: [-4552.61, -4107.38]) than patients undergoing CAS. Conclusions: In short-term outcomes, CEA was associated with a lower risk of postoperative ventricular arrhythmia, neurological complications, in-hospital mortality, and lower cost as compared with CAS.

The quality of systematic reviews/meta-analyses published in the field of bariatrics: A cross-sectional systematic survey using AMSTAR 2 and ROBIS

High-quality systematic reviews (SR) and meta-analyses (MA) are considered to be reliable sources of information. This study aims to assess the quality of studies published as SR or MA in the field of bariatrics in 2016 and 2017. We identified SR and MA in the field of bariatrics by searching electronic databases (MEDLINE, Embase, and Cochrane Database of Systematic Reviews). Eligible studies were those identified as SR/MA in the title/abstract, which aimed to assess any outcome in patients with morbid obesity undergoing or scheduled to undergo bariatric surgery. Two authors independently reviewed all titles and abstracts, assessed full texts of potentially eligible studies, and assessed the quality of included studies. Any discrepancies were resolved by the third reviewer. We evaluated the quality and risk of bias of each SR/MA using AMSTAR 2 checklist and ROBIS tool, respectively. Seventy-eight of 4236 references met inclusion criteria and were assessed for their quality/risk of bias. The methodological quality of 99% of all papers was classified as "critically low." A total of 6% of the studies were at low risk of bias, and 78% were assessed as being at high risk of bias. The methodological quality of studies published in 2016 and 2017 as SR/MA is highly unsatisfactory.