In Cochrane reviews, risk of bias assessments for allocation concealment were frequently not in line with Cochrane's Handbook guidance

Effect of traditional Chinese exercises on the physical and mental health of stroke patients: a meta-analysis

Background

This meta-analysis aimed to investigate the effects of traditional Chinese exercises on stroke and explore the dose-response relationship between the type of exercise and exercise duration with respect to physical and mental health.

Methods

We searched PubMed, Web of Science, EBSCO, China National Knowledge Infrastructure (CNKI), Wan Fang Database, and China Science and Technology Journal Database to identify published randomized controlled trials (RCTs) related to stroke patients and traditional Chinese exercise that met the evaluation criteria, from the inception of the database until September 2022. After assessing the quality of the literature, we used RevMan5.4 for data analysis.

Results

A total of 27 RCTs were included. The results of the meta-analysis revealed that motor function [MD = 4.79, 95% CI: (4.14, 5.43), p < 0.001], balance function [MD = 5.25, 95% CI: (3.93, 6.56), p < 0.001], and mental health [MD = -2.42, 95% CI: (-3.92 to -0.92), p = 0.002] were significantly better in the experimental group than in the control group.

Conclusion

Traditional Chinese exercises have a positive effect on the physical and mental health of stroke patients, with the most significant benefit observed in balance function. While traditional Chinese exercises improve exercise capacity and mental health in stroke patients, these improvements are not directly correlated with longer practice time, and there appear to be certain limitations regarding duration.

Remodeling and Regenerative Properties of Fully Absorbable Meshes for Abdominal Wall Defect Repair: A Systematic Review and Meta-Analysis of Animal Studies

Fully absorbable meshes can repair abdominal wall defects and effectively reduce the incidence of complications, but different types of fully absorbable meshes have different remodeling and regeneration effects. In order to investigate and compare the effects of different fully absorbable meshes on remodeling and regeneration in animals and reduce the biological risk of clinical translation, SYRCLE was adopted to evaluate the methodological quality of the included studies, and GRADE and ConQual were used to evaluate the quality of evidence. According to the inclusion and exclusion criteria, a total of 22 studies related to fully absorbable meshes were included in this systematic review. These results showed that fiber-based synthetic materials and fiber-based natural materials exhibited better restorative and regenerative effects indicated by infiltration and neovascularization, when compared with a porcine acellular dermal matrix. In addition, the human acellular dermal matrix was found to have a similar regenerative effect on the host extracellular matrix and scaffold degradation compared to the porcine acellular dermal matrix, porcine intestinal submucosa, and fiber-based natural materials, but it offered higher tensile strength than the other three. The quality of the evidence in this field was found to be poor. The reasons for downgrading were analyzed, and recommendations for future research included more rigor in study design, more transparency in result reporting, more standardization of animal models and follow-up time for better evaluation of the remodeling and regenerative performance of abdominal wall hernia repair meshes, and less biological risk in clinical translation.

Frequency of use and adequacy of Cochrane risk of bias tool 2 in non-Cochrane systematic reviews published in 2020: Meta-research study

Risk of bias (RoB) assessment is essential to the systematic review methodology. The new version of the Cochrane RoB tool for randomized trials (RoB 2) was published in 2019 to address limitations identified since the first version of the tool was published in 2008 and to increase the reliability of assessments. This study analyzed the frequency of usage of the RoB 2 and the adequacy of reporting the RoB 2 assessments in non-Cochrane reviews published in 2020. This meta-research study included non-Cochrane systematic reviews of interventions published in 2020. For the reviews that used the RoB 2 tool, we analyzed the reporting of the RoB 2 assessment. Among 3880 included reviews, the Cochrane RoB 1 tool was the most frequently used (N = 2228; 57.4%), followed by the Cochrane RoB 2 tool (N = 267; 6.9%). From 267 reviews that reported using the RoB 2 tool, 213 (79.8%) actually used it. In 26 (12.2%) reviews, erroneous statements were used to indicate the RoB 2 assessment. Only 20 (9.4%) reviews presented a complete RoB 2 assessment with a detailed table of answers to all signaling questions. The judgment of risk of bias by the RoB 2 tool was not justified by a comment in 158 (74.2%) reviews. Only in 33 (14.5%) of reviews the judgment in all domains was justified in the accompanying comment. In most reviews (81.7%), the RoB was inadequately assessed at the study level. In conclusion, the majority of non-Cochrane reviews published in 2020 still used the Cochrane RoB 1 tool. Many reviews used the RoB 2 tool inadequately. Further studies about the uptake and the use of the RoB 2 tool are needed.

Author instructions in biomedical journals infrequently address systematic review reporting and methodology: a cross-sectional study

OBJECTIVES

We aimed to analyze how instructions for authors in journals indexed in MEDLINE address systematic review (SR) reporting and methodology.

STUDY DESIGN AND SETTING

We analyzed instructions for authors in 20% of MEDLINE-indexed journals listed in the online catalog of the National Library of Medicine on July 27, 2021. We extracted data only from the instructions published in English. We extracted data on the existence of instructions for reporting and methodology of SRs.

RESULTS

Instructions from 1,237 journals mentioned SRs in 45% (n = 560) of the cases. Systematic review (SR) registration was mentioned in 104/1,237 (8%) of instructions. Guidelines for reporting SR protocols were found in 155/1,237 (13%) of instructions. Guidelines for reporting SRs were explicitly mentioned in 461/1,237 (37%), whereas the EQUATOR (Enhancing the Quality and Transparency of Health Research) network was referred to in 474/1,237 (38%) of instructions. Less than 2% (n = 20) of instructions mentioned risk of bias and meta-analyses; less than 1% mentioned certainty of evidence assessment, methodological expectations, updating of SRs, overviews of SRs, or scoping reviews.

CONCLUSION

Journals indexed in MEDLINE rarely provide instructions for authors regarding SR reporting and methodology. Such instructions could potentially raise authors' awareness and improve how SRs are prepared and reported.

Anti-tumour necrosis factor-induced skin rashes in inflammatory bowel disease: a systematic review and evidence-based management algorithm

BACKGROUND

Anti-tumour necrosis factor alpha (anti-TNF) agents are a highly effective treatment for inflammatory bowel disease (IBD). Skin lesions, including psoriasiform, eczematous and lupoid eruptions, may paradoxically result from anti-TNF use and cause significant morbidity leading to discontinuation of therapy. There are no consensus guidelines on the management of these lesions.

AIMS

This systematic review considers the existing evidence regarding cutaneous complications of anti-TNF therapy in IBD and the development of an algorithm for management.

METHODS

A systematic review was performed by searching Medline (Pubmed) and Embase for articles published from inception to January 2021. The following search terms were used 'anti-tumour necrosis factor alpha', 'infliximab', 'adalimumab', 'certolizumab', 'golimumab', 'inflammatory bowel disease', 'Crohn disease', 'Ulcerative colitis', 'psoriasis', 'psoriasiform', 'dermatitis', 'lupus', 'skin lesion' and 'skin rash'. Reference lists of relevant studies were reviewed to identify additional suitable studies.

RESULTS

Thirty-four studies were included in the review. Eczema can generally be managed with topical agents and the anti-TNF can be continued, while the development of lupus requires immediate cessation of the anti-TNF and consideration of alternative immunomodulators. Management of psoriasis and psoriasiform lesions may follow a step-wise algorithm where topical treatments will be trialled in less severe cases, with recourse to an alternative anti-TNF or a switch to an alternative class of biological agent.

CONCLUSION

Assessment of anti-TNF skin lesions should be performed in conjunction with a dermatologist and rheumatologist in complex cases. High-quality prospective studies are needed to clarify the validity of these algorithms in the future.

Application of enhanced recovery after surgery in perioperative patients undergoing laparoscopic surgery for gastric cancer: A meta-analysis

BACKGROUND

To analyze the effect of enhanced recovery after surgery (ERAS) in perioperative patients undergoing laparoscopic surgery for gastric cancer (GC).

METHODS

We searched the literature databases of PubMed (https://pubmed.ncbi.nlm.nih.gov/) and Web of Science (https://www.webofscience.com/) for studies related to ERAS and laparoscopic surgery for GC from their inception till October 2022, and the retrieved articles were further screened for analysis. The literature quality was evaluated based on the Cochrane risk of bias tool. The endpoints included the first postoperative exhaust time, first postoperative bowel movement, length of stay, complications and hospitalization expenses. The meta-analysis was performed using RevMan 5.4 software.

RESULTS

11 studies were included, involving 2039 patients. Meta-analysis showed that the first postoperative exhaust and bowel movement were earlier and the length of stay was shorter in the experimental group (e.g.,) than in the control group (CG) (P < .05). Moreover, a lower incidence of complications and markedly reduced hospitalization expenses were determined in for example (P < .05). Due to the heterogeneity in the comparison of indicators such as the first postoperative exhaust and bowel movement across articles, a funnel plot was drawn for observations. The plot was found to be basically symmetrical, indicating small results bias and reliable reference significance of our findings.

CONCLUSION

ERAS for perioperative patients undergoing laparoscopic surgery for GC can effectively shorten the postoperative rehabilitation cycle of patients, improve surgical safety and reduce treatment costs.

Effect of omentopexy/gastropexy on gastrointestinal symptoms after laparoscopic sleeve gastrectomy: A meta-analysis of randomized controlled trials and systematic review

INTRODUCTION

The increased prevalence of obesity worldwide and low incidence of postoperative complications make the laparoscopic sleeve gastrectomy (LSG) a clear public choice for obese-related individuals. Pre-existing studies reported contentious outcomes regarding the association with gastrointestinal symptoms after adding omentopexy (Ome) or gastropexy (Gas) to LSG. The present meta-analysis attempted to evaluate the pros and cons of operating Ome/Gas after LSG concerning gastrointestinal symptoms.

METHODS

The data extraction and study quality assessment were independently performed by two individuals. The PubMed, EMBASE, Scopus, and Cochrane Library databases were systematically searched up to October 1, 2022, using the keywords LSG, omentopexy, and gastropexy to identify randomized controlled trial studies.

RESULTS

Of the original 157 records, 13 studies with 3515 patients were included. LSG with Ome/Gas excels the LSG group in nausea (odds ratio [OR] = 0.57; 95% CI[0.46, 0.70]; P < .00001), reflux (OR = 0.57; 95% CI [0.46, 0.70]; P < .00001), vomiting (OR = 0.41; 95% CI [0.25, 0.67]; P = .0004) on gastrointestinal symptoms and bleeding (OR = 0.36; 95% CI [0.22, 0.59]; P < .0001), leakage (OR = 0.19; 95% CI [0.09, 0.43]; P < .0001), gastric torsion (OR = 0.23; 95% CI [0.07, 0.75]; P = .01) on post-LSG complications. Further, LSG with Ome/Gas was superior to LSG regarding the result of excess body mass index loss in 1 year after surgery (mean difference = 1.83; 95% CI [0.59, 3.07]; P = .004). However, no significant associations were shown between groups in wound infection and the resulting weight or body mass index 1 year after surgery. Of note, subgroup analysis indicated that gastroesophageal reflux disease can be alleviated by adding Ome/Gas post-LSG in those who used small bougies from 32 to 36 Fr (OR = 0.24; 95% CI [0.17, 0.34]; P < .00001) in contrast with large bougies over 36 Fr.

CONCLUSION

Most results elucidated the impact of adding Ome/Gas after LSG in reducing the incidence of gastrointestinal symptoms. Additionally, more studies should be conducted to find the relations between other indicators in the present analysis due to the poor cases.

Adherence of systematic reviews to Cochrane RoB2 guidance was frequently poor: a meta epidemiological study

OBJECTIVES

To assess whether the use of the revised Cochrane risk of bias tool for randomized trials (RoB2) in systematic reviews (SRs) adheres to RoB2 guidance.

METHODS

We searched MEDLINE, Embase, Cochrane Library from 2019 to May 2021 to identify SRs using RoB2. We analyzed methods and results sections to see whether risk of bias was assessed at outcome measure level and applied to primary outcomes of the SR as per RoB2 guidance. The relation between SR characteristics and adequacy of RoB2 use was examined by logistic regression analysis.

RESULTS

Two hundred-eight SRs were included. We could assess adherence in 137 SRs as 12 declared using RoB2 but actually used RoB1 and 59 did not report the number of primary outcomes. The tool usage was adherent in 69.3% SRs. Considering SRs with multiple primary outcomes, adherence dropped to 28.8%. We found a positive association between RoB2 guidance adherence and the methodological quality of the reviews assessed by AMSTAR2 (p-for-trend 0.007). Multivariable regression analysis suggested journal impact factor [first quartile vs. other quartiles] was associated with RoB2 adherence (OR 0.34; 95% CI: 0.16-0.72).

CONCLUSIONS

Many SRs did not adhere to RoB2 guidance as they applied the tool at the study level rather than at the outcome measure level. Lack of adherence was more likely among low and very low quality reviews.

Restrictions and their reporting in systematic reviews of effectiveness: an observational study

BACKGROUND

Restrictions in systematic reviews (SRs) can lead to bias and may affect conclusions. Therefore, it is important to report whether and which restrictions were used. This study aims to examine the use of restrictions regarding language, publication period, and study type, as well as the transparency of reporting in SRs of effectiveness.

METHODS

A retrospective observational study was conducted with a random sample of 535 SRs of effectiveness indexed in PubMed between 2000 and 2019. The use of restrictions and their reporting were analysed using descriptive statistics.

RESULTS

Of the total 535 SRs included, four out of every ten (41.3%) lacked information on at least one of the three restrictions considered (language, publication period, or study type). Overall, 14.6% of SRs did not provide information on restrictions regarding publication period, 19.1% regarding study type, and 18.3% regarding language. Of all included SRs, language was restricted in 46.4%, and in more than half of the SRs with restricted language (130/248), it was unclear whether the restriction was applied during either the search or the screening process, or both. The restrictions were justified for publication period in 22.2% of the respective SRs (33/149), study type in 6.5% (28/433), and language in 3.2% (8/248). Differences in reporting were found between countries as well as between Cochrane and non-Cochrane reviews.

CONCLUSIONS

This study suggests that there is a lack of transparency in reporting on restrictions in SRs. Authors as well as editors and reviewers should be encouraged to improve the reporting and justification of restrictions to increase the transparency of SRs.

Effects of Periodontal Treatment on Levels of Proinflammatory Cytokines in Patients with Chronic Periodontitis: A Meta-Analysis

Background

During the progression of chronic periodontitis (CP), changes in the levels of inflammatory factors are detected in serum and gingival sulcus fluid (GCF). The aim of this meta-analysis was to systematically evaluate the effect of periodontal treatment on GCF and serum proinflammatory cytokines (IL-6, TNF-α, and IL-8) in patients with CP.

Methods

Literature searches were performed through PubMed, Web of Science, Embase, China National Knowledge Infrastructure (CNKI), and Wanfang Database. Randomized controlled trials comparing cytokine levels in periodontal treatment (experimental group) and control group between 2015 and 2020 were included.

Results

There were a total of 13 studies included with 1220 patients. There were 630 cases in the experimental group and 590 cases in the control group. The meta-analysis showed that IL-6 levels in the GCF (SMD = −2.88, 95% CI (-3.68, -2.09), P < 0.001) and serum (SMD = −1.27, 95% CI (-1.72, -0.81), P < 0.001) were significantly lower in the experimental group compared with those before treatment. In addition, IL-8 levels in the GCF (SMD = −2.08, 95% CI (-3.40, -0.76), P < 0.001) and serum (SMD = −1.73, 95% CI (-2.76, -0.70), P < 0.001) were decreased after periodontal treatment, but more than that, a decrease was observed in TNF-α levels of GCF (SMD = −3.98, 95% CI (-5.23, -2.73), P < 0.001) and serum (SMD = −1.80, 95% CI (-3.16, -0.45), P < 0.001) after treatment.

Conclusion

After periodontal therapy, the proinflammatory cytokines in the GCF and serum of patients with CP were significantly decreased compared with those before treatment, and the efficacy was remarkable.

Comparing the efficacy of catheter ablation strategies for persistent atrial fibrillation: a Bayesian analysis of randomized controlled trials

BACKGROUND

Catheter ablation has been recommended as the first-line treatment option for selected patients with atrial fibrillation (AF). However, a widely accepted ablation strategy for persistent AF (perAF) has not yet been established. The benefits of ablation strategies are not conclusive for perAF. There is an urgent need to systematically analyze the results of previous studies and rank these treatment strategies to guide clinical practice.

METHODS

Randomized controlled trials (RCTs) on ablation for perAF were included. The primary outcome was recurrence of atrial tachyarrhythmia (AT) after a single ablation procedure. A Bayesian random-effects network meta-analysis model was fitted.

RESULTS

Twenty-three studies were included in the analysis. A total of 3394 patients and 22 ablation strategies were found in the involved studies. The ablation strategy of pulmonary vein isolation (PVI) + electrical box isolation of the left atrial posterior wall (PBOX) + non-PV trigger ablation (NPV) showed the best treatment effect in terms of the primary outcome. The individualized ablation strategies of mapping and ablation combined with PVI, such as PVI + rotors, PVI + dispersion areas, and PVI + low voltage zone (LVZ) also showed a better ablation effect in perAF.

CONCLUSIONS

PVI ablation is a widely used strategy in perAF and is recognized as a cornerstone procedure for perAF. The PVI + PBOX + NPV strategy showed the highest rank in our analysis. Mapping and ablation strategies that could provide individualized substrate modification also showed a better rank in our analysis and are believed to be a promising direction for the treatment of perAF.

Reevaluating the Use of Sound Therapy for Tinnitus Management: Perspectives on Relevant Systematic Reviews

PURPOSE

Tinnitus is a highly prevalent condition that can severely reduce health functioning. In spite of extant clinical practice guidelines (CPGs), implementation of these CPGs is relatively uncommon. As a result, patients seeking professional services for tinnitus often have no assurance of receiving evidence-based care. The purpose of this tutorial was to clarify the evidence for sound therapy so that it may be included in future CPGs for tinnitus.

METHOD

"Best clinical evidence" is obtained from high-quality systematic reviews, which are generally considered the highest level of evidence. Our review of recent, comprehensive, high-quality systematic reviews of interventions for tinnitus concludes that cognitive behavioral therapy is the only effective intervention, though the strength of evidence was generally rated as low in these reviews. Although trials of sound therapy for tinnitus have been included in these reviews, they have been rated as having high risk of bias (RoB) and not included in syntheses or rated as insufficient strength of evidence.

RESULTS

Conclusions from these and other reviews have influenced recommendations made in CPGs for tinnitus. These conclusions, however, can make it appear that an intervention for tinnitus is not effective, even if the opposite is true. We contend that the strict inclusion criteria for these reviews are counterproductive and have the effect of obscuring decades of evidence demonstrating the clinical effectiveness of sound therapies for tinnitus. Ultimately, this process has resulted in many patients not receiving sound therapy, despite what should be sufficient evidence that this is an effective form of intervention.

CONCLUSION

If we rely on systematic reviews using contemporary RoB assessment criteria for studies published prior to these reporting guidelines, then we risk excluding important conclusions regarding interventions that could help patients in need.

Role of endoscopic ultrasound-guided liver biopsy: a meta-analysis

BACKGROUND

Endoscopic ultrasound-guided liver biopsy (EUS-LB) is an evolving technique. In this meta-analysis, we aimed to evaluate the value of EUS-LB for parenchymal and focal liver lesions. Besides, we aimed to assess the influences of needle-related factors on the performance of EUS-LB. Additionally, we aimed to assess the influence of various criteria on specimen adequacy.

METHODS

We searched the PubMed, Embase, Cochrane Library databases up to 10 October 2021. The primary outcome was diagnostic yield, specimen adequacy, qualified specimens evaluated by rapid on-site evaluation (ROSE). The secondary outcome was adverse events. Subgroup analyses were based on needle type, needle size, fine-needle biopsy (FNB) needle type. A sensitivity analysis was conducted on specimen adequacy based on two definition criteria.

RESULTS

In total, 33 studies were included. Pooled rates of diagnostic yield, specimen adequacy, qualified specimen by ROSE, adverse events were 95%, 84%, 93%, 3%. Subgroup analyses showed that Acquire needles generated higher diagnostic yield than SharkCore needles (99% vs. 88%, p = .047). Additionally, FNB needles demonstrated a higher rate of adverse events than FNA needles (6% vs. 1%, p = .028). Sensitivity analysis on specimen adequacy based on various criteria demonstrated that the specimen adequacy rate defined by the AASLD criterion was lower than that of the commonly-used criterion (37% vs. 84%, p = .001).

CONCLUSION

EUS-LB is effective and safe for liver biopsy. Acquire needles provide better specimens than SharkCore needles. FNB needles may increase the risk of adverse events compared with FNA needles. The AASLD criterion is harder to achieve than the commonly-used criterion.

A Potential Mechanism of Kidney-Tonifying Herbs Treating Unexplained Recurrent Spontaneous Abortion: Clinical Evidence From the Homogeneity of Embryo Implantation and Tumor Invasion

Background: Kidney-tonifying herbs (KTHs) are widely used to treat unexplained recurrent spontaneous abortion (URSA) based on the theory of traditional Chinese medicine (TCM). However, there is still a lack of systematic evaluation and mechanistic explanation for these treatments.

Objective: The purpose of this study was to assess the clinical efficacy, and to investigate the potential mechanisms, of KTH based on TCM for the treatment of URSA.

Methods: A systematic literature search was conducted within PubMed, Embase, China Biomedical Literature database, Web of Science (WOS), China National Knowledge Infrastructure (CNKI) database, and the Wanfang database to find articles reporting on the Chinese herbal formula based around KTH for treating URSA, which were published between January 2010 and June 2021. A full bibliometric analysis was carried out; in addition, randomized controlled trial (RCT) articles were selected for systematic evaluation and meta-analysis. The drugs with the highest frequency of KTHs were screened for meta-analysis. Finally, network analysis and molecular docking were used to study the key components and potential pathway of KTHs in the treatment of URSA.

Results: The meta-analysis included nine RCTs involving 1,054 subjects. Compared with the control groups, the clinical efficacy of TCM-based KTHs in the treatment of URSA patients significantly improved outcomes. Additionally, a component target pathway network was identified, which included 32 potential blood activating components and 113 main targets. Japonine, sopranol, lysine, and matrine were considered the most important bioactive molecules for KTHs. The key potential therapeutic pathway for URSA was a tumor-related signaling pathway. The target genes for URSA regulated by KTHs were highly similar to tumor biological processes such as the regulation of apoptotic signaling pathways, inflammatory responses, angiogenesis, and epithelial metabolic transition.

Conclusion: KTH has great potential for treating URSA. Because the maintenance of pregnancy has a high similarity with tumor invasion, the research relating to tumor mechanisms should also be followed up as it may lead to new ideas and breakthroughs for research into URSA. At the same time, embryonic and decidual cells share a high degree of cellular heterogeneity and spatial structural complexity with tumor cells, and a single cell combined with spatial omics may be the best future approach for validating KTH mechanisms.

Hemoperfusion in combination with hemofiltration for acute severe organophosphorus pesticide poisoning: A systematic review and meta-analysis

Background:

Acute severe organophosphorus pesticide poisoning (ASOPP) is one of the major diseases that endanger human life and health. However, the effects of conventional therapy including gastric lavages, mechanical ventilation, muscarinic antagonist drugs, and cholinesterase reactivators were uncertain. This meta-analysis aims to investigate the safety and efficacy of hemoperfusion combined with hemofiltration besides routine therapy for ASOPP.

Materials and Methods:

A comprehensive search for candidate publications was performed through PubMed, Medline, Cochrane Library, WanFang, Chinese Biomedical Literature, and China National Knowledge Infrastructure from database inception to May 12, 2020. The retrieved studies were screened by the predefined inclusion and exclusion criteria. The data of important end points were extracted. The risk ratio (RR) and weighted mean difference (WMD) were pooled for categorical variables and continuous variables, respectively. Meta-analyses and publication bias were conducted by using STATA software version 15.1.

Results:

A total of 11 randomized controlled trials with 811 patients were included. Compared to conventional therapy group, patients in the hemoperfusion plus hemofiltration group were significantly superior with regard to mortality (RR 0.38, 95% confidence interval [CI] [0.25, 0.57], P < 0.001), total atropine dosing (WMD −147.34 mg, 95% CI [−199.49, −95.18], P < 0.001), duration of mechanical ventilation (WMD −2.34 days, 95% CI [−3.77, −0.92], P < 0.001), cholinesterase recovery time (WMD −2.49 days, 95% CI [−3.14, −1.83], P < 0.001), and length of stay (WMD −4.52 days, 95% CI [−5.31, −3.73], P < 0.001).

Conclusion:

Combined hemoperfusion and hemofiltration was a very safe and effective treatment protocol for ASOPP, not only resulting in significantly decreased mortality but also resulting in reduced total atropine dosing, duration of mechanical ventilation, cholinesterase recovery time, and length of stay.

Tranexamic Acid versus Epsilon-Aminocaproic Acid in Total Knee Arthroplasty: A Meta-Analysis

Objective

At present, the effect of tranexamic acid (TXA) and epsilon-aminocaproic acid (EACA) on total knee arthroplasty (TKA) remains controversial. Therefore, the aim of this meta-analysis is to compare the differences between the effects of TXA and EACA in TKA.

Methods

We used electronic databases, including PubMed, Embase, MEDLINE, Ovid, ScienceDirect, Cochran Library, Google Scholar, clinical trial, and Chinese related databases, for literature search to find any effect of TXA and EACA in TKA. The differences between groups were compared by odds ratio (OR), weighted mean difference (WMD), and 95% confidence interval (CI). A total of four studies, including 3 randomized controlled trials (RCT) and 1 cohort study, were involved in this meta-analysis, involving 1836 participants. Among these participants, 816 belonged to the TXA group and 1020 belonged to the EACA group.

Results

Meta-analysis indicated no difference in surgery time (WMD = 0.01, 95% CI -0.35 to 0.36), total amount of blood loss (WMD = 0.14, 95% CI -0.09 to 0.37), transfusion rate (OR = 0.74, 95% CI 0.20 to 2.78), transfusion units per patient (SMD = -0.15, 95% CI -0.54 to 0.25), complications (OR = 0.75, 95% CI 0.37 to 1.55), and length of stay (SMD = -0.01, 95% CI -0.11 to 0.08).

Conclusions

Our results suggest that the effect of TXA is not superior to EACA in TKA. However, this conclusion still needs to be further confirmed by multicenter and large-sample clinical trials.

Probiotic and prebiotic interventions for non-alcoholic fatty liver disease: a systematic review and network meta-analysis

Many studies have associated altered intestinal bacterial communities and non-alcoholic fatty liver disease, but the putative effects are inconclusive. The purpose of this network meta-analysis (NMA) was to evaluate the effects of probiotics, prebiotics, and synbiotics on non-alcoholic fatty liver disease through randomised intervention trials. Literature searches were performed until March 2020. For each outcome, a random NMA was performed, the surface under the cumulative ranking curve (SUCRA) was determined. A total of 22 randomised trials comparing prebiotic, probiotic, and synbiotic treatments included 1301 participants. Considering all seven results (aspartate aminotransferase, alanine aminotransferase, body mass index, weight, total cholesterol, high-density lipoprotein cholesterol, and low-density lipoprotein cholesterol) together, the highest SUCRA values are probiotics (94%), synbiotics (61%) and prebiotics (56%), respectively. NMA results provide evidence that probiotics, prebiotics, and synbiotics can alleviate non-alcoholic fatty liver disease. However, due to the lack of high-quality randomised trials, this research also has some limitations.

Failure rates associated with guided versus non-guided dental implant placement: a systematic review and meta-analysis

Objective

The purpose of the systematic review and meta-analysis was to evaluate implant failure rates and their association with guided and free-hand implant placement techniques.

Materials and methods

A literature search was conducted across PubMed, Medline via Ovid, Cochrane database, and Google Scholar. The search was completed in September 2020. Series of meta-analyses were conducted to compare implant failure rates with guided and free-hand techniques.

Results

A total of 3387 articles were identified from the electronic search. After applying the inclusion criteria, eight articles were selected for qualitative assessment and four for quantitative synthesis (meta-analysis). The included studies had a risk ratio of 0.29 (95% CI: 0.15, 0.58), P < 0.001 for the use of guided implant placement. Implant failure rates were affected by the different placement techniques indicated by the test for overall effect (Z = 3.53, P = 0.0004). The incidence of implant failure in guided surgery versus free-hand surgery was found to be 2.25% and 6.42%, respectively.

Conclusion

Both guided and free-hand implant placement techniques resulted in a high implant survival rate. However, implant failure rates were almost three times higher in the free-hand implant placement category. A guided implant placement approach is recommended for a successful outcome.

Assessing the risk of performance and detection bias in Cochrane reviews as a joint domain is less accurate compared to two separate domains

Background

Initially, the Cochrane risk of bias (RoB) tool had a domain for “blinding of participants, personnel and outcome assessors”. In the 2011 tool, the assessment of blinding was split into two domains: blinding of participants and personnel (performance bias) and blinding of outcome assessors (detection bias). The aims of this study were twofold; first, to analyze the frequency of usage of the joint blinding domain (a single domain for performance and detection bias), and second, to assess the proportion of adequate assessments made in the joint versus single RoB domains for blinding by comparing whether authors’ RoB judgments were supported by explanatory comments in line with the Cochrane Handbook recommendations.

Methods

We extracted information about the assessment of blinding from RoB tables (judgment, comment, and whether it was specified which outcome type; e.g., objective, subjective) of 729 Cochrane reviews published in 2015-2016. In the Cochrane RoB tool, judgment (low, unclear or high risk) needs to be accompanied by a transparent comment, in which authors provide a summary justifying RoB judgment, to ensure transparency in how these judgments were reached. We reassessed RoB based on the supporting comments reported in Cochrane RoB tables, in line with instructions from the Cochrane Handbook. Then, we compared our new assessments to judgments made by Cochrane authors. We compared the frequency of adequate judgments in reviews with two separate domains for blinding versus those with a joint domain for blinding.

Results

The total number of assessments for performance bias was 6918, with 8656 for detection bias and 3169 for the joint domain. The frequency of adequate assessments was 74% for performance bias, 78% for detection bias, and 59% for the joint domain. The lowest frequency of adequate assessments was found when Cochrane authors judged low risk – 47% in performance bias, 62% in detection bias, and 31% in the joint domain. The joint domain and detection bias domain had a similar proportion of specified outcome types (17% and 18%, respectively).

Conclusions

Splitting joint RoB assessment about blinding into two domains was justified because the frequency of adequate judgments was higher in separate domains. Specification of outcome types in RoB domains should be further scrutinized.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01339-1.

Enhanced access to recommendations from the Cochrane Handbook for improving authors' judgments about risk of bias: A randomized controlled trial

This randomized controlled trial (RCT) aimed to test the efficacy of enhanced access to Cochrane Handbook (Handbook) recommendations for judging the 2011 Cochrane risk of bias (RoB) domains for improving the adequacy of RoB judgments. Parallel-group RCT with a 1:1 allocation ratio (N = 2271 per group) was conducted. Eligible participants were corresponding authors of all published Cochrane reviews and protocols. After allocation by a random number generator, participants received 20 scenarios for assessing RoB. The intervention group was shown tables from the Handbook with instructions for assessing 2011 RoB tool together with scenarios they were supposed to assess-enhanced access to the Handbook. The control group was shown only a general link to the Handbook. The primary outcome was the proportion of participants that made an adequate judgment of RoB scenarios for analyzed domains. There were 240 responses out of 2020 delivered e-mail invitations in the intervention and 197/2254 in the control group. Only five participants from the intervention group judged RoB adequately in all the 20 scenarios and no one in the control group. The proportion of participants who adequately assessed all the scenarios within a domain was significantly higher in the intervention than in the control group. The frequency of adequate RoB judgments was 7.1% (95% CI: 5.0-9.3%, p < 0.001) higher in the intervention group (76.2%) than in the control group (69.0%). The enhanced access yields more adequate RoB assessments and could be incorporated in software supporting the RoB tool.

Agreement in Risk of Bias Assessment Between RobotReviewer and Human Reviewers: An Evaluation Study on Randomised Controlled Trials in Nursing-Related Cochrane Reviews

PURPOSE

RobotReviewer is a machine learning system for semi-automated assistance in risk of bias assessment. The tools's performance in randomized controlled trials (RCTs) in the field of nursing remains unknown. We aimed therefore to evaluate the agreement in risk of bias assessment between RobotReviewer and human reviewers.

DESIGN

Evaluation study using a retrospective diagnostic design.

METHODS

We used RobotReviewer as the index test and human reviewers' risk of bias assessment reported in Cochrane reviews as the reference test. A convenience sample of electronically available English-language full texts of RCTs included in Cochrane reviews with nurs* in the title were eligible for inclusion. In this context, we assessed random sequence generation, allocation concealment, and blinding (personnel or participants and assessors) corresponding to Cochrane risk of bias version 2011. Two independent research teams performed and double-checked data extraction and analysis. We calculated sensitivity, specificity, receiver operating characteristic (ROC) curve, the area under the ROC curve, predictive values, observed percentage of agreement, and Cohen's kappa (including confidence intervals, if applicable).

FINDINGS

The selection process yielded 190 RCTs published between 1958 and 2016 in 23 Cochrane reviews published between 2000 and 2018. Missing assessments of risk of bias domains in Cochrane reviews or RobotReviewer yielded varying sample sizes per risk of bias domain. Sensitivity ranged from 0.44 to 0.88 and specificity from 0.48 to 0.95. Positive predictive value was highest for allocation concealment (0.79) and lowest for blinding assessors (0.25). Cohen's kappa was moderate for randomization (0.52), allocation concealment (0.60), and for blinding of personnel/patients (0.43). Blinding of outcome assessors had only slight agreement (0.04).

CONCLUSIONS

This is the first evaluation of risk of bias assessment by RobotReviewer in RCTs included in nursing-related Cochrane reviews. It yielded a moderate degree of agreement with human reviewers for randomization and allocation concealment, and an adequate sensitivity for detecting low risk of selection bias.

CLINICAL RELEVANCE

Based on our results, using the RobotReviewer for risk of bias assessment in RCTs can be supportive in some risk of bias domains. However, human reviewers should supervise the semi-automated assessment process.

Second-generation cryoballoon vs. contact-force sensing radiofrequency catheter ablation in atrial fibrillation: a meta-analysis of randomized controlled trials

OBJECTIVES

It is imperative to understand the influence of second-generation cryoballoon (CB-2) and contact-force sensing radiofrequency ablation (CF-RF) on clinical outcomes in atrial fibrillation (AF). This updated meta-analysis of randomized controlled trials (RCTs) examined the efficacy and safety of CB-2 vs. CF-RF in patients with AF.

METHODS

RCTs on the use of CB-2 vs. CF-RF in patients with AF were included. The primary outcome was the recurrence of AF, and the key secondary outcomes included serious complications, acute pulmonary vein isolation (PVI), procedure duration, and fluoroscopy time.

RESULTS

A total of 261 articles were identified, and five studies with a total of 845 participants were included in the study. A total of 93% of participants had paroxysmal AF, 7% of participants had persistent AF, and none of participants had permanent AF. There were 499 participants in the CB-2 arm and 346 in the CF-RF arm. AF recurrence was comparable in the CB-2 group (30.3%) and the CF-RF group (29.2%) (OR = 0.93; 95%CI = 0.56-1.54; P = 0.79; I² = 48%). There were no statistical differences in acute PVI (P = 0.92; I² = 0%) and serious complications (P = 0.87; I² = 47%) between the two groups. The procedure duration was shorter in the CB-2 group than in the CF-RF group (MD = - 13.39; 95%CI = - 15.58, - 7.19; P < 0.0001; I² = 59%).

CONCLUSION

Our study demonstrated that CB-2 and CF-RF had comparable recurrences of AF and similar incidences of serious complications in AF patients during the ablation process.

Adequacy of risk of bias assessment in surgical vs non-surgical trials in Cochrane reviews: a methodological study

Background

Bias in randomized controlled trials (RCTs) can lead to underestimation or overestimation of the true effects of interventions. Surgical RCTs may suffer from the risk of bias (RoB) that is avoidable in trials of other interventions, and vice versa. We aimed to compare the adequacy of RoB assessments in surgical versus non-surgical RCTs included in Cochrane reviews and to assess the most common differences in those RoB assessments. Due to specificities of surgical trials, i.e. difficulties associated with blinding of surgical interventions, we hypothesized that assessments of surgical trials may be more adequate, compared to RCTs of non-surgical interventions.

Methods

This was a methodological study, analyzing methods of published Cochrane systematic reviews. Data were extracted from RoB tables in Cochrane reviews (judgments and accompanying explanatory comment) for the following four RoB domains used in the 2011 Cochrane RoB tool: randomization, allocation concealment, blinding of participants and personnel, and blinding of outcome assessors. We defined adequate assessments as those that were in line with instructions from the Cochrane Handbook for Systematic Reviews of Interventions. The prevalence of adequate assessments was compared in surgical versus non-surgical trials. The most common differences in both groups of reviews were presented.

Results

In 729 analyzed Cochrane reviews, there were 10,537 included trials. The prevalence of adequate RoB judgments made by Cochrane authors ranged from 87.9, 95%CI (87.3 to 88.6%) for randomization to 70.7, 95%CI (69.8 to 71.5%) for blinding of participants and personnel. For all analyzed RoB domains, the prevalence of adequate RoB domains was higher in surgical trials than in non-surgical trials. For two RoB domains assessing blinding, this difference between surgical and non-surgical trials was statistically significant (P < 0.001), while the difference was not significant for the RoB domain regarding randomization (P = 0.124) and allocation concealment (P = 0.039, β < 0.8).

Conclusions

RoB judgments were more in line with instructions from the Cochrane Handbook when Cochrane reviews assessed surgical trials, compared to those that analyzed non-surgical interventions. However, further steps are warranted to scrutinize RoB assessment in trials of both surgical and non-surgical interventions.

Agreement of Risk-of -Bias varied in systematic reviews on acupuncture and was associated with methodological quality

OBJECTIVES

The objective of the study was to evaluate the consistency of risk of bias assessments for overlapping randomized controlled trials (RCTs) included in systematic reviews (SRs) on acupuncture.

STUDY DESIGN AND SETTING

Databases were searched for acupuncture SRs. A weighted kappa (κ) statistic was calculated, and logistic regression was used to explore the factors of disagreements.

RESULTS

We included 241 RCTs from 109 SRs on acupuncture. The percentage disagreements ranged from 25% to 44%, with moderate agreement for random sequence generation (κ = 0.57), allocation concealment (κ = 0.50), and incomplete outcome data (κ = 0.50), besides fair agreement for blinding of participants and personnel (κ = 0.44), blinding of outcome assessment (κ = 0.31), and selective reporting (κ = 0.39). Only 19% RCTs were evaluated completely consistent. Methodological quality (random sequence generation, odds ratio (OR) = 3.46), international cooperation (allocation concealment, OR = 0.14; incomplete outcome data, OR = 0.14; selective reporting, OR = 0.05), and risk of bias reporting completeness score (selective reporting, OR = 0.53) significantly affected the relative odds of disagreements.

CONCLUSION

The level of agreement varied from fair to moderate agreement depending on the risk of bias domain. Methodological quality appears to be an overarching factor to account for disagreements.

Risk of Bias and Quality of Reporting in Colon and Rectal Cancer Systematic Reviews Cited by National Comprehensive Cancer Network Guidelines

INTRODUCTION

Given the changing landscape of colorectal cancer, systematic reviews are likely to play a key role in advancing the understanding of prevention, diagnosis, and treatment.

METHODS

We conducted a cross-sectional investigation of the risk of bias and reporting quality of systematic reviews referenced by colon and rectal cancer National Comprehensive Cancer Network (NCCN) guidelines. We used two widely accepted tools: Risk of Bias in Systematic reviews (ROBIS) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA).

RESULTS

Using ROBIS, only 3 (4.8%) systematic reviews were judged with low risk of bias, 35 (55.6%) systematic reviews were judged with unclear risk of bias, and 25 (39.7%) systematic reviews were judged with high risk of bias. Across all systematic reviews, the individual bias domains at the highest risk of bias were domains 1 (protocol and eligibility criteria) and 2 (methods to identify and select studies). Across all studies, the median adherence to PRISMA was 74.1% (IQR 69.2-80.0%), corresponding to approximately 20 of 27 items.

CONCLUSIONS

Systematic reviews cited in NCCN guidelines for colon and rectal cancer are frequently at unclear or high risk of bias and do not report key systematic review items that are important for the critical appraisal of results.

Quality of evidence in a post-Soviet country: evaluation of methodological quality of controlled clinical trials published in national journals from Uzbekistan

Background

Most researchers in Uzbekistan prefer to publish their reports in journals of their home country. Moreover, the proportion of healthcare practitioners who prefer to use these national sources of information also remains high. However, the quality of publications from national journals, in post-Soviet countries, has not been systematically evaluated until now. The primary objective of this study was to evaluate the quality of randomized controlled trials’ (RCTs) reports published in medical journals from Uzbekistan. We supposed that reports had at least minimal quality to contribute to the higher quality of healthcare.

Methods

To evaluate the quality of RCTs, we selected two journals from the list of national medical journals for which background information was provided. We decided to select articles from journals that had the highest subscription rate and were likely to have the highest impact on clinical decisions. The journals were Medical Journal of Uzbekistan and Paediatrics. Only issues published in 2007–2017 were considered for evaluation. Two evaluators independently scored RCTs and controlled clinical trials (CCTs) reported in the journals. The 5-point scale developed by Jadad et al. was used to evaluate the quality of reports. Consensus-based decision was made about the final score of each report.

Results

We reviewed 1311 studies in the two journals and found 380 clinical trials reports for the final evaluation. Our main finding was that none of the reports received a final score of more than 1, with an absolute agreement between evaluators. A median score of the studied reports was equal to 0, predicting a very low quality of controlled trials reported in the national journals (Wilcoxon signed-rank test p = 1.0; 95% CI = 0–0).

Conclusions

We believe that quality of reports about controlled trials, in Uzbekistan, can be considered insufficient to contribute to the higher quality of care and patients’ safety. In the worst case, such condition can cause serious damage to the public health and lead to ineffective use of resources in the country. Therefore, the better reporting and organization of RCTs and CCTs should become a main goal of all stakeholders interested in the effective and safe healthcare in the country.

Sound Therapy for Tinnitus: Options for Audiologists

Purpose

Sound therapy to reduce the emotional and functional effects of tinnitus has been used by audiologists since the 1970s when Jack Vernon introduced the masking method to provide a sense of relief from tinnitus distress. Our group renamed masking sound as “soothing” sound and distinguished it from “interesting” and “background” sound, each of which has a different purpose for tinnitus sound therapy. Other methods of sound therapy have the potential to reduce the “sensation” of tinnitus, including notched noise, matched noise, desynchronization, and residual inhibition. The purpose of this article is to provide an overview of the different sound therapy approaches to serve as a resource for audiologists who often provide sound therapy to their patients with tinnitus.

Conclusion

Although, according to systematic reviews, sound therapy does not have strong evidence for treatment of tinnitus, it is nonetheless well evidenced both through abundant research and clinical utilization mostly by audiologists. It is unknown if any one form of sound therapy is superior to any other.

Assessing risk of bias judgments for blinding of outcome assessors in Cochrane reviews

Aim: Adequate judging of risk of bias (RoB) for blinding of outcome assessors (detection bias) is important for supporting highest level of evidence. Materials & methods: Judgments and supporting comments for detection bias were retrieved from RoB tables reported in Cochrane reviews. We categorized comments, and then compared judgment and supporting comment with instructions from the Cochrane Handbook. Results: We analyzed 8656 judgments for detection bias from 7626 trials included in 575 reviews. Overall, 1909 judgments (22%) were not in line with the Cochrane Handbook. In 9% of trials, the authors split the detection bias domain according to outcomes. Here, prevalence of inadequate judgments was 19%. Conclusion: Interventions to improve RoB assessments in systematic reviews should be explored.

Methodological tools and sensitivity analysis for assessing quality or risk of bias used in systematic reviews published in the high-impact anesthesiology journals

Background

A crucial element in the systematic review (SR) methodology is the appraisal of included primary studies, using tools for assessment of methodological quality or risk of bias (RoB). SR authors can conduct sensitivity analyses to explore whether their results are sensitive to exclusion of low quality studies or a high RoB. However, it is unknown which tools do SR authors use for assessing quality/RoB, and how they set threshold for quality/RoB in sensitivity analyses. The aim of this study was to assess quality/RoB assessment tools, the types of sensitivity analyses and quality/RoB thresholds for sensitivity analyses used within SRs published in high-impact pain/anesthesiology journals.

Methods

This was a methodological study. We analyzed SRs published from January 2005 to June 2018 in the 25% highest-ranking journals within the Journal Citation Reports (JCR) “Anesthesiology” category. We retrieved the SRs from PubMed. Two authors independently screened records, full texts, and extracted data on quality/RoB tools and sensitivity analyses. We extracted data about quality/RoB tools, types of sensitivity analyses and the thresholds for quality/RoB used in them.

Results

Out of 678 analyzed SRs, 513 (76%) reported the use of quality/RoB assessments. The most commonly reported tools for assessing quality/RoB in the studies were the Cochrane tool for risk of bias assessment (N = 251; 37%) and Jadad scale (N = 99; 15%). Meta-analysis was conducted in 451 (66%) of SRs and sensitivity analysis in 219/451 (49%). Most commonly, sensitivity analysis was conducted to explore the influence of study quality/RoB (90/219; 41%) on the results. Quality/RoB thresholds used for sensitivity analysis for those studies were clearly reported in 47 (52%) articles that used them. The quality/RoB thresholds used for sensitivity analyses were highly heterogeneous and inconsistent, even when the same tool was used.

Conclusions

A quarter of SRs reported using quality/RoB assessments, and some of them cited tools that are not meant for assessing quality/RoB. Authors who use quality/RoB to explore the robustness of their results in meta-analyses use highly heterogeneous quality/RoB thresholds in sensitivity analyses. Better methodological consistency for quality/RoB sensitivity analyses is needed.

Cochrane risk of bias tool was used inadequately in the majority of non-Cochrane systematic reviews

OBJECTIVES

To analyze how many non-Cochrane systematic reviews (NCSRs) used Cochrane's risk of bias (RoB) tool, domains they used, and whether judgments and comments about RoB were in line with Cochrane Handbook.

METHODS

This was a methodological (research-on-research) study. We retrieved NCSRs from PubMed, extracted information about methods used for RoB assessment, and if they used 2011 Cochrane RoB tool, we analyzed their RoB methods and compared them with Cochrane Handbook guidance.

RESULTS

We included 508 NCSRs; 431 (85%) reported they analyzed RoB, and 269 (53%) used Cochrane RoB tool. Only 16 of those 269 (5.9%) reported both a judgment and a supporting comment in the Cochrane RoB table in the manuscript (N = 4) or in a supplementary file (N = 12). Fifteen reviews, with 158 included trials, used judgments low/high/unclear; 41% of analyzed available judgments were inadequate, either because judgment was not in line with comment or comment was missing.

CONCLUSIONS

Most NCSRs use Cochrane RoB tool to assess RoB, but most of them reported it incompletely, with high prevalence of inadequate judgments. Authors, editors, and peer-reviewers should make an effort to improve completeness and adequacy of Cochrane RoB assessment in non-Cochrane reviews.

External factors may influence Cochrane reviewers when classifying the risk of bias of original reports

OBJECTIVE

The objective of the study was to explore contextual factors associated with high or low risk-of-bias judgment in case of incomplete or unclear information in study reports.

STUDY DESIGN AND SETTING

Research-on-research study, using matched case-control design, with a sample of 304 randomized controlled trials (RCTs) included in two Cochrane reviews for which there was disagreement on the risk-of-bias judgment related to incomplete or unclear information in the study report. A case was defined as an RCT judged at high or low risk of bias; a control was the same RCT judged at unclear risk. We used a conditional logistic regression model for analysis.

RESULTS

Review authors being also authors of the RCT were more likely to assess an item at low risk of bias than unclear (OR: 11.71; 95% CI: 1.39-98.76). Earlier trials in a review were more often assigned a low risk (OR: 0.37; [0.15-0.96]). Review groups and authors that had completed a lower number of reviews slightly more often assigned a low risk, whereas others reported "unclear" (OR: 0.97, [95% CI: 0.95-0.99] for groups) and 0.97 (95% CI: 0.95-0.998) for authors).

CONCLUSIONS

Risk-of-bias assessment of RCTs in case of incomplete or unclear information may be affected by contextual factors.

Inter-review agreement of risk-of-bias judgments varied in Cochrane reviews

OBJECTIVES

The objective of the study was to measure the level of agreement between Cochrane reviews of overlapping randomized controlled trials (RCTs) regarding risk-of-bias (RoB) judgments.

STUDY DESIGN AND SETTING

On November 5, 2017, the Cochrane Database of Systematic Reviews was searched for Cochrane reviews on tobacco. Reviews that included overlapping RCTs were included. RoB judgments were extracted from RoB tables using automated data scraping with manual verification and adjustments. Agreement between the reviews was calculated using Conger's generalized kappa coefficient (κ) and raw agreement (a).

RESULTS

We included 53 Cochrane reviews of 376 RCTs. For the RoB domain "random sequence generation," the level of agreement between the reviews was substantial with κ = 0.63 (95% confidence interval: 0.56 to 0.71; a = 0.80). There was slight-to-moderate agreement between the reviews regarding the domains "allocation concealment": κ = 0.51 (0.41 to 0.61), a = 0.75; "blinding": κ = 0.19 (0.02 to 0.37), a = 0.52; "blinding of outcome assessment": κ = 0.43 (0.14 to 0.72) a = 0.67; and "incomplete outcome data": κ = 0.15 (-0.03 to 0.32), a = 0.64. For "blinding of participants and personnel" and "selective reporting", κ could not be calculated. The raw agreement was 0.40 and 0.42, respectively.

CONCLUSION

The level of agreement between Cochrane reviews regarding RoB judgments ranged from slight to substantial depending on the RoB domain. Further investigations regarding reasons for variation and interventions to improve agreement are needed.

Overall bias methods and their use in sensitivity analysis of Cochrane reviews were not consistent

OBJECTIVE

The objective of the study was to analyze methods of assessing "overall bias" in Cochrane reviews of interventions published in the Cochrane Database of Systematic Reviews and sensitivity analyses related to overall risk of bias (RoB).

STUDY DESIGN AND SETTING

From Cochrane reviews published within 3 years, from July 2015 to June 2018, we extracted data regarding methods of judging overall bias for a single trial, as well as details regarding methods used in frequency of RoB in sensitivity analyses.

RESULTS

Of the 1,452 analyzed Cochrane reviews, 409 mentioned assessment of overall RoB on a study level. In 107 reviews, authors clearly specified key domains that determined the overall RoB, whereas in the remaining reviews, assessment of overall bias was not in line with the Cochrane Handbook. Among 268 Cochrane reviews that had any RoB-related sensitivity analysis, in 56 (21%) reviews, the authors reported a significant change for at least one outcome compared with the initial analysis.

CONCLUSION

Highly heterogeneous approaches to summarizing overall RoB on a study level and using RoB for sensitivity analyses may yield inconsistent and incomparable results across Cochrane reviews.

Efficacy and safety of modified-release paracetamol for acute and chronic pain: a systematic review protocol

INTRODUCTION

Paracetamol (acetaminophen) is widely used for management of mild-to-moderate pain and reduction of fever. It is available as immediate release (IR) and modified-release (MR) formulation. In 2017, European Medicines Agency recommended a suspension of marketing of MR paracetamol in the European Union. Benefit-risk balance of these products has been assessed as negative as data showed that existing procedures for overdose management may not be efficient. Since MR paracetamol is still available in other countries (Australia and USA) and there is no available systematic review (SR) of efficacy and safety of MR paracetamol in the literature, we have decided to perform one to evaluate available data from randomised clinical trials (RCTs).

METHODS AND ANALYSIS

Using predefined search criteria, we will search EMBASE, MEDLINE, Cochrane Central Register of Controlled Trials, ClinicalTrials.gov and WHO International Clinical Trials Registry Platform to identify RCTs evaluating efficacy and safety of MR paracetamol alone in any dose or duration for any pain. Participants are defined as adults and adolescents (over 12 years). Primary efficacy outcomes will be pain intensity, pain relief and sleep. Primary safety outcomes will be the number of patients experiencing any (serious) adverse event, the number of patients withdrawn due to adverse events and the number of patients with gastrointestinal and hepatic adverse events. Data analysis will be subdivided based on different clinical syndromes. Meta-analysis will be conducted if possible. Cochrane risk of bias (RoB) tool with seven dimensions will be used to assess RoB of individual studies.

ETHICS AND DISSEMINATION

This SR will include only data collected from trial reports; therefore, an ethical approval will not be sought. We will publish the protocol and our findings in peer-reviewed journals.

PROSPERO REGISTRATION NUMBER

CRD42018115769.

Overall bias and sample sizes were unchanged in ICU trials over time: a meta-epidemiological study

OBJECTIVE

To assess time trends in risk of bias (RoB) and sample sizes in randomized clinical trials (RCTs) of adult intensive care unit (ICU) patients.

STUDY DESIGN AND SETTING

A meta-epidemiological study of RCTs from Cochrane systematic reviews assessing interventions in adult ICU patients. Using run charts, we assessed time trends in the annual proportion of RCTs with overall low RoB, the annual median sample sizes, and the annual proportion of RCTs with low, unclear, and high RoB in individual bias domains.

RESULTS

We included 604 RCTs published between 1977 and 2018 from 53 Cochrane systematic reviews. Only 6.8% of the RCTs had overall low RoB. We observed only random variation in the annual proportions of RCTs with overall low RoB, in the annual median sample sizes and in most individual bias domains. For "allocation concealment," we observed an increase in the proportion of low RoB RCTs and a decrease in the unclear RoB RCTs.

CONCLUSIONS

Few RCTs in adult ICU patients had overall low RoB. We found no evidence of an increase in RCTs with overall low RoB or in the median sample sizes over time. The only individual RoB domain with better ratings over time was "allocation concealment."

Risk of bias judgments for random sequence generation in Cochrane systematic reviews were frequently not in line with Cochrane Handbook

BACKGROUND

Assessing the risk of bias (RoB) in included studies is one of the key methodological aspects of systematic reviews. Cochrane systematic reviews appraise RoB of randomised controlled trials (RCTs) with the Cochrane RoB tool. Detailed instructions for using the Cochrane RoB tool are provided in the Cochrane Handbook for Systematic Reviews of Interventions (The Cochrane Handbook). The purpose of this study was to analyse whether Cochrane authors use adequate judgments about the RoB for random sequence generation of RCTs included in Cochrane reviews.

METHODS

We extracted authors' judgments (high, low or unclear RoB) and supports for judgments (comments accompanying judgments which explain the rationale for a judgment) for random sequence generation of included RCTs from RoB tables of Cochrane reviews using automated data scraping. We categorised all supporting comments, analysed the number and type of various supporting comments and assessed adequacy of RoB judgment for randomisation in line with recommendations from the Cochrane Handbook.

RESULTS

We analysed 10,103 RCTs that were included in 704 Cochrane reviews. For 5,706 RCTs, randomisation was not described, but for the remaining RCTs, it was indicated that randomisation was performed using computer/software/internet (N = 2,850), random number table (N = 883), mechanical method (N = 359) or it was incomplete/inappropriate (N = 305). Overall, 1,220/10,103 trials (12%) did not have a RoB judgment in line with Cochrane Handbook guidance about randomisation. The highest proportion of misjudgements was found for trials with high RoB (28%), followed by those with low (20%) or unclear (3%). Therefore, one in eight judgments for the analysed domain in Cochrane reviews was not in line with Cochrane Handbook, and one in four if the judgment was "high risk".

CONCLUSION

Authors of Cochrane reviews often make judgments about the RoB related to random sequence generation that are not in line with instructions given in the Cochrane Handbook, which compromises the reliability of the systematic reviews. Our results can help authors of both Cochrane and non-Cochrane reviews which use Cochrane RoB tool to avoid making common mistakes when assessing RoB in included trials.

The judgement of biases included in the category "other bias" in Cochrane systematic reviews of interventions: a systematic survey

BACKGROUND

Clinical decisions are made based on Cochrane reviews, but the implementation of results of evidence syntheses such as Cochrane reviews is problematic if the evidence is not prepared consistently. All systematic reviews should assess the risk of bias (RoB) in included studies, and in Cochrane reviews, this is done by using Cochrane RoB tool. However, the tool is not necessarily applied according to the instructions. In this study, we aimed to determine the types of bias and their corresponding judgements noted in the 'other bias' domain of Cochrane RoB tool.

METHODS

We analyzed Cochrane reviews that included randomized controlled trials (RCTs) and extracted data regarding 'other bias' from the RoB table and accompanying support for the judgment. We categorized different types of other bias.

RESULTS

We analyzed 768 Cochrane reviews that included 11,369 RCTs. There were 602 (78%) Cochrane reviews that had 'other bias' domain in the RoB tool, and they included a total of 7811 RCTs. In the RoB table of 337 Cochrane reviews for at least one of the included trials it was indicated that no other bias was found and supporting explanations were inconsistently judged as low, unclear or high RoB. In the 524 Cochrane reviews that described various sources of other bias, there were 5762 individual types of explanations which we categorized into 31 groups. The judgments of the same supporting explanations were highly inconsistent. We found numerous other inconsistencies in reporting of sources of other bias in Cochrane reviews.

CONCLUSION

Cochrane authors mention a wide range of sources of other bias in the RoB tool and they inconsistently judge the same supporting explanations. Inconsistency in appraising risk of other bias hinders reliability and comparability of Cochrane systematic reviews. Discrepant and erroneous judgments of bias in evidence synthesis may hinder implementation of evidence in routine clinical practice and reduce confidence in otherwise trustworthy sources of information. These results can help authors of Cochrane and non-Cochrane reviews to gain insight into various sources of other bias that can be found in trials, and also to help them avoid mistakes that were recognized in published Cochrane reviews.

Assessments of attrition bias in Cochrane systematic reviews are highly inconsistent and thus hindering trial comparability

Background

An important part of the systematic review methodology is appraisal of the risk of bias in included studies. Cochrane systematic reviews are considered golden standard regarding systematic review methodology, but Cochrane’s instructions for assessing risk of attrition bias are vague, which may lead to inconsistencies in authors’ assessments. The aim of this study was to analyze consistency of judgments and support for judgments of attrition bias in Cochrane reviews of interventions published in the Cochrane Database of Systematic Reviews (CDSR).

Methods

We analyzed Cochrane reviews published from July 2015 to June 2016 in the CDSR. We extracted data on number of included trials, judgment of attrition risk of bias for each included trial (low, unclear or high) and accompanying support for the judgment (supporting explanation). We also assessed how many Cochrane reviews had different judgments for the same supporting explanations.

Results

In the main analysis we included 10,292 judgments and supporting explanations for attrition bias from 729 Cochrane reviews. We categorized supporting explanations for those judgments into four categories and we found that most of the supporting explanations were unclear. Numerical indicators for percent of attrition, as well as statistics related to attrition were judged very differently. One third of Cochrane review authors had more than one category of supporting explanation; some had up to four different categories. Inconsistencies were found even with the number of judgments, names of risk of bias domains and different judgments for the same supporting explanations in the same Cochrane review.

Conclusion

We found very high inconsistency in methods of appraising risk of attrition bias in recent Cochrane reviews. Systematic review authors need clear guidance about different categories they should assess and judgments for those explanations. Clear instructions about appraising risk of attrition bias will improve reliability of the Cochrane’s risk of bias tool, help authors in making decisions about risk of bias and help in making reliable decisions in healthcare.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0717-9) contains supplementary material, which is available to authorized users.

Disagreements in risk of bias assessment for randomised controlled trials included in more than one Cochrane systematic reviews: a research on research study using cross-sectional design

OBJECTIVES

Assess the frequency and reasons for disagreements in risk of bias assessments for randomised controlled trials (RCTs) included in more than one Cochrane review.

DESIGN

Research on research study, using cross-sectional design.

DATA SOURCES

2796 Cochrane reviews published between March 2011 and September 2014.

DATA SELECTION

RCTs included in more than one review.

DATA EXTRACTION

Risk of bias assessment and support for judgement for five key risk of bias items.

DATA SYNTHESIS

For each item, we compared risk of bias assessment made in each review and calculated proportion of agreement. Two reviewers independently analysed 50% of all disagreements by comparing support for each judgement with information from study report to evaluate whether disagreements were related to a difference in information (eg, contact the study author) or a difference in interpretation (same support for judgement but different interpretation). They also identified main reasons for different interpretation.

RESULTS

1604 RCTs were included in more than one review. Proportion of agreement ranged from 57% (770/1348 trials) for incomplete outcome data to 81% for random sequence generation (1193/1466). Most common source of disagreement was difference in interpretation of the same information, ranging from 65% (88/136) for random sequence generation to 90% (56/62) for blinding of participants and personnel. Access to different information explained 32/136 (24%) disagreements for random sequence generation and 38/205 (19%) for allocation concealment. Disagreements related to difference in interpretation were frequently related to incomplete or unclear reporting in the study report (83% of disagreements related to different interpretation for random sequence generation).

CONCLUSIONS

Risk of bias judgements of RCTs included in more than one Cochrane review differed substantially. Most disagreements were related to a difference in interpretation of an incomplete or unclear description in the study report. A clearer guidance on common causes of incomplete information may improve agreement.