Impact of a short version of the CONSORT checklist for peer reviewers to improve the reporting of randomised controlled trials published in biomedical journals: study protocol for a randomised controlled trial

Evaluation of Reporting Quality of Glaucoma Randomized Controlled Trial Abstracts: Current Status and Future Perspectives

The aim of this study was to explore adherence to the Consolidated Standards of Reporting Trials (CONSORT) reporting standards in abstracts of randomized controlled trials on glaucoma. A cross-sectional observational study was conducted on the aforementioned abstracts, indexed in MEDLINE/PubMed between the years 2017 and 2021. In total, 302 abstracts met the inclusion criteria and were further analyzed. The median score of CONSORT-A items was 8 (interquartile range, 7-10) out of 17 (47.0%). Most analyzed studies were conducted in a single center (80.5%) and the abstracts were predominantly structured (95.0%). Only 20.5% of the abstracts adequately described the trial design, while randomization and funding were described by 6.0% of the abstracts. Higher overall scores were associated with structured abstracts, a multicenter setting, statistically significant results, funding by industry, a higher number of participants, and having been published in journals with impact factors above four (p < 0.001, respectively). The results of this study indicate a suboptimal adherence to CONSORT-A reporting standards, especially in particular items such as randomization and funding. Since these factors could contribute to the overall quality of the trials and further translation of trial results into clinical practice, an improvement in glaucoma research reporting transparency is needed.

Reviewer training for improving grant and journal peer review

BACKGROUND

Funders and scientific journals use peer review to decide which projects to fund or articles to publish. Reviewer training is an intervention to improve the quality of peer review. However, studies on the effects of such training yield inconsistent results, and there are no up-to-date systematic reviews addressing this question.

OBJECTIVES

To evaluate the effect of peer reviewer training on the quality of grant and journal peer review.

SEARCH METHODS

We used standard, extensive Cochrane search methods. The latest search date was 27 April 2022.

SELECTION CRITERIA

We included randomized controlled trials (RCTs; including cluster-RCTs) that evaluated peer review with training interventions versus usual processes, no training interventions, or other interventions to improve the quality of peer review.

DATA COLLECTION AND ANALYSIS

We used standard Cochrane methods. Our primary outcomes were 1. completeness of reporting and 2. peer review detection of errors. Our secondary outcomes were 1. bibliometric scores, 2. stakeholders' assessment of peer review quality, 3. inter-reviewer agreement, 4. process-centred outcomes, 5. peer reviewer satisfaction, and 6. completion rate and speed of funded projects. We used the first version of the Cochrane risk of bias tool to assess the risk of bias, and we used GRADE to assess the certainty of evidence.

MAIN RESULTS

We included 10 RCTs with a total of 1213 units of analysis. The unit of analysis was the individual reviewer in seven studies (722 reviewers in total), and the reviewed manuscript in three studies (491 manuscripts in total). In eight RCTs, participants were journal peer reviewers. In two studies, the participants were grant peer reviewers. The training interventions can be broadly divided into dialogue-based interventions (interactive workshop, face-to-face training, mentoring) and one-way communication (written information, video course, checklist, written feedback). Most studies were small. We found moderate-certainty evidence that emails reminding peer reviewers to check items of reporting checklists, compared with standard journal practice, have little or no effect on the completeness of reporting, measured as the proportion of items (from 0.00 to 1.00) that were adequately reported (mean difference (MD) 0.02, 95% confidence interval (CI) -0.02 to 0.06; 2 RCTs, 421 manuscripts). There was low-certainty evidence that reviewer training, compared with standard journal practice, slightly improves peer reviewer ability to detect errors (MD 0.55, 95% CI 0.20 to 0.90; 1 RCT, 418 reviewers). We found low-certainty evidence that reviewer training, compared with standard journal practice, has little or no effect on stakeholders' assessment of review quality in journal peer review (standardized mean difference (SMD) 0.13 standard deviations (SDs), 95% CI -0.07 to 0.33; 1 RCT, 418 reviewers), or change in stakeholders' assessment of review quality in journal peer review (SMD -0.15 SDs, 95% CI -0.39 to 0.10; 5 RCTs, 258 reviewers). We found very low-certainty evidence that a video course, compared with no video course, has little or no effect on inter-reviewer agreement in grant peer review (MD 0.14 points, 95% CI -0.07 to 0.35; 1 RCT, 75 reviewers). There was low-certainty evidence that structured individual feedback on scoring, compared with general information on scoring, has little or no effect on the change in inter-reviewer agreement in grant peer review (MD 0.18 points, 95% CI -0.14 to 0.50; 1 RCT, 41 reviewers, low-certainty evidence).

AUTHORS' CONCLUSIONS

Evidence from 10 RCTs suggests that training peer reviewers may lead to little or no improvement in the quality of peer review. There is a need for studies with more participants and a broader spectrum of valid and reliable outcome measures. Studies evaluating stakeholders' assessments of the quality of peer review should ensure that these instruments have sufficient levels of validity and reliability.

Evaluating the completeness of the reporting of abstracts since the publication of the CONSORT extension for abstracts: an evaluation of randomized controlled trial in ten nursing journals

BACKGROUND

As a practice-oriented discipline, strict adherence to reporting guidelines is particularly important in randomized controlled trial (RCT) abstracts of the nursing area. However, whether abstract reports after 2010 have complied with the Consolidated Standards of Reporting Trials for Abstracts (CONSORT-A) guideline is unclear. This study aimed to evaluate whether the publication of CONSORT-A has improved abstract reporting in nursing and explores the factors associated with better adherence to the guidelines.

METHODS

We searched the Web of Science for 200 RCTs randomly selected from ten nursing journals. We used a data extraction form based on CONSORT-A, including 16 items, to analyze the reporting adherence to the guidelines, and the reporting rate of each item and the total score for each abstract were used to indicate adherence and overall quality score (OQS, range 0-16). A comparison of the total mean score between the two periods was made, and affecting factors were analyzed.

RESULTS

In the studies we included, 48 abstracts were published pre-CONSORT-A whereas 152 post-CONSORT-A. The overall mean score for reporting adherence to 16 items was 7.41 ± 2.78 and 9.16 ± 2.76 for pre- and post-CONSORT-A, respectively (total score: 16). The most poorly reported items are "harms (0%)," "outcomes in method (8.5%)," "randomization (25%)," and "blinding (6.5%)." Items including the year of publication, impact factor, multiple center trial, word count, and structured abstract are significantly associated with higher adherence.

CONCLUSIONS

The adherence to abstract reporting in nursing literature has improved since the CONSORT-A era, but the overall completeness of RCT abstracts remained low. A joint effort by authors, editors, and journals is necessary to improve reporting quality of RCT abstracts.

Reminding Peer Reviewers of Reporting Guideline Items to Improve Completeness in Published Articles: Primary Results of 2 Randomized Trials

Importance

Numerous studies have shown that adherence to reporting guidelines is suboptimal.

Objective

To evaluate whether asking peer reviewers to check if specific reporting guideline items were adequately reported would improve adherence to reporting guidelines in published articles.

Design, Setting, and Participants

Two parallel-group, superiority randomized trials were performed using manuscripts submitted to 7 biomedical journals (5 from the BMJ Publishing Group and 2 from the Public Library of Science) as the unit of randomization, with peer reviewers allocated to the intervention or control group.

Interventions

The first trial (CONSORT-PR) focused on manuscripts that presented randomized clinical trial (RCT) results and reported following the Consolidated Standards of Reporting Trials (CONSORT) guideline, and the second trial (SPIRIT-PR) focused on manuscripts that presented RCT protocols and reported following the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guideline. The CONSORT-PR trial included manuscripts that described RCT primary results (submitted July 2019 to July 2021). The SPIRIT-PR trial included manuscripts that contained RCT protocols (submitted June 2020 to May 2021). Manuscripts in both trials were randomized (1:1) to the intervention or control group; the control group received usual journal practice. In the intervention group of both trials, peer reviewers received an email from the journal that asked them to check whether the 10 most important and poorly reported CONSORT (for CONSORT-PR) or SPIRIT (for SPIRIT-PR) items were adequately reported in the manuscript. Peer reviewers and authors were not informed of the purpose of the study, and outcome assessors were blinded.

Main Outcomes and Measures

The difference in the mean proportion of adequately reported 10 CONSORT or SPIRIT items between the intervention and control groups in published articles.

Results

In the CONSORT-PR trial, 510 manuscripts were randomized. Of those, 243 were published (122 in the intervention group and 121 in the control group). A mean proportion of 69.3% (95% CI, 66.0%-72.7%) of the 10 CONSORT items were adequately reported in the intervention group and 66.6% (95% CI, 62.5%-70.7%) in the control group (mean difference, 2.7%; 95% CI, -2.6% to 8.0%). In the SPIRIT-PR trial, of the 244 randomized manuscripts, 178 were published (90 in the intervention group and 88 in the control group). A mean proportion of 46.1% (95% CI, 41.8%-50.4%) of the 10 SPIRIT items were adequately reported in the intervention group and 45.6% (95% CI, 41.7% to 49.4%) in the control group (mean difference, 0.5%; 95% CI, -5.2% to 6.3%).

Conclusions and Relevance

These 2 randomized trials found that it was not useful to implement the tested intervention to increase reporting completeness in published articles. Other interventions should be assessed and considered in the future.

Trial Registration

ClinicalTrials.gov Identifiers: NCT05820971 (CONSORT-PR) and NCT05820984 (SPIRIT-PR).

[The PRISMA 2020 statement: an updated guideline for reporting systematic reviewsDeclaración PRISMA 2020: una guía actualizada para la publicación de revisiones sistemáticas]

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews.

Peer Reviewed Evaluation of Registered End-Points of Randomised Trials (the PRE-REPORT study): a stepped wedge, cluster-randomised trial

OBJECTIVE

To test whether providing relevant clinical trial registry information to peer reviewers evaluating trial manuscripts decreases discrepancies between registered and published trial outcomes.

DESIGN

Stepped wedge, cluster-randomised trial, with clusters comprised of eligible manuscripts submitted to each participating journal between 1 November 2018 and 31 October 2019.

SETTING

Thirteen medical journals.

PARTICIPANTS

Manuscripts were eligible for inclusion if they were submitted to a participating journal during the study period, presented results from the primary analysis of a clinical trial, and were peer reviewed.

INTERVENTIONS

During the control phase, there were no changes to pre-existing peer review practices. After journals crossed over into the intervention phase, peer reviewers received a data sheet describing whether trials were registered, the initial registration and enrolment dates, and the registered primary outcome(s) when enrolment began.

MAIN OUTCOME MEASURE

The presence of a clearly defined, prospectively registered primary outcome consistent with the primary outcome in the published trial manuscript, as determined by two independent outcome assessors.

RESULTS

We included 419 manuscripts (243 control and 176 intervention). Participating journals published 43% of control-phase manuscripts and 39% of intervention-phase manuscripts (model-estimated percentage difference between intervention and control trials = -10%, 95% CI -25% to 4%). Among the 173 accepted trials, published primary outcomes were consistent with clearly defined, prospectively registered primary outcomes in 40 of 105 (38%) control-phase trials and 27 of 68 (40%) intervention-phase trials. A linear mixed model did not show evidence of a statistically significant primary outcome effect from the intervention (estimated difference between intervention and control=-6% (90% CI -27% to 15%); one-sided p value=0.68).

CONCLUSIONS

These results do not support use of the tested intervention as implemented here to increase agreement between prospectively registered and published trial outcomes. Other approaches are needed to improve the quality of outcome reporting of clinical trials.

TRIAL REGISTRATION NUMBER

ISRCTN41225307.

Did an introduction of CONSORT for abstracts guidelines improve reporting quality of randomised controlled trials' abstracts on Helicobacter pylori infection? Observational study

OBJECTIVE

To determine abstracts' adherence to the Consolidated Standards of Reporting Trials for Abstracts (CONSORT-A) statement and to explore the factors associated with reporting quality.

DESIGN

An observational study.

SETTING

Abstracts of randomised controlled trials published between 2010 and 2019, found searching the MEDLINE database.

PARTICIPANTS

A total of 451 abstracts of the clinical trials on Helicobacter pylori infections were included.

PRIMARY AND SECONDARY OUTCOME MEASURES

Abstracts' reporting quality was determined by assessing their adherence to 17-item CONSORT-A checklist, with overall score being calculated as the sum of items that were adequately reported for each abstract. Additional factors that might influence the reporting quality of the abstracts were analysed, with univariate and multivariate linear regression used to determine how those factors influenced the overall reporting quality.

RESULTS

Included abstracts had an overall median quality score of 8/17 (IQR 7-9). Large proportions of abstracts adequately reported interventions, participants, objectives, numbers randomised and conclusions (97.1, 99.3, 89.1. 94.7 and 98.4% of abstracts, respectively). Trial design, randomisation, blinding and funding were severely under-reported with only 8.0, 2.7, 11.0 and 2.0% of abstracts reporting each item. Overall quality scores for H. pylori abstracts were higher in association with CONSORT-A endorsement (B=5.698; 95% CI 1.781 to 9.615), pharmacological interventions (B=4.063; 95% CI 0.224 to 7.902), multicentre settings (B=5.057; 95% CI 2.370 to 7.743), higher numbers of participants (B=3.607; 95% CI 1.272 to 5.942), hospital settings (B=4.827; 95% CI 1.753 to 7.901) and longer abstracts (B=3.878; 95% CI 0.787 to 6.969 for abstracts with 251-300 words and B=7.404; 95% CI 3.930 to 10.878 for abstracts with more than 300 words).

CONCLUSIONS

The overall reporting quality of abstracts was inadequate. The endorsement of CONSORT-A guidelines by more journals might improve the standards of reporting.

A Scoping Review of Four Decades of Outcomes in Nonsurgical Root Canal Treatment, Nonsurgical Retreatment, and Apexification Studies: Part 3-A Proposed Framework for Standardized Data Collection and Reporting of Endodontic Outcome Studies

INTRODUCTION

Despite initiatives to standardize and improve reporting of rapidly growing endodontic outcome research studies, issues related to missing and ambiguous information are still of great concern. In this article, we propose a framework for standardized data collection and a compiled checklist for reporting of various study designs on endodontic outcome.

METHODS

A comprehensive search was carried out to locate randomized controlled trials, cohorts, case-control studies, or case series of >100 patients that reported on endodontic outcomes. We reviewed these articles to develop a Data Collection Template and compiled a checklist for reporting of future endodontic outcome research.

RESULTS

Out of 354 eligible articles previously reported in our scoping review on endodontic outcome studies, 109 articles were selected and screened for study variables or levels of categorization. Our complied Data Collection Template was developed in 19 domains to highlight important demographic, preoperative, intraoperative, and postoperative variables. Because of the specific needs for endodontic outcome literature, we also proposed a compiled checklist (consisting of 4 main domains) to facilitate the reporting of various study designs on endodontic outcome studies. This checklist included simple descriptions of the required items and examples on reporting from published endodontic studies.

CONCLUSIONS

By facilitating the collection and reporting of relevant research data by investigators in private practice and academia, we hope that the proposed Data Collection Template and reporting guideline can highlight the importance of standardization among clinicians and researchers while producing valid scientific information that will support evidence-based treatment decisions.

The PRISMA 2020 statement: an updated guideline for reporting systematic reviews

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews. Full English text available from:www.revespcardiol.org/en.

Development of a checklist to detect errors in meta-analyses in systematic reviews of interventions: study protocol

Background : Systematic reviews underpin clinical practice and policies that guide healthcare decisions. A core component of many systematic reviews is meta-analysis, which is a statistical synthesis of results across studies. Errors in the conduct and interpretation of meta-analysis can lead to incorrect conclusions regarding the benefits and harms of interventions; and studies have shown that these errors are common. Enabling peer reviewers to better detect errors in meta-analysis through the use of a checklist provides an opportunity for these errors to be rectified before publication. To our knowledge, no such checklist exists. Objective : To develop and evaluate a checklist to detect errors in pairwise meta-analyses in systematic reviews of interventions. Methods : We will undertake a four-step process to develop the checklist. First, we will undertake a systematic review of studies that have evaluated errors in the conduct and interpretation of meta-analysis to generate a bank of items to consider for the checklist. Second, we will undertake a survey of systematic review methodologists and statisticians to seek their views on which items, of the bank of items generated in step 1, are most important to include in the checklist. Third, we will hold a virtual meeting to agree upon which items to include in the checklist. Fourth, before finalising the checklist, we will pilot with editors and peer reviewers of journals. Conclusion : The developed checklist is intended to help journal editors and peer reviewers identify errors in the application and interpretation of meta-analyses in systematic reviews. Fewer errors in the conduct and improved interpretation will lead to more accurate review findings and conclusions to inform clinical practice.

A scoping review on biomedical journal peer review guides for reviewers

Background

Peer review is widely used in academic fields to assess a manuscript’s significance and to improve its quality for publication. This scoping review will assess existing peer review guidelines and/or checklists intended for reviewers of biomedical journals and provide an overview on the review guidelines.

Methods

PubMed, Embase, and Allied and Complementary Medicine (AMED) databases were searched for review guidelines from the date of inception until February 19, 2021. There was no date restriction nor article type restriction. In addition to the database search, websites of journal publishers and non-publishers were additionally hand-searched.

Results

Of 14,633 database publication records and 24 website records, 65 publications and 14 websites met inclusion criteria for the review (78 records in total). From the included records, a total of 1,811 checklist items were identified. The items related to Methods, Results, and Discussion were found to be the highly discussed in reviewer guidelines.

Conclusion

This review identified existing literature on peer review guidelines and provided an overview of the current state of peer review guides. Review guidelines were varying by journals and publishers. This calls for more research to determine the need to use uniform review standards for transparent and standardized peer review.

Protocol registration

The protocol for this study has been registered at Research Registry (www.researchregistry.com): reviewregistry881.

The PRISMA 2020 statement: An updated guideline for reporting systematic reviews

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews.

The PRISMA 2020 statement: an updated guideline for reporting systematic reviews

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews.

The PRISMA 2020 statement: An updated guideline for reporting systematic reviews

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews.

The PRISMA 2020 statement: An updated guideline for reporting systematic reviews

Matthew Page and co-authors describe PRISMA 2020, an updated reporting guideline for systematic reviews and meta-analyses.