Reporting of planned statistical methods in published surgical randomised trial protocols: a protocol for a methodological systematic review

Describing the landscape of nutrition- and diet-related randomized controlled trials: metaresearch study of protocols published between 2012 and 2022

BACKGROUND

Publishing protocols promotes transparency and reproducibility. The scope and methods of protocols for nutrition- and diet-related randomized controlled trials (RCTs) have not been investigated yet.

OBJECTIVES

This study aims to map the landscape of nutrition- and diet-related interventions research.

METHODS

We conducted a metaresearch of nutrition-and diet-related RCT protocols published between January 2012 and March 2022, in any language, targeting human participants, evaluating nutrition interventions isolated or combined. A systematic search of the literature was conducted in 6 online databases. Bibliometric information, study characteristics, and research transparency practices data were collected from the included publications. The instructions for authors of journals with publications in our sample were checked for endorsement of reporting guidelines. Mentions to reporting guidelines in the included protocols were also checked.

RESULTS

Among the 62,319 records retrieved, 1068 were eligible. The number of published protocols increased annually, with a mean of 103 (range: 32-163) publications/y. Protocols were published in 148 journals, 50 of them (33.8%) endorsed Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT), 111 (75.3%) Consolidated Standards of Reporting Trials (CONSORT), and 4 (2.7%) Template for Intervention Description and Replication (TIDieR), whereas 343 (32.1%) protocols mentioned SPIRIT, 297 (27.8%) CONSORT, and 20 (1.9%) TIDieR. Most protocols reported the RCT registration number (n = 1006; 94.2%) and included statements about conflicts of interest (n = 952; 89.1%) and funding (n = 994; 93.2%). About two-thirds of protocols focused on adults or elderly participants (n = 677; 63.4%). Most protocols described 1 isolated nutrition- or diet-related intervention (n = 724; 67.8%), which were most frequently "supplementation, supplements or fortification" (n = 405; 37.9%) or "nutrition education, counseling or coordination of care" (n = 354; 33.1%). The most frequent primary outcomes reported were related to clinical status (n = 308; 28.8%).

CONCLUSIONS

The number of protocols for nutrition- or diet-related RCTs published is increasing, supporting the raising awareness and the importance of promoting these publications. The support and mention of relevant reporting guidelines by journals and researchers, respectively, remain far from ideal.

Reporting completeness of nutrition and diet-related randomised controlled trials protocols

BACKGROUND AND AIMS

There is a need to consolidate reporting guidance for nutrition randomised controlled trial (RCT) protocols. The reporting completeness in nutrition RCT protocols and study characteristics associated with adherence to SPIRIT and TIDieR reporting guidelines are unknown. We, therefore, assessed reporting completeness and its potential predictors in a random sample of published nutrition and diet-related RCT protocols.

METHODS

We conducted a meta-research study of 200 nutrition and diet-related RCT protocols published in 2019 and 2021 (aiming to consider periods before and after the start of the COVID pandemic). Data extraction included bibliometric information, general study characteristics, compliance with 122 questions corresponding to items and subitems in the SPIRIT and TIDieR checklists combined, and mention to these reporting guidelines in the publications. We calculated the proportion of protocols reporting each item and the frequency of items reported for each protocol. We investigated associations between selected publication aspects and reporting completeness using linear regression analysis.

RESULTS

The majority of protocols included adults and elderly as their study population (n = 73; 36.5%), supplementation as intervention (n = 96; 48.0%), placebo as comparator (n = 89; 44.5%), and evaluated clinical status as the outcome (n = 80; 40.0%). Most protocols described a parallel RCT (n = 188; 94.0%) with a superiority framework (n = 141; 70.5%). Overall reporting completeness was 52.0% (SD = 10.8%). Adherence to SPIRIT items ranged from 0% (n = 0) (data collection methods) to 98.5% (n = 197) (eligibility criteria). Adherence to TIDieR items ranged from 5.5% (n = 11) (materials used in the intervention) to 98.5% (n = 197) (description of the intervention). The multivariable regression analysis suggests that a higher number of authors [β = 0.53 (95%CI: 0.28-0.78)], most recent published protocols [β = 3.19 (95%CI: 0.24-6.14)], request of reporting guideline checklist during the submission process by the journal [β = 6.50 (95%CI: 2.56-10.43)] and mention of SPIRIT by the authors [β = 5.15 (95%CI: 2.44-7.86)] are related to higher reporting completeness scores.

CONCLUSIONS

Reporting completeness in a random sample of 200 diet or nutrition-related RCT protocols was low. Number of authors, year of publication, self-reported adherence to SPIRIT, and journals' endorsement of reporting guidelines seem to be positively associated with reporting completeness in nutrition and diet-related RCT protocols.

Recruitment into randomised trials of arteriovenous grafts: A systematic review

Although randomised controlled trials (RCT) are considered the optimal form of evidence, there are relatively few in surgery. Surgical RCT are particularly likely to be discontinued with poor recruitment cited as a leading reason. Surgical RCT present challenges over and above those seen in drug trials as the treatment under study may vary between procedures, between surgeons in one unit, and between units in multi-centred RCT. The most contentious and debated area of vascular access remains the role of arteriovenous grafts, and thus the quality of the data that is used to support opinions, guidelines and recommendations is critical. The aim of this review was to determine the extent of variation in the planning and recruitment in all RCT involving AVG. The findings of this are stark: there have been only 31 RCT performed in 31 years, the vast majority of which exhibited major limitations severe enough to undermine the results. This underlines the need for better quality RCT and data, and further inform the design of future studies. Perhaps most fundamental is the planning for a RCT that accounts for the intended population, the uptake of a RCT and the attrition for the significant co-morbidity in this population.

Prediction model protocols indicate better adherence to recommended guidelines for study conduct and reporting

BACKGROUND AND OBJECTIVE

Protocols are invaluable documents for any research study, especially for prediction model studies. However, the mere existence of a protocol is insufficient if key details are omitted. We reviewed the reporting content and details of the proposed design and methods reported in published protocols for prediction model research.

METHODS

We searched MEDLINE, Embase, and the Web of Science Core Collection for protocols for studies developing or validating a diagnostic or prognostic model using any modeling approach in any clinical area. We screened protocols published between Jan 1, 2022 and June 30, 2022. We used the abstract, introduction, methods, and discussion sections of The Transparent Reporting of a multivariable prediction model of Individual Prognosis Or Diagnosis (TRIPOD) statement to inform data extraction.

RESULTS

We identified 30 protocols, of which 28 were describing plans for model development and six for model validation. All protocols were open access, including a preprint. 15 protocols reported prospectively collecting data. 21 protocols planned to use clustered data, of which one-third planned methods to account for it. A planned sample size was reported for 93% development and 67% validation analyses. 16 protocols reported details of study registration, but all protocols reported a statement on ethics approval. Plans for data sharing were reported in 13 protocols.

CONCLUSION

Protocols for prediction model studies are uncommon, and few are made publicly available. Those that are available were reasonably well-reported and often described their methods following current prediction model research recommendations, likely leading to better reporting and methods in the actual study.

Assessment of 30 Years of Randomized Controlled Trials in The American Journal of Sports Medicine: 1990-2020

Background

Randomized controlled trials (RCTs) stand atop the evidence-based hierarchy of study designs for their ability to arrive at results with the lowest risk of bias. Even for RCTs, however, critical appraisal is essential before applying results to clinical practice.

Purpose

To analyze the quality of reporting of RCTs published in The American Journal of Sports Medicine (AJSM) from 1990 to 2020 and to identify trends over time and areas of improvement for future trials.

Study Design

Systematic review; Level of evidence, 1.

Methods

We queried the AJSM database for RCTs published between January 1990 and December 2020. Data pertaining to study characteristics were recorded. Quality assessments were conducted using the Detsky quality-of-reporting index and the modified Cochrane risk-of-bias (mROB) tool. Univariate and multivariable models were generated to establish factors with associations to study quality. The Fragility Index was calculated for eligible studies.

Results

A total of 277 RCTs were identified with a median sample size of 70 patients. A total of 19 RCTs were published between 1990 and 2000 (t₁); 82 RCTs between 2001 and 2010 (t₂); and 176 RCTs between 2011 and 2020 (t₃). From t₁ to t₃, significant increases were observed in the overall mean-transformed Detsky score (from 68.2% ± 9.8% to 87.4% ± 10.2%, respectively; P < .001) and mROB score (from 4.7 ± 1.6 to 6.9 ± 1.6, respectively; P < .001). Multivariable regression analysis revealed that trials with follow-up periods of <5 years clearly stated primary outcomes, and a focus on the elbow, shoulder, or knee were associated with higher mean-transformed Detsky and mROB scores. The median Fragility Index was 2 (interquartile range, 0-5) for trials with statistically significant. Studies with small sample sizes (<100 patients) were more likely to have low Fragility Index scores and less likely to have a statistically significant finding in any outcome.

Conclusion

The quantity and quality of published RCTs published in AJSM increased over the past 3 decades. However, single-center trials with small sample sizes were prone to fragile results.

Protocol for a meta-research study of protocols for diet or nutrition-related trials published in indexed journals: general aspects of study design, rationale and reporting limitations

INTRODUCTION

The Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) reporting guideline establishes a minimum set of items to be reported in any randomised controlled trial (RCT) protocol. The Template for Intervention Description and Replication (TIDieR) reporting guideline was developed to improve the reporting of interventions in RCT protocols and results papers. Reporting completeness in protocols of diet or nutrition-related RCTs has not been systematically investigated. We aim to identify published protocols of diet or nutrition-related RCTs, assess their reporting completeness and identify the main reporting limitations remaining in this field.

METHODS AND ANALYSIS

We will conduct a meta-research study of RCT protocols published in journals indexed in at least one of six selected databases between 2012 and 2022. We have run a search in PubMed, Embase, CINAHL, Web of Science, PsycINFO and Global Health using a search strategy designed to identify protocols of diet or nutrition-related RCTs. Two reviewers will independently screen the titles and abstracts of records yielded by the search in Rayyan. The full texts will then be read to confirm protocol eligibility. We will collect general study features (publication information, types of participants, interventions, comparators, outcomes and study design) of all eligible published protocols in this contemporary sample. We will assess reporting completeness in a randomly selected sample of them and identify their main reporting limitations. We will compare this subsample with the items in the SPIRIT and TIDieR statements. For all data collection, we will use data extraction forms in REDCap. This protocol is registered on the Open Science Framework (DOI: 10.17605/OSF.IO/YWEVS).

ETHICS AND DISSEMINATION

This study will undertake a secondary analysis of published data and does not require ethical approval. The results will be disseminated through journals and conferences targeting stakeholders involved in nutrition research.

Reporting quality of clinical trial protocols: a repeated cross-sectional study about the Adherence to SPIrit Recommendations in Switzerland, CAnada and GErmany (ASPIRE-SCAGE)

OBJECTIVES

Comprehensive protocols are key for the planning and conduct of randomised clinical trials (RCTs). Evidence of low reporting quality of RCT protocols led to the publication of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) checklist in 2013. We aimed to examine the quality of reporting of RCT protocols from three countries before and after the publication of the SPIRIT checklist.

DESIGN

Repeated cross sectional study.

SETTING

Swiss, German and Canadian research ethics committees (RECs).

PARTICIPANTS

RCT protocols approved by RECs in 2012 (n=257) and 2016 (n=292).

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcomes were the proportion of reported SPIRIT items per protocol and the proportion of trial protocols reporting individual SPIRIT items. We compared these outcomes in protocols approved in 2012 and 2016, and built regression models to explore factors associated with adherence to SPIRIT. For each protocol, we also extracted information on general trial characteristics and assessed whether individual SPIRIT items were reported RESULTS: The median proportion of reported SPIRIT items among RCT protocols showed a non-significant increase from 72% (IQR, 63%-79%) in 2012 to 77% (IQR, 68%-82%) in 2016. However, in a preplanned subgroup analysis, we detected a significant improvement in investigator-sponsored protocols: the median proportion increased from 64% (IQR, 55%-72%) in 2012 to 76% (IQR, 64%-83%) in 2016, while for industry-sponsored protocols median adherence was 77% (IQR 72%-80%) for both years. The following trial characteristics were independently associated with lower adherence to SPIRIT: single-centre trial, no support from a clinical trials unit or contract research organisation, and investigator-sponsorship.

CONCLUSIONS

In 2012, industry-sponsored RCT protocols were reported more comprehensively than investigator-sponsored protocols. After publication of the SPIRIT checklist, investigator-sponsored protocols improved to the level of industry-sponsored protocols, which did not improve.

Outcomes reported in randomised controlled trials of major depressive disorder in older adults: protocol for a methodological review

INTRODUCTION

Major depressive disorder (MDD or depression) is prevalent among adults aged 65 years and older. The effectiveness and safety of interventions used to treat depression is often assessed through randomised controlled trials (RCTs). However, heterogeneity in the selection, measurement and reporting of outcomes in RCTs renders comparisons between trial results, interpretability and generalisability of findings challenging. There is presently no core outcome set (COS) for use in RCTs that assess interventions for older adults with MDD. We will conduct a methodological review of the literature for outcomes reported in trials for adults 65 years and older with depression to assess the heterogeneity of outcome measures.

METHODS AND ANALYSIS

RCTs evaluating pharmacotherapy, psychotherapy, or any other treatment intervention for older adults with MDD published in the last 10 years will be located using electronic database searches (MEDLINE, Embase, PsycINFO and the Cochrane Central Register of Controlled Trials). Reviewers will conduct title and abstract screening, full-text screening and data extraction of trials eligible for inclusion independently and in duplicate. Outcomes will be synthesised and mapped to core outcome-domain frameworks. We will summarise characteristics associated with trials and outcomes.

ETHICS AND DISSEMINATION

We hope that findings from our methodological review will reduce variability in outcome selection, measurement and reporting and facilitate the development of a COS for older adults with MDD. Our review will also inform evidence synthesis efforts in identifying the best treatment practices for this clinical population. Ethics approval is not required, as this study is a literature review.

PROSPERO REGISTRATION NUMBER

CRD42021244753.

Rationale and design of repeated cross-sectional studies to evaluate the reporting quality of trial protocols: the Adherence to SPIrit REcommendations (ASPIRE) study and associated projects

BACKGROUND

Clearly structured and comprehensive protocols are an essential component to ensure safety of participants, data validity, successful conduct, and credibility of results of randomized clinical trials (RCTs). Funding agencies, research ethics committees (RECs), regulatory agencies, medical journals, systematic reviewers, and other stakeholders rely on protocols to appraise the conduct and reporting of RCTs. In response to evidence of poor protocol quality, the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guideline was published in 2013 to improve the accuracy and completeness of clinical trial protocols. The impact of these recommendations on protocol completeness and associations between protocol completeness and successful RCT conduct and publication remain uncertain.

OBJECTIVES AND METHODS

Aims of the Adherence to SPIrit REcommendations (ASPIRE) study are to investigate adherence to SPIRIT checklist items of RCT protocols approved by RECs in the UK, Switzerland, Germany, and Canada before (2012) and after (2016) the publication of the SPIRIT guidelines; determine protocol features associated with non-adherence to SPIRIT checklist items; and assess potential differences in adherence across countries. We assembled an international cohort of RCTs based on 450 protocols approved in 2012 and 402 protocols approved in 2016 by RECs in Switzerland, the UK, Germany, and Canada. We will extract data on RCT characteristics and adherence to SPIRIT for all included protocols. We will use multivariable regression models to investigate temporal changes in SPIRIT adherence, differences across countries, and associations between SPIRIT adherence of protocols with RCT registration, completion, and publication of results. We plan substudies to examine the registration, premature discontinuation, and non-publication of RCTs; the use of patient-reported outcomes in RCT protocols; SPIRIT adherence of RCT protocols with non-regulated interventions; the planning of RCT subgroup analyses; and the use of routinely collected data for RCTs.

DISCUSSION

The ASPIRE study and associated substudies will provide important information on the impact of measures to improve the reporting of RCT protocols and on multiple aspects of RCT design, trial registration, premature discontinuation, and non-publication of RCTs observing potential changes over time.

An Assessment of Randomized Controlled Trial Quality in The Journal of Bone & Joint Surgery: Update from 2001 to 2013

BACKGROUND

The quality of reporting of randomized controlled trials (RCTs) published in The Journal of Bone & Joint Surgery (JBJS) from 1988 to 2000 was previously analyzed. The purpose of this current study was to analyze the quality of reporting of RCTs published in JBJS from 2001 to 2013 to identify trends over time and potential areas of improvement for future clinical trials.

METHODS

A manual search of the JBJS database identified RCTs published between January 2001 and December 2013. Quality assessments, using the Detsky quality-of-reporting index (Detsky score), a modified Cochrane risk-of-bias tool, and abstraction of relevant data identifying predictors of quality, were conducted.

RESULTS

A total of 5,780 publications were identified in JBJS from 2001 to 2013, with 285 RCTs (4.9%), representing an increase from the prior 13-year period. The overall mean transformed Detsky score (and standard error) increased significantly (p < 0.001) from 68.1% ± 1.67% to 76.24% ± 0.72%. The percentage of multicenter RCTs decreased from 67% to 31%. The percentage of positive trials also decreased from 80% to 50.5%, as did the mean sample size (212 to 166). Regression analysis indicated that trials with an epidemiologist as the first author and nonsurgical trials were significantly associated (p = 0.001) with a higher overall trial quality score. The categories of the lowest mean methodology scores were randomization and concealment, eligibility criteria, and reasons for patient exclusion, as identified with the Detsky score, and patient and assessor blinding, as identified with the risk-of-bias assessment.

CONCLUSIONS

The quantity and quality of published RCTs in JBJS have increased in the 2001 to 2013 time period compared with the previous time period. Although these improvements are encouraging, trends to smaller, single-center trials were also observed. To efficiently determine the efficacy of orthopaedic treatments and limit bias, high-quality randomized trials of appropriate sample size and rigorous design are needed.

Enhancing research publications and advancing scientific writing in health research collaborations: sharing lessons learnt from the trenches

BACKGROUND

Disseminating research protocols, processes, methods or findings via peer-reviewed publications has substantive merits and benefits to various stakeholders.

PURPOSE

In this article, we share strategies to enhance research publication contents (ie, what to write about) and to facilitate scientific writing (ie, how to write) in health research collaborations.

METHODS

Empirical experience sharing.

RESULTS

To enhance research publication contents, we encourage identifying appropriate opportunities for publications, publishing protocols ahead of results papers, seeking publications related to methodological issues, considering justified secondary analyses, and sharing academic process or experience. To advance writing, we suggest setting up scientific writing as a goal, seeking an appropriate mentorship, making full use of scientific meetings and presentations, taking some necessary formal training in areas such as effective communication and time and stress management, and embracing the iterative process of writing.

CONCLUSION

All the strategies we share are dependent upon each other; and they advocate gradual academic accomplishments through study and training in a "success-breeds-success" way. It is expected that the foregoing shared strategies in this paper, together with other previous guidance articles, can assist one with enhancing research publications, and eventually one's academic success in health research collaborations.