Evaluation of Journal Registration Policies and Prospective Registration of Randomized Clinical Trials of Nonregulated Health Care Interventions

Misleading presentations in functional food trials led by contract research organizations were frequently observed in Japan: meta-epidemiological study

OBJECTIVES

The functional food market has experienced significant growth, leading to an uptick in clinical trials conducted by contract research organizations (CROs). Research focusing on CRO-managed trials and the communication of trial outcomes to the consumer market remains underexplored. This metaepidemiological study aims to evaluate the quality of randomized controlled trials (RCTs) facilitated by prominent CROs in Japan and to examine the quality of the representations used to convey their results to consumers.

STUDY DESIGN AND SETTING

This study focused on the food trials that were registered in the University Hospital Medical Information Network Clinical Trial Registry or the International Clinical Trials Registry Platform by the top 5 CROs. Press releases of study results or advertisements of food products based on the study results were identified by conducting a Google search. The risk of bias in the RCT publications was independently assessed by 2 reviewers, who also evaluated the presence of "spin" in the abstracts and full texts. An assessment of "spin" in press releases/advertisements was undertaken.

RESULTS

A total of 76 RCT registrations, 32 RCT publications, and 11 press releases/advertisements were included. Approximately 72% of the RCT publications exhibited a high risk of bias due to selective outcome reporting. "Spin" was present in the results of the abstract (72%), abstract conclusion (81%), full-text results (44%), and full-text conclusion (84%). "Spin" appeared in 73% of press releases/advertisements due to the selective outcome reporting.

CONCLUSION

Functional food presentations in Japan frequently contained "spin." The Japanese government should more rigorously check whether food manufacturers report outcomes selectively.

Is there 'trustworthy' evidence for using manual therapy to treat patients with shoulder dysfunction?: A systematic review

The primary objective of this review was to create a 'trustworthy,' living systematic review and meta-analysis for the application of manual therapy interventions in treating patients with shoulder dysfunction. Included studies were English-language randomized controlled trials published between 1/1/2010 and 8/3/2023, with searches performed in: PubMed, Cochrane Central Register of Controlled Trials (CENTRAL), CINHAL, ProQuest Nursing & Allied Health, EBSCO Medline, and PEDro. The population of focus included adults 18 years and older with musculoskeletal impairments related to shoulder dysfunction. Our primary outcomes included pain and region-specific outcome measures. We excluded trials, including participants having shoulder dysfunction resulting from surgery, radicular pain, instability/dislocation, fracture, lymphedema, and radiation. Our screening methodology was based upon a previously published 'trustworthy' systematic review protocol. This included the application of our PICOTS criteria in addition to screening for prospective clinical trial registration and following of prospective intent, as well as assessment of PEDro scores, risk-of-bias ratings, GRADE scoring, and examination of confidence in estimated effects. Twenty-six randomized controlled trials met our PICOTS criteria; however, only 15 of these were registered. Only three were registered prospectively. Two of these did not have discussions and conclusions that aligned with their primary outcome. The remaining single study was found to have a high risk-of-bias, meaning the remainder of the protocol could not be employed and that no randomized controlled trials could undergo further assessment or meta-analysis. The results of this systematic review indicate there are no 'trustworthy' randomized controlled trials examining the effectiveness of manual therapy interventions for the treatment of patients with shoulder dysfunction, as defined by the prospectively established methodology. Therefore, these findings signal that creating a 'trustworthy,' living systematic review on this clinically relevant topic is not yet possible due to a lack of 'trustworthy' randomized controlled trials.

Assessment of consistency between peer-reviewed publications and clinical trial registrations in nursing journals

BACKGROUND

The inconsistencies between randomized clinical trials (RCTs) registrations and peer-reviewed publications may distort trial results and threaten the validity of evidence-based medicine. Previous studies have found many inconsistencies between RCTs registrations and peer-reviewed publications, and outcome reporting bias is prevalent.

AIMS

The aims of this review were to assess whether the primary outcomes and other data reported in publications and registered records in RCTs of nursing journals were consistent and whether discrepancies in the reporting of primary outcomes favored statistically significant results. Moreover, we reviewed the proportion of RCTs for prospective registration.

METHODS

We systematically searched PubMed for RCTs published in the top 10 nursing journals between March 5, 2020, and March 5, 2022. Registration numbers were extracted from the publications, and registered records were identified from the registration platforms. The publications and registered records were compared to identify consistency. Inconsistencies were subdivided into discrepancies and omissions.

RESULTS

A total of 70 RCTs published in seven journals were included. The inconsistencies involved sample size estimation (71.4%), random sequence generation (75.7%), allocation concealment (97.1%), blinding (82.9%), primary outcomes (60.0%) and secondary outcomes (84.3%). Among the inconsistencies in the primary outcomes, 21.4% were due to discrepancies and 38.6% resulted from omissions. Fifty-three percent (8/15) presented discrepancies in the primary outcomes that favored statistically significant results. Additionally, although only 40.0% of the studies were prospective registrations, the number of prospectively registered trials has trended upward over time.

LINKING EVIDENCE TO ACTION

While not including all RCTs in the nursing field, our sample reflected a general trend: inconsistencies between publications and trial registrations were prevalent in the included nursing journals. Our research helps to provide a way to improve the transparency of research reports. Ensuring that clinical practice has access to transparent and reliable research results are essential to achieve the best possible evidence-based medicine.

Estimating the prevalence of discrepancies between study registrations and publications: a systematic review and meta-analyses

OBJECTIVES

Prospectively registering study plans in a permanent time-stamped and publicly accessible document is becoming more common across disciplines and aims to reduce risk of bias and make risk of bias transparent. Selective reporting persists, however, when researchers deviate from their registered plans without disclosure. This systematic review aimed to estimate the prevalence of undisclosed discrepancies between prospectively registered study plans and their associated publication. We further aimed to identify the research disciplines where these discrepancies have been observed, whether interventions to reduce discrepancies have been conducted, and gaps in the literature.

DESIGN

Systematic review and meta-analyses.

DATA SOURCES

Scopus and Web of Knowledge, published up to 15 December 2019.

ELIGIBILITY CRITERIA

Articles that included quantitative data about discrepancies between registrations or study protocols and their associated publications.

DATA EXTRACTION AND SYNTHESIS

Each included article was independently coded by two reviewers using a coding form designed for this review (osf.io/728ys). We used random-effects meta-analyses to synthesise the results.

RESULTS

We reviewed k=89 articles, which included k=70 that reported on primary outcome discrepancies from n=6314 studies and, k=22 that reported on secondary outcome discrepancies from n=1436 studies. Meta-analyses indicated that between 29% and 37% (95% CI) of studies contained at least one primary outcome discrepancy and between 50% and 75% (95% CI) contained at least one secondary outcome discrepancy. Almost all articles assessed clinical literature, and there was considerable heterogeneity. We identified only one article that attempted to correct discrepancies.

CONCLUSIONS

Many articles did not include information on whether discrepancies were disclosed, which version of a registration they compared publications to and whether the registration was prospective. Thus, our estimates represent discrepancies broadly, rather than our target of undisclosed discrepancies between prospectively registered study plans and their associated publications. Discrepancies are common and reduce the trustworthiness of medical research. Interventions to reduce discrepancies could prove valuable.

REGISTRATION

osf.io/ktmdg. Protocol amendments are listed in online supplemental material A.

Credibility at stake: only two-thirds of randomized trials of nutrition interventions are registered and lack transparency in outcome and treatment effect definitions

OBJECTIVES

This study aimed to investigate the adherence of randomized controlled trials of nutrition interventions to transparency practices informing assessments of selective reporting biases, including the availability of a trial registration entry, protocol and statistical analysis plan (SAP).

STUDY DESIGN AND SETTING

Retrospective observational study with cross-sectional design. We systematically searched for trials published from 1 July 2019, to 30 June 2020, and included a randomly selected sample of 400 studies. We searched for registry entries, protocols, and SAPs for all included studies. We extracted data to characterize the disclosure of sufficient information in the available materials to inform assessments of selective reporting biases, considering the definition of outcome domain, measure, metric, method of aggregation, time point, analysis population, methods to handle missing data and method of adjustment.

RESULTS

Most trials (69%) were registered, but these often lacked sufficient specification of outcomes and intended treatment effects. Protocols and SAPs provided more details but were less often available (14% and 3%, respectively), and even then, almost all studies presented limited information to inform the assessments of risk of bias due to the selection of the reported result.

CONCLUSION

Lack of full specification of outcomes and intended treatment effects hinder a full adherence of randomized controlled trials of nutrition interventions to transparency practices and may affect their credibility.

Impact of searching clinical trials registers in systematic reviews of pharmaceutical and non-pharmaceutical interventions: Reanalysis of meta-analyses

Systematic reviewers are advised to search trials registers to minimise risk of reporting biases. However, there has been little research on the impact of searching trials registers on the results of meta-analyses. We aimed to evaluate the impact of searching clinical trials registers for systematic reviews of pharmaceutical or non-pharmaceutical interventions. We searched PubMed, Scopus, Science Citation Index and Social Sciences Citation Index, and Education Collection for systematic reviews with meta-analyses indexed from 2 November to 2 December 2020. A random sample of systematic reviews was initially drawn, and for reviews which considered randomised trials eligible for inclusion, which had not searched a trials register, we searched ClinicalTrials.gov, EudraCT, ANZCTR, and the WHO ICTRP search portal for eligible trials. We compared meta-analytic effect estimates before and after including results from additional trials identified. We found additional trials for 63% (63/101) of eligible reviews; however, trials with results that could contribute to a meta-analysis were identified for only 20% (20/101) of the reviews. On average, there was no difference in the meta-analytic effect estimates before versus after adding the new trials. In summary, searching clinical trial registers led to identification of additional trials for many reviews; however, very few trials had results available for inclusion in meta-analyses. Including results from the new trials led to no change in the meta-analytic estimates, on average. Trials registers would be even more valuable to systematic reviewers if more trialists made use of them (i.e., registered their trials and posted results in a timely manner).

A comparison of registered and published primary outcomes in clinical trials of opioid use disorder: ACTTION review and recommendations

BACKGROUND AND AIMS

Prospective trial registration can increase research integrity. This Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks (ACTTION) review was designed to compare the primary outcomes (PO) reported in registries with associated publications for opioid use disorder (OUD) clinical trials.

DESIGN

The World Health Organization's International Clinical Trials Registry Platform (ICTRP) was searched for completed trials (2010 through 2019). Associated publications were identified and paired with trial registry data based on the publication date.

MEASUREMENTS

Reviewers independently rated the occurrence of discrepancies between the POs in the registry compared to the publication. An analysis of prospective versus retrospective registration was also completed.

FINDINGS

One-hundred and forty trials were identified in the search, and 43 registry-publication pairs evaluated. Only 34 of the 43 pairs could be examined for discrepancies because nine did not report a PO in registry and publication. Of the 34 pairs, only four met rigorous criteria for prospective trial registration and had an exact match of POs. In contrast, the majority of the 34 trials, or 80%, had inconsistent POs (e.g., registered secondary outcomes published as primary; the timing of PO not specified) and/or were retrospectively registered.

CONCLUSIONS

Many clinical trials focused on OUD have not met the standards of trial registration, such as consistent reporting of POs and prospective registration. Failure to properly register trial characteristics undermines the validity of research findings and can delay the development of life-saving treatments. Recommendations for improving prospective trial reporting practices are provided.

Effects of mental health interventions among people hospitalized with COVID-19 infection: A systematic review of randomized controlled trials

OBJECTIVE

We evaluated the effects of mental health interventions among people hospitalized with COVID-19.

METHODS

We conducted a systematic review and searched 9 databases (2 Chinese-language) from December 31, 2019 to June 28, 2021. Eligible randomized controlled trials assessed interventions among hospitalized COVID-19 patients that targeted mental health symptoms. Due to the poor quality of trials, we sought to verify accuracy of trial reports including results.

RESULTS

We identified 47 randomized controlled trials from China (N = 42), Iran (N = 4) and Turkey (N = 1) of which 21 tested the efficacy of psychological interventions, 5 physical and breathing exercises, and 21 a combination of interventions. Trial information could only be verified for 3 trials of psychological interventions (cognitive behavioral, guided imagery, multicomponent online), and these were the only trials with low risk of bias on at least 4 of 7 domains. Results could not be pooled or interpreted with confidence due to the degree of poor reporting and trial quality, the frequency of what were deemed implausibly large effects, and heterogeneity.

CONCLUSION

Trials of interventions to address mental health in hospitalized COVID-19 patients, collectively, are not of sufficient quality to inform practice. Health care providers should refer to existing expert recommendations and standard hospital-based practices.

REGISTRATION

PROSPERO (CRD42020179703); registered on April 17, 2020.

A comparative analysis of important public clinical trial registries, and a proposal for an interim ideal one

Background

It is an ethical and scientific obligation to register each clinical trial, and report its results, accurately, comprehensively and on time. The WHO recognizes 17 public registries as Primary Registries, and has also introduced a set of minimal standards in the International Standards for Clinical Trial Registries (ISCTR) that primary registries need to implement. These standards are categorized into nine sections—Content, Quality and Validity, Accessibility, Unambiguous Identification, Technical Capacity, Administration and Governance, the Trial Registration Data Set (TRDS), Partner registries and Data Interchange Standards. This study compared the WHO’s primary registries, and the US’s ClinicalTrials.gov, to examine the implementation of ISCTR, with the aim of defining features of an interim ideal registry.

Methods and findings

The websites of the 18 registries were evaluated for 14 features that map to one or more of the nine sections of ISCTR, and assigned scores for their variations of these features. The assessed features include the nature of the content; the number and nature of fields to conduct a search; data download formats; the nature of the audit trail; the health condition category; the documentation available on a registry website; etc. The registries received scores for their particular variation of a given feature based on a scoring rationale devised for each individual feature analysed. Overall, the registries received between 27% and 80% of the maximum score of 94. The results from our analysis were used to define a set of features of an interim ideal registry.

Conclusions

To the best of our knowledge, this is the first study to quantify the widely divergent quality of the primary registries’ compliance with the ISCTR. Even with this limited assessment, it is clear that some of the registries have much work to do, although even a few improvements would significantly improve them.

Steps toward preregistration of research on research integrity

BACKGROUND

A proposal to encourage the preregistration of research on research integrity was developed and adopted as the Amsterdam Agenda at the 5th World Conference on Research Integrity (Amsterdam, 2017). This paper reports on the degree to which abstracts of the 6th World Conference in Research Integrity (Hong Kong, 2019) reported on preregistered research.

METHODS

Conference registration data on participants presenting a paper or a poster at 6th WCRI were made available to the research team. Because the data set was too small for inferential statistics this report is limited to a basic description of results and some recommendations that should be considered when taking further steps to improve preregistration.

RESULTS

19% of the 308 presenters preregistered their research. Of the 56 usable cases, less than half provided information on the six key elements of the Amsterdam Agenda. Others provided information that invalidated their data, such as an uninformative URL. There was no discernable difference between qualitative and quantitative research.

CONCLUSIONS

Some presenters at the WCRI have preregistered their research on research integrity, but further steps are needed to increase frequency and completeness of preregistration. One approach to increase preregistration would be to make it a requirement for research presented at the World Conferences on Research Integrity.

Study designs and statistical approaches to suicide and prevention research in real-world data

OBJECTIVE

To provide researchers, clinicians and policy makers with a primer to study designs, statistical approaches and graphical reporting methods for suicide research in real world data (RWD).

METHODS

Study designs, statistical method and graphical reporting standards are detailed with examples from the recently published literature.

RESULTS

Data sources and codes for identifying suicidal behavior are described. Study designs are described in detail for post-market surveillance, retrospective cohort studies, case control and nested case-control studies, and self-controlled (within-individual) studies including applications of marginal structural models. Graphical reporting of designs is described using an original research study.

CONCLUSIONS

Compared to RCTs, RWE studies offer larger sample sizes, greater generalizability, and real-world validity. However, these non-experimental data risk uncontrolled confounding and potential introduction of bias unless data, design and statistical approaches are rigorously aligned.

Group sample sizes in nonregulated health care intervention trials described as randomized controlled trials were overly similar

OBJECTIVES

We evaluated whether sample sizes in different arms of two-arm parallel group randomized controlled trials of nonregulated interventions were systematically closer in size than would plausibly occur by chance if simple randomization had been applied.

STUDY DESIGN AND SETTING

We searched PubMed for trials of nonregulated health care interventions that did not report using restricted randomization from journals in behavioral sciences and psychology, nursing, nutrition and dietetics, rehabilitation, and surgery. We emailed trial authors to clarify randomization procedures.

RESULTS

We identified 148 nonregulated intervention trials that indicated they used simple randomization. Difference in trial arm sizes was smaller than would be predicted by chance if simple randomization had occurred in all trials (P < 0.001). Rather than approximately half of the trials being within a 50% prediction interval for the difference, 96% had differences within this interval. Results were similar and statistically significant (P < 0.001) for trials that were published in journals with impact factors ≥ 4 and when stratified by type of nonregulated intervention.

CONCLUSION

There is a need for education and better understanding of clinical trial methods to ensure that randomization procedures are implemented as intended and reported fully and accurately.

Discrepancies in the Registries of Diet vs Drug Trials

This cross-sectional study examines discrepancies between registered protocols and subsequent publications for drug and diet trials whose findings were published in prominent clinical journals in the last decade.

Investigation of Risk Of Bias due to Unreported and SelecTively included results in meta-analyses of nutrition research: the ROBUST study protocol

Background: Dietary guidelines should be informed by systematic reviews (SRs) of the available scientific evidence. However, if the SRs that underpin dietary guidelines are flawed in their design, conduct or reporting, the recommendations contained therein may be misleading or harmful. To date there has been little empirical investigation of bias due to selective inclusion of results, and bias due to missing results, in SRs of food/diet-outcome relationships. Objectives: To explore in SRs with meta-analyses of the association between food/diet and health-related outcomes: (i) whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available; (ii) what impact selective inclusion of study effect estimates may have on meta-analytic effects, and; (iii) the risk of bias due to missing results (publication bias and selective non-reporting bias) in meta-analyses. Methods: We will systematically search for SRs with meta-analysis of the association between food/diet and health-related outcomes in a generally healthy population, published between January 2018 and June 2019. We will randomly sort titles and abstracts and screen them until we identify 50 eligible SRs. The first reported meta-analysis of a binary or continuous outcome in each SR (the 'index meta-analysis') will be evaluated. We will extract from study reports all study effect estimates that were eligible for inclusion in the index meta-analyses (e.g. from multiple instruments and time points) and will quantify and test for evidence of selective inclusion of results. We will also assess the risk of bias due to missing results in the index meta-analyses using a new tool (ROB-ME). Ethics and dissemination: Ethics approval is not required because information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. We will make all data collected from this study publicly available via the Open Science Framework.

Investigation of Risk Of Bias due to Unreported and SelecTively included results in meta-analyses of nutrition research: the ROBUST study protocol

Background: Dietary guidelines should be informed by systematic reviews (SRs) of the available scientific evidence. However, if the SRs that underpin dietary guidelines are flawed in their design, conduct or reporting, the recommendations contained therein may be misleading or harmful. To date there has been little empirical investigation of bias due to selective inclusion of results, and bias due to missing results, in SRs of food/diet-outcome relationships. Objectives: To explore in SRs with meta-analyses of the association between food/diet and health-related outcomes: (i) whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available; (ii) what impact selective inclusion of study effect estimates may have on meta-analytic effects, and; (iii) the risk of bias due to missing results (publication bias and selective non-reporting bias) in meta-analyses. Methods: We will systematically search for SRs with meta-analysis of the association between food/diet and health-related outcomes in a generally healthy population, published between January 2018 and June 2019. We will randomly sort titles and abstracts and screen them until we identify 50 eligible SRs. The first reported meta-analysis of a binary or continuous outcome in each SR (the 'index meta-analysis') will be evaluated. We will extract from study reports all study effect estimates that were eligible for inclusion in the index meta-analyses (e.g. from multiple instruments and time points) and will quantify and test for evidence of selective inclusion of results. We will also assess the risk of bias due to missing results in the index meta-analyses using a new tool (ROB-ME). Ethics and dissemination: Ethics approval is not required because information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. We will make all data collected from this study publicly available via the Open Science Framework.

Adverse event reporting and trial registration in venous leg ulcer trials published since the 2001 CONSORT statement revision: A systematic review

AIM

To be in accord with the Consolidated Standards of Reporting Trials (CONSORT) Statement, all important adverse events in randomised controlled trials (RCTs) should be reported, as well as trial registration. Neither concern has been investigated in venous leg ulcer trials. We therefore aimed to quantify and explore compliance with adverse event reporting and trials registration in RCTs that reported interventions for treating venous leg ulceration.

MATERIALS AND METHODS

We searched the Cochrane Controlled Trials Register, Medline, Embase, and CINAHL for studies reported between 2001 and 2017. Included studies must have been described as randomised controlled trials evaluating any intervention in a VLU population. Data was then extracted by one author into a standard form and checked by a second author.

RESULTS

We screened 3100 titles and identified 204 trials involving pharmaceuticals (82), medicated and non-medicated devices (102), organisational (5) or other interventions (15) published in 76 journals. Eighty-four trials reported adverse events (41.2%), while 18 reported no events occurred (8.8%) and 78 did not report adverse events (38.2%). Types of adverse events reported included all-cause (20.1%), ulcer-related only (38.2%), treatment-related only (11.3%) and serious adverse events only (1.0%). Only 38 trials were registered (18.6%). Trial registration was associated with reporting of any adverse events (Odds Ratio 3.0, 95%CI 1.1-7.9), as was the trial being a pharmaceutical trial (Odds Ratio 2.9, 95%CI 1.5-5.7) or a multicentre trial (Odds Ratio 4.2, 95%CI 2.2-8.1).

CONCLUSION

Adverse event reporting in VLU trials is variable with about one third of trials not reporting on adverse events at all. Trials registration is a the modifiable factor associated with better reporting of adverse events. Journal editors could explore how they can promote trials registration to enhance better reporting of harms in VLU trials.