A systematic review of comparisons between protocols or registrations and full reports in primary biomedical research

Adherence to Registration and Selective Outcome Reporting in Randomized Clinical Trials Published in Endodontic Journals Over the Past 5 Years: A Meta-Research Study

INTRODUCTION

Prospective registration of randomized clinical trials (RCTs) is highly recommended to ensure research transparency and prevent selective outcome reporting (SOR). This study aimed to evaluate the adherence to registration and the presence of SOR in RCTs published in endodontic journals over the past 5 years.

METHODS

Electronic searches were conducted in PubMed and the libraries of the Journal of Endodontics, International Endodontic Journal, European Endodontic Journal, and Australian Endodontic Journal. Two reviewers were involved in the study selection and evaluation. Publications were assessed for key methodological aspects, including the presence and timing of trial registration. RCT registries were examined to identify discrepancies between publication and registered protocols and the presence of SOR. Logistic regression was used to explore the effect of study variables on registration practices and SOR.

RESULTS

Of the 144 RCTs included, 104 (72.2%) were registered. Among those registered, only 19 (18.3%) adhered to prospective registration. Registration practice increased by 53% per year (OR 1.53; 95% CI: 1.34-2.08; P < .01). Discrepancies between publication and protocol were identified in 55.8% of studies, primarily related to sample size (33.7%). SOR was identified in 41 trials (39.4%), mainly due to discrepancies in the outcome time frame (18.3%). Studies evaluating multiple outcomes had 4.95 times higher odds of exhibiting SOR (OR 4.95; 95% CI: 1.63-12.95; P < .01). Furthermore, studies that were registered retrospectively or exhibited discrepancies between publication and protocol accounted for 6.10 times (OR 6.10; 95% CI: 1.81-18.96; P = .03) and 5.61 times (OR 5.61; 95% CI: 2.93-16.58; P < .01) higher odds of exhibiting SOR, respectively.

CONCLUSIONS

RCTs published in endodontic journals over the past 5 years presented low adherence to prospective trial registration and a high prevalence of SOR.

Implementation of SAMPL Guidelines in 100 clinical medicine articles: enhancing statistical reporting and recommendations for biomedical journals

BACKGROUND

Contemporary observations indicate insufficient quality in the reporting of statistical data. Despite the publication of the SAMPL guidelines in 2015, they have not been widely adopted. The aim of this article is to highlight the incorporation of SAMPL Guidelines in the statistical reviews of articles related to clinical medicine, as well as the changes implemented by authors in revised manuscripts as a result of such guidance. An additional objective is to provide recommendations for biomedical journals regarding the necessity of integrating SAMPL Guidelines into their daily practices.

METHODS

The study incorporated 100 selected statistical reviews of original clinical medicine articles from 8 biomedical journals, conducted between 2016 and 2023. Each of these reviews suggested specific SAMPL Guidelines to be implemented in the revised manuscript. It was evaluated which specific SAMPL Guidelines were most frequently enforced and what changes resulted from their implementation.

RESULTS

Seventy-five percent of the manuscripts in question garnered acceptance following a solitary round of statistical evaluation. Among the most frequently recommended and subsequently implemented SAMPL Guidelines by the authors are a more thorough description of the purpose of the applied statistical tests (65%), indication of the practical significance of the obtained results, including calculation of relevant effect size measures (64%), analysis of assumptions necessary for the application of specific statistical tests (58%), and consideration of the impact of outlier values on the obtained results (34%).

CONCLUSION

To improve the quality of statistical reporting in biomedical journals, greater emphasis should be placed on implementing SAMPL Guidelines.

Reporting quality of acupuncture overviews: A methodological investigation based on the PRIOR statement

OBJECTIVE

Acupuncture overviews are increasing rapidly; however, their reporting quality is yet unclear. We aimed to investigate the reporting quality of relevant overviews according to the preferred reporting items for overviews of reviews (PRIOR) statement.

METHODS

We systematically searched PubMed from inception to August 16, 2022 for overviews on acupuncture therapies. Reporting quality of included overviews was evaluated using the PRIOR statement, and the results were cross-checked. Multiple linear regression analysis was used to assess the predictors of the reporting completeness. GraphPad 9.4 was utilized to generate an evidence map, Excel 2019 was used to extract and manage data, and R 4.2.3 was used for data analysis.

RESULTS

A total of 49 overviews published from 2006 to 2022 were included, of which China ranked first with 38 overviews. The most frequently searched database was PubMed/ Medline (n = 48, 98%), and commonly used methodological quality assessment tool was AMSTAR-2 (n = 14, 29%). The overarching themes centered on acupuncture for obstetrics, gynecology, reproductive diseases, as well as depression, anxiety, and insomnia. Reporting quality needs to be improved involving the definition of systematic reviews (SRs), overlap of primary studies and SRs, methods for managing discrepant data across SRs, risk of bias in primary studies, heterogeneity, and sensitivity analysis of synthesized results, reporting bias assessment, and registration and protocol. Moreover, publication in recent years and receiving funding support were significantly associated with higher overall reporting quality score (P < 0.05).

CONCLUSION

Based on the PRIOR statement, this methodological study indicates that the reporting quality of the included acupuncture overviews is poor. In the future, authors of overviews are encouraged to use the PRIOR statement for standardized reporting. Furthermore, it is recommended that journal editors mandate the inclusion of this statement in authors' reports and require a complete PRIOR checklist.

Exploring open science practices in behavioural public policy research

In their book 'Nudge: Improving Decisions About Health, Wealth and Happiness', Thaler & Sunstein (2009) argue that choice architectures are promising public policy interventions. This research programme motivated the creation of 'nudge units', government agencies which aim to apply insights from behavioural science to improve public policy. We closely examine a meta-analysis of the evidence gathered by two of the largest and most influential nudge units (DellaVigna & Linos (2022 Econometrica 90, 81-116 (doi:10.3982/ECTA18709))) and use statistical techniques to detect reporting biases. Our analysis shows evidence suggestive of selective reporting. We additionally evaluate the public pre-analysis plans from one of the two nudge units (Office of Evaluation Sciences). We identify several instances of excellent practice; however, we also find that the analysis plans and reporting often lack sufficient detail to evaluate (unintentional) reporting biases. We highlight several improvements that would enhance the effectiveness of the pre-analysis plans and reports as a means to combat reporting biases. Our findings and suggestions can further improve the evidence base for policy decisions.

Dissemination of Registered COVID-19 Clinical Trials (DIRECCT): a cross-sectional study

BACKGROUND

The results of clinical trials should be completely and rapidly reported during public health emergencies such as COVID-19. This study aimed to examine when, and where, the results of COVID-19 clinical trials were disseminated throughout the first 18 months of the pandemic.

METHODS

Clinical trials for COVID-19 treatment or prevention were identified from the WHO ICTRP database. All interventional trials with a registered completion date ≤ 30 June 2021 were included. Trial results, published as preprints, journal articles, or registry results, were located using automated and manual techniques across PubMed, Google Scholar, Google, EuropePMC, CORD-19, the Cochrane COVID-19 Study Register, and clinical trial registries. Our main analysis reports the rate of dissemination overall and per route, and the time from registered completion to results using Kaplan-Meier methods, with additional subgroup and sensitivity analyses reported.

RESULTS

Overall, 1643 trials with completion dates ranging from 46 to 561 days prior to the start of results searches were included. The cumulative probability of reporting was 12.5% at 3 months from completion, 21.6% at 6 months, and 32.8% at 12 months. Trial results were most commonly disseminated in journals (n = 278 trials, 69.2%); preprints were available for 194 trials (48.3%), 86 (44.3%) of which converted to a full journal article. Trials completed earlier in the pandemic were reported more rapidly than those later in the pandemic, and those involving ivermectin were more rapidly reported than other common interventions. Results were robust to various sensitivity analyses except when considering only trials in a "completed" status on the registry, which substantially increased reporting rates. Poor trial registry data on completion status and dates limits the precision of estimates.

CONCLUSIONS

COVID-19 trials saw marginal increases in reporting rates compared to standard practice; most registered trials failed to meet even the 12-month non-pandemic standard. Preprints were common, complementing journal publication; however, registries were underutilized for rapid reporting. Maintaining registry data enables accurate representation of clinical research; failing to do so undermines these registries' use for public accountability and analysis. Addressing rapid reporting and registry data quality must be emphasized at global, national, and institutional levels.

Reasons for missing evidence in rehabilitation meta-analyses: a cross-sectional meta-research study

BACKGROUND

Systematic reviews of randomized controlled trials are the best evidence for informing on intervention effectiveness. Their results, however, can be biased due to omitted evidence in the quantitative analyses. We aimed to assess the proportion of randomized controlled trials omitted from meta-analyses in the rehabilitation field and explore related reasons.

METHODS

This is a cross-sectional meta-research study. For each systematic review included in a published selected sample in the rehabilitation field, we identified an index meta-analysis on the primary outcome and the main comparison. We then looked at all the studies considered eligible for the chosen comparison in the systematic review and identified those trials that have been omitted (i.e., not included) from each index meta-analysis. Reasons for omission were collected based on an eight-reason classification. We used descriptive statistics to describe the proportion of omitted trials overall and according to each reason.

RESULTS

Starting from a cohort of 827 systematic reviews, 131 index meta-analyses comprising a total of 1761 eligible trials were selected. Only 16 index meta-analyses included all eligible studies while 15 omitted studies without providing references. From the remaining 100 index meta-analyses, 717 trials (40,7%) were omitted overall. Specific reasons for omission were: "unable to distinguish between selective reporting and inadequate planning" (39,3%, N = 282), "inadequate planning" (17%, N = 122), "justified to be not included" (15,1%, N = 108), "incomplete reporting" (8,4%, N = 60), "selective reporting" (3,3%, N = 24) and other situations (e.g., outcome present but no motivation for omission) (5,2%, N = 37). The 11,7% (N = 84) of omitted trials were not assessed due to non-English language or full text not available.

CONCLUSIONS

Almost half of the eligible trials were omitted from their index meta-analyses. Better reporting, protocol registration, definition and adoption of core outcome sets are needed to prevent omission of evidence in systematic reviews.

Why humans kill animals and why we cannot avoid it

Killing animals has been a ubiquitous human behaviour throughout history, yet it is becoming increasingly controversial and criticised in some parts of contemporary human society. Here we review 10 primary reasons why humans kill animals, discuss the necessity (or not) of these forms of killing, and describe the global ecological context for human killing of animals. Humans historically and currently kill animals either directly or indirectly for the following reasons: (1) wild harvest or food acquisition, (2) human health and safety, (3) agriculture and aquaculture, (4) urbanisation and industrialisation, (5) invasive, overabundant or nuisance wildlife control, (6) threatened species conservation, (7) recreation, sport or entertainment, (8) mercy or compassion, (9) cultural and religious practice, and (10) research, education and testing. While the necessity of some forms of animal killing is debatable and further depends on individual values, we emphasise that several of these forms of animal killing are a necessary component of our inescapable involvement in a single, functioning, finite, global food web. We conclude that humans (and all other animals) cannot live in a way that does not require animal killing either directly or indirectly, but humans can modify some of these killing behaviours in ways that improve the welfare of animals while they are alive, or to reduce animal suffering whenever they must be killed. We encourage a constructive dialogue that (1) accepts and permits human participation in one enormous global food web dependent on animal killing and (2) focuses on animal welfare and environmental sustainability. Doing so will improve the lives of both wild and domestic animals to a greater extent than efforts to avoid, prohibit or vilify human animal-killing behaviour.

Estimating the prevalence of discrepancies between study registrations and publications: a systematic review and meta-analyses

OBJECTIVES

Prospectively registering study plans in a permanent time-stamped and publicly accessible document is becoming more common across disciplines and aims to reduce risk of bias and make risk of bias transparent. Selective reporting persists, however, when researchers deviate from their registered plans without disclosure. This systematic review aimed to estimate the prevalence of undisclosed discrepancies between prospectively registered study plans and their associated publication. We further aimed to identify the research disciplines where these discrepancies have been observed, whether interventions to reduce discrepancies have been conducted, and gaps in the literature.

DESIGN

Systematic review and meta-analyses.

DATA SOURCES

Scopus and Web of Knowledge, published up to 15 December 2019.

ELIGIBILITY CRITERIA

Articles that included quantitative data about discrepancies between registrations or study protocols and their associated publications.

DATA EXTRACTION AND SYNTHESIS

Each included article was independently coded by two reviewers using a coding form designed for this review (osf.io/728ys). We used random-effects meta-analyses to synthesise the results.

RESULTS

We reviewed k=89 articles, which included k=70 that reported on primary outcome discrepancies from n=6314 studies and, k=22 that reported on secondary outcome discrepancies from n=1436 studies. Meta-analyses indicated that between 29% and 37% (95% CI) of studies contained at least one primary outcome discrepancy and between 50% and 75% (95% CI) contained at least one secondary outcome discrepancy. Almost all articles assessed clinical literature, and there was considerable heterogeneity. We identified only one article that attempted to correct discrepancies.

CONCLUSIONS

Many articles did not include information on whether discrepancies were disclosed, which version of a registration they compared publications to and whether the registration was prospective. Thus, our estimates represent discrepancies broadly, rather than our target of undisclosed discrepancies between prospectively registered study plans and their associated publications. Discrepancies are common and reduce the trustworthiness of medical research. Interventions to reduce discrepancies could prove valuable.

REGISTRATION

osf.io/ktmdg. Protocol amendments are listed in online supplemental material A.

Completeness and consistency of primary outcome reporting in COVID-19 publications in the early pandemic phase: a descriptive study

BACKGROUND

The COVID-19 pandemic saw a steep increase in the number of rapidly published scientific studies, especially early in the pandemic. Some have suggested COVID-19 trial reporting is of lower quality than typical reports, but there is limited evidence for this in terms of primary outcome reporting. The objective of this study was to assess the prevalence of completely defined primary outcomes reported in registry entries, preprints, and journal articles, and to assess consistent primary outcome reporting between these sources.

METHODS

This is a descriptive study of a cohort of registered interventional clinical trials for the treatment and prevention of COVID-19, drawn from the DIssemination of REgistered COVID-19 Clinical Trials (DIRECCT) study dataset. The main outcomes are: 1) Prevalence of complete primary outcome reporting; 2) Prevalence of consistent primary outcome reporting between registry entry and preprint as well as registry entry and journal article pairs.

RESULTS

We analyzed 87 trials with 116 corresponding publications (87 registry entries, 53 preprints and 63 journal articles). All primary outcomes were completely defined in 47/87 (54%) registry entries, 31/53 (58%) preprints and 44/63 (70%) journal articles. All primary outcomes were consistently reported in 13/53 (25%) registry-preprint pairs and 27/63 (43%) registry-journal article pairs. No primary outcome was specified in 13/53 (25%) preprints and 8/63 (13%) journal articles. In this sample, complete primary outcome reporting occurred more frequently in trials with vs. without involvement of pharmaceutical companies (76% vs. 45%), and in RCTs vs. other study designs (68% vs. 49%). The same pattern was observed for consistent primary outcome reporting (with vs. without pharma: 56% vs. 12%, RCT vs. other: 43% vs. 22%).

CONCLUSIONS

In COVID-19 trials in the early phase of the pandemic, all primary outcomes were completely defined in 54%, 58%, and 70% of registry entries, preprints and journal articles, respectively. Only 25% of preprints and 43% of journal articles reported primary outcomes consistent with registry entries.

Registered report: Survey on attitudes and experiences regarding preregistration in psychological research

BACKGROUND

Preregistration, the open science practice of specifying and registering details of a planned study prior to knowing the data, increases the transparency and reproducibility of research. Large-scale replication attempts for psychological results yielded shockingly low success rates and contributed to an increasing demand for open science practices among psychologists. However, preregistering one's studies is still not the norm in the field. Here, we conducted a study to explore possible reasons for this discrepancy.

METHODS

In a mixed-methods approach, we conducted an online survey assessing attitudes, motivations, and perceived obstacles with respect to preregistration. Respondents (N = 289) were psychological researchers that were recruited through their publications on Web of Science, PubMed, PSYNDEX, and PsycInfo, and preregistrations on OSF Registries. Based on the theory of planned behavior, we predicted that positive attitudes (moderated by the perceived importance of preregistration) as well as a favorable subjective norm and higher perceived behavioral control positively influence researchers' intention to preregister (directional hypothesis 1). Furthermore, we expected an influence of research experience on attitudes and perceived motivations and obstacles regarding preregistration (non-directional hypothesis 2). We analyzed these hypotheses with multiple regression models and included preregistration experience as a control variable.

RESULTS

Researchers' attitudes, subjective norms, perceived behavioral control, and the perceived importance of preregistration significantly predicted researchers' intention to use preregistration in the future (see hypothesis 1). Research experience influenced both researchers' attitudes and their perception of motivations to preregister, but not the perception of obstacles (see hypothesis 2). Descriptive reports on researchers' attitudes, motivations and obstacles regarding preregistration are provided.

DISCUSSION

Many researchers had already preregistered and had a rather positive attitude toward preregistration. Nevertheless, several obstacles were identified that may be addressed to improve and foster preregistration.

Selective outcome reporting bias is highly prevalent in randomized clinical trials of nonsurgical periodontal therapy

Selective outcome reporting (SOR) is a type of bias that can compromise the validity of results and affect evidence-based practice. SOR can overestimate the effect of an intervention and lead to conclusions that a treatment is effective when it is not. This study aimed to investigate the prevalence of SOR in publications of RCTs on nonsurgical periodontal therapy (NSPT) and to verify associated factors. The protocols were searched and selected on the www.clinicaltrials.gov platform up to January 16, 2022. Corresponding publications were identified, and data extraction and discrepancy analysis were performed. The risk of bias was assessed according to the RoB2 tool. One hundred forty-five studies (174 publications) were included. The prevalence of SOR was 49.7% and was unclear in nearly one third of studies (27.6%). Only 31.7% of the primary outcomes were completely described in the publications. The overall risk of bias was high in 60% of the included studies. SOR was associated with statistical significance (p < .001), and multiple publications of the same study (p = .005). Our study demonstrated the high prevalence of SOR, highlighting the need to improve the quality of reporting of RCTs on NSPT studies.

Changes to design and analysis elements of research plans during randomised controlled trials in Australia

OBJECTIVES

To investigate the frequency and legitimacy of substantive changes to the research plans of published randomised controlled trials (RCTs) undertaken in Australia.

DESIGN

Comparison of methodology and analysis plans for RCTs specified in protocol documents (full protocols, published protocol articles, statistical analysis plans, Australian New Zealand Clinical Trials Registry [ANZCTR] registration entries) and described in publications of primary results.

SETTING, PARTICIPANTS

181 RCTs registered with the ANZCTR, 1 September 2007 - 31 December 2013, for which primary results had been published.

MAIN OUTCOME MEASURE

Changes made to research plan, both overall and by specific item (primary outcome, analysis set, eligibility criteria, sample size, primary analysis method, and treatment arms included in the primary comparison in multi-arm trials); trial characteristics associated with changes.

RESULTS

Protocol documents were available for 124 of 181 eligible RCTs (69%; 46 publicly available, 78 provided by trial groups on request). Full audit of RCTs with protocols found clear or probable changes in 111 trials (90%), for 101 of which (91%) it was unclear whether changes had been made blinded to treatment outcomes. After seeking clarification from investigators, changes to 78 trials were confirmed (63%), for 61 of which (78%) changes were made blinded to treatment outcomes. Any change was less likely for trials with publicly available protocols than for trials for which we needed to request protocols (odds ratio, 0.22; 95% CI, 0.06-0.77). Limited reviews of trials without protocols identified that changes had been made to 42 of 57 trials (74%).

CONCLUSION

Changes to RCT study plans in Australia are both frequent and usually made appropriately blinded to treatment outcomes. However, the documentation of changes made to RCT protocols should be formalised to improve transparency.

Quality Output Checklist and Content Assessment (QuOCCA): a new tool for assessing research quality and reproducibility

Research must be well designed, properly conducted and clearly and transparently reported. Our independent medical research institute wanted a simple, generic tool to assess the quality of the research conducted by its researchers, with the goal of identifying areas that could be improved through targeted educational activities. Unfortunately, none was available, thus we devised our own. Here, we report development of the Quality Output Checklist and Content Assessment (QuOCCA), and its application to publications from our institute's scientists. Following consensus meetings and external review by statistical and methodological experts, 11 items were selected for the final version of the QuOCCA: research transparency (items 1-3), research design and analysis (items 4-6) and research reporting practices (items 7-11). Five pairs of raters assessed all 231 articles published in 2017 and 221 in 2018 by researchers at our institute. Overall, the results were similar between years and revealed limited engagement with several recommended practices highlighted in the QuOCCA. These results will be useful to guide educational initiatives and their effectiveness. The QuOCCA is brief and focuses on broadly applicable and relevant concepts to open, high-quality, reproducible and well-reported science. Thus, the QuOCCA could be used by other biomedical institutions and individual researchers to evaluate research publications, assess changes in research practice over time and guide the discussion about high-quality, open science. Given its generic nature, the QuOCCA may also be useful in other research disciplines.

Access to unpublished protocols and statistical analysis plans of randomised trials

BACKGROUND

Access to protocols and statistical analysis plans (SAPs) increases the transparency of randomised trial by allowing readers to identify and interpret unplanned changes to study methods, however they are often not made publicly available. We sought to determine how often study investigators would share unavailable documents upon request.

METHODS

We used trials from two previously identified cohorts (cohort 1: 101 trials published in high impact factor journals between January and April of 2018; cohort 2: 100 trials published in June 2018 in journals indexed in PubMed) to determine whether study investigators would share unavailable protocols/SAPs upon request. We emailed corresponding authors of trials with no publicly available protocol or SAP up to four times.

RESULTS

Overall, 96 of 201 trials (48%) across the two cohorts had no publicly available protocol or SAP (11/101 high-impact cohort, 85/100 PubMed cohort). In total, 8/96 authors (8%) shared some trial documentation (protocol only [n = 5]; protocol and SAP [n = 1]; excerpt from protocol [n = 1]; research ethics application form [n = 1]). We received protocols for 6/96 trials (6%), and a SAP for 1/96 trial (1%). Seventy-three authors (76%) did not respond, 7 authors responded (7%) but declined to share a protocol or SAP, and eight email addresses were invalid (8%). A total of 329 emails were sent (an average of 41 emails for every trial which sent documentation). After emailing authors, the total number of trials with an available protocol increased by only 3%, from 52% in to 55%.

CONCLUSIONS

Most study investigators did not share their unpublished protocols or SAPs upon direct request. Alternative strategies are needed to increase transparency of randomised trials and ensure access to protocols and SAPs.

From registration, protocol to report: are COVID-19-related RCTs in mainland China consistent? A systematic review of clinical trial registry and literature

OBJECTIVE

To provide a comprehensive review of registered COVID-19-related randomised controlled trials (RCTs) in mainland China and evaluate the transparency of reporting through comparison of registrations, protocols and full reports.

DESIGN

Systematic review of trial registrations and publications.

DATA SOURCES

International Clinical Trials Registry Platform, Chinese Clinical Trial Registry, ClinicalTrials.gov, the ISRCTN registry and EU Clinical Trial Register were accessed on 1 February 2022. Publications were searched in PubMed, Embase, Cochrane Library, Google Scholar, CNKI.net and Wanfangdata from 10 February 2022 to 12 February 2022.

ELIGIBILITY CRITERIA

Eligible trials were COVID-19 related RCTs carried out in mainland China. Observational studies, non-randomised trials and single-arm trials were excluded.

DATA EXTRACTION AND SYNTHESIS

Two reviewers independently extracted data from registrations, publications and performed risk of bias assessment for trial reports. Information provided by registrations and publications was compared. The findings were summarised with descriptive statistics.

RESULTS

The number of eligible studies was 415. From these studies 20 protocols and 77 RCT reports were published. Seven trials published both protocol and RCT full report. Between registrations and publications, discrepancy or omission was found in sample size (7, 35.0% for protocols and 47, 61.0% for reports, same below), trial setting (13, 65.0% and 43, 55.8%), inclusion criteria (12, 60.0% and 57, 74.0%), exclusion criteria (10, 50.0% and 54, 70.1%), masking method (9, 45.0% and 35, 45.5%) and primary outcome or time frame of primary outcome measurement (14, 70.0% and 51, 66.2%). Between protocols and full reports, 5 (71.4%) reports had discrepancy in primary outcome or time frame of primary outcome measurement.

CONCLUSIONS

Discrepancy among registrations, protocols and reports revealed compromised transparency in reporting of COVID-19-related RCTs in mainland China. The importance of trial registration should be further emphasised to enhance transparent RCT reporting.

A comparison of registered and published primary outcomes in clinical trials of opioid use disorder: ACTTION review and recommendations

BACKGROUND AND AIMS

Prospective trial registration can increase research integrity. This Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks (ACTTION) review was designed to compare the primary outcomes (PO) reported in registries with associated publications for opioid use disorder (OUD) clinical trials.

DESIGN

The World Health Organization's International Clinical Trials Registry Platform (ICTRP) was searched for completed trials (2010 through 2019). Associated publications were identified and paired with trial registry data based on the publication date.

MEASUREMENTS

Reviewers independently rated the occurrence of discrepancies between the POs in the registry compared to the publication. An analysis of prospective versus retrospective registration was also completed.

FINDINGS

One-hundred and forty trials were identified in the search, and 43 registry-publication pairs evaluated. Only 34 of the 43 pairs could be examined for discrepancies because nine did not report a PO in registry and publication. Of the 34 pairs, only four met rigorous criteria for prospective trial registration and had an exact match of POs. In contrast, the majority of the 34 trials, or 80%, had inconsistent POs (e.g., registered secondary outcomes published as primary; the timing of PO not specified) and/or were retrospectively registered.

CONCLUSIONS

Many clinical trials focused on OUD have not met the standards of trial registration, such as consistent reporting of POs and prospective registration. Failure to properly register trial characteristics undermines the validity of research findings and can delay the development of life-saving treatments. Recommendations for improving prospective trial reporting practices are provided.

Subgroup Analysis in Pulmonary Hypertension-Specific Therapy Clinical Trials: A Systematic Review

Pulmonary hypertension (PH) treatment decisions are driven by the results of randomized controlled trials (RCTs). Subgroup analyses are often performed to assess whether the intervention effect will change due to the patient's characteristics, thus allowing for individualized decisions. This review aimed to evaluate the appropriateness and interpretation of subgroup analyses performed in PH-specific therapy RCTs published between 2000 and 2020. Claims of subgroup effects were evaluated with prespecified criteria. Overall, 30 RCTs were included. Subgroup analyses presented: a high number of subgroup analyses reported, lack of prespecification, and lack of interaction tests. The trial protocol was not available for most RCTs; significant differences were found in those articles that published the protocol. Authors reported 13 claims of subgroup effect, with 12 claims meeting four or fewer of Sun's criteria. Even when most RCTs were generally at low risk of bias and were published in high-impact journals, the credibility and general quality of subgroup analyses and subgroup claims were low due to methodological flaws. Clinicians should be skeptical of claims of subgroup effects and interpret subgroup analyses with caution, as due to their poor quality, these analyses may not serve as guidance for personalized care.

Registered Report Protocol: Survey on attitudes and experiences regarding preregistration in psychological research

BACKGROUND

Preregistration, the open science practice of specifying and registering details of a planned study prior to knowing the data, increases the transparency and reproducibility of research. Large-scale replication attempts for psychological results yielded shockingly low success rates and contributed to an increasing demand for open science practices among psychologists. However, preregistering one's studies is still not the norm in the field. Here, we propose a study to explore possible reasons for this discrepancy.

METHODS

In a mixed-methods approach, an online survey will be conducted, assessing attitudes, motivations, and perceived obstacles with respect to preregistration. Participants will be psychological researchers that will be recruited by scanning research articles on Web of Science, PubMed, PSYNDEX, and PsycInfo, and preregistrations on OSF Registries (targeted sample size: N = 296). Based on the theory of planned behavior, we predict that positive attitudes (moderated by the perceived importance of preregistration) as well as a favorable subjective norm and higher perceived behavioral control positively influence researchers' intention to preregister (hypothesis 1). Furthermore, we expect an influence of research experience on attitudes and perceived motivations and obstacles regarding preregistration (hypothesis 2). We will analyze these hypotheses with multiple regression models, and will include preregistration experience as control variable.

Selective outcome reporting in root coverage randomized clinical trials

BACKGROUND

Outcome discrepancies between protocols and respective publications represent a concerning bias. The purpose of this study was to assess the prevalence of selective outcome reporting (SOR) in root coverage randomized clinical trials (RCTs).

METHODS

Published root coverage RCTs (July 2005 to March 2020) were included if a corresponding protocol could be identified in a public registry. Discrepancies between protocol and its correspondent publication(s) were compared regarding primary and secondary outcomes and other study characteristics. Associations between trial characteristics and SOR were evaluated.

RESULTS

Forty four studies (54 publications) were included. The majority of studies (77.3%) were retrospectively registered. SOR was frequent (40.9% of trials) and consisted of primary outcome downgrade (22.7%); secondary outcome upgrade (11.4%); new primary outcome introduced in publication (25%); protocol primary outcome omitted from publication (13.6%) and discrepancy in primary outcome timing (18.2%). SOR was unclear in 20.5% of studies and favoured statistical significance in 12 studies (27.3%). SOR was significantly associated with study significance (p < 0.001) and unclear outcome definition in the publication (p < 0.001). Only a third (32.8%) of primary outcomes were completely defined.

CONCLUSIONS

The present study identified high prevalence of SOR in root coverage RCTs.

Protocol registration issues of systematic review and meta-analysis studies: a survey of global researchers

Background

Although protocol registration of systematic reviews/meta-analysis (SR/MA) is still not mandatory, it is highly recommended that authors publish their SR/MA protocols prior to submitting their manuscripts for publication as recommended by the Cochrane guidelines for conducting SR/MAs. our aim was to assess the awareness, obstacles, and opinions of SR/MA authors about the protocol registration process.

Methods

A cross-sectional survey study included the authors who published SR/MAs during the period from 2010 to 2016, and they were contacted for participation in our survey study. They were identified through the literature search of SR/MAs in Scopus database. An online questionnaire was sent to each participant via e-mail after receiving their approval to join the study. We have sent 6650 emails and received 275 responses.

Results

A total of 270 authors responses were complete and included in the final analysis. Our results has shown that PROSPERO was the most common database used for protocol registration (71.3%). The registration-to-acceptance time interval in PROSPERO was less than 1 month (99.1%). Almost half of the authors (44.2%) did not register their protocols prior to publishing their SR/MAs and according to their opinion that the other authors lack knowledge of protocol importance and mandance to be registered, was the most commonly reported reason (44.9%). A significant percenatge of respondents (37.4%) believed that people would steal their ideas from protocol databases, while only 5.3% reported that their SR/MA had been stolen. However, the majority (72.9%) of participants have agreed that protocol registries play a role in preventing unnecessary duplication of reviews. Finally, 37.4% of participants agree that SR/MA protocol registration should be mandatory.

Conclusion

About half of the participants believes that the main reason for not registering protocols, is that the other authors lack knowledge concerning obligation and importance to register the SR/MA protocols in advance. Therefore, tools should be available to mandate protocol registration of any SRs beforehand and increasing awareness about the benefits of protocol registration among researchers.

Evidence of unexplained discrepancies between planned and conducted statistical analyses: a review of randomised trials

BACKGROUND

Choosing or altering the planned statistical analysis approach after examination of trial data (often referred to as 'p-hacking') can bias the results of randomised trials. However, the extent of this issue in practice is currently unclear. We conducted a review of published randomised trials to evaluate how often a pre-specified analysis approach is publicly available, and how often the planned analysis is changed.

METHODS

A review of randomised trials published between January and April 2018 in six leading general medical journals. For each trial, we established whether a pre-specified analysis approach was publicly available in a protocol or statistical analysis plan and compared this to the trial publication.

RESULTS

Overall, 89 of 101 eligible trials (88%) had a publicly available pre-specified analysis approach. Only 22/89 trials (25%) had no unexplained discrepancies between the pre-specified and conducted analysis. Fifty-four trials (61%) had one or more unexplained discrepancies, and in 13 trials (15%), it was impossible to ascertain whether any unexplained discrepancies occurred due to incomplete reporting of the statistical methods. Unexplained discrepancies were most common for the analysis model (n = 31, 35%) and analysis population (n = 28, 31%), followed by the use of covariates (n = 23, 26%) and the approach for handling missing data (n = 16, 18%). Many protocols or statistical analysis plans were dated after the trial had begun, so earlier discrepancies may have been missed.

CONCLUSIONS

Unexplained discrepancies in the statistical methods of randomised trials are common. Increased transparency is required for proper evaluation of results.

Comparison between protocols and publications for prognostic and predictive cancer biomarker studies

BACKGROUND

Method prespecification in study protocols is important for controlling bias in reports. The primary goal of this study was to assess potential for discordance between study protocols and publications reporting predictive or prognostic cancer biomarker research. Secondary objectives included comparing characteristics of publications with accessible protocols compared to those without.

METHODS

Publications reporting predictive or prognostic cancer biomarker research were identified from 15 major journals, 2012-2015. Protocols were sought online or through repeated queries of corresponding authors. The following four items were extracted: (1) biomarkers, (2) biospecimen/assays, (3) sample size, (4) endpoints. We defined "explicit discordance" as the presence of major inconsistencies on these items.

RESULTS

Of 149 eligible publications, we obtained 19 eligible protocols online (13%). Out of a random sample of 103 publications where protocols were not available online, 12 protocols (12%) were furnished by corresponding authors; 8 (8% of authors) explicitly stated the absence of a protocol. Among 24 retrospective cohort studies, no protocol could be accessed. We found explicit discordance between publications and protocols for 18 studies (58%), in particular choice of biomarkers (36%), biospecimen/assays (6%), or endpoints (29%).

CONCLUSION

Protocols are generally not accessible or not used for cancer biomarker studies. Publications were often explicitly discordant with protocols, particularly regarding biomarkers and endpoints. Our findings point to common unaddressed risk of bias in publications of major journals reporting the relationship between cancer biomarkers and clinical endpoints.

Selective outcome reporting in randomized clinical trials of dental implants

AIM

Selective outcome reporting (SOR) is a type of bias that occurs when the primary outcome of a trial protocol is changed or omitted in the paper. The purpose of this study was to evaluate the prevalence of SOR in publications of randomized clinical trials (RCT) concerning dental implants.

MATERIALS AND METHODS

Two reviewers independently screened protocols registered at ClinicalTrials.gov until February/2019. If the protocol met the eligibility criteria, the reviewers tried to identify the corresponding publication. Data extraction was carried out by the same reviewers. Primary and secondary outcomes were recorded for each trial and compared to outcomes previously described in protocols.

RESULTS

A total of 49 protocols were included. SOR was identified in 27 (55.1%) trials. The major discrepancies were as follows: protocol-defined primary outcome omitted in the publication (n = 6, 12.2%), new primary outcome introduced (n = 8, 16.3%), discrepancy in the primary outcome time frame (n = 17, 34.7%) and new secondary outcome introduced (n = 31, 63.3%). SOR was significantly associated with industry funding (p = 0.04) and timing of registration (p = 0.04).

CONCLUSIONS

Our results indicate a high rate of SOR in dental implants clinical trials. Use of registry data during the peer-review process may help decreasing SOR.

A guide to the reporting of protocols of pilot and feasibility trials

Publishing protocols of trials including protocols of pilot and feasibility trials—designed to inform the designs of main trials—has been advocated as an important strategy towards improving transparency in the conduct and reporting of main trials and pilot/feasibility trials. This editorial aims to provide some general guidance on how to report protocols of pilot and feasibility trials, drawing upon two available resources—the CONSORT extension to pilot trials and the SPIRIT guideline for main trials. We describe how these might be adapted for the reporting of protocol manuscripts of pilot and feasibility trials for submission in Pilot and Feasibility Studies journal.

Poor statistical reporting, inadequate data presentation and spin persist despite editorial advice

The Journal of Physiology and British Journal of Pharmacology jointly published an editorial series in 2011 to improve standards in statistical reporting and data analysis. It is not known whether reporting practices changed in response to the editorial advice. We conducted a cross-sectional analysis of reporting practices in a random sample of research papers published in these journals before (n = 202) and after (n = 199) publication of the editorial advice. Descriptive data are presented. There was no evidence that reporting practices improved following publication of the editorial advice. Overall, 76-84% of papers with written measures that summarized data variability used standard errors of the mean, and 90-96% of papers did not report exact p-values for primary analyses and post-hoc tests. 76-84% of papers that plotted measures to summarize data variability used standard errors of the mean, and only 2-4% of papers plotted raw data used to calculate variability. Of papers that reported p-values between 0.05 and 0.1, 56-63% interpreted these as trends or statistically significant. Implied or gross spin was noted incidentally in papers before (n = 10) and after (n = 9) the editorial advice was published. Overall, poor statistical reporting, inadequate data presentation and spin were present before and after the editorial advice was published. While the scientific community continues to implement strategies for improving reporting practices, our results indicate stronger incentives or enforcements are needed.