Frequency and reasons for outcome reporting bias in clinical trials: interviews with trialists

Correcting for outcome reporting bias in a meta-analysis: A meta-regression approach

Outcome reporting bias (ORB) refers to the biasing effect caused by researchers selectively reporting outcomes within a study based on their statistical significance. ORB leads to inflated effect size estimates in meta-analysis if only the outcome with the largest effect size is reported due to ORB. We propose a new method (CORB) to correct for ORB that includes an estimate of the variability of the outcomes' effect size as a moderator in a meta-regression model. An estimate of the variability of the outcomes' effect size can be computed by assuming a correlation among the outcomes. Results of a Monte-Carlo simulation study showed that the effect size in meta-analyses may be severely overestimated without correcting for ORB. Estimates of CORB are close to the true effect size when overestimation caused by ORB is the largest. Applying the method to a meta-analysis on the effect of playing violent video games on aggression showed that the effect size estimate decreased when correcting for ORB. We recommend to routinely apply methods to correct for ORB in any meta-analysis. We provide annotated R code and functions to help researchers apply the CORB method.

Trials and tribulations of transparency related to inconsistencies between plan and conduct in peer-reviewed physiotherapy publications: A methodology review

RATIONALE

The physiotherapy profession strives to be a leader in providing quality care and strongly recognizes the value of research to guide clinical practice. Adherence to guidelines for research reporting and conduct is a significant step towards high-quality, transparent and reproducible research.

AIM/OBJECTIVE

Assess integrity between planned and conducted methodology in randomized controlled trials (RCTs) and systematic reviews (SRs) published in physiotherapy journals.

METHODS

Eighteen journals were manually searched for RCTs and SRs published from 1 July 2021 through 31 December 2021. Studies were included if the journal or specific study was indexed in PubMed and published/translated in English. Descriptive statistics determined congruence between preregistration data and publication.

RESULTS

Forty RCTs and 68 SRs were assessed. Forty-three SRs included meta-analysis (MA). Of the 34 registered RCTs, 7 (20.6%) had no discrepancy between the registration and publication. Two trials (5.9%) addressed all discrepancies, 4 (11.8%) addressed some and 21 (61.8%) did not address any discrepancies. Of the 36 registered MAs, 33 (91.7%) had discrepancies between the registration and publication. Two (5.6%) addressed all discrepancies and three (8.3%) had no discrepancies. Eight SRs without MA published information not matching their registration, and none provided justification for the discrepancies.

CONCLUSION

Most RCTs/SRs were registered; the majority had discrepancies between preregistration and publication, potentially influencing the outcomes and interpretations of findings. Journals should require preregistration and compare the submission with the registration information when assessing publication suitability. Readers should be aware of these inconsistencies and their implications when interpreting and translating results into practice.

How do trial teams plan for retention during the design stage of the trial? A scoping review

BACKGROUND

Retention to trials is important to ensure the results of the trial are valid and reliable. The SPIRIT guidelines (18b) require "plans to promote participant retention and complete follow-up, including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols" be included in trial protocols. It is unknown how often protocols report this retention information. The purpose of our scoping review is to establish if, and how, trial teams report plans for retention during the design stage of the trial.

MATERIALS AND METHODS

A scoping review with searches in key databases (PubMed, Scopus, EMBASE, CINAHL (EBSCO), and Web of Science from 2014 to 2019 inclusive) to identify randomised controlled trial protocols. We produced descriptive statistics on the characteristics of the trial protocols and also on those adhering to SPIRIT item 18b. A narrative synthesis of the retention strategies was also conducted.

RESULTS

Eight-hundred and twenty-four protocols met our inclusion criteria. RCTs (n = 722) and pilot and feasibility trial protocols (n = 102) reported using the SPIRIT guidelines during protocol development 35% and 34.3% of the time respectively. Of these protocols, only 9.5% and 11.4% respectively reported all aspects of SPIRIT item 18b "plans to promote participant retention and to complete follow-up, including list of any outcome data for participants who discontinue or deviate from intervention protocols". Of the RCT protocols, 36.8% included proactive "plans to promote participant retention" regardless of whether they reported using SPIRIT guidelines or not. Most protocols planned "combined strategies" (48.1%). Of these, the joint most commonly reported were "reminders and data collection location and method" and "reminders and monetary incentives". The most popular individual retention strategy was "reminders" (14.7%) followed by "monetary incentives- conditional" (10.2%). Of the pilot and feasibility protocols, 40.2% included proactive "plans to promote participant retention" with the use of "combined strategies" being most frequent (46.3%). The use of "monetary incentives - conditional" (22%) was the most popular individual reported retention strategy.

CONCLUSION

There is a lack of reporting of plans to promote participant retention in trial protocols. Proactive planning of retention strategies during the trial design stage is preferable to the reactive implementation of retention strategies. Prospective retention planning and clear communication in protocols may inform more suitable choice, costing and implementation of retention strategies and improve transparency in trial conduct.

Reasons for missing evidence in rehabilitation meta-analyses: a cross-sectional meta-research study

BACKGROUND

Systematic reviews of randomized controlled trials are the best evidence for informing on intervention effectiveness. Their results, however, can be biased due to omitted evidence in the quantitative analyses. We aimed to assess the proportion of randomized controlled trials omitted from meta-analyses in the rehabilitation field and explore related reasons.

METHODS

This is a cross-sectional meta-research study. For each systematic review included in a published selected sample in the rehabilitation field, we identified an index meta-analysis on the primary outcome and the main comparison. We then looked at all the studies considered eligible for the chosen comparison in the systematic review and identified those trials that have been omitted (i.e., not included) from each index meta-analysis. Reasons for omission were collected based on an eight-reason classification. We used descriptive statistics to describe the proportion of omitted trials overall and according to each reason.

RESULTS

Starting from a cohort of 827 systematic reviews, 131 index meta-analyses comprising a total of 1761 eligible trials were selected. Only 16 index meta-analyses included all eligible studies while 15 omitted studies without providing references. From the remaining 100 index meta-analyses, 717 trials (40,7%) were omitted overall. Specific reasons for omission were: "unable to distinguish between selective reporting and inadequate planning" (39,3%, N = 282), "inadequate planning" (17%, N = 122), "justified to be not included" (15,1%, N = 108), "incomplete reporting" (8,4%, N = 60), "selective reporting" (3,3%, N = 24) and other situations (e.g., outcome present but no motivation for omission) (5,2%, N = 37). The 11,7% (N = 84) of omitted trials were not assessed due to non-English language or full text not available.

CONCLUSIONS

Almost half of the eligible trials were omitted from their index meta-analyses. Better reporting, protocol registration, definition and adoption of core outcome sets are needed to prevent omission of evidence in systematic reviews.

Completeness and consistency of primary outcome reporting in COVID-19 publications in the early pandemic phase: a descriptive study

BACKGROUND

The COVID-19 pandemic saw a steep increase in the number of rapidly published scientific studies, especially early in the pandemic. Some have suggested COVID-19 trial reporting is of lower quality than typical reports, but there is limited evidence for this in terms of primary outcome reporting. The objective of this study was to assess the prevalence of completely defined primary outcomes reported in registry entries, preprints, and journal articles, and to assess consistent primary outcome reporting between these sources.

METHODS

This is a descriptive study of a cohort of registered interventional clinical trials for the treatment and prevention of COVID-19, drawn from the DIssemination of REgistered COVID-19 Clinical Trials (DIRECCT) study dataset. The main outcomes are: 1) Prevalence of complete primary outcome reporting; 2) Prevalence of consistent primary outcome reporting between registry entry and preprint as well as registry entry and journal article pairs.

RESULTS

We analyzed 87 trials with 116 corresponding publications (87 registry entries, 53 preprints and 63 journal articles). All primary outcomes were completely defined in 47/87 (54%) registry entries, 31/53 (58%) preprints and 44/63 (70%) journal articles. All primary outcomes were consistently reported in 13/53 (25%) registry-preprint pairs and 27/63 (43%) registry-journal article pairs. No primary outcome was specified in 13/53 (25%) preprints and 8/63 (13%) journal articles. In this sample, complete primary outcome reporting occurred more frequently in trials with vs. without involvement of pharmaceutical companies (76% vs. 45%), and in RCTs vs. other study designs (68% vs. 49%). The same pattern was observed for consistent primary outcome reporting (with vs. without pharma: 56% vs. 12%, RCT vs. other: 43% vs. 22%).

CONCLUSIONS

In COVID-19 trials in the early phase of the pandemic, all primary outcomes were completely defined in 54%, 58%, and 70% of registry entries, preprints and journal articles, respectively. Only 25% of preprints and 43% of journal articles reported primary outcomes consistent with registry entries.

PROTOCOL: Assessment of outcome reporting bias in studies included in Campbell systematic reviews

This is the protocol for a Campbell systematic review. The objectives are as follows: To identify methods used to assess the risk of outcome reporting bias (ORB) in studies included in recent Campbell systematic reviews of intervention effects. The review will answer the following questions: What proportion of recent Campbell reviews included assessment of ORB? How did recent reviews define levels of risk of ORB (what categories, labels, and definitions did they use)? To what extent and how did these reviews use study protocols as sources of data on ORB? To what extent and how did reviews document reasons for judgments about risk of ORB? To what extent and how did reviews assess the inter-rater reliability of ORB ratings? To what extent and how were issues of ORB considered in the review's abstract, plain language summary, and conclusions?

The transparency of reporting 'harms' encountered with the surgically assisted acceleration of orthodontic tooth movement in the published randomized controlled trials: a meta-epidemiological study

BACKGROUND

Surgical-assisted accelerated orthodontics (SAAO) has become very popular recently. Therefore, this study aimed to investigate the extent to which researchers adhere to Item 19 (harms) of the Consolidated Standards of Reporting Trials (CONSORT) in the published studies in the field of SAAO. In addition, the study evaluated the possible association between harm reporting and the human development index (HDI) of the recruited research sample country, CiteScore-based quartile (CSBQ) of the publishing journal, invasiveness of the surgical intervention (ISI), and the type of orthodontic tooth movement (TOTM). Moreover, it aimed to summarize the different possible harms and complications that maybe encountered in the course of SAAO.

MATERIALS AND METHODS

Electronic searching of six databases was conducted for SAAO-related English RCTs published between January 2000 and April 2022. For the RCTs that did not report harms, information was sought by contacting the corresponding authors. Descriptive statistics of the evaluated variables were performed. The association between 'harm reporting' and the HDI of the research team, the BDRQ of the publication journal, the ISI, and the TOTM were investigated. Binary logistic regression was used, and the odds ratios (ORs) with 95% confidence interval (CIs) of the evaluated variables were obtained. Moreover, the risk of bias of the included RCTs was assessed using the RoB2 tool.

RESULTS

Among the 91 included RCTs, 54 RCTs (59.3%) did not adhere to reporting harm associated with the SAAO. The non-adherence was significantly associated with the ISI (OR 0.16; CI 0.03-0.73; p < 0.018) for invasive methods compared with minimally invasive ones). There was a significant positive correlation between harm reporting and both the CSBQ of the publishing journal and the HDI of the recruited research sample country (p = 0.001, p = 0.003, respectively). On the contrary, a non-significant association was found between harm reporting and the type of OTM (p = 0.695). The incidence of harms associated with SAAO was approximately 17.5%.

LIMITATIONS

Assessment was restricted to English RCTs related to SAAO.

CONCLUSION AND IMPLICATIONS

The adherence to reporting harms in the field of SAAO was deficient. Efforts should be made by authors, peer reviewers, and editors to improve compliance with the CONSORT guidelines regarding harms reporting. Additionally, there is a wide spectrum of harms that could be associated with SAAO that the practitioner should pay attention to and alert the patient to the possibility of their occurrence.

A systematic review of reported outcomes in people with lower limb chronic venous insufficiency of the deep veins

OBJECTIVE

The prevalence of lower limb chronic venous insufficiency (CVI) of the deep veins is increasing and presents a significant burden to patients and health care services. To improve the evaluation of interventions it is necessary to standardise their reporting. The aim of this study was to perform a systematic review of the outcomes of interventions delivered to people with CVI of the deep veins as part of the development of a novel core outcome set (COS).

METHODS

Following the Core Outcome Measures in Effectiveness Trials (COMET) framework for COS development, a systematic review was conducted to PRISMA guidance. The protocol was preregistered on PROSPERO (CRD42021236795). MEDLINE, Embase, Emcare, CINAHL, and the Cochrane Central Register of Controlled Trials (CENTRAL), Cochrane Database of Systematic Reviews and Clinicaltrials.gov were searched from January 2018 to January 2021. Clinical trials and observational studies involving more than 20 participants, reporting outcomes for patients with CVI of the deep veins were eligible. Outcomes were extracted verbatim, condensed into agreed outcome terms and coded into domains using standard COMET taxonomy. Outcome reporting consistency, where outcomes were fully reported throughout the methods and results of their respective articles was also assessed.

RESULTS

Some 103 studies were eligible. There were 1183 verbatim outcomes extracted, spanning 22 domains. No outcome was reported unanimously, with the most widely reported outcome of primary patency featuring in 51 articles (<50%). There was a predominant focus on reporting clinical outcomes (n = 963 [81%]), with treatment durability (n = 278 [23%]) and clinical severity (n = 108 [9%]) reported frequently. Life impact outcomes were relatively under-reported (n = 60 [5%]). Outcome reporting consistency was poor, with just 50% of outcomes reported fully.

CONCLUSIONS

Outcome reporting in studies of people with CVI of the deep veins is currently heterogeneous. Life impact outcomes, which likely reflect patients' priorities are under-reported. This study provides the first step in the development of a COS for people with lower limb CVI of the deep veins.

Changes to design and analysis elements of research plans during randomised controlled trials in Australia

OBJECTIVES

To investigate the frequency and legitimacy of substantive changes to the research plans of published randomised controlled trials (RCTs) undertaken in Australia.

DESIGN

Comparison of methodology and analysis plans for RCTs specified in protocol documents (full protocols, published protocol articles, statistical analysis plans, Australian New Zealand Clinical Trials Registry [ANZCTR] registration entries) and described in publications of primary results.

SETTING, PARTICIPANTS

181 RCTs registered with the ANZCTR, 1 September 2007 - 31 December 2013, for which primary results had been published.

MAIN OUTCOME MEASURE

Changes made to research plan, both overall and by specific item (primary outcome, analysis set, eligibility criteria, sample size, primary analysis method, and treatment arms included in the primary comparison in multi-arm trials); trial characteristics associated with changes.

RESULTS

Protocol documents were available for 124 of 181 eligible RCTs (69%; 46 publicly available, 78 provided by trial groups on request). Full audit of RCTs with protocols found clear or probable changes in 111 trials (90%), for 101 of which (91%) it was unclear whether changes had been made blinded to treatment outcomes. After seeking clarification from investigators, changes to 78 trials were confirmed (63%), for 61 of which (78%) changes were made blinded to treatment outcomes. Any change was less likely for trials with publicly available protocols than for trials for which we needed to request protocols (odds ratio, 0.22; 95% CI, 0.06-0.77). Limited reviews of trials without protocols identified that changes had been made to 42 of 57 trials (74%).

CONCLUSION

Changes to RCT study plans in Australia are both frequent and usually made appropriately blinded to treatment outcomes. However, the documentation of changes made to RCT protocols should be formalised to improve transparency.

A longitudinal assessment of trial protocols approved by research ethics committees: The Adherance to SPIrit REcommendations in the UK (ASPIRE-UK) study

Background

To assess the quality of reporting of RCT protocols approved by UK research ethics committees before and after the publication of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guideline.

Methods

We had access to RCT study protocols that received ethical approval in the UK in 2012 (n=103) and 2016 (n=108). From those, we assessed the adherence to the 33 SPIRIT items (i.e. a total of 64 components of the 33 SPIRIT items). We descriptively analysed the adherence to SPIRIT guidelines as proportion of adequately reported items (median and interquartile range [IQR]) and stratified the results by year of approval and sponsor.

Results

The proportion of reported SPIRIT items increased from a median of 64.9% (IQR, 57.6–69.2%) in 2012 to a median of 72.5% (IQR, 65.3–78.3%) in 2016. Industry-sponsored RCTs reported more SPIRIT items in 2012 (median 67.4%; IQR, 64.1–69.4%) compared to non-industry-sponsored trials (median 59.8%; IQR, 46.5–67.7%). This gap between industry- and non-industry-sponsored trials increased in 2016 (industry-sponsored: median 75.6%; IQR, 71.2–79.0% vs non-industry-sponsored: median 65.3%; IQR, 51.6–76.3%).

Conclusions

The adherence to SPIRIT guidelines has improved in the UK from 2012 to 2016 but remains on a modest level, especially for non-industry-sponsored RCTs.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-022-06516-1.

Reporting quality of clinical trial protocols: a repeated cross-sectional study about the Adherence to SPIrit Recommendations in Switzerland, CAnada and GErmany (ASPIRE-SCAGE)

OBJECTIVES

Comprehensive protocols are key for the planning and conduct of randomised clinical trials (RCTs). Evidence of low reporting quality of RCT protocols led to the publication of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) checklist in 2013. We aimed to examine the quality of reporting of RCT protocols from three countries before and after the publication of the SPIRIT checklist.

DESIGN

Repeated cross sectional study.

SETTING

Swiss, German and Canadian research ethics committees (RECs).

PARTICIPANTS

RCT protocols approved by RECs in 2012 (n=257) and 2016 (n=292).

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcomes were the proportion of reported SPIRIT items per protocol and the proportion of trial protocols reporting individual SPIRIT items. We compared these outcomes in protocols approved in 2012 and 2016, and built regression models to explore factors associated with adherence to SPIRIT. For each protocol, we also extracted information on general trial characteristics and assessed whether individual SPIRIT items were reported RESULTS: The median proportion of reported SPIRIT items among RCT protocols showed a non-significant increase from 72% (IQR, 63%-79%) in 2012 to 77% (IQR, 68%-82%) in 2016. However, in a preplanned subgroup analysis, we detected a significant improvement in investigator-sponsored protocols: the median proportion increased from 64% (IQR, 55%-72%) in 2012 to 76% (IQR, 64%-83%) in 2016, while for industry-sponsored protocols median adherence was 77% (IQR 72%-80%) for both years. The following trial characteristics were independently associated with lower adherence to SPIRIT: single-centre trial, no support from a clinical trials unit or contract research organisation, and investigator-sponsorship.

CONCLUSIONS

In 2012, industry-sponsored RCT protocols were reported more comprehensively than investigator-sponsored protocols. After publication of the SPIRIT checklist, investigator-sponsored protocols improved to the level of industry-sponsored protocols, which did not improve.

Registry versus publication: discrepancy of primary outcomes and possible outcome reporting bias in child and adolescent mental health

Outcome reporting bias is one of the fundamental forms of publication bias. It implies publishing only outcomes that have positive results. The aim of this observational study was to explore primary outcome discrepancies between registry of clinical trials and their corresponding publications, since these can indicate outcome reporting bias in child mental health. Data were extracted from completed interventional clinical trials from ClinicalTrial.gov registry and its Archive site. Trials were registered under "Behaviours and Mental Disorders" category, and conducted on underage participants (0-17 years). Their primary outcomes were compared to those published in publication which had a corresponding NCT number stated in the text. Sixteen percent of trials did not have the minimum information on primary outcome stated in the registry-neither the measure used nor the measurement time points; 38.9% of trials had the minimum information stated to describe primary outcome, while only 3.3% of trials had all the necessary elements stated in the registry. Most of the publication in our sample had positive results (66.4%). Half of the trials registered before completion had non-matching primary outcomes in the registry and publication; 85.4% of trials with non-matching outcomes indicated possible outcome reporting bias for some of the primary outcome. Middle-sized trials and industry-funded trials were related with higher quality of primary outcome registration. Industry funding was related with positive findings in publication. Non-industry funding proved to be the only significant predictor of discrepancy between registered and published primary outcomes, and possible outcome reporting bias. Journal impact factor was not related with any of the outcome measures. The main limitation of the study is that it primarily offers an insight into discrepancy of registered and published outcomes. The methodology does not imply an access to results of unpublished outcomes - therefore, it was not possible to determine the presence of the bias with sufficient certainty in large number of trials. Further research should be done with improved methodology and additional data.

Selective Reporting of Outcomes in Tinnitus Trials: Comparison of Trial Registries With Corresponding Publications

Objectives: We aimed to study the prevalence of selective reporting of primary and secondary outcomes in tinnitus trials and to examine if selective reporting of outcome measures is influenced by the nature and direction of its results.

Background: Selective reporting of outcome measures has been reported in several biomedical fields and can influence the clinical usefulness and implementation of outcomes of clinical trials. It is reported as one of the obstacles in finding an effective intervention for tinnitus.

Methods: ClinicalTrials.gov (CT.gov) was used to identify all registered interventional tinnitus trials up to December 2015. A standardized search was used to find corresponding publications up to March 2018. The prespecified outcomes in CT.gov were compared with the outcomes reported in corresponding publication(s). The effects of the (lack of) statistical significance of trial results and the effects of funding source on record adherence were evaluated. Changes in registration elements were assessed with the Archive site of CT.gov.

Results: We found corresponding publications for 60 (64.5%) of 93 eligible tinnitus trials registered in CT.gov. Of all the publications, five (7.5%) fully reported outcome measures entirely in line with the prespecified outcome measures. Discrepancies between the prespecified and reported outcomes were found in a total of 51 (76.1%) of the studies for primary outcomes, whereas 62 (92.5%) of the studies had discrepancies in secondary outcomes. In secondary outcomes, statistical significance of trial results influenced CT.gov record adherence. In addition, there was a statistically significant difference in the rate of discrepancy in industry-funded [n = 98 (87.5%) discrepant outcomes] and non-industry funded trials [n = 172 (74.5%) discrepant outcomes] (p = 0.01). Finally, 15 (25.9%) trialists made modifications in registered outcome measures during or after the trial period.

Conclusion: Tinnitus trials suffer from substantial outcome reporting bias. Awareness of its presence must be raised to limit the obstacles of finding an effective intervention for tinnitus.

Rationale and design of repeated cross-sectional studies to evaluate the reporting quality of trial protocols: the Adherence to SPIrit REcommendations (ASPIRE) study and associated projects

BACKGROUND

Clearly structured and comprehensive protocols are an essential component to ensure safety of participants, data validity, successful conduct, and credibility of results of randomized clinical trials (RCTs). Funding agencies, research ethics committees (RECs), regulatory agencies, medical journals, systematic reviewers, and other stakeholders rely on protocols to appraise the conduct and reporting of RCTs. In response to evidence of poor protocol quality, the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guideline was published in 2013 to improve the accuracy and completeness of clinical trial protocols. The impact of these recommendations on protocol completeness and associations between protocol completeness and successful RCT conduct and publication remain uncertain.

OBJECTIVES AND METHODS

Aims of the Adherence to SPIrit REcommendations (ASPIRE) study are to investigate adherence to SPIRIT checklist items of RCT protocols approved by RECs in the UK, Switzerland, Germany, and Canada before (2012) and after (2016) the publication of the SPIRIT guidelines; determine protocol features associated with non-adherence to SPIRIT checklist items; and assess potential differences in adherence across countries. We assembled an international cohort of RCTs based on 450 protocols approved in 2012 and 402 protocols approved in 2016 by RECs in Switzerland, the UK, Germany, and Canada. We will extract data on RCT characteristics and adherence to SPIRIT for all included protocols. We will use multivariable regression models to investigate temporal changes in SPIRIT adherence, differences across countries, and associations between SPIRIT adherence of protocols with RCT registration, completion, and publication of results. We plan substudies to examine the registration, premature discontinuation, and non-publication of RCTs; the use of patient-reported outcomes in RCT protocols; SPIRIT adherence of RCT protocols with non-regulated interventions; the planning of RCT subgroup analyses; and the use of routinely collected data for RCTs.

DISCUSSION

The ASPIRE study and associated substudies will provide important information on the impact of measures to improve the reporting of RCT protocols and on multiple aspects of RCT design, trial registration, premature discontinuation, and non-publication of RCTs observing potential changes over time.

Influence and management of conflicts of interest in randomised clinical trials: qualitative interview study

OBJECTIVE

To characterise and analyse the experiences of trial researchers of if and how conflicts of interest had unduly influenced clinical trials they had worked on, what management strategies they had used to minimise any potential influence, and their experiences and views on conflicts of interest more generally.

DESIGN

Qualitative interview study.

PARTICIPANTS

Trial researchers who had participated in at least 10 clinical trials with methodological or statistical expertise. Researchers differed by geographical location, educational background, and experience with different types of funders. Interviewees were identified by searches on Web of Science and snowball sampling. 52 trial researchers were approached by email; 20 agreed to be interviewed.

SETTING

Interviews conducted by telephone, recorded, transcribed verbatim, imported to NVivo 12, and analysed by systematic text condensation. Semistructured interviews focused on financial and non-financial conflicts of interest.

RESULTS

The interviewees had participated in a median of 37.5 trials and were mainly male physicians who had experience with commercial and non-commercial trial funders. Two predefined themes (influence of conflicts of interest and management strategies) and two additional themes (definition and reporting of conflicts of interest) emerged. Examples of perceived influence of conflicts of interest were: choice of inferior comparator, manipulation of the randomisation process, prematurely stopping the trials, fabrication of data, blocking access to data, and spin (eg, overly favourable interpretation of the results). Examples of strategies to manage conflicts of interest were: disclosure procedures, exclusion of the funder from design and analysis, independent committees, contracts ensuring complete access to the data, and no restriction by the funder on analysis and reporting. Interviewees used different definitions or thresholds for what they considered to be conflicts of interest, and they described different criteria for when to report them. Some interviewees considered non-commercial financial conflicts of interest (eg, funding of trials by governmental health agencies with a political agenda) to be equally or more important than commercial financial conflicts of interest (eg, funding by drug and device companies), but more challenging to report and manage.

CONCLUSION

This study described how trial researchers perceive conflicts of interest unduly influencing clinical trials they had worked on, and the management strategies they used to prevent these influences. The results indicated considerable variability in researchers' understanding of what conflicts of interest are and when they should be reported.

A historical review of publication bias

Publication bias is a well-known threat to the validity of meta-analyses and, more broadly, the reproducibility of scientific findings. When policies and recommendations are predicated on an incomplete evidence base, it undermines the goals of evidence-based decision-making. Great strides have been made in the last 50 years to understand and address this problem, including calls for mandatory trial registration and the development of statistical methods to detect and correct for publication bias. We offer an historical account of seminal contributions by the evidence synthesis community, with an emphasis on the parallel development of graph-based and selection model approaches. We also draw attention to current innovations and opportunities for future methodological work.

Distance sampling for epidemiology: an interactive tool for estimating under-reporting of cases from clinic data

BACKGROUND

Distance sampling methods are widely used in ecology to estimate and map the abundance of animal and plant populations from spatial survey data. The key underlying concept in distance sampling is the detection function, the probability of detecting the occurrence of an event as a function of its distance from the observer, as well as other covariates that may influence detection. In epidemiology, the burden and distribution of infectious disease is often inferred from cases that are reported at clinics and hospitals. In areas with few public health facilities and low accessibility, the probability of detecting a case is also a function of the distance between an infected person and the "observer" (e.g. a health centre). While the problem of distance-related under-reporting is acknowledged in public health; there are few quantitative methods for assessing and correcting for this bias when mapping disease incidence. Here, we develop a modified version of distance sampling for prediction of infectious disease incidence by relaxing some of the framework's fundamental assumptions. We illustrate the utility of this approach using as our example malaria distribution in rural Burkina Faso, where there is a large population at risk but relatively low accessibility of health facilities.

RESULTS

The modified distance-sampling framework was used to predict the probability of reporting malaria infection at 8 rural clinics, based on road-travel distances from villages. The rate at which reporting probability dropped with distance varied between clinics, depending on road and clinic positions. The probability of case detection was estimated as 0.3-1 in the immediate vicinity of the clinic, dropping to 0.1-0.6 at a travel distance of 10 km, and effectively zero at distances > 30-40 km.

CONCLUSIONS

To enhance the method's strategic impact, we provide an interactive mapping tool (as a self-contained R Shiny app) that can be used by non-specialists to interrogate model outputs and visualize how the overall probability of under-reporting and the catchment area of each clinic is influenced by changing the number and spatial allocation of health centres.

Reporting of health-related quality of life endpoints in National Cancer Institute-supported cancer treatment trials

BACKGROUND

The importance of capturing and reporting health-related quality of life (HRQOL) in clinical trials has been increasingly recognized in the oncology field. As a result, the National Cancer Institute (NCI) began to provide support for correlative HRQOL studies in cancer treatment trials. The current study was conducted to assess the publication rate of HRQOL correlative studies in NCI-supported treatment trials and to identify potential factors positively or negatively associated with publication rates.

METHODS

The NCI conducted a retrospective review of existing NCI databases to identify cancer treatment trials that had obtained additional NCI funding for the assessment of HRQOL and to determine the extent to which funded HRQOL studies have been completed and published in a peer-reviewed journal.

RESULTS

Of the 108 included trials, 58 (54%) had a parent trial (PT) publication; of these, 36 trials (62%) had a published HRQOL result: 20 as an independent publication and 16 that were included and/or reported in the PT publication. The length of time between trial activation and closure, as well as the specific cancer, appeared to be associated with the publication rates.

CONCLUSIONS

The results of the current study demonstrated that approximately 45% of the PT publications were followed by a HRQOL publication within 1 year, to allow the knowledge to be used in patient treatment decision making. The authors believe the current analysis is an important first step toward a better understand of the challenges that researchers face when reporting HRQOL endpoints.

A Methodologic Systematic Review of Mobile Health Behavior Change Randomized Trials

CONTEXT

Mobile health helps providers offer accessible, affordable, tailored behavior change interventions. However, research assessing mobile health interventions may feature methodologic shortcomings and poor reporting. This review aims to summarize the characteristics, methods, and intervention reporting of RCTs evaluating mobile health behavior change interventions.

EVIDENCE ACQUISITION

This was a methodologic systematic review of RCTs assessing mobile health behavior change interventions published in PubMed from January 1, 2014 to January 1, 2018, in journals with the upper half of Impact Factors (Clarivate Analytics). Three reviewers independently extracted sample characteristics. Primary outcomes were classified as patient-important or not using definitions from the literature. Any non-patient-important outcomes were then reclassified by a panel of 3 patients. Intervention reporting was assessed by the mobile health Evidence Reporting and Assessment checklist. Data were analyzed in December 2018.

EVIDENCE SYNTHESIS

Most of the 231 included RCTs assessed text messaging (51%) or smartphone app (28%) interventions aiming to change nutrition and physical activity (36%) or treatment adherence (25%). Only 8% of RCTs had a patient-important primary outcome, follow-up of ≥6 months, and intent-to-treat analysis. Most primary outcomes were behavioral measures (60%). Follow-up was <3 months in 29% of RCTs. Regarding reporting, 12 of the 16 checklist items were reported in less than half of RCTs (e.g., usability/content testing, 32%; data security, 13%).

CONCLUSIONS

Reports of RCTs assessing mobile health behavior change interventions lack information that would be useful for providers, including reporting of long-term intervention impact on patient-important primary outcomes and information needed for intervention replicability.

Selective outcome reporting in randomized clinical trials of dental implants

AIM

Selective outcome reporting (SOR) is a type of bias that occurs when the primary outcome of a trial protocol is changed or omitted in the paper. The purpose of this study was to evaluate the prevalence of SOR in publications of randomized clinical trials (RCT) concerning dental implants.

MATERIALS AND METHODS

Two reviewers independently screened protocols registered at ClinicalTrials.gov until February/2019. If the protocol met the eligibility criteria, the reviewers tried to identify the corresponding publication. Data extraction was carried out by the same reviewers. Primary and secondary outcomes were recorded for each trial and compared to outcomes previously described in protocols.

RESULTS

A total of 49 protocols were included. SOR was identified in 27 (55.1%) trials. The major discrepancies were as follows: protocol-defined primary outcome omitted in the publication (n = 6, 12.2%), new primary outcome introduced (n = 8, 16.3%), discrepancy in the primary outcome time frame (n = 17, 34.7%) and new secondary outcome introduced (n = 31, 63.3%). SOR was significantly associated with industry funding (p = 0.04) and timing of registration (p = 0.04).

CONCLUSIONS

Our results indicate a high rate of SOR in dental implants clinical trials. Use of registry data during the peer-review process may help decreasing SOR.

Achieving explanatory depth and spatial breadth in infectious disease modelling: Integrating active and passive case surveillance

Ideally, the data used for robust spatial prediction of disease distribution should be both high-resolution and spatially expansive. However, such in-depth and geographically broad data are rarely available in practice. Instead, researchers usually acquire either detailed epidemiological data with high resolution at a small number of active sampling sites, or more broad-ranging but less precise data from passive case surveillance. We propose a novel inferential framework, capable of simultaneously drawing insights from both passive and active data types. We developed a Bayesian latent point process approach, combining active data collection in a limited set of points, where in-depth covariates are measured, with passive case detection, where error-prone, large-scale disease data are accompanied only by coarse or remotely-sensed covariate layers. Using the example of malaria, we tested our method's efficiency under several hypothetical scenarios of reported incidence in different combinations of imperfect detection and spatial complexity of the environmental variables. We provide a simple solution to a widespread problem in spatial epidemiology, combining latent process modelling and spatially autoregressive modelling. By using active sampling and passive case detection in a complementary way, we achieved the best-of-both-worlds, in effect, a formal calibration of spatially extensive, error-prone data by localised, high-quality data.

Interventions to Improve Antimicrobial Stewardship for Older People in Care Homes: A Systematic Review

BACKGROUND

Inappropriate antimicrobial prescribing has been reported in care homes. This may result in serious drug-related adverse events, Clostridium difficile colonization, and the development of antimicrobial resistance among care home residents. Interventions to improve antibiotic prescribing in nursing homes have been reported through clinical trials, but whether antifungal and antiviral prescribing and residential homes have been considered, or how outcomes were measured and reported in such interventions, remains unclear.

OBJECTIVES

Our aims were to evaluate the effect of interventions to improve antimicrobial stewardship in care homes and to report the outcomes used in these trials.

METHODS

We searched 11 electronic databases and five trial registries for studies published until 30 November 2018. Inclusion criteria for the review were randomized controlled trials, targeting care home residents and healthcare professionals, providing interventions to improve antimicrobial prescribing compared with usual care or other interventions. The Cochrane tools for assessing risk of bias were used for quality assessment. A narrative approach was taken because of heterogeneity across the studies.

RESULTS

Five studies met the inclusion criteria. The studies varied in terms of types of infection, key targets, delivery of interventions, and reported outcomes. In total, 27 outcomes were reported across the studies, with seven not prespecified in the methods. The interventions had little impact on adherence to guidelines and prevalence of antimicrobial prescribing; they appeared to decrease total antimicrobial consumption but were unlikely to have affected overall hospital admissions and mortality. The overall quality of evidence was low because the risk of bias was high across the studies.

CONCLUSION

The interventions had limited effect on improving antimicrobial prescribing but did not appear to cause harm to care home residents. The low quality of evidence and heterogeneity in outcome measurement suggest the need for future well-designed studies and the development of a core outcome set to best evaluate the effectiveness of antimicrobial stewardship in care homes.

Peer reviewed evaluation of registered end-points of randomised trials (the PRE-REPORT study): protocol for a stepped-wedge, cluster-randomised trial

INTRODUCTION

Clinical trials are critical to the advancement of medical knowledge. However, the reliability of trial conclusions depends in part on consistency between pre-planned and reported study outcomes. Unfortunately, selective outcome reporting, in which outcomes reported in published manuscripts differ from pre-specified study outcomes, is common. Trial registries such as ClinicalTrials.gov have the potential to help identify and stop selective outcome reporting during peer review by allowing peer reviewers to compare outcomes between registry entries and submitted manuscripts. However, the persistently high rate of selective outcome reporting among published clinical trials indicates that the current peer review process at most journals does not effectively address the problem of selective outcome reporting.

METHODS AND ANALYSIS

PRE-REPORT is a stepped-wedge cluster-randomised trial that will test whether providing peer reviewers with a summary of registered, pre-specified primary trial outcomes decreases inconsistencies between prospectively registered and published primary outcomes. Peer reviewed manuscripts describing clinical trial results will be included. Eligible manuscripts submitted to each participating journal during the study period will comprise each cluster. After an initial control phase, journals will transition to the intervention phase in random order, after which peer reviewers will be emailed registry information consisting of the date of registration and any prospectively defined primary outcomes. Blinded outcome assessors will compare registered and published primary outcomes for all included trials. The primary PRE-REPORT outcome is the presence of a published primary outcome that is consistent with a prospectively defined primary outcome in the study's trial registry. The primary outcome will be analysed using a mixed effect logistical regression model to compare results between the intervention and control phases.

ETHICS AND DISSEMINATION

The Cooper Health System Institutional Review Board determined that this study does not meet criteria for human subject research. Findings will be published in peer-reviewed journals.

TRIAL REGISTRATION NUMBER

ISRCTN41225307; Pre-results.

Evaluation of Journal Registration Policies and Prospective Registration of Randomized Clinical Trials of Nonregulated Health Care Interventions

IMPORTANCE

Many interventions that are important to the health care of patients are not subject to regulation by the US Food and Drug Administration (FDA) or comparable regulatory bodies in other nations.

OBJECTIVE

To determine whether specialty journals that publish trials of primarily nonregulated health care interventions require prospective registration and whether the prospective registration policies are associated with the publication of prospectively registered trials, trials with adequately registered outcomes, and trials with primary outcomes consistent with the registered primary outcomes.

DESIGN AND METHODS

PubMed was searched daily, from March 18, 2016, to September 17, 2016, for nonregulated intervention randomized clinical trials. The search included all journals in the Clarivate Analytics Science Citation Index Expanded categories of behavioral sciences, nursing, nutrition and dietetics, psychology, rehabilitation, and surgery. Trials of interventions not subject to FDA regulation were included. One investigator extracted journal registration policy and trial registration status. Two investigators independently extracted trial registration and publication characteristics.

MAIN OUTCOMES AND MEASURES

For journals, the main outcome was the trial registration policy. For trials, the main outcomes were prospective registration, adequacy of outcome registration, and concordance of registered with published primary outcomes.

RESULTS

In total, 953 nonregulated intervention trials published in 254 journals were identified. Prospective registration was required for publication by 29 (11.4%) of 254 journals, and an additional 12 journals (4.7%) had conditional date-based requirements. Only 189 (19.8%) of the 953 trials were registered prospectively, including 33 of 98 published in journals with prospective registration policies as compared with 156 of 855 in journals without policies (33.7% vs 18.2%; P = .004). Among the 17 journals that required prospective registration and had at least 2 included trials, none had a prospective registration of more than 50%. In journals with policies, only 3 of 98 trials included primary outcomes consistent with prospectively, adequately registered outcomes, as compared with 34 of 852 trials in journals without policies (3.1% vs 4.0%; P = .62).

CONCLUSIONS AND RELEVANCE

Few journals in behavioral sciences or psychology, nursing, nutrition and dietetics, rehabilitation, and surgery require prospective trial registration, and those with existing registration policies rarely enforce them; this finding suggests that strategies for encouraging prospective registration of clinical trials not subject to FDA regulation should be developed and tested.

COMPare: Qualitative analysis of researchers' responses to critical correspondence on a cohort of 58 misreported trials

BACKGROUND

Discrepancies between pre-specified and reported outcomes are an important and prevalent source of bias in clinical trials. COMPare (Centre for Evidence-Based Medicine Outcome Monitoring Project) monitored all trials in five leading journals for correct outcome reporting, submitted correction letters on all misreported trials in real time, and then monitored responses from editors and trialists. From the trialists' responses, we aimed to answer two related questions. First, what can trialists' responses to corrections on their own misreported trials tell us about trialists' knowledge of correct outcome reporting? Second, what can a cohort of responses to a standardised correction letter tell us about how researchers respond to systematic critical post-publication peer review?

METHODS

All correspondence from trialists, published by journals in response to a correction letter from COMPare, was filed and indexed. We analysed the letters qualitatively and identified key themes in researchers' errors about correct outcome reporting, and approaches taken by researchers when their work was criticised.

RESULTS

Trialists frequently expressed views that contradicted the CONSORT (Consolidated Standards of Reporting Trials) guidelines or made inaccurate statements about correct outcome reporting. Common themes were: stating that pre-specification after trial commencement is acceptable; incorrect statements about registries; incorrect statements around the handling of multiple time points; and failure to recognise the need to report changes to pre-specified outcomes in the trial report. We identified additional themes in the approaches taken by researchers when responding to critical correspondence, including the following: ad hominem criticism; arguing that trialists should be trusted, rather than follow guidelines for trial reporting; appealing to the existence of a novel category of outcomes whose results need not necessarily be reported; incorrect statements by researchers about their own paper; and statements undermining transparency infrastructure, such as trial registers.

CONCLUSIONS

Researchers commonly make incorrect statements about correct trial reporting. There are recurring themes in researchers' responses when their work is criticised, some of which fall short of the scientific ideal. Research on methodological shortcomings is now common, typically in the form of retrospective cohort studies describing the overall prevalence of a problem. We argue that prospective cohort studies which additionally issue correction letters in real time on each individual flawed study-and then follow-up responses from trialists and journals-are more impactful, more informative for those consuming the studies critiqued, more informative on the causes of shortcomings in research, and a better use of research resources.

Collaboration between academics and industry in clinical trials: cross sectional study of publications and survey of lead academic authors

OBJECTIVES

To determine the role of academic authors, funders, and contract research organisations in industry funded trials of vaccines, drugs, and devices and to determine lead academic authors' experiences with industry funder collaborations.

DESIGN

Cross sectional analysis of trial publications and survey of lead academic authors.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

The most recent 200 phase III and IV trials of vaccines, drugs, and devices with full industry funding, at least one academic author, published in one of the top seven high impact general medical journals (New England Journal of Medicine, Lancet, JAMA, BMJ, Annals of Internal Medicine, JAMA Internal Medicine, and PLoS Medicine).

RESULTS

Employees of industry funders co-authored 173 (87%) of publications; 183 (92%) trials reported involvement of funders in design, and 167 (84%) reported involvement of academic authors. Data analysis involved the funder in 146 (73%) trials and the academic authors in 79 (40%). Trial reporting involved the funder in 173 (87%) trials and academic authors in 197 (99%). Contract research organisations were involved in the reporting of 123 (62%) trials.Eighty (40%) of 200 lead academic authors responded to the survey. Twenty nine (33%) of the 80 responders reported that academics had final say on the design. Ten responders described involvement of an unnamed funder and/or contract research organisation employee in the data analysis and/or reporting. Most academic authors found the collaboration with industry funder beneficial, but 3 (4%) experienced delay in publication due to the industry funder and 9 (11%) reported disagreements with the industry funder, mostly concerning trial design and reporting.

CONCLUSIONS

Industry employees and academic authors are involved in the design, conduct, and reporting of most industry funded trials in high impact journals. However, data analysis is often conducted without academic involvement. Academics view the collaboration as beneficial, but some report loss of academic freedom.

The Impact of Targeted Rheumatoid Arthritis Pharmacologic Treatment on Mental Health: A Systematic Review and Network Meta-Analysis

Rheumatoid arthritis (RA) pharmacotherapy may impact mental health outcomes by improving pain and stiffness, potentially by targeting inflammatory processes common to RA and depression. The objectives of this review were to ascertain the frequency of mental health assessments in RA pharmacotherapy trials, quantify the efficacy of RA pharmacotherapy for mental health outcomes, and explore the clinical and demographic factors related to mental health outcomes. Effective pharmacotherapy alone is unlikely to substantially improve mental health outcomes in most patients with RA. Integrated mental health care provided within routine clinical practice is essential to optimize mental and physical health outcomes.

A protocol developing, disseminating and implementing a core outcome set for infertility

STUDY QUESTIONS

We aim to produce, disseminate and implement a core outcome set for future infertility research.

WHAT IS KNOWN ALREADY

Randomized controlled trials (RCTs) evaluating infertility treatments have reported many different outcomes, which are often defined and measured in different ways. Such variation contributes to an inability to compare, contrast and combine results of individual RCTs. The development of a core outcome set will ensure outcomes important to key stakeholders are consistently collected and reported across future infertility research.

STUDY DESIGN, SIZE, DURATION

This is a consensus study using the modified Delphi method. All stakeholders, including healthcare professionals, allied healthcare professionals, researchers and people with lived experience of infertility will be invited to participate.

PARTICIPANTS/MATERIALS, SETTING, METHODS

An international steering group, including people with lived experience of infertility, healthcare professionals, allied healthcare professionals and researchers, has been formed to guide the development of this core outcome set. Potential core outcomes have been identified through a comprehensive literature review of RCTs evaluating treatments for infertility and will be entered into a modified Delphi method. Participants will be asked to score potential core outcomes on a nine-point Likert scale anchored between one (not important) and nine (critical). Repeated reflection and rescoring should promote convergence towards consensus ‘core’ outcomes. We will establish standardized definitions and recommend high-quality measurement instruments for individual core outcomes.

STUDY FUNDING/COMPETING INTEREST(S)

This project is funded by the Royal Society of New Zealand Catalyst Fund (3712235). BWM reports consultancy fees from Guerbet, Merck, and ObsEva. R.S.L. reports consultancy fees from Abbvie, Bayer, Fractyl and Ogeda and research sponsorship from Ferring. S.B. is the Editor-in-Chief of Human Reproduction Open. The remaining authors declare no competing interests.

Risk Factors for Antisocial Behavior in Low- and Middle-Income Countries: A Systematic Review of Longitudinal Studies

Violent crime is a major cause of social instability, injury, and death in low- and middle-income countries. Longitudinal studies in high-income countries have provided important evidence on developmental precursors of violence and other antisocial behaviors. However, there may be unique influences or different risk factor effects in other social settings. Extensive searches in seven languages and screening of over 60,000 references identified 39 longitudinal studies of antisocial behavior in low- and middle-income countries. Many risk factors have roughly the same average effects as when studied in high-income countries. Stability of aggression over a 3-year period is almost identical across low- and middle-income countries and high-income countries. Dimensions of comorbid psychopathology such as low self-control, hyperactivity, and sensation seeking are associated with antisocial behavior in low- and middle-income countries, but some early physical health factors have consistently weak or null effects.

Antibiotic therapy for preventing infections in people with acute stroke

BACKGROUND

Stroke is the main cause of disability in high-income countries and ranks second as a cause of death worldwide. Infections occur frequently after stroke and may adversely affect outcome. Preventive antibiotic therapy in the acute phase of stroke may reduce the incidence of infections and improve outcome. In the previous version of this Cochrane Review, published in 2012, we found that antibiotics did reduce the risk of infection but did not reduce the number of dependent or deceased patients. However, included studies were small and heterogeneous. In 2015, two large clinical trials were published, warranting an update of this Review.

OBJECTIVES

To assess the effectiveness and safety of preventive antibiotic therapy in people with ischaemic or haemorrhagic stroke. We wished to determine whether preventive antibiotic therapy in people with acute stroke:• reduces the risk of a poor functional outcome (dependency and/or death) at follow-up;• reduces the occurrence of infections in the acute phase of stroke;• reduces the occurrence of elevated body temperature (temperature ≥ 38° C) in the acute phase of stroke;• reduces length of hospital stay; or• leads to an increased rate of serious adverse events, such as anaphylactic shock, skin rash, or colonisation with antibiotic-resistant micro-organisms.

SEARCH METHODS

We searched the Cochrane Stroke Group Trials Register (25 June 2017); the Cochrane Central Register of Controlled Trials (CENTRAL; 2017, Issue 5; 25 June 2017) in the Cochrane Library; MEDLINE Ovid (1950 to 11 May 2017), and Embase Ovid (1980 to 11 May 2017). In an effort to identify further published, unpublished, and ongoing trials, we searched trials and research registers, scanned reference lists, and contacted trial authors, colleagues, and researchers in the field.

SELECTION CRITERIA

Randomised controlled trials (RCTs) of preventive antibiotic therapy versus control (placebo or open control) in people with acute ischaemic or haemorrhagic stroke.

DATA COLLECTION AND ANALYSIS

Two review authors independently selected articles and extracted data; we discussed and resolved discrepancies at a consensus meeting with a third review author. We contacted study authors to obtain missing data when required. An independent review author assessed risk of bias using the Cochrane 'Risk of bias' tool. We calculated risk ratios (RRs) for dichotomous outcomes, assessed heterogeneity amongst included studies, and performed subgroup analyses on study quality.

MAIN RESULTS

We included eight studies involving 4488 participants. Regarding quality of evidence, trials showed differences in study population, study design, type of antibiotic, and definition of infection; however, primary outcomes among the included studies were consistent. Mortality rate in the preventive antibiotic group was not significantly different from that in the control group (373/2208 (17%) vs 360/2214 (16%); RR 1.03, 95% confidence interval (CI) 0.87 to 1.21; high-quality evidence). The number of participants with a poor functional outcome (death or dependency) in the preventive antibiotic therapy group was also not significantly different from that in the control group (1158/2168 (53%) vs 1182/2164 (55%); RR 0.99, 95% CI 0.89 to 1.10; moderate-quality evidence). However, preventive antibiotic therapy did significantly reduce the incidence of 'overall' infections in participants with acute stroke from 26% to 19% (408/2161 (19%) vs 558/2156 (26%); RR 0.71, 95% CI 0.58 to 0.88; high-quality evidence). This finding was highly significant for urinary tract infections (81/2131 (4%) vs 204/2126 (10%); RR 0.40, 95% CI 0.32 to 0.51; high-quality evidence), whereas no preventive effect for pneumonia was found (222/2131 (10%) vs 235/2126 (11%); RR 0.95, 95% CI 0.80 to 1.13; high-quality evidence). No major side effects of preventive antibiotic therapy were reported. Only two studies qualitatively assessed the occurrence of elevated body temperature; therefore, these results could not be pooled. Only one study reported length of hospital stay.

AUTHORS' CONCLUSIONS

Preventive antibiotics had no effect on functional outcome or mortality, but significantly reduced the risk of 'overall' infections. This reduction was driven mainly by prevention of urinary tract infection; no effect for pneumonia was found.

A systematic review of comparisons between protocols or registrations and full reports in primary biomedical research

BACKGROUND

Prospective study protocols and registrations can play a significant role in reducing incomplete or selective reporting of primary biomedical research, because they are pre-specified blueprints which are available for the evaluation of, and comparison with, full reports. However, inconsistencies between protocols or registrations and full reports have been frequently documented. In this systematic review, which forms part of our series on the state of reporting of primary biomedical, we aimed to survey the existing evidence of inconsistencies between protocols or registrations (i.e., what was planned to be done and/or what was actually done) and full reports (i.e., what was reported in the literature); this was based on findings from systematic reviews and surveys in the literature.

METHODS

Electronic databases, including CINAHL, MEDLINE, Web of Science, and EMBASE, were searched to identify eligible surveys and systematic reviews. Our primary outcome was the level of inconsistency (expressed as a percentage, with higher percentages indicating greater inconsistency) between protocols or registration and full reports. We summarized the findings from the included systematic reviews and surveys qualitatively.

RESULTS

There were 37 studies (33 surveys and 4 systematic reviews) included in our analyses. Most studies (n = 36) compared protocols or registrations with full reports in clinical trials, while a single survey focused on primary studies of clinical trials and observational research. High inconsistency levels were found in outcome reporting (ranging from 14% to 100%), subgroup reporting (from 12% to 100%), statistical analyses (from 9% to 47%), and other measure comparisons. Some factors, such as outcomes with significant results, sponsorship, type of outcome and disease speciality were reported to be significantly related to inconsistent reporting.

CONCLUSIONS

We found that inconsistent reporting between protocols or registrations and full reports of primary biomedical research is frequent, prevalent and suboptimal. We also identified methodological issues such as the need for consensus on measuring inconsistency across sources for trial reports, and more studies evaluating transparency and reproducibility in reporting all aspects of study design and analysis. A joint effort involving authors, journals, sponsors, regulators and research ethics committees is required to solve this problem.

Outcomes in randomised controlled trials in prevention and management of carious lesions: a systematic review

Background

Inconsistent outcome reporting is one significant hurdle to combining results from trials into systematic reviews. Core outcome sets (COS) can reduce this barrier. The aim of this review was to map outcomes reported in caries prevention and management randomised controlled trials (RCT) as a first step to COS development. We also investigated RCT characteristics and reporting of primary outcomes and sample size calculations.

Methods

PubMed, Embase, Web of Knowledge and Cochrane CENTRAL were systematically searched (1 January 1968 to 25 August 2015). Inclusion criteria: RCTs comparing any technique for prevention or management of caries with another or placebo and RCTs comparing interventions to support patients undergoing treatment of caries (without setting, dentition or age restrictions). Categories were developed through piloting and group consensus and outcomes grouped accordingly.

Results

Of 4773 search results, 764 were potentially relevant, full text was available for 731 papers and 605 publications met the inclusion criteria and were included. For all outcomes across the time periods 1968–1980 and 2001–2010, reporting of outcome ‘caries experience’ reduced from 39% to 18%; ‘clinical performance of the restoration’ reporting increased from 33% to 42% although there was a reduction to 22% in 2011–2015. Emerging outcome domains include ‘lesion activity’ and ‘pulp health-related outcomes’, accounting for 1% and 0%, respectively, during 1968–1980 and 10% and 4% for 2011–2015. Reporting ‘resource efficiency’ and ‘quality of life measures’ have remained at a low level. No publications reported tooth survival independent of an index such as DMFT or equivalent. Primary outcomes were only identified as such in 414 (68%) of the reports.

Conclusions

Over the past 50 years, outcome reporting for trials on prevention and management of carious lesions have tended to focus on outcomes measuring caries experience and restoration material clinical performance with lesion activity and cost-effectiveness increasingly being reported. Patient-reported and patient-focused outcomes are becoming more common (although as secondary outcomes) but remain low in use. The challenge with developing a COS will be balancing commonly previously reported outcomes against those more relevant for the future.

Trial registration

PROSPERO, CRD42015025310. Registered on 14 August 2015, Trials (Schwendicke et al., Trials 16:397, 2015) and COMET initiative online (COMET, 2017).

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-017-2256-1) contains supplementary material, which is available to authorized users.

The perceived feasibility of methods to reduce publication bias

Publication bias is prevalent within the scientific literature. Whilst there are multiple ideas on how to reduce publication bias, only a minority of journals have made substantive changes to address the problem. We aimed to explore the perceived feasibility of strategies to reduce publication bias by gauging opinions of journal editors (n = 73) and other academics/researchers (n = 160) regarding nine methods of publishing and peer-reviewing research: mandatory publication, negative results journals/articles, open reviewing, peer-review training and accreditation, post-publication review, pre-study publication of methodology, published rejection lists, research registration, and two-stage review. Participants completed a questionnaire asking both quantitative (multiple choice or Likert scales) and qualitative (open-ended) questions regarding the barriers to implementing each suggestion, and their strengths and limitations. Participants were asked to rate the nine suggestions, then choose the method they felt was most effective. Mandatory publication was most popularly selected as the 'most effective' method of reducing publication bias for editors (25%), and was the third most popular choice for academics/researchers (14%). The most common selection for academics/researchers was two-stage review (26%), but fewer editors prioritised this (11%). Negative results journals/articles were the second and third most common choices for academics/researchers (21%) and editors (16%), respectively. Editors more commonly chose research registration as 'most effective' (21%), which was favoured by only 6% of academics/researchers. Whilst mandatory publication was generally favoured by respondents, it is infeasible to trial at a journal level. Where suggestions have already been implemented (e.g. negative results journals/articles, trial registration), efforts should be made to objectively assess their efficacy. Two-stage review should be further trialled as its popularity amongst academics/researchers suggests it may be well received, though editors may be less receptive. Several underlying barriers to change also emerged, including scientific culture, impact factors, and researcher training; these should be further explored to reduce publication bias.

Effects of Qigong on symptom management in cancer patients: A systematic review

PURPOSE

Qigong as a complementary and alternative therapy is often used by cancer patients for symptom management. The aim of this systematic review is to critically evaluate the effectiveness of Qigong on symptom management among cancer patients.

METHODS

A systematic search was conducted in the following databases from their inception through May 2016: Cochrane Library, PubMed/MEDLINE, CINAHL, PsycINFO, PEDRO. All controlled clinical trials of Qigong among cancer patients were included. The strength of the evidence was evaluated for all included studies using the Oxford Centre for Evidence-based Medicine Levels of Evidence. The risk of bias was assessed using the Cochrane Collaboration's Tool for Assessing Risk of Bias.

RESULTS

Twenty-two studies including fifteen randomized controlled trials and eight controlled clinical trials examined the efficacy of Qigong in symptom management among patients with various cancers. Results of these studies indicated that symptoms in the Qigong group were significantly improved or there was an observed positive trend from pre-to post-interventions scores for physical symptoms and psychological symptoms.

CONCLUSION

The effectiveness of Qigong as a health practice adopted by cancer patients to manage their symptoms during their cancer journey is not proven, but there are promising results that need further verification in future research.

Outcome reporting bias in randomized-controlled trials investigating antipsychotic drugs

Recent literature hints that outcomes of clinical trials in medicine are selectively reported. If applicable to psychotic disorders, such bias would jeopardize the reliability of randomized clinical trials (RCTs) investigating antipsychotics and thus their extrapolation to clinical practice. We therefore comprehensively examined outcome reporting bias in RCTs of antipsychotic drugs by a systematic review of prespecified outcomes on ClinicalTrials.gov records of RCTs investigating antipsychotic drugs in schizophrenia and schizoaffective disorder between 1 January 2006 and 31 December 2013. These outcomes were compared with outcomes published in scientific journals. Our primary outcome measure was concordance between prespecified and published outcomes; secondary outcome measures included outcome modifications on ClinicalTrials.gov after trial inception and the effects of funding source and directionality of results on record adherence. Of the 48 RCTs, 85% did not fully adhere to the prespecified outcomes. Discrepancies between prespecified and published outcomes were found in 23% of RCTs for primary outcomes, whereas 81% of RCTs had at least one secondary outcome non-reported, newly introduced, or changed to a primary outcome in the respective publication. In total, 14% of primary and 44% of secondary prespecified outcomes were modified after trial initiation. Neither funding source (P=0.60) nor directionality of the RCT results (P=0.10) impacted ClinicalTrials.gov record adherence. Finally, the number of published safety endpoints (N=335) exceeded the number of prespecified safety outcomes by 5.5 fold. We conclude that RCTs investigating antipsychotic drugs suffer from substantial outcome reporting bias and offer suggestions to both monitor and limit such bias in the future.

A systematic review of the processes used to link clinical trial registrations to their published results

BACKGROUND

Studies measuring the completeness and consistency of trial registration and reporting rely on linking registries with bibliographic databases. In this systematic review, we quantified the processes used to identify these links.

METHODS

PubMed and Embase databases were searched from inception to May 2016 for studies linking trial registries with bibliographic databases. The processes used to establish these links were categorised as automatic when the registration identifier was available in the bibliographic database or publication, or manual when linkage required inference or contacting of trial investigators. The number of links identified by each process was extracted where available. Linear regression was used to determine whether the proportions of links available via automatic processes had increased over time.

RESULTS

In 43 studies that examined cohorts of registry entries, 24 used automatic and manual processes to find articles; 3 only automatic; and 11 only manual (5 did not specify). Twelve studies reported results for both manual and automatic processes and showed that a median of 23% (range from 13 to 42%) included automatic links to articles, while 17% (range from 5 to 42%) of registry entries required manual processes to find articles. There was no evidence that the proportion of registry entries with automatic links had increased (R ² = 0.02, p = 0.36). In 39 studies that examined cohorts of articles, 21 used automatic and manual processes; 9 only automatic; and 2 only manual (7 did not specify). Sixteen studies reported numbers for automatic and manual processes and indicated that a median of 49% (range from 8 to 97%) of articles had automatic links to registry entries, and 10% (range from 0 to 28%) required manual processes to find registry entries. There was no evidence that the proportion of articles with automatic links to registry entries had increased (R ² = 0.01, p = 0.73).

CONCLUSIONS

The linkage of trial registries to their corresponding publications continues to require extensive manual processes. We did not find that the use of automatic linkage has increased over time. Further investigation is needed to inform approaches that will ensure publications are properly linked to trial registrations, thus enabling efficient monitoring of trial reporting.

Agreement on what to measure in randomised controlled trials in burn care: study protocol for the development of a core outcome set

INTRODUCTION

In 2004, nearly 11 million severely burn-injured patients required medical care worldwide. Burns cause prolonged hospitalisation and long-term disability. Although mortality has been reduced, morbidity remains significant.Burn care is costly and decision-making is challenging. A range of procedures are performed at different times after injury; new technology is emerging and alternate care pathways are regularly introduced. Data to guide evidence-based decision-making are lacking. Researchers use different outcomes to assess recovery, so it is not possible to combine trial information to draw meaningful conclusions. Early recovery measures include length of hospital stay, healing time and treatment complications. Longer-term outcomes include issues with function, cosmesis and psychological health. Reporting an agreed set of the most important outcomes (core outcome set (COS)) in randomised controlled trials (RCTs) will allow effective evidence synthesis to support clinical decisions. Patient input will ensure relevance.

METHODS AND ANALYSIS

The aim is to produce a burn COS for RCT reporting. A long list of outcomes will be identified through systematic reviews of clinical and patient-reported outcomes. Additional outcomes will be identified from interviews with patients over 10 years, parents of children of any age and multidisciplinary professionals. A two-stage modified Delphi exercise will be undertaken to prioritise and condense the list, with patients (n=150) at different stages of recovery. We will also include nursing, therapy (n=100) and medical staff (n=100). A reduced list will be taken to consensus meetings with families and clinical staff to achieve a final COS.

ETHICS AND DISSEMINATION

A COS will reduce outcome reporting heterogeneity in burn care research, allowing more effective use of research funding and facilitating evidence synthesis and evidence-based clinical decision-making. Stakeholders will include journal editors, health commissioners, researchers, patients and professionals. The study has ethical approval and is registered with Core Outcome Measures in Effectiveness Trials Initiative (http://www.comet-initiative.org/studies/details/798?result=true).

Evidence of selective reporting bias in hematology journals: A systematic review

Introduction

Selective reporting bias occurs when chance or selective outcome reporting rather than the intervention contributes to group differences. The prevailing concern about selective reporting bias is the possibility of results being modified towards specific conclusions. In this study, we evaluate randomized controlled trials (RCTs) published in hematology journals, a group in which selective outcome reporting has not yet been explored.

Methods

Our primary goal was to examine discrepancies between the reported primary and secondary outcomes in registered and published RCTs concerning hematological malignancies reported in hematology journals with a high impact factor. The secondary goals were to address whether outcome reporting discrepancies favored statistically significant outcomes, whether a pattern existed between the funding source and likelihood of outcome reporting bias, and whether temporal trends were present in outcome reporting bias. For trials with major outcome discrepancies, we contacted trialists to determine reasons for these discrepancies. Trials published between January 1, 2010 and December 31, 2015 in Blood; British Journal of Haematology; American Journal of Hematology; Leukemia; and Haematologica were included.

Results

Of 499 RCTs screened, 109 RCTs were included. Our analysis revealed 118 major discrepancies and 629 total discrepancies. Among the 118 discrepancies, 30 (25.4%) primary outcomes were demoted, 47 (39.8%) primary outcomes were omitted, and 30 (25.4%) primary outcomes were added. Three (2.5%) secondary outcomes were upgraded to a primary outcome. The timing of assessment for a primary outcome changed eight (6.8%) times. Thirty-one major discrepancies were published with a P-value and twenty-five (80.6%) favored statistical significance. A majority of authors whom we contacted cited a pre-planned subgroup analysis as a reason for outcome changes.

Conclusion

Our results suggest that outcome changes occur frequently in hematology trials. Because RCTs ultimately underpin clinical judgment and guide policy implementation, selective reporting could pose a threat to medical decision making.

Adapting Internet-Delivered Parent-Child Interaction Therapy to Treat Co-Occurring Disruptive Behavior and Callous-Unemotional Traits: A Case Study

Disruptive behavior disorders (DBD) are highly prevalent, emerge in early childhood, exhibit considerable stability across time, and are associated with profound disability. When DBD co-occur with callous-unemotional (CU) traits (i.e., lack of empathy/guilt), the risk of early-onset, stable, and severe disruptive behavior is even higher, relative to DBD alone. Early intervention is critical, and there is robust empirical support for the efficacy of parent management training (PMT) for reducing disruptive behavior in young children. However, broad access to these interventions is hindered by numerous systemic barriers, including geographic disparities in availability of services. To overcome these barriers and enhance access and quality of care to underserved communities, several PMT programs have been adapted to online delivery formats, including Parent-Child Interaction Therapy (PCIT). PCIT is an evidence-supported treatment that attempts to reduce disruptive child behavior by improving the parent–child relationship and implementing consistent and effective discipline strategies. Comer and colleagues proposed an online adaptation of PCIT (I-PCIT) that is delivered using video teleconferencing (VTC). I-PCIT was implemented with the family of a 5-year-old Australian boy presenting with clinically significant disruptive behavior and CU traits living in a rural community. Findings from this case report (a) document an improvement in disruptive behavior that was maintained to follow-up and (b) provide preliminary support for adapting PCIT to online delivery formats to enhance accessibility of services and improve child and parent outcomes.