Are reports of randomized controlled trials improving over time? A systematic review of 284 articles published in high-impact general and specialized medical journals

Quality of reporting inflammatory bowel disease randomised controlled trials: a systematic review

OBJECTIVE

Our objective was to perform a systemic evaluation of the risk of bias in randomised controlled trial (RCT) reports published on inflammatory bowel disease (IBD).

DESIGN

We assessed the risk of bias using the Cochrane tool, as indicators of poor methodology or subsequently poor reporting. We systematically selected, with dual independent judgements, all studies published on IBD with no time limits and assessed the methodological quality of included studies again using independent dual ratings.

RESULTS

563 full texts were included after selection and review. No abstract publications were free of any source of bias. Full-text publications still fared badly, as only 103 full-text papers exhibited a low risk of bias in all reporting domains when excluding blinding. RCTs published in journals with higher impact factor (IF) were associated with an overall reduced rate of being at high risk. However, only 6% of full RCT publications in journals with an IF greater than 10, published in the past 5 years, were free of bias.The trend over time is towards improved reporting in all areas. Trials published by larger author teams, in full-text form and by industry and public sponsorship were positively correlated with a lower risk of bias. Only allocation concealment showed a statistically significant improvement with time (p=0.037).

CONCLUSION

These findings are consistent with those of other specialties in the literature. While this unclear risk of bias may represent poor reporting of methods instead of poor methodological quality, it leaves readers and future secondary researchers with significant questions regarding such key issues.

Randomized controlled trials in lung cancer surgery: How are we doing?

Objective

Randomized control trials are considered the highest level of evidence, yet the scalability and practicality of implementing randomized control trials in the thoracic surgical oncology space are not well described. The aim of this study is to understand what types of randomized control trials have been conducted in thoracic surgical oncology and ascertain their success rate in completing them as originally planned.

Methods

The ClinicalTrials.gov database was queried in April 2023 to identify registered randomized control trials performed in patients with lung cancer who underwent surgery (by any technique) as part of their treatment.

Results

There were 68 eligible randomized control trials; 33 (48.5%) were intended to examine different perioperative patient management strategies (eg, analgesia, ventilation, drainage) or to examine different intraoperative technical aspects (eg, stapling, number of ports, port placement, ligation). The number of randomized control trials was relatively stable over time until a large increase in randomized control trials starting in 2016. Forty-four of the randomized control trials (64.7%) were open-label studies, 43 (63.2%) were conducted in a single facility, 66 (97.1%) had 2 arms, and the mean number of patients enrolled per randomized control trial was 236 (SD, 187). Of 21 completed randomized control trials (31%), the average time to complete accrual was 1605 days (4.4 years) and average time to complete primary/secondary outcomes and adverse events collection was 2125 days (5.82 years).

Conclusions

Given the immense investment of resources that randomized control trials require, these findings suggest the need to scrutinize future randomized control trial proposals to assess the likelihood of successful completion. Future study is needed to understand the various contributing factors to randomized control trial success or failure.

Analysis of congenital heart disease research: Mapping impact, production and global collaboration

Background

Congenital heart disease (CHD) is a severe childhood health problem worldwide. This study analyzes the scientific production on CHD published in Scopus during the last decade.

Methods

This study assessed the research on CHD conducted during the 2011-2020 period. Indexed publications were identified using a list of keywords consisting of MeSH (Medical Subject Heading) and Emtree (EMBASE) terms for the category "Congenital Heart Defects." The studies obtained were analyzed using the SciVal platform, and the production, impact, and collaboration indicators were considered. Additionally, an analysis of co-occurrence between countries and keywords was carried out with the VOSviewer software.

Results

The study found 8934 articles on CHD. A growing trend was observed during the last decade. The distribution of the scientific production at the international level shows that most publications are from the United States, China, and the United Kingdom, whereas Latin American countries have the lowest production. The articles analyzed contain a total of 89,835 citations. Most studies are published in high-impact journals (Q1), with 3273 studies corresponding to 39.3% of the total production analyzed. In the keyword analysis, we found five areas investigated.

Conclusions

Scientific production has shown sustained growth in the past years. Increased scientific production in specific research areas may have helped decrease morbimortality due to CHD.

Reporting quality was suboptimal in a systematic review of randomized controlled trials with adaptive designs

OBJECTIVE

The study was conducted to evaluate the reporting quality of randomized controlled trials (RCTs) that use an adaptive design (AD) based on the 2020 AD Consolidated Standards for Reporting Trials 2010 extension (ACE) guidelines and identify factors associated with better reporting quality.

STUDY DESIGN AND SETTING

PubMed, Embase, Cochrane, Web of Science, and Google Scholar were searched until November 1, 2022. Multivariable linear regression analysis was performed to investigate potential predictors.

RESULTS

In total, 109 RCTs were included in our study. The mean compliance rate for the ACE checklist items was 69.75% ± 16.02. Key methodological items including allocation concealment and its implementations were poorly reported. There was also suboptimal reporting of checklist items related to the conduct of interim analyzes. Multivariable regression analysis showed better reporting quality with trial registration, nonindustry affiliation (first author), a sample size of >100, general medical journal type, publication date (≥2020), funding, and disclosure of the number of interim analyzes.

CONCLUSION

Our study showed that RCTs with AD had suboptimal reporting of 2020 ACE checklist items, particularly AD-specific items. Following the development of ACE guidelines, stricter adherence to the ACE guideline is necessary to improve their reporting quality. Pre-ACE and post-ACE adherence comparisons can be conducted in the future.

Transparent transdisciplinary reporting in geriatric research using the EQUATOR Network

This editorial comments on the article by Carpenter et al. in this issue.

CYP35 family in Caenorhabditis elegans biological processes: fatty acid synthesis, xenobiotic metabolism, and stress responses

With more than 80 cytochrome P450 (CYP) encoding genes found in the nematode Caenorhabditis elegans (C. elegans), the cyp35 genes are one of the important genes involved in many biological processes such as fatty acid synthesis and storage, xenobiotic stress response, dauer and eggshell formation, and xenobiotic metabolism. The C. elegans CYP35 subfamily consisted of A, B, C, and D, which have the closest homolog to human CYP2 family. C. elegans homologs could answer part of the hunt for human disease genes. This review aims to provide an overview of CYP35 in C. elegans and their human homologs, to explore the roles of CYP35 in various C. elegans biological processes, and how the genes of cyp35 upregulation or downregulation are influenced by biological processes, upon exposure to xenobiotics or changes in diet and environment. The C. elegans CYP35 gene expression could be upregulated by heavy metals, pesticides, anti-parasitic and anti-chemotherapeutic agents, polycyclic aromatic hydrocarbons (PAHs), nanoparticles, drugs, and organic chemical compounds. Among the cyp35 genes, cyp-35A2 is involved in most of the C. elegans biological processes regulation. Further venture of cyp35 genes, the closest homolog of CYP2 which is the largest family of human CYPs, may have the power to locate cyps gene targets, discovery of novel therapeutic strategies, and possibly a successful medical regime to combat obesity, cancers, and cyps gene-related diseases.

Role of Placental Glucose Transporters in Determining Fetal Growth

Maternal nutrient availability and its transport through the placenta are crucial for fetal development. Nutrients are transported to the fetus via specific transporters present on the microvillous (MVM) and basal membrane (BM) of the placenta. Glucose is the most abundant nutrient transferred to the fetus and plays a key role in the fetal growth and development. The transfer of glucose across the human placenta is directly proportional to maternal glucose concentrations, and is mediated by glucose transporter family proteins (GLUTs). Maternal glucose concentration influences expression and activity of GLUTs in the MVM (glucose uptake) and BM (glucose delivery). Alteration in the number and function of these transporters may affect the growth and body composition of the fetus. The thin-fat phenotype of the Indian baby (low ponderal index, high adiposity) is proposed as a harbinger of future metabolic risk. We propose that placental function mediated through nutrient transporters contributes to the phenotype of the baby, specifically that glucose transporters will influence neonatal fat. This review discusses the role of various glucose transporters in the placenta in determining fetal growth and body composition, in light of the above hypothesis.

[A network meta-analysis of the efficacy and safety of baloxavir marboxil versus neuraminidase inhibitors for the treatment of influenza in otherwise healthy patients]

AIM

Baloxavir marboxil (baloxavir) is the first cap-dependent endonuclease inhibitor being studied for the treatment of influenza in single oral dosing regimen. This network meta-analysis (NMA) evaluated the efficacy and safety of baloxavir compared to other antivirals for influenza in otherwise healthy patients.

METHODS

A systematic literature review was performed on 14 November 2016 in Medline, Embase, CENTRAL, and ICHUSHI to identify randomized controlled trials assessing antivirals for influenza. A NMA including 22 trials was performed to compare the efficacy and safety of baloxavir with other antivirals.

RESULTS

The time to alleviation of all symptoms was significantly shorter for baloxavir compared to zanamivir (difference in median time 19.96 h; 95% CrI [3.23, 39.07]). The time to cessation of viral shedding was significantly shorter for baloxavir than zanamivir and oseltamivir (47.00 h; 95% CrI [28.18, 73.86] and 56.03 h [33.74, 87.86], respectively). The mean decline in virus titer from baseline to 24 h was significantly greater for baloxavir than for the other drugs. Other differences in efficacy outcomes were not significant. No significant differences were found between baloxavir and the other antivirals for safety, except total drug-related adverse events where baloxavir demonstrated a decrease compared to oseltamivir and laninamivir.

CONCLUSION

The NMA suggests that baloxavir demonstrated better or similar efficacy results compared to other antivirals with a comparable safety profile. Baloxavir led to a significant decrease in viral titer versus zanamivir, oseltamivir and peramivir and decreased viral shedding versus zanamivir and oseltamivir.

A network meta-analysis of the efficacy and safety of baloxavir marboxil versus neuraminidase inhibitors for the treatment of influenza in otherwise healthy patients

Objective: Baloxavir marboxil (baloxavir) is the first cap-dependent endonuclease inhibitor being studied for the treatment of influenza in single oral dosing regimen. This network meta-analysis (NMA) evaluated the efficacy and safety of baloxavir compared to other antivirals for influenza in otherwise healthy patients. Methods: A systematic literature review was performed on 14 November 2016 in Medline, Embase, CENTRAL, and ICHUSHI to identify randomized controlled trials assessing antivirals for influenza. A NMA including 22 trials was performed to compare the efficacy and safety of baloxavir with other antivirals. Results: The time to alleviation of all symptoms was significantly shorter for baloxavir compared to zanamivir (difference in median time 19.96 h; 95% CrI [3.23, 39.07]). The time to cessation of viral shedding was significantly shorter for baloxavir than zanamivir and oseltamivir (47.00 h; 95% CrI [28.18, 73.86] and 56.03 h [33.74, 87.86], respectively). The mean decline in virus titer from baseline to 24 h was significantly greater for baloxavir than for the other drugs. Other differences in efficacy outcomes were not significant. No significant differences were found between baloxavir and the other antivirals for safety, except total drug-related adverse events where baloxavir demonstrated a decrease compared to oseltamivir and laninamivir. Conclusions: The NMA suggests that baloxavir demonstrated better or similar efficacy results compared to other antivirals with a comparable safety profile. Baloxavir led to a significant decrease in viral titer versus zanamivir, oseltamivir and peramivir and decreased viral shedding versus zanamivir and oseltamivir.

Reporting quality of European and Croatian health practice guidelines according to the RIGHT reporting checklist

Background

Health practice guidelines (HPGs) are important tools for the translation of evidence into practice. Reporting Items for Practice Guidelines in HealThcare (RIGHT) checklist provides guidance on reporting health practice guidelines (HPGs). We assessed the reporting completeness and quality of a set of national (Croatian) and relevant transnational (European) HPGs.

Methods

The national sample included all HPGs published in the official journal of the Croatian Medical Association in 2014–2016. We searched PubMed to identify relevant European guidelines (n = 24). Two independent reviewers assessed the adherence with the items on the RIGHT checklist. Kappa score was used to measure the level of agreement. Frequentist and Bayes statistics Bayes factor (BF₁₀) was used to evaluate the differences between the national and transnational HPGs.

Results

Overall, Croatian and European HPGs adhered to less than 50% of RIGHT checklist items. Croatian HPGs reported a median of 14.0 (95% CI 13.0–15.0) RIGHT reporting items, and European counterparts reported a median of 16.0 (95% CI 14.0–17.2) out of the total of 35 checklist items (Mann Whitney U test, P = 0.048; BF₁₀ = 1.543). European HPGs were better than Croatian HPGs in reporting stakeholder involvement and values and preferences (BF₁₀ = 80.63), as well as describing the implications of costs and resources (BF₁₀ = 55.15). Croatian HPGs better reported HPGs specified aims (BF₁₀ = 16.90), primary intended users (BF₁₀ = 8.70), and sources of funding (BF₁₀ = 122.90). Most insufficiently reported items for both HPG sets were defining the guideline questions and clear outcomes, quality assurance, management of funding and conflicts of interest, and guideline limitations.

Conclusions

Important methodological details are missing from most published HPGs at national and transnational levels. To ensure better quality and adequate use of HPGs, reporting guidelines should be endorsed and used by developers and users alike.

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0828-4) contains supplementary material, which is available to authorized users.

Comparison of the Quality of Pediatric Randomized Controlled Trials Published in Both Nursing and Medical Journals: Adherence to the CONSORT Statement

BACKGROUND

Nursing care should be based on scientific evidence. However, studies must be performed rigorously with accurate reporting for their findings to be applicable to practice. Since the body of scientific nursing literature is broad, the quality and validity of its findings should be regularly controlled and verified to ensure their application and their practical impact.

PURPOSE

To compare reporting quality of pediatric randomized controlled trial (RCT) articles in nursing and medical journals.

METHODS

Randomly selected articles were reviewed and scored to assess the number of CONSORT items that were adequately reported, generating a CONSORT score. The CONSORT scores for 28 items were compared between the two journal types.

RESULTS AND DISCUSSION

The CONSORT scores by journal type were not significantly different: (19.2 [16.2; 22] for medical journals and 19.5 [16.1; 21.5] for nursing journals, p = .77). The reporting of CONSORT items was poor for both journal types. However, there were two significant differences: item 19 (Declaration of all important harm or unintended effects, p = .0006) and item 23 (Registration number of the study, p = .0003), were reported more often in medical journals. The adherence of journals to the CONSORT statement and large sample size was associated with better quality of the reporting of studies.

CONCLUSIONS

Based on reporting quality, nursing studies have the same scientific credibility and rigor as medical studies in the pediatric field.

LINKING EVIDENCE TO ACTION

The findings of this study could help researchers improve the reporting of their studies and highlight the importance of reporting quality for future knowledge transfer and practical use. The quality of research and its reporting is necessary to improve knowledge transfer into practice.

How to Report Exotic Animal Research

Reporting the results of primary research is a key step in knowledge creation. Many well-conducted studies are rejected by journal editors, criticized by peers, or unsuitable for systematic reviewers because of poor reporting. This article summarizes the most important methodological items to report when writing an original research article.

Avoidable waste related to inadequate methods and incomplete reporting of interventions: a systematic review of randomized trials performed in Sub-Saharan Africa

Background

Randomized controlled trials (RCTs) are needed to improve health care in Sub-Saharan Africa (SSA). However, inadequate methods and incomplete reporting of interventions can prevent the transposition of research in practice which leads waste of research. The aim of this systematic review was to assess the avoidable waste in research related to inadequate methods and incomplete reporting of interventions in RCTs performed in SSA.

Methods

We performed a methodological systematic review of RCTs performed in SSA and published between 1 January 2014 and 31 March 2015. We searched PubMed, the Cochrane library and the African Index Medicus to identify reports. We assessed the risk of bias using the Cochrane Risk of Bias tool, and for each risk of bias item, determined whether easy adjustments with no or minor cost could change the domain to low risk of bias. The reporting of interventions was assessed by using standardized checklists based on the Consolidated Standards for Reporting Trials, and core items of the Template for Intervention Description and Replication. Corresponding authors of reports with incomplete reporting of interventions were contacted to obtain additional information. Data were descriptively analyzed.

Results

Among 121 RCTs selected, 74 (61%) evaluated pharmacological treatments (PTs), including drugs and nutritional supplements; and 47 (39%) nonpharmacological treatments (NPTs) (40 participative interventions, 1 surgical procedure, 3 medical devices and 3 therapeutic strategies). Overall, the randomization sequence was adequately generated in 76 reports (62%) and the intervention allocation concealed in 48 (39%). The primary outcome was described as blinded in 46 reports (38%), and incomplete outcome data were adequately addressed in 78 (64%). Applying easy methodological adjustments with no or minor additional cost to trials with at least one domain at high risk of bias could have reduced the number of domains at high risk for 24 RCTs (19%). Interventions were completely reported for 73/121 (60%) RCTs: 51/74 (68%) of PTs and 22/47 (46%) of NPTs. Additional information was obtained from corresponding authors for 11/48 reports (22%).

Conclusion

Inadequate methods and incomplete reporting of published SSA RCTs could be improved by easy and inexpensive methodological adjustments and adherence to reporting guidelines.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-017-2034-0) contains supplementary material, which is available to authorized users.

Mobile access to virtual randomization for investigator-initiated trials

Background/aims Randomization is indispensable in clinical trials in order to provide unbiased treatment allocation and a valid statistical inference. Improper handling of allocation lists can be avoided using central systems, for example, human-based services. However, central systems are unaffordable for investigator-initiated trials and might be inaccessible from some places, where study subjects need allocations. We propose mobile access to virtual randomization, where the randomization lists are non-existent and the appropriate allocation is computed on demand. Methods The core of the system architecture is an electronic data capture system or a clinical trial management system, which is extended by an R interface connecting the R server using the Java R Interface. Mobile devices communicate via the representational state transfer web services. Furthermore, a simple web-based setup allows configuring the appropriate statistics by non-statisticians. Our comprehensive R script supports simple randomization, restricted randomization using a random allocation rule, block randomization, and stratified randomization for un-blinded, single-blinded, and double-blinded trials. For each trial, the electronic data capture system or the clinical trial management system stores the randomization parameters and the subject assignments. Results Apps are provided for iOS and Android and subjects are randomized using smartphones. After logging onto the system, the user selects the trial and the subject, and the allocation number and treatment arm are displayed instantaneously and stored in the core system. So far, 156 subjects have been allocated from mobile devices serving five investigator-initiated trials. Conclusion Transforming pre-printed allocation lists into virtual ones ensures the correct conduct of trials and guarantees a strictly sequential processing in all trial sites. Covering 88% of all randomization models that are used in recent trials, virtual randomization becomes available for investigator-initiated trials and potentially for large multi-center trials.

Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a prospective study

BACKGROUND

The Cochrane risk of bias tool is commonly criticized for having a low reliability. We aimed to investigate whether training of raters, with objective and standardized instructions on how to assess risk of bias, can improve the reliability of the Cochrane risk of bias tool.

METHODS

In this pilot study, four raters inexperienced in risk of bias assessment were randomly allocated to minimal or intensive standardized training for risk of bias assessment of randomized trials of physical therapy treatments for patients with knee osteoarthritis pain. Two raters were experienced risk of bias assessors who served as reference. The primary outcome of our study was between-group reliability, defined as the agreement of the risk of bias assessments of inexperienced raters with the reference assessments of experienced raters. Consensus-based assessments were used for this purpose. The secondary outcome was within-group reliability, defined as the agreement of assessments within pairs of inexperienced raters. We calculated the chance-corrected weighted Kappa to quantify agreement within and between groups of raters for each of the domains of the risk of bias tool.

RESULTS

A total of 56 trials were included in our analysis. The Kappa for the agreement of inexperienced raters with reference across items of the risk of bias tool ranged from 0.10 to 0.81 for the minimal training group and from 0.41 to 0.90 for the standardized training group. The Kappa values for the agreement within pairs of inexperienced raters across the items of the risk of bias tool ranged from 0 to 0.38 for the minimal training group and from 0.93 to 1 for the standardized training group. Between-group differences in Kappa for the agreement of inexperienced raters with reference always favored the standardized training group and was most pronounced for incomplete outcome data (difference in Kappa 0.52, p < 0.001) and allocation concealment (difference in Kappa 0.30, p = 0.004).

CONCLUSIONS

Intensive, standardized training on risk of bias assessment may significantly improve the reliability of the Cochrane risk of bias tool.

Deficiencies of effectiveness of intervention studies in veterinary medicine: a cross-sectional survey of ten leading veterinary and medical journals

The validity of studies that assess the effectiveness of an intervention (EoI) depends on variables such as the type of study design, the quality of their methodology, and the participants enrolled. Five leading veterinary journals and 5 leading human medical journals were hand-searched for EoI studies for the year 2013. We assessed (1) the prevalence of randomized controlled trials (RCTs) among EoI studies, (2) the type of participants enrolled, and (3) the methodological quality of the selected studies. Of 1707 eligible articles, 590 were EoI articles and 435 RCTs. Random allocation to the intervention was performed in 52% (114/219; 95%CI:45.2-58.8%) of veterinary EoI articles, against 87% (321/371; 82.5-89.7%) of human EoI articles (adjusted OR:9.2; 3.4-24.8). Veterinary RCTs were smaller (median: 26 animals versus 465 humans) and less likely to enroll real patients, compared with human RCTs (OR:331; 45-2441). Only 2% of the veterinary RCTs, versus 77% of the human RCTs, reported power calculations, primary outcomes, random sequence generation, allocation concealment and estimation methods. Currently, internal and external validity of veterinary EoI studies is limited compared to human medical ones. To address these issues, veterinary interventional research needs to improve its methodology, increase the number of published RCTs and enroll real clinical patients.

Associated Factors and Consequences of Risk of Bias in Randomized Controlled Trials of Yoga: A Systematic Review

BACKGROUND

Bias in randomized controlled trials (RCTs) of complementary therapy interventions seems to be associated with specific factors and to potentially distort the studies' conclusions. This systematic review assessed associated factors of risk of bias and consequences for the studies' conclusions in RCTs of yoga as one of the most commonly used complementary therapies.

METHODS

Medline/PubMed, Scopus, IndMED and the Cochrane Library were searched through February 2014 for yoga RCTs. Risk of selection bias was assessed using the Cochrane tool and regressed to a) publication year; b) country of origin; c) journal type; and d) impact factor using multiple logistic regression analysis. Likewise, the authors' conclusions were regressed to risk of bias.

RESULTS

A total of 312 RCTs were included. Impact factor ranged from 0.0 to 39.2 (median = 1.3); 60 RCT (19.2%) had a low risk of selection bias, and 252 (80.8%) had a high or unclear risk of selection bias. Only publication year and impact factor significantly predicted low risk of bias; RCTs published after 2001 (adjusted odds ratio (OR) = 12.6; 95% confidence interval (CI) = 1.7, 94.0; p<0.001) and those published in journals with impact factor (adjusted OR = 2.6; 95%CI = 1.4, 4.9; p = 0.004) were more likely to have low risk of bias. The authors' conclusions were not associated with risk of bias.

CONCLUSIONS

Risk of selection bias was generally high in RCTs of yoga; although the situation has improved since the publication of the revised CONSORT statement 2001. Pre-CONSORT RCTs and those published in journals without impact factor should be handled with increased care; although risk of bias is unlikely to distort the RCTs' conclusions.

Impact of an online writing aid tool for writing a randomized trial report: the COBWEB (Consort-based WEB tool) randomized controlled trial

BACKGROUND

Incomplete reporting is a frequent waste in research. Our aim was to evaluate the impact of a writing aid tool (WAT) based on the CONSORT statement and its extension for non-pharmacologic treatments on the completeness of reporting of randomized controlled trials (RCTs).

METHODS

We performed a 'split-manuscript' RCT with blinded outcome assessment. Participants were masters and doctoral students in public health. They were asked to write, over a 4-hour period, the methods section of a manuscript based on a real RCT protocol, with a different protocol provided to each participant. Methods sections were divided into six different domains: 'trial design', 'randomization', 'blinding', 'participants', 'interventions', and 'outcomes'. Participants had to draft all six domains with access to the WAT for a random three of six domains. The random sequence was computer-generated and concealed. For each domain, the WAT comprised reminders of the corresponding CONSORT item(s), bullet points detailing all the key elements to be reported, and examples of good reporting. The control intervention consisted of no reminders. The primary outcome was the mean global score for completeness of reporting (scale 0-10) for all domains written with or without the WAT.

RESULTS

Forty-one participants wrote 41 different manuscripts of RCT methods sections, corresponding to 246 domains (six for each of the 41 protocols). All domains were analyzed. For the primary outcome, the mean (SD) global score for completeness of reporting was higher with than without use of the WAT: 7.1 (1.2) versus 5.0 (1.6), with a mean (95 % CI) difference 2.1 (1.5-2.7; P <0.01). Completeness of reporting was significantly higher with the WAT for all domains except for blinding and outcomes.

CONCLUSION

Use of the WAT could improve the completeness of manuscripts reporting the results of RCTs.

TRIAL REGISTRATION

Clinicaltrials.gov ( http://clinicaltrials.gov NCT02127567 , registration date first received April 29, 2014).

Registration of randomized clinical trials--a challenge

BACKGROUND

Registration of interventional studies is necessary according to the Declaration of Helsinki but implementation has been a challenge for many journals. Acta Anaesthesiologica Scandinavica (Acta) requires registration for studies conducted after January 1(st) 2010. We aimed to assess the proportion of correctly registered randomized controlled trials (RCTs) published in Acta from 2009 to 2014.

METHODS

We manually searched all Acta issues from 2009 to 2014 for RCTs. Information about timing of data collection and registration in trial registries was extracted. We classified RCTs as correctly registered when it could be verified that patient enrolment was started after registration in a trial registry.

RESULTS

We identified 200 RCTs. Dates for patient enrolment were not specified in 51 (25.5%). The proportion of correctly registered trials increased significantly from 17.1% (19/111) for trials starting enrolment before 2010 to 63.2% after 2010 (24/38, P < 0.01). Most clinical trials were registered at clinicaltrials.gov.

CONCLUSION

Many published randomized controlled trials from Acta Anaesthesiologica Scandinavica were not adequately registered but the requirement of trial registration has resulted in a significant increase in the proportion of correctly registered trials.

Guidelines for the Reporting of Treatment Trials for Alcohol Use Disorders

BACKGROUND

The primary goals in conducting clinical trials of treatments for alcohol use disorders (AUDs) are to identify efficacious treatments and determine which treatments are most efficacious for which patients. Accurate reporting of study design features and results is imperative to enable readers of research reports to evaluate to what extent a study has achieved these goals. Guidance on quality of clinical trial reporting has evolved substantially over the past 2 decades, primarily through the publication and widespread adoption of the Consolidated Standards of Reporting Trials statement. However, there is room to improve the adoption of those standards in reporting the design and findings of treatment trials for AUD.

METHODS

This paper provides a narrative review of guidance on reporting quality in AUD treatment trials.

RESULTS

Despite improvements in the reporting of results of treatment trials for AUD over the past 2 decades, many published reports provide insufficient information on design or methods.

CONCLUSIONS

The reporting of alcohol treatment trial design, analysis, and results requires improvement in 4 primary areas: (i) trial registration, (ii) procedures for recruitment and retention, (iii) procedures for randomization and intervention design considerations, and (iv) statistical methods used to assess treatment efficacy. Improvements in these areas and the adoption of reporting standards by authors, reviewers, and editors are critical to an accurate assessment of the reliability and validity of treatment effects. Continued developments in this area are needed to move AUD treatment research forward via systematic reviews and meta-analyses that maximize the utility of completed studies.

Changes to registration elements and results in a cohort of Clinicaltrials.gov trials were not reflected in published articles

OBJECTIVES

To assess effectiveness of legislative initiatives to stimulate public registration of trial results, we assessed adherence to protocol and results reporting, changes to registry, and publication data for randomized controlled trials (RCTs) after introduction of Food and Drug Administration Amendment Act (FDAAA).

STUDY DESIGN AND SETTING

Observational study of a cohort of ClinicalTrials.gov registered FDAAA-covered RCTs found through ClinicalTrials.gov between 2009 and 2012 and data from corresponding publications. WHO Minimum Data Set items were abstracted by one author and verified by the other author.

RESULTS

Among 81 eligible trials, most were industry-funded, with a drug intervention in parallel assignment. Secondary outcomes at the initial and last registration were omitted for 17% and 19.7% of RCTs, respectively. RCT registration changes mostly involved scientific title (18.8%). Inclusion criteria omission was most common (88%) in publications. Inferential statistical methods for primary and secondary outcomes matched between registry and publication for 53.4% and 28.6% of RCTs, respectively. Serious and other adverse events (AEs) that were absent for 23.8% and 4.8% of RCTs, respectively, were published as nonoccurring.

CONCLUSION

Discrepancies remain relatively high between registered and published outcomes, particularly regarding registered omissions in publications and concomitant reporting, nature of statistical method used, and reporting of AEs. This seriously undermines transparency of clinical trials and needs immediate attention of all stakeholders in health research.

CONSORT item reporting quality in the top ten ranked journals of critical care medicine in 2011: a retrospective analysis

INTRODUCTION

Reporting randomised controlled trials is a key element in order to disseminate research findings. The CONSORT statement was introduced to improve the reporting quality. We assessed the adherence to the CONSORT statement of randomised controlled trials published 2011 in the top ten ranked journals of critical care medicine (ISI Web of Knowledge 2011, Thomson Reuters, London UK).

METHODS

Design. We performed a retrospective cross sectional data analysis. Setting. This study was executed at the University Hospital of RWTH, Aachen. Participants. We selected the following top ten listed journals according to ISI Web of Knowledge (Thomson Reuters, London, UK) critical care medicine ranking in the year 2011: American Journal of Respiratory and Critical Care Medicine, Critical Care Medicine, Intensive Care Medicine, CHEST, Critical Care, Journal of Neurotrauma, Resuscitation, Pediatric Critical Care Medicine, Shock and Minerva Anestesiologica. Main outcome measures. We screened the online table of contents of each included journal, to identify the randomised controlled trials. The adherence to the items of the CONSORT Checklist in each trial was evaluated. Additionally we correlated the citation frequency of the articles and the impact factor of the respective journal with the amount of reported items per trial.

RESULTS

We analysed 119 randomised controlled trials and found, 15 years after the implementation of the CONSORT statement, that a median of 61,1% of the checklist-items were reported. Only 55.5% of the articles were identified as randomised trials in their titles. The citation frequency of the trials correlated significantly (rs = 0,433; p<0,001 and r = 0,331; p<0,001) to the CONSORT statement adherence. The impact factor showed also a significant correlation to the CONSORT adherence (r = 0,386; p<0,001).

CONCLUSION

The reporting quality of randomised controlled trials in the field of critical care medicine remains poor and needs considerable improvement.

What affects authors' and editors' use of reporting guidelines? Findings from an online survey and qualitative interviews

Objectives

To identify and understand, through data from multiple sources, some of the factors that affect authors’ and editors’ decisions to use reporting guidelines in the publication of health research.

Design

Mixed methods study comprising an online survey and semi-structured interviews with a sample of authors (online survey: n = 56; response rate = 32%; semi-structured interviews: n = 5) and journal editors (online survey: n = 43; response rate = 27%; semi-structured interviews: n = 6) involved in publishing health and medical research. Participants were recruited from an earlier study examining the effectiveness of the TREND reporting guideline.

Results

Four types of factors interacted to affect authors’ and editors’ likelihood of reporting guideline use; individual (e.g. having multiple reasons for use of reporting guidelines); the professional culture in which people work; environmental (e.g. policies of journals); and, practical (e.g. having time to use reporting guidelines). Having multiple reasons for using reporting guidelines was a particularly salient factor in facilitating reporting guidelines use for both groups of participants.

Conclusions

Improving the completeness and consistency of reporting of research studies is critical to the integrity and synthesis of health research. The use of reporting guidelines offers one potentially efficient and effective means for achieving this, but decisions to use (or not use) reporting guidelines take many factors into account. These findings could be used to inform future studies that might, for example, test the factors that we have identified within a wider theoretical framework for understanding changes in professional practices. The use of reporting guidelines by senior professionals appears to shape the expectations of what constitutes best practice and can be assimilated into the culture of a field or discipline. Without evidence of effectiveness of reporting guidelines, and sustained, multifaceted efforts to improve reporting, little progress seems likely to be made.

Information on new drugs at market entry: retrospective analysis of health technology assessment reports versus regulatory reports, journal publications, and registry reports

BACKGROUND

When a new drug becomes available, patients and doctors require information on its benefits and harms. In 2011, Germany introduced the early benefit assessment of new drugs through the act on the reform of the market for medicinal products (AMNOG). At market entry, the pharmaceutical company responsible must submit a standardised dossier containing all available evidence of the drug's added benefit over an appropriate comparator treatment. The added benefit is mainly determined using patient relevant outcomes. The "dossier assessment" is generally performed by the Institute for Quality and Efficiency in Health Care (IQWiG) and then published online. It contains all relevant study information, including data from unpublished clinical study reports contained in the dossiers. The dossier assessment refers to the patient population for which the new drug is approved according to the summary of product characteristics. This patient population may comprise either the total populations investigated in the studies submitted to regulatory authorities in the drug approval process, or the specific subpopulations defined in the summary of product characteristics ("approved subpopulations").

OBJECTIVE

To determine the information gain from AMNOG documents compared with non-AMNOG documents for methods and results of studies available at market entry of new drugs. AMNOG documents comprise dossier assessments done by IQWiG and publicly available modules of company dossiers; non-AMNOG documents comprise conventional, publicly available sources-that is, European public assessment reports, journal publications, and registry reports. The analysis focused on the approved patient populations.

DESIGN

Retrospective analysis.

DATA SOURCES

All dossier assessments conducted by IQWiG between 1 January 2011 and 28 February 2013 in which the dossiers contained suitable studies allowing for a full early benefit assessment. We also considered all European public assessment reports, journal publications, and registry reports referring to these studies and included in the dossiers.

DATA ANALYSIS

We assessed reporting quality for each study and each available document for eight methods and 11 results items (three baseline characteristics and eight patient relevant outcomes), and dichotomised them as "completely reported" or "incompletely reported (including items not reported at all)." For each document type we calculated the proportion of items with complete reporting for methods and results, for each item and overall, and compared the findings.Results 15 out of 27 dossiers were eligible for inclusion and contained 22 studies. The 15 dossier assessments contained 28 individual assessments of 15 total study populations and 13 approved subpopulations. European public assessment reports were available for all drugs. Journal publications were available for 14 out of 15 drugs and 21 out of 22 studies. A registry report in ClinicalTrials.gov was available for all drugs and studies; however, only 11 contained results. In the analysis of total study populations, the AMNOG documents reached the highest grade of completeness, with about 90% of methods and results items completely reported. In non-AMNOG documents, the rate was 75% for methods and 52% for results items; journal publications achieved the best rates, followed by European public assessment reports and registry reports. The analysis of approved subpopulations showed poorer complete reporting of results items, particularly in non-AMNOG documents (non-AMNOG versus AMNOG: 11% v 71% for overall results items and 5% v 70% for patient relevant outcomes). The main limitation of our analysis is the small sample size.

CONCLUSION

Conventional, publicly available sources provide insufficient information on new drugs, especially on patient relevant outcomes in approved subpopulations. This type of information is largely available in AMNOG documents, albeit only partly in English. The AMNOG approach could be used internationally to develop a comprehensive publication model for clinical studies and thus represents a key open access measure.

Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a study protocol

BACKGROUND

The Cochrane risk of bias (RoB) tool has been widely embraced by the systematic review community, but several studies have reported that its reliability is low. We aim to investigate whether training of raters, including objective and standardized instructions on how to assess risk of bias, can improve the reliability of this tool. We describe the methods that will be used in this investigation and present an intensive standardized training package for risk of bias assessment that could be used by contributors to the Cochrane Collaboration and other reviewers.

METHODS/DESIGN

This is a pilot study. We will first perform a systematic literature review to identify randomized clinical trials (RCTs) that will be used for risk of bias assessment. Using the identified RCTs, we will then do a randomized experiment, where raters will be allocated to two different training schemes: minimal training and intensive standardized training. We will calculate the chance-corrected weighted Kappa with 95% confidence intervals to quantify within- and between-group Kappa agreement for each of the domains of the risk of bias tool. To calculate between-group Kappa agreement, we will use risk of bias assessments from pairs of raters after resolution of disagreements. Between-group Kappa agreement will quantify the agreement between the risk of bias assessment of raters in the training groups and the risk of bias assessment of experienced raters. To compare agreement of raters under different training conditions, we will calculate differences between Kappa values with 95% confidence intervals.

DISCUSSION

This study will investigate whether the reliability of the risk of bias tool can be improved by training raters using standardized instructions for risk of bias assessment. One group of inexperienced raters will receive intensive training on risk of bias assessment and the other will receive minimal training. By including a control group with minimal training, we will attempt to mimic what many review authors commonly have to do, that is-conduct risk of bias assessment in RCTs without much formal training or standardized instructions. If our results indicate that an intense standardized training does improve the reliability of the RoB tool, our study is likely to help improve the quality of risk of bias assessments, which is a central component of evidence synthesis.