Methodological reporting of randomized clinical trials in major gastroenterology and hepatology journals in 2006

Systematic review of sample size calculations and reporting in randomized controlled trials in ophthalmology over a 20-year period

PURPOSE

Randomized Controlled Trials (RCTs) are considered the gold standard for the practice of evidence-based medicine. The purpose of this study is to systematically assess the reporting of sample size calculations in ophthalmology RCTs in 5 leading journals over a 20-year period. Reviewing sample size calculations in ophthalmology RCTs will shed light on the methodological quality of RCTs and, by extension, on the validity of published results.

METHODS

The MEDLINE database was searched to identify full reports of RCTs in the journals Ophthalmology, JAMA Ophthalmology, American Journal of Ophthalmology, Investigative Ophthalmology and Visual Science, and British Journal of Ophthalmology between January and December of the years 2000, 2010 and 2020. Screening identified 559 articles out of which 289 met the inclusion criteria for this systematic review. Data regarding sample size calculation reporting and trial characteristics was extracted for each trial by independent investigators.

RESULTS

In 2020, 77.9% of the RCTs reported sample size calculations as compared with 37% in 2000 (p < 0.001) and 60.7% in 2010 (p = 0.012). Studies reporting all necessary parameters for sample size recalculation increased significantly from 17.2% in 2000 to 39.3% in 2010 and 43.0% in 2020 (p < 0.001). Reporting of funding was greater in 2020 (98.8%) compared with 2010 (89.3%) and 2000 (53.1%). Registration in a clinical trials database occurred more frequently in 2020 (94.2%) compared to 2000 (1.2%; p < 0.001) and 2010 (68%; p < 0.001). In 2020, 38.4% of studies reported different sample sizes in the online registry from the published article. Overall, the most studied area in 2000 was glaucoma (29.6% of RCTs), whereas in 2010 and 2020, it was retina (40.2 and 37.2% of the RCTs, respectively). The number of patients enrolled in a study and the number of eyes studied was significantly greater in 2020 compared to 2000 and 2010 (p < 0.001).

CONCLUSION

Sample size calculation reporting in ophthalmology RCTs has improved significantly between the years 2000 and 2020 and is comparable to other fields in medicine. However, reporting of certain parameters remains inconsistent with current publication guidelines.

Lowering the threshold of statistical significance in gastroenterology trials

A multidisciplinary international cohort of 72 expert statisticians and researchers recently proposed lowering the p value threshold from 0.05 to 0.005 to mitigate distortion of trial results and decrease bias. We hereby explored how a change to the p value threshold may alter the statistical significance of primary endpoints in gastroenterology (GE) randomized control trials (RCTs). We analyzed RCTs published in the 20 highest ranked GE and medicine journals. For each trial, we extracted the p values for the corresponding primary endpoints. We retrieved 233 RCTs, of which 159 were included in the final analysis yielding 202 primary endpoints. Of these endpoints, 60% had a p value less than 0.05 and when a threshold of less than 0.005 was applied, approximately 50% retained significance. We endorse a lower p value threshold as an actionable, provisional measure for improving statistical inference in GE RCTs until more long-term solutions become available.

Systematic characterization of gastrointestinal clinical trials

BACKGROUND

Clinical guidelines are commonly based on inadequate evidence, suggesting deficiencies in the present portfolio of clinical research.

AIMS

To investigate characteristics of clinical trials examining gastrointestinal (GI) diseases registered in ClinicalTrials.gov.

METHODS

A cross-sectional analysis of 13,647 GI trials and 111,535 non-GI trials initiated between January 1997 and September 2013 was performed. Entries were sorted by operational status, purpose, interventions, trial design, and epochs to identify trends and interactions in trial properties.

RESULTS

The global production of GI trials has remained static in recent years and a majority of research efforts are focused on a few diseases. While GI trials are generally produced by highly populated US states and countries, they are also seldom larger than 500 patients. The likelihood of using data monitoring committees, randomization, and double blinding in GI trials has increased over time, though a substantial fraction of GI trials still do not employ rigorous trial designs. While levels of GI trials correlate with disease burden, the explained variance of GI trials by disease burden worldwide is poor.

CONCLUSION

GI trials are chiefly concentrated in few diseases and highly populated regions, exhibit heterogeneous trends and methodologies, and are sensitive to disease burdens, though more so within North America than worldwide.

Effect of Perioperative Probiotics and Synbiotics on Postoperative Infections After Gastrointestinal Surgery: A Systematic Review With Meta-Analysis

BACKGROUND

Bacterial infection following gastrointestinal surgery remains a common morbidity. The aim of this study was to estimate the effect of the perioperative use of probiotics and synbiotics on postoperative infections.

MATERIALS AND METHODS

We searched PubMed, Embase, and the Cochrane Library to identify pertinent randomized controlled trials (RCTs). The primary outcome was postoperative infection rate. The secondary outcomes were length of hospital and intensive care unit (ICU) stay, length of antibiotic therapy, and mortality. The pooled outcomes were calculated using random effects models.

RESULTS

Twenty-eight RCTs involving 2511 patients were included in this systematic review. The incidence of infectious complications was lower among patients who received probiotics/synbiotics than among the controls (odds ratio [OR] = 0.35; 95% confidence interval [CI], 0.24-0.50), particularly regarding respiratory (OR = 0.44; 95% CI, 0.28-0.68), urinary tract (OR = 0.30; 95% CI, 0.16-0.55), and wound infections (OR = 0.58; 95% CI, 0.42-0.80). The lengths of hospital stay (mean difference [MD] = -3.20; 95% CI, -4.87 to -1.54) and duration of antibiotic therapy (MD = -3.40; 95% CI, -4.67 to -2.13) were shorter for patients who received probiotics/synbiotics than for controls. There were no significant differences in mortality (OR = 1.19; 95% CI, 0.52-2.74) or length of ICU stay (MD = -0.46; 95% CI, -1.07 to 0.14) between the compared groups.

CONCLUSION

Probiotics and synbiotics may prevent postoperative infections in patients undergoing gastrointestinal surgery. However, the results need to be interpreted with caution due to the risk of bias in the included studies and the potential publication bias.

Methodological Reporting Quality of Randomized Controlled Trials in 3 Leading Diabetes Journals From 2011 to 2013 Following CONSORT Statement: A System Review

To appraise the current reporting methodological quality of randomized clinical trials (RCTs) in 3 leading diabetes journals.We systematically searched the literature for RCTs in Diabetes Care, Diabetes and Diabetologia from 2011 to 2013.Characteristics were extracted based on Consolidated Standards of Reporting Trials (CONSORT) statement. Generation of allocation, concealment of allocation, intention-to-treat (ITT) analysis and handling of dropouts were defined as primary outcome and "low risk of bias." Sample size calculation, type of intervention, country, number of patients, funding source were also revealed and descriptively reported. Trials were compared among journals, study years, and other characters.A total of 305 RCTs were enrolled in this study. One hundred eight (35.4%) trials reported adequate generation of allocation, 87 (28.5%) trials reported adequate concealment of allocation, 53 (23.8%) trials used ITT analysis, and 130 (58.3%) trials were adequate in handling of dropouts. Only 15 (4.9%) were "low risk of bias" trials. Studies at a large scale (n > 100) or from European presented with more "low risk of bias" trials than those at a small scale (n ≤ 100) or from other regions. No improvements were found in these 3 years.This study shows that methodological reporting quality of RCTs in the major diabetes journals remains suboptimal. It can be further improved to meet and keep up with the standards of the CONSORT statement.

Methodological reporting quality of randomized controlled trials in three spine journals from 2010 to 2012

PURPOSE

To elucidate the methodological reporting quality of randomized controlled trials (RCTs) in three spine journals from 2010 to 2012.

METHODS

In this study, we summarized the methodological report of RCTs in three major spine journals, including the Spine Journal, Spine and the European Spine Journal from 2010 to 2012. The methodological reporting quality, including the allocation sequence generation, allocation concealment, blinding and sample size calculation, was revealed. Number of patients, funding source, type of intervention and country were also retrieved from each trial. The methodological reporting quality was descriptively reported.

RESULTS

Ninety trials were involved and 57.8% (52/90) reported adequate allocation sequence generation, 46.7% (42/90) reported adequate allocation concealment, 34.4% (31/90) reported adequate blinding and 37.8% (34/90) reported adequate sample size calculation.

CONCLUSIONS

This study shows that the methodological reporting quality of RCTs in the spine field needs further improvement.

Quality of reporting of randomized controlled trials published in Intensive Care Medicine from 2001 to 2010

PURPOSE

To evaluate the methodological quality of randomized controlled trials (RCTs) published in Intensive Care Medicine from 2001 to 2010, and to compare it with a previous review of RCTs published from 1975 to 2000.

METHODS

We assessed the quality of reporting of randomization, blinding and participant flow, both individually and combined within the Jadad scale, and compared them with findings from our previous review. For RCTs published from 2001 to 2010, we also evaluated the frequency of distorted finding presentation (spin) and inflated predicted treatment effect (delta inflation).

RESULTS

In the 221 RCTs from 2001 to 2010, the sample size was significantly larger than in the older series, and there was a higher proportion of studies with negative findings. Reporting of the rationale for sample size estimation and allocation concealment increased significantly, but reporting of other important individual methodological components did not change substantially compared with the previous period and remained low. Among RCTs from 2001 to 2010, a spin strategy was used in 69 of 111 RCTs with statistically negative results, while delta inflation was present in 7 of 11 RCTs evaluating survival as a primary outcome. Papers with higher Jadad scores were cited more often than the others.

CONCLUSIONS

Quality of reporting of RCTs published in Intensive Care Medicine has only partly improved over time, and spin and delta bias are of frequent occurrence. There is a need for stronger adherence to CONSORT recommendations, with special emphasis on accurate description of randomization and blindness, and correct reporting of statistically non-significant results.

[Quality of reporting in studies on bipolar disorders: implications for the development of guidelines]

BACKGROUND

Selective publishing as well as inadequate reporting of clinical trials entail a risk of bias in clinical decision making. Therefore the CONSORT statement was introduced to improve the quality of reporting of randomized controlled trials (RCT). This study aimed to assess the quality of reporting of RCTs on pharmacological treatment of bipolar disorder in relation to publication period and endorsement of publication guidelines.

METHODS

In the context of the development of the German evidence and consensus-based S3 guidelines for diagnosis and therapy of bipolar disorders a systematic literature search was carried out to identify all RCTs published between 2000 and 2010 relevant to the pharmacological treatment of bipolar disorders. An adapted checklist based on the CONSORT statement was used to assess the quality of reporting.

RESULTS

A total of 134 RCTs were included in this analysis. Of the 72 checklist items, 43% were generally reported adequately (reported in  ≥ 75% of all trials) and 25% inadequately (reported in  < 25% of all trials). Reporting was generally poor for randomization, effect size (reported in 22%) and number needed to treat (NNT 16%). No consistent trend could be shown for improvement in quality of reporting over time or for journals that do or do not endorse the URM (uniform requirements for manuscripts submitted to biomedical journals).

CONCLUSIONS

Clinical investigators as well as editors and reviewers should be further encouraged to follow publication guidelines otherwise trials have to be downgraded or excluded from systematic evaluations.

Methodological reporting of randomized controlled trials in major hepato-gastroenterology journals in 2008 and 1998: a comparative study

Background

It was still unclear whether the methodological reporting quality of randomized controlled trials (RCTs) in major hepato-gastroenterology journals improved after the Consolidated Standards of Reporting Trials (CONSORT) Statement was revised in 2001.

Methods

RCTs in five major hepato-gastroenterology journals published in 1998 or 2008 were retrieved from MEDLINE using a high sensitivity search method and their reporting quality of methodological details were evaluated based on the CONSORT Statement and Cochrane Handbook for Systematic Reviews of interventions. Changes of the methodological reporting quality between 2008 and 1998 were calculated by risk ratios with 95% confidence intervals.

Results

A total of 107 RCTs published in 2008 and 99 RCTs published in 1998 were found. Compared to those in 1998, the proportion of RCTs that reported sequence generation (RR, 5.70; 95%CI 3.11-10.42), allocation concealment (RR, 4.08; 95%CI 2.25-7.39), sample size calculation (RR, 3.83; 95%CI 2.10-6.98), incomplete outecome data addressed (RR, 1.81; 95%CI, 1.03-3.17), intention-to-treat analyses (RR, 3.04; 95%CI 1.72-5.39) increased in 2008. Blinding and intent-to-treat analysis were reported better in multi-center trials than in single-center trials. The reporting of allocation concealment and blinding were better in industry-sponsored trials than in public-funded trials. Compared with historical studies, the methodological reporting quality improved with time.

Conclusion

Although the reporting of several important methodological aspects improved in 2008 compared with those published in 1998, which may indicate the researchers had increased awareness of and compliance with the revised CONSORT statement, some items were still reported badly. There is much room for future improvement.

Reporting characteristics of cancer pain: a systematic review and quantitative analysis of research publications in palliative care journals

Objective:

A common disorder requiring symptom palliation in palliative and end-of-life care is cancer. Cancer pain is recognized as a global health burden. This paper sought to systematically examine the extent to which there is an adequate scientific research base on cancer pain and its reporting characteristics in the palliative care journal literature.

Materials and Methods:

Search conducted in MEDLINE and CINAHL sought to locate all studies published in 19 palliative/ hospice/ supportive/ end-of-life care journals from 2009 to 2010. The journals included were: American Journal of Hospice and Palliative Care, BMC Palliative Care, Current Opinion in Supportive and Palliative Care, End of Life Care Journal, European Journal of Palliative Care, Hospice Management Advisor, Indian Journal of Palliative Care, International Journal of Palliative Nursing, Internet Journal of Pain Symptom Control and Palliative Care, Journal of Pain and Palliative Care Pharmacotherapy, Journal of Palliative Care, Journal of Palliative Medicine, Journal of Social Work in End-of-life and Palliative Care, Journal of Supportive Oncology, Palliative Medicine, Palliative and Supportive Care, and Supportive Care in Cancer. Journal contents were searched to identify studies that included cancer pain in abstract.

Results:

During the years 2009 and 2010, of the selected 1,569 articles published in the journals reviewed, only 5.86% (92 articles) were on cancer pain.

Conclusion:

While researchers in the field of palliative care have studied cancer pain, the total percentage for studies is still a low 5.86%. To move the field of palliative care forward so that appropriate guidelines for cancer pain management can be developed, it is critical that more research be reported upon which to base cancer pain therapy in an evidence-based palliative care model.

Meta-analysis: erythromycin before endoscopy for acute upper gastrointestinal bleeding

BACKGROUND

Studies evaluating the effect of erythromycin on patients with acute upper gastrointestinal bleeding (UGIB) had been reported, but the results were inconclusive.

AIMS

To compare erythromycin with control in patients with acute UGIB by performing a meta-analysis.

METHODS

Electronic databases including PubMed, EMBASE and the Cochrane Library, Science Citation Index, were searched to find relevant randomised controlled trials (RCTs). Two reviewers independently identified relevant trials evaluating the effect of erythromycin on patients with acute UGIB. Outcome measures were the incidence of empty stomach, need for second endoscopy, blood transfusion, length of hospital stay, endoscopic procedure time and mortality.

RESULTS

Four RCTs including 335 patients were identified. Meta-analysis demonstrated the incidence of empty stomach was significantly increased in patients receiving erythromycin (active group 69%, control group 37%, P<0.00001). The need for second endoscopy, amount of blood transfusion and the length of hospital stay were also significantly reduced (all P<0.05). A trend for shorter endoscopic procedure time and decreased mortality rate was observed.

CONCLUSIONS

Prophylactic erythromycin is useful for patients with upper gastrointestinal bleeding to decrease the amount of blood in the stomach and reduce the need for second endoscopy, amount of blood transfusion. It may shorten the length of hospital stay, but its effects on mortality need further larger trials to be confirmed.

Hepato-biliary clinical trials and their inclusion in the Cochrane Hepato-Biliary Group register and reviews

BACKGROUND AND AIMS

The Cochrane Hepato-Biliary Group (CHBG) is one of the 52 collaborative review groups within The Cochrane Collaboration. The activities of the CHBG focus on collecting hepato-biliary randomized clinical trials (RCT) and controlled clinical trials (CCT), and including them in systematic reviews with meta-analyses of the trials. In this overview, we present the growth of The CHBG Controlled Trials Register, as well as the systematic reviews that have been produced since March 1996.

RESULTS

The CHBG register includes almost 11,000 RCT and 700 CCT publications. The earliest RCT in the register were published in 1955, and the earliest CCT in 1945. From 1945 to 1980, there were less than 100 publications each year. From 1981 to 1997, their number increased from over 100 to 600 a year. After 1997, the number of publications seems to have been decreasing. The CHBG has published 199 protocols for systematic reviews and 107 systematic reviews through to August 2009 in which 21% of the RCT and CCT were included. The CHBG reviews have been cited approximately 1200 times.

CONCLUSIONS

A large amount of work has been carried out since 1996. However, there is still much to do, as the CHBG register contains a great number of RCT and CCT on topics that have not yet been systematically reviewed.

Internal validity of randomized controlled trials reported in major gastrointestinal and surgical endoscopy journals in 2008

BACKGROUND

Assessment of the reported methodology for randomized controlled trials (RCTs) in the major gastrointestinal and surgical endoscopy journals has never been reported.

METHODS

Generation of the allocation sequence, allocation concealment, double blinding, sample size calculation, number of patients, and funding source included in methodologic quality were retrieved from each trial, and all the relevant trials were identified by a search of all clinical trials published in 2008 in four major gastrointestinal and surgical endoscopy journals.

RESULTS

A total of 64 trials were included in the final analysis, which found that 50% (32/64) of all trials reported adequate generation of the allocation sequence, 58% (37/64) reported adequate allocation concealment, 47% (30/64) reported adequate blinding, 47% (30/64) reported adequate sample size calculation, and 67% (43/64) failed to disclose the funding source.

CONCLUSIONS

The study showed that the quality of the reported methodology in the major gastrointestinal and surgical endoscopy journals needs great improvement.