A 30 year perspective of the quality of evidence published in 25 clinical journals: signs of change?

Clinical research quo vadis? Trends in reporting of clinical trials and observational study designs over two decades

Background

Multiple classifications have been developed that classify the medical literature into different levels of evidence to facilitate the evaluation of study results and practice of evidence-based medicine. The suggested hierarchies of evidence are generally based on the type of study design; randomized, controlled clinical trials constitute the top level of evidence while case reports rank the lowest among epidemiologic study designs. However, little is known about the frequency with which different study designs appear in the medical literature overall. The purpose of this study was to describe trends in the frequency of reports of randomized control trials (RCTs) as compared to other study designs in the medical literature over two decades.

Methods

Data about the prevalence of various types of study designs in the medical literature over the last two decades (years 1990 - 2009) were abstracted from PubMed, validated and subjected to cross-sectional and longitudinal analysis.

Results

In the last 20 years, the annual rate of publication of journal articles has more than doubled. During this period, the percentage of observational studies increased from 29.9% to 40.5%, the percentage of reports of RCTs increased minimally, and there was a striking decline in the percentage of case reports (from 49.8% to 33.6%) in the medical literature overall. In contrast, in three selected, highly cited medical journals, the percentage of reports of RCTs increased by almost 10%. Surprisingly, the percentage of articles classified as case reports also increased (from 36.3% to 43.8%) in these three journals, while the percentage of reports of cohort and case-control studies decreased.

Conclusion

Though the relative frequency of reports from RCTs has not changed substantially in the last 20 years, cohort studies and case-control studies have largely supplanted simple case reports. In contrast, in high impact journals, the representation of RCTs and case reports has increased, with corresponding declines in reports based on other study designs. Further research will be needed to determine whether those trends in publication have resulted in more robust evidence and faster advancement of medical knowledge.

Improving levels of evidence in studies published in spinal journals from 1983 to 2011

INTRODUCTION

The evidence basis for spinal surgical practice is closely related to the quality of clinical studies published. In the hierarchy of study designs, the quality of evidence from a randomised controlled trial (RCT) and meta-analysis are considered better than other types of prospective and retrospective study designs. We assessed the level of evidence in studies published from 1983 to 2011 in 4 journals, publishing predominantly on spine related topics.

METHODS

Computerised search of the Medline database was undertaken to evaluate the articles published in 4 'spinal' journals (Spine, European Spine Journal, Journal of Spinal Disorders and Techniques and Spinal Cord) from 1983 to 2011. The numbers of RCTs, meta-analyses and other clinical trials (non-RCT) over the time periods 1983-7, 1988-92, 1993-7, 1998-2002, 2003-7 and 2008-11 were compared.

RESULTS

From a total number of 21775 articles evaluated, there were 888 (4.1%) RCTs, 93 (0.4%) meta-analyses and 1355 (6.2%) other clinical trials. Overall, comparing the time periods 1983-7 with 2008-11, there were increases in the proportion of RCTs (1.6% to 5.1%), meta-analysis (0% to 0.6%) and other clinical trials (0.9% to 6.3%) and a decrease in the proportion of other articles published (97.6% to 88.1%). These changes appear to have plateaued after 2003-7.

CONCLUSIONS

Although RCTs, meta-analysis and other clinical trials form a small proportion of the studies published in leading 'spinal' journals, there have been an encouraging increase in their proportion over the years. This is comparable with other larger surgical specialities and provides a better evidence basis for clinical practice in spinal disorders.

Level of evidence in four selected rehabilitation journals

OBJECTIVE

To investigate the methodologic quality and level of evidence of publications in major peer-reviewed general rehabilitation journals (Archives of Physical Medicine and Rehabilitation [APMR], American Journal of Physical Medicine and Rehabilitation [AJPMR], Clinical Rehabilitation [CR], and Physical Therapy [PT]).

DESIGN

Descriptive, comparative.

MAIN OUTCOME MEASURES

All the articles published in AJPMR, APMR, CR, and PT between January 2005 and December 2009 were investigated. Type of study and level of evidence were recorded for all articles. Selection and assessment of articles were based on the title and abstract by 2 independent raters.

RESULTS

The most frequently published reports were randomized controlled trials (12.7%), followed by cross-sectional studies (12.1%), case reports/case series (10.3%), validation studies (9.3%), cohort studies (8.9%), clinical trials (7.5%), case-control studies (6.8%), and other study types (32.4%). When the articles were classified according to their level of evidence, level I studies most frequently appeared in CR (29.1%), followed by PT (11.0%), APMR (10.5%), and AJPMR (7.1%). Most of the meta-analyses (10) were in APMR, and there were none in AJPMR.

CONCLUSIONS

Randomized controlled trials and meta-analyses form only a small proportion of articles published in the current rehabilitation literature. The numbers of randomized controlled trials and meta-analysis are comparable with those in other fields.

Pharmacy resident project publication rates and study designs from 1981, 1991, and 2001

PURPOSE

The publication rates and characteristics of residency projects presented at the Southeastern Residency Conference (SERC) in 1981, 1991, and 2001 are described.

METHODS

SERC abstracts from 1981, 1991, and 2001 were searched in MEDLINE and International Pharmaceutical Abstracts with a time limit of six years after the conference. All potential positive publications were reviewed and validated by a second investigator. Publication rates and other secondary variables, including study design, authors' degrees, reporting of results, time to publication, and journal of publication, were compared, with differences noted.

RESULTS

A total of 272 abstracts were evaluated. Publication in a journal was achieved for 18 (20%) of 90 projects presented in 1981, 11 (16%) of 70 projects presented in 1991, and 14 (13%) of 112 projects presented in 2001. The median time to publication for all abstracts was 22.8 months. Publications rates decreased at each subsequent time period. Projects published in journals with high impact factors had a longer median time to publication compared with other projects. Abstracts that reported results were published more often than those that did not (43% versus 23%), as were those that included a physician coauthor compared with those that did not (32% versus 19%). Observational studies were the most common study designs and were most published.

CONCLUSION

Approximately 16% of pharmacy residents' research projects presented at SERC in 1981, 1991, and 2001 were published. Projects with observational study designs were the most common study type and were the most commonly published.

Over 50 years of trial in Acta Psychiatrica Scandinavica: a survey

Background

Randomised controlled trials are the gold standard for evaluating mental health care interventions. We assessed the content and quality of trials published in Acta Psychiatrica Scandinavica and Supplementum since 1948.

Methods

All trials were identified manually, quality assessed, data extracted, and sought on Medline.

Results

About 8.6% of all reports in the journal were clinical trials (n = 582) with the peak frequency in the 1980s. Most originate from Europe (80%) and focus on depression (~38%) or schizophrenia (27%). The median sample size is 44. We found only two trials that fully met the criteria of quality reporting RCTs set by CONSORT statements (0.34%) since 1996. Less than 50% of records were possible to identify by a Medline search using broad methodological terms.

Conclusion

Acta is a major source of health trials. The standard of reporting is similar to other journals but better adherence to CONSORT would ensure higher quality of reports and better dissemination.

Quality of evidence published in clinical journals: signs of change?

This commentary provides a consensus view by two reviewers on a paper appearing in this issue that investigates the change in the quality of evidence published in clinical journals over a period of 25 years.