Accuracy of references in the orthopaedic literature

Accuracy of citation and quotation in foot and ankle surgery journals

BACKGROUND

A recent review of technical editing of research suggests that over one third of references cited in articles in medical journals have some inaccuracies and one fifth of quotations to references in these articles are not accurate.

METHODS

Two hundred and forty-nine citation references and 408 quotes from 25 articles published in 5 orthopaedic journals were randomly selected to determine referencing accuracy. The presence of citation errors was examined by 1 of the authors while the presence of quotation errors was determined by 2 of the authors. Full copies of articles as well as the references were obtained to compare the accuracies.

RESULTS

The total citation error rate was 41% (103 out of 249 references), and the total quotation error rate was 20% (80 out of 408 quotes) for the 5 orthopaedic journals.

CONCLUSION

Citation and quotation errors were still relatively common in orthopaedic journals. While we did not identify any factors associated with citation and quotation errors, the use of technical editing may reduce the amount of citation errors.

CLINICAL RELEVANCE

Readers and authors should be aware that many citations of studies are inaccurate and one should review the original source if it is to be used in another publication or to guide clinical treatment.

Reference accuracy in peer-reviewed pediatric orthopaedic literature

BACKGROUND

Reference accuracy of articles published in the biomedical literature is determined by the presence of citation and quotation errors. A recent review demonstrated that the median citation error rate per biomedical journal was 39%, and the median quotation error rate per journal was 20%. Reference accuracy in pediatric orthopaedic articles has not been previously reported, to our knowledge.

METHODS

Two hundred references from twenty articles published in four peer-reviewed orthopaedic journals were randomly selected for assessment of citation and quotation accuracy. Full-text copies of all original references were obtained by interlibrary loan methods and reviewed directly to establish citation accuracy. The presence of citation errors was determined by a single investigator. The relevance of citation errors was determined by assessing the ease of reference retrieval through PubMed. Quotation accuracy was determined by two examiners who reviewed each of the twenty articles and 200 references to compare the claims made for the references in the article against the data and opinions expressed in the actual reference.

RESULTS

The total citation error rate across all of the journals was 26% (fifty-one of 200 references) with a 95% confidence interval of 16.5% to 37.3%. The median citation error rate per journal was 27% (range, 10% to 38%). Although citation errors were common, most were of minimal significance, as 196 of the 200 references could be retrieved with ease from PubMed. The total quotation error rate across all of the articles was 38% (152 of 398 reference citations) with a 95% confidence interval of 30.1% to 47.0%. The median quotation error rate per journal was 38% (range, 28% to 46%).

CONCLUSIONS

Citation and quotation errors are common in the pediatric orthopaedic literature. Reference accuracy continues to be a substantial problem in the biomedical literature despite recent technological advances such as online databases, easily accessible search engines, and widely available bibliographic software.

Reference accuracy in four rehabilitation journals

OBJECTIVE

To investigate the incidence of reference errors in major peer-reviewed general physical therapy and rehabilitation journals (American Journal of Physical Medicine and Rehabilitation (AJPMR), Archives of Physical Medicine and Rehabilitation (APMR), Clinical Rehabilitation (CR) and Physical Therapy (PT)).

DESIGN

Descriptive, comparative.

MAIN OUTCOME MEASURES

All issues of the AJPMR, APMR, CR and PT between 2003 and 2007 were studied. For each journal, references from articles were consecutively numbered, and using a random number generator, 100 references were selected from each journal. For each reference, ease of retrieval on MEDLINE and the presence of citation errors were noted. If discrepancies were identified, the reference was compared with the original publication. Two observers independently evaluated each reference for citation errors.

RESULTS

The total number of citations with errors among all published journals was 123 (30.7%). The reference error rates by journal ranged from 23% to 44%. Most errors (48.0%) occurred in the author element, followed by the title (31.7%), journal (8.9%), page (5.7%), year (4.1%), and volume (1.6%). Only 8 (2%) were likely to make retrieval of the reference difficult.

CONCLUSIONS

Errors in references still appear in current physical therapy and rehabilitation literature, but most are not severe.

Accuracy of references in burns journals

AIMS

To study the incidence and risk factors for citation and quotation errors in two major burns surgery journals.

METHODS

120 references were randomly selected from original articles published in the following two journals - January to December 2006 issues of Burns and Journal of Burn Care & Research. For each reference, the ease of retrieval on PubMed and the presence of citation errors were noted. Two independent observers analysed each reference for quotation errors. The characteristics of the root article, that is, type of study, author numbers, number of references and article word count were noted.

RESULTS

Of the 120 selected references, 117 referred to articles from indexed medical journals published in English. Among these, 4 articles could not be retrieved due to fatal citation errors (3.3%). A further 12 citation errors were noted giving a total citation error rate of 13.3% (95% CI: 6.74-19.93%). Of the 117 references analysed, the quotation error rate was 13.7% (95% CI: 8.6-19.5%) half of which were major errors. There was no significant association between the combined error rate per article and the journal (Kruskal-Wallis test; p=0.861, type of study (Kruskal-Wallis test; p=0.717), author numbers (Spearman's rho=0.197, p=0.423), article length (Spearman's rho=0.118, p=0.705) or references per article (Spearman's rho=0.229, p=0.189).

CONCLUSION

Significant numbers of citation and quotation errors still appear in current burns literature. Incorrect spelling of author names and partial omissions of article titles were the two most common errors. No observable underlying factors were identified in this study. The present results serve as a reminder to authors, editors and peer reviewers for more care of citation accuracy when striving for their common goal of scientific excellence.

Accuracy of references in general surgical journals--an old problem revisited

BACKGROUND

Reference errors in biomedicals journals are well documented. Increasing use of electronic databases and bibliographic software may change the nature and frequency of errors.

AIM

To study the current incidence of reference errors in four major general surgical journals.

METHODS

Seventy-five references were randomly selected from original articles published in one issue of each of four general surgical journals. For each reference, ease of retrieval on PubMed and the presence of citation errors were noted. Two observers independently reviewed each reference for quotation errors.

RESULTS

Of the 300 selected references, 261 from indexed English language biomedical journals were analysed. Retrieval from PubMed was impossible or difficult in six instances, giving a major citation error rate of 2.3%. Overall (major and minor) citation error rate was 11.1%. Of the 258 references that could be retrieved, 20 (7.8%) had quotation errors, 80% of which were considered major. The overall citation error rate was significantly different across the four journals. There was moderate correlation between quotation error rate and number of references in each original article.

CONCLUSION

Errors in references still appear in current surgical literature. Solutions to address this problem have been discussed.

How accurate are the references in Emergency Medical Journal?

OBJECTIVE

To determine the accuracy of references in Emergency Medicine Journal during 2003.

MATERIALS AND METHODS

All references cited in Emergency Medicine Journal during 2003 were examined carefully, and the accuracy of the citations was checked against reliable electronic and manual resources. References were categorised as correct or incorrect. The errors were classified as minor if the integrity of the reference was not greatly compromised and major if the error severely detracted from the quality of the reference.

RESULTS

Overall, errors were found in 19% of all citations checked (n = 2561), and in 8% the errors were major and markedly detracted from the quality of the reference.

CONCLUSIONS

Citation errors reflect badly on authors and the publishing journal and may reflect underlying flaws in other areas of the research published. It is hoped that identification of this problem will lead to attempts to improve the accuracy of reference citation in the emergency medicine literature and to an improvement in the credibility of research in our specialty.

How evidence-based are advertisements in journals regarding the subspecialty of rheumatology?

OBJECTIVES

In rheumatology, five different billion-dollar drugs have emerged in recent years, making this subspecialty the focus of extensive advertising campaigns. Considering this development and the fact that the scientific content of advertisements has been questioned, we initiated a study to determine how evidence-based advertisements are in four leading journals of rheumatology.

METHODS

Advertisements were extracted from the journals Arthritis and Rheumatism, Rheumatology, Seminars in Arthritis and Rheumatism and the Journal of Rheumatology, and were evaluated using a standardized score form.

RESULTS

In total, 353 advertisements were obtained, of which 84 were unique. Of the 300 references provided by these unique advertisements, 53 (18%) were considered 'supporting'. In addition, 87 (29%) of the 300 references referred to randomized controlled trials (RCTs), of which 49% supported the claim. The vast majority of RCTs (97%) were sponsored by the advertising company. In the 84 unique advertisements 288 claims were made, of which 190 provided one or more references. Of these 190 claims, 33 (17%) were judged 'well supported'. In total, only four (5%) of the 84 different advertisements were judged 'well supported' and 11 (13%) of the 84 were judged 'misleading' because they contained one or more misleading claims, i.e. claims with references to articles contradicting the claim.

CONCLUSIONS

This study indicates that few advertisements in rheumatology journals are entirely evidence-based.

Accuracy of references in five biomedical informatics journals

OBJECTIVE

To determine the rate and type of errors in biomedical informatics journal article references.

METHODS

References in articles from the first 2004 issues of five biomedical informatics journals, Journal of the American Medical Informatics Association, Journal of Biomedical Informatics, International Journal of Medical Informatics, Methods of Information in Medicine, and Artificial Intelligence in Medicine were compared with MEDLINE for journal, authors, title, year, volume, and page number accuracy. If discrepancies were identified, the reference was compared with the original publication. Two reviewers independently evaluated each reference.

RESULTS

The five journal issues contained 37 articles. Among the 656 eligible references, 225 (34.3%) included at least one error. Among the 225 references, 311 errors were identified. One or more errors were found in the bibliography of 31 (84%) of the 37 articles. The reference error rates by journal ranged from 22.1% to 40.7%. Most errors (39.0%) occurred in the author element, followed by the journal (31.2%), title (17.7%), page (7.4%), year (3.5%), and volume (1.3%) information.

CONCLUSION

The study identified a considerable error rate in the references of five biomedical informatics journals. Authors are responsible for the accuracy of references and should more carefully check them, possibly using informatics-based assistance.

Reference accuracy in the Journal of Hand Surgery

PURPOSE

An accurate listing of cited references in journal articles is important for the reader. Many studies have revealed errors in the references cited in a variety of subspecialty journals. We believe that a similar amount of errors, relative to other subspecialty journals, exists in the Journal of Hand Surgery.

METHODS

One hundred randomly selected references from each of the 1985 and 1995 Journal of Hand Surgery volumes were analyzed for accuracy of the journal name, title, author name(s), year, volume, and page number(s). References were considered without error if they matched the original article exactly.

RESULTS

In 1985, 30% of the references examined contained one or more errors as compared with 11% in 1995.

CONCLUSIONS

These error rates are similar to those found in studies of other biomedical journals. Perhaps the decrease in the quantity of errors present in 1995 versus that of 1985 is related to the implementation of a more stringent, revised set of instructions to submitting authors imposed in 1995.