Accuracy of referencing in the ophthalmic literature

Quotation Accuracy of Systematic Review and Meta-Analysis Protocols on Acupuncture

Currently, published systematic review protocols (SR protocols) have increasingly become a new trend in fields such as acupuncture and are therefore a new source of quotations in these fields. Systematic reviews are considered the pinnacle of the evidence pyramid as they embody comprehensive literature searching. Quotations are key elements to achieve this goal as they can support the assertions of the original authors, but the ‘misquotation’ exists, too, and they can be misleading to the reader. The aim of this study was to examine the quotation accuracy of SR protocols in a meta-analysis on acupuncture research. We searched SCOPUS through 31 December, 2020, and each protocol and its citations were analyzed and classified as correct or incorrect. We used descriptive statistics to report the quotation errors and characteristics of the included protocols. The results showed 248 SR protocols, where 124 protocols received quotations and 38 quotations (31.4%) were erroneous. Only 11 (4.4%) of the published SRs and SR protocols had been published previously. Furthermore, the scientific journal in which the most SR protocols were published was Medicine (193; 77.8%), followed by BMJ Open (39; 15.7%). Authors from China (86.5%) were the most productive in publishing SRs and SR protocols. Finally, we concluded that the number of SR protocols and meta-analyses published in scientific journals and indexed by databases exceeds the publication capacity of the SRs associated with them, generating scientific literature that does not make any novel contribution to knowledge.

Quotation errors in general science journals

Due to the incremental nature of scientific discovery, scientific writing requires extensive referencing to the writings of others. The accuracy of this referencing is vital, yet errors do occur. These errors are called ‘quotation errors’. This paper presents the first assessment of quotation errors in high-impact general science journals. A total of 250 random citations were examined. The propositions being cited were compared with the referenced materials to verify whether the propositions could be substantiated by those materials. The study found a total error rate of 25%. This result tracks well with error rates found in similar studies in other academic fields. Additionally, several suggestions are offered that may help to decrease these errors and make similar studies more feasible in the future.

Normalization of Deviation: Quotation Error in Human Factors

Objective The objective of this paper is to examine quotation error in human factors. Background Science progresses through building on the work of previous research. This requires accurate quotation. Quotation error has a number of adverse consequences: loss of credibility, loss of confidence in the journal, and a flawed basis for academic debate and scientific progress. Quotation error has been observed in a number of domains, including marine biology and medicine, but there has been little or no previous study of this form of error in human factors, a domain that specializes in the causes and management of error. Methods A study was conducted examining quotation accuracy of 187 extracts from 118 published articles that cited a control article (Vaughan's 1996 book: The Challenger Launch Decision: Risky Technology, Culture, and Deviance at NASA). Results Of extracts studied, 12.8% ( n = 24) were classed as inaccurate, with 87.2% ( n = 163) being classed as accurate. A second dimension of agreement was examined with 96.3% ( n = 180) agreeing with the control article and only 3.7% ( n = 7) disagreeing. The categories of accuracy and agreement form a two by two matrix. Conclusion Rather than simply blaming individuals for quotation error, systemic factors should also be considered. Vaughan's theory, normalization of deviance, is one systemic theory that can account for quotation error. Application Quotation error is occurring in human factors and should receive more attention. According to Vaughan's theory, the normal everyday systems that promote scholarship may also allow mistakes, mishaps, and quotation error to occur.

Accuracy of cited "facts" in medical research articles: A review of study methodology and recalculation of quotation error rate

Previous reviews estimated that approximately 20 to 25% of assertions cited from original research articles, or "facts," are inaccurately quoted in the medical literature. These reviews noted that the original studies were dissimilar and only began to compare the methods of the original studies. The aim of this review is to examine the methods of the original studies and provide a more specific rate of incorrectly cited assertions, or quotation errors, in original research articles published in medical journals. Additionally, the estimate of quotation errors calculated here is based on the ratio of quotation errors to quotations examined (a percent) rather than the more prevalent and weighted metric of quotation errors to the references selected. Overall, this resulted in a lower estimate of the quotation error rate in original medical research articles. A total of 15 studies met the criteria for inclusion in the primary quantitative analysis. Quotation errors were divided into two categories: content ("factual") or source (improper indirect citation) errors. Content errors were further subdivided into major and minor errors depending on the degree that the assertion differed from the original source. The rate of quotation errors recalculated here is 14.5% (10.5% to 18.6% at a 95% confidence interval). These content errors are predominantly, 64.8% (56.1% to 73.5% at a 95% confidence interval), major errors or cited assertions in which the referenced source either fails to substantiate, is unrelated to, or contradicts the assertion. Minor errors, which are an oversimplification, overgeneralization, or trivial inaccuracies, are 35.2% (26.5% to 43.9% at a 95% confidence interval). Additionally, improper secondary (or indirect) citations, which are distinguished from calculations of quotation accuracy, occur at a rate of 10.4% (3.4% to 17.5% at a 95% confidence interval).

Quotation accuracy in medical journal articles-a systematic review and meta-analysis

Background. Quotations and references are an indispensable element of scientific communication. They should support what authors claim or provide important background information for readers. Studies indicate, however, that quotations not serving their purpose-quotation errors-may be prevalent. Methods. We carried out a systematic review, meta-analysis and meta-regression of quotation errors, taking account of differences between studies in error ascertainment. Results. Out of 559 studies screened we included 28 in the main analysis, and estimated major, minor and total quotation error rates of 11,9%, 95% CI [8.4, 16.6] 11.5% [8.3, 15.7], and 25.4% [19.5, 32.4]. While heterogeneity was substantial, even the lowest estimate of total quotation errors was considerable (6.7%). Indirect references accounted for less than one sixth of all quotation problems. The findings remained robust in a number of sensitivity and subgroup analyses (including risk of bias analysis) and in meta-regression. There was no indication of publication bias. Conclusions. Readers of medical journal articles should be aware of the fact that quotation errors are common. Measures against quotation errors include spot checks by editors and reviewers, correct placement of citations in the text, and declarations by authors that they have checked cited material. Future research should elucidate if and to what degree quotation errors are detrimental to scientific progress.

The qualitative problem of major quotation errors, as illustrated by 10 different examples in the headache literature

There are two types of errors when references are used in the scientific literature: citation errors and quotation errors, and these errors have in reviews mainly been evaluated quantitatively. Quotation errors are the major problem, and 1 review reported 6% major quotation errors. The objective of this listing of quotation errors is to illustrate by qualitative analysis of different types of 10 major quotation errors how and possibly why authors misquote references. The author selected for review the first 10 different consecutive major quotation errors encountered from his reading of the headache literature. The characteristics of the 10 quotation errors ranged considerably. Thus, in a review of migraine therapy in a very prestigious medical journal, the superiority of a new treatment (sumatriptan) vs an old treatment (aspirin plus metoclopramide) was claimed despite no significant difference for the primary efficacy measure in the trial. One author, in a scientific debate, referred to the lack of dilation of the middle meningeal artery in spontaneous migraine despite the fact that only 1 migraine attack was studied. The possibility for creative major quotation errors in the medical literature is most likely infinite. Qualitative evaluations, as the present, of major quotation errors will hopefully result in more general awareness of quotation problems in the medical literature. Even if the final responsibility for correct use of quotations is with the authors, the referees, the experts with the knowledge needed to spot quotation errors, should be more involved in ensuring correct and fair use of references. Finally, this paper suggests that major misleading quotations, if pointed out by readers of the journal, should, as a rule, be corrected by way of an erratum statement.

Accuracy of citation and quotation in foot and ankle surgery journals

BACKGROUND

A recent review of technical editing of research suggests that over one third of references cited in articles in medical journals have some inaccuracies and one fifth of quotations to references in these articles are not accurate.

METHODS

Two hundred and forty-nine citation references and 408 quotes from 25 articles published in 5 orthopaedic journals were randomly selected to determine referencing accuracy. The presence of citation errors was examined by 1 of the authors while the presence of quotation errors was determined by 2 of the authors. Full copies of articles as well as the references were obtained to compare the accuracies.

RESULTS

The total citation error rate was 41% (103 out of 249 references), and the total quotation error rate was 20% (80 out of 408 quotes) for the 5 orthopaedic journals.

CONCLUSION

Citation and quotation errors were still relatively common in orthopaedic journals. While we did not identify any factors associated with citation and quotation errors, the use of technical editing may reduce the amount of citation errors.

CLINICAL RELEVANCE

Readers and authors should be aware that many citations of studies are inaccurate and one should review the original source if it is to be used in another publication or to guide clinical treatment.

Reference accuracy in peer-reviewed pediatric orthopaedic literature

BACKGROUND

Reference accuracy of articles published in the biomedical literature is determined by the presence of citation and quotation errors. A recent review demonstrated that the median citation error rate per biomedical journal was 39%, and the median quotation error rate per journal was 20%. Reference accuracy in pediatric orthopaedic articles has not been previously reported, to our knowledge.

METHODS

Two hundred references from twenty articles published in four peer-reviewed orthopaedic journals were randomly selected for assessment of citation and quotation accuracy. Full-text copies of all original references were obtained by interlibrary loan methods and reviewed directly to establish citation accuracy. The presence of citation errors was determined by a single investigator. The relevance of citation errors was determined by assessing the ease of reference retrieval through PubMed. Quotation accuracy was determined by two examiners who reviewed each of the twenty articles and 200 references to compare the claims made for the references in the article against the data and opinions expressed in the actual reference.

RESULTS

The total citation error rate across all of the journals was 26% (fifty-one of 200 references) with a 95% confidence interval of 16.5% to 37.3%. The median citation error rate per journal was 27% (range, 10% to 38%). Although citation errors were common, most were of minimal significance, as 196 of the 200 references could be retrieved with ease from PubMed. The total quotation error rate across all of the articles was 38% (152 of 398 reference citations) with a 95% confidence interval of 30.1% to 47.0%. The median quotation error rate per journal was 38% (range, 28% to 46%).

CONCLUSIONS

Citation and quotation errors are common in the pediatric orthopaedic literature. Reference accuracy continues to be a substantial problem in the biomedical literature despite recent technological advances such as online databases, easily accessible search engines, and widely available bibliographic software.

Accuracy of references in burns journals

AIMS

To study the incidence and risk factors for citation and quotation errors in two major burns surgery journals.

METHODS

120 references were randomly selected from original articles published in the following two journals - January to December 2006 issues of Burns and Journal of Burn Care & Research. For each reference, the ease of retrieval on PubMed and the presence of citation errors were noted. Two independent observers analysed each reference for quotation errors. The characteristics of the root article, that is, type of study, author numbers, number of references and article word count were noted.

RESULTS

Of the 120 selected references, 117 referred to articles from indexed medical journals published in English. Among these, 4 articles could not be retrieved due to fatal citation errors (3.3%). A further 12 citation errors were noted giving a total citation error rate of 13.3% (95% CI: 6.74-19.93%). Of the 117 references analysed, the quotation error rate was 13.7% (95% CI: 8.6-19.5%) half of which were major errors. There was no significant association between the combined error rate per article and the journal (Kruskal-Wallis test; p=0.861, type of study (Kruskal-Wallis test; p=0.717), author numbers (Spearman's rho=0.197, p=0.423), article length (Spearman's rho=0.118, p=0.705) or references per article (Spearman's rho=0.229, p=0.189).

CONCLUSION

Significant numbers of citation and quotation errors still appear in current burns literature. Incorrect spelling of author names and partial omissions of article titles were the two most common errors. No observable underlying factors were identified in this study. The present results serve as a reminder to authors, editors and peer reviewers for more care of citation accuracy when striving for their common goal of scientific excellence.

Technical editing of research reports in biomedical journals

BACKGROUND

Most journals try to improve their articles by technical editing processes such as proof-reading, editing to conform to 'house styles', grammatical conventions and checking accuracy of cited references. Despite the considerable resources devoted to technical editing, we do not know whether it improves the accessibility of biomedical research findings or the utility of articles. This is an update of a Cochrane methodology review first published in 2003.

OBJECTIVES

To assess the effects of technical editing on research reports in peer-reviewed biomedical journals, and to assess the level of accuracy of references to these reports.

SEARCH STRATEGY

We searched The Cochrane Library Issue 2, 2007; MEDLINE (last searched July 2006); EMBASE (last searched June 2007) and checked relevant articles for further references. We also searched the Internet and contacted researchers and experts in the field.

SELECTION CRITERIA

Prospective or retrospective comparative studies of technical editing processes applied to original research articles in biomedical journals, as well as studies of reference accuracy.

DATA COLLECTION AND ANALYSIS

Two review authors independently assessed each study against the selection criteria and assessed the methodological quality of each study. One review author extracted the data, and the second review author repeated this.

MAIN RESULTS

We located 32 studies addressing technical editing and 66 surveys of reference accuracy. Only three of the studies were randomised controlled trials. A 'package' of largely unspecified editorial processes applied between acceptance and publication was associated with improved readability in two studies and improved reporting quality in another two studies, while another study showed mixed results after stricter editorial policies were introduced. More intensive editorial processes were associated with fewer errors in abstracts and references. Providing instructions to authors was associated with improved reporting of ethics requirements in one study and fewer errors in references in two studies, but no difference was seen in the quality of abstracts in one randomised controlled trial. Structuring generally improved the quality of abstracts, but increased their length. The reference accuracy studies showed a median citation error rate of 38% and a median quotation error rate of 20%.

AUTHORS' CONCLUSIONS

Surprisingly few studies have evaluated the effects of technical editing rigorously. However there is some evidence that the 'package' of technical editing used by biomedical journals does improve papers. A substantial number of references in biomedical articles are cited or quoted inaccurately.

Citation accuracy in the oral and maxillofacial surgery literature

PURPOSE

Readers frequently assume that the references in an article published in a well-respected, peer-reviewed journal will be cited correctly. However, previous studies have shown that this is not always true. The purpose of this study was to evaluate the citation accuracy in 5 oral and maxillofacial surgery journals.

MATERIALS AND METHODS

A search was done using PubMed (http://www.ncbi.nlm.nih.gov/sites/entrez?db=PubMed) to locate the articles cited in 108 references in each journal. If the article could not be located, a search was done in the specific journal in the medical library. The references were then divided into those with major errors that prevented the article from being located and those with minor errors that still permitted the article to be found.

RESULTS

There were a significant number of citations with major errors in each of the journals that made it impossible to find these articles. No correlation was found between the number of major or minor errors and the rank of the senior author or if the article was submitted from a non English-speaking country.

CONCLUSIONS

This study shows that there needs to be a greater effort on the part of authors to provide accurate citations in their articles.

The physical environment and physical activity: a critical appraisal of review articles

BACKGROUND

Over the last few years an increasing number of studies investigating the association between the physical environment and physical activity have been published. Many reviews have also summarized this emerging body of research, and such review papers are frequently used by public health policymakers and researchers themselves to inform decision making.

METHODS

This paper systematically appraises methodologic aspects of literature reviews examining the relationship between physical activity and the physical environment published in peer-reviewed journals between 2000 and 2005. Eleven reviews and their antecedent source papers were examined.

RESULTS

The majority of these reviews omitted between one third and two thirds of the studies that could have been eligible for inclusion at the time they conducted the review. Methodologic information on how the review was conducted was not always provided. Furthermore, in some cases results of a study were reported incorrectly, or physical environmental aspects were conflated with social environmental or cognitive factors. Moreover, when results were reported incorrectly, physical environmental variables were almost always reported as significantly associated with physical activity, when these associations were nonsignificant, or were not assessed as part of the primary study.

CONCLUSIONS

Users of reviews in this field should be aware that there are significant methodologic variations among them, and that some reviews may include only a sample of the relevant primary studies. However, this is difficult to determine given the frequent incompleteness of review method reporting. Greater standardization in the reporting of review methods may assist with future efforts to summarize studies of the relationship between physical environments and physical activity.