Technical editing of research reports in biomedical journals

Citation of updated and co-published Cochrane Methodology Reviews

BACKGROUND

To evaluate the number of citations for Cochrane Methodology Reviews after they have been updated or co-published in another journal, and the effect of co-publishing the review on the co-publishing journal's impact factor (IF).

METHODS

We identified all Cochrane Methodology Reviews published in the Cochrane Database of Systematic Reviews (CDSR) before 2018 and searched for co-published versions in the Web of Science Core Collection database up to 16 August 2022. The included reviews were in two cohorts: those that had been published and updated in CDSR and those that had been published in CDSR and co-published in another journal. The primary outcome measured the citation number to updated and original reviews in the first five years after publication of the updated review, and assessed the citation number of co-published and non-co-published reviews in the first five years after publication of the co-published version. The secondary outcome was the ratio of an adjusted IF and the actual IF of the co-publishing journal.

RESULTS

Eight updated and six original reviews were identified for the updated cohort of reviews, and four co-published reviews were included in the co-published cohort. The original reviews continued to be cited after the update was published but the median for the total number of citations was non-significantly higher for the updated reviews than for their original version[161 (Interquartile range (IQR) 85, 198) versus 113 (IQR 15, 433)]. The median number of total citations [362 (IQR 179, 840) versus 145 (IQR 75, 445)] and the median number of citations to the review in the first five years after co-publication combined and in each of those years was higher in the co-published group than in the non-co-published group. One of the three journals that co-published Reviews in the first year and two journals in the second year had a lower IF after co-publication.

CONCLUSIONS

Earlier versions of Cochrane Methodology Reviews continue to be cited after an update is published, which raises doubts about whether those citing are using the most recent evidence or are aware of the update. Co-publication facilitates broader application and dissemination of Cochrane methodology evidence.

Quotation errors related to the wound management of open lower limb fractures (WOLLF) randomized clinical trial

PURPOSE

We assessed the accuracy of quotations in the published literature of the Wound Management of Open Lower Limb Fractures (WOLLF) Randomized Clinical Trial.

METHODS

A literature search was performed to identify all citations of WOLLF from 2018 to 2021. Duplicate papers, non-English language papers, textbooks, technical tip videos, Ph.D. theses, and other grey literature were excluded from analysis. Eligible publications had their full text assessed by 2 independent reviewers who used a validated framework of error classification.

RESULTS

We identified 62 original papers that met our eligibility criteria and referenced WOLLF. Of the 62 papers, 12 contained a quotation error (12/62, 19%). Errors were classified into major and minor. There were 7 major errors and 5 minor errors. The majority of quotation errors (7/12, 58%) occurred due to multi-referencing, where groups of references were used to support single assertions. There was substantial agreement between the two independent reviewers as determined by a Kappa coefficient of 0.761.

CONCLUSION

Our study demonstrated a 19% quotation error rate in the literature regarding WOLLF. The majority were due to multi-referencing. We suggest that both authors and reviewers carefully check the source literature of key references.

Quotation Accuracy of Systematic Review and Meta-Analysis Protocols on Acupuncture

Currently, published systematic review protocols (SR protocols) have increasingly become a new trend in fields such as acupuncture and are therefore a new source of quotations in these fields. Systematic reviews are considered the pinnacle of the evidence pyramid as they embody comprehensive literature searching. Quotations are key elements to achieve this goal as they can support the assertions of the original authors, but the ‘misquotation’ exists, too, and they can be misleading to the reader. The aim of this study was to examine the quotation accuracy of SR protocols in a meta-analysis on acupuncture research. We searched SCOPUS through 31 December, 2020, and each protocol and its citations were analyzed and classified as correct or incorrect. We used descriptive statistics to report the quotation errors and characteristics of the included protocols. The results showed 248 SR protocols, where 124 protocols received quotations and 38 quotations (31.4%) were erroneous. Only 11 (4.4%) of the published SRs and SR protocols had been published previously. Furthermore, the scientific journal in which the most SR protocols were published was Medicine (193; 77.8%), followed by BMJ Open (39; 15.7%). Authors from China (86.5%) were the most productive in publishing SRs and SR protocols. Finally, we concluded that the number of SR protocols and meta-analyses published in scientific journals and indexed by databases exceeds the publication capacity of the SRs associated with them, generating scientific literature that does not make any novel contribution to knowledge.

Quotation errors related to the Proximal Fracture of the Humerus Evaluation by Randomization (ProFHER) study

AIM

The aim of this study was to assess the accuracy of quotations of the Proximal Fracture of the Humerus Evaluation by Randomization (ProFHER) study in the published literature.

METHODS

A literature search was performed from March 2015 to November 2019 to identify all papers that reference ProFHER since its publication. Full text articles were reviewed by two independent reviewers using a validated framework of assessing quotation errors. A kappa co-efficient was calculated to assess interobserver reliability of the reviewers.

RESULTS

There were 260 individual ProFHER quoted references within the 138 included articles. We identified 35/260 quotation errors (13%). Of these, 10/35 (29%) were major quotation errors and 25/35 (71%) minor quotation errors. There was substantial interobserver agreement when errors were classified. Of the 10 major errors, six quotations were not substantiated by the results of ProFHER and three were unrelated to ProFHER. One paper contained a quotation error that contradicted the results of ProFHER. Of the 25 minor errors, 19 oversimplified or generalised the conclusions of ProFHER and six contained numerical or grammatical errors.

CONCLUSION

The current study demonstrated substantial inaccuracies in quotations of the Proximal Fracture of the Humerus Evaluation by Randomization study. Vigilance is recommended when quoting the literature and reviewing submitted papers in order to prevent the perpetuation of misquoted data.

Make quotations great again: a proposal to reduce false-knowledge

The last decades saw remarkable change in the way healthcare professionals generate and consume medical knowledge. Information management technologies have evolved considerably, yet medical publications continue to use a referencing system that has changed very little since the turn of the 20th century. Research suggests that up to one in five referenced claims quotes the original text inaccurately. Many authors, perhaps inadvertently, contribute to this process by citing non-primary data and amplifying the errors of their predecessors. Erroneous claims are propagated, accumulate into false belief systems and generate inaccurate knowledge. Updating the referencing system to provide additional information to support each referenced claim (eg, the location of the referenced statement in the original text and the nature of that text) could, perhaps, address this cycle of inaccuracy. We believe such changes in the referencing system would prompt authors to rigorously verify referenced claims and provide readers with context to inform a critical evaluation of the text. We detail our proposal for changes in the notations used for referencing, as well as in the information provided within reference lists. We also discuss some barriers and solutions to the adoption of our proposal.

Guidelines for Editing Biomedical Journals: Recommended by Academy of Medical Sciences of Bosnia and Herzegovina

Background:

Enormous number of medical journals published around the globe requires standardization of editing practice.

Objective:

The aim of this article was to enlist main principles of editing biomedical scientific journals adopted at annual meeting of Academy of Medical Sciences of Bosnia & Herzegovina (AMSB&H).

Methods:

The evidence for writing this Guideline was systematically searched for during September 2020 in the PUBMED and GOOGLE SCHOLAR databases. The inclusion criteria were: original studies, systematic reviews, invited expert opinions, guidelines and editorials. The exclusion criteria were narrative reviews and uninvited opinion articles. The retrieved evidence was analyzed by members of the AMSB&H, then discussed at 2020 annual meeting of the AMSB&H and adopted by nominal group technique.

Results:

In total 14 recommendations were made, based on A to C class of evidence. The editors should educate potential authors and instruct them how to structure their manuscript, how to write every segment of the manuscript, and take care about correct use of statistical tests. Plagiarism detection softwares should be used regularly, and statistical and technical editing should be rigorous and thorough. International standards of reporting specific types of studies should be followed, and principles of ethical and responsible behavior of editors, reviewers and authors should be published on the journal’s web site. The editors should insist on registration of clinical studies before submission, and check whether non-essential personal information is removed from the articles; when essential personal information has to be included, an article should not be published without signed informed consent by the patient to whom these information relate.

Conclusions:

Principles of editing biomedical scientific journals recommended in this guideline should serve as one of the means of improving medical journals’ quality.

Occurrence and nature of questionable research practices in the reporting of messages and conclusions in international scientific Health Services Research publications: a structured assessment of publications authored by researchers in the Netherlands

OBJECTIVES

Explore the occurrence and nature of questionable research practices (QRPs) in the reporting of messages and conclusions in international scientific Health Services Research (HSR) publications authored by researchers from HSR institutions in the Netherlands.

DESIGN

In a joint effort to assure the overall quality of HSR publications in the Netherlands, 13 HSR institutions in the Netherlands participated in this study. Together with these institutions, we constructed and validated an assessment instrument covering 35 possible QRPs in the reporting of messages and conclusions. Two reviewers independently assessed a random sample of 116 HSR articles authored by researchers from these institutions published in international peer-reviewed scientific journals in 2016.

SETTING

Netherlands, 2016.

SAMPLE

116 international peer-reviewed HSR publications.

MAIN OUTCOME MEASURES

Median number of QRPs per publication, the percentage of publications with observed QRP frequencies, occurrence of specific QRPs and difference in total number of QRPs by methodological approach, type of research and study design.

RESULTS

We identified a median of six QRPs per publication out of 35 possible QRPs. QRPs occurred most frequently in the reporting of implications for practice, recommendations for practice, contradictory evidence, study limitations and conclusions based on the results and in the context of the literature. We identified no differences in total number of QRPs in papers based on different methodological approach, type of research or study design.

CONCLUSIONS

Given the applied nature of HSR, both the severity of the identified QRPs, and the recommendations for policy and practice in HSR publications warrant discussion. We recommend that the HSR field further define and establish its own scientific norms in publication practices to improve scientific reporting and strengthen the impact of HSR. The results of our study can serve as an empirical basis for continuous critical reflection on the reporting of messages and conclusions.

Biomedical text mining for research rigor and integrity: tasks, challenges, directions

An estimated quarter of a trillion US dollars is invested in the biomedical research enterprise annually. There is growing alarm that a significant portion of this investment is wasted because of problems in reproducibility of research findings and in the rigor and integrity of research conduct and reporting. Recent years have seen a flurry of activities focusing on standardization and guideline development to enhance the reproducibility and rigor of biomedical research. Research activity is primarily communicated via textual artifacts, ranging from grant applications to journal publications. These artifacts can be both the source and the manifestation of practices leading to research waste. For example, an article may describe a poorly designed experiment, or the authors may reach conclusions not supported by the evidence presented. In this article, we pose the question of whether biomedical text mining techniques can assist the stakeholders in the biomedical research enterprise in doing their part toward enhancing research integrity and rigor. In particular, we identify four key areas in which text mining techniques can make a significant contribution: plagiarism/fraud detection, ensuring adherence to reporting guidelines, managing information overload and accurate citation/enhanced bibliometrics. We review the existing methods and tools for specific tasks, if they exist, or discuss relevant research that can provide guidance for future work. With the exponential increase in biomedical research output and the ability of text mining approaches to perform automatic tasks at large scale, we propose that such approaches can support tools that promote responsible research practices, providing significant benefits for the biomedical research enterprise.

Accuracy of cited "facts" in medical research articles: A review of study methodology and recalculation of quotation error rate

Previous reviews estimated that approximately 20 to 25% of assertions cited from original research articles, or "facts," are inaccurately quoted in the medical literature. These reviews noted that the original studies were dissimilar and only began to compare the methods of the original studies. The aim of this review is to examine the methods of the original studies and provide a more specific rate of incorrectly cited assertions, or quotation errors, in original research articles published in medical journals. Additionally, the estimate of quotation errors calculated here is based on the ratio of quotation errors to quotations examined (a percent) rather than the more prevalent and weighted metric of quotation errors to the references selected. Overall, this resulted in a lower estimate of the quotation error rate in original medical research articles. A total of 15 studies met the criteria for inclusion in the primary quantitative analysis. Quotation errors were divided into two categories: content ("factual") or source (improper indirect citation) errors. Content errors were further subdivided into major and minor errors depending on the degree that the assertion differed from the original source. The rate of quotation errors recalculated here is 14.5% (10.5% to 18.6% at a 95% confidence interval). These content errors are predominantly, 64.8% (56.1% to 73.5% at a 95% confidence interval), major errors or cited assertions in which the referenced source either fails to substantiate, is unrelated to, or contradicts the assertion. Minor errors, which are an oversimplification, overgeneralization, or trivial inaccuracies, are 35.2% (26.5% to 43.9% at a 95% confidence interval). Additionally, improper secondary (or indirect) citations, which are distinguished from calculations of quotation accuracy, occur at a rate of 10.4% (3.4% to 17.5% at a 95% confidence interval).

Quotation accuracy in medical journal articles-a systematic review and meta-analysis

Background. Quotations and references are an indispensable element of scientific communication. They should support what authors claim or provide important background information for readers. Studies indicate, however, that quotations not serving their purpose-quotation errors-may be prevalent. Methods. We carried out a systematic review, meta-analysis and meta-regression of quotation errors, taking account of differences between studies in error ascertainment. Results. Out of 559 studies screened we included 28 in the main analysis, and estimated major, minor and total quotation error rates of 11,9%, 95% CI [8.4, 16.6] 11.5% [8.3, 15.7], and 25.4% [19.5, 32.4]. While heterogeneity was substantial, even the lowest estimate of total quotation errors was considerable (6.7%). Indirect references accounted for less than one sixth of all quotation problems. The findings remained robust in a number of sensitivity and subgroup analyses (including risk of bias analysis) and in meta-regression. There was no indication of publication bias. Conclusions. Readers of medical journal articles should be aware of the fact that quotation errors are common. Measures against quotation errors include spot checks by editors and reviewers, correct placement of citations in the text, and declarations by authors that they have checked cited material. Future research should elucidate if and to what degree quotation errors are detrimental to scientific progress.

The qualitative problem of major quotation errors, as illustrated by 10 different examples in the headache literature

There are two types of errors when references are used in the scientific literature: citation errors and quotation errors, and these errors have in reviews mainly been evaluated quantitatively. Quotation errors are the major problem, and 1 review reported 6% major quotation errors. The objective of this listing of quotation errors is to illustrate by qualitative analysis of different types of 10 major quotation errors how and possibly why authors misquote references. The author selected for review the first 10 different consecutive major quotation errors encountered from his reading of the headache literature. The characteristics of the 10 quotation errors ranged considerably. Thus, in a review of migraine therapy in a very prestigious medical journal, the superiority of a new treatment (sumatriptan) vs an old treatment (aspirin plus metoclopramide) was claimed despite no significant difference for the primary efficacy measure in the trial. One author, in a scientific debate, referred to the lack of dilation of the middle meningeal artery in spontaneous migraine despite the fact that only 1 migraine attack was studied. The possibility for creative major quotation errors in the medical literature is most likely infinite. Qualitative evaluations, as the present, of major quotation errors will hopefully result in more general awareness of quotation problems in the medical literature. Even if the final responsibility for correct use of quotations is with the authors, the referees, the experts with the knowledge needed to spot quotation errors, should be more involved in ensuring correct and fair use of references. Finally, this paper suggests that major misleading quotations, if pointed out by readers of the journal, should, as a rule, be corrected by way of an erratum statement.

A retrospective analysis of reported errata in five leading medical journals in 2012

Background

Although medical publications are frequently used as the source of information, the prevalence of errata remains unclear. The objective of this study was to examine peer-review and publication processes of medical journals as well as to determine the occurrence of reported errata in medical journals and timeliness in identifying and correcting errata.

Methods

Five medical journals, New England Journal of Medicine, Annals of Internal Medicine, British Medical Journal, Journal of American Medical Association, and Lancet, were evaluated. The characteristics of these journals were obtained from editors’ survey. All these journals report errata noted in their prior publications. We retrospectively analyzed all errata reported from January 1, 2012, to December 31, 2012. The mean number of reported errata per issue, the most common errata, and the mean time to report errata were calculated.

Results

The journals had high impact factors (14–51), received 3,200 to more than 15,000 submissions in 2012, and utilized two or more external reviewers and usually two or more editors for any accepted articles. All the journals edited the accepted articles, including references, figures, and tables for style. A mean of 1.3 articles with ≥1 errata was reported per issue (a total of 306 articles with errata in 226 issues). Errata in author's information, numeric errata, and errata in the figures and tables were the most common errata. The mean time to report the errata was 122 days.

Conclusion

The high-impact journals, with extensive pre-publication review, reported relatively few errata per issue. The delay in reporting errata needs further exploration.

Reporting of methods was better in the Clinical Trials Registry-India than in Indian journal publications

OBJECTIVE

We sought to evaluate if editorial policies and the reporting quality of randomized controlled trials (RCTs) had improved since our 2004-05 survey of 151 RCTs in 65 Indian journals, and to compare reporting quality of protocols in the Clinical Trials Registry-India (CTRI).

STUDY DESIGN AND SETTING

An observational study of endorsement of Consolidated Standards for the Reporting of Trials (CONSORT) and International Committee of Medical Journal Editors (ICMJE) requirements in the instructions to authors in Indian journals, and compliance with selected requirements in all RCTs published during 2007-08 vs. our previous survey and between all RCT protocols in the CTRI on August 31, 2010 and published RCTs from both surveys.

RESULTS

Journal policies endorsing the CONSORT statement (22/67, 33%) and ICMJE requirements (35/67, 52%) remained suboptimal, and only 4 of 13 CONSORT items were reported in more than 50% of the 145 RCTs assessed. Reporting of ethical issues had improved significantly, and that of methods addressing internal validity had not improved. Adequate methods were reported significantly more frequently in 768 protocols in the CTRI, than in the 296 published trials.

CONCLUSION

The CTRI template facilitates the reporting of valid methods in registered trial protocols. The suboptimal compliance with CONSORT and ICMJE requirements in RCTs published in Indian journals reduces credibility in the reliability of their results.

Predictors of the accuracy of quotation of references in peer-reviewed orthopaedic literature in relation to publications on the scaphoid

Using inaccurate quotations can propagate misleading information, which might affect the management of patients. The aim of this study was to determine the predictors of quotation inaccuracy in the peer-reviewed orthopaedic literature related to the scaphoid. We randomly selected 100 papers from ten orthopaedic journals. All references were retrieved in full text when available or otherwise excluded. Two observers independently rated all quotations from the selected papers by comparing the claims made by the authors with the data and expressed opinions of the reference source. A statistical analysis determined which article-related factors were predictors of quotation inaccuracy. The mean total inaccuracy rate of the 3840 verified quotes was 7.6%. There was no correlation between the rate of inaccuracy and the impact factor of the journal. Multivariable analysis identified the journal and the type of study (clinical, biomechanical, methodological, case report or review) as important predictors of the total quotation inaccuracy rate.

We concluded that inaccurate quotations in the peer-reviewed orthopaedic literature related to the scaphoid were common and slightly more so for certain journals and certain study types. Authors, reviewers and editorial staff play an important role in reducing this inaccuracy.

CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials

Overwhelming evidence shows the quality of reporting of randomised controlled trials (RCTs) is not optimal. Without transparent reporting, readers cannot judge the reliability and validity of trial findings nor extract information for systematic reviews. Recent methodological analyses indicate that inadequate reporting and design are associated with biased estimates of treatment effects. Such systematic error is seriously damaging to RCTs, which are considered the gold standard for evaluating interventions because of their ability to minimise or avoid bias. A group of scientists and editors developed the CONSORT (Consolidated Standards of Reporting Trials) statement to improve the quality of reporting of RCTs. It was first published in 1996 and updated in 2001. The statement consists of a checklist and flow diagram that authors can use for reporting an RCT. Many leading medical journals and major international editorial groups have endorsed the CONSORT statement. The statement facilitates critical appraisal and interpretation of RCTs. During the 2001 CONSORT revision, it became clear that explanation and elaboration of the principles underlying the CONSORT statement would help investigators and others to write or appraise trial reports. A CONSORT explanation and elaboration article was published in 2001 alongside the 2001 version of the CONSORT statement. After an expert meeting in January 2007, the CONSORT statement has been further revised and is published as the CONSORT 2010 Statement. This update improves the wording and clarity of the previous checklist and incorporates recommendations related to topics that have only recently received recognition, such as selective outcome reporting bias. This explanatory and elaboration document-intended to enhance the use, understanding, and dissemination of the CONSORT statement-has also been extensively revised. It presents the meaning and rationale for each new and updated checklist item providing examples of good reporting and, where possible, references to relevant empirical studies. Several examples of flow diagrams are included. The CONSORT 2010 Statement, this revised explanatory and elaboration document, and the associated website (www.consort-statement.org) should be helpful resources to improve reporting of randomised trials.

Checking reference lists to find additional studies for systematic reviews

BACKGROUND

Checking reference lists to identify relevant studies for systematic reviews is frequently recommended by systematic review manuals and is often undertaken by review authors. To date, no systematic review has explicitly examined the effectiveness of checking reference lists as a method to supplement electronic searching.

OBJECTIVES

To investigate the effectiveness of checking reference lists for the identification of additional, relevant studies for systematic reviews. Effectiveness is defined as the proportion of relevant studies identified by review authors solely by checking reference lists.

SEARCH STRATEGY

We searched the databases of The Cochrane Library (Issue 3, 2008), Library and Information Science abstracts (LISA) (1969 to July 2008) and MEDLINE (1966 to July 2008). We contacted experts in systematic review methods and examined reference lists of articles.

SELECTION CRITERIA

Studies of any design which examined checking reference lists as a search method for systematic reviews in any area. The primary outcome was the additional yield of relevant studies (i.e. studies not found through any other search methodologies); other outcomes were publication types identified and data pertaining to the costs (e.g. cost-effectiveness, cost-efficiency) of checking reference lists.

DATA COLLECTION AND ANALYSIS

We summarized data descriptively.

MAIN RESULTS

We included 12 studies (in 13 publications) in this review, but interpretability and generalizability of these studies is difficult and the study designs used were at high risk of bias. The additional yield (calculated by dividing the additional 'unique' yield identified by checking reference lists by the total number of studies found to be eligible within the study) of relevant studies identified through checking reference lists ranged from 2.5% to 42.7%. Only two studies reported yield information by publication type (dissertations and systematic reviews). No cost data were reported although one study commented that it was impossible to isolate the time spent on reference tracking since this was done in parallel with the critical appraisal of each paper, and for that particular study costs were not specifically estimated.

AUTHORS' CONCLUSIONS

There is some evidence to support the use of checking reference lists for locating studies in systematic reviews. However, this evidence is derived from weak study designs. In situations where the identification of all relevant studies through handsearching and database searching is difficult, it would seem prudent that authors of reviews check reference lists to supplement their searching. The challenge, therefore, is for review authors to recognize those situations.