Abstracts for reports of randomised trials of COVID-19 interventions had low quality and high spin

Massive covidization and subsequent decovidization of the scientific literature involved 2 million authors

OBJECTIVES

We aimed to examine the growth trajectory and impact of COVID-19-related papers in the scientific literature and how the scientific workforce engaged in this work.

STUDY DESIGN AND SETTING

We used Scopus data to August 1, 2024, and a search string for COVID-19-related publications. Authors of COVID-19 work were mapped against databases of top-cited authors.

RESULTS

Scopus indexed 718,660 COVID-19-related publications. As the proportion of all indexed scientific publications, COVID-19-related publications peaked in September 2021 (4.7%) remained at 4.3%-4.6% for another year and then gradually declined but was still 1.9% in July 2024. COVID-19-related publications included 1,978,612 unique authors: 1,127,215 authors had ≥5 full papers in their career and 53,418 authors were in the top 2% of their scientific subfield. Authors with >10%, >30%, and >50% of their total career citations attributed to COVID-19-related publications were 376,942, 201,702, and 125,523, respectively. As of August 1, 2024, 65 of the top 100 most cited papers published in 2020 were COVID-19-related, declining to 24/100, 19/100, 7/100, and 5/100 for the most cited papers published in 2021, 2022, 2023, and 2024, respectively. Across 174 scientific subfields, 132 had ≥10% of their active influential (top 2% by composite citation indicator) authors publish something on COVID-19 during 2020-2024. Among the 300 authors with highest composite citation indicator specifically for their COVID-19-related publications, 41 were editors or journalists or columnists.

CONCLUSION

COVID-19 massively engaged the scientific workforce in unprecedented ways. As the pandemic ended, there has been a sharp decline in the overall volume and high impact of newly published COVID-19-related publications.

PLAIN LANGUAGE SUMMARY

We evaluated Scopus, a bibliometric database, for the increase and waning of the COVID-19 scientific literature. Until August 1, 2024, we identified 718,660 COVID-19-related publications indexed in Scopus that had involved 1,978,612 unique authors. The rise and subsequent decline pattern of COVID-19 publications was similar to other previous epidemics like Zika, Ebola, and H1N1, but at a far larger, unprecedented scale. 125,523 authors had >50% of their total career citations attributed to COVID-19 papers. 132/174 scientific subfields had at least one of every 10 of their top-cited authors publish something on COVID-19 during 2020-2024. Many influential authors were editors or journalists or columnists. Overall, COVID-19 massively engaged a huge number of authors and created a vast literature. As the interest has now sharply declined, one needs to examine what this immense COVID-19 scientific workforce will do in the future.

Spin in randomized controlled trials of pharmacology in COVID-19: A systematic review

Spin, defined as the misrepresentation of the results of a study, could negate the validity of scientific findings. To explore the manifestation of spin, and identify the factors affecting spin in COVID-19 RCTs, a systematic review was performed from PubMed/Medline, National Institutes of Health, EMBASE, Cochrane, and Web of Science. RCTs on pharmacotherapy for COVID-19 with nonsignificant primary outcomes published in 2020 were included. 21 abstracts (33.9%) and 28 main texts (45.2%) were found to contain spin in at least one section. In the conclusion section, other spin strategies beautifying their findings that were not included in the abstract were found in the main texts. More factors influencing the level of spin were found in abstracts than in the main texts, but most of the levels of spin in abstracts were comparable to those in the main texts. Although common factors that affected the manifestation of spin in the main texts and abstracts were the sample size and type of journal, further research to determine multicollinearity between significant factors and the manifestation of spin is required.

Abstracts of randomized controlled trials in splint therapy for temporomandibular disorders: Reporting quality and spin

BACKGROUND

Poor reporting quality and spin in randomized controlled trial (RCT) abstracts can lead to misinterpretation and distorted interpretation of results.

OBJECTIVES

This methodological study aimed to assess the reporting quality and spin among RCT abstracts on splint therapy for temporomandibular disorders (TMD) and explore the association between spin and potentially related factors.

METHODS

The authors searched PubMed for RCTs on splint therapy for TMD. The reporting quality of each abstract was assessed using the original 16-item CONSORT for abstracts checklist. The authors evaluated the presence and characteristics of spin only in abstracts with nonsignificant primary outcomes according to pre-determined spin strategies. Logistic regression analyses were performed to identify factors associated with the presence of spin.

RESULTS

A total of 148 abstracts were included in the reporting quality evaluation. The mean overall CONSORT score (OCS) was 5.86 (score range: 0-16). Only interventions, objectives and conclusions were adequately reported. Of the 61 RCT abstracts included for spin analysis, spin was identified in 38 abstracts (62.3%), among which 32 abstracts (52.3%) had spin in the Results section and 21 (34.4%) had spin in the Conclusions section. A significantly lower presence of spin was found in studies with exact p-value reporting (OR: 0.170; 95% CI: 0.032-0.887; p = .036) and a two-arm comparison design (OR: 11.777; 95% CI: 2.171-63.877; p = .004).

CONCLUSIONS

The reporting quality of RCT abstracts on splint therapy for TMD is suboptimal and the prevalence of spin is high. More awareness and joint efforts are needed to improve reporting quality and minimize spin.

Assessment of Noninferiority Margins in Cardiovascular Medicine Trials

Background

Noninferiority trials are increasingly common in cardiovascular medicine, but their reporting and interpretation are challenging, particularly when an absolute risk difference is used as noninferiority margin.

Objectives

This study aimed to investigate the effect of using absolute rather than relative noninferiority margins in cardiovascular trials.

Methods

We reviewed noninferiority trials presented at major cardiovascular conferences from 2015 to 2022 and published within the same period. Based on the actual versus anticipated event rates in the control group, we recalculated the absolute noninferiority margin and re-assessed the trial results. The primary outcome of interest was the proportion of trials with a different interpretation after recalculation. Additionally, we analyzed the conclusion statements of these trials to determine if cautionary notes for the interpretation of study results were included.

Results

We analyzed a total of 768 trials, of which 88 had a noninferiority design and 66 used an absolute noninferiority margin. Of 48 comparisons from 45 trials qualifying for the analysis, 11 (22.9%) had divergent results after recalculation of the absolute noninferiority margin based on the observed rather than anticipated event rate. Ten trials originally claiming noninferiority, did not meet it after the margin recalculation. All of them did not include statements suggesting cautionary interpretation of the study results in the conclusion section. Compared with the other trials, these displayed a larger median difference between anticipated and recalculated noninferiority margins (44.7% [IQR: 38.6%-56.7%] vs 15.3% [IQR: -1.5% to 28.9%]; P < 0.001).

Conclusions

Recalculating noninferiority margins based on actual event rates, rather than anticipated ones, led to different outcomes in approximately 1 out of 4 cardiovascular trials, with most divergent trials lacking cautionary interpretation. These findings emphasize the importance of using or supplementing the relative noninferiority margin, particularly in studies with significant deviations between observed and expected event rates. This underscores the critical need for enhanced methodological and reporting standards in noninferiority trials, especially those employing absolute margins.

Evaluating the completeness of the reporting of abstracts since the publication of the CONSORT extension for abstracts: an evaluation of randomized controlled trial in ten nursing journals

BACKGROUND

As a practice-oriented discipline, strict adherence to reporting guidelines is particularly important in randomized controlled trial (RCT) abstracts of the nursing area. However, whether abstract reports after 2010 have complied with the Consolidated Standards of Reporting Trials for Abstracts (CONSORT-A) guideline is unclear. This study aimed to evaluate whether the publication of CONSORT-A has improved abstract reporting in nursing and explores the factors associated with better adherence to the guidelines.

METHODS

We searched the Web of Science for 200 RCTs randomly selected from ten nursing journals. We used a data extraction form based on CONSORT-A, including 16 items, to analyze the reporting adherence to the guidelines, and the reporting rate of each item and the total score for each abstract were used to indicate adherence and overall quality score (OQS, range 0-16). A comparison of the total mean score between the two periods was made, and affecting factors were analyzed.

RESULTS

In the studies we included, 48 abstracts were published pre-CONSORT-A whereas 152 post-CONSORT-A. The overall mean score for reporting adherence to 16 items was 7.41 ± 2.78 and 9.16 ± 2.76 for pre- and post-CONSORT-A, respectively (total score: 16). The most poorly reported items are "harms (0%)," "outcomes in method (8.5%)," "randomization (25%)," and "blinding (6.5%)." Items including the year of publication, impact factor, multiple center trial, word count, and structured abstract are significantly associated with higher adherence.

CONCLUSIONS

The adherence to abstract reporting in nursing literature has improved since the CONSORT-A era, but the overall completeness of RCT abstracts remained low. A joint effort by authors, editors, and journals is necessary to improve reporting quality of RCT abstracts.

Spin in the abstracts of randomized controlled trials of nurse-led care: A cross-sectional study

BACKGROUND

Nurse-led models of care had been reported as effective intervention approaches for improving health management and reducing hospitalizations of target patients in a number of studies. However, the reporting quality of studies in the field varied and there was a lack of literature evaluation.

OBJECTIVE

The objective of the study was to assess the reporting quality and spin of abstracts of published randomized controlled trials which had statistically not significant primary outcomes. Moreover, potential factors associated with the presence of spin were also assessed.

METHODS

Studies on nurse-led care were retrieved from PubMed from January 1st, 2017, to December 31st, 2021. Only randomized controlled trials with statistically not significant primary outcomes were included. Study screening and data extraction were carried out by two reviewers independently. The reporting quality of each abstract was evaluated by the Consolidated Standards of Reporting Trials statement, and spin strategies were analyzed using a pre-designed assessment form. Potential predictors for the presence of spin were analyzed by multivariate logistic regressions.

RESULTS

The overall reporting quality of the included 75 randomized controlled trial abstracts was not satisfying, with a median score of 16-item Consolidated Standards of Reporting Trials statement at 6 (IQR 5, 8). Forty abstracts used at least one spin strategy in abstracts. Among them, 18 (45.0 %) used spin strategies in the result section and 39 (97.5 %) had spin in the conclusion section. The most common spin strategy identified in abstracts was focusing on statistically significant secondary outcomes (12/40, 30.0 %) in the result section and claiming benefit with no consideration of statistically not significant results for the primary outcomes (32/40, 80.0 %) in the conclusion section. Based on the definition, 29 (72.5 %) abstracts were assessed to have high level of spin in the conclusions of abstracts. By multivariate logistic regression analyses, it was found that only geographic origin (reference: studies from Asian countries, OR = 0.118, 95 % CI 0.027 to 0.511, P = 0.004) and the Consolidated Standards of Reporting Trials statement score (reference: lower score, OR = 0.625, 95 % CI 0.470 to 0.829, P = 0.001) were significantly associated with the presence of spin in abstracts.

CONCLUSION

Among the randomized controlled trials with statistically not significant primary outcomes in the field of nurse-led care, the reporting quality of abstracts needs to be improved. Trials from Asian countries and with lower Consolidated Standards of Reporting Trials statement scores are more likely to present spin in abstracts. Findings reported in the result and conclusion sections of these abstracts need to be interpreted with caution.

Abstracts of randomized controlled trials in the field of dentofacial trauma: Reporting quality and spin

BACKGROUND/AIM

Abstracts of randomized controlled trials (RCTs) provide a summary of the entire trial report. Their transparent, detailed, and accurate reporting is essential for clinical decision-making and evidence-based dental practice. The aim of this study was to assess the reporting quality and prevalence of spin in abstracts of RCTs in the field of dentofacial trauma.

MATERIALS AND METHODS

The PubMed database was searched to identify RCT reports published between 2017 and 2021 in the field of dentofacial trauma. The reporting quality of abstracts was assessed according to the 16-item CONSORT for Abstracts checklist and measured with an overall quality score (OQS, range: 0-16). Linear regression analyses were used to identify factors associated with reporting quality. Among the included RCTs, parallel-group RCTs with non-significant primary outcomes were selected for spin assessment using predefined spin strategies.

RESULTS

One hundred and twelve eligible abstracts were identified and included. The mean OQS was 4.51 (SD, 1.35; 95% CI, 4.26-4.76). Abstracts with more than 250 words (p = .004) and a structured format (p = .032) had significantly better reporting quality. Of the 30 abstracts that were eligible for spin assessment, spin was identified in 23 (76.7%). Among these, spin was observed in the Conclusions sections of 22 abstracts (73.3%) and the Results sections of 9 abstracts (30.0%).

CONCLUSIONS

Among RCT abstracts in the field of dentofacial trauma, the reporting quality was sub-optimal and the prevalence of spin was relatively high. Strict adherence to the CONSORT for Abstracts guidelines is needed to ensure complete and transparent reporting. Relevant stakeholders need to make concerted efforts to avoid spin.

Reporting quality for abstracts of randomised trials on child and adolescent depression prevention: a meta-epidemiological study on adherence to CONSORT for abstracts

OBJECTIVES

This study aimed to investigate adherence to Consolidated Standards of Reporting Trials (CONSORT) for abstracts in reports of randomised trials on child and adolescent depression prevention. Secondary objective was to examine factors associated with overall reporting quality.

DESIGN

Meta-epidemiological study.

DATA SOURCES

We searched MEDLINE, EMBASE, PsycINFO, PsycArticles and CENTRAL.

ELIGIBILITY CRITERIA

Trials were eligible if the sample consisted of children and adolescents under 18 years with or without an increased risk for depression or subthreshold depression. We included reports published from 1 January 2003 to 8 August 2020 on randomised controlled trials (RCTs) and cluster randomised trials (CRTs) assessing universal, selective and indicated interventions aiming to prevent the onset of depression or reducing depressive symptoms.

DATA EXTRACTION AND SYNTHESIS

As the primary outcome measure, we assessed for each trial abstract whether information recommended by CONSORT was adequately reported, inadequately reported or not reported. Moreover, we calculated a summative score of overall reporting quality and analysed associations with trial and journal characteristics.

RESULTS

We identified 169 eligible studies, 103 (61%) RCTs and 66 (39%) CRTs. Adequate reporting varied considerably across CONSORT items: while 9 out of 10 abstracts adequately reported the study objective, no abstract adequately provided information on blinding. Important adverse events or side effects were only adequately reported in one out of 169 abstracts. Summative scores for the abstracts' overall reporting quality ranged from 17% to 83%, with a median of 40%. Scores were associated with the number of authors, abstract word count, journal impact factor, year of publication and abstract structure.

CONCLUSIONS

Reporting quality for abstracts of trials on child and adolescent depression prevention is suboptimal. To help health professionals make informed judgements, efforts for improving adherence to reporting guidelines for abstracts are needed.

Massive covidization of research citations and the citation elite

The COVID-19 pandemic saw a massive mobilization of the scientific workforce. We evaluated the citation impact of COVID-19 publications relative to all scientific work published in 2020 to 2021, finding that 20% of citations received to papers published in 2020 to 2021 were to COVID-19–related papers. Across science, 98 of the 100 most-cited papers published in 2020 to 2021 were related to COVID-19. A large number of scientists received large numbers of citations to their COVID-19 work, often exceeding the citations they had received to all their work during their entire career. We document a strong covidization of research citations across science. This may have major repercussions for research priorities and the evolution of research on COVID-19 and beyond.

Massive scientific productivity accompanied the COVID-19 pandemic. We evaluated the citation impact of COVID-19 publications relative to all scientific work published in 2020 to 2021 and assessed the impact on scientist citation profiles. Using Scopus data until August 1, 2021, COVID-19 items accounted for 4% of papers published, 20% of citations received to papers published in 2020 to 2021, and >30% of citations received in 36 of the 174 disciplines of science (up to 79.3% in general and internal medicine). Across science, 98 of the 100 most-cited papers published in 2020 to 2021 were related to COVID-19; 110 scientists received ≥10,000 citations for COVID-19 work, but none received ≥10,000 citations for non–COVID-19 work published in 2020 to 2021. For many scientists, citations to their COVID-19 work already accounted for more than half of their total career citation count. Overall, these data show a strong covidization of research citations across science, with major impact on shaping the citation elite.

A Synthesis of the Formats for Correcting Erroneous and Fraudulent Academic Literature, and Associated Challenges

Academic publishing is undergoing a highly transformative process, and many established rules and value systems that are in place, such as traditional peer review (TPR) and preprints, are facing unprecedented challenges, including as a result of post-publication peer review. The integrity and validity of the academic literature continue to rely naively on blind trust, while TPR and preprints continue to fail to effectively screen out errors, fraud, and misconduct. Imperfect TPR invariably results in imperfect papers that have passed through varying levels of rigor of screening and validation. If errors or misconduct were not detected during TPR's editorial screening, but are detected at the post-publication stage, an opportunity is created to correct the academic record. Currently, the most common forms of correcting the academic literature are errata, corrigenda, expressions of concern, and retractions or withdrawals. Some additional measures to correct the literature have emerged, including manuscript versioning, amendments, partial retractions and retract and replace. Preprints can also be corrected if their version is updated. This paper discusses the risks, benefits and limitations of these forms of correcting the academic literature.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10838-022-09607-4.

The rapid, massive growth of COVID-19 authors in the scientific literature

We examined the extent to which the scientific workforce in different fields was engaged in publishing COVID-19-related papers. According to Scopus (data cut, 1 August 2021), 210 183 COVID-19-related publications included 720 801 unique authors, of which 360 005 authors had published at least five full papers in their career and 23 520 authors were at the top 2% of their scientific subfield based on a career-long composite citation indicator. The growth of COVID-19 authors was far more rapid and massive compared with cohorts of authors historically publishing on H1N1, Zika, Ebola, HIV/AIDS and tuberculosis. All 174 scientific subfields had some specialists who had published on COVID-19. In 109 of the 174 subfields of science, at least one in 10 active, influential (top 2% composite citation indicator) authors in the subfield had authored something on COVID-19. Fifty-three hyper-prolific authors had already at least 60 (and up to 227) COVID-19 publications each. Among the 300 authors with the highest composite citation indicator for their COVID-19 publications, most common countries were USA (n = 67), China (n = 52), UK (n = 32) and Italy (n = 18). The rapid and massive involvement of the scientific workforce in COVID-19-related work is unprecedented and creates opportunities and challenges. There is evidence for hyper-prolific productivity.