Spin in Published Reports of Tinnitus Randomized Controlled Trials: Evidence of Overinterpretation of Results

Linguistic spin in randomized controlled trials about age-related macular degeneration

Objective

To evaluate the prevalence, type and extent of linguistic spin in randomized controlled trials (RCTs) exploring interventions in patients with age-related macular degeneration (AMD), as well as to investigate whether study variables were correlated with linguistic spin.

Study design and setting

PubMed was searched from 2011 to 2020 to identify RCTs including patients with AMD. Two authors independently assessed a total of 96 RCTs. Linear regression analyses were performed to investigate whether linguistic spin was correlated with predefined study variables.

Results

Linguistic spin was found in 61 of 96 abstracts (63.5%) and in 90 of 96 main texts (93.8%). Use of words pointing out the beneficial effect of a treatment and the use of '(statistically) significant/significance' without reporting a P-value or a 95% confidence interval (CI) were the most frequently identified categories of linguistic spin. Sample size was significantly correlated with the total linguistic spin score (95% CI 0.38-5.23, P = 0.02).

Conclusion

A high prevalence and extent of linguistic spin in RCTs about AMD was found. We highlighted the importance of objective reporting and awareness of linguistic spin among ophthalmologists and other readers.

Trends in the Use of Promotional Language (Hype) in Abstracts of Successful National Institutes of Health Grant Applications, 1985-2020

IMPORTANCE

The integrity of the grant application process is important to the success of the entire research enterprise. However, little information is available concerning the prevalence and evolution of subjective or promotional language ("hype") that has the potential to undermine objectivity in the writing and evaluation of grant applications.

OBJECTIVE

To assess changes over time in the use of hype in abstracts of National Institutes of Health (NIH) grant applications.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study assessed the prevalence of promotional adjectives in abstracts in the NIH archive from 1985 to 2020.

MAIN OUTCOMES AND MEASURES

From all abstracts in the NIH RePORTER (Research Portfolio Online Reporting Tools: Expenditures and Results) archive, adjectives were automatically extracted, and their frequencies in the most recent year (2020) were assessed relative to the start year (1985). Adjectives that shifted significantly in frequency and that carried a promotional sense (ie, hype) were retained, and patterns of change were assessed by plotting yearly frequencies (1985-2020). By grouping the adjectives based on shared semantic properties, broad meanings commonly expressed by hype were identified. Absolute change was measured as the difference in normalized frequency between 1985 and 2020. Relative change was measured as the percentage change in normalized frequency in 2020 relative to 1985, or the first year of occurrence.

RESULTS

In total, 901 717 abstracts were analyzed and 139 adjective forms were identified as hype. Among these 139 adjective forms, 130 hype adjectives increased in frequency by 7690 words per million (wpm) (mean [SD] relative increase, 1378% [3132%]), while 9 hype adjectives decreased in frequency by 686 wpm (mean [SD] relative decrease, 44% [18%]). The largest absolute increases were for the terms novel (1054 wpm), critical (555 wpm), and key (461 wpm), while the largest relative increases were for the terms sustainable (25 157%), actionable (16 114%), and scalable (13 029%). Hype most often serves to promote the significance, novelty, scale, and rigor of a project; the utility of the expected outcomes; the qualities of the investigators and research environment; and the gravity of the problem; as well as conveying the personal attitudes of the applicants.

CONCLUSIONS AND RELEVANCE

Levels of hype in successful NIH grant applications have increased over time from 1985 to 2020. The findings in this study should serve to sensitize applicants, reviewers, and funding agencies to the increasing prevalence of subjective, promotional language in funding applications.