Misrepresentation and distortion of research in biomedical literature

Are Causal Statements Reported in Pharmacovigilance Disproportionality Analyses Using Individual Case Safety Reports Exaggerated in Related Citations? A Meta-epidemiological Study

BACKGROUND

Previous meta-epidemiological surveys have found considerable misinterpretation of results of disproportionality analyses. We aim to explore the relationship between the strength of causal statements used in title and abstract conclusions of pharmacovigilance disproportionality analyses and the strength of causal language used in citing studies.

METHODS

On March 30, 2022, we selected the 30 disproportionality studies with the highest Altmetric Attention Scores. For each article, we extracted all citing studies using the Dimension database (n = 1434). In parallel, two authors assessed the strength of causal statements in the title and abstract conclusions of source articles and in the paragraph of citing studies. Based on previous studies, the strength of causal language was quantified based on a four-level scale (1-appropriate interpretation; 2-ambiguous interpretation; 3-conditionally causal; 4-unconditionally causal). Discrepancies were solved by discussion until consensus among the team. We assessed the association between the strength of causal statements in source articles and citing studies, separately for the title and abstract conclusions, through multinomial regression models.

RESULTS

Overall, 27% (n = 8) of source studies used unconditionally causal statements in their title, 30% (n = 9) in their abstract conclusion, and 17% (n = 5) in both. Only 20% (n = 6) used appropriate statements in their title and in their abstract's conclusions. Among the 622 citing studies analyzed, 285 (45.8%) used unconditionally causal statements when referring to the findings from disproportionality analysis, and only 164 (26.4%) used appropriate language. Multinomial models found that the strength of causal statements in citing studies was positively associated with the strength of causal language used in abstract conclusions of source articles (Likelihood Ratio Test (LogLRT) p < 0.00001) but not in the titles. In particular, among studies citing source articles with appropriate interpretation, 30.2% (95% confidence interval [CI] 22.8-37.6) contained unconditionally causal statements in their abstract conclusions, versus 56.4% (95% CI 48.7-64.2) for studies citing source articles with unconditionally causal statements.

CONCLUSIONS

Nearly half of the studies citing pharmacovigilance disproportionality analyses results used causal claims, particularly when the causal language used in the source article was stronger. There is a need for higher caution when writing, interpreting, and citing disproportionality studies.

Evaluation of Spin in Clinical Trials of Mesenchymal Stromal Cells for the Treatment of Knee Osteoarthritis: A Systematic Review

BACKGROUND

The regenerative potential of mesenchymal stromal cells (MSCs) has sparked interest in their use for knee osteoarthritis. Concurrently, there have been investigations on how data in scientific journals are reported and how they may influence readers' interpretations, or "spin bias." These studies are at risk for bias, given the limited number of patients and inconsistent blinding or controls. The risk of spin bias also complicates the interpretation, as results may be presented in a way that favors a particular outcome.

PURPOSE

To quantify and characterize spin bias in clinical trials of MSCs for knee osteoarthritis.

STUDY DESIGN

Systematic review.

METHODS

PubMed and Embase searches were conducted using the terms "mesenchymal stem cells" or "MSCs" and "knee arthritis" or "osteoarthritis" and "therapy" or "treatment" or "regeneration." Overall, 2 independent reviewers classified spin as high, moderate, or low and as 1 of 3 types: (1) emphasizing statistically significant results, (2) interpreting nonsignificant results as treatment effectiveness, and (3) claiming treatment benefits despite nonsignificant results. Journals were categorized as orthopaedic or nonorthopaedic. Descriptive statistics, the chi-square test, and the Fisher exact test were used to analyze the data, with alpha set at P < .05.

RESULTS

Among the 54 studies, spin was found in 80.0% of articles, with 14.5% having high, 25.5% moderate, and 40.0% low levels of spin. Type 1 was found in 54.5% of articles, type 2 in 18.2%, and type 3 in 29.1%. Spin was less frequently observed in the Methods section of articles compared with the abstract (17.52; P = .003). Reports on adipose-derived MSCs were associated with a higher frequency and level of spin compared with reports on MSCs from other sources (18.92; P = .026). There was no difference in the frequency of spin between orthopaedic and nonorthopaedic journals (0.48; P = .49) and no association with the impact factor (5.34; P = .07). There was no association between spin and financial disclosures (0.02; P = .577).

CONCLUSION

Spin bias was present in most MSC-related trials for knee osteoarthritis, with a higher frequency among those that utilized adipose-derived MSCs. Understanding the prevalence and strategies of spin can mitigate any potential misinterpretations of study outcomes.

The mockery that confounds better treatment of confounding in epidemiology: The change in estimate fallacy

Confounding is one of the most infamous bugbears of epidemiology, used by some to dismiss the field's utility outright. The subject has received considerable attention from epidemiologists and the field boasts a remarkable arsenal for addressing the issue. However, it appears that there are still misconceptions about how to identify variables that cause confounding (a lack of exchangeability) in epidemiologic practice. In this commentary, I examine whether analysis of the properties of change-in-estimate method for identification of confounding, exemplified by two highly cited papers, has been appropriately cited in published reports and whether it was utilized to improve epidemiologic practice. I conclude that the myth that a change-in-estimate criterion of 10 % is legitimate for identifying confounding persists in epidemiological practice, despite having been discredited by several independent research groups decades ago. Speculations on possible solutions to this problem are offered, but my work's main contribution is identification of a problem of how methodological advances in epidemiology may be misapplied. There currently do not exist any universal criteria for identification of confounding! "Citation without representation" or biased presentation of conclusions of methodological research may be pervasive.

Public engagement with research reproducibility

Public engagement with reproducibility is crucial for fostering trust in science. This Community Page outlines, through the example of baking Christmas tree meringues, how scientists can effectively engage and educate the public about the importance of reproducibility in research.

A cross-sectional study assessing visual abstracts of randomized trials revealed inadequate reporting and high prevalence of spin

OBJECTIVES

Visual abstracts (VAs) lack study-specific reporting guidelines and are increasingly used as stand-alone sources in medical research dissemination although not designed for this purpose. Therefore, our objectives were to describe 1) completeness of reporting in VAs and corresponding written abstracts (WAs) of randomized controlled trials (RCTs), and 2) the extent and type of spin (ie, any reporting pattern that could distort result interpretation and mislead readers) in VAs and WAs of RCTs with a statistically nonsignificant primary outcome.

STUDY DESIGN AND SETTING

We conducted a cross-sectional study evaluating VAs and WAs of RCTs published between January 1, 2021, and March 3, 2023. We searched MEDLINE via PubMed for reports of RCTs published in the 15 highest impact factor journals from six medical fields (among which 34 journals producing VAs of RCTs were identified). One reviewer identified primary reports of RCTs published with a VA and randomly selected a maximum of 10 reports from each journal to avoid overrepresentation. The completeness of reporting assessment was based on the Consolidated Standards of Reporting Trials extension for abstracts. Spin was explored using a standardized spin classification for RCTs with statistically nonsignificant primary outcome results. Both assessments were conducted in duplicate, with discussion until consensus in case of discrepancy.

RESULTS

A random sample of 253 reports from 34 journals was identified. The information provided in VAs was frequently incomplete: primary outcome identification, primary outcome results, and harms were respectively described or displayed in only 47% (n = 116/247), 30% (n = 75/247), and 35% (n = 88/253). Reporting was slightly better for some items in WAs, although still unsatisfactory. Among trials with nonsignificant primary outcome results (n = 101), 57% (n = 58) of the VAs and 55% (n = 56) of the WAs exhibited at least 1 type of spin. Posthoc analyses showed VAs produced by journal editors of high-impact general medical journals were more complete and more accurate than those produced by specialty journals or authors.

CONCLUSION

The information conveyed in VAs was frequently incomplete and inaccurate, highlighting the urgent need to refer to appropriate specific reporting guidelines to avoid misinterpretation by readers.

Several methods for assessing research waste in reviews with a systematic search: a scoping review

Background

Research waste is present in all study designs and can have significant consequences for science, including reducing the reliability of research findings and contributing to the inefficient use of resources. Estimates suggest that as much as 85% of all biomedical research is wasted. However, it is uncertain how avoidable research waste is assessed in specific types of study designs and what methods could be used to examine different aspects of research waste. We aimed to investigate which methods, systematic reviews, scoping reviews, and overviews of reviews discussing research waste, have used to assess avoidable research waste.

Materials and Methods

We published a protocol in the Open Science Framework prospectively (https://osf.io/2fbp4). We searched PubMed and Embase with a 30-year limit (January 1993-August 2023). The concept examined was how research waste and related synonyms (e.g., unnecessary, redundant, duplicate, etc.) were assessed in reviews with a systematic search: systematic, scoping, or overviews of reviews. We extracted data on the method used in the review to examine for research waste and for which study design this method was applied.

Results

The search identified 4,285 records of which 93 reviews with systematic searches were included. The reviews examined a median of 90 (range 10-6,781) studies, where the study designs most commonly included were randomized controlled trials (48%) and systematic reviews (33%). In the last ten years, the number of reports assessing research waste has increased. More than 50% of examined reviews reported evaluating methodological research waste among included studies, typically using tools such as one of Cochrane Risk of Bias tools (n = 8) for randomized controlled trials or AMSTAR 1 or 2 (n = 12) for systematic reviews. One fourth of reviews assessed reporting guideline adherence to e.g., CONSORT (n = 4) for randomized controlled trials or PRISMA (n = 6) for systematic reviews.

Conclusion

Reviews with systematic searches focus on methodological quality and reporting guideline adherence when examining research waste. However, this scoping review revealed that a wide range of tools are used, which may pose difficulties in comparing examinations and performing meta-research. This review aids researchers in selecting methodologies and contributes to the ongoing discourse on optimizing research efficiency.

Common Critiques and Recommendations for Studies in Neurology Using Machine Learning Methods

Machine learning (ML) methods are becoming more prevalent in the neurology literature as alternatives to traditional statistical methods to address challenges in the analysis of modern data sets. Despite the increase in the popularity of ML methods in neurology studies, some authors do not fully address all items recommended in reporting guidelines. The authors of this Research Methods article are members of the Neurology® editorial board and have reviewed many studies using ML methods. In their review reports, several critiques often appear, which could be avoided if guidance were available. In this article, we detail common critiques found in ML research studies and make recommendations for how to avoid them. The first critique involves misalignment of the study goals and the analysis conducted. The second critique focuses on ML terminology being appropriately used. Critiques 3-6 are related to the study design: justifying sample sizes and the suitability of the data set for the study goals, describing the ML analysis pipeline sufficiently, quantifying the amount of missing data and providing information about missing data handling, and including uncertainty estimates for key metrics. The seventh critique focuses on fairly describing both strengths and limitations of the ML study, including the analysis methodology and results. We provide examples in neurology for each critique and guidance on how to avoid the critique. Overall, we recommend that authors use ML-specific checklists developed by research consortia for designing and reporting studies using ML. We also recommend that authors involve both a statistician and an ML expert in work that uses ML. Although our list of critiques is not exhaustive, our recommendations should help improve the quality and rigor of ML studies. ML has great potential to revolutionize neurology, but investigators need to conduct and report the results in a way that allows readers to fully evaluate the benefits and limitations of ML approaches.

Reevaluating ADHD and its First-Line Treatment: Insights from DSM-5-TR and Modern Approaches

Is Attention Deficit Hyperactivity Disorder (ADHD) a "brain disorder"? Should it be managed regularly with stimulant drugs? This article critically examines the evolving biomedical discourse surrounding these questions through a close inspection of the latest edition of the influential psychiatric manual - the DSM-5-TR - as well as additional authoritative sources (e.g., previous DSM editions, consensus statements, FDA communications). The DSM-5-TR acknowledges that "no biological marker is diagnostic for ADHD" and that "meta-analyses of all neuroimaging studies do not show differences between individuals with ADHD and control subjects." The authors of the DSM-5-TR, therefore, conclude that "until these issues are resolved, no form of neuroimaging can be used for diagnosis of ADHD." These statements, along with biases in the neuroimaging literature and additional empirical evidence presented in the article, challenge popular myths about the neurobiological basis of ADHD. Similarly, common beliefs about the first-line treatment of ADHD with stimulant drugs are being increasingly questioned today. For instance, the DSM-5-TR's section on Stimulant-Related Disorders introduces a new diagnostic entity named: Stimulant-Induced Mild Neurocognitive Disorder. This addition aligns with a recent FDA Drug Safety Communication for "all prescription stimulants," which highlights longstanding concerns regarding the safety of medications prescribed to millions of diagnosed individuals, primarily children. The FDA now mandates that "the Boxed Warning, FDA's most prominent warning, will describe the risks of misuse, abuse, addiction, and overdose," emphasizing that such "misuse and abuse of prescription stimulants can result in overdose and death." In light of these challenges to the biomedical discourse, this article offers a neurodiversity-oriented alternative. Using evolutionary principles and historical context, it argues that most cases of ADHD fall under the DSM's socio-philosophical category of "conflicts that are primarily between the individual and society" (similar to homosexuality, which was removed from the DSM in 1973), and are therefore "not mental disorders".

Most Systematic Reviews and Meta-analyses Reporting Clinical Outcomes of the Remplissage Procedure Have at Least 1 Form of Spin

Purpose

To determine the prevalence of spin in systematic reviews (SRs) and meta-analyses of clinical studies of the remplissage procedure.

Methods

Two reviewers independently performed a literature search of the PubMed, Scopus, and Embase databases using the search term "remplissage" in accordance with Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines. The full article of each included SR was assessed for the presence of the 15 most common types of spin. Methodologic quality was assessed using the second version of A Measurement Tool to Assess Systematic Reviews (AMSTAR 2).

Results

A total of 15 SRs (8 accompanied by meta-analyses; 6 Level III and 9 Level IV) were included. Overall, 13 SRs (86.7%) contained at least 1 form of spin, with 33 unique instances of spin recorded; the mean frequency was 2.2 ± 1.3 (range, 0-4). The most prevalent form of spin, present in 11 studies (73%), was type 9 ("conclusion claims the beneficial effect of the experimental treatment despite reporting bias"). There were 14 uses of spin classified as misleading reporting, 16 classified as misleading interpretation, and 3 classified as inappropriate extrapolation. The mean 5-year impact factor of the publishing journals was 4.4 ± 0.9 (range, 0-6.1), the mean number of citations per SR was 33.3 ± 24.9 (range, 0-55), and the mean number of citations per month since publication was 0.68 ± 0.44 (range, 0-1.48). According to the AMSTAR 2 assessment, confidence in the results of the SRs was rated as critically low for 20% of reviews, low for 33.3%, and moderate for 46.7%.

Conclusions

Most SRs of the remplissage procedure are affected by the presence of spin. Favorable reporting was observed in the absence of definite findings, as was minimization of drawbacks for certain populations.

Level of Evidence

Level IV, systematic review of Level III and IV studies.

Promotional language in radiology publications: increasing use of "excellent", "favorable", "promising", "robust", and "unique"

PURPOSE

To investigate if radiology researchers are increasingly promoting their scientific findings by more frequently using positive words in their publications.

MATERIALS AND METHODS

This study included all articles that were published in 14 general radiology journals between 2003 and 2022. The title and abstract of each article were assessed for the presence of positive, negative, neutral, and random words, according to predefined sets of words for each category. Usage of positive, negative, neutral, and random words was calculated for each year and corrected for the total number of articles in each year. Temporal trends between 2002 and 2023 and the relationship between positive word usage and journal impact factor (IF) were assessed.

RESULTS

Positive word usage (Mann-Kendall tau of 0.895, P< 0.001) and neutral word usage (Mann-Kendall tau of 0.463, P = 0.005) showed significant upward temporal trends. Negative word usage and random word usage did not show any significant temporal trends. Five positive words showed significantly increased usage over time and were present in more than 1 % of titles/abstracts in at least one year: "excellent" (Mann-Kendall tau of 0.800, P< 0.001), "favorable" (Mann-Kendall tau of 0.547, P< 0.001), "promising" (Mann-Kendall tau of 0.607, P< 0.001), "robust" (Mann-Kendall tau of 0.737, P< 0.001), and "unique" (Mann-Kendall tau of 0.747, P< 0.001). There was no significant association between positive word usage and journal IF.

CONCLUSION

Radiology researchers appear to increasingly promote their scientific findings by more frequently using positive words in their publications over the past two decades.

KEY POINTS

· Positive word usage in titles/abstracts has strongly increased between 2003-2022. · "Excellent", "favorable", "promising", "robust", and "unique" were most often used. · This trend occurred in all general radiology journals, regardless of impact factor.

CITATION FORMAT

· Kwee T, Kwee R. Promotional language in radiology publications: increasing use of "excellent", "favorable", "promising", "robust", and "unique". Fortschr Röntgenstr 2024; 196: 945 - 955.

Promoting Data Sharing: The Moral Obligations of Public Funding Agencies

Sharing research data has great potential to benefit science and society. However, data sharing is still not common practice. Since public research funding agencies have a particular impact on research and researchers, the question arises: Are public funding agencies morally obligated to promote data sharing? We argue from a research ethics perspective that public funding agencies have several pro tanto obligations requiring them to promote data sharing. However, there are also pro tanto obligations that speak against promoting data sharing in general as well as with regard to particular instruments of such promotion. We examine and weigh these obligations and conclude that all things considered funders ought to promote the sharing of data. Even the instrument of mandatory data sharing policies can be justified under certain conditions.

Development and Evaluation of a Framework for Identifying and Addressing Spin for Harms in Systematic Reviews of Interventions

"Spin" refers to misleading reporting, interpretation, and extrapolation of findings in primary and secondary research (such as in systematic reviews). The study of spin primarily focuses on beneficial outcomes. The objectives of this research were threefold: first, to develop a framework for identifying spin associated with harms in systematic reviews of interventions; second, to apply the framework to a set of reviews, thereby pinpointing instances where spin may be present; and finally, to revise the spin examples, offering guidance on how spin can be rectified.

The authors developed their framework through an iterative process that engaged an international group of researchers specializing in spin and reporting bias. The framework comprises 12 specific types of spin for harms, grouped by 7 categories across the 3 domains (reporting, interpretation, and extrapolation). The authors subsequently gathered instances of spin from a random sample of 100 systematic reviews of interventions. Of the 58 reviews that assessed harm and the 42 that did not, they found that 28 (48%) and 6 (14%), respectively, had at least 1 of the 12 types of spin for harms. Inappropriate extrapolation of the results and conclusions for harms to populations, interventions, outcomes, or settings not assessed in a review was the most common category of spin in 17 of 100 reviews.

The authors revised the examples to remove spin, taking into consideration the context (for example, medical discipline, source population), findings for harms, and methodological limitations of the original reviews. They provide guidance for authors, peer reviewers, and editors in recognizing and rectifying or (preferably) avoiding spin, ultimately enhancing the clarity and accuracy of harms reporting in systematic review publications.

Spin in dental publications: a scoping review

The aim of this review was to map the practice of spin in scientific publications in the dental field. After registering the review protocol (osf.io/kw5qv/), a search was conducted in MEDLINE via PubMed, CENTRAL, Embase, Scopus, LILACS, ClinicalTrials.gov, and OpenGrey databases in June 2023. Any study that evaluated the presence of spin in dentistry was eligible. Data were independently extracted in duplicate by two reviewers. After removing duplicates, 4888 records were screened and 38 were selected for full-text review. Thirteen studies met the eligibility criteria, all of which detected the presence of spin in the primary studies, with the prevalence of spin ranging from 30% to 86%. The most common types of spin assessed in systematic reviews were failure to mention adverse effects of interventions and to report the number of studies/patients contributing to the meta-analysis of main outcomes. In randomized controlled trials, there was a focus on statistically significant within-group and between-group comparisons for primary or secondary outcomes (in abstract results) and claiming equivalence/noninferiority/similarity for statistically nonsignificant results (in abstract conclusions). The practice of spin is widespread in dental scientific literature among different specialties, journals, and countries. Its impact, however, remains poorly investigated.

Exploring decision-makers' challenges and strategies when selecting multiple systematic reviews: insights for AI decision support tools in healthcare

BACKGROUND

Systematic reviews (SRs) are being published at an accelerated rate. Decision-makers may struggle with comparing and choosing between multiple SRs on the same topic. We aimed to understand how healthcare decision-makers (eg, practitioners, policymakers, researchers) use SRs to inform decision-making and to explore the potential role of a proposed artificial intelligence (AI) tool to assist in critical appraisal and choosing among SRs.

METHODS

We developed a survey with 21 open and closed questions. We followed a knowledge translation plan to disseminate the survey through social media and professional networks.

RESULTS

Our survey response rate was lower than expected (7.9% of distributed emails). Of the 684 respondents, 58.2% identified as researchers, 37.1% as practitioners, 19.2% as students and 13.5% as policymakers. Respondents frequently sought out SRs (97.1%) as a source of evidence to inform decision-making. They frequently (97.9%) found more than one SR on a given topic of interest to them. Just over half (50.8%) struggled to choose the most trustworthy SR among multiple. These difficulties related to lack of time (55.2%), or difficulties comparing due to varying methodological quality of SRs (54.2%), differences in results and conclusions (49.7%) or variation in the included studies (44.6%). Respondents compared SRs based on the relevance to their question of interest, methodological quality, and recency of the SR search. Most respondents (87.0%) were interested in an AI tool to help appraise and compare SRs.

CONCLUSIONS

Given the identified barriers of using SR evidence, an AI tool to facilitate comparison of the relevance of SRs, the search and methodological quality, could help users efficiently choose among SRs and make healthcare decisions.

Evaluation of spin in systematic reviews on the use of tendon transfer for massive irreparable rotator cuff tears

PURPOSE

To identify, describe and account for the incidence of spin in systematic reviews and meta-analyses of tendon transfer for the treatment of massive, irreparable rotator cuff tears. The secondary objective was to characterize the studies in which spin was identified and to determine whether identifiable patterns exist among studies with spin.

METHODS

This study was conducted per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Each abstract was assessed for the presence of the 15 most common types of spin derived from a previously established methodology. General data that were extracted included study title, authors, publication year, journal, level of evidence, study design, funding source, reported adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, preregistration of the study protocol, and methodologic quality per A Measurement Tool to Assess Systematic Reviews Version 2 (AMSTAR 2).

RESULTS

The search yielded 53 articles, of which 13 were included in the final analysis. Articles were excluded if they were not published in a peer reviewed journal, not written in English, utilized cadaveric or nonhuman models, or lacked an abstract with accessible full text. 53.8% (7/13) of the included studies contained at least 1 type of spin in the abstract. Type 5 spin ("The conclusion claims beneficial effect of the experimental treatment despite a high risk of bias in primary studies") was the most common, appearing in 23.1% (3/13) of included abstracts. Nine of the spin categories did not appear in any of the included abstracts. A lower AMSTAR 2 score was significantly associated with the presence of spin in the abstract (P < .006).

CONCLUSION

Spin is highly prevalent in the abstracts of systematic reviews and meta-analyses concerning tendon transfer for massive rotator cuff tears. A lower overall AMSTAR 2 rating was associated with a higher incidence of spin. Future studies should continue to explore the prevalence of spin in orthopedic literature and identify any factors that may contribute to its presence.

Promotional language and the adoption of innovative ideas in science

How are the merits of innovative ideas communicated in science? Here, we conduct semantic analyses of grant application success with a focus on scientific promotional language, which may help to convey an innovative idea's originality and significance. Our analysis attempts to surmount the limitations of prior grant studies by examining the full text of tens of thousands of both funded and unfunded grants from three leading public and private funding agencies: the NIH, the NSF, and the Novo Nordisk Foundation, one of the world's largest private science funding foundations. We find a robust association between promotional language and the support and adoption of innovative ideas by funders and other scientists. First, a grant proposal's percentage of promotional language is associated with up to a doubling of the grant's probability of being funded. Second, a grant's promotional language reflects its intrinsic innovativeness. Third, the percentage of promotional language is predictive of the expected citation and productivity impact of publications that are supported by funded grants. Finally, a computer-assisted experiment that manipulates the promotional language in our data demonstrates how promotional language can communicate the merit of ideas through cognitive activation. With the incidence of promotional language in science steeply rising, and the pivotal role of grants in converting promising and aspirational ideas into solutions, our analysis provides empirical evidence that promotional language is associated with effectively communicating the merits of innovative scientific ideas.

Five aspects of research waste in biomedicine: A scoping review

BACKGROUND

The number of published journal articles has grown exponentially during the last 30 years, which may have led to some wasteful research. However, the terminology associated with research waste remains unclear. To address this, we aimed to identify, define, and categorize the aspects of research waste in published biomedical reports.

METHODS

In this scoping review, we systematically searched for biomedical literature reports from 1993 to 2023 in two databases, focusing on those addressing and defining research waste. Through data charting, we analyzed and categorized the aspects of research waste.

RESULTS

Based on 4285 initial records in the searches, a total of 832 reports were included in the analysis. The included reports were primarily narrative reviews (26%) and original reports (21%). We categorized research waste into five aspects: methodological, invisible, negligible, underreported, and structural (MINUS) research waste. More than half of the reports (56%) covered methodological research waste concerning flaws in study design, study conduct, or analysis. Invisible research waste covered nonpublication, discontinuation, and lack of data-sharing. Negligible research waste primarily concerned unnecessary repetition, for example, stemming from the absence of preceding a trial with a systematic review of the literature. Underreported research waste mainly included poor reporting, resulting in a lack of transparency. Structural research waste comprised inadequate management, collaboration, prioritization, implementation, and dissemination.

CONCLUSION

MINUS encapsulates the five main aspects of research waste. Recognizing these aspects of research waste is important for addressing and preventing further research waste and thereby ensuring efficient resource allocation and scientific integrity.

SPIN-PM: a consensus framework to evaluate the presence of spin in studies on prediction models

OBJECTIVES

To develop a framework to identify and evaluate spin practices and its facilitators in studies on clinical prediction model regardless of the modeling technique.

STUDY DESIGN AND SETTING

We followed a three-phase consensus process: (1) premeeting literature review to generate items to be included; (2) a series of structured meetings to provide comments discussed and exchanged viewpoints on items to be included with a panel of experienced researchers; and (3) postmeeting review on final list of items and examples to be included. Through this iterative consensus process, a framework was derived after all panel's researchers agreed.

RESULTS

This consensus process involved a panel of eight researchers and resulted in SPIN-Prediction Models which consists of two categories of spin (misleading interpretation and misleading transportability), and within these categories, two forms of spin (spin practices and facilitators of spin). We provide criteria and examples.

CONCLUSION

We proposed this guidance aiming to facilitate not only the accurate reporting but also an accurate interpretation and extrapolation of clinical prediction models which will likely improve the reporting quality of subsequent research, as well as reduce research waste.

The REporting of A Disproportionality Analysis for DrUg Safety Signal Detection Using Individual Case Safety Reports in PharmacoVigilance (READUS-PV): Explanation and Elaboration

In pharmacovigilance, disproportionality analyses based on individual case safety reports are widely used to detect safety signals. Unfortunately, publishing disproportionality analyses lacks specific guidelines, often leading to incomplete and ambiguous reporting, and carries the risk of incorrect conclusions when data are not placed in the correct context. The REporting of A Disproportionality analysis for drUg Safety signal detection using individual case safety reports in PharmacoVigilance (READUS-PV) statement was developed to address this issue by promoting transparent and comprehensive reporting of disproportionality studies. While the statement paper explains in greater detail the procedure followed to develop these guidelines, with this explanation paper we present the 14 items retained for READUS-PV guidelines, together with an in-depth explanation of their rationale and bullet points to illustrate their practical implementation. Our primary objective is to foster the adoption of the READUS-PV guidelines among authors, editors, peer reviewers, and readers of disproportionality analyses. Enhancing transparency, completeness, and accuracy of reporting, as well as proper interpretation of their results, READUS-PV guidelines will ultimately facilitate evidence-based decision making in pharmacovigilance.

The Reporting of a Disproportionality Analysis for Drug Safety Signal Detection Using Individual Case Safety Reports in PharmacoVigilance (READUS-PV): Development and Statement

BACKGROUND AND AIM

Disproportionality analyses using reports of suspected adverse drug reactions are the most commonly used quantitative methods for detecting safety signals in pharmacovigilance. However, their methods and results are generally poorly reported in published articles and existing guidelines do not capture the specific features of disproportionality analyses. We here describe the development of a guideline (REporting of A Disproportionality analysis for drUg Safety signal detection using individual case safety reports in PharmacoVigilance [READUS-PV]) for reporting the results of disproportionality analyses in articles and abstracts.

METHODS

We established a group of 34 international experts from universities, the pharmaceutical industry, and regulatory agencies, with expertise in pharmacovigilance, disproportionality analyses, and assessment of safety signals. We followed a three-step process to develop the checklist: (1) an open-text survey to generate a first list of items; (2) an online Delphi method to select and rephrase the most important items; (3) a final online consensus meeting.

RESULTS

Among the panel members, 33 experts responded to round 1 and 30 to round 2 of the Delphi and 25 participated to the consensus meeting. Overall, 60 recommendations for the main body of the manuscript and 13 recommendations for the abstracts were retained by participants after the Delphi method. After merging of some items together and the online consensus meeting, the READUS-PV guidelines comprise a checklist of 32 recommendations, in 14 items, for the reporting of disproportionality analyses in the main body text and four items, comprising 12 recommendations, for abstracts.

CONCLUSIONS

The READUS-PV guidelines will support authors, editors, peer-reviewers, and users of disproportionality analyses using individual case safety report databases. Adopting these guidelines will lead to more transparent, comprehensive, and accurate reporting and interpretation of disproportionality analyses, facilitating the integration with other sources of evidence.

Decompression Alone vs Decompression and Fusion: Spin in Abstracts of Systematic Reviews and Meta-Analysis

STUDY DESIGN

Cross-sectional.

OBJECTIVES

Spin, a prevalent bias, can distort outcomes in well-validated research. Treatment of lumbar stenosis with spondylolisthesis through decompression alone (DA) vs decompression and fusion (DF) remains a controversial topic. We aimed to identify the prevalence of spin in meta-analyses and systematic reviews regarding DA vs DF in the treatment of spinal stenosis with concomitant degenerative spondylolisthesis.

METHODS

MEDLINE, Web of Science, and Google Scholar were searched for meta-analyses and systematic reviews comparing DA vs DF treatment of lumbar spinal stenosis accompanied by degenerative spondylolisthesis. Each study was evaluated for the nine most severe spin types. We also explored the association between spin and methodological quality of a systematic review using the revised A Measurement Tool to Assess Systematic Reviews appraisal tool.

RESULTS

The search yielded 1506 articles, of which 13 met inclusion. It was found that 46.2% (6/13) of the articles contained spin within the abstract. Of the nine most severe types, type 5 was found to be the most prevalent (4/13, 30.8%), followed by types 9 (2/13, 15.4%), 3 (1/13, 7.7%), 4 (1/13, 7.7%), and 6 (1/13, 7.7%). Spin types 1, 2, 7 and 8 were not found. According to AMSTAR-2, 53.8% (7/13) of the studies were appraised as "critically low" quality and 46.2% (6/13) as "low" quality.

CONCLUSION

This study demonstrated the presence of spin in 46.2% of abstracts pertaining to the treatment for spinal stenosis with degenerative spondylolisthesis. Spine surgeons should learn to recognize spin as they review articles before implementing them into practice.

Reporting of interventional clinical trial results in an academic center: a survey of completed studies

BACKGROUND

The dissemination of clinical trial results is an important scientific and ethical endeavour. This survey of completed interventional studies in a French academic center describes their reporting status.

METHODS

We explored all interventional studies sponsored by Rennes University Hospital identified on the French Open Science Monitor which tracks trials registered on EUCTR or clinicaltrials.gov, and provides an automatic assessment of the reporting of results. For each study, we ascertained the actual reporting of results using systematic searches on the hospital internal database, bibliographic databases (Google Scholar, PubMed), and by contacting all principal investigators (PIs). We describe several features (including total budget and numbers of trial participants) of the studies that did not report any results.

RESULTS

The French Open Science Monitor identified 93 interventional studies, among which 10 (11%) reported results. In contrast, our survey identified 36 studies (39%) reporting primary analysis results and an additional 18 (19%) reporting results for secondary analyses (without results for their primary analysis). The overall budget for studies that did not report any results was estimated to be €5,051,253 for a total of 6,735 trial participants. The most frequent reasons for the absence of results reported by PIs were lack of time for 18 (42%), and logistic difficulties (e.g. delay in obtaining results or another blocking factor) for 12 (28%). An association was found between non-publication and negative results (adjusted Odds Ratio = 4.70, 95% Confidence Interval [1.67;14.11]).

CONCLUSIONS

Even allowing for the fact that automatic searches underestimate the number of studies with published results, the level of reporting was disappointingly low. This amounts to a waste of trial participants' implication and money. Corrective actions are needed.

TRIAL REGISTRATION

https://osf.io/q5hcs.

Alpha-blockers: the magic pill for endourology-The great delusion

PURPOSE

The present paper takes a different and more critical look at the role of alpha-blockers, sometimes nicknamed as "magical pills", in particular for stone disease and medical expulsive therapy (MET).

METHODS

A non-systematic narrative review was performed, synthesizing pertinent information from selected articles, and critically evaluating their conclusions. Sometimes different views on alpha-blockers were laid bare, including curiosities or other entertaining nuances suitable to the present topic, but always maintaining sharp objectivity and the foremost scientific rigor.

RESULTS AND CONCLUSIONS

Alpha-blockers seem to be a panacea, being used to treat a wide variety of non-urological diseases and conditions. Urological applications include erectile dysfunction to benign prostatic hyperplasia, from incontinence to urinary retention, or even to facilitate urinary stone passage along the urinary tract. Due to its versatility, alpha-blockers appear to be the Swiss army knife of urological medications. However, the efficacy of alpha-blockers for MET, pain management, or facilitating upper tract access is very disappointing, bringing no, or in some instances, only marginal benefits. Their treatment results are far from being significant or impressive let alone magical. Regular sexual intercourse is an effective alternative to alpha-blockers, providing faster ureteral stone expulsion rates and reducing the need for pain medication. Most of the research supporting alpha-blockers has been based on single-center, underpowered, low-quality studies. These low-quality studies biased several subsequent meta-analyses, contaminating them with their low-quality data, enhancing and prolonging this delusion. These results emphasize the need for large, multi-centric, unbiased, randomized, double-blinded, placebo-controlled trials to prevent future year-long delusions that may afflict any medical field.

On hidden factors and design-associated errors that may lead to data misinterpretation: An example from preclinical research on the potential seasonality of neonatal seizures

Unintentional misinterpretation of research in published biomedical reports that is not based on statistical flaws is often underrecognized, despite its possible impact on science, clinical practice, and public health. Important causes of such misinterpretation of scientific data, resulting in either false positive or false negative conclusions, include design-associated errors and hidden (or latent) variables that are not easily recognized during data analysis. Furthermore, cognitive biases, such as the inclination to seek patterns in data whether they exist or not, may lead to misinterpretation of data. Here, we give an example of these problems from hypothesis-driven research on the potential seasonality of neonatal seizures in a rat model of birth asphyxia. This commentary aims to raise awareness among the general scientific audience about the issues related to the presence of unintentional misinterpretation in published reports.

Evaluation of spin in reviews of biodegradable balloon spacers for massive irreparable rotator cuff tears

HYPOTHESIS

Clinical studies are often at risk of spin, a form of bias where beneficial claims are overstated while negative findings are minimized or dismissed. Spin is often more problematic in abstracts given their brevity and can result in the misrepresentation of a study's actual findings. The goal of this study is to aggregate primary and secondary studies reporting the clinical outcomes of the use of subacromial balloon spacers in the treatment of massive irreparable rotator cuff tears to identify the incidence of spin and find any significant association with study design parameters.

MATERIALS AND METHODS

This study was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Independent searches were completed on 2 databases (PubMed and Embase) for primary studies, systematic and current concepts reviews, and meta-analyses and the results were compiled. Two authors independently screened the studies using a predetermined inclusion criteria and aggregated data including titles, publication journals and years, authors, study design, etc. Each study was independently assessed for the presence of 15 different types of spin. Statistical analysis was conducted to identify associations between study characteristics and spin.

RESULTS

Twenty-nine studies met the inclusion criteria for our analysis, of which 10 were reviews or meta-analyses and 19 were primary studies. Spin was identified in every study except for 2 (27/29, 93.1%). Type 3 spin, "Selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention" and type 9 spin, "Conclusion claims the beneficial effect of the experimental treatment despite reporting bias" were most frequently noted in our study, both observed in 12/29 studies (41.4%). Date of publication, and adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses or "The International Prospective Register of Systematic Reviews" were study characteristics associated with a higher rate of certain types of spin. There was a statistically significant association between disclosure of external study funding source and the presence of spin type 4, but none of the other forms of spin.

CONCLUSION

Spin is highly prevalent in the abstracts of primary studies, systematic reviews, and meta-analyses discussing the use of subacromial balloon spacer technology in the treatment of massive irreparable rotator cuff tears. Our findings revealed that spin in the abstract tended to favor the balloon spacer intervention. Further efforts are required in the future to mitigate spin within the abstracts of published manuscripts.

The risk of normative bias in reporting empirical research: lessons learned from prenatal screening studies about the prominence of acknowledged limitations

Empirical data can be an extremely powerful and influential tool in bioethical research. However, when researchers or policy makers look for answers to ethical questions by engaging with empirical research, there can be a tendency (conscious or unconscious) to shape, report, and use empirical research in a way that confirms their own preferred ethical conclusions. This skewing effect - what we call 'normative bias' - is often so subtle it falls short of clear misconduct and thus can be difficult to call out. However, we argue that this subtle influence of bias has the potential to significantly influence debate and policy around highly sensitive ethical issues and must be guarded against. In this paper we share the lessons we have learned through a journey of self-reflection around the effect that normative bias can have when reporting on and referring to empirical data relating to ethical issues. We use a variety of papers from our area of the ethics of routine prenatal screening to illustrate these subtle but often powerfully distorting effects of bias. Our aim in doing so is not to criticise the work of others, as we recognise our own normative bias, but to improve awareness of this issue, remind the need for reflexivity to guard against our own biases, and introduce a new criterion - the idea of a 'limitation prominence assessment' - that can work as a practical way to evaluate the seriousness of the limitations of an empirical study and thus, the risks of the study being misread or misinterpreted through superficial reading.

Response to Medical Assistance in Dying, Palliative Care, Safety, and Structural Vulnerability

This report, signed by >170 scholars, clinicians, and researchers in palliative care and related fields, refutes the claims made by the previously published Medical Assistance in Dying, Palliative Care, Safety, and Structural Vulnerability. That report attempted to argue that structural vulnerability was not a concern in the provision of assisted dying (AD) by a selective review of evidence in medical literature and population studies. It claimed that palliative care has its own safety concerns, and that "misuse" of palliative care led to reports of wrongful death. We and our signatories do not feel that the conclusions reached are supported by the evidence provided in the contested report. The latter concluded that the logical policy response would be to address the root causes of structural vulnerability rather than restrict access to AD. Our report, endorsed by an international community of palliative care professionals, believes that public policy should aim to reduce structural vulnerability and, at the same time, respond to evidence-based cautions about AD given the potential harm.

Peer Review in JAACAP Open: Promoting Integrity While Advancing Open Science

A critical piece in the launch of JAACAP Open is the establishment of a high-quality and robust peer review process for incoming submissions. Indeed, peer review is the backbone of our scientific process. Here, we will discuss the importance of peer review, describe the process as we are expanding the JAACAP journal family, and explain why and how you can be involved in the peer review process.

Presentation and publication skills: Publication governance and pitfalls to avoid

There are several pitfalls in the publication process that researchers can fall victim to, and these can occur knowingly or unknowingly. Although some of these errors may have occurred in good faith, disregard of publication governance is a dangerous practice and could bring authors and their co-authors into disrepute. We highlight some of these potential pitfalls, acquaint the reader with some rules that need to be adhered to in research and publishing, and help the reader learn how to avoid tripping-up on the road to publication.

Selective and deceptive citation in the construction of dueling consensuses

The COVID-19 pandemic provides a unique opportunity to study science communication and, in particular, the transmission of consensus. In this study, we show how "science communicators," writ large to include both mainstream science journalists and practiced conspiracy theorists, transform scientific evidence into two dueling consensuses using the effectiveness of masks as a case study. We do this by compiling one of the largest, hand-coded citation datasets of cross-medium science communication, derived from 5 million Twitter posts of people discussing masks. We find that science communicators selectively uplift certain published works while denigrating others to create bodies of evidence that support and oppose masks, respectively. Anti-mask communicators in particular often use selective and deceptive quotation of scientific work and criticize opposing science more than pro-mask communicators. Our findings have implications for scientists, science communicators, and scientific publishers, whose systems of sharing (and correcting) knowledge are highly vulnerable to what we term adversarial science communication.

Inaccuracy in the Scientific Record and Open Postpublication Critique

There is growing evidence that the published psychological literature is marred by multiple errors and inaccuracies and often fails to reflect the changing nature of the knowledge base. At least four types of error are common-citation error, methodological error, statistical error, and interpretation error. In the face of the apparent inevitability of these inaccuracies, core scientific values such as openness and transparency require that correction mechanisms are readily available. In this article, I reviewed standard mechanisms in psychology journals and found them to have limitations. The effects of more widely enabling open postpublication critique in the same journal in addition to conventional peer review are considered. This mechanism is well established in medicine and the life sciences but rare in psychology and may assist psychological science to correct itself.

Limited online training opportunities exist for scholarly peer reviewers

OBJECTIVES

To create a comprehensive list of all openly available online trainings in scholarly peer review and to analyze their characteristics.

STUDY DESIGN AND SETTING

A systematic review of online training material in scholarly peer review openly accessible between 2012 and 2022. Training characteristics were presented in evidence tables and summarized narratively. A risk of bias tool was purpose-built for this study to evaluate the included training material as evidence-based.

RESULTS

Fourty-two training opportunities in manuscript peer review were identified, of which only twenty were openly accessible. Most were online modules (n = 12, 60%) with an estimated completion time of less than 1 hour (n = 13, 65%). Using our ad hoc risk of bias tool, four sources (20%) met our criteria of evidence-based.

CONCLUSION

Our comprehensive search of the literature identified 20 openly accessible online training materials in manuscript peer review. For such a crucial step in the dissemination of literature, a lack of training could potentially explain disparities in the quality of scholarly publishing.

Spin on adverse effects in abstracts of systematic reviews of orthodontic interventions: a cross-sectional study (part 2)

BACKGROUND

It is critical that abstracts of systematic reviews transparently report both the beneficial and adverse effects of interventions without misleading the readers. This cross-sectional study assessed whether adverse effects of interventions were reported or considered in abstracts of systematic reviews of orthodontic interventions and whether spin on adverse effects was identified when comparing the abstracts with what was sought and reported in these reviews.

METHODS

This cross-sectional study (part 2 of 2) used the same sample of 98 systematic reviews orthodontic interventions as used in part 1. Eligible reviews were retrieved from the Cochrane Database of Systematic Reviews and the 5 leading orthodontic journals between August 1 2009 and July 31 2021. Prevalence proportions were sought for 3 outcomes as defined in the published protocol. Univariable logistic regression models were built to explore associations between the presence of spin in the abstract and a series of predictors. Odds ratios (OR) 95% confidence intervals (95% CI) were used to quantify the strength of associations and their precision.

RESULTS

76.5% (75/98) of eligible reviews reported or considered (i.e., discussed, weighted etc.) potential adverse effects of orthodontic interventions in the abstract and the proportion of spin on adverse effects was 40.8% (40/98) in the abstract of these reviews. Misleading reporting was the predominant category of spin, i.e., 90% (36/40). Our explorative analyses found that compared to the Cochrane Database of Systematic Reviews all 5 orthodontic journals had similar odds of the presence of spin on adverse effects in abstracts of systematic reviews of orthodontic interventions. The odds of the presence of spin did not change over the sampled years (OR: 1.03, 95% CI: 0.9 to 1.16) and did not depend on the number of authors (OR: 0.93, 95% CI: 0.71 to 1.21), or on the type of orthodontic intervention (OR: 1.1, 95% CI: 0.45 to 2.67), or whether conflicts of interests were reported (OR: 0.74, 95% CI: 0.32 to 1.68).

CONCLUSION

End users of systematic reviews of orthodontic interventions have to be careful when interpreting results on adverse effects in the abstracts of these reviews, because they could be jeopardized by uncertainties such as not being reported and misleading reporting as a result of spin.

Dopamine-independent development and maintenance of mouse striatal medium spiny neuron dendritic spines

Striatal medium spiny neurons (MSNs) and striatal dopamine (DA) innervation are profoundly important for brain function such as motor control and cognition. A widely accepted theory posits that striatal DA loss causes (or leads to) MSN dendritic atrophy. However, examination of the literature indicates that the data from Parkinson's disease (PD) patients and animal PD models were contradictory among studies and hard to interpret. Here we have re-examined the potential effects of DA activity on MSN morphology or lack thereof. We found that in 15-day, 4- and 12-month old Pitx3 null mutant mice that have severe DA denervation in the dorsal striatum while having substantial residual DA innervation in the ventral striatum, MSN dendrites and spine numbers were similar in dorsal and ventral striatum, and also similar to those in normal mice. In 15-day, 4- and 12-month old tyrosine hydroxylase knockout mice that cannot synthesize L-dopa and thus have no endogenous DA in the entire brain, MSN dendrites and spine numbers were also indistinguishable from age-matched wild-type (WT) mice. Furthermore, in adult WT mice, unilateral 6-OHDA lesion at 12 months of age caused an almost complete striatal DA denervation in the lesioned side, but MSN dendrites and spine numbers were similar in the lesioned and control sides. Taken together, our data indicate that in mice, the development and maintenance of MSN dendrites and spines are DA-independent such that DA depletion does not trigger MSN dendritic atrophy; our data also suggest that the reported MSN dendritic atrophy in PD may be a component of neurodegeneration in PD rather than a consequence of DA denervation.

Systematic review finds "spin" practices and poor reporting standards in studies on machine learning-based prediction models

OBJECTIVES

We evaluated the presence and frequency of spin practices and poor reporting standards in studies that developed and/or validated clinical prediction models using supervised machine learning techniques.

STUDY DESIGN AND SETTING

We systematically searched PubMed from 01/2018 to 12/2019 to identify diagnostic and prognostic prediction model studies using supervised machine learning. No restrictions were placed on data source, outcome, or clinical specialty.

RESULTS

We included 152 studies: 38% reported diagnostic models and 62% prognostic models. When reported, discrimination was described without precision estimates in 53/71 abstracts (74.6% [95% CI 63.4-83.3]) and 53/81 main texts (65.4% [95% CI 54.6-74.9]). Of the 21 abstracts that recommended the model to be used in daily practice, 20 (95.2% [95% CI 77.3-99.8]) lacked any external validation of the developed models. Likewise, 74/133 (55.6% [95% CI 47.2-63.8]) studies made recommendations for clinical use in their main text without any external validation. Reporting guidelines were cited in 13/152 (8.6% [95% CI 5.1-14.1]) studies.

CONCLUSION

Spin practices and poor reporting standards are also present in studies on prediction models using machine learning techniques. A tailored framework for the identification of spin will enhance the sound reporting of prediction model studies.

Spin in the Abstracts of Meta-analyses and Systematic Reviews: Quadriceps Tendon Graft for Anterior Cruciate Ligament Reconstruction

BACKGROUND

Spin is a reporting bias that misrepresents research. Ultimately it can affect surgeon decision making and patient care. Anterior cruciate ligament (ACL) reconstruction is common, but debate continues over optimal treatment modalities.

PURPOSE

To identify the prevalence of spin in meta-analysis and systematic review abstracts regarding the treatment of ACL injuries with quadriceps tendon graft.

STUDY DESIGN

Cross-sectional study.

METHODS

Electronic libraries (MEDLINE, Embase, Web of Science, Google Scholar) were searched for meta-analyses and systematic reviews regarding the treatment of ACL tears with quadriceps tendon graft. The 9 most severe types of spin commonly found in abstracts were used as an evaluation tool to assess the articles. Two reviewers each performed a blinded assessment of each article for spin. A third reviewer helped after review was done to address any discrepancies between the original reviewers. Further evaluation included year of publication, number of citations, journal impact factor, and A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR-2) score.

RESULTS

The electronic database search resulted in 986 articles, of which 13 met our inclusion criteria. After review, we found that 53.8% (7/13) of the included articles contained 1 of the 9 most severe forms of spin. Of the 13 articles, 15.4% (n = 2) contained 2 types of spin, and 38.5% (n = 5) contained 1 type of spin. No studies contained ≥3 types of spin. Of the types of spin evaluated, the most prevalent (n = 4; 30.8%) was type 3 ("selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention"). All studies, regardless of the presence of spin, were found to be low or critically low quality according to the AMSTAR-2 assessment.

CONCLUSION

This study demonstrated the presence of spin in 53.8% of meta-analysis and systematic review abstracts pertaining to quadriceps tendon graft for ACL reconstruction. Orthopaedic surgeons should learn to recognize spin as they review articles when deciding the treatment course for ACL injuries. Additionally, strict criteria should be considered to reduce the prevalence of spin in orthopaedic literature.

Identifying spin bias of nonsignificant findings in biomedical studies

OBJECTIVE

The purpose of this research note is to share a technique for the identification of spin bias that we developed as part of a living systematic review on the cardiovascular testing of e-cigarette substitution for cigarette smoking. While some researchers have remarked on the subjective nature of ascertaining spin bias, our technique objectively documents forms of spin bias arising from the misrepresentation of nonsignificant findings and from the omission of data.

RESULTS

We offer a two-step process for the identification of spin bias consisting of tracking data and findings and recording of data discrepancies by describing how the spin bias was produced in the text. In this research note, we give an example of the documentation of spin bias from our systematic review. Our experience was that nonsignificant results were presented as causal or even as significant in the Discussion of studies. Spin bias distorts scientific research and misleads readers; therefore it behooves peer reviewers and journal editors to make the effort to detect and correct it.

Overinterpretation of findings in machine learning prediction model studies in oncology: a systematic review

OBJECTIVES

In biomedical research, spin is the overinterpretation of findings, and it is a growing concern. To date, the presence of spin has not been evaluated in prognostic model research in oncology, including studies developing and validating models for individualized risk prediction.

STUDY DESIGN AND SETTING

We conducted a systematic review, searching MEDLINE and EMBASE for oncology-related studies that developed and validated a prognostic model using machine learning published between 1st January, 2019, and 5th September, 2019. We used existing spin frameworks and described areas of highly suggestive spin practices.

RESULTS

We included 62 publications (including 152 developed models; 37 validated models). Reporting was inconsistent between methods and the results in 27% of studies due to additional analysis and selective reporting. Thirty-two studies (out of 36 applicable studies) reported comparisons between developed models in their discussion and predominantly used discrimination measures to support their claims (78%). Thirty-five studies (56%) used an overly strong or leading word in their title, abstract, results, discussion, or conclusion.

CONCLUSION

The potential for spin needs to be considered when reading, interpreting, and using studies that developed and validated prognostic models in oncology. Researchers should carefully report their prognostic model research using words that reflect their actual results and strength of evidence.

NSF Fellows' perceptions about incentives, research misconduct, and scientific integrity in STEM academia

There is increased concern about perverse incentives, quantitative performance metrics, and hyper-competition for funding and faculty positions in US academia. Recipients of the prestigious National Science Foundation Graduate Research Fellowships (n = 244) from Civil and Environmental Engineering (45.5%) and Computer Science and Engineering (54.5%) were anonymously surveyed to create a baseline snapshot of their perceptions, behaviors and experiences. NSF Fellows ranked scientific advancement as the top metric for evaluating academics followed by publishing in high-impact journals, social impact of research, and publication/citation counts. The self-reported rate of academic cheating was 16.7% and of research misconduct was 3.7%. Thirty-one percent of fellows reported direct knowledge of graduate peers cheating, and 11.9% had knowledge of research misconduct by colleagues. Only 30.7% said they would report suspected misconduct. A majority of fellows (55.3%) felt that mandatory ethics trainings left them unprepared for dealing with ethical issues. Fellows stated academic freedom, flexible schedules and opportunity to mentor students were the most positive aspects of academia, whereas pressures for funding, publication, and tenure were cited as the most negative aspects. These data may be useful in considering how to better prepare STEM graduate trainees for academic careers.

Overinterpretation of findings in diagnostic accuracy studies of infectious diseases

OBJECTIVES

To assess the prevalence of overly positive interpretation - also called 'spin'- of results in diagnostic accuracy studies of infectious diseases and to identify suggestions for improvement.

METHODS

A PubMed search was performed to identify diagnostic accuracy studies of infectious diseases published between January and March 2019. Each article was assessed by two authors independently to identify study characteristics and forms of actual and potential overinterpretation. 'Actual overinterpretation' was defined as conclusions that were not based on the study aims or conclusions that were more favourable than was justified by the study findings. There are other practices that may result in the overinterpretation of study findings and these have been described as 'potential overinterpretation'.

RESULTS

The final analysis included 120 studies. Favourable or promising recommendations were made in the main text of 101 (84%) of the included studies. Evidence of actual overinterpretation (spin) was found in 30 (25%) articles, with 22 studies reporting a conclusion that did not match the study aims and 56 studies with a more positive conclusion in the abstract than in the main text. All analysed studies exhibited at least one form of potential overinterpretation, with was most commonly a lack of sample size calculation (n = 109, 91%) and not reporting a null hypothesis (n = 115, 96%).

CONCLUSIONS

Evidence of overinterpretation of results was found in a third of the included studies. We propose possible interventions to prevent overly positive interpretation of results in diagnostic accuracy studies.

'Spin' among abstracts of randomised controlled trials in sleep medicine: A research-on-research study

STUDY OBJECTIVES

'Spin', using reporting strategies to distort study results, can mislead readers of medical research. This study aimed to evaluate the prevalence and characteristics of 'spin' among randomised controlled trial (RCT) abstracts published in sleep medicine journals, and to identify factors associated with its presence and severity.

METHODS

The search for RCTs published between 2010 and 2020 were conducted in seven reputable journals of sleep medicine. Abstracts of RCTs with statistically nonsignificant primary outcomes were included and analysed for 'spin', according to pre-determined 'spin' strategies. Chi-square tests or logistic regression analyses were performed to detect the association between characteristics of included abstracts and the presence and severity of 'spin'.

RESULTS

A total of 114 RCT abstracts were included in this study, of which 89 (78.1%) were identified as having at least one type of 'spin' strategy. Sixty-six abstracts (57.9%) had 'spin' in the Results section, 82 (71.9%) abstracts presented with 'spin' in the Conclusions section. The presence of 'spin' varied significantly among RCTs based on the different categories of research area (P=0.047) and the statistician involvement (P=0.045). Furthermore, research area (P=0.019) and funding status (P=0.033) were significant factors associated with the severity of 'spin'.

CONCLUSIONS

The prevalence of 'spin' is high among RCT abstracts in sleep medicine. This raise the need for researchers, editors and other stakeholders to be aware of the issue of 'spin' and make joint efforts to eliminate it in future publications.