"Only 50% of randomized trials have high level of confidence in arthroscopy and sports medicine"-a spin-based assessment

Reporting Bias in the Form of Positive Spin Is Highly Prevalent in Abstracts of Systematic Reviews on Primary Repair of the Anterior Cruciate Ligament

PURPOSE

To analyze reporting bias in the form of spin present in systematic reviews and meta-analyses on the topic of primary anterior cruciate ligament (ACL) repair.

METHODS

The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines were followed throughout this study. Peer-reviewed systematic reviews were collected from 3 databases (PubMed, Scopus, and SPORTDiscus), and their abstracts were assessed for the 15 most common types of spin. Articles were excluded if they were not published in English, had no evidence, were retracted, were published without an abstract, did not have full text available, or included cadaveric or nonhuman subjects. Full text quality was assessed using AMSTAR 2 (A Measurement Tool to Assess Systematic Reviews Version 2). Fisher exact tests were used to examine associations between the different types of spin and other study characteristics such as AMSTAR 2 confidence rating, study design, and level of evidence.

RESULTS

Spin was present in the abstracts of 13 of 15 articles (86.7%). There were significant associations between PRISMA adherence and lower incidences of spin types 3, 6, and 8 (P = .029 for each). A critically low AMSTAR 2 confidence rating was significantly associated with an increased incidence of spin type 9 (P = .01), and a higher AMSTAR 2 score was significantly associated with decreased spin type 4 and type 5 (P = .039 and P = .048, respectively). A more recent year of publication was correlated with a lower incidence of spin type 14 (P = .044).

CONCLUSIONS

Spin is present in most systematic reviews and meta-analyses regarding primary repair of the ACL, with two-thirds of abstracts spinning evidence in favor of ACL repair. Standardized guidelines including the PRISMA guidelines and the AMSTAR 2 assessment tool were negatively correlated with spin. More recently published articles were found to contain significantly less spin, as were articles published in journals with higher Clarivate Impact Factors and Scopus CiteScores.

LEVEL OF EVIDENCE

Level V, systematic review of Level III through V studies.

Spin in the abstracts of randomized controlled trials of nurse-led care: A cross-sectional study

BACKGROUND

Nurse-led models of care had been reported as effective intervention approaches for improving health management and reducing hospitalizations of target patients in a number of studies. However, the reporting quality of studies in the field varied and there was a lack of literature evaluation.

OBJECTIVE

The objective of the study was to assess the reporting quality and spin of abstracts of published randomized controlled trials which had statistically not significant primary outcomes. Moreover, potential factors associated with the presence of spin were also assessed.

METHODS

Studies on nurse-led care were retrieved from PubMed from January 1st, 2017, to December 31st, 2021. Only randomized controlled trials with statistically not significant primary outcomes were included. Study screening and data extraction were carried out by two reviewers independently. The reporting quality of each abstract was evaluated by the Consolidated Standards of Reporting Trials statement, and spin strategies were analyzed using a pre-designed assessment form. Potential predictors for the presence of spin were analyzed by multivariate logistic regressions.

RESULTS

The overall reporting quality of the included 75 randomized controlled trial abstracts was not satisfying, with a median score of 16-item Consolidated Standards of Reporting Trials statement at 6 (IQR 5, 8). Forty abstracts used at least one spin strategy in abstracts. Among them, 18 (45.0 %) used spin strategies in the result section and 39 (97.5 %) had spin in the conclusion section. The most common spin strategy identified in abstracts was focusing on statistically significant secondary outcomes (12/40, 30.0 %) in the result section and claiming benefit with no consideration of statistically not significant results for the primary outcomes (32/40, 80.0 %) in the conclusion section. Based on the definition, 29 (72.5 %) abstracts were assessed to have high level of spin in the conclusions of abstracts. By multivariate logistic regression analyses, it was found that only geographic origin (reference: studies from Asian countries, OR = 0.118, 95 % CI 0.027 to 0.511, P = 0.004) and the Consolidated Standards of Reporting Trials statement score (reference: lower score, OR = 0.625, 95 % CI 0.470 to 0.829, P = 0.001) were significantly associated with the presence of spin in abstracts.

CONCLUSION

Among the randomized controlled trials with statistically not significant primary outcomes in the field of nurse-led care, the reporting quality of abstracts needs to be improved. Trials from Asian countries and with lower Consolidated Standards of Reporting Trials statement scores are more likely to present spin in abstracts. Findings reported in the result and conclusion sections of these abstracts need to be interpreted with caution.

Analysis of Spin in RCTs of Spine Surgery Using ORG-LOC Grading Tool

BACKGROUND

Even in highly credible research models, such as randomised control trials (RCTs), many pitfalls do exist that a practitioner must be aware of, to get the actual sense of the research. The one such pitfall that is much more common but ironically less explored is the Writers' bias or the spin. Particularly in the abstracts, it is a potential source of deception to the readers.

METHODS

We selected 250 recent RCTs from the top 5 spine journals. Baseline data and CONSORT Adherence Score (CAS) were collected. Abstracts of the RCTs were graded using the level of confidence (LOC) grading tool developed by the Orthopaedic Research Group. All possible associations of spin were studied to assess the significance.

RESULTS

The median CAS was 11 (IQR 10-12). Only 47.6% (n = 119) articles had High LOC with no or one non-critical spin in abstract. 12.4% (n = 31) had Moderate LOC and 28% (n = 70) had Low LOC. The rest had Critically Low LOC with more than one critical spin. Of the variables analyzed in multivariate regression analysis, only CAS had a (negative) correlation with the LOC of the abstracts.

CONCLUSIONS

Spin-based grading of RCTs is the need of the hour to aid readers to interpret the true essence of research papers. 40% of the RCTs in top spine journals had low to critically low LOC. Objective structuring of abstracts with adherence to CONSORT guidelines is the way forward to prevent spin.

Evaluation of spin in systematic reviews and meta-analyses of superior capsular reconstruction

BACKGROUND

Small, preliminary studies and the systematic reviews on superior capsular reconstruction (SCR) that collate data are at increased risk spin. This study's primary objective was to identify, describe, and account for the incidence of spin in systematic reviews of SCR. This study's secondary objective was to characterize the studies in which spin was identified to determine whether identifiable patterns of characteristics exist among studies with spin.

METHODS

This study was conducted per Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines using a predetermined protocol. A search was conducted on the PubMed and Embase databases for systematic reviews and meta-analyses on SCR. Screening and data extraction were conducted independently by 2 authors. Each included study's abstract was assessed for the presence of the 15 most common types of spin, with full texts reviewed during cases of disagreement or for clarification. General data that were extracted included study title, authors, publication year, journal, level of evidence, study design, funding source, reported adherence to PRISMA guidelines, preregistration of the study protocol, and primary and secondary outcome measures. Full texts were used in the assessment of study quality per AMSTAR 2.

RESULTS

We identified 53 studies during our search, of which 17 met the inclusion criteria. At least 1 form of spin was observed in all 17 studies. The most common types of spin were type 5 ("The conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies") and type 9 ("Conclusion claims the beneficial effect of the experimental treatment despite reporting bias"), both of which were observed in 11 studies (11 of 17, 65%). A statistically significant association between lower level of evidence and type 5 ("The conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies") was observed (P = .0175). A statistically significant association was also found between more recent year of publication and the spin category misleading interpretation (P = .0398), and between lower AMSTAR 2 score and type 13 ("Failure to specify the direction of the effect when it favors the control intervention") (P = .0260). No other statistical associations between other study characteristics were observed.

CONCLUSION

Spin is highly prevalent in abstracts of SCR systematic reviews and meta-analyses. An association was found between the presence of spin and lower level of evidence, year of publication, and AMSTAR 2 ratings.