1
|
Kim MS, Hasan LK, Fathi A, Hasan SK, Haratian A, Bolia IK, Petrigliano FA, Weber AE, Gamradt SC, Liu JN. Evaluation of spin in systematic reviews and meta-analyses of superior capsular reconstruction. J Shoulder Elbow Surg 2022; 31:1743-1750. [PMID: 35472573 DOI: 10.1016/j.jse.2022.03.015] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/16/2022] [Revised: 03/11/2022] [Accepted: 03/14/2022] [Indexed: 02/01/2023]
Abstract
BACKGROUND Small, preliminary studies and the systematic reviews on superior capsular reconstruction (SCR) that collate data are at increased risk spin. This study's primary objective was to identify, describe, and account for the incidence of spin in systematic reviews of SCR. This study's secondary objective was to characterize the studies in which spin was identified to determine whether identifiable patterns of characteristics exist among studies with spin. METHODS This study was conducted per Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines using a predetermined protocol. A search was conducted on the PubMed and Embase databases for systematic reviews and meta-analyses on SCR. Screening and data extraction were conducted independently by 2 authors. Each included study's abstract was assessed for the presence of the 15 most common types of spin, with full texts reviewed during cases of disagreement or for clarification. General data that were extracted included study title, authors, publication year, journal, level of evidence, study design, funding source, reported adherence to PRISMA guidelines, preregistration of the study protocol, and primary and secondary outcome measures. Full texts were used in the assessment of study quality per AMSTAR 2. RESULTS We identified 53 studies during our search, of which 17 met the inclusion criteria. At least 1 form of spin was observed in all 17 studies. The most common types of spin were type 5 ("The conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies") and type 9 ("Conclusion claims the beneficial effect of the experimental treatment despite reporting bias"), both of which were observed in 11 studies (11 of 17, 65%). A statistically significant association between lower level of evidence and type 5 ("The conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies") was observed (P = .0175). A statistically significant association was also found between more recent year of publication and the spin category misleading interpretation (P = .0398), and between lower AMSTAR 2 score and type 13 ("Failure to specify the direction of the effect when it favors the control intervention") (P = .0260). No other statistical associations between other study characteristics were observed. CONCLUSION Spin is highly prevalent in abstracts of SCR systematic reviews and meta-analyses. An association was found between the presence of spin and lower level of evidence, year of publication, and AMSTAR 2 ratings.
Collapse
Affiliation(s)
- Michael S Kim
- USC Epstein Family Center for Sports Medicine, Department of Orthopaedic Surgery, Keck Medicine of USC, Los Angeles, CA, USA
| | - Laith K Hasan
- USC Epstein Family Center for Sports Medicine, Department of Orthopaedic Surgery, Keck Medicine of USC, Los Angeles, CA, USA
| | - Amir Fathi
- USC Epstein Family Center for Sports Medicine, Department of Orthopaedic Surgery, Keck Medicine of USC, Los Angeles, CA, USA
| | - Shurooq K Hasan
- Department of Health Policy and Management, School of Public Health, University of Maryland, College Park, MD, USA
| | - Aryan Haratian
- USC Epstein Family Center for Sports Medicine, Department of Orthopaedic Surgery, Keck Medicine of USC, Los Angeles, CA, USA
| | - Ioanna K Bolia
- USC Epstein Family Center for Sports Medicine, Department of Orthopaedic Surgery, Keck Medicine of USC, Los Angeles, CA, USA
| | - Frank A Petrigliano
- USC Epstein Family Center for Sports Medicine, Department of Orthopaedic Surgery, Keck Medicine of USC, Los Angeles, CA, USA
| | - Alexander E Weber
- USC Epstein Family Center for Sports Medicine, Department of Orthopaedic Surgery, Keck Medicine of USC, Los Angeles, CA, USA
| | - Seth C Gamradt
- USC Epstein Family Center for Sports Medicine, Department of Orthopaedic Surgery, Keck Medicine of USC, Los Angeles, CA, USA
| | - Joseph N Liu
- USC Epstein Family Center for Sports Medicine, Department of Orthopaedic Surgery, Keck Medicine of USC, Los Angeles, CA, USA.
| |
Collapse
|
2
|
Nowlin R, Wirtz A, Wenger D, Ottwell R, Cook C, Arthur W, Sallee B, Levin J, Hartwell M, Wright D, Sealey M, Zhu L, Vassar M. Spin in Abstracts of Systematic Reviews and Meta-analyses of Melanoma Therapies: Cross-sectional Analysis. JMIR DERMATOLOGY 2022; 5:e33996. [PMID: 37632865 PMCID: PMC10334896 DOI: 10.2196/33996] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2021] [Revised: 01/01/2022] [Accepted: 01/03/2022] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND Spin is defined as the misrepresentation of a study's results, which may lead to misperceptions or misinterpretation of the findings. Spin has previously been found in randomized controlled trials and systematic reviews of acne vulgaris treatments and treatments of various nondermatological conditions. OBJECTIVE The purpose of this study was to quantify the presence of spin in abstracts of systematic reviews and meta-analyses of melanoma therapies and identify any related secondary characteristics of these articles. METHODS We used a cross-sectional approach on June 2, 2020, to search the MEDLINE and Embase databases from their inception. To meet inclusion criteria, a study was required to be a systematic review or meta-analysis pertaining to the treatment of melanoma in human subjects, and reported in English. We used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) definition of systematic reviews and meta-analyses. Data were extracted in a masked, duplicate fashion. We conducted a powered bivariate linear regression and calculated odds ratios for each study characteristic. RESULTS A total of 200 systematic reviews met the inclusion criteria. We identified spin in 38% (n=76) of the abstracts. The most common type of spin found was type 3 (selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention), occurring 40 times; the least common was type 2 (title claims or suggests a beneficial effect of the experimental intervention not supported by the findings), which was not present in any included abstracts. We found that abstracts pertaining to pharmacologic interventions were 3.84 times more likely to contain spin. The likelihood of an article containing spin has decreased annually (adjusted odds ratio 0.91, 95% CI 0.84-0.99). No significant correlation between funding source or other study characteristics and the presence of spin was identified. CONCLUSIONS We have found that spin is fairly common in the abstracts of systematic reviews of melanoma treatments, but the prevalence of spin in these abstracts has been declining from 1992-2020.
Collapse
Affiliation(s)
- Ross Nowlin
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, United States
| | - Alexis Wirtz
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, United States
| | - David Wenger
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, United States
| | - Ryan Ottwell
- Department of Internal Medicine, University of Oklahoma College of Community Medicine, Tulsa, OK, United States
- Department of Dermatology, St. Joseph Mercy Hospital, Ann Arbor, MI, United States
| | - Courtney Cook
- Department of Dermatology, University of Oklahoma Health Sciences Center, Oklahoma City, OK, United States
| | - Wade Arthur
- Department of Internal Medicine, University of Arkansas for Medical Sciences, Fayetteville, AR, United States
| | - Brigitte Sallee
- Department of Dermatology, University of Oklahoma Health Sciences Center, Oklahoma City, OK, United States
| | - Jarad Levin
- Department of Dermatology, University of Oklahoma Health Sciences Center, Oklahoma City, OK, United States
| | - Micah Hartwell
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, United States
- Department of Psychiatry and Behavioral Sciences, Oklahoma State University Center for Health Sciences, Tulsa, OK, United States
| | - Drew Wright
- Samuel J. Wood Library and C.V. Starr Biomedical Information Center, Weill Cornell Medical College, New York, NY, United States
| | - Meghan Sealey
- Department of Statistics, Oklahoma State University, Stillwater, OK, United States
| | - Lan Zhu
- Department of Statistics, Oklahoma State University, Stillwater, OK, United States
| | - Matt Vassar
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, United States
- Department of Psychiatry and Behavioral Sciences, Oklahoma State University Center for Health Sciences, Tulsa, OK, United States
| |
Collapse
|
3
|
Ottwell R, Esmond L, Rea W, Hartwell M, Som M, Harris R, Miao Z, Zhu L, Arthur W, Brachtenbach T, Wright DN, Vassar M. Spin Infrequently Occurs in Abstracts of Systematic Reviews For The Pharmacological Treatment of Type 2 Diabetes Mellitus. Diabet Med 2021; 38:e14653. [PMID: 34289158 DOI: 10.1111/dme.14653] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/26/2021] [Accepted: 07/13/2021] [Indexed: 12/19/2022]
Abstract
AIMS Currently, there is a growing body of research demonstrating that spin - the misinterpretation and distortion of a study's findings - is common in different fields of medicine. To our knowledge, no study has investigated its presence in systematic reviews focused on diabetic therapies. METHODS We performed a cross-sectional study by searching MEDLINE and Embase for systematic reviews focused on pharmacologic treatments for type 2 diabetes mellitus. Our search retrieved 26,490 records, from which 199 studies were extracted in a masked, duplicate fashion. Each study was evaluated for the nine most severe types of spin and other study design parameters. Spin was presented as frequencies and odds ratios to identify associations between study characteristics. RESULTS Spin was identified in the abstracts of 15 systematic reviews (15/199, 7.5%). Spin type 5 was the most common type identified (7/199, 3.5%). Spin types 1, 2, 4, and 8 were not identified. In the last 5 years (2016-2021), 7 systematic reviews contained spin within their abstract. There was no association between spins presence and any extracted study characteristic . CONCLUSIONS Our findings show that spin infrequently occurs in abstracts of systematic reviews focused on pharmacologic therapies for type 2 diabetes mellitus. However, any amount of spin can lead to the distortion of a reader's interpretation of the study's findings. Thus, we provide recommendations with rationale to prevent spin in future systematic reviews.
Collapse
Affiliation(s)
- Ryan Ottwell
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
- Department of Internal Medicine, University of Oklahoma, School of Community Medicine, Tulsa, OK, USA
- Samuel J. Wood Library & C.V. Starr Biomedical Information Center, Weill Cornell Medical College, New York, NY, USA
| | - Lindy Esmond
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - William Rea
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
- Kansas City University of Medicine and Biosciences, Joplin, MO, USA
| | - Micah Hartwell
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
- Department of Psychiatry and Behavioral Sciences, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - Mousumi Som
- Department of Internal Medicine, Oklahoma State University Medical Center, Tulsa, OK, USA
| | - Rachael Harris
- Department of Internal Medicine, Oklahoma State University Medical Center, Tulsa, OK, USA
| | - Zhuqi Miao
- Spears School of Business, Oklahoma State University, Stillwater, OK, USA
| | - Lan Zhu
- Department of Statistics, Oklahoma State University, Stillwater, OK, USA
| | - Wade Arthur
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - Travis Brachtenbach
- Department of Internal Medicine, Oklahoma State University Medical Center, Tulsa, OK, USA
| | - Drew N Wright
- Spears School of Business, Oklahoma State University, Stillwater, OK, USA
| | - Matt Vassar
- Office of Medical Student Research, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
- Department of Psychiatry and Behavioral Sciences, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| |
Collapse
|