Assessment of reporting quality of abstracts of systematic reviews with meta-analysis using PRISMA-A and discordance in assessments between raters without prior experience

Evaluation of Spin in the Abstracts of Systematic Reviews and Meta-Analyses of Ulnar Collateral Ligament Reconstruction

Purpose

To identify the quantity and types of spin present in systematic reviews and meta-analyses of ulnar collateral ligament reconstruction (UCLR) outcomes and to characterize the studies with spin to determine if any patterns exist.

Methods

This study was conducted per Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. In August 2022, PubMed, Scopus, and SportDiscus databases were searched using the terms "ulnar collateral ligament reconstruction" AND "systematic review" OR "meta-analysis." Each abstract was assessed for the presence of the 15 most common types of spin derived from a previously established methodology. General data that were extracted included study title, authors, publication year, journal, level of evidence, study design, funding source, reported adherence to PRISMA guidelines, preregistration of the study protocol, and methodologic quality per A Measurement Tool to Assess Systematic Reviews Version 2 (AMSTAR 2).

Results

In total, 122 studies were identified during the preliminary search, of which 19 met the inclusion criteria. Each study had at least 1 form of spin. The most common type of spin identified was type 5 ("The conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies") (7/19, 36.8%). AMSTAR type 9 ("Did the review authors use a satisfactory technique for assessing the RoB [risk of bias] in individual studies that were included in the review?") was associated with both a lower Clarivate Impact Factor (P = .001) and a lower Scopus CiteScore (P = .015). Studies receiving external funding were associated with the failure to satisfy AMSTAR type 3 ("Did the review authors explain their selection of the study designs for inclusion in the review?") (P = .047).

Conclusions

Spin is highly prevalent in the abstracts of systematic reviews and meta-analyses that investigate the outcomes of UCLR.

Clinical Relevance

Spin has been identified in peer-reviewed articles published on various topics, including many in orthopaedics. Systematic reviews and meta-analyses contain the most comprehensive evidence regarding a clinical question, so it is important to identify spin that may be included in these reports. Greater efforts are needed to ensure that the abstracts of papers accurately represent the results in the full text.

A meta-evaluation of the quality of reporting and execution in ecological meta-analyses

Quantitatively summarizing results from a collection of primary studies with meta-analysis can help answer ecological questions and identify knowledge gaps. The accuracy of the answers depends on the quality of the meta-analysis. We reviewed the literature assessing the quality of ecological meta-analyses to evaluate current practices and highlight areas that need improvement. From each of the 18 review papers that evaluated the quality of meta-analyses, we calculated the percentage of meta-analyses that met criteria related to specific steps taken in the meta-analysis process (i.e., execution) and the clarity with which those steps were articulated (i.e., reporting). We also re-evaluated all the meta-analyses available from Pappalardo et al. [1] to extract new information on ten additional criteria and to assess how the meta-analyses recognized and addressed non-independence. In general, we observed better performance for criteria related to reporting than for criteria related to execution; however, there was a wide variation among criteria and meta-analyses. Meta-analyses had low compliance with regard to correcting for phylogenetic non-independence, exploring temporal trends in effect sizes, and conducting a multifactorial analysis of moderators (i.e., explanatory variables). In addition, although most meta-analyses included multiple effect sizes per study, only 66% acknowledged some type of non-independence. The types of non-independence reported were most often related to the design of the original experiment (e.g., the use of a shared control) than to other sources (e.g., phylogeny). We suggest that providing specific training and encouraging authors to follow the PRISMA EcoEvo checklist recently developed by O'Dea et al. [2] can improve the quality of ecological meta-analyses.

Cochrane review abstracts are getting longer, but this has no large impact on the reporting quality

OBJECTIVE

To evaluate the reporting of abstracts of Cochrane Reviews according to PRISMA-A and to investigate a possible association with the abstract´s length.

METHODS

This is a retrospective, observational study based on all Cochrane Reviews indexed in Medline (via PubMed) until November 18, 2022. In the second part, a random sample of 440 abstracts was drawn, in which PRISMA-A adherence was assessed by two independent reviewers. Analyses were stratified by the year of publication and the number of words.

RESULTS

Overall, the median number of words of the 15,188 included abstracts was 469 (IQR 389-686 words), steadily increasing from 353 words in 2000 to 838 words in 2022, with less than one percent of the abstracts in 2022 having ≤ 300 words (in 2000: 30.7%). Analyses on PRISMA-A adherence in the random sample showed a mean score of 6.1 out of 12 fully reported items. Stratified by year, PRISMA-A adherence increased, with higher word counts in 2000-2010 and 2011-2015, while there was no difference in PRISMA-A adherence by abstract length in 2016-2022.

CONCLUSION

Over the years, abstracts of Cochrane Reviews have become longer, running up to 1000 words. This conflicts with the Cochrane Handbook, which recommends a maximum length of 400 until it was aligned with MECIR in 2019, which has recommended a length of <700 words since 2012 but allows up to 1000 words. It is debatable whether such long abstracts meet the key goals of abstracts of being informative, accurate, appealing, and concise.

Systematic metareview of prediction studies demonstrates stable trends in bias and low PROBAST inter-rater agreement

OBJECTIVES

To (1) explore trends of risk of bias (ROB) in prediction research over time following key methodological publications, using the Prediction model Risk Of Bias ASsessment Tool (PROBAST) and (2) assess the inter-rater agreement of the PROBAST.

STUDY DESIGN AND SETTING

PubMed and Web of Science were searched for reviews with extractable PROBAST scores on domain and signaling question (SQ) level. ROB trends were visually correlated with yearly citations of key publications. Inter-rater agreement was assessed using Cohen's Kappa.

RESULTS

One hundred and thirty nine systematic reviews were included, of which 85 reviews (containing 2,477 single studies) on domain level and 54 reviews (containing 2,458 single studies) on SQ level. High ROB was prevalent, especially in the Analysis domain, and overall trends of ROB remained relatively stable over time. The inter-rater agreement was low, both on domain (Kappa 0.04-0.26) and SQ level (Kappa -0.14 to 0.49).

CONCLUSION

Prediction model studies are at high ROB and time trends in ROB as assessed with the PROBAST remain relatively stable. These results might be explained by key publications having no influence on ROB or recency of key publications. Moreover, the trend may suffer from the low inter-rater agreement and ceiling effect of the PROBAST. The inter-rater agreement could potentially be improved by altering the PROBAST or providing training on how to apply the PROBAST.

Do Meta-Analyses of Intervention/Prevention Programs in the Field of Criminology Meet the Tests of Transparency and Reproducibility?

While assessments of transparent reporting practices in meta-analyses are not uncommon in the field of health sciences interventions, they are limited in the social sciences and to our knowledge are non-existent in criminology. Modified PRISMA 2020 checklists were used to assess transparency and reproducibility of reporting for a sample of 33 meta-analyses of intervention/prevention evaluations published in scholarly journals between 2016 and 2021. Results indicate that the average rate of transparent reporting practices was 63%; adherence varied considerably across studies and subscales, with low rates of adherence for some core checklist items. Overwhelmingly, studies were not reproducible in their entirety; article word count was significantly correlated with reproducibility (r = 0.4028, p < .03). These findings suggest that substantial changes to reporting practices are necessary to meet traditional meta-analytic claims of transparency and reproducibility. Study limitations include sample size, coding instruments, and coding subjectivity.

Definition, harms, and prevention of redundant systematic reviews

BACKGROUND

Along with other types of research, it has been stated that the extent of redundancy in systematic reviews has reached epidemic proportions. However, it was also emphasized that not all duplication is bad, that replication in research is essential, and that it can help discover unfortunate behaviors of scientists. Thus, the question is how to define a redundant systematic review, the harmful consequences of such reviews, and what we could do to prevent the unnecessary amount of this redundancy.

MAIN BODY

There is no consensus definition of a redundant systematic review. Also, it needs to be defined what amount of overlap between systematic reviews is acceptable and not considered a redundancy. One needs to be aware that it is possible that the authors did not intend to create a redundant systematic review. A new review on an existing topic, which is not an update, is likely justified only when it can be shown that the previous review was inadequate, for example, due to suboptimal methodology. Redundant meta-analyses could have scientific, ethical, and economic questions for researchers and publishers, and thus, they should be avoided, if possible. Potential solutions for preventing redundant reviews include the following: (1) mandatory prospective registration of systematic reviews; (2) editors and peer reviewers rejecting duplicate/redundant and inadequate reviews; (3) modifying the reporting checklists for systematic reviews; (4) developing methods for evidence-based research (EBR) monitoring; (5) defining systematic reviews; (6) defining the conclusiveness of systematic reviews; (7) exploring interventions for the adoption of methodological advances; (8) killing off zombie reviews (i.e., abandoned registered reviews); (9) better prevention of duplicate reviews at the point of registration; (10) developing living systematic reviews; and (11) education of researchers.

CONCLUSIONS

Disproportionate redundancy of the same or very similar systematic reviews can lead to scientific, ethical, economic, and societal harms. While it is not realistic to expect that the creation of redundant systematic reviews can be completely prevented, some preventive measures could be tested and implemented to try to reduce the problem. Further methodological research and development in this field will be welcome.

THE REPORTING QUALITY OF SYSTEMATIC REVIEW ABSTRACTS IN LEADING GENERAL DENTAL JOURNALS: A METHODOLOGICAL STUDY

OBJECTIVE

To assess the reporting quality of systematic review (SR) abstracts published in leading general dental journals according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Abstracts (PRISMA-A) guidelines, and to identify factors associated with overall reporting quality.

METHODS

We identified SR abstracts published in 10 leading general dental journals and assessed their reporting quality. For each abstract, an overall reporting score (ORS, range: 0-13) was calculated. Risk ratio (RR) was calculated to compare the reporting quality of abstracts in Pre-PRISMA (2011-2012) and Post-PRISMA (2017-2018) periods. Univariable and multivariable linear regression analyses were performed to identify factors associated with reporting quality.

RESULTS

A total of 104 eligible abstracts were included. The mean ORS was 5.59 (SD = 1.48) and 6.97 (1.74) respectively in the Pre- and Post-PRISMA abstracts, with statistically significant difference (mean difference = 1.38; 95% CI: 0.70, 2.05). Reporting of the exact P-value (B = 1.22; 95% CI: 0.45, 1.99) was a significant predictor of higher reporting quality.

CONCLUSION

The reporting quality of SR abstracts published in leading general dental journals improved after the release of PRISMA-A guidelines, but is still suboptimal. Relevant stakeholders need to work together to enhance the reporting quality of SR abstracts in dentistry.

Reporting quality of abstracts of systematic reviews/meta-analyses: An appraisal of Arab Journal of Urology across 12 years: the PRISMA-Abstracts checklist

Objective

We appraised the reporting quality of abstracts of systematic reviews/meta-analyses (SR/MAs) published in one urology journal and explored associations between abstract characteristics and completeness of reporting.

Methods

The Arab Journal of Urology (AJU) was searched for SR/MAs published between January 2011 and 31 May 2022. SR/MAs with structured abstract and quantitative synthesis were eligible. Two reviewers simultaneously together selected the SR/MAs by title, screened the abstracts, and included those based on inclusion/exclusion criteria. Data of a range of characteristics were extracted from each SR/MAs into a spreadsheet. To gauge completeness of reporting, the PRISMA-Abstract checklist (12 items) was used to appraise the extent to which abstracts adhered to the checklist. For each abstract, we computed item, section, and overall adherence. Chi-square and t-tests compared the adherence scores. Univariate and multivariate analyses identified the abstract characteristics associated with overall adherence.

Results

In total, 66 SR/MAs published during the examined period; 62 were included. Partial reporting was not uncommon. In terms of adherence to the 12 PRISMA-A items were: two items exhibited 100% adherence (title, objectives); five items had 80% to <100% adherence (interpretation, included studies, synthesis of results, eligibility criteria, and information sources); two items displayed 40% to <80% adherence (description of the effect, strengths/limitations of evidence); and three items had adherence that fell between 0% and 1.6% (risk of bias, funding/conflict of interest, registration). Multivariable regression revealed two independent predictors of overall adherence: single-country authorship (i.e. no collaboration) was associated with higher overall adherence (P = 0.046); and abstracts from South America were associated with lower overall adherence (P = 0.04).

Conclusion

This study is the first to appraise abstracts of SR/MAs in urology. For high-quality abstracts, improvements are needed in the quality of reporting. Adoption/better adherence to PRISMA-A checklist by editors/authors could improve the reporting quality and completeness of SR/MAs abstracts.

Reporting according to the preferred reporting items for systematic reviews and meta-analyses for abstracts (PRISMA-A) depends on abstract length

OBJECTIVE

To evaluate reporting of abstracts of systematic reviews according to the preferred reporting items for systematic reviews and meta-analyses for abstracts (PRISMA-A) 2013 checklist.

STUDY DESIGN AND SETTING

A random sample of 534 systematic reviews on effectiveness indexed in PubMed between 2000 and 2019 was assessed. Adherence of abstracts to PRISMA-A was analysed using descriptive statistics. Results were stratified by number of words, structure, and year of publication.

RESULT

The mean score of fully reported PRISMA-A items was 5.4 of 12, with adherence varying widely between items (0% to 98.8%). Cochrane reviews received higher mean total scores than non-Cochrane reviews (6.3 vs. 5.2). Adherence to PRISMA-A increased linearly with increasing word count. In non-Cochrane reviews, authors of structured abstracts more often adhered to PRISMA-A than those of unstructured abstracts. No improvements in reporting of abstracts were found after the implementation of PRISMA-A in 2013.

CONCLUSION

Adherence to PRISMA-A shows great potential for improvement. Therefore, authors, editors, and reviewers should be made aware of PRISMA-A by referring to it in the journal submission guidelines. As adherence to PRISMA-A increases with the number of words, journals should consider to increase the word limit to 250-300 words.

Spin in Abstracts of Systematic Reviews and Meta-analyses of Melanoma Therapies: Cross-sectional Analysis

BACKGROUND

Spin is defined as the misrepresentation of a study's results, which may lead to misperceptions or misinterpretation of the findings. Spin has previously been found in randomized controlled trials and systematic reviews of acne vulgaris treatments and treatments of various nondermatological conditions.

OBJECTIVE

The purpose of this study was to quantify the presence of spin in abstracts of systematic reviews and meta-analyses of melanoma therapies and identify any related secondary characteristics of these articles.

METHODS

We used a cross-sectional approach on June 2, 2020, to search the MEDLINE and Embase databases from their inception. To meet inclusion criteria, a study was required to be a systematic review or meta-analysis pertaining to the treatment of melanoma in human subjects, and reported in English. We used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) definition of systematic reviews and meta-analyses. Data were extracted in a masked, duplicate fashion. We conducted a powered bivariate linear regression and calculated odds ratios for each study characteristic.

RESULTS

A total of 200 systematic reviews met the inclusion criteria. We identified spin in 38% (n=76) of the abstracts. The most common type of spin found was type 3 (selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention), occurring 40 times; the least common was type 2 (title claims or suggests a beneficial effect of the experimental intervention not supported by the findings), which was not present in any included abstracts. We found that abstracts pertaining to pharmacologic interventions were 3.84 times more likely to contain spin. The likelihood of an article containing spin has decreased annually (adjusted odds ratio 0.91, 95% CI 0.84-0.99). No significant correlation between funding source or other study characteristics and the presence of spin was identified.

CONCLUSIONS

We have found that spin is fairly common in the abstracts of systematic reviews of melanoma treatments, but the prevalence of spin in these abstracts has been declining from 1992-2020.

Frequency and characteristics of promissory conference abstracts, i.e. abstracts without results, accepted at Cochrane Colloquia 1994-2020

Background

The purpose of a conference abstract is to summarize the main points of a research-related report that will be presented at an academic conference. However, some conferences accept and publish abstracts without results, which is contrary to the basic idea of a conference abstract as a dissemination tool. A conference abstract without results included is called a “promissory abstract”. This study aimed to analyze the frequency and characteristics of promissory conference abstracts, i.e. abstracts submitted without results, accepted at Cochrane Colloquia.

Methods

We analyzed 8297 conference abstracts accepted at 25 Cochrane Colloquia, organized in 1994–2020, which were publicly available on the website of the Cochrane Library. Two authors screened abstracts to identify promissory abstracts. We extracted characteristics of promissory abstracts.

Results

Among abstracts accepted for Cochrane Colloquia, 8.7% were promissory; 475 (66%) were accepted as poster presentations, 241 (34%) as oral presentations and 1 as a workshop. The median number of authors in promissory abstracts was 4 (interquartile range: 3 to 6 authors). In 245 (34%) promissory abstracts, affiliations of authors were not reported. The authors were most commonly affiliated with the following countries: UK (472; 36%), Canada (N = 123; 26%), China (N = 76; 16%), United States (N = 66; 14%) and Australia (N = 53; 11%). There were 512 (71%) promissory abstracts in which study design was not reported.

Conclusion

Promissory abstracts were commonly accepted at Cochrane Colloquia. Such abstracts deserve further attention, as they are detrimental in terms of the dissemination of new knowledge presented at a conference. Conference organizers could ask authors to update the abstract results subsequently to enable the dissemination of information presented at a conference.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01442-3.

Evaluation of spin in the abstracts of systematic reviews and meta-analyses of studies on opioid use disorder

BACKGROUND

Spin, or the inappropriate formatting of information to emphasize certain outcomes, should not be present in research. This study focuses on identifying and characterizing the presence of spin in systematic review and meta-analysis abstracts that focus on the treatment of opioid use disorder. Methods: Search strategies were developed to identify studies pertaining to the treatment of opioid use disorder. The studies were then screened by two authors. These qualifying studies were then evaluated for the presence of spin within their abstracts by two trained authors. These studies were also evaluated by the AMSTAR-2 standards to evaluate the quality of the qualifying systematic reviews by two trained reviewers. Results: The sample in this study included 113 systematic reviews and meta-analyses. Spin was present in 20 of these studies (20/113, 17.7%). The most common spin form was spin type 3 (6/20, 30%), followed by types 5 and 9 (both 4/20, 20%), type 6 (3/20, 15%), type 7 (2/20, 10%), and type 8 (1/20, 5%). The remaining spin types 1, 2, and 4 were not present in the sample. Of the 113 included studies, the most common intervention type was pharmacologic (93/113, 82%). No significant association was found between the quality of a systematic review and the presence of spin. Conclusions: Findings in this study show positive trends in prevalence of five forms of spin evaluated in abstracts of systematic reviews and meta-analyses looking at treatments for opioid use disorder. However, study quality had no significant association with the presence of spin. Misrepresentation of results, or spin, may alter a clinician's perceptions about treatment efficacies. Therefore, increasing physician awareness of spin may improve clinical decision-making.

There is still room for improvement in the completeness of abstract reporting according to the PRISMA-A checklist: a cross-sectional study on systematic reviews in periodontology

BACKGROUND

To evaluate the completeness of reporting abstracts of systematic reviews (SRs) before and after the publication of the PRISMA-A checklist in 2013 and to assess if an association exists between abstract characteristics and the completeness of reporting.

METHODS

A systematic search of the literature was conducted in the PubMed and Scopus databases in March 2020. The search focused on the SRs of evaluations of interventions published since 2002 in the field of periodontology. The abstracts of the selected SRs were divided into two groups before and after publication of the PRISMA-A checklist in 2013, and compliance with the 12 items reported in the checklist was evaluated by three calibrated evaluators.

RESULTS

A set of 265 abstracts was included in the study. The total score before (mean score, 53.78%; 95% CI, 51.56-55.90%) and after (mean score, 56.88%; 95% CI, 55.39-58.44%) the publication of the PRISMA-A statement exhibited a statistically significant improvement (P = 0.012*). Nevertheless, only the checklist items included studies and synthesis of the results displayed a statistically significant change after guideline publication. The total PRISMA-A score was higher in the meta-analysis group and in articles authored by more than four authors.

CONCLUSIONS

The impact of the PRISMA-A was statistically significant, but the majority of the items did not improve after its introduction. The editors and referees of periodontal journals should promote adherence to the checklist to improve the quality of the reports and provide readers with better insight into the characteristics of published studies.

Evaluation of spin in the abstracts of systematic reviews and meta-analyses related to the treatment of proximal humerus fractures

BACKGROUND

Research has shown that many physicians rely solely on abstracts to make clinical decisions. However, many abstracts have been shown to be misleading. The primary objective of this study was to identify the prevalence of spin - bias towards particular results - within the abstracts of systematic reviews and meta-analyses pertaining to the treatment of proximal humerus fractures, one of the most common osteoporotic fractures among elderly patients.

METHODS

We systematically searched MEDLINE and Embase databases to identify systematic reviews and meta-analyses examining the treatment of proximal humerus fractures. Screening and data extraction occurred in a masked, duplicate fashion. The nine most severe types of spin that occur within abstracts were extracted along with study characteristics, including journal recommendations to adhere to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) and year in which the review was performed, to identify potential associations. We subsequently explored the association between spin and the methodological quality of a systematic review using the revised A MeaSurement Tool to Assess systematic Reviews (AMSTAR 2) appraisal instrument.

RESULTS

Our search retrieved 505 articles, of which 73 systematic reviews met inclusion criteria. We found that 34.2% (25/73) of the included systematic reviews contained spin. Spin type 3 (selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention) was the most common type identified (12/73, 16.4%). Three spin types were not identified in any of the abstracts. Spin was 3.2 (OR 3.2; 95% CI, 1.02-10.02) times more likely to be present in systematic reviews published in journals recommending adherence to PRISMA. Furthermore, the odds of an abstract containing spin was 1.25 (OR 1.25; 95% CI, 1.02-1.52) times more likely to be present in systematic reviews for each year after 2000. No other study characteristics were associated with spin. The methodological quality of 24 studies were rated as "critically low" (32.9%), 14 were "low" (19.2%), 28 were "moderate" (38.4%), and 7 were "high" (9.6%), but these findings were not associated with spin.

CONCLUSION

Spin was present in systematic review abstracts regarding treatment of proximal humerus fractures. Measures such as education on the subject of spin and improved reporting standards should be implemented to increase awareness and reduce incidence of spin in abstracts.

LEVEL OF EVIDENCE OF THE STUDY PERFORMED

Basic Science Study; Research Methodology.

Reporting quality of systematic review abstracts in operative dentistry: An assessment using the PRISMA for Abstracts guidelines

OBJECTIVES

To assess and compare the reporting quality of systematic review (SR) abstracts in operative dentistry published before and after the release of Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Abstracts (PRISMA-A), and to identify factors associated with reporting quality.

METHODS

PubMed was searched for abstracts published during 2010-2012 (Pre-PRISMA period) and 2017-2019 (Post-PRISMA period). Reporting quality was assessed and scored using a modified 13-item PRSIMA-A checklist. Risk ratio (RR) was used to compare the adequate reporting rate of each item between the two periods. Univariable and multivariable linear regression analyses were performed to identify factors associated with reporting quality.

RESULTS

A total of 160 abstracts were included and assessed. Only four items ('objective', 'results of main outcomes', 'description of the effect' and 'interpretation') were adequately reported in most abstracts (>75 %). According to the multivariable analysis, greater word count (P = 0.001), being published in the Post-PRISMA period (P = 0.025) and geographic origin from Asia (P = 0.025) or South America (P = 0.015) were significantly associated with higher reporting quality.

CONCLUSIONS/CLINICAL SIGNIFICANCE

The reporting quality of SR abstracts in operative dentistry had improved significantly after the publication of PRISMA-A, but was still suboptimal. Researchers, reviewers and journal editors in operative dentistry need to be familiar with the PRISMA-A checklist, and make concerted efforts to improve the reporting of SR abstracts.

A tutorial on methodological studies: the what, when, how and why

BACKGROUND

Methodological studies - studies that evaluate the design, analysis or reporting of other research-related reports - play an important role in health research. They help to highlight issues in the conduct of research with the aim of improving health research methodology, and ultimately reducing research waste.

MAIN BODY

We provide an overview of some of the key aspects of methodological studies such as what they are, and when, how and why they are done. We adopt a "frequently asked questions" format to facilitate reading this paper and provide multiple examples to help guide researchers interested in conducting methodological studies. Some of the topics addressed include: is it necessary to publish a study protocol? How to select relevant research reports and databases for a methodological study? What approaches to data extraction and statistical analysis should be considered when conducting a methodological study? What are potential threats to validity and is there a way to appraise the quality of methodological studies?

CONCLUSION

Appropriate reflection and application of basic principles of epidemiology and biostatistics are required in the design and analysis of methodological studies. This paper provides an introduction for further discussion about the conduct of methodological studies.

An Evaluation of the Presence of Spin in the Abstracts of Tonsillectomy Systematic Reviews

OBJECTIVES/HYPOTHESIS

Spin-the practice of adding or omitting information intentionally or unintentionally to make the results of a study more favorable-may influence clinical decision making, especially when present in study abstracts. Here, we quantify and characterize the presence of spin in the abstracts of systematic reviews regarding tonsillectomy.

METHODS

This study is an analysis of systematic review abstracts. Searches were conducted on September 23, 2019 on PubMed and Embase using the advanced search feature to retrieve systematic reviews regarding tonsillectomies. The nine most severe forms of spin were then evaluated. Spin was classified by two investigators in parallel, with each blinded to the classifications of the other. Study characteristics were also recorded in duplicate. Consensus meetings between investigators were held to resolve disagreements.

RESULTS

In the 85 included systematic reviews, at least one form of spin was present in 44.7% (38/85) of abstracts. Journals with higher impact factors were less likely to contain spin in the abstracts of systematic reviews (point biserial correlation coefficient of -0.30). No statistically significant associations were found between the presence of spin and intervention type (P = .56) or adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (P = .08); however, there was a significant association between spin and funding source (P = .03).

CONCLUSIONS

Spin was common in the abstracts of our sample of tonsillectomy systematic reviews. Researchers, clinicians, and peer reviewers could benefit from learning to recognize spin in medical literature. Further research is needed into the effects of spin on clinical decision making.

LEVEL OF EVIDENCE

NA Laryngoscope, 2020.

Has the Reporting Quality of Systematic Review Abstracts in Nursing Improved Since the Release of PRISMA for Abstracts? A Survey of High-Profile Nursing Journals

BACKGROUND

The PRISMA for Abstracts (PRISMA-A) was developed to guide authors to present a structured abstract. However, the adherence of abstracts to these guidelines in some areas was of concern.

AIMS

To determine whether the publication of PRISMA-A resulted in an improvement in the abstracts reported with nursing systematic reviews (SRs).

METHODS

This was a cross-sectional study. We searched PubMed for randomized controlled trials-based SRs published in top-tier nursing journals. A PRISMA-A checklist was used to assess abstracts in the SR included. Total score on checklists, comparison of total scores between two periods, and effect factors were analyzed.

RESULTS

Overall, abstract reporting compliance with PRISMA-A has not improved significantly with the time span. Of the 81 SRs, 74.1% were structured. About half reported eligibility criteria, information sources, and description of the effect as recommended. Registration status was reported only in 4.9%. The reporting quality was significantly higher for journals with higher impact factors (p < .001).

LINKING EVIDENCE TO ACTION

Although not inclusive of all SRs in the nursing field, our sample reflects the general trend that there was no significant improvement in the compliance of SR abstracts reported in nursing with the release of PRISMA-A. There is room for improvement, as most items have not been fully reported.

Evaluation of Spin in the Abstracts of Systematic Reviews and Meta-Analyses Focused on the Treatment of Acne Vulgaris: Cross-Sectional Analysis

Background

Spin is the misrepresentation of study findings, which may positively or negatively influence the reader’s interpretation of the results. Little is known regarding the prevalence of spin in abstracts of systematic reviews, specifically systematic reviews pertaining to the management and treatment of acne vulgaris.

Objective

The primary objective of this study was to characterize and determine the frequency of the most severe forms of spin in systematic review abstracts and to evaluate whether various study characteristics were associated with spin.

Methods

Using a cross-sectional study design, we searched PubMed and EMBASE for systematic reviews focusing on the management and treatment of acne vulgaris. Our search returned 316 studies, of which 36 were included in our final sample. To be included, each systematic review must have addressed either pharmacologic or nonpharmacologic treatment of acne vulgaris. These studies were screened, and data were extracted in duplicate by two blinded investigators. We analyzed systematic review abstracts for the nine most severe types of spin.

Results

Spin was present in 31% (11/36) of abstracts. A total of 12 examples of spin were identified in the 11 abstracts containing spin, with one abstract containing two instances of spin. The most common type of spin, selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention, was identified five times (5/12, 42%). A total of 44% (16/36) of studies did not report a risk of bias assessment. Of the 11 abstracts containing spin, six abstracts (55%) had not reported a risk of bias assessment or performed a risk of bias assessment but did not discuss it. Spin in abstracts was not significantly associated with a specific intervention type, funding source, or journal impact factor.

Conclusions

Spin is present in the abstracts of systematic reviews and meta-analyses covering the treatment of acne vulgaris. This paper raises awareness of spin in abstracts and emphasizes the importance of its recognition, which may lead to fewer incidences of spin in future studies.

The collaboration and reporting quality of social welfare systematic reviews in the Campbell Collaboration online library

Background

To analyze the collaboration and reporting quality of the systematic reviews of social welfare in the Campbell collaboration online library.

Methods

The Campbell collaboration online library was searched for systematic reviews of social welfare and the basic information extracted in order to assess the reporting quality of systematic reviews using a MOOSE checklist. BICOMS-2 and UCINET software were used to produce the social network, and Comprehensive Meta Analysis (Version 2) and STATA 13.0 were used to analyze the related data.

Results

Fifty-seven systematic reviews of social welfare were included. Twenty-eight items of the included social welfare systematic reviews were rated as complete (≥70%). There were significant differences between ≤2013 and ≥ 2014 in five items. These differences were as follows: research published by one organization or more than one organization in one item, more than three authors or less than four authors in two items, and one country or more than one country in six items. It’s completed about researches with more than one organization, three authors or more than one country. Some items were found to have a low reporting rate of studies published before 2014, by one organization, with less than four authors or one country, respectively. The social network of authors and organizations showed good collaboration.

Conclusions

Some items could be further improved with regard to the rate of reporting systematic reviews of social welfare in the Campbell collaboration online library. This could improve the overall quality of social welfare systematic reviews.