Assessment of the abstract reporting of systematic reviews of dose-response meta-analysis: a literature survey

Cochrane review abstracts are getting longer, but this has no large impact on the reporting quality

OBJECTIVE

To evaluate the reporting of abstracts of Cochrane Reviews according to PRISMA-A and to investigate a possible association with the abstract´s length.

METHODS

This is a retrospective, observational study based on all Cochrane Reviews indexed in Medline (via PubMed) until November 18, 2022. In the second part, a random sample of 440 abstracts was drawn, in which PRISMA-A adherence was assessed by two independent reviewers. Analyses were stratified by the year of publication and the number of words.

RESULTS

Overall, the median number of words of the 15,188 included abstracts was 469 (IQR 389-686 words), steadily increasing from 353 words in 2000 to 838 words in 2022, with less than one percent of the abstracts in 2022 having ≤ 300 words (in 2000: 30.7%). Analyses on PRISMA-A adherence in the random sample showed a mean score of 6.1 out of 12 fully reported items. Stratified by year, PRISMA-A adherence increased, with higher word counts in 2000-2010 and 2011-2015, while there was no difference in PRISMA-A adherence by abstract length in 2016-2022.

CONCLUSION

Over the years, abstracts of Cochrane Reviews have become longer, running up to 1000 words. This conflicts with the Cochrane Handbook, which recommends a maximum length of 400 until it was aligned with MECIR in 2019, which has recommended a length of <700 words since 2012 but allows up to 1000 words. It is debatable whether such long abstracts meet the key goals of abstracts of being informative, accurate, appealing, and concise.

Systematic metareview of prediction studies demonstrates stable trends in bias and low PROBAST inter-rater agreement

OBJECTIVES

To (1) explore trends of risk of bias (ROB) in prediction research over time following key methodological publications, using the Prediction model Risk Of Bias ASsessment Tool (PROBAST) and (2) assess the inter-rater agreement of the PROBAST.

STUDY DESIGN AND SETTING

PubMed and Web of Science were searched for reviews with extractable PROBAST scores on domain and signaling question (SQ) level. ROB trends were visually correlated with yearly citations of key publications. Inter-rater agreement was assessed using Cohen's Kappa.

RESULTS

One hundred and thirty nine systematic reviews were included, of which 85 reviews (containing 2,477 single studies) on domain level and 54 reviews (containing 2,458 single studies) on SQ level. High ROB was prevalent, especially in the Analysis domain, and overall trends of ROB remained relatively stable over time. The inter-rater agreement was low, both on domain (Kappa 0.04-0.26) and SQ level (Kappa -0.14 to 0.49).

CONCLUSION

Prediction model studies are at high ROB and time trends in ROB as assessed with the PROBAST remain relatively stable. These results might be explained by key publications having no influence on ROB or recency of key publications. Moreover, the trend may suffer from the low inter-rater agreement and ceiling effect of the PROBAST. The inter-rater agreement could potentially be improved by altering the PROBAST or providing training on how to apply the PROBAST.

Reporting according to the preferred reporting items for systematic reviews and meta-analyses for abstracts (PRISMA-A) depends on abstract length

OBJECTIVE

To evaluate reporting of abstracts of systematic reviews according to the preferred reporting items for systematic reviews and meta-analyses for abstracts (PRISMA-A) 2013 checklist.

STUDY DESIGN AND SETTING

A random sample of 534 systematic reviews on effectiveness indexed in PubMed between 2000 and 2019 was assessed. Adherence of abstracts to PRISMA-A was analysed using descriptive statistics. Results were stratified by number of words, structure, and year of publication.

RESULT

The mean score of fully reported PRISMA-A items was 5.4 of 12, with adherence varying widely between items (0% to 98.8%). Cochrane reviews received higher mean total scores than non-Cochrane reviews (6.3 vs. 5.2). Adherence to PRISMA-A increased linearly with increasing word count. In non-Cochrane reviews, authors of structured abstracts more often adhered to PRISMA-A than those of unstructured abstracts. No improvements in reporting of abstracts were found after the implementation of PRISMA-A in 2013.

CONCLUSION

Adherence to PRISMA-A shows great potential for improvement. Therefore, authors, editors, and reviewers should be made aware of PRISMA-A by referring to it in the journal submission guidelines. As adherence to PRISMA-A increases with the number of words, journals should consider to increase the word limit to 250-300 words.

Vaccination for Human Papillomavirus: an historic and bibliometric study

A systematic literature review was conducted to describe in a historical perspective the evolution of studies concerning HPV vaccination. The search identified 794 articles of which 568 were included. The first article was published in 2001, and the maximum annual number of publications was reached in 2014. The average number of authors per paper was 8.8. Papers originated from 49 different countries, with the USA accounted for the maximum number of publications (n = 217). Efficacy (46.5%) and safety (31.0%) were the most prevalent objectives. Clinical trials constituted the largest group of methods (37.9%). Chronological trends did not reveal any lasting curve-crossings, indicating that the priority topics have remained the same. The geographical origin of these studies raises questions about the transposability of the results to populations where HPV vaccination has been studied only a little. This study could help guide future research to less-studied research objectives, particularly for vaccines.

An Evaluation of the Presence of Spin in the Abstracts of Tonsillectomy Systematic Reviews

OBJECTIVES/HYPOTHESIS

Spin-the practice of adding or omitting information intentionally or unintentionally to make the results of a study more favorable-may influence clinical decision making, especially when present in study abstracts. Here, we quantify and characterize the presence of spin in the abstracts of systematic reviews regarding tonsillectomy.

METHODS

This study is an analysis of systematic review abstracts. Searches were conducted on September 23, 2019 on PubMed and Embase using the advanced search feature to retrieve systematic reviews regarding tonsillectomies. The nine most severe forms of spin were then evaluated. Spin was classified by two investigators in parallel, with each blinded to the classifications of the other. Study characteristics were also recorded in duplicate. Consensus meetings between investigators were held to resolve disagreements.

RESULTS

In the 85 included systematic reviews, at least one form of spin was present in 44.7% (38/85) of abstracts. Journals with higher impact factors were less likely to contain spin in the abstracts of systematic reviews (point biserial correlation coefficient of -0.30). No statistically significant associations were found between the presence of spin and intervention type (P = .56) or adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (P = .08); however, there was a significant association between spin and funding source (P = .03).

CONCLUSIONS

Spin was common in the abstracts of our sample of tonsillectomy systematic reviews. Researchers, clinicians, and peer reviewers could benefit from learning to recognize spin in medical literature. Further research is needed into the effects of spin on clinical decision making.

LEVEL OF EVIDENCE

NA Laryngoscope, 2020.