An Evaluation of Primary Studies Published in Predatory Journals Included in Systematic Reviews From High-Impact Dermatology Journals: Cross-sectional Study

BACKGROUND

Predatory publishing is a deceptive form of publishing that uses unethical business practices, minimal to no peer review processes, or limited editorial oversight to publish articles. It may be problematic to our highest standard of scientific evidence-systematic reviews-through the inclusion of poor-quality and unusable data, which could mislead results, challenge outcomes, and undermine confidence. Thus, there is a growing concern surrounding the effects predatory publishing may have on scientific research and clinical decision-making.

OBJECTIVE

The objective of this study was to evaluate whether systematic reviews published in top dermatology journals contain primary studies published in suspected predatory journals (SPJs).

METHODS

We searched PubMed for systematic reviews published in the top five dermatology journals (determined by 5-year h-indices) between January 1, 2019, and May 24, 2021. Primary studies were extracted from each systematic review, and the publishing journal of these primary studies was cross-referenced using Beall's List and the Directory of Open Access Journals. Screening and data extraction were performed in a masked, duplicate fashion. We performed chi-square tests to determine possible associations between a systematic review's inclusion of a primary study published in a SPJ and particular study characteristics.

RESULTS

Our randomized sample included 100 systematic reviews, of which 31 (31%) were found to contain a primary study published in a SPJ. Of the top five dermatology journals, the Journal of the American Academy of Dermatology had the most systematic reviews containing a primary study published in an SPJ. Systematic reviews containing a meta-analysis or registered protocol were significantly less likely to contain a primary study published in a SPJ. No statistically significant associations were found between other study characteristics.

CONCLUSIONS

Studies published in SPJs are commonly included as primary studies in systematic reviews published in high-impact dermatology journals. Future research is needed to investigate the effects of including suspected predatory publications in scientific research.

The Development of a Checklist to Enhance Methodological Quality in Intervention Programs

The methodological quality of primary studies is an important issue when performing meta-analyses or systematic reviews. Nevertheless, there are no clear criteria for how methodological quality should be analyzed. Controversies emerge when considering the various theoretical and empirical definitions, especially in relation to three interrelated problems: the lack of representativeness, utility, and feasibility. In this article, we (a) systematize and summarize the available literature about methodological quality in primary studies; (b) propose a specific, parsimonious, 12-items checklist to empirically define the methodological quality of primary studies based on a content validity study; and (c) present an inter-coder reliability study for the resulting 12-items. This paper provides a precise and rigorous description of the development of this checklist, highlighting the clearly specified criteria for the inclusion of items and a substantial inter-coder agreement in the different items. Rather than simply proposing another checklist, however, it then argues that the list constitutes an assessment tool with respect to the representativeness, utility, and feasibility of the most frequent methodological quality items in the literature, one that provides practitioners and researchers with clear criteria for choosing items that may be adequate to their needs. We propose individual methodological features as indicators of quality, arguing that these need to be taken into account when designing, implementing, or evaluating an intervention program. This enhances methodological quality of intervention programs and fosters the cumulative knowledge based on meta-analyses of these interventions. Future development of the checklist is discussed.