Assessment of Duplicate Evidence in Systematic Reviews of Imaging Findings of Children With COVID-19

Methods proposed for monitoring the implementation of evidence-based research: a cross-sectional study

OBJECTIVES

Evidence-based research (EBR) is the systematic and transparent use of prior research to inform a new study so that it answers questions that matter in a valid, efficient, and accessible manner. This study surveyed experts about existing (e.g., citation analysis) and new methods for monitoring EBR and collected ideas about implementing these methods.

STUDY DESIGN AND SETTING

We conducted a cross-sectional study via an online survey between November 2022 and March 2023. Participants were experts from the fields of evidence synthesis and research methodology in health research. Open-ended questions were coded by recurring themes; descriptive statistics were used for quantitative questions.

RESULTS

Twenty-eight expert participants suggested that citation analysis should be supplemented with content evaluation (not just what is cited but also in which context), content expert involvement, and assessment of the quality of cited systematic reviews. They also suggested that citation analysis could be facilitated with automation tools. They emphasized that EBR monitoring should be conducted by ethics committees and funding bodies before the research starts. Challenges identified for EBR implementation monitoring were resource constraints and clarity on responsibility for EBR monitoring.

CONCLUSION

Ideas proposed in this study for monitoring the implementation of EBR can be used to refine methods and define responsibility but should be further explored in terms of feasibility and acceptability. Different methods may be needed to determine if the use of EBR is improving over time.

A Decade of Efforts to Add Value to Child Health Research Practices

OBJECTIVE

To identify practices that add value to improve the design, conduct, and reporting of child health research and reduce research waste.

STUDY DESIGN

In order to categorize the contributions of members of Standards for Research (StaR) in Child Health network, we developed a novel Child Health Improving Research Practices (CHIRP) framework comprised of 5 domains meant to counteract avoidable child health research waste and improve quality: 1) address research questions relevant to children, their families, clinicians, and researchers; 2) apply appropriate research design, conduct and analysis; 3) ensure efficient research oversight and regulation; 4) Provide accessible research protocols and reports; and 5) develop unbiased and usable research reports, including 17 responsible research practice recommendations. All child health research relevant publications by the 48 original StaR standards' authors over the last decade were identified, and main topic areas were categorized using this framework.

RESULTS

A total of 247 publications were included in the final sample: 100 publications (41%) in domain 1 (3 recommendations), 77 publications (31%) in domain 2 (3), 35 publications (14%) in domain 3 (4), 20 publications (8%) in domain 4 (4), and 15 publications (6%) in domain 5 (3). We identified readily implementable "responsible" research practices to counter child health research waste and improve quality, especially in the areas of patients and families' engagement throughout the research process, developing Core Outcome Sets, and addressing ethics and regulatory oversight issues.

CONCLUSION

While most of the practices are readily implementable, increased awareness of methodological issues and wider guideline uptake is needed to improve child health research. The CHIRP Framework can be used to guide responsible research practices that add value to child health research.

Quality of systematic reviews on timing of complementary feeding for early childhood allergy prevention

BACKGROUND

Only rigorously prepared analyses can provide the highest level of evidence to inform decision-making. Several recent systematic reviews (SRs) examined the hypothesis that the early introduction of specific allergenic complementary foods (CFs) to infants may lead to a lower incidence of one or more allergic outcomes. However, the methodological rigour and quality of reporting of SRs in this area has not yet been systematically evaluated.

METHODS

We comprehensively searched PubMed, Medline (Ovid), and Web of Science Core Collection on 13th January 2022, using a pre-specified and tested search syntax for SRs with RCT evidence on the early introduction of allergenic CFs as a means for allergy prevention in infants and children. We examined the quality and risk of bias (RoB) using AMSTAR-2 and ROBIS tools, examined adherence to the Preferred Reporting Items for SRs and Meta-Analyses (PRISMA), and checked whether certainty of the evidence was assessed.

RESULTS

Twelve SRs were included. Application of both tools resulted in similar overall judgements in terms of direction and extent for nine of the 12 SRs. Nine SRs were found to be of critically low to low quality according to AMSTAR-2 and to be at high RoB according to ROBIS. One SR received a moderate quality rating (AMSTAR-2) and high RoB rating (ROBIS). However, for two SRs, judgements between AMSTAR-2 and ROBIS were at stark variance. Only two SRs fully adhered to the PRISMA checklist. Six SRs evaluated the certainty of the body of RCT evidence. Several SRs failed to consider unpublished studies either by an explicit a priori exclusion or by inadequate search strategies.

CONCLUSIONS

Well-conducted SRs are important for decision-making and informing guideline development, the quality of their methodology should therefore be considered. The methodological rigour and the reporting quality of SRs on the timing of CF for allergy prevention must be improved.

REGISTRATION

https://osf.io/7cs4b .

Definition, harms, and prevention of redundant systematic reviews

BACKGROUND

Along with other types of research, it has been stated that the extent of redundancy in systematic reviews has reached epidemic proportions. However, it was also emphasized that not all duplication is bad, that replication in research is essential, and that it can help discover unfortunate behaviors of scientists. Thus, the question is how to define a redundant systematic review, the harmful consequences of such reviews, and what we could do to prevent the unnecessary amount of this redundancy.

MAIN BODY

There is no consensus definition of a redundant systematic review. Also, it needs to be defined what amount of overlap between systematic reviews is acceptable and not considered a redundancy. One needs to be aware that it is possible that the authors did not intend to create a redundant systematic review. A new review on an existing topic, which is not an update, is likely justified only when it can be shown that the previous review was inadequate, for example, due to suboptimal methodology. Redundant meta-analyses could have scientific, ethical, and economic questions for researchers and publishers, and thus, they should be avoided, if possible. Potential solutions for preventing redundant reviews include the following: (1) mandatory prospective registration of systematic reviews; (2) editors and peer reviewers rejecting duplicate/redundant and inadequate reviews; (3) modifying the reporting checklists for systematic reviews; (4) developing methods for evidence-based research (EBR) monitoring; (5) defining systematic reviews; (6) defining the conclusiveness of systematic reviews; (7) exploring interventions for the adoption of methodological advances; (8) killing off zombie reviews (i.e., abandoned registered reviews); (9) better prevention of duplicate reviews at the point of registration; (10) developing living systematic reviews; and (11) education of researchers.

CONCLUSIONS

Disproportionate redundancy of the same or very similar systematic reviews can lead to scientific, ethical, economic, and societal harms. While it is not realistic to expect that the creation of redundant systematic reviews can be completely prevented, some preventive measures could be tested and implemented to try to reduce the problem. Further methodological research and development in this field will be welcome.

COVID-19 L·OVE repository is highly comprehensive and can be used as a single source for COVID-19 studies

Background and Objective

The coronavirus disease 2019 Living OVerview of Evidence (COVID-19 L·OVE) is a public repository and classification platform for COVID-19 articles. The repository contains more than 430,000 articles as of September 20, 2021 and intends to provide a one-stop shop for COVID-19 evidence. Considering that systematic reviews conduct high-quality searches, this study assesses the comprehensiveness and currency of the repository against the total number of studies in a representative sample of COVID-19 systematic reviews.

Methods

Our sample was generated from all the studies included in the systematic reviews of COVID-19 published during April 2021. We estimated the comprehensiveness of COVID-19 L·OVE repository by determining how many of the individual studies in the sample were included in the COVID-19 L·OVE repository. We estimated the currency as the percentage of studies that was available in the COVID-19 L·OVE repository at the time the systematic reviews conducted their own search.

Results

We identified 83 eligible systematic reviews that included 2,132 studies. COVID-19 L·OVE had an overall comprehensiveness of 99.67% (2,125/2,132). The overall currency of the repository, that is, the proportion of articles that would have been obtained if the search of the reviews was conducted in COVID-19 L·OVE instead of searching the original sources, was 96.48% (2,057/2,132). Both the comprehensiveness and the currency were 100% for randomized trials (82/82).

Conclusion

The COVID-19 L·OVE repository is highly comprehensive and current. Using this repository instead of traditional manual searches in multiple databases can save a great amount of work to people conducting systematic reviews and would improve the comprehensiveness and timeliness of evidence syntheses. This tool is particularly important for supporting living evidence synthesis processes.