Scoping review of COVID-19-related systematic reviews and meta-analyses: can we really have confidence in their results?

What can we learn from historical pandemics? A systematic review of the literature

What are the insights from historical pandemics for policymaking today? We carry out a systematic review of the literature on the impact of pandemics that occurred since the Industrial Revolution and prior to Covid-19. Our literature searches were conducted between June 2020 and September 2023, with the final review encompassing 169 research papers selected for their relevance to understanding either the demographic or economic impact of pandemics. We include literature from across disciplines to maximise our knowledge base, finding many relevant articles in journals which would not normally be on the radar of social scientists. Our review identifies two gaps in the literature: (1) the need to study pandemics and their effects more collectively rather than looking at them in isolation; and (2) the need for more study of pandemics besides 1918 Spanish Influenza, especially milder pandemic episodes. These gaps are a consequence of academics working in silos, failing to draw on the skills and knowledge offered by other disciplines. Synthesising existing knowledge on pandemics in one place provides a basis upon which to identify the lessons in preparing for future catastrophic disease events.

Are COVID-19 systematic reviews up to date and can we tell? A cross-sectional study

BACKGROUND

COVID-19 led to a rapid acceleration in the number of systematic reviews. Readers need to know how up to date evidence is when selecting reviews to inform decisions. This cross-sectional study aimed to evaluate how easily the currency of COVID-19 systematic reviews published early in the pandemic could be determined and how up to date these reviews were at the time of publication.

METHODS

We searched for systematic reviews and meta-analyses relevant to COVID-19 added to PubMed in July 2020 and January 2021, including any that were first published as preprints. We extracted data on the date of search, number of included studies, and date first published online. For the search date, we noted the format of the date and where in the review this was reported. A sample of non-COVID-19 systematic reviews from November 2020 served as a comparator.

RESULTS

We identified 246 systematic reviews on COVID-19. In the abstract of these reviews, just over half (57%) reported the search date (day/month/year or month/year) while 43% failed to report any date. When the full text was considered, the search date was missing from 6% of reviews. The median time from last search to publication online was 91 days (IQR 63-130). Time from search to publication was similar for the subset of 15 rapid or living reviews (92 days) but shorter for the 29 reviews published as preprints (37 days). The median number of studies or publications included per review was 23 (IQR 12-40). In the sample of 290 non-COVID SRs, around two-thirds (65%) reported the search date while a third (34%) did not include any date in the abstract. The median time from search to publication online was 253 days (IQR 153-381) and each review included a median of 12 studies (IQR 8-21).

CONCLUSIONS

Despite the context of the pandemic and the need to easily ascertain the currency of systematic reviews, reporting of the search date information for COVID-19 reviews was inadequate. Adherence to reporting guidelines would improve the transparency and usefulness of systematic reviews to users.

Potential limitations in systematic review studies assessing the effect of the main intervention for treatment/therapy of COVID-19 patients: An overview

Background

Although several studies have assessed the safety, efficacy, and effectiveness of interventions in treating the COVID-19, many of them have limitations that can have an immense impact on their results. This study aims to assess the potential limitations in systematic reviews (SRs) that evaluate the effect of interventions on the treatment of the COVID-19.

Methods

PubMed, Scopus, and Web of Sciences (WOS) databases were searched from inception to January 1, 2022. All systematic reviews investigated the effectiveness, efficacy, safety, and outcome of the main intervention (Favipiravir, Remdesivir, Hydroxychloroquine, Ivermectin, Lopinavir/Ritonavir, or Tocilizumab) for the treatment of COVID-19 patients and reported the potential limitations of the included studies. We assessed the quality of the included studies using the Quality Assessment Tool (QAT) for review articles. We conducted a content analysis and prepared a narrative summary of the limitations.

Results

Forty-six studies were included in this review. Ninety one percent of the included studies scored as strong quality and the remaining (9%) as moderate quality. Only 29.7% of the included systematic reviews have a registered protocol. 26% of the included studies mentioned a funding statement. The main limitations of the included studies were categorized in 10 domains: sample size, heterogeneity, follow-up, treatment, including studies, design, definitions, synthesis, quality, and search.

Conclusion

Various limitations have been reported in all the included studies. Indeed, the existence of limitations in studies can affect their results, therefore, identifying these limitations can help researchers design better studies. As a result, stronger studies with more reliable results will be reported and disseminated. Further research on COVID-19 SRs is essential to improve research quality and also, efficiency among scientists across the world.

Characteristics of registered and published systematic reviews focusing on the prevention of COVID-19: a meta-research study

OBJECTIVE

We investigated characteristics of systematic reviews (SRs) assessing measures to prevent COVID-19 by (1) identifying SR registrations in Prospective Register of Systematic Reviews (PROSPERO), (2) identifying published SRs in COVID-19 Living Overview of the Evidence (L-OVE) and (3) estimating the proportion of PROSPERO registrations published as full SR between 8 and 16 months after registration.

STUDY DESIGN

This meta-research study is part of the German CEOsys project, www.covid-evidenz.de. We searched PROSPERO entries registered between 1 January 2020 and 31 August 2020, and we searched COVID-19 L-OVE for published SRs (search date: 5 May 2021) focusing on measures to prevent COVID-19 and SARS-CoV-2 transmission. The two samples were screened for eligibility and key characteristics were extracted and summarised.

RESULTS

Of 612 PROSPERO registrations, 47 focused on prevention and were included. The preventive measures included public health interventions (20), followed by personal protective equipment (10), vaccinations (9) and others (8). In total, 13 of 47 (28%) PROSPERO registrations had been published as full SR (as preprint only (6), as peer-reviewed article only (6), as preprint and peer-reviewed article (1)). Median time between PROSPERO registration and publication was 5 months for peer-reviewed SRs and 2 months for preprints.Of the 2182 entries identified in COVID-19 L-OVE, 51 published SRs focused on prevention and were included. Similar to the PROSPERO sample, most published SRs focused on public health interventions (21). The number of included primary studies ranged between 0 and 64 (median: 7). Nine published SRs did not include any studies because of a lack of primary studies.

CONCLUSION

Considering the urgent information needs of policymakers and the public, our findings reveal the high-speed publication of preprints and lack of primary studies in the beginning of the COVID-19 crisis. Further meta-research on COVID-19 SRs is important to improve research efficiency among researchers across the world.

PROSPERO REGISTRATION NUMBER

CRD42021240423.