Methodological challenges for living systematic reviews conducted during the COVID-19 pandemic: A concept paper

Implications of living evidence formats for coverage decisions in the German health care system

Decision-makers consult systematic reviews and clinical guidelines to make informed coverage decisions based on the current state of evidence. Outdated recommendations in rapidly evolving areas such as lung cancer treatment, are challenging. The COVID-19 pandemic highlighted the need for good decision-making under uncertainty. The descriptive analysis of two samples of evidence bases for evidence synopses to prepare the decision on appropriate comparators shows that living systematic reviews and living clinical guidelines are rare (41/5,463; 0.75%) but present, with COVID-19 being the most common indication. We also describe some characteristics and quality issues of these living formats in the German context. We note an overlap between living and rapid formats, where updates may not adhere to methodological standards in evidence selection, appraisal and formulation of recommendations, or may lack transparency in their methodological processes. The need for critical appraisal of living formats is highlighted as crucial aspect. The production of living systematic reviews and clinical guidelines requires considerable resources and expertise. While there is a need for timeliness in decision making, especially in situations of high uncertainty such as the COVID-19 pandemic, the trade-off between time and quality needs to be balanced. The focus should therefore be on how best to select and process recommendations that are relevant for updating and those that are not. Regularly updated systematic reviews and clinical guidelines that adhere to recommended standards are important for decision-making bodies such as the Federal Joint Committee (G-BA). Transparent documentation of the process and methods used increases confidence in decision-making, even when the evidence base is not perfect.

COVID-19 vaccine evidence monitoring assisted by artificial Intelligence: An emergency system implemented by the Public Health Agency of Canada to capture and describe the trajectory of evolving pandemic vaccine literature

Background

The COVID-19 pandemic resulted in a rapid accumulation of novel vaccine research evidence. As a means to monitor this evidence, the Public Health Agency of Canada (PHAC) created the Evidence eXtraction Team for Research Analysis (EXTRA), which contributed to situational awareness in Canada through a bibliographic repository used to support decision-making by the National Advisory Committee on Immunization. We describe the process by which this literature was identified and catalogued, and provide an overview of characteristics in the identified literature.

Methods

To expedite the process, PHAC leveraged an artificial intelligence (AI) tool to assist in the screening and selection of relevant articles. Literature search results were initially screened by AI, then manually reviewed for relevance. Relevant articles were tagged using controlled vocabulary and stored in a bibliographic repository. This repository was analyzed to identify trends in vaccine research over time according to several key characteristics.

Results

As of December 31, 2023, EXTRA's repository contained 19,050 articles relevant to PHAC's immunization mandate. The majority of these articles (63.9 %) were identified between August 2021 and January 2023, with an average of 20 relevant articles added daily during this period. Nearly 14,000 articles reported on mRNA vaccines. Safety outcomes were most frequently reported (n = 8,289), followed by immunogenicity (n = 7,269) and efficacy/effectiveness (n = 3,246). COVID-19 vaccine literature output started to decrease in mid-2023, two years after the initial dramatic increase in mid-2021.

Conclusions

This hybrid (AI and human) approach was critical for PHAC situational awareness and the development of timely vaccine guidance in Canada during the COVID-19 pandemic. Given the volume of data and analyses required, the AI-augmented processes made this massive undertaking manageable. Analysis of COVID-19 vaccine research patterns supports projections of research volume, type, and rate that will help predict resourcing and information needs to plan future emergency vaccine guidance activities.

A living critical interpretive synthesis to yield a framework on the production and dissemination of living evidence syntheses for decision-making

BACKGROUND

The COVID-19 pandemic has had an unprecedented impact in the global research production and has also increased research waste. Living evidence syntheses (LESs) seek to regularly update a body of evidence addressing a specific question. During the COVID-19 pandemic, the production and dissemination of LESs emerged as a cornerstone of the evidence infrastructure. This critical interpretive synthesis answers the questions: What constitutes an LES to support decision-making?; when should one be produced, updated, and discontinued?; and how should one be disseminated?

METHODS

Searches included the Cochrane Library, EMBASE (Ovid), Health Systems Evidence, MEDLINE (Ovid), PubMed, and Web of Science up to 23 April 2024 and included articles that provide any insights on addressing the compass questions on LESs. Articles were selected and appraised, and their insights extracted. An interpretive and iterative coding process was used to identify relevant thematic categories and create a conceptual framework.

RESULTS

Among the 16,630 non-duplicate records identified, 208 publications proved eligible. Most were non-empirical articles, followed by actual LESs. Approximately one in three articles were published in response to the COVID-19 pandemic. The conceptual framework addresses six thematic categories: (1) what is an LES; (2) what methodological approaches facilitate LESs production; (3) when to produce an LES; (4) when to update an LES; (5) how to make available the findings of an LES; and (6) when to discontinue LES updates.

CONCLUSION

LESs can play a critical role in reducing research waste and ensuring alignment with advisory and decision-making processes. This critical interpretive synthesis provides relevant insights on how to better organize the global evidence architecture to support their production.

TRIAL REGISTRATION

PROSPERO registration: CRD42021241875.

Challenges of consolidating evidence collected during a pandemic and lessons for the future

OBJECTIVE

To illustrate the challenges encountered when gathering rapidly synthesized evidence in response to the coronavirus disease 2019 (COVID-19) pandemic.

METHODS

In this article, we describe the challenges encountered when we performed a systematic literature review (SLR) of randomized controlled trials (RCTs) on the efficacy and safety of treatments for severe COVID-19. The methods of the SLR are described in full, to show the context of our objectives. Then we use the results of the SLR to demonstrate the problems of producing synthesized evidence in this setting.

RESULTS

Various challenges were identified during this SLR. These were primarily a result of heterogeneity in the study methodology of eligible studies. Definitions of the patient populations and outcome measurements were highly variable and the majority of studies demonstrated a high risk of bias, preventing quantitative synthesis of the collated evidence.

CONCLUSION

Consolidating evidence from RCTs evaluating COVID-19 interventions was problematic. Guidance is needed for scenarios with high rapid output in primary research.

Building a sharable literature collection to advance the science and practice of implementation facilitation

Background

Implementation science seeks to produce generalizable knowledge on strategies that promote the adoption and sustained use of evidence-based innovations. Literature reviews on specific implementation strategies can help us understand how they are conceptualized and applied, synthesize findings, and identify knowledge gaps. Although rigorous literature reviews can advance scientific knowledge and facilitate theory development, they are time-consuming and costly to produce. Improving the efficiency of literature review processes and reducing redundancy of effort is especially important for this rapidly developing field. We sought to amass relevant literature on one increasingly used evidence-based strategy, implementation facilitation (IF), as a publicly available resource.

Methods

We conducted a rigorous systematic search of PubMed, CINAHL, and Web of Science citation databases for peer-reviewed, English-language articles with "facilitation" and a combination of other terms published from January 1996 to December 2021. We searched bibliographies of articles published from 1996 to 2015 and identified articles during the full text review that reported on the same study. Two authors screened 3,168 abstracts. After establishing inter-rater reliability, they individually conducted full-text review of 786 relevant articles. A multidisciplinary team of investigators provided recommendations for preparing and disseminating the literature collection.

Findings

The literature collection is comprised of 510 articles. It includes 277 empirical studies of IF and 77 other articles, including conceptual/theoretical articles, literature reviews, debate papers and descriptions of large-scale clinical initiatives. Over half of the articles were published between 2017 and 2021. The collection is publicly available as an Excel file and as an xml file that can be imported into reference management software.

Conclusion

We created a publicly accessible collection of literature about the application of IF to implement evidence-based innovations in healthcare. The comprehensiveness of this collection has the potential to maximize efficiency and minimize redundancy in scientific inquiry about this strategy. Scientists and practitioners can use the collection to more rapidly identify developments in the application of IF and to investigate a wide range of compelling questions on its use within and across different healthcare disciplines/settings, countries, and payer systems. We offer several examples of how this collection has already been used.

[A Continuous Literature Review for Mental Health Surveillance during the Covid-19 Pandemic in Germany - Method, Selected Results and Lessons Learned]

In order to provide an up-to-date overview on changes in population's mental health during the COVID-19 pandemic, a continuous literature review was conducted. Building on a rapid review, systematic and hand searches were conducted monthly until December 31, 2022. Studies were assessed for observation periods, risk of bias and outcomes. Trends in depressive symptoms in adults were summarized by vote counting. 102 publications were included from 62 studies in the adult population. Studies declined over the course of the pandemic. Overall, 37% of the studies and 56% of the publications can assess trends in the population reliably. Among evidence for changes in depressive symptoms deteriorations predominated at last. The heterogeneity of results published by the end of 2022 limits evidence syntheses. Evidence of deterioration requires further surveillance. A continuous review can indicate evidence gaps at an early stage.

Needs and feasibility of living systematic reviews (LSRs): Experience from LSRs on COVID-19 vaccine effectiveness

During 2021 and 2023, a team of researchers at the Robert Koch Institute (RKI) and partnering institutions conducted two living systematic reviews (LSRs) on the effectiveness of COVID-19 vaccines in different age groups to inform recommendations of the Standing Committee on Vaccination in Germany (Ständige Impfkommission, STIKO). Based on our experience from the realization of these LSRs, we developed certain criteria to assess the needs and feasibility of conducting LSRs. Combining these with previously established criteria, we developed the following set to inform future planning of LSRs for STIKO: Needs criterion (N)1: Relevance of the research question, N2: Certainty of evidence (CoE) at baseline; N3: Expected need for Population-Intervention-Comparator-Outcome (PICO) adaptations; N4: Expected new evidence over time; N5: Expected impact of new evidence on CoE; Feasibility criterion (F)1: Availability of sufficient human resources; F2: Feasibility of timely dissemination of the results to inform decision-making. For each criterion we suggest rating options which may support the decision to conduct an LSR or other forms of evidence synthesis when following the provided flowchart. The suggested criteria were developed on the basis of the experiences from exemplary reviews in a specific research field (i.e., COVID-19 vaccination), and did not follow a formal development or validation process. However, these criteria might also be useful to assess whether questions from other research fields can and should be answered using the LSR approach, or assist in determining whether the use of an LSR is sensible and feasible for specific questions in health policy and practice.

Experience report of two living systematic Cochrane reviews on COVID-19

INTRODUCTION

Up-to-date systematic reviews (SRs) are essential for making evidence-based decisions. During the 2019 coronavirus (COVID-19) pandemic, there was a particular need for up-to-date evidence, making the living systematic review (LSR) approach an appropriate review type. However, this approach poses certain challenges.

OBJECTIVE AND OUTLINE

We aim to provide practice insights and report challenges that we faced while conducting two Cochrane LSRs on COVID-19 treatments with (i) convalescent plasma and (ii) systemic corticosteroids. We address our objective with an experience report and share challenges of the following components based on Iannizzi et al. (2022): study design, publication types, intervention/comparator, outcomes, search strategy, review updates and transparent reporting of differences between review updates.

RESULTS

Regarding the study design, the plasma LSR included different study designs because RCT data were not available at the beginning of the pandemic, whereas for the corticosteroids LSR, which started several months later, RCT data were already available. The challenges in both LSRs included the publication types (preprints were included with caution) and the intervention/comparator, for instance the unavailability of standard of care for either LSR, or SARS-CoV-2 variants occurrence. Further challenges in both LSRs occurred in the components "outcome sets" (which had to be adjusted) and "literature search". The decision criteria for updating were based on important studies and available resources in both LSRs and policy relevance in the plasma LSR. Transparent reporting of the differences between the various update versions were discussed for both LSRs.

DISCUSSION AND CONCLUSION

In summary, there are similarities and differences regarding challenges of review components for both LSRs. It is important to keep in mind that the two LSR examples presented here were conducted in the wake of the COVID-19 pandemic. Therefore, many of the challenges are attributable to the pandemic and are not specific to LSRs, such as constant adjustments of the outcome sets or changes in the database search. Nevertheless, we believe that some of these aspects are helpful for LSR authors and are applicable to other LSRs outside the pandemic context, particularly in areas where new evidence is rapidly emerging.

Proposed triggers for retiring a living systematic review

Living systematic reviews (LSRs) are systematic reviews that are continually updated, incorporating relevant new evidence as it becomes available. LSRs are critical for decision-making in topics where the evidence continues to evolve. It is not feasible to continue to update LSRs indefinitely; however, guidance on when to retire LSRs from the living mode is not clear. We propose triggers for making such a decision. The first trigger is to retire LSRs when the evidence becomes conclusive for the outcomes that are required for decision-making. Conclusiveness of evidence is best determined based on the GRADE certainty of evidence construct, which is more comprehensive than solely relying on statistical considerations. The second trigger to retire LSRs is when the question becomes less pertinent for decision-making as determined by relevant stakeholders, including people affected by the problem, healthcare professionals, policymakers and researchers. LSRs can also be retired from a living mode when new studies are not anticipated to be published on the topic and when resources become unavailable to continue updating. We describe examples of retired LSRs and apply the proposed approach using one LSR about adjuvant tyrosine kinase inhibitors in high-risk renal cell carcinoma that we retired from a living mode and published its last update.

How to update a living systematic review and keep it alive during a pandemic: a practical guide

BACKGROUND

The covid-19 pandemic has highlighted the role of living systematic reviews. The speed of evidence generated during the covid-19 pandemic accentuated the challenges of managing high volumes of research literature.

METHODS

In this article, we summarise the characteristics of ongoing living systematic reviews on covid-19, and we follow a life cycle approach to describe key steps in a living systematic review.

RESULTS

We identified 97 living systematic reviews on covid-19, published up to 7th November 2022, which focused mostly on the effects of pharmacological interventions (n = 46, 47%) or the prevalence of associated conditions or risk factors (n = 30, 31%). The scopes of several reviews overlapped considerably. Most living systematic reviews included both observational and randomised study designs (n = 45, 46%). Only one-third of the reviews has been updated at least once (n = 34, 35%). We address practical aspects of living systematic reviews including how to judge whether to start a living systematic review, methods for study identification and selection, data extraction and evaluation, and give recommendations at each step, drawing from our own experience. We also discuss when it is time to stop and how to publish updates.

CONCLUSIONS

Methods to improve the efficiency of searching, study selection, and data extraction using machine learning technologies are being developed, their performance and applicability, particularly for reviews based on observational study designs should improve, and ways of publishing living systematic reviews and their updates will continue to evolve. Finally, knowing when to end a living systematic review is as important as knowing when to start.

Pragmatic and open science-based solution to a current problem in the reporting of living systematic reviews

Living systematic reviews (LSRs) are an increasingly common approach to keeping reviews up to date, in which new relevant studies are incorporated as they become available, so as to inform healthcare policy and practice in a timely manner. While journal publishers have been exploring the publication of LSRs using different updating and publishing approaches, readers cannot currently assess if the evidence underpinning a published LSR is up to date, as neither the search details, the selection process, nor the list of identified studies is made available between the publication of updates. We describe a new method to transparently report the living evidence surveillance process that occurs between published LSR versions. We use the example of the living Cochrane Review on nirmatrelvir combined with ritonavir (Paxlovid) for preventing and treating COVID-19 to illustrate how this can work in practice. We created a publicly accessible spreadsheet on the Open Science Framework platform, linking to the living Cochrane Review, that details the search and study selection process, enabling readers to track the progress of eligible ongoing or completed studies. Further automation of the evidence surveillance process should be explored.

Methodological quality and reporting quality of COVID-19 living systematic review: a cross-sectional study

OBJECTIVES

The main objective of this study is to evaluate the methodological quality and reporting quality of living systematic reviews (LSRs) on Coronavirus disease 2019 (COVID-19), while the secondary objective is to investigate potential factors that may influence the overall quality of COVID-19 LSRs.

METHODS

Six representative databases, including Medline, Excerpta Medica Database (Embase), Cochrane Library, China national knowledge infrastructure (CNKI), Wanfang Database, and China Science, Technology Journal Database (VIP) were systematically searched for COVID-19 LSRs. Two authors independently screened articles, extracted data, and then assessed the methodological and reporting quality of COVID-19 LSRs using the "A Measurement Tool to Assess systematic Reviews-2" (AMSTAR-2) tool and "Preferred Reporting Items for Systematic reviews and Meta-Analyses" (PRISMA) 2020 statement, respectively. Univariate linear regression and multivariate linear regression were used to explore eight potential factors that might affect the methodological quality and reporting quality of COVID-19 LSRs.

RESULTS

A total of 64 COVID-19 LSRs were included. The AMSTAR-2 evaluation results revealed that the number of "yes" responses for each COVID-19 LSR was 13 ± 2.68 (mean ± standard deviation). Among them, 21.9% COVID-19 LSRs were rated as "high", 4.7% as "moderate", 23.4% as "low", and 50% as "critically low". The evaluation results of the PRISMA 2020 statement showed that the sections with poor adherence were methods, results and other information. The number of "yes" responses for each COVID-19 LSR was 21 ± 4.18 (mean ± standard deviation). The number of included studies and registration are associated with better methodological quality; the number of included studies and funding are associated with better reporting quality.

CONCLUSIONS

Improvement is needed in the methodological and reporting quality of COVID-19 LSRs. Researchers conducting COVID-19 LSRs should take note of the quality-related factors identified in this study to generate evidence-based evidence of higher quality.

[Concepts to support dynamic updating processes of guidelines and their practical implementation-Systematic literature review]

BACKGROUND

Clinical practice guidelines (CPG), which are crucial for evidence-based healthcare, should be kept up to date, especially on topics where emerging evidence could modify one of the recommendations with implications for the healthcare service; however, an updating process, which is practicable for guideline developers as well as users represents a challenge.

OBJECTIVE

This article provides an overview of the currently discussed methodological approaches for the dynamic updating of guidelines and systematic reviews.

MATERIAL AND METHODS

As part of a scoping review, a literature search was conducted in the databases MEDLINE, EMBASE (via Ovid), Scopus, Epistemonikos, medRxiv, as well as in study and guideline registers. Concepts on the dynamic updating of guidelines and systematic reviews or dynamically updated guidelines and systematic reviews or their protocols published in English or German were included.

RESULTS

The publications included most frequently described the following main processes that must be adapted in dynamic updating processes 1) the establishment of continuously working guideline groups, 2) networking between guidelines, 3) the formulation and application of prioritization criteria, 4) the adaptation of the systematic literature search and 5) the use of software tools for increased efficiency and digitalization of the guidelines.

CONCLUSION

The transformation to living guidelines requires a change in the needs for temporal, personnel and structural resources. The digitalization of guidelines and the use of software to increase efficiency are necessary instruments, but alone do not guarantee the realization of living guidelines. A process in which dissemination and implementation must also be integrated is necessary. Standardized best practice recommendations on the updating process are still lacking.

Horizon scanning, rapid reviews and living evidence to support decision-making: lessons from the work of the Critical Intelligence Unit in New South Wales, Australia during the COVID-19 pandemic

The COVID-19 pandemic has seen an increase in rapidly disseminated scientific evidence and highlighted that traditional evidence synthesis methods, such as time and resource intensive systematic reviews, may not be successful in responding to rapidly evolving policy and practice needs. In New South Wales (NSW) Australia, the Critical Intelligence Unit (CIU) was established early in the pandemic and acted as an intermediary organisation. It brought together clinical, analytical, research, organisational and policy experts to provide timely and considered advice to decision-makers. This paper provides an overview of the functions, challenges and future implications of the CIU, particularly the Evidence Integration Team. Outputs from the Evidence Integration Team included a daily evidence digest, rapid evidence checks and living evidence tables. These products have been widely disseminated and used to inform policy decisions in NSW, making valuable impacts. Changes and innovations to evidence generation, synthesis and dissemination in response to the COVID-19 pandemic provide an opportunity to shift the way evidence is used in future. The experience and methods of the CIU have potential to be adapted and applied to the broader health system nationally and internationally.

From standard systematic reviews to living systematic reviews

Systematic reviews (SRs) have become a central tool for evidence-based health care over the last 30 years. The number of SRs being published has increased steadily. However, concerns have been raised regarding the duplication of work, methodological flaws and the currency of many systematic reviews, also in the context of the COVID-19 pandemic. Living systematic reviews (LSRs) offer a new approach to updating systematic reviews, particularly in high-priority research fields that face the challenge of dynamically evolving and sometimes uncertain evidence. Continual updates serve to ensure that LSRs remain current and methodologically rigorous. As a new element of the evidence ecosystem, LSRs can inform living guidelines and recommendations, user-adapted formats, decisions at the patient and system level as well as gaps in primary research.

Conceptualizing the reporting of living systematic reviews

OBJECTIVES

As part of an effort to develop an extension of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 2020 statement for living systematic reviews (LSRs), we discuss conceptual issues relevant to the reporting of LSRs and highlight a few challenges.

METHODS

Discussion of conceptual issues based on a scoping review of the literature and discussions among authors.

RESULTS

We first briefly describe aspects of the LSR production process relevant to reporting. The production cycles differ by whether the literature surveillance identifies new evidence and whether newly identified evidence is judged to be consequential. This impacts the timing, content, and format of LSR versions. Second, we discuss four types of information that are specific to the reporting of LSRs: justification for adopting the living mode, LSR specific methods, changes between LSR versions, and LSR updating status. We also discuss the challenge of conveying changes between versions to the reader. Third, we describe two commonly used reporting formats of LSRs: full and partial reports. Although partial reports are easier to produce and publish, they lead to the scattering of information across different versions. Full reports ensure the completeness of reporting. We discuss the implications for the extension of the PRISMA 2020 statement for LSRs.

CONCLUSION

We argue that a dynamic publication platform would facilitate complete and timely reporting of LSRs.

Paper 4: Search methods and approaches for living guidelines

OBJECTIVES

To describe the key features of a continual evidence surveillance process that can be implemented for living guidelines and to outline the considerations and trade-offs in adopting different approaches.

STUDY DESIGN AND SETTING

Members of the Australian Living Evidence Consortium (ALEC), National Institute of Health and Care Excellence (NICE), and the US GRADE Network (USGN) shared their practical experiences of and approaches to establishing surveillance systems for living guidelines. We identified several common components of evidence surveillance and listed the key features and considerations for each component drawn from case studies, highlighting differences with standard guidelines.

RESULTS

We developed guidance that covers the initial information needed to support decisions around suitability for living mode and the practical considerations in setting up continual search surveillance systems (search frequency, sources to search, use of automation, reporting the search, ongoing resources, and evaluation). The case studies draw on our experiences with developing guidelines for COVID-19, as well as for other conditions such as stroke and diabetes, and cover a range of practical approaches, including the use of automation.

CONCLUSION

This paper highlights different approaches to continual evidence surveillance that can be implemented in living guidelines.

When we shouldn't borrow information from an existing network of trials for planning a new trial

Introduction: To achieve higher power or increased precision for a new trial, methods based on updating network meta-analysis (NMA) have been proposed by researchers. However, this approach could potentially lead to misinterpreted results and misstated conclusions. This work aims to investigate the potential inflation of type I error risk when a new trial is conducted only when, based on a p-value of the comparison in the existing network, a "promising" difference between two treatments is noticed. Methods: We use simulations to evaluate the scenarios of interest. In particular, a new trial is to be conducted independently or depending on the results from previous NMA in various scenarios. Three analysis methods are applied to each simulation scenario: with the existing network, sequential analysis and without the existing network. Results: For the scenario that the new trial will be conducted only when a promising finding (p-value <5%) is indicated by the existing network, the type I error risk increased dramatically (38.5% in our example data) when analyzed with the existing network and sequential analysis. The type I error is controlled at 5% when analyzing the new trial without the existing network. Conclusion: If the intention is to combine a trial result with an existing network of evidence, or if it is expected that the trial will eventually be included in a network meta-analysis, then the decision that a new trial is performed should not depend on a statistically "promising" finding indicated by the existing network.

[Evidence syntheses in public health: An overview]

BACKGROUND

Since the beginning of the COVID-19 pandemic, there has been a high demand for rapid evidence syntheses to answer urgent public health questions. This article provides an overview of different types of reviews for public health questions and a synthesis of existing recommendations for the preparation of reviews. The aim is to support the planning of one's own review and the critical evaluation of published reviews.

METHODS

The basis of this summary is an extensive search for guidelines and recommendations for different review types. Furthermore, internal journal clubs were held to determine knowledge needs and to critically discuss the various review types. Relating to the dissemination of results, fact sheets were developed for the individual review types including the most important information, prerequisites and work steps, as well as a decision tree for identifying the appropriate review type for the respective question.

RESULTS

Of the review types identified, Systematic, Rapid, Scoping, Umbrella, and Narrative Reviews were considered in more detail because they are particularly relevant to public health issues. Together with scoping and umbrella reviews, systematic reviews have the highest resource requirements due to the demands for extensive, systematic evidence synthesis and reproducibility. Rapid methods can accelerate the review process, for example by a very narrowly formulated question, a limited literature search, or the execution of certain steps by one instead of two persons.

DISCUSSION

Systematic Reviews may be considered as the gold standard, but they were developed primarily for clinical questions relating to interventions. This article, however, focusses on review types that consider the diversity of questions as well as the predominant use of quantitative methods in the field of public health. The fact sheets developed and the decision tree should enable low-threshold access to reviews while linking the perspectives of research and resource planning. They complement existing guidelines and recommendations.

CONCLUSION

To answer the diverse spectrum of public health questions, various types of reviews with various requirements and approaches are available. Given this diversity, a systematic introduction can be helpful for researchers planning or assessing a review.

Methods for developing and reporting living evidence synthesis

BACKGROUND AND OBJECTIVES

Identify currently methodological aspects proposed for planning, conducting, and reporting living evidence (LE) synthesis. Develop a preliminary checklist of key LE synthesis elements.

METHODS

A survey of methodological articles describing or analyzing methods for the design, conduction, or reporting of LE synthesis.

RESULTS

Twelve methodological articles were identified and analyzed. Key elements were related to: i) definition of LE and characteristics of LE synthesis, ii) methods and tools for the living process, iii) new evidence integration (methods and considerations), iv) updates dissemination and publication, v) revisiting living parameters, and vi) protocol considerations for LE synthesis.

CONCLUSION

This survey displays basic methodological concepts that can drive the development of LE synthesis and identifies specific aspects with opportunities for development. The potential impact of the LE approach calls for a change in the current evidence synthesis updating processes to more open, collaborative, transparent, and efficient systems. LE approaches also challenge journal editors to shift toward more efficient processes for synthesis update dissemination, which minimizes the risks of reliability of published information.

Community engagement to support COVID-19 vaccine uptake: a living systematic review protocol

INTRODUCTION

Widespread vaccination against COVID-19 is one of the most effective ways to control, and ideally, end the global COVID-19 pandemic. Vaccine hesitancy and vaccine rates vary widely across countries and populations and are influenced by complex sociocultural, political, economic and psychological factors. Community engagement is an integral strategy within immunisation campaigns and has been shown to improve vaccine acceptance. As evidence on community engagement to support COVID-19 vaccine uptake is emerging and constantly changing, research that lessens the knowledge-to-practice gap by providing regular and up-to-date evidence on current best-practice is essential.

METHODS AND ANALYSIS

A living systematic review will be conducted which includes an initial systematic review and bimonthly review updates. Searching and screening for the review and subsequent updates will be done in four streams: a systematic search of six databases, grey literature review, preprint review and citizen sourcing. The screening will be done by a minimum of two reviewers at title/abstract and full-text in Covidence, a systematic review management software. Data will be extracted across predefined fields in an excel spreadsheet that includes information about article characteristics, context and population, community engagement approaches, and outcomes. Synthesis will occur using the convergent integrated approach. We will explore the potential to quantitatively synthesise primary outcomes depending on heterogeneity of the studies.

ETHICS AND DISSEMINATION

The initial review and subsequent bimonthly searches and their results will be disseminated transparently via open-access methods. Quarterly briefs will be shared on the reviews' social media platforms and across other interested networks and repositories. A dedicated web link will be created on the Community Health-Community of Practice site for sharing findings and obtaining feedback. A mailing list will be developed and interested parties can subscribe for updates.

PROSPERO REGISTRATION NUMBER

CRD42022301996.

Characteristics of Living Systematic Review for COVID-19

Purpose

The systematic review aims to analyze and summarize the characteristics of living systematic review (LSR) for coronavirus disease 2019 (COVID-19).

Methods

Six databases including Medline, Excerpta Medica (Embase), Cochrane Library, China National Knowledge Infrastructure (CNKI), Wanfang Database and China Science, and Technology Journal Database (VIP), were searched as the source of basic information and methodology of LSR. Descriptive analytical methods were used to analyze the included COVID-19 LSRs, and the study characteristics of COVID-19 LSRs were further assessed.

Results

Sixty-four COVID-19 LSRs were included. Eighty-nine point one percent of LSRs were published on Science Citation Index (SCI) journals, and 64.1% publication with an impact factor (IF) >5 and 17.2% with an IF >15 among SCI journals. The first unit of the published LSRs for COVID-19 came from 19 countries, with the largest contribution from the UK (17.2%, 11/64). Forty point six percent of LSRs for COVID-19 were related to therapeutics topic which was considered the most concerned perspective for LSRs for COVID-19. Seventy-six point six percent of LSRs focused on the general population, with less attention to children, pregnant women and the elderly. However, the LSR for COVID-19 was reported incomplete on “living” process, including 40.6% of studies without search frequency, 79.7% of studies without screening frequency, 20.3% of studies without update frequency, and 65.6% of studies without the timing or criteria of transitioning LSR out of living mode.

Conclusion

Although researchers in many countries have applied LSRs to COVID-19, most of the LSRs for COVID-19 were incomplete in reporting on the “living” process and less focused on special populations. This could reduce the confidence of health-care providers and policy makers in the results of COVID-19 LSR, thereby hindering the translation of evidence on COVID-19 LSR into clinical practice. It was necessary to explicitly enact preferred reporting items for systematic reviews and meta-analyses (PRISMA) to improve the reporting quality of LSR and support ongoing efforts of therapeutics research for special patients with COVID-19.

Lessons learnt: Undertaking rapid reviews on public health and social measures during a global pandemic

Public health and social measures (PHSM) have been central to the COVID‐19 response. Consequently, there has been much pressure on decision‐makers to make evidence‐informed decisions and on researchers to synthesize the evidence regarding these measures. This article describes our experiences, responses and lessons learnt regarding key challenges when planning and conducting rapid reviews of PHSM during the COVID‐19 pandemic. Stakeholder consultations and scoping reviews to obtain an overview of the evidence inform the scope of reviews that are policy‐relevant and feasible. Multiple complementary reviews serve to examine the benefits and harms of PHSM across different populations and contexts. Conceiving reviews of effectiveness as adaptable living reviews helps to respond to evolving evidence needs and an expanding evidence base. An appropriately skilled review team and good planning, coordination and communication ensures smooth and rigorous processes and efficient use of resources. Scientific rigor, the practical implications of PHSM‐related complexity and likely time savings should be carefully weighed in deciding on methodological shortcuts. Making the best possible use of modeling studies represents a particular challenge, and methods should be carefully chosen, piloted and implemented. Our experience raises questions regarding the nature of rapid reviews and regarding how different types of evidence should be considered in making decisions about PHSM during a global pandemic. We highlight the need for readily available protocols for conducting studies on the effectiveness, unintended consequences and implementation of PHSM in a timely manner, as well as the need for rapid review standards tailored to “rapid” versus “emergency” mode reviewing.

Extension of the PRISMA 2020 statement for living systematic reviews (LSRs): protocol

Background: While the PRISMA 2020 statement is intended to guide the reporting of original systematic reviews, updated systematic reviews, and living systematic reviews (LSRs), its explanation and elaboration document notes that additional considerations for updated systematic reviews and LSRs may need to be addressed. This paper reports the protocol for developing an extension of the PRISMA 2020 statement for LSRs. Methods: We will follow the EQUATOR Network's guidance for developing health research reporting guidelines. We will review the literature to identify possible items of the PRISMA 2020 checklist that need modification, as well as new items that need to be added. Then, we will survey representatives of different stakeholder groups for their views on the proposed modifications of the PRISMA 2020 checklist. We will summarize, present, and discuss the results of the survey in an online meeting, aiming to reach consensus on the content of the LSR extension. We will then draft the checklist, explanation and elaboration for each item, and flow diagram for the PRISMA 2020 extension. Then, we will share these initial documents with stakeholder representatives for final feedback and approval. Discussion: We anticipate that the PRISMA 2020 extension for LSRs will benefit LSR authors, editors, and peer reviewers of LSRs, as well as different users of LSRs, including guideline developers, policy makers, healthcare providers, patients, and other stakeholders.