The Use of Rapid Review Methods for the U.S. Preventive Services Task Force

Rapid review: A review of methods and recommendations based on current evidence

Rapid review (RR) could accelerate the traditional systematic review (SR) process by simplifying or omitting steps using various shortcuts. With the increasing popularity of RR, numerous shortcuts had emerged, but there was no consensus on how to choose the most appropriate ones. This study conducted a literature search in PubMed from inception to December 21, 2023, using terms such as "rapid review" "rapid assessment" "rapid systematic review" and "rapid evaluation". We also scanned the reference lists and performed citation tracking of included impact studies to obtain more included studies. We conducted a narrative synthesis of all RR approaches, shortcuts and studies assessing their effectiveness at each stage of RRs. Based on the current evidence, we provided recommendations on utilizing certain shortcuts in RRs. Ultimately, we identified 185 studies focusing on summarizing RR approaches and shortcuts, or evaluating their impact. There was relatively sufficient evidence to support the use of the following shortcuts in RRs: limiting studies to those published in English-language; conducting abbreviated database searches (e.g., only searching PubMed/MEDLINE, Embase, and CENTRAL); omitting retrieval of grey literature; restricting the search timeframe to the recent 20 years for medical intervention and the recent 15 years for reviewing diagnostic test accuracy; conducting a single screening by an experienced screener. To some extent, the above shortcuts were also applicable to SRs. This study provided a reference for future RR researchers in selecting shortcuts, and it also presented a potential research topic for methodologists.

Rapid Reviews Methods Series: Involving patient and public partners, healthcare providers and policymakers as knowledge users

Rapid reviews (RRs) are a helpful evidence synthesis tool to support urgent and emergent decision-making in healthcare. RRs involve abbreviating systematic review methods and are conducted in a condensed timeline to meet the decision-making needs of organisations or groups that commission them. Knowledge users (KUs) are those individuals, typically patient and public partners, healthcare providers, and policy-makers, who are likely to use evidence from research, including RRs, to make informed decisions about health policies, programmes or practices. However, research suggests that KU involvement in RRs is often limited or overlooked, and few RRs include patients as KUs. Existing RR methods guidance advocates involving KUs but lacks detailed steps on how and when to do so. This paper discusses the importance of involving KUs in RRs, including patient and public involvement to ensure RRs are fit for purpose and relevant for decision-making. Opportunities to involve KUs in planning, conduct and knowledge translation of RRs are outlined. Further, this paper describes various modes of engaging KUs during the review lifecycle; key considerations researchers should be mindful of when involving distinct KU groups; and an exemplar case study demonstrating substantive involvement of patient partners and the public in developing RRs. Although involving KUs requires time, resources and expertise, researchers should strive to balance 'rapid' with meaningful KU involvement in RRs. This paper is the first in a series led by the Cochrane Rapid Reviews Methods Group to further guide general RR methods.

What are the best methods for rapid reviews of the research evidence? A systematic review of reviews and primary studies

Rapid review methodology aims to facilitate faster conduct of systematic reviews to meet the needs of the decision-maker, while also maintaining quality and credibility. This systematic review aimed to determine the impact of different methodological shortcuts for undertaking rapid reviews on the risk of bias (RoB) of the results of the review. Review stages for which reviews and primary studies were sought included the preparation of a protocol, question formulation, inclusion criteria, searching, selection, data extraction, RoB assessment, synthesis, and reporting. We searched 11 electronic databases in April 2022, and conducted some supplementary searching. Reviewers worked in pairs to screen, select, extract data, and assess the RoB of included reviews and studies. We included 15 systematic reviews, 7 scoping reviews, and 65 primary studies. We found that several commonly used shortcuts in rapid reviews are likely to increase the RoB in the results. These include restrictions based on publication date, use of a single electronic database as a source of studies, and use of a single reviewer for screening titles and abstracts, selecting studies based on the full-text, and for extracting data. Authors of rapid reviews should be transparent in reporting their use of these shortcuts and acknowledge the possibility of them causing bias in the results. This review also highlights shortcuts that can save time without increasing the risk of bias. Further research is needed for both systematic and rapid reviews on faster methods for accurate data extraction and RoB assessment, and on development of more precise search strategies.

Evaluation of the interim Cochrane rapid review methods guidance-A mixed-methods study on the understanding of and adherence to the guidance

The Cochrane Rapid Review Methods Group (RRMG) first released interim guidance in March 2020 to support authors in conducting rapid reviews (RRs). The objective of this mixed-methods study was to assess the adherence and investigate authors' understanding of the RRMG guidance. We identified all documents citing the Interim Cochrane RRMG guidance up to February 17, 2022 and performed an exploratory adherence analysis. We interviewed 20 RR authors to assess the recommendations' comprehensibility and reasons for any deviations. Further, we surveyed nine authors of COVID-19-related Cochrane reviews for their reasons for not conducting a RR. We analyzed 128 RRs (111 non-Cochrane, 17 Cochrane) that cited the RRMG guidance. Several recommendations were not followed by a large proportion of RR authors such as stepwise approach to study design inclusion or peer review of search strategies, whereas others were exceeded, for example, dual independent screening of abstracts/full texts. The most reported reasons for deviating from the guidance were time constraints, unclarities in the recommended approach, or inapplicability to the specific RR. Overall, the guidance was viewed as user-friendly; however, without pre-existing knowledge of systematic review (SR) conduct, the application was perceived as difficult. The main reasons for conducting a full SR over a RR were late availability of the guidance, preset mandate to conduct a SR, uncertainty regarding methodological distinctions between SR and RR, and inapplicability to the evidence base. Clarifications are warranted throughout the Interim Cochrane RRMG guidance to ensure that users with various experience levels can understand and apply its recommendations accordingly.

Rapid literature review: definition and methodology

Introduction: A rapid literature review (RLR) is an alternative to systematic literature review (SLR) that can speed up the analysis of newly published data. The objective was to identify and summarize available information regarding different approaches to defining RLR and the methodology applied to the conduct of such reviews. Methods: The Medline and EMBASE databases, as well as the grey literature, were searched using the set of keywords and their combination related to the targeted and rapid review, as well as design, approach, and methodology. Of the 3,898 records retrieved, 12 articles were included. Results: Specific definition of RLRs has only been developed in 2021. In terms of methodology, the RLR should be completed within shorter timeframes using simplified procedures in comparison to SLRs, while maintaining a similar level of transparency and minimizing bias. Inherent components of the RLR process should be a clear research question, search protocol, simplified process of study selection, data extraction, and quality assurance. Conclusions: There is a lack of consensus on the formal definition of the RLR and the best approaches to perform it. The evidence-based supporting methods are evolving, and more work is needed to define the most robust approaches.

Approaches to enabling rapid evaluation of innovations in health and social care: a scoping review of evidence from high-income countries

OBJECTIVE

The COVID-19 pandemic increased the demand for rapid evaluation of innovation in health and social care. Assessment of rapid methodologies is lacking although challenges in ensuring rigour and effective use of resources are known. We mapped reports of rapid evaluations of health and social care innovations, categorised different approaches to rapid evaluation, explored comparative benefits of rapid evaluation, and identified knowledge gaps.

DESIGN

Scoping review.

DATA SOURCES

MEDLINE, EMBASE and Health Management Information Consortium (HMIC) databases were searched through 13 September 2022.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

We included publications reporting primary research or methods for rapid evaluation of interventions or services in health and social care in high-income countries.

DATA EXTRACTION AND SYNTHESIS

Two reviewers developed and piloted a data extraction form. One reviewer extracted data, a second reviewer checked 10% of the studies; disagreements and uncertainty were resolved through consensus. We used narrative synthesis to map different approaches to conducting rapid evaluation.

RESULTS

We identified 16 759 records and included 162 which met inclusion criteria.We identified four main approaches for rapid evaluation: (1) Using methodology designed specifically for rapid evaluation; (2) Increasing rapidity by doing less or using less time-intensive methodology; (3) Using alternative technologies and/or data to increase speed of existing evaluation method; (4) Adapting part of non-rapid evaluation.The COVID-19 pandemic resulted in an increase in publications and some limited changes in identified methods. We found little research comparing rapid and non-rapid evaluation.

CONCLUSIONS

We found a lack of clarity about what 'rapid evaluation' means but identified some useful preliminary categories. There is a need for clarity and consistency about what constitutes rapid evaluation; consistent terminology in reporting evaluations as rapid; development of specific methodologies for making evaluation more rapid; and assessment of advantages and disadvantages of rapid methodology in terms of rigour, cost and impact.

Paper 2: Performing rapid reviews

BACKGROUND

Health policy-makers must often make decisions in compressed time frames and with limited resources. Hence, rapid reviews have become a pragmatic alternative to comprehensive systematic reviews. However, it is important that rapid review methods remain rigorous to support good policy development and decisions. There is currently little evidence about which streamlined steps in a rapid review are less likely to introduce unacceptable levels of uncertainty while still producing a product that remains useful to policy-makers.

METHODS

This paper summarizes current research describing commonly used methods and practices that are used to conduct rapid reviews and presents key considerations and options to guide methodological choices for a rapid review.

RESULTS

The most important step for a rapid review is for an experienced research team to have early and ongoing engagement with the people who have requested the review. A clear research protocol, derived from a needs assessment conducted with the requester, serves to focus the review, defines the scope of the rapid review, and guides all subsequent steps. Common recommendations for rapid review methods include tailoring the literature search in terms of databases, dates, and languages. Researchers can consider using a staged search to locate high-quality systematic reviews and then subsequently published primary studies. The approaches used for study screening and selection, data extraction, and risk-of-bias assessment should be tailored to the topic, researcher experience, and available resources. Many rapid reviews use a single reviewer for study selection, risk-of-bias assessment, or data abstraction, sometimes with partial or full verification by a second reviewer. Rapid reviews usually use a descriptive synthesis method rather than quantitative meta-analysis. Use of brief report templates and standardized production methods helps to speed final report publication.

CONCLUSIONS

Researchers conducting rapid reviews need to make transparent methodological choices, informed by stakeholder input, to ensure that rapid reviews meet their intended purpose. Transparency is critical because it is unclear how or how much streamlined methods can bias the conclusions of reviews. There are not yet internationally accepted standards for conducting or reporting rapid reviews. Thus, this article proposes interim guidance for researchers who are increasingly employing these methods.

Methodological guidance for rapid reviews in healthcare: a scoping review

OBJECTIVE

The aim of the present work was to identify published methodological guidance for rapid reviews (RRs) and to analyze the recommendations with regard to time-saving measures.

STUDY DESIGN AND SETTING

A literature search was performed in PubMed and EMBASE in November 2020. In addition, a search based on Google Scholar and websites of governmental and non-governmental organizations was conducted. Literature screening was carried out by two researchers independently.

RESULTS

A total of 34 publications were included. These describe 38 distinct RR types. The timeframe to complete the identified RR types ranges from 24 hours to six months (mean time 2.2 months). For most RR types a specific research question (n=21) and a prioritizing search (n=25; preference for e.g. systematic reviews, meta-analyses) is employed. Different approaches such as reduced personnel in literature screening (n=21) and data extraction (n=21) are recommended. The majority of RR types include a bias assessment (n=28) and suggest a narrative report focusing on safety and efficacy.

CONCLUSION

The included RR types are heterogeneous in terms of completion time, considered domains and strategies to alter the standard systematic review methods. A rationale for the recommended shortcuts is rarely presented. This article is protected by copyright. All rights reserved.

Semi-automated Tools for Systematic Searches

Traditionally, literature identification for systematic reviews has relied on a two-step process: first, searching databases to identify potentially relevant citations, and then manually screening those citations. A number of tools have been developed to streamline and semi-automate this process, including tools to generate terms; to visualize and evaluate search queries; to trace citation linkages; to deduplicate, limit, or translate searches across databases; and to prioritize relevant abstracts for screening. Research is ongoing into tools that can unify searching and screening into a single step, and several protype tools have been developed. As this field grows, it is becoming increasingly important to develop and codify methods for evaluating the extent to which these tools fulfill their purpose.

Scoping review on clinical definition of bovine respiratory disease complex and related clinical signs in dairy cows

Bovine respiratory disease complex (BRD) is a worldwide multifactorial infectious disease. Antimicrobials are commonly used for treating BRD because bacteria are often involved. The clinical diagnosis of BRD is a challenge, especially in adult dairy cows, where information on this syndrome is scant. Having a definition based on consistent and reliable clinical signs would improve the accuracy of BRD diagnosis and could help to develop an optimal treatment approach by an early detection. The aim of this scoping review was to review clinical signs that could be recognized by producers in dairy cattle suffering from naturally occurring infectious respiratory disease, as reported in the literature. A review of the literature was performed for articles published between January 1, 1990 and January 1, 2020. The search of literature in English, French, and Italian languages included 2 different databases (Pubmed, https://pubmed.ncbi.nlm.nih.gov/; CAB abstract, https://www.cabi.org/publishing-products/cab-abstracts/). Clinical signs were categorized as follows: (1) "general manifestations of disease," which included behavioral changes or fever; (2) "alterations in respiratory function," which included clinical signs specifically associated with the respiratory tract examination; and (3) "clinical signs of other body systems," which included clinical signs related to other systems such as diarrhea or subcutaneous emphysema. The focus of the review was on clinical signs that could be monitored by animal handlers and producers. A total of 1,067 titles were screened, and 23 studies were finally included. The most common general clinical signs were increased body temperature (reported in 83% of studies, n = 19), change in feed intake (26%, n = 6), altered mentation (22%, n = 5), and decreased milk production (17%, n = 4). The alterations in respiratory function noted were nasal discharge (74%, n = 17), cough (65%, n = 15), altered respiratory dynamic or dyspnea (61%, n = 14), increased respiratory rate (43%, n = 10), and ocular discharge or lacrimation (30%, n = 7). The clinical signs associated with infectious respiratory disease reported in the 23 studies generally lacked a clear description of what constitutes a deviation from normality (0-50% of studies clearly reported what was considered normal versus abnormal depending on the clinical signs). This limitation prevented any comparison between studies that apparently reported the same "clinical sign," but possibly referred to a different assessment and definition of what was considered normal versus abnormal. Therefore, the definition of clinical signs in a repeatable way with validated interobserver agreement to determine the optimal combination for the diagnosis of BRD in dairy cows is needed. This could lead to a more judicious use of antimicrobials for respiratory disease in adult dairy cows.

Methodological Considerations for Auditory Training Interventions for Adults With Hearing Loss: A Rapid Review

Purpose The aim of this study was to evaluate literature on auditory training published since 2013. Method A rapid review or a streamlined approach to systematically identify and summarize relevant studies was performed. Selected health sciences databases were searched using a search strategy developed with the PICO (population, intervention, comparison, and outcome) framework. Studies eligible for inclusion had older adult participants with hearing loss and utilized technology-based auditory training in laboratory or home settings. Results The study quality of most studies was found to be low to moderate, with concentrations between low and moderate. Major issues were related to study design and reporting, such as the need for blinding and a control group, larger sample sizes, and a follow-up for long-term outcomes of auditory training interventions. Wide variability in training approaches, participant backgrounds (e.g., audiograms, hearing aid use), and outcome measures are also noted. Conclusions Evidence on the effectiveness of auditory training is mixed. Future research should include high-quality randomized controlled trials with representative populations; follow-up periods to study long-term effects; and exploration of behavioral, electrophysiological, and self-reported outcome measures. Recommendations for study designs and methodologies are also discussed.

Exploring the effectiveness of telehealth interventions for diagnosis, contact tracing and care of Corona Virus Disease of 2019 (COVID19) patients in sub Saharan Africa: a rapid review

The efficacy of leveraging telehealth services on clinical outcomes remains scarcely documented. We conducted a rapid review to explore the effectiveness of telehealth interventions for the diagnosis, contact tracing and care of the corona virus disease of 2019 in sub Saharan Africa. Using MEDLINE, Science Direct and Cochrane Library databases, a review was conducted during the month of July 2020 of peer reviewed articles reporting on the use of telehealth interventions in sub-Saharan Africa. All the studies were assessed against the inclusion criteria by two independent reviewers. The 7 studies included in the synthesis were conducted in 2 countries [Nigeria (× 1), Zimbabwe (× 1)], and commentaries covering the entire sub Saharan Africa in general (× 4) and to Uganda (× 1). All the included articles and commentaries were published in 2020. We established that mobile applications are effective in providing information for referrals of potential patients infected by COVID 19 and provides convenient access to routine care without the risk of exposure through close contact. In countries such as Nigeria, mobile positioning data significantly improved decision making, capacity and scope of contact tracing and surveillance of known contacts of confirmed cases. We noted that collaborations between the government, mobile network operators and technology companies were utilised for successful mobile positioning data tracing interventions for COVID patients. Mobile applications such as the Vula platform in South Africa were also noted as effective in providing psychological support to health care workers attending to patients.

Operational Considerations for Physical Therapy During COVID-19: A Rapid Review

OBJECTIVE

Physical therapists play an important role in responding to pandemic and physical disaster situations. Existing literature can provide guidance to health care leadership teams to appropriately and safely leverage physical therapy resources and skill sets during the COVID-19 pandemic. The purpose of this study was to provide a review of the pandemic and physical disaster management literature to summarize physical therapy-specific operational considerations to assist hospital-based leadership teams in planning and response efforts during the COVID-19 pandemic.

METHODS

A rapid review was conducted over a 4-week time frame (April-May 2020). The review team included 3 physical therapist clinician researchers, a health librarian, and a member of the physical therapy leadership team. The initial search strategy identified 303 articles, 80 of which were retained for full-text screening. Twenty articles were included in the review.

RESULTS

Five main categories of operational considerations for physical therapy during the COVID-19 pandemic were identified: (1) organizational actions, (2) staffing considerations, (3) physical therapist roles, (4) physical resources, and (5) other considerations. Additional relevant information from physical therapists' experiences in physical disaster situations was also summarized.

CONCLUSION

The evidence presented within this review can be used to inform facility-based and regional planning efforts during the current COVID-19 pandemic and in general preparedness planning.

IMPACT

Physical therapists have an important role to play in response efforts related to major events that impact health and wellness, including the COVID-19 pandemic. Evidence-informed, facility-based, and regional planning during the current COVID-19 pandemic will help physical therapists enhance their role in treating patients in physical therapy and rehabilitation settings.

Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews

OBJECTIVES

To develop methods guidance to support the conduct of rapid reviews (RRs) produced within Cochrane and beyond, in response to requests for timely evidence syntheses for decision-making purposes including urgent health issues of high priority.

STUDY DESIGN AND SETTING

Interim recommendations were informed by a scoping review of the underlying evidence, primary methods studies conducted, and a survey sent to 119 representatives from 20 Cochrane entities, who were asked to rate and rank RR methods across stages of review conduct. Discussions among those with expertise in RR methods further informed the list of recommendations with accompanying rationales provided.

RESULTS

Based on survey results from 63 respondents (53% response rate), 26 RR methods recommendations are presented for which there was a high or moderate level of agreement or scored highest in the absence of such agreement. Where possible, how recommendations align with Cochrane methods guidance for systematic reviews is highlighted.

CONCLUSION

The Cochrane Rapid Reviews Methods Group offers new, interim guidance to support the conduct of RRs. Because best practice is limited by the lack of currently available evidence for some RR methods shortcuts taken, this guidance will need to be updated as additional abbreviated methods are evaluated.

Defining Rapid Reviews: a systematic scoping review and thematic analysis of definitions and defining characteristics of rapid reviews

BACKGROUND AND OBJECTIVE

Rapid reviews were first mentioned in the literature in 1997, when Best et al. described the rapid health technology assessment program in the south and west regions of England but did not provide a formal definition. More recently, the only consensus around a rapid review definition is that a formal definition does not exist. The primary aim of this work is to create a repository of existing definitions and to identify key themes, which may help the knowledge synthesis community in defining rapid review products.

METHODS

A systematic scoping review was performed to identify definitions used in journal-published rapid reviews written in English between 2017 and January 2019. We searched Medline, Embase Classic + Embase, PsycINFO, ERIC, Cochrane Library, CINAHL, and Web of Science on December 21, 2018. Two reviewers performed study selection and data extraction using a priori-defined methods published in a protocol. Definitions from rapid review methods articles (published from 1997 onward) identified in another scoping review were added to the results, and all definitions were thematically analyzed using NVivo. A quantitative analysis was also performed around studies cited.

RESULTS

Definitions from 216 rapid reviews and 90 rapid review methods articles were included in the thematic analysis. Eight key themes were identified: accelerated/rapid process or approach, variation in methods shortcuts, focus/depth/breadth of scope, compare and contrast to a full traditional systematic review, stakeholder rationale, resource efficiency rationale, systematic approach, bias/limitations. Secondary referencing was a common occurrence.

CONCLUSION

Thematic analysis performed in this systematic scoping review has allowed for the creation of a suggested definition for rapid reviews that can be used to inform the systematic review community.

Lung Cancer Screening with Low-Dose CT: a Meta-Analysis

BACKGROUND

Randomized controlled trials have evaluated the efficacy of low-dose CT (LDCT) lung cancer screening on lung cancer (LC) outcomes.

OBJECTIVE

Meta-analyze LDCT lung cancer screening trials.

METHODS

We identified studies by searching PubMed, Google Scholar, the Cochrane Registry, ClinicalTrials.gov , and reference lists from retrieved publications. We abstracted data on study design features, stage I LC diagnoses, LC and overall mortality, false positive results, harm from invasive diagnostic procedures, overdiagnosis, and significant incidental findings. We assessed study quality using the Cochrane risk-of-bias tool. We used random-effects models to calculate relative risks and assessed effect modulators with subgroup analyses and meta-regression.

RESULTS

We identified 9 studies that enrolled 96,559 subjects. The risk of bias across studies was judged to be low. Overall, LDCT screening significantly increased the detection of stage I LC, RR = 2.93 (95% CI, 2.16-3.98), I² = 19%, and reduced LC mortality, RR = 0.84 (95% CI, 0.75-0.93), I² = 0%. The number needed to screen to prevent an LC death was 265. Women had a lower risk of LC death (RR = 0.69, 95% CI, 0.40-1.21) than men (RR = 0.86, 95% CI, 0.66-1.13), p value for interaction = 0.11. LDCT screening did not reduce overall mortality, RR = 0.96 (95% CI, 0.91-1.01), I² = 0%. The pooled false positive rate was 8% (95% CI, 4-18); subjects with false positive results had < 1 in 1000 risk of major complications following invasive diagnostic procedures. The most valid estimates for overdiagnosis and significant incidental findings were 8.9% and 7.5%, respectively.

DISCUSSION

LDCT screening significantly reduced LC mortality, though not overall mortality, with women appearing to benefit more than men. The estimated risks for false positive results, screening complications, overdiagnosis, and incidental findings were low. Long-term survival data were available only for North American and European studies limiting generalizability.

Combining abbreviated literature searches with single-reviewer screening: three case studies of rapid reviews

BACKGROUND

Decision-makers increasingly request rapid answers to clinical or public health questions. To save time, personnel, and financial resources, rapid reviews streamline the methodological steps of the systematic review process. We aimed to explore the validity of a rapid review approach that combines a substantially abbreviated literature search with a single-reviewer screening of abstracts and full texts using three case studies.

METHODS

We used a convenience sample of three ongoing Cochrane reviews as reference standards. Two reviews addressed oncological topics and one addressed a public health topic. For each of the three topics, three reviewers screened the literature independently. Our primary outcome was the change in conclusions between the rapid reviews and the respective Cochrane reviews. In case the rapid approach missed studies, we recalculated the meta-analyses for the main outcomes and asked Cochrane review authors if the new body of evidence would change their original conclusion compared with the reference standards. Additionally, we assessed the sensitivity of the rapid review approach compared with the results of the original Cochrane reviews.

RESULTS

For the two oncological topics (case studies 1 and 2), the three rapid reviews each yielded the same conclusions as the Cochrane reviews. However, the authors would have had less certainty about their conclusion in case study 2. For case study 3, the public health topic, only one of the three rapid reviews led to the same conclusion as the Cochrane review. The other two rapid reviews provided insufficient information for the authors to draw conclusions. Using the rapid review approach, the sensitivity was 100% (3 of 3) for case study 1. For case study 2, the three rapid reviews identified 40% (4 of 10), 50% (5 of 10), and 60% (6 of 10) of the included studies, respectively; for case study 3, the respective numbers were 38% (8 of 21), 43% (9 of 21), and 48% (10 of 21).

CONCLUSIONS

Within the limitations of these case studies, a rapid review approach that combines abbreviated literature searches with single-reviewer screening may be feasible for focused clinical questions. For complex public health topics, sensitivity seems to be insufficient.

Error rates of human reviewers during abstract screening in systematic reviews

Background

Automated approaches to improve the efficiency of systematic reviews are greatly needed. When testing any of these approaches, the criterion standard of comparison (gold standard) is usually human reviewers. Yet, human reviewers make errors in inclusion and exclusion of references.

Objectives

To determine citation false inclusion and false exclusion rates during abstract screening by pairs of independent reviewers. These rates can help in designing, testing and implementing automated approaches.

Methods

We identified all systematic reviews conducted between 2010 and 2017 by an evidence-based practice center in the United States. Eligible reviews had to follow standard systematic review procedures with dual independent screening of abstracts and full texts, in which citation inclusion by one reviewer prompted automatic inclusion through the next level of screening. Disagreements between reviewers during full text screening were reconciled via consensus or arbitration by a third reviewer. A false inclusion or exclusion was defined as a decision made by a single reviewer that was inconsistent with the final included list of studies.

Results

We analyzed a total of 139,467 citations that underwent 329,332 inclusion and exclusion decisions from 86 unique reviewers. The final systematic reviews included 5.48% of the potential references identified through bibliographic database search (95% confidence interval (CI): 2.38% to 8.58%). After abstract screening, the total error rate (false inclusion and false exclusion) was 10.76% (95% CI: 7.43% to 14.09%).

Conclusions

This study suggests important false inclusion and exclusion rates by human reviewers. When deciding the validity of a future automated study selection algorithm, it is important to keep in mind that the gold standard is not perfect and that achieving error rates similar to humans may be adequate and can save resources and time.

Palliative Care Evidence Review Service (PaCERS): a knowledge transfer partnership

The importance of linking evidence into practice and policy is recognised as a key pillar of a prudent approach to healthcare; it is of importance to healthcare professionals and decision-makers across the world in every speciality. However, rapid access to evidence to support service redesign, or to change practice at pace, is challenging. This is particularly so in smaller specialties such as Palliative Care, where pressured multidisciplinary clinicians lack time and skill sets to locate and appraise the literature relevant to a particular area. Therefore, we have initiated the Palliative Care Evidence Review Service (PaCERS), a knowledge transfer partnership through which we have developed a clear methodology to conduct evidence reviews to support professionals and other decision-makers working in palliative care.

PaCERS methodology utilises modified systematic review methods as there is no agreed definition or an accepted methodology for conducting rapid reviews. This paper describes the stages involved based on our iterative recent experiences and engagement with stakeholders, who are the potential beneficiaries of the research. Uniquely, we emphasise the process and opportunities of engagement with the clinical workforce and policy-makers throughout the review, from developing and refining the review question at the start through to the importance of demonstrating impact. We are faced with the challenge of the trade-off between the timely transfer of evidence against the risk of impacting on rigour. To address this issue, we try to ensure transparency throughout the review process. Our methodology aligns with key principles of knowledge synthesis in defining a process that is transparent, robust and improving the efficiency and timeliness of the review.

Our reviews are clinically or policy driven and, although we use modified systematic review methods, one of the key differences between published review processes and our review process is in our relationship with the requester. This streamlining approach to synthesising evidence in a timely manner helps to inform decisions faced by clinicians and decision-makers in healthcare settings, supporting, at pace, knowledge transfer and mobilisation.

Why equating all evidence searches to systematic reviews defies their role in information seeking

All too often the quality and rigor of topic investigations is inaccurately conveyed to information professionals, resulting in a mischaracterization of the research, which, if left unchecked and published, may in turn mislead potential readers. Accurately understanding and categorizing the types of topic investigation searches that are requested of information professionals is critical to both meeting requestors’ needs and reflecting their intended methodological approaches. Information professionals’ expertise can be an invaluable resource to guide users through the investigative and publication process.

Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review

BACKGROUND

Stringent requirements exist regarding the transparency of the study selection process and the reliability of results. A 2-step selection process is generally recommended; this is conducted by 2 reviewers independently of each other (conventional double-screening). However, the approach is resource intensive, which can be a problem, as systematic reviews generally need to be completed within a defined period with a limited budget. The aim of the following methodological systematic review was to analyse the evidence available on whether single screening is equivalent to double screening in the screening process conducted in systematic reviews.

METHODS

We searched Medline, PubMed and the Cochrane Methodology Register (last search 10/2018). We also used supplementary search techniques and sources ("similar articles" function in PubMed, conference abstracts and reference lists). We included all evaluations comparing single with double screening. Data were summarized in a structured, narrative way.

RESULTS

The 4 evaluations included investigated a total of 23 single screenings (12 sets for screening involving 9 reviewers). The median proportion of missed studies was 5% (range 0 to 58%). The median proportion of missed studies was 3% for the 6 experienced reviewers (range: 0 to 21%) and 13% for the 3 reviewers with less experience (range: 0 to 58%). The impact of missing studies on the findings of meta-analyses had been reported in 2 evaluations for 7 single screenings including a total of 18,148 references. In 3 of these 7 single screenings - all conducted by the same reviewer (with less experience) - the findings would have changed substantially. The remaining 4 of these 7 screenings were conducted by experienced reviewers and the missing studies had no impact or a negligible on the findings of the meta-analyses.

CONCLUSIONS

Single screening of the titles and abstracts of studies retrieved in bibliographic searches is not equivalent to double screening, as substantially more studies are missed. However, in our opinion such an approach could still represent an appropriate methodological shortcut in rapid reviews, as long as it is conducted by an experienced reviewer. Further research on single screening is required, for instance, regarding factors influencing the number of studies missed.

Care of acute conditions and chronic diseases in Canada and the United States: Rapid systematic review and meta-analysis

This study tested the hypothesis that socioeconomically vulnerable Canadians with diverse acute conditions or chronic diseases have health care access and survival advantages over their counterparts in the USA. A rapid systematic review retrieved 25 studies (34 independent cohorts) published between 2003 and 2018. They were synthesized with a streamlined meta-analysis. Very low-income Canadian patients were consistently and highly advantaged in terms of health care access and survival compared with their counterparts in the USA who lived in poverty and/or were uninsured or underinsured. In aggregate and controlling for specific conditions or diseases and typically 4 to 9 comorbid factors or biomarkers, Canadians' chances of receiving better health care were estimated to be 36% greater than their American counterparts (RR=1.36, 95% CI 1.35-1.37). This estimate was significantly larger than that based on general patient or non-vulnerable population comparisons (RR=1.09, 95% CI 1.08-1.10). Contrary to prevalent political rhetoric, three studies observed that Americans experience more than twice the risk of long waits for breast or colon cancer care or of dying while they wait for an organ transplant (RR=2.36, 95% CI 2.09-2.66). These findings were replicated across externally valid national studies and more internally valid, metropolitan or provincial/state comparisons. Socioeconomically vulnerable Canadians are consistently and highly advantaged on health care access and outcomes compared to their American counterparts. Less vulnerable comparisons found more modest Canadian advantages. The Affordable Care Act ought to be fully supported including the expansion of Medicaid across all states. Canada's single payer system ought to be maintained and strengthened, but not through privatization.

Screening for Syphilis Infection in Pregnant Women: Updated Evidence Report and Systematic Review for the US Preventive Services Task Force

IMPORTANCE

The incidence of syphilis and congenital syphilis in the United States has increased after reaching historic lows in the early 2000s.

OBJECTIVE

To systematically review literature on the effectiveness and harms of screening for syphilis in pregnancy and the harms of penicillin treatment in pregnancy to inform the US Preventive Services Task Force.

DATA SOURCES

MEDLINE, PubMed, and the Cochrane Central Register of Controlled Trials for relevant English-language literature, published from January 1, 2008, to June 2, 2017. Ongoing surveillance was conducted through November 22, 2017.

STUDY SELECTION

Studies conducted in countries categorized as "high" or "very high" on the Human Development Index that explicitly addressed 1 of 3 a priori-defined key questions.

DATA EXTRACTION AND SYNTHESIS

Independent critical appraisal and data abstraction by 2 reviewers. Data from included studies were narratively synthesized without pooling data.

MAIN OUTCOMES AND MEASURES

Incidence of congenital syphilis; any harms of screening or penicillin treatment in pregnancy.

RESULTS

Seven studies in 8 publications were included. One observational study evaluated the implementation of syphilis screening in pregnancy in 2 441 237 women in China. From 2002 to 2012, screening for syphilis in all pregnant women increased from 89.8% to 97.2%, and the incidence of congenital syphilis decreased from 109.3 to 9.4 cases per 100 000 live births. Five studies (n = 21 795) evaluated the false-positive findings of treponemal tests and 1 study (n = 318) evaluated the false-negative findings of nontreponemal tests. These studies found that false-positives with treponemal-specific enzyme or chemiluminescent immunoassays were common (46.5%-88.2%), therefore warranting reflexive (automatic confirmatory) testing for all positive test findings. One study (n = 318) found no false-negatives with treponemal tests, and 1 study (n = 139) demonstrated the prozone phenomenon (false-negative response from high antibody titer) with rapid plasma reagin screening using undiluted samples (2.9%). No studies were identified for harms of penicillin in pregnancy.

CONCLUSIONS AND RELEVANCE

Screening for syphilis infection in pregnant women is associated with reduced incidence of congenital syphilis, and available evidence supports the need for reflexive testing for positive test results.