A retrospective comparison of systematic reviews with same-topic rapid reviews

A Guide for Schools on Student-Directed Suicide Prevention Programs Eligible for Implementation under the STANDUP Act, a Rapid Review and Evidence Synthesis

This review evaluates the strength of evidence for school-based mental health and suicide prevention programs that meet the legal eligibility criteria of the Suicide Training and Awareness Nationally Delivered for Universal Prevention Act of 2021 (STANDUP Act). Included studies were aggregated by program and a program's overall body of evidence was evaluated using the LEGEND system. Requirements for implementation were also documented. We identified 29 studies, which, when aggregated, encompassed 12 unique programs that meet the statute's evidence-based criteria. All four outcomes described in the statute were measured, with help-seeking being the most commonly measured. Two programs were assigned a high level of evidence in decreasing suicidal thoughts and behaviors. The findings serve as a resource for school officials in identifying evidence-based mental health and suicide prevention programs and understanding the resources needed for implementation.

Rapid review: A review of methods and recommendations based on current evidence

Rapid review (RR) could accelerate the traditional systematic review (SR) process by simplifying or omitting steps using various shortcuts. With the increasing popularity of RR, numerous shortcuts had emerged, but there was no consensus on how to choose the most appropriate ones. This study conducted a literature search in PubMed from inception to December 21, 2023, using terms such as "rapid review" "rapid assessment" "rapid systematic review" and "rapid evaluation". We also scanned the reference lists and performed citation tracking of included impact studies to obtain more included studies. We conducted a narrative synthesis of all RR approaches, shortcuts and studies assessing their effectiveness at each stage of RRs. Based on the current evidence, we provided recommendations on utilizing certain shortcuts in RRs. Ultimately, we identified 185 studies focusing on summarizing RR approaches and shortcuts, or evaluating their impact. There was relatively sufficient evidence to support the use of the following shortcuts in RRs: limiting studies to those published in English-language; conducting abbreviated database searches (e.g., only searching PubMed/MEDLINE, Embase, and CENTRAL); omitting retrieval of grey literature; restricting the search timeframe to the recent 20 years for medical intervention and the recent 15 years for reviewing diagnostic test accuracy; conducting a single screening by an experienced screener. To some extent, the above shortcuts were also applicable to SRs. This study provided a reference for future RR researchers in selecting shortcuts, and it also presented a potential research topic for methodologists.

Production and use of rapid responses during the COVID-19 pandemic in Quebec (Canada): perspectives from evidence synthesis producers and decision makers

BACKGROUND

The COVID-19 pandemic has required evidence to be made available more rapidly than usual, in order to meet the needs of decision makers in a timely manner. These exceptional circumstances have caused significant challenges for organizations and teams responsible for evidence synthesis. They had to adapt to provide rapid responses to support decision-making. This study aimed to document (1) the challenges and adaptations made to produce rapid responses during the pandemic, (2) their perceived usefulness, reported use and factors influencing their use and (3) the methodological adaptations made to produce rapid responses.

METHODS

A qualitative study was conducted in 2021 with eight organizations in the health and social services system in Quebec (Canada), including three institutes with a provincial mandate. Data collection included focus groups (n = 9 groups in 8 organizations with 64 participants), interviews with decision makers (n = 12), and a document analysis of COVID-19 rapid responses (n = 128). A thematic analysis of qualitative data (objectives 1 and 2) and a descriptive analysis of documents (objective 3) were conducted.

RESULTS

The results highlight the teams and organizations' agility to deal with the many challenges encountered during the pandemic (e.g., increased their workloads, adoption of new technological tools or work processes, improved collaboration, development of scientific monitoring, adaptation of evidence synthesis methodologies and products). The challenge of balancing rigor and speed was reported by teams and organizations. When available at the right time, rapid responses have been reported as a useful tool for informing or justifying decisions in a context of uncertainty. Several factors that may influence their use were identified (e.g., clearly identify needs, interactions with producers, perceived rigor and credibility, precise and feasible recommendations). Certain trends in the methodological approaches used to speed up the evidence synthesis process were identified.

CONCLUSIONS

This study documented rapid responses producers' experiences during the COVID-19 pandemic in Quebec, and decision makers who requested, consulted, or used these products. Potential areas of improvements are identified such as reinforce coordination, improve communication loops, clarify guidelines or methodological benchmarks, and enhance utility of rapid response products for decision makers.

What are the best methods for rapid reviews of the research evidence? A systematic review of reviews and primary studies

Rapid review methodology aims to facilitate faster conduct of systematic reviews to meet the needs of the decision-maker, while also maintaining quality and credibility. This systematic review aimed to determine the impact of different methodological shortcuts for undertaking rapid reviews on the risk of bias (RoB) of the results of the review. Review stages for which reviews and primary studies were sought included the preparation of a protocol, question formulation, inclusion criteria, searching, selection, data extraction, RoB assessment, synthesis, and reporting. We searched 11 electronic databases in April 2022, and conducted some supplementary searching. Reviewers worked in pairs to screen, select, extract data, and assess the RoB of included reviews and studies. We included 15 systematic reviews, 7 scoping reviews, and 65 primary studies. We found that several commonly used shortcuts in rapid reviews are likely to increase the RoB in the results. These include restrictions based on publication date, use of a single electronic database as a source of studies, and use of a single reviewer for screening titles and abstracts, selecting studies based on the full-text, and for extracting data. Authors of rapid reviews should be transparent in reporting their use of these shortcuts and acknowledge the possibility of them causing bias in the results. This review also highlights shortcuts that can save time without increasing the risk of bias. Further research is needed for both systematic and rapid reviews on faster methods for accurate data extraction and RoB assessment, and on development of more precise search strategies.

Approaches to enabling rapid evaluation of innovations in health and social care: a scoping review of evidence from high-income countries

OBJECTIVE

The COVID-19 pandemic increased the demand for rapid evaluation of innovation in health and social care. Assessment of rapid methodologies is lacking although challenges in ensuring rigour and effective use of resources are known. We mapped reports of rapid evaluations of health and social care innovations, categorised different approaches to rapid evaluation, explored comparative benefits of rapid evaluation, and identified knowledge gaps.

DESIGN

Scoping review.

DATA SOURCES

MEDLINE, EMBASE and Health Management Information Consortium (HMIC) databases were searched through 13 September 2022.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

We included publications reporting primary research or methods for rapid evaluation of interventions or services in health and social care in high-income countries.

DATA EXTRACTION AND SYNTHESIS

Two reviewers developed and piloted a data extraction form. One reviewer extracted data, a second reviewer checked 10% of the studies; disagreements and uncertainty were resolved through consensus. We used narrative synthesis to map different approaches to conducting rapid evaluation.

RESULTS

We identified 16 759 records and included 162 which met inclusion criteria.We identified four main approaches for rapid evaluation: (1) Using methodology designed specifically for rapid evaluation; (2) Increasing rapidity by doing less or using less time-intensive methodology; (3) Using alternative technologies and/or data to increase speed of existing evaluation method; (4) Adapting part of non-rapid evaluation.The COVID-19 pandemic resulted in an increase in publications and some limited changes in identified methods. We found little research comparing rapid and non-rapid evaluation.

CONCLUSIONS

We found a lack of clarity about what 'rapid evaluation' means but identified some useful preliminary categories. There is a need for clarity and consistency about what constitutes rapid evaluation; consistent terminology in reporting evaluations as rapid; development of specific methodologies for making evaluation more rapid; and assessment of advantages and disadvantages of rapid methodology in terms of rigour, cost and impact.

Rapid evidence synthesis to enable innovation and adoption in health and social care

BACKGROUND

The rapid identification and adoption of effective innovations in healthcare is a known challenge. The strongest evidence base for innovations can be provided by evidence synthesis, but this is frequently a lengthy process and even rapid versions of this can be time-consuming and complex. In the UK, the Accelerated Access Review and Academic Health Science Network (AHSN) have provided the impetus to develop a consistently rapid process to support the identification and adoption of high-value innovations in the English NHS.

METHODS

The Greater Manchester Applied Research Collaboration (ARC-GM) developed a framework for a rapid evidence synthesis (RES) approach, which is highly integrated within the innovation process of the Greater Manchester AHSN and the associated healthcare and research ecosystem. The RES uses evidence synthesis approaches and draws on the GRADE Evidence to Decision framework to provide rapid assessments of the existing evidence and its relevance to specific decision problems. We implemented this in a real-time context of decision-making around adoption of innovative health technologies.

RESULTS

Key stakeholders in the Greater Manchester decision-making process for healthcare innovations have found that our approach is both timely and flexible; it is valued for its combination of rigour and speed. Our RES approach rapidly and systematically identifies, appraises and contextualises relevant evidence, which can then be transparently incorporated into decisions about the wider adoption of innovations. The RES also identifies limitations in existing evidence for innovations and this can inform subsequent evaluations. There is substantial interest from other ARCs and AHSNs in implementing a similar process. We are currently exploring methods to make completed RES publicly available. We are also exploring methods to evaluate the impact of using RES as more implementation decisions are made.

CONCLUSIONS

The RES framework we have implemented combines transparency and consistency with flexibility and rapidity. It therefore maximises utility in a real-time decision-making context for healthcare innovations.

Medical Health Data-Driven Physical Education Scheme: Public Environment-Oriented Exercise Health Management

Smart wearable devices can encourage users to take an active part in exercise to a certain extent. The most important reason is that their entertainment function can bring users a better exercise experience, which is also one of the reasons why most people wear smart wearable devices during exercise. Another reason is that the medical and health data feedback of smart wearable devices can play a stimulating role, which is also the motivation of this study. Through the feedback of health data, students can perceive their own exercise situation, and teachers can plan more targeted exercise courses based on medical and health data, so as to improve the quality of physical education. This paper studies the origin and development of health data-driven learning; clarifies the logical mechanism between health data-driven learning and physical education; analyzes the internal needs of physical education design; discusses the characteristics of thinking, process, tools, and other elements of health data-driven learning in the new era of physical education reform; and carries out physical education teaching practice. Experiments have proved that health data-driven physical education is beneficial for improving students' emotional attitude and values, optimizing learning process and methods, and strengthening knowledge literacy and exercise skills. However, in today's poor public environment, the medical and health data-driven physical education scheme proposed in this paper is also applicable to the public-oriented exercise health management. Medical Health; Public Environment; Health Management; Data Analysis.

Paper 2: Performing rapid reviews

BACKGROUND

Health policy-makers must often make decisions in compressed time frames and with limited resources. Hence, rapid reviews have become a pragmatic alternative to comprehensive systematic reviews. However, it is important that rapid review methods remain rigorous to support good policy development and decisions. There is currently little evidence about which streamlined steps in a rapid review are less likely to introduce unacceptable levels of uncertainty while still producing a product that remains useful to policy-makers.

METHODS

This paper summarizes current research describing commonly used methods and practices that are used to conduct rapid reviews and presents key considerations and options to guide methodological choices for a rapid review.

RESULTS

The most important step for a rapid review is for an experienced research team to have early and ongoing engagement with the people who have requested the review. A clear research protocol, derived from a needs assessment conducted with the requester, serves to focus the review, defines the scope of the rapid review, and guides all subsequent steps. Common recommendations for rapid review methods include tailoring the literature search in terms of databases, dates, and languages. Researchers can consider using a staged search to locate high-quality systematic reviews and then subsequently published primary studies. The approaches used for study screening and selection, data extraction, and risk-of-bias assessment should be tailored to the topic, researcher experience, and available resources. Many rapid reviews use a single reviewer for study selection, risk-of-bias assessment, or data abstraction, sometimes with partial or full verification by a second reviewer. Rapid reviews usually use a descriptive synthesis method rather than quantitative meta-analysis. Use of brief report templates and standardized production methods helps to speed final report publication.

CONCLUSIONS

Researchers conducting rapid reviews need to make transparent methodological choices, informed by stakeholder input, to ensure that rapid reviews meet their intended purpose. Transparency is critical because it is unclear how or how much streamlined methods can bias the conclusions of reviews. There are not yet internationally accepted standards for conducting or reporting rapid reviews. Thus, this article proposes interim guidance for researchers who are increasingly employing these methods.

Characteristics, quality and volume of the first 5 months of the COVID-19 evidence synthesis infodemic: a meta-research study

OBJECTIVE

The academic and scientific community has reacted at pace to gather evidence to help and inform about COVID-19. Concerns have been raised about the quality of this evidence. The aim of this review was to map the nature, scope and quality of evidence syntheses on COVID-19 and to explore the relationship between review quality and the extent of researcher, policy and media interest.

DESIGN AND SETTING

A meta-research: systematic review of reviews.

INFORMATION SOURCES

PubMed, Epistemonikos COVID-19 evidence, the Cochrane Library of Systematic Reviews, the Cochrane COVID-19 Study Register, EMBASE, CINAHL, Web of Science Core Collection and the WHO COVID-19 database, searched between 10 June 2020 and 15 June 2020.

ELIGIBILITY CRITERIA

Any peer-reviewed article reported as a systematic review, rapid review, overview, meta-analysis or qualitative evidence synthesis in the title or abstract addressing a research question relating to COVID-19. Articles described as meta-analyses but not undertaken as part of a systematic or rapid review were excluded.

STUDY SELECTION AND DATA EXTRACTION

Abstract and full text screening were undertaken by two independent reviewers. Descriptive information on review type, purpose, population, size, citation and attention metrics were extracted along with whether the review met the definition of a systematic review according to six key methodological criteria. For those meeting all criteria, additional data on methods and publication metrics were extracted.

RISK OF BIAS

For articles meeting all six criteria required to meet the definition of a systematic review, AMSTAR-2 ((A MeaSurement Tool to Assess systematic Reviews, version 2.0) was used to assess the quality of the reported methods.

RESULTS

2334 articles were screened, resulting in 280 reviews being included: 232 systematic reviews, 46 rapid reviews and 2 overviews. Less than half reported undertaking critical appraisal and a third had no reproducible search strategy. There was considerable overlap in topics, with discordant findings. Eighty-eight of the 280 reviews met all six systematic review criteria. Of these, just 3 were rated as of moderate or high quality on AMSTAR-2, with the majority having critical flaws: only a third reported registering a protocol, and less than one in five searched named COVID-19 databases. Review conduct and publication were rapid, with 52 of the 88 systematic reviews reported as being conducted within 3 weeks, and a half published within 3 weeks of submission. Researcher and media interest, as measured by altmetrics and citations, was high, and was not correlated with quality.

DISCUSSION

This meta-research of early published COVID-19 evidence syntheses found low-quality reviews being published at pace, often with short publication turnarounds. Despite being of low quality and many lacking robust methods, the reviews received substantial attention across both academic and public platforms, and the attention was not related to the quality of review methods.

INTERPRETATION

Flaws in systematic review methods limit the validity of a review and the generalisability of its findings. Yet, by being reported as 'systematic reviews', many readers may well regard them as high-quality evidence, irrespective of the actual methods undertaken. The challenge especially in times such as this pandemic is to provide indications of trustworthiness in evidence that is available in 'real time'.

PROSPERO REGISTRATION NUMBER

CRD42020188822.

Principles and framework for assessing the risk of bias for studies included in comparative quantitative environmental systematic reviews

The internal validity of conclusions about effectiveness or impact in systematic reviews, and of decisions based on them, depends on risk of bias assessments being conducted appropriately. However, a random sample of 50 recently-published articles claiming to be quantitative environmental systematic reviews found 64% did not include any risk of bias assessment, whilst nearly all that did omitted key sources of bias. Other limitations included lack of transparency, conflation of quality constructs, and incomplete application of risk of bias assessments to the data synthesis. This paper addresses deficiencies in risk of bias assessments by highlighting core principles that are required for risk of bias assessments to be fit-for-purpose, and presenting a framework based on these principles to guide review teams on conducting risk of bias assessments appropriately and consistently. The core principles require that risk of bias assessments be Focused, Extensive, Applied and Transparent (FEAT). These principles support risk of bias assessments, appraisal of risk of bias tools, and the development of new tools. The framework follows a Plan-Conduct-Apply-Report approach covering all stages of risk of bias assessment. The scope of this paper is comparative quantitative environmental systematic reviews which address PICO or PECO-type questions including, but not limited to, topic areas such as environmental management, conservation, ecosystem restoration, and analyses of environmental interventions, exposures, impacts and risks.

Methodological guidance for rapid reviews in healthcare: a scoping review

OBJECTIVE

The aim of the present work was to identify published methodological guidance for rapid reviews (RRs) and to analyze the recommendations with regard to time-saving measures.

STUDY DESIGN AND SETTING

A literature search was performed in PubMed and EMBASE in November 2020. In addition, a search based on Google Scholar and websites of governmental and non-governmental organizations was conducted. Literature screening was carried out by two researchers independently.

RESULTS

A total of 34 publications were included. These describe 38 distinct RR types. The timeframe to complete the identified RR types ranges from 24 hours to six months (mean time 2.2 months). For most RR types a specific research question (n=21) and a prioritizing search (n=25; preference for e.g. systematic reviews, meta-analyses) is employed. Different approaches such as reduced personnel in literature screening (n=21) and data extraction (n=21) are recommended. The majority of RR types include a bias assessment (n=28) and suggest a narrative report focusing on safety and efficacy.

CONCLUSION

The included RR types are heterogeneous in terms of completion time, considered domains and strategies to alter the standard systematic review methods. A rationale for the recommended shortcuts is rarely presented. This article is protected by copyright. All rights reserved.

Changes in Youth Mental Health, Psychological Wellbeing, and Substance Use During the COVID-19 Pandemic: A Rapid Review

Since the onset of the COVID-19 pandemic, researchers around the world have made efforts to assess its impact on youth mental health; however, the breadth of this topic has impeded a clear assessment of pandemic outcomes. This study aimed to address this gap by reviewing changes in youth (age ≤ 25) mental health, psychological wellbeing, substance use, and the use or delivery of relevant services during the pandemic. PubMed and Embase were searched in May 2021 to conduct a rapid review of the literature. The results encompass 156 primary publications and are reported using a narrative synthesis. Studies of mental health (n = 122) and psychological wellbeing (n = 28) generally indicated poor outcomes in many settings. Publications regarding substance use (n = 41) noted overall declines or unchanged patterns. Studies of service delivery (n = 12) indicated a generally positive reception for helplines and telehealth, although some youth experienced difficulties accessing services. The findings indicate negative impacts of the pandemic on youth mental health, with mixed results for substance use. Services must support marginalized youth who lack access to telehealth.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40894-022-00185-6.

Opportunities to improve reporting of rapid response in health technology assessment

Introduction

Mini health technology assessment (HTA) reports have been used to support policy makers and health systems by providing a timely summary of scientific evidence. The objective of this meta-epidemiologic study was to evaluate the quality of reporting of mini-HTA reports published in Brazil.

Methods

An electronic search for all mini-HTA reports published between 2014 and March 2019 was conducted in the SISREBRATS and CONITEC databases. The study selection and data extraction were performed by two independent assessors. The following data were extracted: bibliographic data; research question; characteristics of the population, health technologies and outcomes assessed; eligibility criteria; information about searches and study selection; risk of bias assessment; quality of evidence assessment; synthesis of results; and recommendation about the technology evaluated. A descriptive analysis was used to summarize the information retrieved from all the included mini-HTA reports.

Results

We included 103 mini-HTA reports, the great majority of which (92.3 percent) focused on the coverage of the technologies in the healthcare system, with more than 60 percent being about drugs. Only five mini-HTA reports (4.8 percent) gave reasons for the choice of outcomes, and fifteen (14.5 percent) discriminated between primary and secondary outcomes. All mini-HTAs reported the databases searched and 99 percent of them reported using Medline. Sixty percent of the mini-HTA reported assessing the risk of bias, and 52 percent reported assessing the quality of evidence.

Conclusion

The quality of reporting of the mini-HTA reports performed in Brazil is insufficient and needs to be improved to guarantee transparency and replicability.

Rapid evidence synthesis approach for limits on the search date: How rapid could it be?

Rapid reviews have been widely employed to support timely decision-making, and limiting the search date is the most popular approach in published rapid reviews. We assessed the accuracy and workload of search date limits on the meta-analytical results to determine the best rapid strategy. The meta-analyses data were collected from the Cochrane Database of Systematic Reviews (CDSR). We emulated the rapid reviews by limiting the search date of the original CDSR to the recent 40, 35, 30, 25, 20, 15, 10, 7, 5, and 3 years, and their results were compared to the full meta-analyses. A random sample of 10% was drawn to repeat the literature search by the same timeframe limits to measure the relative workload reduction (RWR). The relationship between accuracy and RWR was established. We identified 21,363 meta-analyses of binary outcomes and 7683 meta-analyses of continuous outcomes from 2693 CDSRs. Our results suggested that under a maximum tolerance of 5% and 10% on the bias of magnitude, a limit on the recent 20 years can achieve good accuracy and at the same time save the most workload. Under the tolerance of 15% and 20% on the bias, a limit on the recent 10 years and 15 years could be considered. Limiting the search date is a valid rapid method to produce credible evidence for timely decisions. When conducting rapid reviews, researchers should consider both the accuracy and workload to make an appropriate decision.

Discrete choice experiment to determine preferences of decision-makers in healthcare for different formats of rapid reviews

BACKGROUND

Time-saving formats of evidence syntheses have been developed to fulfill healthcare policymakers' demands for timely evidence-based information. A discrete choice experiment (DCE) with decision-makers and people involved in the preparation of evidence syntheses was undertaken to elicit preferences for methodological shortcuts in the conduct of abbreviated reviews.

METHODS

D-efficient scenarios, each containing 14 pairwise comparisons, were designed for the DCE: the development of an evidence synthesis in 20 working days (scenario 1) and 12 months (scenario 2), respectively. Six attributes (number of databases, number of reviewers during screening, publication period, number of reviewers during data extraction, full-text analysis, types of HTA domains) with 2 to 3 levels each were defined. These were presented to the target population in an online survey. The relative importance of the individual attributes was determined using logistic regression models.

RESULTS

Scenario 1 was completed by 36 participants and scenario 2 by 26 participants. The linearity assumption was confirmed by the full model. In both scenarios, the linear difference model showed a preference for higher levels for "number of reviewers during data extraction", followed by "number of reviewers during screening" and "full-text analysis". Subgroup analyses showed that preferences were influenced by participation in the preparation of evidence syntheses.

CONCLUSION

The surveyed persons expressed preferences for quality standards in the process of literature screening and data extraction.

Rapid reviews: A critical perspective

BACKGROUND

The high scientific uncertainty of many far-reaching and serious political decisions during the "coronavirus crisis" underpins the enormous importance of having evidence syntheses that are quickly available and at the same time reliable. As these requirements can only be insufficiently fulfilled by systematic reviews due to the high amount of time required, abbreviated evidence syntheses in the form of rapid reviews are becoming increasingly popular.

PURPOSE

This commentary aims to enhance methodological and methodical discussions and research about abbreviated evidence syntheses.

METHODS

A selective literature search and evaluation focussing on research dealing with rapid reviews.

RESULTS

In rapid reviews, a wide variety of methods can be used to speed up the process of literature search and evaluation, while at the same time maintaining the principles of methodological quality and transparent reporting. But do rapid reviews currently keep what they promise? We discuss the increasing trend towards rapid reviews, giving the currently available evidence on the topic some critical reflection. Following this discussion, we will finally derive demands that go beyond the topic of rapid reviews alone.

Choose your shortcuts wisely: COVID-19 rapid reviews of traditional, complementary and integrative medicine

Background: The COVID-19 pandemic has led to an explosion of rapid reviews geared towards providing time sensitive answers for clinical and policy decision-makers. Rapid reviews (RRs) strike a balance between rigour and rapidity to minimise bias and optimise transparency within specified constraints. Methods: This review article appraised the methods and reporting standards of a convenience sample of RR protocols and RRs of COVID-19 clinical management questions, published in the first six-months of 2020. Inclusion criteria were all RR protocols evaluating traditional, complementary, and integrative medicine (TCIM) registered on PROSPERO, and all RRs indexed on PubMed or published on the Oxford COVID-19 Evidence Service. A purpose-specific 9-item reporting checklist reflecting recommended minimum requirements for RRs was applied. Findings were synthesised and narrated in the context of methodological considerations for conducting and reporting RRs of TCIM. Results: Included studies were five RR protocols of TCIM and 16 RRs, of which five considered TCIM. Wide variations in RR methods were proposed or applied, as were the reporting standards. All five RRs that evaluated TCIM had the lowest reporting standards that limited reproducibility and transparency. Despite accepted recommendations, most RRs did not publish a protocol. Conclusions: We propose that specific research disciplines, such as TCIM, have a uniqueness that may lead to unacceptable outputs if minimum methodological standards are not applied. The recommended minimum requirements will optimise the credibility of rapid reviews of TCIM and limit the risk of prematurely disregarding a potentially effective intervention.

Combining abbreviated literature searches with single-reviewer screening: three case studies of rapid reviews

BACKGROUND

Decision-makers increasingly request rapid answers to clinical or public health questions. To save time, personnel, and financial resources, rapid reviews streamline the methodological steps of the systematic review process. We aimed to explore the validity of a rapid review approach that combines a substantially abbreviated literature search with a single-reviewer screening of abstracts and full texts using three case studies.

METHODS

We used a convenience sample of three ongoing Cochrane reviews as reference standards. Two reviews addressed oncological topics and one addressed a public health topic. For each of the three topics, three reviewers screened the literature independently. Our primary outcome was the change in conclusions between the rapid reviews and the respective Cochrane reviews. In case the rapid approach missed studies, we recalculated the meta-analyses for the main outcomes and asked Cochrane review authors if the new body of evidence would change their original conclusion compared with the reference standards. Additionally, we assessed the sensitivity of the rapid review approach compared with the results of the original Cochrane reviews.

RESULTS

For the two oncological topics (case studies 1 and 2), the three rapid reviews each yielded the same conclusions as the Cochrane reviews. However, the authors would have had less certainty about their conclusion in case study 2. For case study 3, the public health topic, only one of the three rapid reviews led to the same conclusion as the Cochrane review. The other two rapid reviews provided insufficient information for the authors to draw conclusions. Using the rapid review approach, the sensitivity was 100% (3 of 3) for case study 1. For case study 2, the three rapid reviews identified 40% (4 of 10), 50% (5 of 10), and 60% (6 of 10) of the included studies, respectively; for case study 3, the respective numbers were 38% (8 of 21), 43% (9 of 21), and 48% (10 of 21).

CONCLUSIONS

Within the limitations of these case studies, a rapid review approach that combines abbreviated literature searches with single-reviewer screening may be feasible for focused clinical questions. For complex public health topics, sensitivity seems to be insufficient.

Impact of telemedicine on diagnosis, clinical management and outcomes in rural trauma patients: A rapid review

Introduction

Rural trauma patients are at increased risk of morbidity and mortality compared to trauma patients treated in urban facilities. Factors contributing to this disparity include differences in resource availability and increased time to definitive treatment for rural patients. Telemedicine can improve the early management of these patients by enabling rural providers to consult with trauma specialists at urban centres. The purpose of this study was to assess the impact of telemedicine utilisation on the diagnosis, clinical management and outcomes of rural trauma patients.

Materials and Methods

A rapid review of the literature was performed using the concepts 'trauma', 'rural' and 'telemedicine'. Fifteen electronic databases were searched from inception to 29^th June 2018. Manual searches were also conducted in relevant systematic reviews, key journals and bibliographies of included studies.

Results

The literature search identified 187 articles, of which 8 articles were included in the review. All 8 studies reported on clinical management, while the impact of telemedicine use on diagnosis and outcomes was reported in 4 and 5 studies, respectively. Study findings suggest that the use of telemedicine may improve patient diagnosis, streamline the process of transferring patients and reduce length of stay. Use of telemedicine had minimal impact on mortality and complications in rural trauma patients.

Conclusions

The evidence identified by this rapid review suggests that telemedicine may improve the diagnosis, management and outcomes of rural trauma patients. Further research is required to validate these findings by performing large and well-designed studies in rural areas, ideally as randomised clinical trials.

What constitutes patient-centred care for women: a theoretical rapid review

BACKGROUND

Women experience disparities in health care delivery and outcomes. Patient-centred care for women (PCCW) is needed. This study examined how PCC has been conceptualized and operationalized in women's health research.

METHODS

We conducted a theoretical rapid review of PCCW in MEDLINE, EMBASE, CINAHL and SCOPUS from 2008 to 2018 for studies involving women aged 18 years or greater with any condition, and analyzed data using an established 6-domain framework of patient-centred communication.

RESULTS

We included 39 studies, which covered the following clinical areas: maternal care, cancer, diabetes, HIV, endometriosis, dementia, distal radius fracture, overactive bladder, and lupus erythematosus. The 34 (87.2%) studies that defined or described PCC varied in the PCC elements they addressed, and none addressed all 6 PCC domains. Common domains were exchanging information (25, 73.5%) and fostering the patient-clinician relationship (22, 64.7%). Fewer studies addressed making decisions (16, 47.1%), enabling patient self-management (15, 44.1%), responding to emotions (12, 35.3%), or managing uncertainty (1, 2.9%). Compared with mixed-gender studies, those comprised largely of women more frequently prioritized exchanging information above other domains. Few studies tested strategies to support PCCW or evaluated the impact of PCCW; those that did demonstrated beneficial impact on patient knowledge, satisfaction, well-being, self-care and clinical outcomes.

CONCLUSIONS

Studies varied in how they conceptualized PCCW, and in many it was defined narrowly. Few studies examined how to implement or measure PCCW; thus, we lack insight on how to operationlize PCCW. Thus, further research is needed to confirm this, and whether PCCW differs across conditions, knowledge needed to inform policies, guidelines and measures aimed at improving health care and associated outcomes for women.

Excluding non-English publications from evidence-syntheses did not change conclusions: a meta-epidemiological study

OBJECTIVES

We aimed to assess whether limiting the inclusion criteria solely to English-language publications affected the overall conclusions of evidence syntheses.

STUDY DESIGN AND SETTING

Our analyses used a dataset of a previous methods study that included 59 randomly selected Cochrane intervention reviews with no language restrictions. First, we ascertained the publication language of all 2,026 included publications. Next, we excluded studies based on the following criteria: (1) publication solely in non-English language, or (2) main publication (in case of multiple publications of the same study) in non-English language. We then re-calculated meta-analyses for outcomes that were presented in the main summary of findings tables of the Cochrane reports. If the direction of the effect estimate or the statistical significance changed, authors of the respective Cochrane reviews were consulted to assess whether the new evidence base would have changed their conclusions. The primary outcome of our analyses examined the proportion of conclusions that would change with the exclusion of non-English publications. We set the threshold for the approach as noninferior if the upper limit of the 95% confidence interval of the proportion of changed conclusions did not cross a margin of 10%.

RESULTS

Across all 59 Cochrane reviews, 29 (49%) included 80 non-English publications. For 16 (27%) of these Cochrane reviews, the exclusion of non-English publications resulted in the exclusion of at least one study. In the remaining 13 Cochrane reviews, the non-English publications were not the only or main publication of the study or they did not contribute to the main summary of the findings table, so their exclusion did not result in an exclusion of the study. Overall, the exclusion of non-English publications led to the exclusion of 31 studies contributing to 40 outcomes. For 38 of the 40 outcomes, the exclusion of non-English studies did not markedly alter the size or direction of effect estimates or statistical significance. In two outcomes, the statistical significance changed, but authors would have still drawn the same conclusion, albeit with less certainty. Thus, the proportion of changed conclusions in our sample was 0.0% (95% CI 0.0-0.6), which indicated the noninferiority of the approach. However, the majority of excluded studies were small.

CONCLUSION

Exclusion of non-English publications from systematic reviews on clinical interventions had a minimal effect on overall conclusions and could be a viable methodological shortcut, especially for rapid reviews.

Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management

BACKGROUND

While systematic review (SR) methods are gaining traction as a method for providing a reliable summary of existing evidence for health risks posed by exposure to chemical substances, it is becoming clear that their value is restricted to a specific range of risk management scenarios - in particular, those which can be addressed with tightly focused questions and can accommodate the time and resource requirements of a systematic evidence synthesis.

METHODS

The concept of a systematic evidence map (SEM) is defined and contrasted to the function and limitations of systematic review (SR) in the context of risk management decision-making. The potential for SEMs to facilitate evidence-based decision-making are explored using a hypothetical example in risk management priority-setting. The potential role of SEMs in reference to broader risk management workflows is characterised.

RESULTS

SEMs are databases of systematically gathered research which characterise broad features of the evidence base. Although not intended to substitute for the evidence synthesis element of systematic reviews, SEMs provide a comprehensive, queryable summary of a large body of policy relevant research. They provide an evidence-based approach to characterising the extent of available evidence and support forward looking predictions or trendspotting in the chemical risk sciences. In particular, SEMs facilitate the identification of related bodies of decision critical chemical risk information which could be further analysed using SR methods, and highlight gaps in the evidence which could be addressed with additional primary studies to reduce uncertainties in decision-making.

CONCLUSIONS

SEMs have strong and growing potential as a high value tool in resource efficient use of existing research in chemical risk management. They can be used as a critical precursor to efficient deployment of high quality SR methods for characterising chemical health risks. Furthermore, SEMs have potential, at a large scale, to support the sort of evidence summarisation and surveillance methods which would greatly increase the resource efficiency, transparency and effectiveness of regulatory initiatives such as EU REACH and US TSCA.

Financial stress and mental health among higher education students in the UK up to 2018: rapid review of evidence

Introduction

In the United Kingdom and many other countries, debt accrued during higher education has increased substantially in recent decades. The prevalence of common mental health problems has also increased alongside these changes. However, it is as yet unclear whether there is an association between financial stress and mental health among higher education students.

Methods

We conducted a rapid review of the peer-reviewed scientific literature. Eligible studies were English-language publications testing the association between any indicator of financial stress and mental health among higher education students in the UK. Papers were located through a systematic search of PsychINFO, PubMed and Embase up to November 2018.

Results

The search strategy yielded 1272 studies—9 met the inclusion criteria. A further two were identified through hand-searching. The median sample size was 408. Only three of seven studies found an association between higher debt and worse mental health. There was a consistent cross-sectional relationship between worse mental health and both experience of financial difficulties (seven of seven studies) and debt worry/financial concern (four of five studies), though longitudinal evidence was mixed and limited to six studies.

Conclusion

Among higher education students in the UK, there is little evidence that the amount of debt is associated with mental health. However, more subjective measures of increased financial stress were more consistently associated with worse mental health outcomes. Nevertheless, the identified evidence was judged to be weak; further research is required to examine whether links between financial stress and mental health outcomes are robust and causal in nature.

Delphi consensus reached to produce a decision tool for SelecTing Approaches for Rapid Reviews (STARR)

OBJECTIVES

There are many rapid review methods; however, there is little pragmatic guidance on which methods to select. This study aimed to reach consensus among international rapid review experts outlining areas to consider when selecting approaches for rapid reviews.

STUDY DESIGN AND SETTING

A two-round modified online Delphi survey was conducted between May and July 2018. Participants were asked to rank the importance of a predefined list of 19 items. A consensus definition of at least 70% agreement for each item was decided a priori.

RESULTS

Thirty experts from ten countries participated in round 1 and 24 in round 2. During round 1, consensus was reached on all items. One additional item on quality assessment was suggested by respondents and comments suggested wording changes to improve clarity and understanding of the tool. Respondents in the second round indicated a high level of importance and all 20 items achieved consensus. These items addressed interaction with commissioners, scoping and searching the evidence-base, data extraction and synthesis methods, and reporting of rapid review methods.

CONCLUSION

International consensus was reached to produce the SelecTing Approaches for Rapid Reviews (STARR) decision tool for planning rapid reviews and will lead to improved shared understanding between review teams and review commissioners.

Decision aids for people with Type 2 diabetes mellitus: an effectiveness rapid review and meta-analysis

AIMS

To perform a rapid review and meta-analysis of randomized controlled trials (RCTs) evaluating patient decision aids (PtDAs) for people with Type 2 diabetes mellitus.

METHODS

We searched Medline and the Cochrane Library for RCTs assessing PtDAs in people with Type 2 diabetes. PtDAs were defined as tools designed to help people engage in decision-making about healthcare options, such as making treatment choices or setting therapeutic goals. The study selection process was facilitated by an automated screening tool to identify RCTs. We classified outcomes into seven domains and conducted meta-analyses using random effects models.

RESULTS

We included a total of 15 studies, nine of which were cluster RCTs, that evaluated 10 PtDAs. Thirteen trials compared a PtDA with usual care or usual care plus educational material, whereas two RCTs compared individually tailored vs. non-tailored PtDAs. Meta-analyses showed a favourable effect of PtDAs compared with usual care in reducing decisional conflict [weighted mean difference (WMD) -4.66, 95% confidence interval (CI) -7.93 to -1.39] and in improving knowledge (WMD 20.46, 95% CI 9.13 to 3.77). Use of PtDAs resulted in more active involvement in decision-making during the consultation, although no effect was evident in terms of glycaemic control or self-reported medication adherence.

CONCLUSIONS

PtDAs for people with Type 2 diabetes can improve the quality of decision-making and increase knowledge transfer. Interpretation of our findings is attenuated due to limitations related to the rapid review approach, including searching only two databases and performing data extraction and risk of bias assessment by a single reviewer.

How is patient-centred care addressed in women's health? A theoretical rapid review

PURPOSE

Efforts are needed to reduce gendered inequities and improve health and well-being for women. Patient-centred care (PCC), an approach that informs and engages patients in their own health, is positively associated with improved care delivery, experiences and outcomes. This study aimed to describe how PCC for women (PCCW) has been conceptualised in research.

METHODS

We conducted a theoretical rapid review of PCCW in four health conditions. We searched MEDLINE, EMBASE, CINAHL, SCOPUS, Cochrane Library and Joanna Briggs index for English-language articles published from January 2008 to February 2018 inclusive that investigated PCC and involved at least 50% women aged 18 or older. We analysed findings using a six-domain PCC framework, and reported findings with summary statistics and narrative descriptions.

RESULTS

After screening 2872 unique search results, we reviewed 51 full-text articles, and included 14 (five family planning, three preventive care, four depression, one cardiovascular disease and one rehabilitation). Studies varied in how they assessed PCC. None examined all six PCC framework domains; least evaluated domains were addressing emotions, managing uncertainty and enabling self-management. Seven studies that investigated PCC outcomes found a positive association with appropriate health service use, disease remission, health self-efficacy and satisfaction with care. Differing views about PCC between patients and physicians, physician PCC attitudes and geographic affluence influenced PCC. No studies evaluated the influence of patient characteristics or tested interventions to support PCCW.

CONCLUSION

There is a paucity of research that has explored or evaluated PCCW in the conditions of interest. We excluded many studies because they arbitrarily labelled many topics as PCC, or simply concluded that PCC was needed. More research is needed to fully conceptualise and describe PCCW across different characteristics and conditions, and to test interventions that improve PCCW. Policies and incentives may also be needed to stimulate greater awareness and delivery of PCCW.

Rapid reviews may produce different results to systematic reviews: a meta-epidemiological study

Objective

To simulate possible changes in systematic review results if rapid review methods were used.

Study Design and Setting

We recalculated meta-analyses for binary primary outcomes in Cochrane systematic reviews, simulating rapid review methods. We simulated searching only PubMed, excluding older articles (5, 7, 10, 15, and 20 years before the search date), excluding smaller trials (<50, <100, and <200 participants), and using the largest trial only. We examined percentage changes in pooled odds ratios (ORs) (classed as no important change [<5%], small [<20%], moderate [<30%], or large [≥30%]), statistical significance, and biases observed using rapid methods.

Results

Two thousand five hundred and twelve systematic reviews (16,088 studies) were included. Rapid methods resulted in the loss of all data in 3.7–44.7% of meta-analyses. Searching only PubMed had the smallest risk of changed ORs (19% [477/2,512] were small changes or greater; 10% [260/2,512] were moderate or greater). Changes in ORs varied substantially with each rapid review method; 8.4–21.3% were small, 1.9–8.8% were moderate, and 4.7–34.1% were large. Changes in statistical significance occurred in 6.5–38.6% of meta-analyses. Changes from significant to nonsignificant were most common (2.1–13.7% meta-analyses). We found no evidence of bias with any rapid review method.

Conclusion

Searching PubMed only might be considered where a ∼10% risk of the primary outcome OR changing by >20% could be tolerated. This could be the case in scoping reviews, resource limitation, or where syntheses are needed urgently. Other situations, such as clinical guidelines and regulatory decisions, favor more comprehensive systematic review methods.

Rapid response in health technology assessment: a Delphi study for a Brazilian guideline

Background

Rapid response in health technology assessment is a synthesis of the best available evidence prepared in a timely manner to meet specific demands. We build a consensus among Brazilian specialists in health technology assessment to propose guidelines for the development of rapid response.

Methods

Based on a systematic review that proposed eight methodological steps to conduct rapid response, we applied a modified Delphi technique (without open questions in the first round) to reach consensus among Brazilian experts in health technology assessment. Twenty participants were invited to judge the feasibility of each methodological step in a five-point Likert scale. Consensus was reached if the step had 70% positive approval or interquartile range ≤ 1.

Results

The achievement of consensus was reached in the second round. Between the first and the second round, we scrutinized all points reported by the experts. The Delphi panel reached consensus of eight steps: definition of the structured question of rapid response (with a restricted scope); definition of the eligibility criteria for study types (preferably systematic reviews); search strategy (language and data limits) and sources of information (minimum two); selection of studies (independently by two responders); critical appraisal of the included studies and the risk of bias for the outcomes of interest; data extraction from the included articles; summary of evidence; and preparation of the report.

Conclusions

The guidelines for rapid response in health technology assessment may help governments to make better decisions in a short period of time (35 days). The adoption of methodological processes should improve both the quality and consistency of health technology assessments of rapid decisions in the Brazilian setting.