Efficiency of pragmatic search strategies to update clinical guidelines recommendations

Evidence surveillance for a living clinical guideline: Case study of the Australian stroke guidelines

BACKGROUND

Continual evidence surveillance is an integral feature of living guidelines. The Australian Stroke Guidelines include recommendations on 100 clinical topics and have been 'living' since 2018.

OBJECTIVES

To describe the approach for establishing and evaluating an evidence surveillance system for the living Australian Stroke Guidelines.

METHODS

We developed a pragmatic surveillance system based on an analysis of the searches for the 2017 Stroke Guidelines and evaluated its reliability by assessing the potential impact on guideline recommendations. Search retrieval and screening workload are monitored monthly, together with the frequency of changes to the guideline recommendations.

RESULTS

Evidence surveillance was guided by practical considerations of efficiency and sustainability. A single PubMed search covering all guideline topics, limited to systematic reviews and randomised trials, is run monthly. The search retrieves about 400 records a month of which a sixth are triaged to the guideline panels for further consideration. Evaluations with Epistemonikos and the Cochrane Stroke Trials Register demonstrated the robustness of adopting this more restrictive approach. Collaborating with the guideline team in designing, implementing and evaluating the surveillance is essential for optimising the approach.

CONCLUSION

Monthly evidence surveillance for a large living guideline is feasible and sustainable when applying a pragmatic approach.

An adapted 'Ottawa' method allowed assessing the need to update topic areas within clinical practice guidelines

OBJECTIVE

To adapt and evaluate a method for assessing the need to update guideline topic areas involving multiple recommendations.

STUDY DESIGN AND SETTING

The 'Ottawa method' uses literature signals to determine changes in evidence that trigger a need to update individual guideline questions. We adapted the Ottawa method to include a process for aggregating updating signals by topic area (e.g., resuscitation) and tested this method using the German guideline on the treatment of patients with severe/multiple injuries. This involved a focused systematic evaluation of current evidence to identify updating signals, and classifying the need to update for each topic area. Then, we surveyed the guideline group online about the modified method.

RESULTS

We conducted focused literature searches for 37 topic areas and screened a mean of 97 abstracts per topic area in 2021. The need to update was high for eight (21.6%), intermediate for eight (21.6%), and low for 21 topic areas (56.8%) based on updating signals. The survey response rate was 56% (24/43). Most guideline group members (94%, 16/17 responders) would use the Ottawa method again, but their comments identified some weaknesses.

CONCLUSION

The modified Ottawa method is a suitable, efficient tool to generate evidence-based updating signals for guideline topic areas involving multiple recommendations. Further fine-tuning is recommended.

Outcome Measures for Dementia With Lewy Body Clinical Trials: A Review

BACKGROUND

Dementia with Lewy bodies (DLB) is one of the most common degenerative dementias. Clinical trials for individuals with DLB are increasing. We aimed to identify commonly used outcome measures for trials in DLB.

METHODS

A pragmatic literature search of PubMed and clinicaltrials.gov identified interventional studies including populations with DLB. Studies were included if they enrolled participants with DLB and met the National Institutes of Health criteria for a clinical trial. Data were collected using standardized forms. Outcome measures were categorized according to core and supportive features of DLB.

RESULTS

After de-duplication, 58 trials were identified. The most common cognitive outcome measures were the Mini Mental State Examination (n=24) and Cognitive Drug Research computerized Assessment System (n=5). The Clinician's Assessment of Fluctuations was the most commonly used measure for fluctuations (n=4). Over half of studies used the Neuropsychiatric Inventory to assess behavioral symptoms (n=31). The Unified Parkinson's Disease Rating Scale was frequently used for motor assessment (n=23).

CONCLUSIONS AND RELEVANCE

Clinical trial outcomes used in DLB are rarely validated in this population and some lack face validity. There is a need to validate existing scales in DLB and develop DLB-specific outcome measures.

Current practices and challenges in adaptation of clinical guidelines: a qualitative study based on semistructured interviews

OBJECTIVE

This study aims to better understand the current practice of clinical guideline adaptation and identify challenges raised in this process, given that published adapted clinical guidelines are generally of low quality, poorly reported and not based on published frameworks.

DESIGN

A qualitative study based on semistructured interviews. We conducted a framework analysis for the adaptation process, and thematic analysis for participants' views and experiences about adaptation process.

SETTING

Nine guideline development organisations from seven countries.

PARTICIPANTS

Guideline developers who have adapted clinical guidelines within the last 3 years. We identified potential participants through published adapted clinical guidelines, recommendations from experts, and a review of the Guideline International Network Conference attendees' list.

RESULTS

We conducted ten interviews and identified nine adaptation methodologies. The reasons for adapting clinical guidelines include developing de novo clinical guidelines, implementing source clinical guidelines, and harmonising and updating existing clinical guidelines. We identified the following core steps of the adaptation process (1) selection of scope and source guideline(s), (2) assessment of source materials (guidelines, recommendations and evidence level), (3) decision-making process and (4) external review and follow-up process. Challenges on the adaptation of clinical guidelines include limitations from source clinical guidelines (poor quality or reporting), limitations from adaptation settings (lacking resources or skills), adaptation process intensity and complexity, and implementation barriers. We also described how participants address the complexities and implementation issues of the adaptation process.

CONCLUSIONS

Adaptation processes have been increasingly used to develop clinical guidelines, with the emergence of different purposes. The identification of core steps and assessment levels could help guideline adaptation developers streamline their processes. More methodological research is needed to develop rigorous international standards for adapting clinical guidelines.

UEG framework for the development of high-quality clinical guidelines

The 48 national member societies and 17 specialist member societies which operate under the umbrella of United European Gastroenterology (UEG) increasingly develop clinical practice guidelines for both national and international implementation. The methodologies and strategies used in these guidelines vary considerably. The UEG Quality of Care Taskforce aimed to provide a framework for quality guidelines in order to assist member societies in the process of developing guidelines, and to provide a tool for readers of guidelines to critically appraise their quality. We outline the steps necessary to begin the guideline development process, how to build working groups, how to search for evidence, how to grade the quality of the evidence, how to reach consensus on statements and how to write the guideline document. We believe that using this framework will increase the potential to produce a high-quality guideline which is transparent, independent, reproducible and implementable.

Nutrimedia: A novel web-based resource for the general public that evaluates the veracity of nutrition claims using the GRADE approach

The objective of Nutrimedia is to evaluate, based on the scientific evidence, the veracity of nutrition claims disseminated to the public by the media. In this article, we describe the methodology, characteristics and contents of this web-based resource, as well as its web traffic and media impact since it was launched. Nutrimedia uses a systematic process to evaluate common beliefs, claims from newspapers and advertising identified and selected by its research team, as well as questions from the public. After formulating a structured question for each claim, we conduct a pragmatic search, prioritizing guidelines and/or systematic reviews. We evaluate the certainty of the evidence using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach, and classify the veracity of each claim into seven categories (true, probably true, possibly true, possibly false, probably false, false, and uncertain). For each evaluation, we develop a scientific report, a plain language summary, a summary of findings table, and, in some cases, a video. From November 2017 to May 2019, we published 30 evaluations (21 were related to foods, six to diets, and three to supplements), most of which were triggered by questions from the public (40%; 12/30). Overall, nearly half of the claims were classified as uncertain (47%; 14/30). Nutrimedia received 47,265 visitors, with a total of 181,360 pages viewed. The project and its results were reported in 84 written media and 386 websites from Spain and 14 other countries, mostly from Latin America. To our knowledge, Nutrimedia is the first web-based resource for the public that evaluates the certainty of evidence and the veracity of nutrition claims using the GRADE approach. The scientific rigor combined with the use of friendly presentation formats are distinctive features of this resource, developed to help the public to make informed choices about nutrition.

Study-based registers reduce waste in systematic reviewing: discussion and case report

BACKGROUND

Maintained study-based registers (SBRs) have, at their core, study records linked to, potentially, multiple other records such as references, data sets, standard texts and full-text reports. Such registers can minimise and refine searching, de-duplicating, screening and acquisition of full texts. SBRs can facilitate new review titles/updates and, within seconds, inform the team about the potential workload of each task.

METHODS

We discuss the advantages/disadvantages of SBRs and report a case of how such a register was used to develop a successful grant application and deliver results-reducing considerable redundancy of effort.

RESULTS

SBRs saved time in question-setting and scoping and made rapid production of nine Cochrane systematic reviews possible.

CONCLUSION

Whilst helping prioritise and conduct systematic reviews, SBRs improve quality. Those funding information specialists for literature reviewing could reasonably stipulate the resulting SBR to be delivered for dissemination and use beyond the life of the project.

GRADE-Based Recommendations for Surgical Repair of Nonruptured Abdominal Aortic Aneurysm

The objective of this study was to provide evidence-based recommendations for endovascular aneurysm repair (EVAR) versus open surgical repair (OSR) for patients with a nonruptured abdominal aortic aneurysm (AAA). We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis statement and adhered to the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework. Both low- and high surgical risk patients treated with EVAR showed decreased 30-day mortality, but the low-risk group had no differences in 4-year mortality. Compared with friendly anatomy, patients with hostile anatomy had an increased risk of type I endoleak. Young patients may prefer OSR. Endovascular aneurysm repair was not cost-effective in Europe. Four conditional recommendations were formulated: (1) OSR for low-risk patients up to 80 years old, (2) EVAR for low-risk patients older than 80 years, (3) EVAR for high-risk patients as long as is anatomically feasible, and (4) OSR in patients in whom it is not anatomically feasible to perform EVAR. Based on GRADE criteria, either OSR or EVAR can be suggested to patients with nonruptured AAA taking into account their surgical risk, hostile anatomy, and age. Given the weakness of the recommendations, personal preferences are determinant.

Intermittent androgen deprivation therapy: recommendations to improve the management of patients with prostate cancer following the GRADE approach

PURPOSE

The purpose of this study was to provide evidence-based recommendations of intermittent androgen deprivation therapy (IADT) compared with continuous androgen deprivation therapy (CADT) for men with prostate cancer (PCA).

METHODS

We conducted a comprehensive search in MEDLINE, EMBASE, The Cochrane Library, CINAHL, and ECONLIT, from the database inception to December 2017. We adhered to the Grading of Recommendations, Assessment, Development and Evaluation framework to assess the quality of the evidence and to formulate recommendations.

RESULTS

We included one systematic review with 15 trials as well as three additional studies that assessed IADT versus CADT, all of them focused on PCA patients in advanced stages. The findings did not show differences for critical and important outcomes, including adverse events. Trials reported the benefits of IADT in terms of selected domains of health-related quality of life, although with high heterogeneity. Evidence quality was considered moderate or low for most of the assessed outcomes. We identified a patient preference study reporting a high preference for IADT, due to issues related to quality of life, general well-being, and side effects, among others. We did not identify economic studies comparing these regimes. We formulate four recommendations: one no-recommendation, one conditional recommendation, and two good practice points.

CONCLUSION

For men in early stages of PCA, it is not possible to make any recommendation about the preferable use of IADT or CADT due to the lack of available evidence. For men in advanced stages of the disease, an IADT should be considered as soon as clinically reasonable (weak recommendation and low certainty of the evidence). Clinicians should discuss the risks and benefits of IADT and CADT with their patients, taking into account their values and preferences.

A systematic decision-making process on the need for updating clinical practice guidelines proved to be feasible in a pilot study

OBJECTIVES

The objective of this study was to test and evaluate a new decision-making process on the need for updating within the update of a German clinical practice guideline (CPG).

STUDY DESIGN AND SETTING

The pilot study comprised (1) limited searches in Pubmed to identify new potentially relevant evidence, (2) an online survey among the members of the CPG group to assess the need for update, and (3) a consensus conference for determination and prioritization of guideline sections with a high need for update. Subsequently, we conducted a second online survey to evaluate the procedure.

RESULTS

The searches resulted in 902 abstracts that were graded as new potentially relevant evidence. Twenty five of 39 members of the CPG group (64%) participated in the online survey. Seventy six percent of those took part in the second online survey. The evaluation study found on average a grade of support of the procedure regarding the determination of the need for update of 3.65 (standard deviation: 0.76) on a likert scale with 1 = "no support" to 5 = "very strong support."

CONCLUSION

The conducted procedure presents a systematic approach for assessing whether and to what extent a CPG requires updating and enables setting priorities for which particular guideline section to update within a CPG.

Updated clinical guidelines experience major reporting limitations

BACKGROUND

The Checklist for the Reporting of Updated Guidelines (CheckUp) was recently developed. However, so far, no systematic assessment of the reporting of updated clinical guidelines (CGs) exists. We aimed to examine (1) the completeness of reporting the updating process in CGs and (2) the inter-observer reliability of CheckUp.

METHODS

We conducted a systematic assessment of the reporting of the updating process in a sample of updated CGs using CheckUp. We performed a systematic search to identify updated CGs published in 2015, developed by a professional society, reporting a systematic review of the evidence, and containing at least one recommendation. Three reviewers independently assessed the CGs with CheckUp (16 items). We calculated the median score per item, per domain, and overall, converting scores to a 10-point scale. Multiple linear regression analyses were used to identify differences according to country, type of organisation, scope, and health topic of updated CGs. We calculated the intraclass coefficient (ICC) and 95% confidence interval (95% CI) for domains and overall score.

RESULTS

We included in total 60 updated CGs. The median domain score on a 10-point scale for presentation was 5.8 (range 1.7 to 10), for editorial independence 8.3 (range 3.3 to 10), and for methodology 5.7 (range 0 to 10). The median overall score on a 10-point scale was 6.3 (range 3.1 to 10). Presentation and justification items at recommendation level (respectively reported by 27 and 38% of the CGs) and the methods used for the external review and implementing changes in practice were particularly poorly reported (both reported by 38% of the CGs). CGs developed by a European or international institution obtained a statistically significant higher overall score compared to North American or Asian institutions (p = 0.014). Finally, the agreement among the reviewers on the overall score was excellent (ICC 0.88, 95% CI 0.75 to 0.95).

CONCLUSIONS

The reporting of updated CGs varies considerably with significant room for improvement. We recommend using CheckUp to assess the updating process in updated CGs and as a blueprint to inform methods and reporting strategies in updating.

Continuous surveillance of a pregnancy clinical guideline: an early experience

BACKGROUND

To date there is no consensus about the optimal strategy for keeping clinical guidelines (CGs) up-to-date. The aims of this study were (1) to develop a continuous surveillance and updating strategy for CGs and (2) to test the strategy in a specific CG.

METHODS

The main steps were as follows: (1) recruiting members for the CG Updating Working Group, (2) mapping the CG, (3) identifying new evidence from the CG Updating Working Group, (4) designing and running restricted literature searches, (5) reviewing drugs and medical devices alerts, (6) screening and assessing the new evidence, (7) reviewing and, if necessary, modifying clinical questions and recommendations, and (8) updating the CG document.

RESULTS

The Pregnancy CG Updating Working Group consisted of 29 members, including clinicians, patients and caregivers, and clinical guideline methodology experts. We selected 69 clinical questions (123 recommendations) from the "Assistance during pregnancy" section. For the first update cycle (32-month duration), 9710 references were identified. Of these, 318 were pertinent, 289 were relevant, and 55 were classified as potential key references. For the second and third update cycles (6-month duration each), 2160 and 2010 references were retrieved, respectively. The continuous surveillance and updating strategy has not yet been completely implemented.

CONCLUSIONS

Further resources are needed in updating the CG field, both for implementing updating strategies and for developing methodological research.

Reporting Items for Updated Clinical Guidelines: Checklist for the Reporting of Updated Guidelines (CheckUp)

BACKGROUND

Scientific knowledge is in constant development. Consequently, regular review to assure the trustworthiness of clinical guidelines is required. However, there is still a lack of preferred reporting items of the updating process in updated clinical guidelines. The present article describes the development process of the Checklist for the Reporting of Updated Guidelines (CheckUp).

METHODS AND FINDINGS

We developed an initial list of items based on an overview of research evidence on clinical guideline updating, the Appraisal of Guidelines for Research and Evaluation (AGREE) II Instrument, and the advice of the CheckUp panel (n = 33 professionals). A multistep process was used to refine this list, including an assessment of ten existing updated clinical guidelines, interviews with key informants (response rate: 54.2%; 13/24), a three-round Delphi consensus survey with the CheckUp panel (33 participants), and an external review with clinical guideline methodologists (response rate: 90%; 53/59) and users (response rate: 55.6%; 10/18). CheckUp includes 16 items that address (1) the presentation of an updated guideline, (2) editorial independence, and (3) the methodology of the updating process. In this article, we present the methodology to develop CheckUp and include as a supplementary file an explanation and elaboration document.

CONCLUSIONS

CheckUp can be used to evaluate the completeness of reporting in updated guidelines and as a tool to inform guideline developers about reporting requirements. Editors may request its completion from guideline authors when submitting updated guidelines for publication. Adherence to CheckUp will likely enhance the comprehensiveness and transparency of clinical guideline updating for the benefit of patients and the public, health care professionals, and other relevant stakeholders.