The UpPriority tool was developed to guide the prioritization of clinical guideline questions for updating

Axial spondyloarthritis guidelines - aiming for maximum impact

PURPOSE OF REVIEW

This review discusses international clinical practice guidelines (CPGs) for axial spondyloarthritis (axSpA) focusing on methodology, guideline quality, and implementation.

RECENT FINDINGS

The Assessment of SpondyloArthritis International Society/European Alliance of Associations for Rheumatology (ASAS/EULAR) and Pan-American League of Associations for Rheumatology (PANLAR) recently published axSpA CPGs and updates of the American College of Rheumatology/Spondylitis Association of America/Spondyloarthritis Research and Treatment Network (ACR/SAA/SPARTAN) and Asia-Pacific League of Associations for Rheumatology (APLAR) CPGs are expected. GRADE has emerged as the dominant framework for CPG development and has been used by three of the four international axSpA guidelines. Notable differences exist among these guidelines in the way that the recommendations are presented. Two of the four acknowledge the need for implementation strategies, but little detail about this is provided. The few studies that have evaluated the implementation of axSpA CPGs have identified poor adherence to recommendations on physical therapy/exercise and disease activity monitoring. Implementation science has identified many barriers and facilitators affecting guideline uptake, including those related to healthcare professionals and to the guidelines themselves. Creation of a tailored implementation plan simultaneously with the CPG is recommended.

SUMMARY

While methodological rigor in the creation of evidence-based recommendations is the focus of CPG development, recommendations must be presented in a user-friendly format that makes them easy to apply. 'Living guidelines' could facilitate keeping content up to date. Implementation is critical for the success of a CPG and should be emphasized in future axSpA guideline updates. Further research is needed to better understand the factors impacting the successful implementation of axSpA CPGs.

Introducing re-weighted range voting in clinical practice guideline prioritization: Development and testing of the re-weighted priority-setting (REPS) tool

We aimed to develop and test a tool based on the re-weighted range voting mechanism to prioritize items (i.e. key questions) in a priority-setting assessment for clinical practice guidelines. The secondary aim was to provide methodological context of the tool. We iteratively developed the tool and used qualitative methods (i.e. think-aloud and semi-structured interviews) to test the tool's usability and make adjustments accordingly. An observational approach was used to test the tool's outcome satisfaction in a real-world priority-setting assessment within a rare-disease guideline of a European Reference Network and under four different conditions in the tool. Four guideline methodologists tested the usability of the tool. The real-world testing was performed with a guideline panel consisting of a core working group, five expertise working groups, and a working group with patient representatives. Thirty-one panel members assigned scores in the priority-setting assessment. Seventeen panel members rated the priority-setting outcome, and sixteen panel members rated the outputs generated under the four conditions. Upon initial use, guideline methodologists found the tool to be quite overwhelming. However, with some initial effort they were able to easily identify the tool's structure. Based on observations and feedback, the tool was further refined and user guidance was developed. Guideline panel members expressed (high) satisfaction with the priority-setting outcome. They particularly preferred the condition when using mean subgroup scores as input or employing aggressive penalties in the weighting method to determine the outputs. The tool generates a ranked list of items and offers flexibility for different choices in priority-setting assessments as long as its input format requirements are met. Although it is not a consensus method, the tool assists in narrowing down a set of priority items. Additional steps in the priority-setting assessment can lead to a consensus being reached regarding the final outcome.

Needs and feasibility of living systematic reviews (LSRs): Experience from LSRs on COVID-19 vaccine effectiveness

During 2021 and 2023, a team of researchers at the Robert Koch Institute (RKI) and partnering institutions conducted two living systematic reviews (LSRs) on the effectiveness of COVID-19 vaccines in different age groups to inform recommendations of the Standing Committee on Vaccination in Germany (Ständige Impfkommission, STIKO). Based on our experience from the realization of these LSRs, we developed certain criteria to assess the needs and feasibility of conducting LSRs. Combining these with previously established criteria, we developed the following set to inform future planning of LSRs for STIKO: Needs criterion (N)1: Relevance of the research question, N2: Certainty of evidence (CoE) at baseline; N3: Expected need for Population-Intervention-Comparator-Outcome (PICO) adaptations; N4: Expected new evidence over time; N5: Expected impact of new evidence on CoE; Feasibility criterion (F)1: Availability of sufficient human resources; F2: Feasibility of timely dissemination of the results to inform decision-making. For each criterion we suggest rating options which may support the decision to conduct an LSR or other forms of evidence synthesis when following the provided flowchart. The suggested criteria were developed on the basis of the experiences from exemplary reviews in a specific research field (i.e., COVID-19 vaccination), and did not follow a formal development or validation process. However, these criteria might also be useful to assess whether questions from other research fields can and should be answered using the LSR approach, or assist in determining whether the use of an LSR is sensible and feasible for specific questions in health policy and practice.

[Development of criteria for the prospective assessment of the need for updating guideline recommendations: The AGIL criteria]

BACKGROUND

Evidence-based guideline and vaccination recommendations should continuously be updated to appropriately support health care decisions. However, resources for updating guidelines are often limited. The aim of this project was to develop a list of criteria for the prospective assessment of the need for updating individual guideline or vaccination recommendations, which can be applied from the time a guideline or guideline update is finalised.

METHODS

In this article we describe the development of the AGIL criteria (Assessment of Guidelines for Updating Recommendations). The AGIL criteria were developed by experienced scientists and experts in the field of guideline development in a multi-step process. The five steps included: 1) development of an initial list of criteria by the project team; 2) online survey of guideline experts on the initial version of the criteria list; 3) revision of the criteria list based on the results of the online survey; 4) workshop on the criteria list at the EbM Congress 2023; 5) creation of version 1.0 of the AGIL criteria based on the workshop results.

RESULTS

The initial list included the following three criteria: 1) relevance of the question 2) availability of new relevant evidence, and 3) impact of potentially new evidence. The response rate of the online survey for fully completed questionnaires was 31.0% (N=195; 630 guideline experts were contacted by email). For 90.3% (n=176) of the respondents, the criteria list included all essential aspects for assessing the need for updating guideline recommendations. More than three quarters of respondents rated the importance of the three criteria as "very important" or "important" (criteria 1-3: 75.3%, 86.1%, 85.2%) and - with the exception of criterion 1 - comprehensibility as "very comprehensible" or "comprehensible" (criteria 1-3: 58.4%, 75.9%, 78.5%). The results of the online survey and the workshop generally confirmed the three criteria with their two sub-questions. The incorporation of all feedback resulted in the AGIL criteria (version 1.0), recapping: 1) relevance of the question regarding a) PICO components and b) other factors, e.g. epidemiological aspects; 2) availability of new evidence a) on health-related benefits and harms and b) on other decision factors, e.g. feasibility, acceptability; 3) impact of new evidence a) on the certainty of evidence on which the recommendation is based and b) on the present recommendation, e.g.

STRENGTH OF RECOMMENDATION

DISCUSSION

The moderate response rate of the online survey may have limited its representativeness. Nevertheless, we consider the response rate to be satisfactory in this research context. The inclusion of many experts in the online survey and the EbM Congress workshop is a strength of the project and supports the quality of the results.

CONCLUSIONS

The AGIL criteria provide a structured guidance for the prospective assessment of the need for updating individual guideline recommendations and other evidence-based recommendations. The implementation and evaluation of the AGIL criteria 1.0 in a field test is planned.

Efficiency based updating of evidence based surgical guidelines - Experiences from a multidisciplinary guideline upon Minimally Invasive Surgery

Background

Updating evidence-based clinical practice guidelines is an onerous process and there is a call for more efficient determination of key questions that need updating. Especially for surgical techniques it is unclear if new evidence will result in substantial changes after wide implementation and if continuous updating is always necessary.

Objectives

This study analyses the impact of updating a surgical guideline and proposes suggestions for optimising this process.

Materials and methods

The Dutch Minimally Invasive Surgery guideline was developed in 2011 and updated in 2021. For both versions a multidisciplinary guideline working group (GDG) was created, that determined key questions. Changes in conclusions and recommendations were analysed by the GDG and statements for expected change of recommendations in the future were made.

Results

15 key questions were formed, of which 12 were updates of the previous guideline. For only 27% of the updated key questions, the conclusions changed. In ten years, the body grew only marginally for most key questions and quality of the evidence did not improve substantially for almost all key questions. However, in this first update of the MIC guideline, many recommendations did change due to a more robust interpretation of the conclusions by the GDG. Based on analysis of this updating process, the GDG expects that only four out of 15 recommendations may change in the future.

Conclusion

We propose an additional step at the end of guideline development and updating, where the necessity for updating in the future is determined for each key question by the GDG, using their valuable knowledge gained from developing or updating the guideline. For surgical guidelines, the authors suggest updating key issues if it includes a relatively newly introduced surgical- or adapted technique or a new patient group. Low quality or small body of evidence should not be a reason in itself for updating, as this mostly does not lead to new evidence-based conclusions. This new step is expected to result in a more efficient prioritising of key questions that need updating.

What’s new?

By adding one additional step at the end of the updating process, the future updating process could become more efficient.

[Expectations of guideline developers on living guidelines]

BACKGROUND

Guidelines are central pillars of high-quality care and care planning. The quality requirements for the development of guidelines and the associated effort are very high. Therefore, more efficient approaches are being forced.

OBJECTIVE

The opportunities and challenges in the introduction of a dynamic updating concept within the framework of the digitalization of guidelines were explored from the perspective of guideline developers in the field of psychiatry. This perspective should be included in the implementation.

MATERIAL AND METHODS

Between January and May 2022 a cross-sectional survey was conducted among guideline developers (N = 561, response 39%) based on a questionnaire developed and tested in advance. Data were analyzed descriptively.

RESULTS

A total of 60% were familiar with the concept of living guidelines. A large proportion endorsed a dynamic updating (83%) and digitalization (88%) of guidelines; however, various challenges are associated with the concept of living guidelines: the risk of inflationary changes (34%), the need for continuity of all actors (53%), involvement of patient and family representatives (37%) and definition of criteria regarding the decision what to change (38%). The vast majority (85%) thought it necessary that guideline development should be followed by implementation projects.

CONCLUSION

German guideline developers are very receptive regarding the implementation of living guidelines; however, they named numerous challenges, which need to be addressed in this approach.

A Delphi consensus checklist helped assess the need to develop rapid guideline recommendations

BACKGROUND AND OBJECTIVES

We aimed to develop a checklist to aid guideline developers in determining which scientific or societal cause ("triggers") are relevant when considering to initiate a rapid recommendation procedure.

METHODS

We conducted a two-round modified Delphi procedure with a panel of Dutch guideline experts, clinicians, and patient representatives. A previously conducted systematic literature review and semistructured interviews with four science journalists were used to generate a list of potential items. This item list was submitted to the panel for discussion, reduction and refinement into a checklist.

RESULTS

Thirteen experts took part. Two questionnaires were completed in which participants scored an initial list of 64 items based on relevance. During two online meetings, the scores were discussed, irrelevant items were removed, and relevant items were reformulated into seven questions. The final "quickscan assessment of the need for a rapid recommendation" covers user perspective, scientific evidence, clinical relevance, clinical practice variation, applicability, quality of care and public health outcomes, and ethical/legal considerations.

CONCLUSION

The quickscan aids guideline developers in systematically assessing whether a trigger expresses a valid need for developing a rapid recommendation. Future research could focus on the applicability and validity of the checklist within guideline development programs.

Methods for living guidelines: early guidance based on practical experience. Paper 3: selecting and prioritizing questions for living guidelines

OBJECTIVES

This article is part of a series on methods for living guidelines, consolidating practical experiences from developing living guidelines. It focuses on methods for identification, selection, and prioritization of clinical questions for a living approach to guideline development.

STUDY DESIGN AND SETTING

Members of the Australian Living Evidence Consortium, the National Institute of Health and Care Excellence and the US Grading of Recommendations, Assessment, Development and Evaluations Network, convened a working group. All members have expertize and practical experience in the development of living guidelines. We collated methods, documents on prioritization from each organization's living guidelines, conducted interviews and held working group discussions. We consolidated these to form best practice principles which were then edited and agreed on by the working group members.

RESULTS

We developed best practice principles for (1) identification, (2) selection, and (3) prioritization, of questions for a living approach to guideline development. Several different strategies for undertaking prioritizing questions are explored.

CONCLUSION

The article provides guidance for prioritizing questions in living guidelines. Subsequent articles in this series explore consumer involvement, search decisions, and methods decisions that are appropriate for questions with different priority levels.

An International Needs Assessment Survey of Guideline Developers Demonstrates Variability in Resources and Challenges to Collaboration between Organizations

BACKGROUND

The development of rigorous, high-quality clinical guidelines increases the need for resources and skilled personnel within guideline-producing organizations. While collaboration between organizations provides a unique opportunity to pool resources and save time and effort, the collaboration presents its own unique challenges.

OBJECTIVE

To assess the perceived needs and current challenges of guideline producers worldwide related to guideline development and collaboration efforts.

DESIGN

Survey questions were developed by the Guidelines International Network and the US GRADE Network, pilot-tested among attendees of a guideline development workshop, and disseminated electronically using convenience and snowball sampling methods.

PARTICIPANTS

A total of 171 respondents representing 30 countries and more than 112 unique organizations were included in this analysis.

MAIN MEASURES

The survey included free-response, multiple-choice, and seven-point Likert-scale questions. Questions assessed respondents' perceived value of guidelines, resource availability and needs, guideline development processes, and collaboration efforts of their organization.

KEY RESULTS

Time required to develop high-quality systematic reviews and guidelines was the most relevant need (median=7; IQR=5.5-7). In-house resources to conduct literature searches (median=4; IQR=3-6) and the resources to develop rigorous guidelines rapidly (median=4; IQR=2-5) were perceived as the least available resources. Difficulties reconciling differences in guideline methodology (median=6; IQR=4-7) and the time required to establish collaborative agreements (median=6; IQR=5-6) were the most relevant barriers to collaboration between organizations. Results also indicated a general need for improvement in conflict of interest (COI) disclosure policies.

CONCLUSION

The survey identified organizational challenges in supporting rigorous guideline development, including the time, resources, and personnel required. Connecting guideline developers to existing databases of high-quality systematic reviews and the use of freely available online platforms may facilitate guideline development. Guideline-producing organizations may also consider allocating resources to hiring or training personnel with expertise in systematic review methodologies or utilizing resources more effectively by establishing collaborations with other organizations.

An adapted 'Ottawa' method allowed assessing the need to update topic areas within clinical practice guidelines

OBJECTIVE

To adapt and evaluate a method for assessing the need to update guideline topic areas involving multiple recommendations.

STUDY DESIGN AND SETTING

The 'Ottawa method' uses literature signals to determine changes in evidence that trigger a need to update individual guideline questions. We adapted the Ottawa method to include a process for aggregating updating signals by topic area (e.g., resuscitation) and tested this method using the German guideline on the treatment of patients with severe/multiple injuries. This involved a focused systematic evaluation of current evidence to identify updating signals, and classifying the need to update for each topic area. Then, we surveyed the guideline group online about the modified method.

RESULTS

We conducted focused literature searches for 37 topic areas and screened a mean of 97 abstracts per topic area in 2021. The need to update was high for eight (21.6%), intermediate for eight (21.6%), and low for 21 topic areas (56.8%) based on updating signals. The survey response rate was 56% (24/43). Most guideline group members (94%, 16/17 responders) would use the Ottawa method again, but their comments identified some weaknesses.

CONCLUSION

The modified Ottawa method is a suitable, efficient tool to generate evidence-based updating signals for guideline topic areas involving multiple recommendations. Further fine-tuning is recommended.

Methodological approaches for developing, reporting, and assessing evidence-based clinical practice guidelines: A systematic survey

OBJECTIVE

To produce a mapping and feature summary of approaches and tools available for the CPG community to develop, report, or assess four types of CPGs: (1) Standard original (or de novo) CPG, (2) Rapid original CPG, (3) Adapted/adopted CPGs, and (4) Updated CPGs.

STUDY DESIGN

The systematic literature search was conducted using Embase and PubMed, covering the period from January 2010 to October 13 2020. Two websites that collect and recommend approaches/tools to develop, report, or assess CPGs were also searched: Guidelines International Network and Equator Network. We screened the search results to include methodological papers that aimed to develop specific approaches/tools to develop, report, or assess any of the aforementioned four CPG types.

RESULTS

Among 10,581 citations, 46 papers reporting 46 approaches/tools were included. Of these 46 approaches/tools, 33 were about CPG development, seven were for CPG reporting, and six for CPG assessment. Among the 33 development approaches/tools, 26 did not state usability or validity information; but nine from 13 reporting or assessment approaches/tools did.

CONCLUSIONS

This study provides a mapping and feature summary of the current available approaches/tools, which serves to improve users' understanding to pave the way for informed choice and application.

Current practices and challenges in adaptation of clinical guidelines: a qualitative study based on semistructured interviews

OBJECTIVE

This study aims to better understand the current practice of clinical guideline adaptation and identify challenges raised in this process, given that published adapted clinical guidelines are generally of low quality, poorly reported and not based on published frameworks.

DESIGN

A qualitative study based on semistructured interviews. We conducted a framework analysis for the adaptation process, and thematic analysis for participants' views and experiences about adaptation process.

SETTING

Nine guideline development organisations from seven countries.

PARTICIPANTS

Guideline developers who have adapted clinical guidelines within the last 3 years. We identified potential participants through published adapted clinical guidelines, recommendations from experts, and a review of the Guideline International Network Conference attendees' list.

RESULTS

We conducted ten interviews and identified nine adaptation methodologies. The reasons for adapting clinical guidelines include developing de novo clinical guidelines, implementing source clinical guidelines, and harmonising and updating existing clinical guidelines. We identified the following core steps of the adaptation process (1) selection of scope and source guideline(s), (2) assessment of source materials (guidelines, recommendations and evidence level), (3) decision-making process and (4) external review and follow-up process. Challenges on the adaptation of clinical guidelines include limitations from source clinical guidelines (poor quality or reporting), limitations from adaptation settings (lacking resources or skills), adaptation process intensity and complexity, and implementation barriers. We also described how participants address the complexities and implementation issues of the adaptation process.

CONCLUSIONS

Adaptation processes have been increasingly used to develop clinical guidelines, with the emergence of different purposes. The identification of core steps and assessment levels could help guideline adaptation developers streamline their processes. More methodological research is needed to develop rigorous international standards for adapting clinical guidelines.

New methods facilitated the process of prioritizing questions and health outcomes in guideline development

BACKGROUND

Health guideline development requires sequential prioritization of the guideline topic, questions, and health outcomes. In this paper we report on new approaches for prioritizing questions and outcomes in guidelines.

METHODS

Ten guideline panels on venous thromboembolism rated potential guideline questions on a 9-point scale according to their overall importance and 6 criteria: common in practice, uncertainty in practice, variation in practice, new evidence available, cost consequences, not previously addressed. We randomized panelists to rate one potential question with and without the 6 criteria. Panelists rated importance of outcomes, defined with health outcome descriptors (HODs), using a 9-point scale, and health utility of outcomes on a visual analogue scale.

RESULTS

Of 469 potential questions identified, 72.5% were rated as important but not of high priority, and 25.4% as high priority. Each criterion was significantly associated with the overall importance rating. The overall importance rating means were 5.96 (SD 2.38) and 6.53 (SD 2.45) (P = 0.25) for those randomized to rate questions with and without the criteria, respectively. The mean importance rating for 121 outcomes was 6.01 (SD 1.25), with 35.5% rated as critical for decision-making. Panelists provided health utility ratings for 127 outcomes, with a minimum mean rating of 0.12 (SD 0.10) and maximum of 0.91 (SD 0.15).

CONCLUSION

Our structured process provided information to help explain perspectives of question importance, to facilitate panels' outcome prioritization, and to facilitate decision-making in guideline development.

Feasibility of national living guideline methods: the Australian Stroke Guidelines

OBJECTIVE

Maintaining clinical guideline currency has been one challenge to traditional guideline development. This paper describes the methods used to maintain a large national guideline for stroke management.

STUDY DESIGN AND SETTING

The Australian Stroke Clinical Guidelines are developed to meet Australian National Health and Medical Research Council (NHMRC) standards. Monthly surveillance is conducted for new systematic reviews and randomised controlled studies. Included studies undergo data extraction followed by preparation of updated evidence-to-decision frameworks which are used to inform updates, or development of new recommendations. Small writing groups made up of clinical experts and those with lived experience review and agree on changes, which are finally reviewed by a multidisciplinary Guidelines Steering Group. Draft changes are developed and published using the online MAGICapp platform, with dissemination and promotion via traditional methods as well as social media.

RESULTS

Each month approximately 350 abstracts are considered, covering 96 clinical topics and taking on average 16 hours to review. There have been four major guideline updates covering 34 new and updated recommendations.

CONCLUSION

It is feasible to use 'living' methods to maintain the Australian Clinical Guidelines for Stroke Management. Further work is now needed to understand the impact of living guidelines.

The UpPriority tool supported prioritization processes for updating clinical guideline questions

OBJECTIVE

We aim to 1) use the UpPriority tool to identify which clinical questions (CQs) within the clinical guidelines (CGs) need to be prioritized for updating and 2) assess the implementation of the tool in a real-world set of CGs.

STUDY DESIGN AND SETTING

We systematically assessed CQs from a sample of CGs developed in the Spanish National Health System CG program. We applied the UpPriority tool to each CG using a step-by-step process that included: 1) establishment of the UpPriority Implementation Working Group, 2) mapping of the original CG questions and recommendations, 3) development of a survey to prioritize CQs, 4) assessment of CQ's priority according to six items, 5) calculation and ranking of priority scores, 6) decision of prioritized CQs for updating, and 7) development of the priority report. We assessed the tool implementation process (appraisers' experience when using the tool) and the inter-observer reliability of the tool, and we provided suggestions for improvement.

RESULTS

We included four CGs with a total of 107 CQs on the following topics: chronic heart failure (10 CQs), inherited retinal dystrophies (39 CQs), menopause (20 CQs), and open-angle glaucoma (38 CQs). We included a total of 30 participants, most of them clinicians that were members of the original CG development groups. CQs were classified in three groups: 1) high priority (CQs prioritized for updating [16/107; 15.0%]), 2) medium priority (CQs that could be prioritized for updating [47/107; 43.9%]), and low priority (CQs that were not prioritized for updating [44/107; 41.1%]). The mean time each appraiser needed to assess the CQs with the tool was 3.8 hours (range 0.5 to 10). Agreement among the appraisers varied among the CGs. Appraisers considered that the tool was useful. We suggest some areas for consideration when using the tool including: 1) identification of key appraisers, 2) customization of training materials, 3) establishment of priority thresholds, and 4) provision of methodological support.

CONCLUSION

The UpPriority is a useful tool to identify which CQs within a CG need to be prioritized for update in a real-world scenario. Recruitment and training of topic experts are the main challenges when using the tool.