Stakeholders apply the GRADE evidence-to-decision framework to facilitate coverage decisions

Evidence-informed guidelines in oral health: insights from a systematic survey

BACKGROUND

Oral diseases are a major global public health problem, impacting the quality of life of those affected. While consensus exists on the importance of high-quality, evidence-informed guidelines to inform practice and public health decisions in medicine, appropriate methodologies and standards are not commonly adhered to among producers of oral health guidelines. This study aimed to systematically identify organizations that develop evidence-informed guidelines in oral health globally and survey the methodological process followed to formulate recommendations.

METHODS

We searched numerous electronic databases, guideline repositories, and websites of guideline developers, scientific societies, and international organizations (January 2012-October 2023) to identify organizations that develop guidelines addressing any oral health topic and that explicitly declare the inclusion of research evidence in their development. Pairs of reviewers independently evaluated potentially eligible organizations according to predefined selection criteria and extracted data about the organization's characteristics, key features of their guidelines, and the process followed when formulating formal recommendations. Descriptive statistics were used to analyze and summarize data.

RESULTS

We included 46 organizations that developed evidence-informed guidelines in oral health. The organizations were mainly professional associations and scientific societies (67%), followed by governmental organizations (28%). In total, organizations produced 55 different guideline document types, most of them containing recommendations for clinical practice (77%). Panels were primarily composed of healthcare professionals (87%), followed by research methodologists (40%), policymakers (24%), and patient partners (18%). Most (60%) of the guidelines reported their funding source, but only one out of three (33%) included a conflict of interest (COI) policy management. The methodology used in the 55 guideline document types varied across the organizations, but only 19 (35%) contained formal recommendations. Half (51%) of the guideline documents referred to a methodology handbook, 46% suggested a structured approach or system for rating the certainty of the evidence and the strength of recommendations, and 37% mentioned using a framework to move from evidence to decisions, with the GRADE-EtD being the most widely used (27%).

CONCLUSION

Our findings underscore the need for alignment and standardization of both terminology and methodologies used in oral health guidelines with current international standards to formulate trustworthy recommendations.

Using evidence to decision frameworks led to guidelines of better quality and more credible and transparent recommendations

BACKGROUND AND OBJECTIVES

To determine whether the use of Evidence to Decision (EtD) frameworks is associated to higher quality of both guidelines and individual recommendations.

METHODS

We identified guidelines recently published by international organizations that have methodological guidance documents for their development. Pairs of researchers independently extracted information on the use of these frameworks, appraised the quality of the guidelines using the Appraisal of Guidelines, Research and Evaluation II Instrument (AGREE-II), and assessed the clinical credibility and implementability of the recommendations with the Appraisal of Guidelines for REsearch & Evaluation Recommendations Excellence (AGREE-REX) tool. We conducted both descriptive and inferential analyses.

RESULTS

We included 66 guidelines from 17 different countries, published in the last 5 years. Thirty guidelines (45%) used an EtD framework to formulate their recommendations. Compared to those that did not use a framework, those using an EtD framework scored higher in all domains of both AGREE-II and AGREE-REX (P < 0.05). Quality scores did not differ between the use of the The Grading of Recommendations Assessment, Development and Evaluation-EtD framework (17 guidelines) or another EtD framework (13 guidelines) (P > 0.05).

CONCLUSION

The use of EtD frameworks is associated with guidelines of better quality, and more credible and transparent recommendations. Endorsement of EtD frameworks by guideline developing organizations will likely increase the quality of their guidelines.

Evidence to Decision frameworks enabled structured and explicit development of healthcare recommendations

OBJECTIVE

To identify and describe the processes suggested for the formulation of healthcare recommendations in health care guidelines available in guidance documents.

METHODS

We searched international databases in May 2020 to retrieve guidance documents published by organizations dedicated to guideline development. Pairs of researchers independently selected and extracted data about the characteristics of the guidance document, including explicit or implicit recommendation-related criteria and processes considered, as well as the use of EtD frameworks.

RESULTS

We included 68 guidance documents. Most organizations reported a system for grading the strength of recommendations (88%), half of them being the GRADE approach. Two out of three guidance documents (66%) proposed the use of a framework to guide the EtD process. The GRADE Evidence to Decision (GRADE-EtD) framework was the most often reported framework (19 organizations, 42%), whereas 20 organizations (44%) proposed their own multi-criteria frameworks. Using any EtD framework was related with a more comprehensive set of recommendation-related criteria compared to no framework, especially for criteria like values, equity, and acceptability.

CONCLUSION

Although limited, the use of EtD frameworks was associated the inclusion of relevant recommendation criteria. Among the EtD structured frameworks, the GRADE-EtD framework offers the most comprehensive perspective for evidence-informed decision-making processes.

Challenges in applying the GRADE approach in public health guidelines and systematic reviews: a concept article from the GRADE Public Health Group

BACKGROUND AND OBJECTIVE

This article explores the need for conceptual advances and practical guidance in the application of the GRADE approach within public health contexts.

METHODS

We convened an expert workshop and conducted a scoping review to identify challenges experienced by GRADE users in public health contexts. We developed this concept article through thematic analysis and an iterative process of consultation and discussion conducted with members electronically and at three GRADE Working Group meetings.

RESULTS

Five priority issues can pose challenges for public health guideline developers and systematic reviewers when applying GRADE: (1) incorporating the perspectives of diverse stakeholders; (2) selecting and prioritizing health and "nonhealth" outcomes; (3) interpreting outcomes and identifying a threshold for decision-making; (4) assessing certainty of evidence from diverse sources, including nonrandomized studies; and (5) addressing implications for decision makers, including concerns about conditional recommendations. We illustrate these challenges with examples from public health guidelines and systematic reviews, identifying gaps where conceptual advances may facilitate the consistent application or further development of the methodology and provide solutions.

CONCLUSION

The GRADE Public Health Group will respond to these challenges with solutions that are coherent with existing guidance and can be consistently implemented across public health decision-making contexts.

Using GRADE methodology to assess innovation of new medicinal products in Italy

AIM

In April 2017 the Italian Medicine Agency (AIFA) developed new criteria to grant any new medicinal product with an innovative designation. The aim of this study is to describe this new model and how it works.

METHODS

A retrospective descriptive analysis was performed on the results of the assessment process of innovativeness of new medicinal products (or therapeutic indications) based on the AIFA's new innovation criteria (therapeutic need, added therapeutic value and quality of clinical evidence through GRADE methodology) carried out between April 2017 and February 2019 and made publicly available on the AIFA website starting from January 2018.

RESULTS

A total of 37 full reports (22 for oncological indications) explaining the rationale for the AIFA's decision is made publicly available on the agency's website. A total of 12 therapeutic indications (5 oncological) were evaluated as fully innovative, 13 indications (11 oncological) were evaluated as conditionally innovative, while 12 indications (6 oncological) as non-innovative.

CONCLUSION

The new AIFA innovation criteria resulted in a much more flexible and transparent model to define and assess what constitutes a therapeutic innovation. In particular, the choice of AIFA to use the GRADE methodology to evaluate the quality of clinical evidence within a process of drug innovativeness assessment is essential for the early identification of the discrepancy between the need for patients of a rapid access to innovative therapies and the available clinical data needed to make decisions on drug innovativeness.

Evidence to Decision framework provides a structured "roadmap" for making GRADE guidelines recommendations

OBJECTIVES

It is unclear how guidelines panelists discuss and consider factors (criteria) that are formally and not formally included in the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach. To describe the use of decision criteria, we explored how panelists adhered to GRADE criteria and sought to identify any emerging non-GRADE criteria when the panelists used the Evidence to Decision (EtD) framework as part of GRADE application.

STUDY DESIGN AND SETTING

We used conventional and summative qualitative analyses to identify themes emerging from face-to-face, panel meeting discussions. Forty-eight members from 12 countries participated in the development of five guidelines for the management of venous thromboembolism by the American Society of Hematology.

RESULTS

Ten themes corresponded to the GRADE approach and represented all panel discussions. Over half (53%) of the total panel discussions concerned the use of research evidence. When evidence was considered sufficient and clear, the decision-making process proved rapid.

CONCLUSION

The GRADE EtD framework provides structure to guidelines panel meetings, and ensures that the panelists consider all established formal GRADE criteria as they decide on the recommendation text, strength, and direction (for or against an intervention). This is the first study assessing the use of GRADE's EtD framework during real-time guidelines development using panel discussions. Given the widespread use of GRADE, this study provides important information for practice recommendations generated when guidelines panels explicitly follow, in a transparent and systematic manner, the structured GRADE EtD framework. By recognizing the extent to which panels discuss and consider GRADE and other (non-GRADE) criteria for producing guideline recommendations, we are one step closer to understanding the decision-making process in panels that use a structured framework such as the GRADE EtD framework.

Developing Evidence to Decision Frameworks and an Interactive Evidence to Decision Tool for Making and Using Decisions and Recommendations in Health Care

Evidence-informed health care decisions and recommendations need to be made systematically and transparently. Mediating technology can help manage boundaries between groups making decisions and target audiences, enhancing salience, credibility, and legitimacy for all. This article describes the development of the Evidence to Decision (EtD) framework and an interactive tool to create and use frameworks (iEtD) to support communication in decision making. Methods: Using a human-centered design approach, we created prototypes employing a broad range of methods to iteratively develop EtD framework content and iEtD tool functionality. Results: We developed tailored EtD frameworks for making evidence-informed decisions and recommendations about clinical practice interventions, diagnostic and screening tests, coverage, and health system and public health options. The iEtD tool provides functionality for preparing frameworks, using them in group discussions, and publishing output for implementation or adaption. EtD and iEtD are intuitive and useful for producers and users of frameworks, and flexible for use across different types of topics, decisions, and organizations. They bring valued structure to panel discussions and transparency to published output. Conclusion: EtD and iEtD can resolve some of the challenges inherent in multicriteria, multistakeholder decision systems. They are freely available online for all to use at https://ietd.epistemonikos.org/ and https://gradepro.org.

Antipsychotic drug exposure and risk of fracture: a systematic review and meta-analysis of observational studies

To investigate the extent to which exposure to first-generation and second-generation antipsychotics (APs) is associated with an increased risk of fractures, with a particular focus on hip fractures, and to ascertain the risk associated with exposure to individual drugs. We included observational studies that reported data on fractures in individuals exposed to APs compared with unexposed individuals or individuals with previous exposure. We extracted information on study design, source of data, population characteristics, outcomes of interest, matching and confounding factors, and used a modified version of the Newcastle-Ottawa Scale to judge study risk of bias. We pooled adjusted estimates of relative effects to generate pooled odds ratios (ORs) and their 95% confidence interval (CI) using a random-effects model. We rated the quality of evidence using the GRADE approach. Of 36 observational studies, 29 proved to have a low risk of bias and seven were found to have a high risk of bias. The risk of hip fracture (OR: 1.57, 95% CI: 1.42-1.74, low quality of evidence) and of any fracture (OR: 1.17, 95% CI: 1.04-1.31, very low quality of evidence) increased with exposure to APs, with similar increases in risk in the first generation and second generation. The risk was similar among different diagnostic categories. The few studies that provided data were insufficient to allow inferences on individual drugs. AP exposure in unselected populations was associated with a 57% increase in the risk of hip fractures and a 17% increase in the risk of any fractures. Between-study heterogeneity limits the confidence in this estimate.

Patient participation in ERS guidelines and research projects: the EMBARC experience

The European Multicentre Bronchiectasis Audit and Research Collaboration (EMBARC) is a European Respiratory Society (ERS) Clinical Research Collaboration dedicated to improving research and clinical care for people with bronchiectasis. EMBARC has created a European Bronchiectasis Registry, funded by the ERS and by the European Union (EU) Innovative Medicines Initiative Programme. From the outset, EMBARC had the ambition to be a patient-focussed project. In contrast to many respiratory diseases, however, there are no specific patient charities or European patient organisations for patients with bronchiectasis and no existing infrastructure for patient engagement. This article describes the experience of EMBARC and the European Lung Foundation in establishing a patient advisory group and then engaging this group in European guidelines, an international registry and a series of research studies. Patient involvement in research, clinical guidelines and educational activities is increasingly advocated and increasingly important. Genuine patient engagement can achieve a number of goals that are critical to the success of an EU project, including focussing activities on patient priorities, allowing patients to direct the clinical and research agenda, and dissemination of guidelines and research findings to patients and the general public. Here, we review lessons learned and provide guidance for future ERS task forces, EU-funded projects or clinical research collaborations that are considering patient involvement.

EDUCATIONAL AIMS

To understand the different ways in which patients can contribute to clinical guidelines, research projects and educational activities.To understand the barriers and potential solutions to these barriers from a physician's perspective, in order to ensure meaningful patient involvement in clinical projects.To understand the barriers and potential solutions from a patient's perspective, in order to meaningfully involve patients in clinical projects.

WITHDRAWN: Value of Prostate Cancer Care: New Information on New Therapies Suggest Less is More

The Publisher regrets that this article is an accidental duplication of an article that has already been published, http://dx.doi.org/10.1016/j.eururo.2017.05.023. The duplicate article has therefore been withdrawn. The full Elsevier Policy on Article Withdrawal can be found at https://www.elsevier.com/about/our-business/policies/article-withdrawal.