Development and Validation of a Tool to Assess the Quality of Clinical Practice Guideline Recommendations

Quality and credibility of clinical practice guidelines recommendations for the management of neonatal hypoglycemia. A protocol for a systematic review and recommendations' synthesis

INTRODUCTION

Hypoglycemia is one of the most frequent metabolic conditions in neonates. Clinical practice guidelines (CPGs) influence clinical practice as high-quality CPGs facilitate the use of evidence in practice. This proposed study aims to systematically identify and appraise CPGs and CPG recommendations (CPGRs) for treating neonatal hypoglycemia (NH).

METHODS AND ANALYSIS

We will conduct searches in MEDLINE, EMBASE, CINAHL, Cochrane Library, LILACS (Latin American & Caribbean Health Sciences Literature), and Epistemonikos. Authors will search CPGs-specific databases and grey literature. Two reviewers will independently perform the titles and abstract screening, full-text review, and data extraction. Two appraisers will assess the quality of the CPGs and their recommendations using AGREE II (Appraisal of Guidelines Research and Evaluation) and AGREE-REX (Appraisal of Guidelines Research and Evaluation-Recommendations Excellence) instruments. Scores of ≥ 60% in the rigour of development domain will be considered for defining high-quality with AGREE II tool. CPGRs with scores >60% in the three domains will be used to determine high quality with the AGREE REX tool. We will perform a synthesis of the CPGRs to identify the consistency among the CPGRs and the methodological quality of primary studies that support them.

ETHICS AND DISSEMINATION

The results will help us to identify the methodological and quality gaps in the existing CPGs for the treatment of NH. Our findings will be submitted to peer-review journals and presented at academic conferences. Based on the study design, approval from the institutional ethics board is not required for this project.

TRIAL REGISTRATIONS

Systematic Review Registration Number (PROSPERO): CRD 42021239921.

Quality appraisal of clinical nutrition practice guidelines for critically ill adult patients: a systematic review using the Advancing Guideline Development, Reporting and Evaluation in Health Care instrument II (AGREE-II) and AGREE-Recommendation Excellence (AGREE-REX)

Nutritional therapy should follow evidence-based practice, thus several societies regarding nutrition and critical care have developed specific Clinical Practice Guidelines (CPG). However, to be regarded as trustworthy, the quality of the CPG for critically ill patients and its recommendations need to be high. This systematic review aimed to appraise the methodology and recommendations of nutrition CPG for critically ill patients. We performed a systematic review (protocol number CRD42020184199) with literature search conducted on PubMed, Embase, Cochrane Library and other four specific databases of guidelines up to October 2021. Two reviewers, independently, assessed titles and abstracts and potentially eligible full-text reports to determine eligibility and subsequently four reviewers appraised the guidelines quality using the Advancing Guideline Development, Reporting and Evaluation in Health Care instrument II (AGREE-II) and AGREE-Recommendation Excellence (AGREE-REX). Ten CPG for nutrition in critically ill patients were identified. Only Academy of Nutrition and Dietetics and European Society of Intensive Care Medicine had a total acceptable quality and were recommended for daily practice according AGREE-II. None of the CPG recommendations had an overall quality score above 70 %, thus being classified as moderate quality according AGREE-REX. The methodological evaluation of the critically ill adult patient CPG revealed significant discrepancies and showed a need for improvement in its development and/or reporting. In addition, recommendations about nutrition care process presented a moderate quality.

What information can we gain from the quality appraisal of guidelines with physical activity recommendations for cancer patients?A systematic review using the AGREE II and AGREE-REX tools

PURPOSE

There has been growing amount of evidence supporting the benefits of physical activity (PA) on oncological patients' cancer-related health outcomes. Although guidelines on cancer rehabilitation are widely available, the varying quality and practical applicability limited the clinical application of PA recommendations. To assist the future development of guidelines, in this systematic review, we evaluated the quality and applicability of current cancer rehabilitation guidelines with PA recommendations and synthesized PA recommendations for the oncological population.

METHODS

A systematic search was conducted in PubMed, CINAHL, PEDro, EMBASE, and guideline repositories to identify guidelines with PA recommendations for cancer patients from 1 May 2016 to 1 June 2022. The quality of included guidelines was appraised using the tools "Appraisal of Guidelines for Research and Evaluation II" (AGREE II) and AGREE-REX (Recommendation Excellence). PA recommendations were synthesized from the guidelines.

RESULTS

Sixteen guidelines were extracted. The AGREE II domain "clarity of presentation" obtained the highest score, while "applicability" received the lowest, ranging from 33.33% to 98.58%. The AGREE-REX domains "values and preferences" and "implementability" generally scored lower and ranged from 45.83% to 74.17% and 55% to 88.33%, respectively. Eight high-quality guidelines were identified, and the included PA recommendations were extracted.

CONCLUSION

There were some disparities in the quality of the included guidelines. Methodological weaknesses were commonly observed in domains "applicability," "values and preferences," and "implementability"; particular attention should be given to these domains when developing future guidelines. Furthermore, this analysis indicated that more rigorous, high-quality studies are needed to generate evidence for supporting PA recommendations and provide guidance on research gaps in the field of cancer rehabilitation.

Capacity Building During Short-Term Surgical Outreach Trips: A Review of What Guidelines Exist

INTRODUCTION

While short-term surgical outreach trips improve access to care in low- and middle-income countries (LMIC), there is rising concern about their long-term impact. In response, many organizations seek to incorporate capacity building programs into their outreach efforts to help strengthen local health systems. Although leading organizations, like the World Health Organization (WHO), advocate for this approach, uniform guidelines are absent.

METHODS

We performed a systematic review, using search terms pertaining to capacity building guidelines during short-term surgical outreach trips. We extracted information on authorship, guideline development methodology, and guidelines relating to capacity building. Guidelines were classified according to the Global-QUEST framework, which outlines seven domains of capacity building on surgical outreach trips. Guideline development methodology frequencies and domain classifications frequencies were calculated; subsequently, guidelines were aggregated to develop a core guideline for each domain.

RESULTS

A total of 35 studies were included. Over 200 individual guidelines were extracted, spanning all seven framework domains. Guidelines were most frequently classified into Coordination and Community Impact domains and least frequently into the Finance domain. Less than half (46%) of studies collaborated with local communities to design the guidelines. Instead, guidelines were predominantly developed through author trip experience.

CONCLUSION

As short-term surgical trips increase, further work is needed to standardize guidelines, create actionable steps, and promote collaborations in order to promote accountability during short-term surgical outreach trips.

A critical appraisal of delirium clinical practice guidelines relevant to the care of older adults in the emergency department with a synthesis of recommendations: an umbrella review protocol

BACKGROUND

Up to 35% of older adults present to the emergency department (ED) with delirium or develop the condition during their ED stay. Delirium associated with an ED visit is independently linked to poorer outcomes such as loss of independence, increased length of hospital stay, and mortality. Improving the quality of delirium care for older ED patients is hindered by a lack of knowledge and standards to guide best practice. High-quality clinical practice guidelines (CPGs) have the power to translate the complexity of scientific evidence into recommendations to improve and standardize practice. This study will identify and synthesize recommendations from high-quality delirium CPGs relevant to the care of older ED patients.

METHODS

We will conduct a multi-phase umbrella review to retrieve relevant CPGs. Quality of the CPGs and their recommendations will be critically appraised using the Appraisal of Guidelines, Research, and Evaluation (AGREE)-II; and Appraisal of Guidelines Research and Evaluation - Recommendations Excellence (AGREE-REX) instruments, respectively. We will also synthesize and conduct a narrative analysis of high-quality CPG recommendations.

DISCUSSION

This review will be the first known evidence synthesis of delirium CPGs including a critical appraisal and synthesis of recommendations. Recommendations will be categorized according to target population and setting as a means to define the bredth of knowledge in this area. Future research will use consensus building methods to identify which are most relevant to older ED patients.

TRIAL REGISTRATION

This study has been registered in the Open Science Framework registries: https://doi.org/10.17605/OSF.IO/TG7S6 .

Use of the AGREE II instrument to evaluate critical care practice guidelines addressing pharmacotherapy

RATIONALE, AIMS AND OBJECTIVES

Clinical practice guidelines (CPGs) have been evaluated for reporting transparency and methodological quality in a number of studies in various disciplines, but few studies have focused on critical care and none on pharmacotherapy-related guidelines specifically. The objective of this study was to evaluate the quality of critical care CPGs with a focus on pharmacotherapy using the Appraisal of Guidelines, Research and Evaluation (AGREE) II instrument.

METHOD

A cross-sectional study of CPGs published from 2013 through August 2021 was conducted. Following establishment of interrater reliability, guidelines were independently evaluated by three reviewers to rate guidelines on criteria set forth by the AGREE II instrument. Domain scores and item scores were calculated using the AGREE II user manual, and results described with descriptive statistics.

RESULTS

Out of 192 guidelines identified, 73 met inclusion criteria and were screened using the AGREE II instrument. Most guidelines were authored by a professional organization or government agency. Domain quality scores were calculated for each domain as recommended by the AGREE II instrument. Domain 4 (clarity of presentation) had the highest AGREE II domain score with a median score of 87.0% (interquartile range: 79.6%-92.6%). Domain 5 (applicability) received the lowest domain score with a mean score of 41.8 ± 21.1%. The majority of guidelines were recommended for use as published or with modifications, while only six guidelines (8.2%) were not recommended for use.

CONCLUSIONS

The majority of critical care guidelines that include pharmacotherapy recommendations were recommended for use by study authors when the AGREE II instrument was applied. While guidelines generally scored highly in clarity of presentation, additional time and effort should focus on providing solutions to guideline implementation and inclusion of patient preferences.

Evidence-basing for quality improvement; bringing clinical practice guidelines closer to their promise of improving care practices

BACKGROUND

Clinical practice guidelines (CPGs) have become central to efforts to change clinical practice and improve the quality of health care. Despite growing attention for rigorous development methodologies, it remains unclear what contribution CPGs make to quality improvement.

AIM

This mixed methods study examines guideline quality in relation to the availability of certain types of evidence and reflects on the implications of CPGs' promise to improve the quality of care practices.

METHODS

The quality of 62 CPGs was assessed with the Appraisal of Guidelines, Research, and Evaluation (AGREE) instrument. Findings were discussed in 19 follow-up interviews to examine how different quality aspects were considered during development.

RESULTS

The AGREE assessment showed that while some quality criteria were met, CPGs have limited coverage of domains such as stakeholder involvement and applicability, which generally lack a 'strong' evidence base (e.g., randomized controlled trials [RCT]). Qualitative findings uncovered barriers that impede the consolidation of evidence-based guideline development and quality improvement including guideline scoping based on the patient-intervention-comparison-outcome (PICO) question format and a lack of clinical experts involved in evidence appraisal. Developers used workarounds to include quality considerations that lack a strong base of RCT evidence, which often ended up in separate documents or appendices.

CONCLUSION

Findings suggest that CPGs mostly fail to integrate different epistemologies needed to inform the quality improvement of clinical practice. To bring CPGs closer to their promise, guideline scoping should maintain a focus on the most pertinent quality issues that point developers toward the most fitting knowledge for the question at hand, stretching beyond the PICO format. To address questions that lack a strong evidence base, developers actually need to appeal to other sources of knowledge, such as quality improvement, expert opinion, and best practices. Further research is needed to develop methods for the robust inclusion of other types of knowledge.

Quality evaluation of clinical practice guidelines for thromboprophylaxis in orthopaedic trauma based on AGREE II and AGREE-REX: a systematic review protocol

INTRODUCTION

Orthopaedic trauma patients are at high risk of venous thromboembolism (VTE). As VTE prophylaxis has gradually raised public concerns, guidelines related to this topic have increased over time. However, the existing recommendations of thromboprophylaxis guidelines in orthopaedic trauma patients are still inconsistent, and the quality of the guidelines and recommendations for the topic still lacks comprehensive assessments. This review aims to critically appraise clinical practice guidelines for thromboprophylaxis in orthopaedic trauma patients.

METHODS AND ANALYSIS

We will conduct a comprehensive literature search up to 31 October 2022 in databases (PubMed, EMBASE, CINAHL, Web of Science, the Cochrane Library, etc), academic websites and guideline repositories. The quality of the guidelines and recommendations will be assessed by five reviewers independently using the Appraisal of Guidelines Research and Evaluation II instrument (AGREE-II) and the AGREE - Recommendation EXcellence (AGREE-REX). We will summarise the characteristics of the guidelines and compare the differences between these recommendations.

ETHICS AND DISSEMINATION

This study will follow the Declaration of Helsinki and has received approval from the Ethics Committee on Biomedical Research, West China Hospital, Sichuan University (ethics approval no. 2021-989). The results will be summarised as a paper, disseminated through peer-reviewed journals, and will help guide further research in the future.

PROTOCOL REGISTRATION NUMBER

CRD42021273405.

Systematic Review of Clinical Practice Guidelines for Insomnia Disorder

OBJECTIVE

This systematic review assessed the quality of clinical practice guidelines (CPGs) on the treatment of insomnia disorder and their reporting of recommendations, while summarizing the evidence and providing guidance on an algorithmic approach to appropriate pharmacological treatment.

METHODS

The PubMed and EMBASE databases, guideline repositories, and specialist association websites were searched. The quality of the CPGs was assessed using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument, complemented by the AGREE-REX (Appraisal of Guidelines REsearch and Evaluation-Recommendations EXcellence). A multidisciplinary team identified the key clinical questions that a clinician would consider when taking an algorithmic approach to the use of medication for patients with insomnia disorder. By using a meta-synthesis approach, recommendations from the CPGs were characterized and summarized via a recommendation matrix.

RESULTS

A total of 10 records that met the inclusion criteria were included and appraised. Four CPGs were rated as high and 3 CPGs were rated as moderate in overall quality. Most of the CPGs recommended pharmacotherapy only if cognitive behavioral therapy for insomnia or other nonpharmacological interventions were unavailable, unsuccessful, or declined by patients. Recommendations on types of medicines and dose and duration of treatment varied and were nonspecific. Few of the CPGs provided recommendations on pharmacotherapy in special populations.

CONCLUSIONS

Indications for starting medications are the only common thread in all of the reviewed CPGs. The CPGs diverged in the choice of first-line pharmacotherapy, and most of the CPGs did not provide recommendations on all subsequent clinical considerations.

Quality and consistency of clinical practice guidelines on the prevention of food allergy and atopic dermatitis: Systematic review protocol

Background and aims

Allergy prevention strategies have gained significant traction as a means to attenuate the growing burden of allergic diseases over the past decade. As the evidence base for primary prevention of food allergy (FA) and atopic dermatitis (AD) is constantly advancing, clinical practice guideline (CPG) recommendations on interventions for FA and AD prevention vary in quality and consistency among professional organizations. We present a protocol for a systematic review of CPGs on primary prevention of FA and AD.

Methods

We will systematically review and appraise all CPGs addressing primary prevention of FA and AD and report our findings according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Electronic databases and manual website searches from January 2011 to March 2021 without language or geographical restrictions, and supplemented by author contact, will generate the list of potentially relevant CPGs to screen. Evaluation of the methodological quality, consistency, and global applicability of shortlisted CPGs will be performed by members of the Allergy Prevention Work Group of the World Allergy Organization (WAO) using the Appraisal of Guidelines for Research and Evaluation (AGREE) II and AGREE-REX (Recommendations EXcellence). instruments. Guideline contents, consistency, and quality of the recommendations will be summarised in tabular and narrative formats. We aim to present consolidated recommendations from international guidelines of the highest methodological quality and applicability, as determined by AGREE II and AGREE-REX.

Dissemination

This systematic review will provide a succinct overview of the quality and consistency of recommendations across all existing CPGs for FA and AD prevention, as well as crucial perspectives on applicability of individual recommendations in different geographical contexts. Results from this systematic review will be reported in a peer-reviewed journal. It will also inform a position statement by WAO to provide a practical framework to guide the development of future guidelines for allergy prevention worldwide.

Prospero registration number

CRD42021265689.

The Quality and Clinical Applicability of Recommendations in Ostomy Guidelines: A Systematic Review

Aim

To systematically search ostomy clinical practice guidelines, critically assess their quality and clinical applicability of recommendations, and summarize the recommendations.

Design

Systematic review.

Data Sources

The PubMed, ProQuest and CINAHL databases, eight guideline databases, and three ostomy institution websites were searched on September 3, 2021.

Review Methods

Appraisal of Guidelines for Research and Evaluation II (AGREE II) and AGREE Recommendation EXcellence (AGREE-REX) were used to assess the guideline.

Results

The initial search identified 1475 documents. Of these, 27 full-text documents were reviewed. Finally, 10 guidelines were included. Among these, the 2019 Registered Nurses’ Association of Ontario (RNAO) guidelines had the highest total scores using AGREE II and AGREE-REX. The 2019 National Institute for Health and Care Excellence (NICE) and 2018 European Hernia Society (EHS) were also ranked as high-quality and evaluated as “recommended.” The median of the “applicability” domain was the lowest (45%) among the six AGREE II domains. The median of the “values and preferences” domain was the lowest (38%) among the three AGREE-REX domains. In total, 172 recommendations were summarized and parastomal hernia received the most attention among the recommendations.

Conclusion

The quality of the 10 clinical practice guidelines varied widely. The three identified high-quality guidelines might be appropriate first choices in daily ostomy care and management practice and can be tailored to the local context. Ostomy guidelines require further improvement in the “applicability” and “values and preferences” domains.

No Patient or Public Contribution

This review only searched and evaluated relevant documents, so such details do not apply to this review.

The Guideline Language and Format Instrument (GLAFI): development process and international needs assessment survey

Background

Successful guideline implementation depends both on factors extrinsic to guidelines and their intrinsic features. In the Guideline Implementability for Decision Excellence Model (GUIDE-M), “communicating” content (language and format) is one of three core determinants of intrinsic implementability, but is seldom addressed. Our aims were to develop a tool that could be used by guideline developers to optimize language and format during development; identify gaps in this type of guidance in existing resources; and evaluate the perceived need for and usefulness of such a tool among guideline developers.

Methods

Our mixed-methods design consisted of (1) content development (selection and organization of evidence-based constructs from the GUIDE-M into a prototype Guideline Language and Format Instrument (GLAFI), followed by face validation with guideline developers); (2) document analysis (duplicate) of seven existing guideline tools to measure coverage of GLAFI items and identify new items; and (3) an international survey of guideline developers (corresponding authors of recent Canadian Medical Association or Guidelines International Network database guidelines) to measure perceived importance of language and format, quality of existing resources, and usefulness of a language and format tool.

Results

GLAFI items were organized into 4 language and 4 format subdomains. In face validation with guideline developers (17 clinicians, 1 methodologist), all agreed that the tool would improve guideline implementability and 93% indicated a desire for regular use. In the existing guideline tool document analysis, only 14/44 (31.8%) GLAFI items were operationalized in at least one tool. We received survey responses from 148/674 (22.0%) contacted guideline authors representing 45 organizations (9 countries). Language was rated as “extremely important” or “important” in determining uptake by 94% of respondents, and format by 84%. Correspondingly, 72% and 70% indicated that their organization would likely use such a tool.

Conclusions

Optimal language and format are fundamental to guideline implementability but often overlooked. The GLAFI tool operationalizes evidence-based constructs, most of which are absent in existing guideline tools. Guideline developers perceive these concepts to be important and express a willingness to use such a tool. The GLAFI should be further tested and refined with guideline developers and its impact on end-users measured.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01219-2.

AGREEing on Clinical Practice Guidelines for Autism Spectrum Disorders in Children: A Systematic Review and Quality Assessment

Background: Autism spectrum disorder (ASD) is a multifaceted neurodevelopmental disorder requiring multimodal intervention and an army of multidisciplinary teams for a proper rehabilitation plan. Accordingly, multiple practice guidelines have been published for different disciplines. However, systematic evidence to detect and intervene must be updated regularly. Our main objective is to compare and summarize the recommendations made in the clinical practice guidelines (CPGs) for ASD in children released from November 2015 to March 2022. Methods: CPGs were subjected to a systematic review. We developed the inclusion and exclusion criteria and health-related questions, then searched and screened for CPGs utilizing bibliographic and CPG databases. Each of the CPGs used in the study were critically evaluated using the Appraisal of Guidelines for REsearch and Evaluation II (AGREE II) instrument. In a realistic comparison table, we summarized the recommendations. Results: Four eligible CPGs were appraised: Australian Autism CRC (ACRC); Ministry of Health New Zealand (NZ); National Institute for Health and Care Excellence (NICE); and Scottish Intercollegiate Guidelines Network, Healthcare Improvement Scotland (SIGN-HIS). The overall assessments of all four CPGs scored greater than 80%; these findings were consistent with the high scores in the six domains of AGREE II, including: (1) scope and purpose, (2) stakeholder involvement, (3) rigor of development, (4) clarity of presentation, (5) applicability, and (6) editorial independence domains. Domain (3) scored 84%, 93%, 86%, and 85%; domain (5) 92%, 89%, 54%, and 85%; and domain (6) 92%, 96%, 88%, and 92% for ACRC, NICE, NZ, and SIGN-HIS, respectively. Overall, there were no serious conflicts between the clinical recommendations of the four CPGs, but some were more comprehensive and elaborative than others. Conclusions: All four assessed evidence-based CPGs demonstrated high methodological quality and relevance for use in practice.

Policy Analysis for Prevention and Control of Influenza in Aged Care

OBJECTIVE

This study aimed to analyze national influenza infection control policy documents within aged care settings by identifying the consistencies, inconsistencies, and gaps with the current evidence and by evaluating methodological quality. Aged care providers can use these findings to identify their policy documents' strengths and weaknesses.

DESIGN

A quality and content analysis of national level policy documents.

SETTING AND PARTICIPANTS

Aged care settings rely on national agencies' policy recommendations to control and prevent outbreaks. There is limited research on the effectiveness of control measures to prevent and treat influenza within aged care settings. Because of the complexities around aged care governance, the primary responsibility in developing a comprehensive facility-level, infection-prevention policy, falls to the providers.

METHODS

The analysis was conducted using the (1) International Appraisal of Guidelines, Research and Evaluation assessment tool, containing 23 items across 6 domains; and the (2) Influenza Related Control Measures in Aged Care settings checklist, developed by the authors, with 82 recommendations covering: medical interventions, nonmedical interventions, and physical layout.

RESULTS

There were 19 documents from 9 different high-income countries, with a moderately high methodological quality in general. The quality assessment's average score was 40.2% (95% CI 31.9%-44.7%). "Stakeholder involvement" ranked third, and "Editorial independence" and "Rigor of development" had the lowest average scores across all domains. The content analysis' average score was 37.2% (95% CI 10.5%-21.5%). The highest scoring document (59.1%) included term definitions, cited evidence for recommendations, and clear measurable instructions. "Physical Layout" had the least coverage and averaged 21.9% (95% CI 4.2%-37.5%), which shows a substantial gap in built environment recommendations.

CONCLUSIONS AND IMPLICATIONS

Existing policy documents vary in their comprehensiveness. The higher scoring documents provide an ideal model for providers. The checklist tools can be used to assess and enhance documents. Further research on document end-user evaluation would be useful, as there is room for improvement in methodological quality and coverage of recommendation coverage, especially related to physical layout.

Methodological Rigor and Transparency in Clinical Practice Guidelines for Nutrition Care in Critically Ill Adults: A Systematic Review Using the AGREE II and AGREE-REX Tools

Background: To evaluate the methodological quality of (1) clinical practice guidelines (CPGs) that inform nutrition care in critically ill adults using the AGREE II tool and (2) CPG recommendations for determining energy expenditure using the AGREE-REX tool. Methods: CPGs by a professional society or academic group, intended to guide nutrition care in critically ill adults, that used a systematic literature search and rated the evidence were included. Four databases and grey literature were searched from January 2011 to 19 January 2022. Five investigators assessed the methodological quality of CPGs and recommendations specific to energy expenditure determination. Scaled domain scores were calculated for AGREE II and a scaled total score for AGREE-REX. Data are presented as medians (interquartile range). Results: Eleven CPGs were included. Highest scoring domains for AGREE II were clarity of presentation (82% [76–87%]) and scope and purpose (78% [66–83%]). Lowest scoring domains were applicability (37% [32–42%]) and stakeholder involvement (46% [33–51%]). Eight (73%) CPGs provided recommendations relating to energy expenditure determination; scores were low overall (37% [36–40%]) and across individual domains. Conclusions: Nutrition CPGs for critically ill patients are developed using systematic methods but lack engagement with key stakeholders and guidance to support application. The quality of energy expenditure determination recommendations is low.

Editor's Choice - European Society for Vascular Surgery Clinical Practice Guideline Development Scheme: An Overview of Evidence Quality Assessment Methods, Evidence to Decision Frameworks, and Reporting Standards in Guideline Development

OBJECTIVE

A structured and transparent approach is instrumental in translating research evidence to health recommendations and evidence informed clinical decisions. The aim was to conduct an overview and analysis of principles and methodologies for health guideline development.

METHODS

A literature review on methodologies, strategies, and fundamental steps in the process of guideline development was performed. The clinical practice guideline development process and methodology adopted by the European Society for Vascular Surgery are also presented.

RESULTS

Sophisticated methodologies for health guideline development are being applied increasingly by national and international organisations. Their overarching principle is a systematic, structured, transparent, and iterative process that is aimed at making well informed healthcare choices. Critical steps in guideline development include the assessment of the certainty of the body of evidence; evidence to decision frameworks; and guideline reporting. The goal of strength of evidence assessments is to provide well reasoned judgements about the guideline developers' confidence in study findings, and several evidence hierarchy schemes and evidence rating systems have been described for this purpose. Evidence to decision frameworks help guideline developers and users conceptualise and interpret the construct of the quality of the body of evidence. The most widely used evidence to decision frameworks are those developed by the GRADE Working Group and the WHO-INTEGRATE, and are structured into three distinct components: background; assessment; and conclusions. Health guideline reporting tools are employed to ensure methodological rigour and transparency in guideline development. Such reporting instruments include the AGREE II and RIGHT, with the former being used for guideline development and appraisal, as well as reporting.

CONCLUSION

This guide will help guideline developers/expert panels enhance their methodology, and patients/clinicians/policymakers interpret guideline recommendations and put them in context. This document may be a useful methodological summary for health guideline development by other societies and organisations.

AGREE-S: AGREE II extension for surgical interventions - United European Gastroenterology and European Association for Endoscopic Surgery methodological guide

BACKGROUND

The Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument has been developed to inform the methodology, reporting and appraisal of clinical practice guidelines. Evidence suggests that the quality of surgical guidelines can be improved, and the structure and content of AGREE II can be modified to help enhance the quality of guidelines of surgical interventions.

OBJECTIVE

To develop an extension of AGREE II specifically designed for guidelines of surgical interventions.

METHODS

In the tripartite Guideline Assessment Project (GAP) funded by United European Gastroenterology and the European Association for Endoscopic Surgery, (i) we assessed the quality of surgical guidelines and we identified factors associated with higher quality (GAP I); (ii) we applied correlation analysis, factor analysis and the item response theory to inform an adaption of AGREE II for the purposes of surgical guidelines (GAP II); and (iii) we developed an AGREE II extension for surgical interventions, informed by the results of GAP I, GAP II, and a Delphi process of stakeholders, including representation from interventional and surgical disciplines; the Guideline International Network (GIN); the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group; the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) initiative; and representation of surgical journal editors and patient/public.

RESULTS

We developed AGREE-S, an AGREE II extension for surgical interventions, which comprises 24 items organized in 6 domains; Scope and purpose, Stakeholders, Evidence synthesis, Development of recommendations, Editorial independence, and Implementation and update. The panel of stakeholders proposed 3 additional items: development of a guideline protocol, consideration of practice variability and surgical/interventional expertise in different settings, and specification of infrastructures required to implement the recommendations. Three of the existing items were amended, 7 items were rearranged among the domains, and one item was removed. The domain Rigour of Development was divided into domains on Evidence Synthesis and Development of Recommendations. The new domain Development of Recommendations incorporates items from the original AGREE II domain Clarity of Presentation.

CONCLUSION

AGREE-S is an evidence-based and stakeholder-informed extension of the AGREE II instrument, that can be used as a guide for the development and adaption of guidelines on surgical interventions.

Reporting and methodological quality of meta-analyses of acupuncture for patients with migraine: A methodological investigation with evidence map

BACKGROUND

Acupuncture has been widely used to relieve migraine-related symptoms. However, the findings of previous systematic reviews (SRs) and meta-analyses (MAs) are still not completely consistent. Their quality is also unknown, so a comprehensive study is needed.

OBJECTIVE

To evaluate the reporting and methodological quality of these MAs concerning acupuncture for migraine, and summarize evidence about the efficacy and safety of acupuncture for migraine.

SEARCH STRATEGY

PubMed, Embase, Cochrane Library, China National Knowledge Infrastructure, Chinese Biomedical Databases, Wanfang Data, and VIP databases were searched from inception to September 2020, with a comprehensive search strategy.

INCLUSION CRITERIA

The pairwise MAs of randomized controlled trials (RCTs) concerning migraine treated by acupuncture or acupuncture-based therapies, with a control group that received sham acupuncture, medication, no treatment, or acupuncture at different acupoints were included.

DATA EXTRACTION AND ANALYSIS

Two independent investigators screened studies, extracted relevant data, and assessed reporting and methodological quality using Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2009 and A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR 2), then all results were cross-checked. Spearman correlation test was used to evaluate the correlation between reporting and methodological quality scores.

RESULTS

A total of 20 MAs were included in this study. The included MAs indicated that acupuncture was efficacious and safe in preventing and treating migraine when compared with control intervention. There was a high correlation between reporting and methodological quality scores (r_s = 0.87, P < 0.001). The quality of the included SRs needs to be improved mainly with regard to protocol and prospective registration, using a comprehensive search strategy, summarizing the strength of evidence body for key outcomes, a full list of excluded studies with reasons for exclusion, reporting of RCTs' funding sources, and assessing the potential impact of risk of bias in RCTs on MA results.

CONCLUSION

Acupuncture is an effective and safe intervention for preventing and treating migraine, and could be considered as a good option for patients with migraine. However, the reporting and methodological quality of MAs included in this overview is suboptimal. In the future, AMSTAR 2 and PRISMA tools should be followed when making and reporting an SR with MA.

Clinical practice guideline recommendations for pediatric injury care: protocol for a systematic review

INTRODUCTION

Evidence suggests the presence of deficiencies in the quality of care provided to up to half of all paediatric trauma patients in Canada, the USA and Australia. Lack of adherence to evidence-based recommendations may be driven by lack of knowledge of clinical practice guidelines (CPGs), heterogeneity in recommendations or concerns about their quality. We aim to systematically review CPG recommendations for paediatric injury care and appraise their quality.

METHODS AND ANALYSIS

We will identify CPG recommendations through a comprehensive search strategy including Medical Literature Analysis and Retrieval System Online, Excerpta Medica dataBASE, Cochrane library, Web of Science, ClinicalTrials and websites of organisations publishing recommendations on paediatric injury care. We will consider CPGs including at least one recommendation targeting paediatric injury populations on any diagnostic or therapeutic intervention from the acute phase of care with any comparator developed in high-income countries in the last 15 years (January 2007 to a maximum of 6 months prior to submission). Pairs of reviewers will independently screen titles, abstracts and full text of eligible articles, extract data and evaluate the quality of CPGs and their recommendations using Appraisal of Guidelines Research and Evaluation (AGREE) II and AGREE Recommendations Excellence instruments, respectively. We will synthesise evidence on recommendations using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence-to-Decision framework and present results within a recommendations matrix.

ETHICS AND DISSEMINATION

Ethics approval is not a requirement as this study is based on available published data. The results of this systematic review will be published in a peer-reviewed journal, presented at international scientific meetings and distributed to healthcare providers.

PROSPERO REGISTRATION NUMBER

International Prospective Register of Systematic Reviews (CRD42021226934).

Recommendations for the pharmacological treatment of treatment-resistant depression: A systematic review protocol

INTRODUCTION

Depression is a serious and widespread mental health disorder. Although effective treatment does exist, a significant proportion of patients with depression fail to respond to antidepressant treatment trials, a condition named treatment-resistant depression. Efficient approach should be given this condition in order to revert the burden caused by depression. Clinical practice guidelines (CPGs) are evidence-based health promotion instruments to improve diagnosis and treatment. CPGs recommendations for treatment-resistant depression must be trustworthy. The objective of the proposed study is to systematically identify, appraise the quality of CPGs for the treatment of depression and elaborate a synthesis of recommendations for treatment-resistant depression of CPGs considered to be of high quality and with high quality recommendations.

METHODS AND ANALYSIS

We will search the databases of organizations, such as PubMed, Embase, Cochrane Library, PsycInfo, and the Virtual Health Library, and organizations that develop CPGs. Three independent researchers will assess the quality of the CPGs and their recommendations using the AGREE II and AGREE-REX instruments, respectively. Given the identification of divergences and convergences as well as weak and strong points among high quality CPGs, our work may help developers, clinicians and eventually patients.

ETHICS AND DISSEMINATION

No ethical approval is required for a systematic review, as no patient data will be used. The research results will be disseminated in conferences and submitted to a peer reviewed journal.

Quality of clinical practice guidelines for gestational and congenital toxoplasmosis: A systematic review of the literature

OBJECTIVE

To evaluate the methodological quality and transparency of the clinical practice guidelines (CPGs) for the prevention, diagnosis, and treatment of gestational and congenital toxoplasmosis (CT).

METHODS

Systematic review of the literature on gestational and CT CPGs conducted in the MEDLINE, Embase, TripDatabase, Biblioteca Virtual en Salud databases and extensive manual searches in 19 CPG repositories. The characteristics of each of the guidelines were extracted using My AGREE PLUS on-line. Three reviewers assessed overall quality using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) tool.

RESULTS

The combined systematic review found 8651 citations. Of them 46 full texts were reviewed, and eight documents were finally included: four toxoplasmosis CPGs, three prenatal care CPGs that included recommendations on toxoplasmosis, and one pregnancy infection guideline that also included recommendations on toxoplasmosis. The AGREE II domains found to have the highest scores were 'clarity of presentation' (85%; [37%-100%]), followed by 'scope and purpose' (73%; [33%-98%]), and 'editorial independence' (51%; [3%-94%]); the domains with the lowest scores were 'rigour of development' (36%; [11%-79%]), 'stakeholder involvement' (34%; [24%-85%]), and 'applicability' (17%; [6%-83%]). The Colombian and Spanish-Agencia de Evaluación de Tecnologías Sanitarias de Andalucía (AETSA) CPGs had the highest global AGREE II scores. Absolute interrater agreement was good to excellent.

CONCLUSION

Substantial quality variation was found among CPGs, which provided recommendations in accordance with the context of the disease in the corresponding country or region. Only two of the CPGs appraised obtained a good score and are classified as 'recommended'.

European clinical practice guidelines on the use of chemotherapy for advanced oesophageal and gastric cancers: a critical review using the AGREE II and the AGREE-REX instruments

PURPOSE

To assess the methodological quality of all relevant and recent European clinical practice guidelines (CPGs) for advanced oesophageal and gastric cancers, and to synthesise their recommendations on the use of chemotherapy.

METHODS

We searched PubMed, EMBASE, guidelines repositories, and other sources from 2010 onwards. We appraised quality using AGREE-II and AGREE-REX.

RESULTS

11 CPGs were included (five high, five low, and one moderate quality). Most guidelines showed deficiencies in the domain "applicability", with only three scoring above 60%. Nine did not report having sought the views and preferences of the target population. The lowest scores for AGREE-REX were item Values and Preferences of Target Users (1.6; SD 1.3), and item Values and Preferences of Policy/Decision-Makers (1.8; SD 1.7). The domain Clinical Applicability got the highest score and the domain Implementability got the lowest.

CONCLUSIONS

An urgent area of research is how to develop credible and implementable recommendations on the clinical use of CT for advanced oesophageal and gastric cancer.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO (CRD42021236753).

Methodological approaches for developing, reporting, and assessing evidence-based clinical practice guidelines: A systematic survey

OBJECTIVE

To produce a mapping and feature summary of approaches and tools available for the CPG community to develop, report, or assess four types of CPGs: (1) Standard original (or de novo) CPG, (2) Rapid original CPG, (3) Adapted/adopted CPGs, and (4) Updated CPGs.

STUDY DESIGN

The systematic literature search was conducted using Embase and PubMed, covering the period from January 2010 to October 13 2020. Two websites that collect and recommend approaches/tools to develop, report, or assess CPGs were also searched: Guidelines International Network and Equator Network. We screened the search results to include methodological papers that aimed to develop specific approaches/tools to develop, report, or assess any of the aforementioned four CPG types.

RESULTS

Among 10,581 citations, 46 papers reporting 46 approaches/tools were included. Of these 46 approaches/tools, 33 were about CPG development, seven were for CPG reporting, and six for CPG assessment. Among the 33 development approaches/tools, 26 did not state usability or validity information; but nine from 13 reporting or assessment approaches/tools did.

CONCLUSIONS

This study provides a mapping and feature summary of the current available approaches/tools, which serves to improve users' understanding to pave the way for informed choice and application.

Quality of clinical practice guidelines and recommendations for the management of pain, sedation, delirium and iatrogenic withdrawal in pediatric intensive care: a systematic review protocol

INTRODUCTION

Inadequate management of pain and sedation in critically ill children can cause unnecessary suffering and agitation, but also delirium and iatrogenic withdrawal. It is, therefore, important to address these four interrelated conditions together. Some clinical practice guidelines (CPGs) are available for the management of pain and sedation, and a few for delirium and iatrogenic withdrawal in the paediatric intensive care unit; none address the four conditions altogether. Critical appraisal of the quality of CPGs is necessary for their recommendations to be adopted into clinical practice. The aim of this systematic review is to identify and appraise the quality of CPGs and recommendations for management of either pain, sedation, delirium and iatrogenic withdrawal.

METHODS AND ANALYSIS

Researchers will conduct a systematic review in electronic databases (Medline ALL (Ovid), Embase.com, CINAHL with Full Text (EBSCO), JBI EBP Database (Ovid)), guideline repositories and websites of professional societies to identify CPGs published from 2010 to date. They will then combine index and free terms describing CPGs with pain, sedation, delirium and withdrawal. The researchers will include CPGs if they can be applied in the paediatric intensive care population (newborns to 18 years old) and include recommendation(s) for assessment of at least one of the four conditions. Two independent reviewers will screen for eligibility, complete data extraction and quality assessments using the Appraisal of Guidelines for Research and Evaluation (AGREE) II and the AGREE Recommendation Excellence instruments. Researchers will report characteristics, content and recommendations from CPGs in tabulated forms.

ETHICS AND DISSEMINATION

Ethical approval is not required for this systematic review. Results will be published in a peer-reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42021274364.

Recommendations for the development of Clinical Practice Guidelines

Clinical practice guidelines are statements that include recommendations intended to optimize patient care, are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options, and ensure that the best available clinical knowledge is used to provide effective and quality care. They can reduce inappropriate care and variability in clinical practice and can support the translation of new research knowledge into clinical practice. Recommendations from clinical practice guidelines can support health professionals by facilitating the decision-making process, empowering them to make more informed health care choices, clarifying which interventions should be priorities based on a favorable trade-off, and discouraging the use of those that have proven ineffective, dangerous, or wasteful. This review aims to summarize the key components of high-quality and trustworthy guidelines. Articles were retrieved from various libraries, databases, and search engines using free-text term searches adapted for different databases, and selected according to author discretion. Clinical practice guidelines in geriatrics can have a major impact on prevention, diagnosis, treatment, rehabilitation, health care, and the management of diseases and conditions, but they should only be implemented when they have high-quality, rigorous, and unbiased methodologies that consider older adult priorities and provide valid recommendations.

Adherence of Clinical Practice Guidelines for Pharmacologic Treatments of Hospitalized Patients With COVID-19 to Trustworthy Standards: A Systematic Review

IMPORTANCE

The COVID-19 pandemic created the need for rapid and urgent guidance for clinicians to manage COVID-19 among patients and prevent transmission.

OBJECTIVE

To appraise the quality of clinical practice guidelines (CPGs) using the National Academy of Medicine (NAM) criteria.

EVIDENCE REVIEW

A search of MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials to December 14, 2020, and a search of related articles to February 28, 2021, that included CPGs developed by societies or by government or nongovernment organizations that reported pharmacologic treatments of hospitalized patients with COVID-19. Teams of 2 reviewers independently abstracted data and assessed CPG quality using the 15-item National Guideline Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) instrument.

FINDINGS

Thirty-two CPGs were included in the review. Of these, 25 (78.1%) were developed by professional societies and emanated from a single World Health Organization (WHO) region. Overall, the CPGs were of low quality. Only 7 CPGs (21.9%) reported funding sources, and 12 (37.5%) reported conflicts of interest. Only 5 CPGs (15.6%) included a methodologist, described a search strategy or study selection process, or synthesized the evidence. Although 14 CPGs (43.8%) made recommendations or suggestions for or against treatments, they infrequently rated confidence in the quality of the evidence (6 of 32 [18.8%]), described potential benefits and harms (6 of 32 [18.8%]), or graded the strength of the recommendations (5 of 32 [15.6%]). External review, patient or public perspectives, or a process for updating were rare. High-quality CPGs included a methodologist and multidisciplinary collaborations involving investigators from 2 or more WHO regions.

CONCLUSIONS AND RELEVANCE

In this review, few COVID-19 CPGs met NAM standards for trustworthy guidelines. Approaches that prioritize engagement of a methodologist and multidisciplinary collaborators from at least 2 WHO regions may lead to the production of fewer, high-quality CPGs that are poised for updates as new evidence emerges.

TRIAL REGISTRATION

PROSPERO Identifier: CRD42021245239.

Appraising clinical applicability of studies: mapping and synthesis of current frameworks, and proposal of the FrACAS framework and VICORT checklist

BACKGROUND

Not all research findings are translated to clinical practice. Reasons for lack of applicability are varied, and multiple frameworks and criteria exist to appraise the general applicability of epidemiological and clinical research. In this two-part study, we identify, map, and synthesize frameworks and criteria; we develop a framework to assist clinicians to appraise applicability specifically from a clinical perspective.

METHODS

We conducted a literature search in PubMed and Embase to identify frameworks appraising applicability of study results. Conceptual thematic analysis was used to synthesize frameworks and criteria. We carried out a framework development process integrating contemporary debates in epidemiology, findings from the literature search and synthesis, iterative pilot-testing, and brainstorming and consensus discussions to propose a concise framework to appraise clinical applicability.

RESULTS

Of the 4622 references retrieved, we identified 26 unique frameworks featuring 21 criteria. Frameworks and criteria varied by scope and level of aggregation of the evidence appraised, target user, and specific area of applicability (internal validity, clinical applicability, external validity, and system applicability). Our proposed Framework Appraising the Clinical Applicability of Studies (FrACAS) classifies studies in three domains (research, practice informing, and practice changing) by examining six criteria sequentially: Validity, Indication-informativeness, Clinical relevance, Originality, Risk-benefit comprehensiveness, and Transposability (VICORT checklist).

CONCLUSIONS

Existing frameworks to applicability vary by scope, target user, and area of applicability. We introduce FrACAS to specifically assess applicability from a clinical perspective. Our framework can be used as a tool for the design, appraisal, and interpretation of epidemiological and clinical studies.

Analysis of the ADA's Standard of Diabetes Care Recommendation for Continuous Glucose Monitoring System

Transparency is a key factor to understand how recommendations were reached and to decide whether to adopt them or not. Currently, the American Diabetes Association (ADA) recommends the usage of real-time continuous glucose monitors in certain subgroups of people living with type 1 diabetes mellitus. In this commentary we sought to critically appraise this recommendation, mainly regarding the outcomes assessed, the evidence used for each outcome, and how the balance of benefits and harms was made. We found that the decision-making process followed to reach the ADA recommendation was not clear, and that the final recommendation does not seem to be supported in the assessed evidence.

Clinical Epidemiology in China series. Paper 4: The reporting and methodological quality of Chinese clinical practice guidelines published between 2014 and 2018: A systematic review

OBJECTIVE

This study aimed to systematically review the methodological and reporting quality of clinical practice guidelines (CPGs) developed in China and published in medical journals between 2014 and 2018.

STUDY DESIGN AND SETTING

We conducted a comprehensive search in multiple databases: MEDLINE (via PubMed), Embase, CBM (China Biology Medicine), CNKI (China National Knowledge Infrastructure) and Wanfang Data. We included all clinical practice guidelines developed in China between 2014 and 2018. The AGREE II tool and the RIGHT checklist were used to appraise the methodological quality and reporting quality of the included guidelines, respectively.

RESULTS

We identified 17,188 records, and included finally 573 CPGs. Most (n=507, 88.5%) were published in Chinese, and 508 (88.7%) were about Western medicine. Only 62 (10.8%) of the guidelines used the GRADE approach. The mean overall score of methodological quality over all guidelines was 19.4%, and the mean scores for the AGREE II domains were 28.6% (Scope and purpose), 17.0% (Stakeholder involvement), 11.7% (Rigor of development), 32.2% (Clarity of presentation), 14.2% (Applicability) and 12.8% (Editorial independence). The mean overall score for reporting quality over all guidelines was 30.2%, with the following mean scores for each RIGHT domain: 55.6% (Basic information), 43.8% (Background), 14.5% (Evidence), 29.2% (Recommendations), 10.7% (Review and quality assurance), 12.6% (Funding and declaration of interest) and 8.4% (Other information). Subgroup analyses found that both the methodological and reporting quality were generally higher among CPGs that used evidence grading systems or reported receiving funding.

CONCLUSION

Both the methodological quality and the reporting quality of CPGs developed in China have improved over time, but are still below the international average.

Quality appraisal of existing guidelines for the management of headache disorders by the AGREE II's method

OBJECTIVE

To evaluate the quality of existing clinical practice guidelines for headache management and their main recommendations.

BACKGROUND

Evidence-based clinical practice guidelines have been developed to support the clinical decision-making. However, to achieve this goal, the quality of these guidelines must be ensured.

METHODS

A systematic search for clinical practice guidelines for headache management was conducted in the PubMed database, in websites of known guideline developers and in websites of known headache associations. The quality appraisal was performed through the Appraisal of Guidelines for Research and Evaluation II method.

RESULTS

Twelve guidelines were evaluated. The domains of rigor of development, applicability, and editorial independence, which most influence the overall quality of guidelines, had the lowest average scores and the highest standard deviation rates (61% ± 23; 37% ± 20; 53% ± 31). The main recommendations regarding medication use for acute treatment of episodic tension-type headache and migraine in adult patients consisted of paracetamol, acetylsalicylic acid, and other nonsteroidal anti-inflammatory drugs in all guidelines.

CONCLUSIONS

The statistical results indicate that the appraised guidelines have room for both individual and collective improvement. In addition, there is a well-established medication recommendation pattern among all guidelines evaluated.

Knowledge translation of the Canadian 24-Hour Movement Guidelines for Adults aged 18-64 years and Adults aged 65 years or older: a collaborative movement guideline knowledge translation process

Establishing a step-by-step process that provides practitioners with a blueprint for translating movement guidelines into action stands to optimize the investment in guideline development, improve guideline promotion and uptake, and ultimately enhance population health. The purpose of this paper is to describe how the Knowledge-to-Action framework and integrated knowledge translation were operationalized to systematically inform our knowledge translation (KT) efforts for the Canadian 24-Hour Movement Guidelines for Adults aged 18-64 years and Adults aged 65 years or older. In October 2018, the need for a KT Process, operating in tandem with the Guideline Development Process, led to the establishment of a KT team with a specific structure and terms of reference. The KT team collaboratively agreed on decision-making principles prior to selecting target audiences to focus their efforts. We undertook formative research to assess the local context and determinants of guideline dissemination and implementation efforts among target audiences. Plans for the subsequent steps and research are outlined. We highlight recommendations and lessons learned for applying the process in other settings. Novelty We outline a collaborative and systematic process and research program for the knowledge translation of movement guidelines. This paper provides an innovative and replicable blueprint to optimize future movement guideline knowledge translation efforts.

EAU, AUA and NICE Guidelines on Surgical and Minimally Invasive Treatment of Benign Prostate Hyperplasia: A Critical Appraisal of the Guidelines Using the AGREE-II Tool

OBJECTIVE

To critically appraise the methodological rigour of the clinical practice guidelines (CPGs) vis-à-vis BPH surgery as used by specialist research associations in the US, Europe and UK, and to compare whether the guidelines cover all or only some of the available treatments.

METHODS

The current guidelines issued by the EUA, AUA and NICE associations have been analyzed by 4 appraisers using the AGREE-II instrument. We also compared the recommendations given in the guidelines for surgical and minimally invasive treatment to find out which of these CPGs include most of the available treatment options.

RESULTS

According to the AGREE II tool, the median scores of domains were: domain 1 scope and purpose 66.7%, domain 2 stakeholder involvement 50.0%, domain 3 rigor of development 65.1%, domain 4 clarity of presentation 80.6%, domain 5 applicability 33.3%, domain 6 editorial independence 72.9%. The overall assessment according to AGREE II is 83.3%. The NICE guideline scored highest on 5 out of 6 domains and the highest overall assessment score (91.6%). The EAU guideline scored lowest on 4 out of 6 domains and has the lowest overall assessment score (79.1%).

CONCLUSIONS

The analyzed CPGs comprehensively highlight the minimally invasive and surgical treatment options for BPH. According to the AGREE II tool, the domains for clarity of presentation and editorial independence received the highest scores. The stakeholder involvement and applicability domains were ranked as the lowest. Improving the CPG in these domains may help to improve the clinical utility and applicability of CPGs.

Food-based dietary guidelines in Spain: an assessment of their methodological quality

BACKGROUND & OBJECTIVES

Food-based dietary guidelines (FBDGs) have been developed to promote healthy diets and prevent chronic diseases. However, the methodological quality of Spanish FBDGs has not been systematically assessed yet. The objective of this review is to identify and assess the methodological quality of Spanish FBDGs, as well as to describe their food guides and key recommendations.

METHODS

We conducted a systematic search to identify Spanish FBDGs targeted at the general population using multiple sources. Two authors independently screened the references, extracted data, and assessed the quality of the FBDGs using the Appraisal of Guidelines for Research and Evaluation (AGREE) II and the Recommendation Excellence (AGREE-REX) instruments. We performed a descriptive analysis of the FBDGs.

RESULTS

We included 19 FBDGs, published between 2007 and 2019. The median scores for each AGREE II domain were: "scope and purpose" 44% (Q1-Q3: 33-61%); "Stakeholder involvement" 31% (11-44%), "rigor of development" 3% (1-14%); "clarity of presentation" 42% (33-47%), "applicability" 0% (0-6%); and "editorial independence" 0% (0-8%). Six FBDGs (32%; 6/19) were categorized as "recommended with modifications", and the rest (68%; 13/19) as "not recommended". None of the FBDGs scored ≥60% in three or more domains, including the "rigor of development" domain. FBDGs indexed in literature databases scored significantly higher in overall rating than those not indexed (P = 0.023). The majority of FBDGs (74%; 14/19) used the pyramid as a food guide representation with a larger number of food levels (3-7 levels). The majority of FBDGs recommended a daily intake of cereals and grains, vegetables, fruits, olive oil and dairy products; a weekly intake of vegetable and animal proteins; and the occasional and limited intake of other food groups (e.g., ultraprocessed foods).

CONCLUSIONS

Overall, the methodological quality of FBDGs is poor showing that only 32% of FBDGs are "recommended for use with modifications". Our results highlight the need to revise, systematize and improve FBDG development processes in Spain.

Appraisal of clinical practice guidelines for the evaluation and management of neck masses in children

AIM

The purpose of this systematic review was to use the Appraisal of Guidelines for Research and Evaluation tool to assess the methodological quality of clinical practice guidelines (CPGs) for the workup and management of paediatric neck masses.

METHODS

MEDLINE, Embase, Cochrane and grey literature were searched to identify CPG incorporating paediatric neck masses. Four authors with previous training of the Appraisal of Guidelines for Research and Evaluation tool evaluated the included studies.

RESULTS

Nine studies met inclusion criteria. The highest scoring domains were 'Scope and Purpose' (74.0 ± 4.5) and 'Clarity of Presentation' (72.9 ± 6.3). The lowest scoring domains were 'Rigour and Development' (18.8 ± 7.5) and 'Applicability' (23.7 ± 6.1). One study was 'High' quality, three received scores of 'Average' and five were found to be 'Low' quality.

CONCLUSION

The majority of paediatric neck mass CPGs were low to average quality. The domains in need of greatest improvement were 'Rigour and Development' and 'Applicability', suggesting significant concerns in current CPGs focused on paediatric neck masses.

How to specify healthcare process improvements collaboratively using rapid, remote consensus-building: a framework and a case study of its application

BACKGROUND

Practical methods for facilitating process improvement are needed to support high quality, safe care. How best to specify (identify and define) process improvements - the changes that need to be made in a healthcare process - remains a key question. Methods for doing so collaboratively, rapidly and remotely offer much potential, but are under-developed. We propose an approach for engaging diverse stakeholders remotely in a consensus-building exercise to help specify improvements in a healthcare process, and we illustrate the approach in a case study.

METHODS

Organised in a five-step framework, our proposed approach is informed by a participatory ethos, crowdsourcing and consensus-building methods: (1) define scope and objective of the process improvement; (2) produce a draft or prototype of the proposed process improvement specification; (3) identify participant recruitment strategy; (4) design and conduct a remote consensus-building exercise; (5) produce a final specification of the process improvement in light of learning from the exercise. We tested the approach in a case study that sought to specify process improvements for the management of obstetric emergencies during the COVID-19 pandemic. We used a brief video showing a process for managing a post-partum haemorrhage in women with COVID-19 to elicit recommendations on how the process could be improved. Two Delphi rounds were then conducted to reach consensus.

RESULTS

We gathered views from 105 participants, with a background in maternity care (n = 36), infection prevention and control (n = 17), or human factors (n = 52). The participants initially generated 818 recommendations for how to improve the process illustrated in the video, which we synthesised into a set of 22 recommendations. The consensus-building exercise yielded a final set of 16 recommendations. These were used to inform the specification of process improvements for managing the obstetric emergency and develop supporting resources, including an updated video.

CONCLUSIONS

The proposed methodological approach enabled the expertise and ingenuity of diverse stakeholders to be captured and mobilised to specify process improvements in an area of pressing service need. This approach has the potential to address current challenges in process improvement, but will require further evaluation.

AGREEing on clinical practice guidelines for idiopathic steroid-sensitive nephrotic syndrome in children

BACKGROUND

Nephrotic syndrome is the most common kidney disease in children worldwide. Our aim was to critically appraise the quality of recent Clinical Practice Guidelines (CPGs) for idiopathic steroid-sensitive nephrotic syndrome (SSNS) in children in addition to summarize and compare their recommendations.

METHODS

Systematic review of CPGs. We identified clinical questions and eligibility criteria and searched and screened for CPGs using bibliographic and CPG databases. Each included CPG was assessed by four independent appraisers using the Appraisal of Guidelines for REsearch & Evaluation II (AGREE-II) instrument. We summarized the recommendations in a comparison practical table.

RESULTS

Our search retrieved 282 citations, of which three CPGs were eligible and appraised: Kidney Disease: Improving Global Outcomes (KDIGO) 2012, Japan Society for Pediatric Nephrology (JSPN) 2014, and American Academy of Pediatrics (AAP) 2009. Among these, the overall assessment of two evidence-based CPGs scored > 70% (KDIGO and JSPN), which was consistent with their higher scores in the six domains of the AGREE II Instrument. In domain 3 (rigor of development), KDIGO, JSPN, and AAP scored 84%, 74%, and 41%, respectively. In domain 5 (applicability), they scored 22%, 16%, and 19%, respectively, and in domain 6 (editorial independence), they scored 94%, 65%, and 88%, respectively.

CONCLUSIONS

The methodological quality of the KDIGO CPG was superior, followed by JSPN and AAP CPGs with the relevant recommendations for use in practice.

SYSTEMATIC REVIEW REGISTRATION

The protocol was registered in the Center for Open Science (OSF) DOI: 10.17605/OSF.IO/6QTMD and in the International prospective register of systematic reviews PROSPERO 2020 CRD42020197511 .

Clinical practice guidelines for recurrent miscarriage in high-income countries: a systematic review

Recurrent miscarriage affects 1-2% of women of reproductive age, depending on the definition used. A systematic review was conducted to identify, appraise and describe clinical practice guidelines (CPG) published since 2000 for the investigation, management, and/or follow-up of recurrent miscarriage within high-income countries. Six major databases, eight guideline repositories and the websites of 11 professional organizations were searched to identify potentially eligible studies. The quality of eligible CPG was assessed using the Appraisal of Guidelines for Research and Evaluation (AGREE II) Tool. A narrative synthesis was conducted to describe, compare and contrast the CPG and recommendations therein. Thirty-two CPG were included, from which 373 recommendations concerning first-trimester recurrent miscarriage were identified across four sub-categories: structure of care (42 recommendations, nine CPG), investigations (134 recommendations, 23 CPG), treatment (153 recommendations, 24 CPG), and counselling and supportive care (46 recommendations, nine CPG). Most CPG scored 'poor' on applicability (84%) and editorial independence (69%); and to a lesser extent stakeholder involvement (38%) and rigour of development (31%). Varying levels of consensus were found across CPG, with some conflicting recommendations. Greater efforts are required to improve the quality of evidence underpinning CPG, the rigour of their development and the inclusion of multi-disciplinary perspectives, including those with lived experience of recurrent miscarriage.

Early Mobilization After Stroke: Do Clinical Practice Guidelines Support Clinicians' Decision-Making?

Importance: Early mobilization, out-of-bed activity, is a component of acute stroke unit care; however, stroke patient heterogeneity requires complex decision-making. Clinically credible and applicable CPGs are needed to support and optimize the delivery of care. In this study, we are specifically exploring the role of clinical practice guidelines to support individual patient-level decision-making by stroke clinicians about early mobilization post-stroke.

Methods: Our study uses a novel, two-pronged approach. (1) A review of CPGs containing recommendations for early mobilization practices published after 2015 was appraised using purposely selected items from the Appraisal of Guidelines Research and Evaluation–Recommendations Excellence (AGREE-REX) tool relevant to decision-making for clinicians. (2) A cross-sectional study involving semi-structured interviews with Australian expert stroke clinicians representing content experts and CPG target users. Every CPG was independently assessed against the AGREE-REX standard by two reviewers. Expert stroke clinicians, invited via email, were recruited between June 2019 to March 2020.The main outcomes from the review was the proportion of criteria addressed for each AGREE-REX item by individual and all CPG(s). The main cross-sectional outcomes were the distributions of stroke clinicians' responses about the utility of CPGs, specific areas of uncertainty in early mobilization decision-making, and suggested parameters for inclusion in future early mobilization CPGs.

Results: In 18 identified CPGs, many did not adequately address the “Evidence” and “Applicability to Patients” AGREE-REX items. Out of 30 expert stroke clinicians (11 physicians [37%], 11 physiotherapists [37%], 8 nurses [26%]; median [IQR] years of experience, 14 [10–25]), 47% found current CPGs “too broad or vague,” while 40% rely on individual clinical judgement and interpretation of the evidence to select an evidence-based choice of action. The areas of uncertainty in decision-making revealed four key suggestions: (1) more granular descriptions of patient and stroke characteristics for appropriate tailoring of decisions, (2) clear statements about when clinical flexibility is appropriate, (3) detailed description of the intervention dose, and (4) physical assessment criteria including safety parameters.

Conclusions: The lack of specificity, clinical applicability, and adaptability of current CPGs to effectively respond to the heterogeneous clinical stroke context has provided a clear direction for improvement.

Integrated Guidance for Enhancing the Care of Familial Hypercholesterolaemia in Australia

Familial hypercholesterolaemia (FH) is a dominant and highly penetrant monogenic disorder present from birth that markedly elevates plasma low-density lipoprotein (LDL)-cholesterol concentration and, if untreated, leads to premature atherosclerosis and coronary artery disease (CAD). There are approximately 100,000 people with FH in Australia. However, an overwhelming majority of those affected remain undetected and inadequately treated, consistent with FH being a leading challenge for public health genomics. To further address the unmet need, we provide an updated guidance, presented as a series of systematically collated recommendations, on the care of patients and families with FH. These recommendations have been informed by an exponential growth in published works and new evidence over the last 5 years and are compatible with a contemporary global call to action on FH. Recommendations are given on the detection, diagnosis, assessment and management of FH in adults and children. Recommendations are also made on genetic testing and risk notification of biological relatives who should undergo cascade testing for FH. Guidance on management is based on the concepts of risk re-stratification, adherence to heart healthy lifestyles, treatment of non-cholesterol risk factors, and safe and appropriate use of LDL-cholesterol lowering therapies, including statins, ezetimibe, proprotein convertase subtilisin/kexin type 9 inhibitors and lipoprotein apheresis. Broad recommendations are also provided for the organisation and development of health care services. Recommendations on best practice need to be underpinned by good clinical judgment and shared decision making with patients and families. Models of care for FH need to be adapted to local and regional health care needs and available resources. A comprehensive and realistic implementation strategy, informed by further research, including assessments of cost-benefit, will be required to ensure that this new guidance benefits all Australian families with or at risk of FH.

The RIGHT Extension Statement for Traditional Chinese Medicine: Development, Recommendations, and Explanation

Nowadays, the number of traditional Chinese medicine (TCM) guidelines is constantly increasing, but its reporting quality remains unsatisfactory. One of the main reasons is that there is a lack of suitable reporting standard to guide it. In response to this long-standing problem, the Reporting Items for practice Guidelines in HealThcare (RIGHT) Working Group has invited a group of TCM clinical experts, methodologists and epidemiology, and developed the RIGHT Extension Statement for TCM (RIGHT-TCM) through a multi-staged development process, including systematic review, reporting quality evaluation and online Delphi expert consensus. The RIGHT-TCM extends two sections of the RIGHT Statement, includes basic information and recommendations section. Seven strong recommendation sub-items were added to RIGHT Statement and formed the final RIGHT-TCM. The group hopes that the RIGHT-TCM may assist TCM guideline developers in reporting guidelines, support journal editors and peer reviewers when considering TCM guideline reports, and help health care practitioners understand and implement a TCM guideline. This article will introduce its background, development, recommendations and explanation.

Assessment of the quality of recommendations from 161 clinical practice guidelines using the Appraisal of Guidelines for Research and Evaluation-Recommendations Excellence (AGREE-REX) instrument shows there is room for improvement

Objective

To assess the quality of recommendations from 161 clinical practice guidelines (CPGs) using AGREE-REX-D (Appraisal of Guidelines REsearch and Evaluation-Recommendations Excellence Draft).

Design

Cross-sectional study

Setting

International CPG community.

Participants

Three hundred twenty-two international CPG developers, users, and researchers.

Intervention

Participants were assigned to appraise one of 161 CPGs selected for the study using the AGREE-REX-D tool

Main outcome measures

AGREE-REX-D scores of 161 CPGs (7-point scale, maximum 7).

Results

Recommendations from 161 CPGs were appraised by 322 participants using the AGREE-REX-D. CPGs were developed by 67 different organizations. The total overall average score of the CPG recommendations was 4.23 (standard deviation (SD) = 1.14). AGREE-REX-D items that scored the highest were (mean; SD): evidence (5.51; 1.14), clinical relevance (5.95; SD 0.8), and patients/population relevance (4.87; SD 1.33), while the lowest scores were observed for the policy values (3.44; SD 1.53), local applicability (3,56; SD 1.47), and resources, tools, and capacity (3.49; SD 1.44) items. CPGs developed by government-supported organizations and developed in the UK and Canada had significantly higher recommendation quality scores with the AGREE-REX-D tool (p < 0.05) than their comparators.

Conclusions

We found that there is significant room for improvement of some CPGs such as the considerations of patient/population values, policy values, local applicability and resources, tools, and capacity. These findings may be considered a baseline upon which to measure future improvements in the quality of CPGs.

Overview of Systematic Reviews with Meta-Analysis Based on Randomized Controlled Trials of Balneotherapy and Spa Therapy from 2000 to 2019

Background

The objectives of this review were to summarize systematic reviews with meta-analysis of balneotherapy (BT) and spa therapy (ST) based on randomized controlled trials, and to provide a perspective for future research.

Methods

Eligible studies were systematic reviews based on randomized controlled trials with meta-analysis that included at least one group treated with BT or ST. We searched the following databases for articles published in English from the year 2000 to 20 November 2019: Cochrane Database Systematic Review, MEDLINE, CINAHL, Web of Science, and Ichushi-Web.

Results

Eighteen studies met all inclusion criteria. Based on the International Classification of Diseases (ICD)-11, among these 18 studies, 8 (44%) were about “#15 Diseases of the musculoskeletal system or connective tissue”, 5 (28%) were about “#21 Symptoms, signs or clinical findings, not elsewhere classified”, 4 (22%) were about “#11 Diseases of the circulatory system”, and 1 study (6%) was about “#8 Diseases of the nervous system”. Both BT and ST provided significant pain relief and improved quality of life in chronic diseases of the musculoskeletal system and connective tissues. Additionally, BT and ST with exercise under water improved physical fitness and function in patients across diseases.

Conclusion

Researchers need to conduct studies on the treatment of many kinds of potential diseases using the keywords of pain relief and QoL. In addition, depending on patients’ symptoms, physical fitness, and disabilities, performing exercise under water may improve treatment effects on physical function and fitness.