High methodologic quality but poor applicability: assessment of the AAOS guidelines using the AGREE II instrument

Determining the methodological rigor and overall quality of out-of-hospital clinical practice guidelines: a scoping review

OBJECTIVES

Out-of-hospital clinical practice guidelines (CPGs) guide paramedics, emergency medical technicians and first responders, but their quality remains uncertain. This scoping review aims to identify, aggregate and describe all literature that has used a structured appraisal instrument to assess the methodological rigor and overall quality of out-of-hospital CPGs.

METHODS

This study was conducted in accordance with the JBI methodology for scoping reviews and involved systematically searching the following databases and/or information sources with no publication or language limit applied: MEDLINE (Ovid), Embase (Elsevier), CINAHL with full text (EBSCO), Scopus (Elsevier), ProQuest Central (ProQuest).

RESULTS

This review identified 15 articles that appraised 311 unique out-of-hospital CPGs. These CPGs ranged in date of publication from 1998 to 2022. The majority of CPGs (267/311) were assessed using the Appraisal of Guidelines for Research & Evaluation (AGREE-II) instrument, with 146 guidelines appraised against two tools. Following aggregation, CPGs scored highest in Domain 4 (clarity of presentation) at 77.7% (SD = 15.1%), and lowest in Domain 5 (applicability) at 42.6% (SD = 23.7%). The average Domain 3 score (rigor of development) was 55.6% (SD = 25.7%). Of CPGs appraised against the AGREE-II instrument, 34.4% met our a priori definition of being high-quality (Domain 3 score of equal to or greater than 75%), while 31.3% were deemed medium-quality (Domain 3 score between 74% and 50%), and 34.3% were considered low-quality (Domain 3 score less than 50%). There were no significant changes observed in the average Domain 3 score over time (p = 0.092). 146 CPGs were assessed against the National Academy of Medicine criteria with 34.9% meeting all elements indicative of being a high-quality guideline, while 39 CPGs were assessed the 2016 National Health and Medical Research Council Standards for Guidelines with 0% meeting all criteria.

CONCLUSIONS

Out-of-hospital CPGs currently have poor methodological rigor and are of medium to low overall quality. These results should be used to inform future research and initiatives that aim to standardize the methods used to develop guidelines used in this healthcare setting.

Evaluation of clinical practice guidelines on treatment of cystic fibrosis: A systematic review

BACKGROUND

Despite the existence of numerous clinical practice guidelines (CPGs) for cystic fibrosis (CF), there is limited understanding of their credibility and consistency. This systematic review aims to comprehensively evaluate the quality of CPGs for CF and its pulmonary complications, focusing on treatment recommendations for pulmonary care.

METHODS

We conducted a comprehensive search across four databases and relevant websites to identify eligible guidelines providing treatment recommendations. The quality of these guidelines was assessed using the Appraisal of Guidelines for Research and Evaluation (AGREE) II tool. Pulmonary treatment recommendations were analyzed and synthesized narratively.

RESULTS

A total of 35 guidelines were identified. Most guidelines were of moderate quality according to the AGREE II instrument, with overall scores ranging from 21·05 to 76·13. Only six guidelines were recommended for use. These guidelines provide 359 pulmonary treatment recommendations for seven primary therapies and others. There was inconsistency in the use of airway clearance therapy, anti-inflammatories, antibiotics, inhaled drugs, and cystic fibrosis transmembrane conductance regulator modulator therapy. Four guidelines conditionally advocated for oral corticosteroids, while six opposed routine inhaled corticosteroids. One guideline discouraged lumacaftor-ivacaftor in the general CF population, two recommended only for children under 12 years old, and another strongly advocated for children between 2 and 5 years of age. However, one guideline noted a lack of evidence to recommend it for children under 6.

CONCLUSION

The quality of CPGs for CF and its pulmonary complications has improved over time, reaching a moderate level generally, but there is still room for further improvement. Future efforts should focus on standardizing methodological frameworks and generating robust clinical evidence to enhance the overall quality and applicability of CF guidelines.

Evidence-based clinical practice guidelines for the management of acute ankle injuries according to: a PRISMA systematic review and quality appraisal with AGREE II

BACKGROUND

Acute ankle injuries are commonly seen in emergency rooms, with significant social impact and potentially devastating consequences. While several clinical practice guidelines (CPGs) related to ankle injuries have been developed by various organizations, there is a lack of critical appraisal of them. The purpose of this systematic review is to identify and critically appraise evidence-based clinical practice guidelines (EB-CPGs) related to acute ankle injuries in adults.

METHOD

We conducted searches in the Cochrane Library, MEDLINE, EMBASE databases, WHO, and reviewed 98 worldwide orthopedic association websites up until early 2023. Two authors independently applied the inclusion and exclusion criteria, and each evidence-based clinical practice guideline (EB-CPG) underwent independent critical appraisal of its content by all four authors using the Appraisal of Guidelines for REsearch and Evaluation (AGREE II) instrument. AGREE II scores for each domain were then calculated.

RESULTS

This review included five evidence-based clinical practice guidelines. The mean scores for all six domains were as follows: Scope and Purpose (87.8%), Stakeholder Involvement (69.2%), Rigour of Development (72.5%), Clarity of Presentation (86.9%), Applicability (45.6%), and Editorial Independence (53.3%).

CONCLUSION

The number of EB-CPGs related to ankle injuries are limited and the overall quality of the existing evidence-based clinical practice guidelines (EB-CPGs) for ankle injuries is not strong, with three of them being outdated. However, valuable guidance related to Ottawa rules, manual therapy, cryotherapy, functional supports, early ambulation, and rehabilitation has been highlighted. Challenges remain in areas such as monitoring and/or auditing criteria, consideration of the target population's views and preferences, and ensuring editorial independence. Future guidelines should prioritize improvements in these domains to enhance the quality and relevance of ankle injury management.

SYSTEMATIC REVIEW

Systematic review.

A Critical Appraisal of the Congress of Neurological Surgeons Evidence-Based Guidelines on the Evaluation and Treatment of Patients With Thoracolumbar Spine Trauma

Background and objective Thoracolumbar spine trauma (TST) is frequently associated with spinal cord injury and other soft tissue and bony injuries. The management of such injuries requires an evidence-based approach. This study used the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument to assess the methodological quality of clinical guidelines for the management of TST published by the Congress of Neurological Surgeons (CNS). Methods All clinical guidelines on TST published by CNS until 2020 were assessed. Five appraisers from three international centers evaluated the quality of eligible clinical guidelines by using AGREE II. Mean AGREE II scores for each domain were determined. In higher-quality domains, the scores for individual items were analyzed. Results A total of 12 guidelines published by CNS on TST were assessed. Mean scores for all six domains were as follows: Scope and Purpose (75.2%), Stakeholder Involvement (45.4%), Rigor of Development (57.0%), Clarity of Presentation (58.7%), Applicability (16.9%), and Editorial Independence (64.1%). The mean score for the overall quality of all CNS guidelines was 52.9% [95% confidence interval (CI): 52.2-53.5%]. The overall agreement among appraisers was excellent [intra-class correlation coefficients (ICCs) for each guideline ranged from 0.903 to 0.963]. Conclusions CNS guidelines for the management of TST demonstrated acceptable quality across most domains; however, the domains of Applicability and Stakeholder Involvement could be further improved in future guideline updates. The assessors concluded that all guidelines could still be recommended for clinical practice with or without modifications.

Compliance with Australian Orthopaedic Association guidelines does not reduce the risk of venous thromboembolism after total hip and knee arthroplasty

Preventing avoidable venous-thrombo-embolism (VTE) is a priority to improve patient and service outcomes after total hip and total knee arthroplasty (THA, TKA), but compliance with relevant clinical guidelines varies. This study aims to determine the degree to which prophylaxis was compliant with Australian Orthopaedic Association (AOA) VTE prophylaxis guidelines and whether non-compliance is associated with increased risk of VTE. A prospective multi-centre cohort study of adults with osteoarthritis undergoing primary TKA/THA was completed at 19 high-volume public and private hospitals. Data were collected prior to surgery and for one-year post-surgery. Logistic regression was undertaken to explore associations between non-compliance with AOA VTE prophylaxis guidelines and symptomatic 90-day VTE outcomes. Data were analysed for 1838 participants from 19 sites. The rate of non-compliance with all clinical guideline recommendations was 20.1% (N = 369), with 14.1% (N = 259) non-compliance for risk-stratified prophylaxis, 35.8% (N = 658) for duration, and 67.8% (N = 1246) for other general recommendations. Symptomatic VTE was experienced up to 90-days post-surgery by 48 people (2.6%). Overall guideline non-compliance (AOR = 0.93, 95%CI = 0.4 to 1.3, p = 0.86) was not associated with a lower risk of symptomatic 90-day VTE. Results were consistent when people with high bleeding risk were excluded (AOR = 0.94, 95%CI = 0.44 to 2.34, p = 0.89). Non-compliance with the AOA VTE prophylaxis guidelines was not associated with risk of 90-day VTE after arthroplasty. This counterintuitive finding is concerning and necessitates a rigorous review of the AOA VTE prevention clinical guideline.

Assessment of the AAOS guidelines using the AGREE II instrument: An update

Background

The American Academy of Orthopaedic Surgeons (AAOS) provides clinical guidelines and frequently updates and expands on the recommendations. High quality, up-to-date, and applicable clinical guidelines are increasingly important tools for physicians to have. Assessing these continuously updating guidelines' overall quality has most recently been done in 2013 when there were 14 guidelines. Since then, nearly all guidelines have been updated and now 28 guidelines are provided. Our goal was to evaluate the updated 2023 AAOS guidelines using the well-established Advancing Guideline Development, Reporting and Evaluation in Health Care (AGREE II) tool.

Methods

On April 23, 2023, all 28 guidelines set forth by AAOS were compiled. Using the AGREE II tool, a well-recognized and established tool for evaluation of guideline quality used across all disciplines, three independent reviewers appraised the current guidelines. These data were used to run descriptive statistics and assessed accordingly.

Results

Across the 28 guidelines, the average AGREE II domain scores were as follows: Scope and Purpose (median score, 85%), Stakeholder Involvement (median score, 87%), Rigor of Development (median score, 87%), Clarity of Presentation (median score, 87%), Applicability (median score, 48%), Editorial Independence (median score, 87%), and Overall Assessment (median score, 80s%).

Conclusions

Five of the six AGREE II domains, including the overall assessment, tested achieved a median score above the 50% mark. These results indicate that the AAOS guidelines met many criteria for high quality based on a critical appraisal.

Clinical Practice Guideline Recommendations in Pediatric Orthopaedic Injury: A Systematic Review

BACKGROUND

Lack of adherence to recommendations on pediatric orthopaedic injury care may be driven by lack of knowledge of clinical practice guidelines (CPGs), heterogeneity in recommendations or concerns about their quality. We aimed to identify CPGs for pediatric orthopaedic injury care, appraise their quality, and synthesize the quality of evidence and the strength of associated recommendations.

METHODS

We searched Medline, Embase, Cochrane CENTRAL, Web of Science and websites of clinical organizations. CPGs including at least one recommendation targeting pediatric orthopaedic injury populations on any diagnostic or therapeutic intervention developed in the last 15 years were eligible. Pairs of reviewers independently extracted data and evaluated CPG quality using the Appraisal of Guidelines Research and Evaluation (AGREE) II tool. We synthesized recommendations from high-quality CPGs using a recommendations matrix based on the GRADE Evidence-to-Decision framework.

RESULTS

We included 13 eligible CPGs, of which 7 were rated high quality. Lack of stakeholder involvement and applicability (i.e., implementation strategies) were identified as weaknesses. We extracted 53 recommendations of which 19 were based on moderate or high-quality evidence.

CONCLUSIONS

We provide a synthesis of recommendations from high-quality CPGs that can be used by clinicians to guide treatment decisions. Future CPGs should aim to use a partnership approach with all key stakeholders and provide strategies to facilitate implementation. This study also highlights the need for more rigorous research on pediatric orthopaedic trauma.

LEVEL OF EVIDENCE

Level II-therapeutic study.

Clinical Practice Guidelines in an Era of Accountability, Saudi Arabia: A Call for Action

INTRODUCTION

Clinical Practice Guidelines (CPGs) development and implementation in the Kingdom of Saudi Arabia are suboptimal. The Kingdom's Vision 2030 envisages a transformational change to achieve an effective, integrated, value-based ecosystem focused on patient health.

OBJECTIVES

This study aimed to develop a CPG appraisal tool that will support the realization of the Kingdom's Vision 2030 through the development of high-quality and highly implementable CPGs. To maximize its impact, all vital healthcare paradigms, such as systems thinking, value-based healthcare, and information technology, will robustly be incorporated in the tool.

METHODS

The Saudi Health Council through its National Center of Evidence-Based Medicine (NCEBM) embarked on a program to develop this appraisal tool. A taskforce of experts was selected based on their experience in evidence-based practice and training. The task force, through a methodology of extensive literature review, deliberation, outside experts' feedback, and Delphi and consensus voting, developed a prototype appraisal tool that was named the Holistic Appraisal Tool for CPGs (HAT-CPG).

RESULTS

The HAT-CPG was developed comprising three sections: an initial basic information section, an internal validity section, and an external validity section with a total of 13 section items and 73 reporting elements.

CONCLUSION

It is envisaged that the Holistic Appraisal Tool will support CPG developers and users in Saudi Arabia in realizing the objectives for which it was developed.

Editor's Choice - Quality Assessment of European Society for Vascular Surgery Clinical Practice Guidelines

OBJECTIVE

An increasing number of clinical practice guidelines (CPGs) have emerged over recent years. To have clinical utility, they need to be rigorously developed and scientifically robust. Instruments have been developed to assess the quality of clinical guideline development and reporting. The aim of this study was to evaluate CPGs from the European Society for Vascular Surgery (ESVS) using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument.

METHODS

CPGs published by the ESVS during the period January 2011 to January 2023 were included. Two independent reviewers assessed the guidelines after receiving training in the use and application of the AGREE II instrument. Inter-reviewer reliability was assessed with the intraclass correlation coefficient. Maximum scaled scores were 100. Statistical analysis was performed using SPSS Statistics v.26.

RESULTS

Sixteen guidelines were included in the study. Good inter-reviewer score reliability was found on statistical analysis (> 0.9). The mean ± standard deviation domain scores were 68.1 ± 20.3% for scope and purpose, 57.1 ± 21.1% for stakeholder involvement, 67.8 ± 19.5% for rigour of development, 78.1 ± 20.6% for clarity of presentation, 50.3 ± 15.4% for applicability, 77.6 ± 17.6% for editorial independence, and 69.8 ± 20.1% for overall quality. Stakeholder involvement and applicability have improved in quality over time but are still the lowest scoring domains.

CONCLUSION

Most ESVS clinical guidelines are of high quality and reporting. There is scope for improvement, specifically by addressing the domains of stakeholder involvement and clinical applicability.

Modern Paediatric Emergency Department: Potential Improvements in Light of New Evidence

The increasing attendance of paediatric emergency departments has become a serious health issue. To reduce an elevated burden of medical errors, inevitably caused by a high level of stress exerted on emergency physicians, we propose potential areas for improvement in regular paediatric emergency departments. In an effort to guarantee the demanded quality of care to all incoming patients, the workflow in paediatric emergency departments should be sufficiently optimised. The key component remains to implement one of the validated paediatric triage systems upon the patient's arrival at the emergency department and fast-tracking patients with a low level of risk according to the triage system. To ensure the patient's safety, emergency physicians should follow issued guidelines. Cognitive aids, such as well-designed checklists, posters or flow charts, generally improve physicians' adherence to guidelines and should be available in every paediatric emergency department. To sharpen diagnostic accuracy, the use of ultrasound in a paediatric emergency department, according to ultrasound protocols, should be targeted to answer specific clinical questions. Combining all mentioned improvements might reduce the number of errors linked to overcrowding. The review serves not only as a blueprint for modernising paediatric emergency departments but also as a bin of useful literature which can be suitable in the paediatric emergency field.

Patient and public involvement in implementation of evidence-based guidance for musculoskeletal conditions: a scoping review of current advances and gaps

Advances in musculoskeletal (MSK) research have been successfully curated into widely endorsed evidence-based recommendations and guidelines. However, there continues to exist significant variations in care and quality of care, and the global health and socio-economic burdens associated with MSK conditions continues to increase. Limited accessibility, and applicability of guideline recommendations have been suggested as contributory factors to less than adequate guideline implementation. Since patient and public involvement (PPI) is being credited with increasing relevance, dissemination and uptake of MSK research, the success of guidelines implementation strategies may also be maximised through increasing opportunities for PPI input. We therefore conducted a scoping review of literature to explore PPI in implementation of evidence-based guidance for MSK conditions. A comprehensive search was used to identify relevant literature in three databases (Medline, Embase, Cinahl) and two large repositories (WHO, G-IN), supplemented by grey literature search. Eligibility was determined with criteria established a priori and narrative synthesis was used to summarise PPI activities, contexts, and impact on implementation of MSK related evidence-based guidance across ten eligible studies (one from a low-and middle-income country LMIC). A prevalence of low-level PPI (mainly consultative activities) was found in the current literature and may partly account for current experiences of significant variations and quality of care for MSK patients. The success of PPI in MSK research may be lessened by the oversight of PPI in implementation. This has implications for both high- and low-resource healthcare systems, especially in LMICs where evidence is limited. Patient and public partnership for mobilising knowledge, maximising guideline uptake, and bridging the research-practice gap particularly in low resource settings remain important and should extend beyond PPI in research and guideline dissemination activities only. This review is a clarion call to stakeholders, and all involved, to transform PPI in MSK research into real world benefits through implementation approaches underpinned by patient and public partnerships. We anticipate that this will enhance and drive quality improvements in MSK care with patients and for patients across health and care settings.

COVID-19 therapies for inpatients: a review and quality assessment of clinical guidelines

Owing to condensed development processes, expanding evidence and differences in healthcare system characteristics, many COVID-19 guidelines differ in their quality and treatment recommendations, which has consequences for clinical practice. This review aimed to identify COVID-19 treatment guidelines, assess their quality and summarise their recommendations. Guidelines were identified for five therapies most commonly used among inpatients with COVID-19 (remdesivir, dexamethasone, tocilizumab, baricitinib and casirivimab/imdevimab) from 11 countries. Guideline quality was assessed using the Appraisal of Guidelines for Research and Evaluation II (AGREE-II) tool. Full details of recommendations and supporting evidence were analysed for high-quality guidelines, defined as those scoring ≥50% in Domain 3 (Rigour of Development) of AGREE-II. Overall, guidelines differed substantially in their quality and, even among high-quality guidelines using the same evidence, recommendations regarding specific therapeutics varied. Potential reasons for this heterogeneity, including the availability and consistency of clinical data, visibility of trial end-points and context-specific factors, are discussed.

Abdominal aortic aneurysm clinical practice guidelines: a methodological assessment using the AGREE II instrument

OBJECTIVES

Abdominal aortic aneurysm (AAA) clinical practice guidelines (CPGs) provide evidence-based information on patient management; however, methodological differences exist in the development of CPGs. This study examines the methodological quality of AAA CPGs using a validated assessment tool.

METHODS

Medline, EMBASE and online CPG databases were searched from 1946 to 31 October 2021. Full-text, English language, evidence-based AAA CPGs were included. Consensus-based CPGs, summaries of CPGs or CPGs which were only available on purchase were excluded. Five reviewers assessed their quality using the Appraisal of Guidelines for Research and Evaluation II instrument. An overall guideline assessment scaled score of ≥80% was considered as the threshold to recommend CPG use in clinical practice.

RESULTS

Seven CPGs were identified. Scores showed good inter-reviewer reliability (intraclass correlation coefficient 0.943, 95% CI 0.915 to 0.964). On average, CPGs performed adequately with mean scaled scores of over 50% in all domains. However, between CPGs, significant methodological heterogeneity was observed in all domains. Four CPGs scored ≥80% (European Society of Cardiology, the Society of Vascular Surgery, the European Society of Vascular Surgery and the National Institute of Health and Care Excellence), supporting their use in clinical practice.

CONCLUSIONS

Four CPGs were considered of adequate methodological quality to recommend their use in clinical practice; nonetheless, these still showed areas for improvement, potentially through performing economic analysis and trial application of recommendations. A structured approach employing validated CPG creation tools should be used to improve rigour of AAA CPGs. Future work should also evaluate recommendation accuracy using validated appraisal tools.

Quality of recent clinical practice guidelines in anaesthesia publications using the Appraisal of Guidelines for Research and Evaluation II instrument

Clinical practice guidelines are a valuable resource aiding medical decision-making based on scientific evidence. In anaesthesia, guidelines are increasing in both number and scope, influencing individual practice and shaping local departmental policy. The aim of this review is to assess the quality of clinical practice guidelines published in high impact anaesthesia journals over the past 5 yr using the internationally validated Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument. A literature search was conducted in Scopus to identify all guidelines published in the top 10 anaesthesia journals as per Clarivate Analytics Impact Factor from 2016 and 2020. Fifty-one guidelines were included for analysis by five independent appraisers using AGREE II. Each guideline was assessed across six domains and 23 items. Individual domain scores were calculated with a threshold agreed via consensus to represent high-quality guidelines. There was a significant increase in overall score over time (P=0.041), driven by Domain 3 (Rigour of Development, P=0.046). The raw overall score for Domain 3, however, was low. The other domains performed as expected based on previous studies, with Domains 1, 4, and 6 achieving high scores and Domains 2 and 5 incurring poor ratings. Most guidelines studied involved international collaboration but emerged from a single professional society. Use of an appraisal tool was stated as high but poorly detailed. The improvement in the overall score of guidelines and rigour of development is promising; however, only seven guidelines met high-quality criteria, suggesting room for improvement for the overall integrity of guidelines in anaesthesia.

Critical Appraisal of Paralyzed Veterans of America Guidelines in Spinal Cord Injury: An International Collaborative Study Using the Appraisal of Guidelines for Research and Evaluation II Instrument (AGREE II)

INTRODUCTION

Spinal cord injuries (SCI) in military personnel, veterans, and others require an evidence-based, multidisciplinary approach to their care. This appraisal used the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument to evaluate the methodological quality of clinical guidelines for the management of SCI published by the Paralyzed Veterans of America (PVA) organization.

MATERIALS AND METHODS

We searched clinical guidelines on SCI published by PVA until December 2019. Four appraisers across three international centers independently evaluated the quality of eligible clinical guidelines using AGREE II. Mean AGREE II scores for each domain were calculated. In higher quality domains, scores for individual items were analyzed.

RESULTS

A total of 12 guidelines published by PVA on SCI were assessed. Mean scores for all six domains were as follows: Scope and Purpose (78.8%), Stakeholder Involvement (63.7%), Rigor of Development (68.4%), Clarity of Presentation (80.1%), Applicability (53.0%), and Editorial Independence (28.5%). The mean score for the overall quality of all PVA guidelines was 71.9% (95% CI: 69.7-74.1). No guideline was assessed as "not recommended" by any appraiser. Overall quality was significantly associated with year of publication (rs = 0.754, P = 0.0046). Overall agreement among appraisers was excellent (intraclass correlation coefficients for each guideline ranged from 0.96 to 0.99).

CONCLUSIONS

PVA guidelines for the management of SCI demonstrated acceptable or good quality across most domains. We recommend the use of PVA guidelines for the assessment and treatment of SCI and related disorders. The quality of PVA guidelines for the management of SCI have improved over time.

Quality of biosafety guidelines for dental clinical practice in world in early COVID-19 pandemic: A systematic review

OBJECTIVES

To conduct a systematic review of coronavirus disease 2019 (COVID-19)-related biosafety guidelines for dental clinical practice in the early stage of the pandemic, focusing on quality assessment.

METHODS

Electronic (via PubMed, Scopus, Web of Science, Latin American and Caribbean Health Sciences Literature database, Brazilian Library in Dentistry, and Cochrane Library) and gray literature searches were performed for documents published up to May 12, 2020. Guidelines updated until April 17, 2021 were identified. Documents were included as guidelines if they (1) consisted of a set of statements, directions, or principles presenting current or future rules or policy; (2) were developed by government agencies, institutions, organizations, or expert panels; and (3) were related to the general conduct of healthcare activities rather a particular condition. Two researchers, using the Appraisal of Guidelines for Research & Evaluation II, independently extracted the recommendations and evaluated the quality of the guidelines.

RESULTS

Twenty-seven documents from 19 countries were included in the review. These documents presented 122 recommendations related to (1) professional biosafety; (2) patients’/companions’ safety; (3) the organization and biosafety of the physical dental facility environment; and (4) the work process in dental care. Overall, the scientific quality of the guidelines was considered low. Some recommendations presented in these guidelines would require further research to establish their effectiveness.

CONCLUSIONS

We found a wide variety of biosafety guidelines for dental practice regarding COVID-19 in the early months of the pandemic, but their quality was low. Biosafety recommendations should be frequently updated.

Clinical practice guidelines for the prevention and management of frailty: A systematic review

AIMS

To systematically search for clinical practice guidelines focusing on the prevention and management of frailty, to evaluate their methodological quality and to synthesize the consensus recommendations.

DESIGN

A systematic review.

DATA SOURCES

Guideline websites, related professional association websites and electronic databases were systematically searched through 4 November 2020.

REVIEW METHODS

We evaluated the methodological quality of the eligible guidelines using the Appraisal of Guidelines Research and Evaluation II (AGREE II). Two reviewers synthesized the consensus recommendations proposed by at least two guidelines.

RESULTS

Eight guidelines met the eligibility criteria and were included in the review. The mean scores of the six domains were as follows: the 'scope and purpose' domain scored 88.0%, the 'clarity of presentation' domain scored 81.9%, the 'stakeholder involvement' domain scored 63.4%, the 'editorial independence' domain scored 62.2%, the 'rigour of development' domain scored 61.1% and the 'applicability' domain scored 57.8%. In total, we synthesized 23 recommendations for the prevention and management of frailty that are consistent among the included guidelines.

CONCLUSIONS

The number of clinical practice guidelines for the prevention and management of frailty is limited. The methodological quality of existing guidelines needs to be improved. Our synthesized findings provide an intuitive, convenient and summative reference resource for frailty prevention and management. It' is worth noting that recommendations described in the included guidelines require additional detail.

IMPACT

Although the prevention and management of frailty is urgent, there is currently a lack of evidences guiding these processes, especially in the prevention. The methodological quality of existing guidelines is insufficient, and the recommendations described in the guidelines require additional detail. Therefore, users of these guidelines, especially nurses, should make a careful decision according to the specific situation when using. Nurses also have a key role in providing more clinical evidences for the improvement of the quality of the guidelines.

Appraisal of the current guidelines for the management of diverticular disease using the Appraisal of Guidelines Research and Evaluation II (AGREE II) instrument

INTRODUCTION

Diverticular disease is one of the most frequent reasons for attending emergency departments and surgical causes of hospital admission. In the past decade, many surgical and gastroenterological societies have published guidelines for the management of diverticular disease. The aim of the present study was to appraise the methodological quality of these guidelines using the Appraisal of Guidelines Research and Evaluation II (AGREE II) tool.

METHODS

PubMed, Embase, Cochrane Library and Google Scholar databases were searched systematically. The methodological quality of the guidelines was appraised independently by five appraisers using the AGREE II instrument.

FINDINGS

A systematic search of the literature identified 12 guidelines. The median overall score of all guidelines was 68%. Across all guidelines, the highest score of 85% was demonstrated in the domain 'Scope and purpose'. The domains 'Clarity and presentation' and 'Editorial independence' both scored a median of 72%. The lowest scores were demonstrated in the domains 'Stakeholder involvement' and 'Applicability' at 46% and 40%, respectively. Overall, the National Institute for Health and Care Excellence (NICE) guidelines performed consistently well, scoring 100% in five of six domains; NICE was one of the few guidelines that specifically reported stakeholder involvement, scoring 97%. Generally, the domain of 'Stakeholder involvement' ranked poorly with seven of twelve guidelines scoring below 50%, with the worst score in this domain demonstrated by Danish guidelines at 25%.

CONCLUSION

Six of twelve guidelines (NICE, American Society of Colon & Rectal Surgeons (ASCRS), European Society of Coloproctology (ESCP), American Gastroenterological Association, German Society of Gastroenterology/German Society for General and Visceral Surgery (German), Netherlands Society of Surgery) scored above 70%. Only three, NICE, ASCRS and ESCP, scored above 75% and were voted unanimously by the appraisers for use as they are. Therefore, use of AGREE II may help improve the methodological quality of guidelines and their future updates.

Methodological quality of guidelines for the management of opioid use disorder: A systematic review

WHAT IS KNOWN AND OBJECTIVE

Opioid Use Disorder (OUD) has a high mortality rate and affects millions of people worldwide. Many organizations and societies develop Clinical Practice Guidelines (CPGs) to serve as a framework for healthcare providers to decide and support best practice to manage and treat OUD. However, not all CPGs sufficiently address all the important aspects of optimal care for managing OUD. This study aims to review current CPGs for management of OUD, evaluate their methodological quality and summarize their recommendations.

METHODS

We conducted this systematic review according to Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA). Various databases were searched for CPGs and Appraisal of Guidelines for Research and Evaluation (AGREE-II) instrument was used to assess the methodological quality. We also summarized the treatments plans of CPGs across continuum of care (diagnosis and assessment, treatment initiation, pharmacotherapy and psychosocial).

RESULTS

This review included 28 CPGs of varying qualities. CPGs from high-income countries and international organizations rated high for their methodological quality. Most CPGs scored high for the scope and purpose domain and scored low for applicability domain. Recommendations for the continuum of care for OUD varied across CPGs. Buprenorphine was recommended in most of the CPGs, followed by methadone. Recommendations for psychosocial interventions also varied, with cognitive behaviour therapies and counselling or education being the common recommendations in many CPGs WHAT IS NEW AND CONCLUSION: We found most CPGs have scope and purpose and clarity of presentation. However, the methodological rigour and applicability scored low. CPGs need to frame health questions in a comprehensible manner and provide an update as evidence grows. It is important for CPG developers to consider methodological quality as a factor when developing CPG recommendations.

Erectile dysfunction management: a critical appraisal of clinical practice guidelines with the AGREE II instrument

Our study aimed to assess the methodological strengths and weaknesses of erectile dysfunction clinical practice guidelines (CPGs) for individuals using the AGREE II tool. Erectile dysfunction related CPGs were identified from three databases: the National Guideline Clearinghouse, the Guidelines International Network, and PubMed between 2000 and 2020. We designed an independent assessment for each of the erectile dysfunction related CPGs using the AGREE II tool. Four appraisers performed these assessments. The literature search identified 8 CPGs that met our inclusion criteria. The evaluation of the AGREE II domains of each individual revealed that the median scores of domains related to applicability were quite low (39%). Also, the median scores of domains related to the rigour of development and the stakeholder involvement were relatively low (53% and 63%). We determined the highest median scores in three AGREE II domains: clarity of presentation (80.5%), editorial independence (77%), and scope and purpose (89.5%). We found that the European Association of Urology (EAU), the American Urological Association (AUA), and the British Society for Sexual Medicine (BSSM) guidelines had >60% in >4 domains and that their average AGREE II scores were over 70%. In the Canadian Diabetic Association (CDA) and the Japanese Society for Sexual Medicine (JSSM) guidelines, we found that >4 domains were >60%, but their average AGREE II scores were below 70%. The British Medical Journal (BMJ), the Canadian Urologic Association (CUA), and the Malaysian Urologic Association (MUA) guidelines had >60% in <3 domains. We highly recommended EAU, AUA and BSSM guidelines, while we moderately recommended CDA and JSSM guidelines. BMJ, CUA and MUA guidelines were weakly recommended. The quality of the guidelines for erectile dysfunction was variable according to AGREE II. We noted significant deficiencies in the methodological quality of the CPGs developed by different organisations in the areas of applicability and rigour of development.

The Quality of Six Clinical Practice Guidelines in Health and Social Sciences: Are We on the Right Track?

Interest in the development and promotion of clinical practice guidelines (CPGs) continues to grow in many professions. However, the potential benefits associated with CPGs are dependent upon their quality. A number of studies have shown that the quality of CPGs varies greatly. Furthermore, the quality of many of the CPGs used in health and social sciences has yet to be examined. In light of this, the aim of this study was to examine the quality of CPGs that focus on intervention and care management in mental health in Quebec. A search of Quebec regulatory bodies websites was conducted and six CPGs were included in this study. The CPGs were assessed by four trained raters using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument. Items scores and domains scores were considered to determine the quality of the six CPGs. Results show that many of the CPGs did not achieve minimum ratings for numerous quality checks. Notably, none of the CPGs were designed using a rigorous methodology, they lacked transparency throughout the development process and insufficient consideration was given to the applicability of the recommendations they included. Because these shortcomings may hinder the efficacy and utilization of CPGs, suggestions to improve the development of CPGs and to improve their quality are discussed.

Assessing the quality of seven clinical practice guidelines from four professional regulatory bodies in Quebec: What's the verdict?

RATIONALE, AIMS, AND OBJECTIVES

Clinical practice guidelines (CPGs) have become a common feature in the health and social care fields, as they promote evidence-based practice and aim to improve quality of care and patient outcome. However, the benefits of the recommendations reported in CPGs are only as good as the quality of the CPGs themselves. Indeed, rigorous development and strategies for reporting are significant precursors to successful implementation of the recommendations that are proposed. Unfortunately, research has demonstrated that there is much variability in their level of quality. Furthermore, the quality of many CPGs has yet to be examined. The aim of the present study was to assess the quality of seven CPGs from four Quebec professional regulatory bodies pertaining to clinical evaluations in the fields of medicine, psychoeducation, psychotherapy, and social work.

METHODS

The seven Quebec CPGs were assessed by four trained appraisers using the Appraisal of Guidelines for Research and Evaluation II guideline evaluation tool.

RESULTS

Results suggest that while some quality criteria were met, most were not, denoting that these CPGs are of sub-optimal quality.

CONCLUSION

Our findings highlight that there is still a lot to be done in order to improve the rigour and transparency with which scientific evidence is assessed and applied when developing CPGs. Impacts regarding the implementation of these CPGs are discussed in light of their use in clinical practice.

Systematic review of global clinical practice guidelines for neonatal hyperbilirubinemia

OBJECTIVE

Hyperbilirubinemia is one of the most common clinical symptoms in newborns. To improve patient outcomes, evidence-based and implementable guidelines are required. However, clinical guidelines may vary in quality, criteria and recommendations among regions and countries. In this study, we aimed to systematically assess the quality of guidelines using the Appraisal of Guidelines for Research & Evaluation (AGREE)-II instrument and summarise the specific recommendations for neonatal hyperbilirubinemia in order to provide suggestions for future guideline development.

DESIGN

Systematic review.

INTERVENTIONS

We searched the PubMed, Embase, Medline and guideline databases for relevant articles on 10 April 2020. The studies were screened by two independent reviewers according to our inclusion criteria. Two reviewers independently extracted the descriptive data. Four appraisers assessed the guidelines using the AGREE-II instrument.

RESULTS

Our systematic review appraised 12 clinical practice guidelines for the diagnosis and management of neonatal hyperbilirubinemia. The 12 guidelines achieved an average score of 36%-89%. The guidelines received the highest scores for clarity of presentation and lowest scores for rigour of development. Most recommendations for diagnosis were relatively consistent, but recommendations regarding risk factors, the initiating threshold of treatment and pharmacotherapy varied.

CONCLUSIONS

Our study revealed that current guidelines vary in the quality of the developing process and are inconsistent with regards to recommendations. Future guidelines should afford more attention to the quality of methodologies in guideline development, and more qualified evidence is needed to standardise the initiating threshold of treatment for neonatal hyperbilirubinemia.

Appraisal of physical health guidelines for severe mental illness

Objectives The aim of this study was to identify current physical health guidelines for severe mental illness (SMI) and appraise them using the Appraisal of Guidelines for Research and Evaluation (AGREE) II Instrument. Methods Relevant research databases and grey literature were systematically searched to identify physical health guidelines for people with SMI. The grey literature was explored by web searches and targeted searches of the English websites of relevant peak bodies and government health agencies from Organisation for Economic Co-operation and Development countries. Included guidelines were independently appraised by two authors (NF and JH) using the AGREE II Instrument. Results Of the 5352 records screened, 33 were assessed for eligibility. Fifteen practice guidelines met the inclusion criteria. The median domain scores as rated by the AGREE II Instrument ranged from 17% to 69%. The World Health Organization guideline, which demonstrated a broad range of clinical recommendations and sound methodological rigour, was rated the best. Conclusions Most guidelines scored poorly when rated by the AGREE II Instrument. However, these guidelines may still be useful in assisting evidence-based clinical practice. The methodological rigour of future guidelines can be improved by ensuring the AGREE II domains are addressed during the development phase. What is known about the topic? Compared with the general population, people with SMI experience greater chronic disease morbidity and mortality. There is limited evidence from randomised controlled trials to guide physical health care monitoring for people with SMI. Current guidelines and practice are largely based on expert consensus, clinical experience and good intentions. What does this paper add? Using the AGREE II Instrument, this paper appraises the current physical health guidelines for people with SMI. The attributes of the guidelines examined included the evidence base, clarity of presentation, applicability in the real world, the involvement of stakeholders and conflicts of interest of various parties involved in guideline development. What are the implications for practitioners? This review highlights the scarcity of high-quality and evidence-based guidelines for clinicians and researchers to address the physical health of people with SMI.

Protocol of an interdisciplinary consensus project aiming to develop an AGREE II extension for guidelines in surgery

INTRODUCTION

Appraisal of Guidelines for Research and Evaluation (AGREE II) is an instrument that informs development, reporting and assessment of clinical practice guidelines. Previous research has demonstrated the need for improvement in methodological and reporting quality of clinical practice guidelines specifically in surgery. We aimed to develop an AGREE II extension document for application in surgical guidelines.

METHODS AND ANALYSIS

We have performed a structured literature review and assessment of guidelines in surgery using the AGREE II instrument. In exploratory analyses, we have identified factors associated with guideline quality. We have performed reliability and factor analyses to inform the development of an extension document. We will summarise this information and present it to a Delphi panel of stakeholders. We will perform iterative Delphi rounds and we will summarise the final results to develop the extension instrument in a dedicated consensus conference.

ETHICS AND DISSEMINATION

Funding bodies will not be involved in the development of the instrument. Research ethics committee and Health Research Authority approval was waived, since this is a professional staff study only and no duty of care lies with the National Health Service to any of the participants. Conflicts of interest, if any, will be addressed by reassigning functions or replacing participants with relevant conflicts. The results will be disseminated through publication in peer reviewed journals, the funders' websites, social media and direct contact with guideline development organisations and peer-reviewed journals that publish guidelines.

A Methodological Assessment of Diabetic Foot Syndrome Clinical Practice Guidelines

OBJECTIVE

Diabetic foot syndrome (DFS) contributes to significant morbidity in diabetic patients. Diagnostic and therapeutic approaches to DFS may be summarised in clinical practice guidelines (CPGs) to aid clinical practice but may only benefit patients if the CPG is of high quality. This study determines the methodological quality of DFS CPGs using a validated assessment tool to identify CPGs adequate for use in clinical practice.

METHODS

Medline, EMBASE, and CPG databases were searched to 31 May 2019. Reference lists were also searched. Full text English evidence based DFS CPGs were included. CPGs based on expert consensus, guideline summaries, or those only available if purchased were excluded. Four reviewers independently assessed methodological quality using the Appraisal of Guidelines for Research and Evaluation II instrument. An overall guideline assessment scaled score of ≥80% was considered to be of adequate quality to recommend use.

RESULTS

Sixteen CPGs were identified. Good inter-reviewer reliability (ICC 0.985, 95% CI 0.980-0.989) was achieved. Poor scores were noted in domains 2 (stakeholder involvement), 5 (applicability), and 6 (editorial independence). Significant methodological heterogeneity was observed in all domains with the most noted in domain 6 (mean scaled score 43.2 ± 32.1%). Four CPGs achieved overall assessment scores of ≥80%.

CONCLUSION

Four CPGs were considered to be adequate for clinical practice based on methodological quality. However, elements of methodological quality were still lacking, and all CPGs had areas for improvement, potentially through increased multidisciplinary team involvement and trial application of recommendations. Methodological rigour may be improved using structured approaches with validated CPG creation tools in the future. Future work should also assess recommendation accuracy using available validated assessment tools.

A methodologic assessment of lymphedema clinical practice guidelines

OBJECTIVE

The objective of this review was to determine the methodologic quality of current lymphedema clinical practice guidelines (CPGs) to assist health care professionals in selecting accessible, high-quality guidance and to identify areas for improvement in future CPGs.

METHODS

MEDLINE, Embase, online CPG databases, and reference lists of included guidelines were searched up to January 31, 2020. Full-text CPGs reporting on evidence-based recommendations in lymphedema diagnosis or management in English were included. CPGs based on expert consensus, CPG summaries, or CPGs that were not freely available were excluded. Two reviewers identified eligible CPGs, extracted data, and assessed their quality independently using the Appraisal of Guidelines for Research and Evaluation II instrument. Significant scoring discrepancies were discussed with a third reviewer. An overall scaled quality score of ≥80% was the threshold to recommend guideline use.

RESULTS

Six relevant CPGs were identified. One was subsequently excluded as its full text could not be obtained. Overall, there was very good inter-reviewer reliability of scores with intraclass correlation coefficient of 0.952 (95% confidence interval, 0.921-0.974). No single CPG scored highest in all domains, with methodologic heterogeneity observed. Poor performance was noted in domain 5 (mean scaled score, 23.8% ± 17.1%) and domain 6 (22.9% ± 26.7%). No CPG achieved an overall scaled quality score of ≥80%, with the top CPG scoring 79.2%.

CONCLUSIONS

According to the defined threshold, no lymphedema CPG was considered adequate for use in clinical practice. All current lymphedema CPGs have areas for improvement with elements of methodologic quality lacking, particularly with respect to rigor of development. A structured approach, guided by the use of CPG creation tools and checklists such as the Appraisal of Guidelines for Research and Evaluation II instrument, should help CPG development groups in improving the quality of future CPGs. This is of particular importance in a complex, multidisciplinary condition such as lymphedema.

Critical appraisal of papers reporting recommendation on sarcopenia using the AGREE II tool: a EuroAIM initiative

BACKGROUND/OBJECTIVES

The growing interest of medical community about sarcopenia resulted in the production of several clinical practice guidelines (CPGs), with an unavoidable variability in terms of the overall quality of those publications. Our aim is to evaluate the quality of CPGs on sarcopenia using the AGREE II instrument.

SUBJECTS/METHODS

We performed an online literature search for sarcopenia CPGs using different databases. Four independent reviewers evaluated the quality of CPGs using the AGREE II instrument. To classify the quality of each guideline, we defined specific thresholds of final score: high-quality if five or more domains scored >60%; average-quality if three or four domains scored >60%; low-quality if ≤2 domains scored >60%.

RESULTS

Our literature search yielded 315 articles, and after applying exclusion criteria our final analysis included 19 CPGs. The overall quality of CPGs was remarkable, as 13/19 (68.4%) were considered of "high-quality" CPGs, with more than four domains reached a score higher than 60%. "Scope and Purpose" and "Clarity of Presentations" had the best domain results (78.4% and 73.8%, respectively), while the two domains with the lowest scores were "Rigor of Development" and "Applicability" (61.5% and 58.7%, respectively). Interobserver variability ranged between moderate (0.624) and fair (0.275).

CONCLUSIONS

Our study showed that the overall quality of CPGs about sarcopenia was noteworthy, as more than two-third of paper obtained a "high-quality" score. The domain "applicability" had the lowest score, suggesting that emphasis should be put on possible strategies for helping other doctors to implement guideline recommendations in clinical practice.

Adhering to Radiographic Clinical Practice Guidelines for Distal Radial Fracture Management Is Associated with Improved Outcomes and Lower Costs

BACKGROUND

The American Academy of Orthopaedic Surgeons Clinical Practice Guideline on the Treatment of Distal Radius Fractures has not been evaluated in clinical practice. We hypothesized that adhering to the distal radial fracture radiographic clinical practice guideline (CPG) improves outcomes and reduces costs.

METHODS

We reviewed 266 patients with distal radial fractures treated at 1 institution. Based on CPG radiographic parameters (Recommendation 3), care was rated as "appropriate" or "inappropriate." QuickDASH (an abbreviated version of the Disabilities of the Arm, Shoulder and Hand [DASH] questionnaire) scores were collected. The direct costs of distal radial fracture care were determined. Descriptive statistics and nonparametric tests were used to evaluate demographic characteristics and outcomes across groups. QuickDASH scores, grouped by postoperative time interval, were analyzed using linear mixed effect models to predict outcome trends.

RESULTS

In this study, 145 patients in the operative treatment group and 121 patients in the nonoperative treatment group were included. Of the 145 patients in the operative treatment group, 6 underwent an inappropriate surgical procedure, limiting any analyses of that group. Of the 121 patients in the nonoperative treatment group, 68 were treated inappropriately. For the patients in the nonoperative treatment group, appropriate care provided a significant outcome benefit by 1 year; the median QuickDASH score was 10.1 points for the appropriate treatment group and 19.5 points for the inappropriate treatment group (p = 0.05). The total direct costs for inappropriate nonoperative treatment were, on average, 60% higher than appropriate nonoperative treatment. In predictive models, patients with appropriate care in the operative treatment group and the nonoperative treatment group had better outcomes than patients with inappropriate nonoperative treatment at all time points after 29 days.

CONCLUSIONS

When nonoperative distal radial fracture management was aligned with radiographic CPG criteria, patients in our cohort had improved patient-reported outcomes with lower costs.

LEVEL OF EVIDENCE

Therapeutic Level III. See Instructions for Authors for a complete description of levels of evidence.

Assessing the methodological quality of the Canadian Psychiatric Association's anxiety and depression clinical practice guidelines

RATIONALE, AIMS, AND OBJECTIVES

Clinical practice guidelines (CPGs) endeavour to incorporate the best available research evidence together with the clinically informed opinions of leading experts in order to guide clinical practice when dealing with a given condition. There has been increased interest in CPGs that are evidence based and that promote best practice, a central component of which is incorporating the best available research predicated on strong study designs. Despite this soaring interest, there remains heterogeneity in the methodological quality of many CPGs, which may have an effect on the quality of services that clinicians offer. In light of this, this study examined the quality of the methodology used to develop two CPGs of the Canadian Psychiatric Association (CPA).

METHOD

The CPA's guidelines for the management of anxiety disorders (2006) and for the treatment of depressive disorders (2001) were assessed by trained raters using the Appraisal of Guidelines for Research and Evaluation II Instrument scale.

RESULTS

The blind ratings of three trained raters demonstrated that the anxiety and depression CPGs had a number of strengths and important weaknesses.

CONCLUSION

Implications for the development of future CPGs on anxiety and depression, including recommendations to improve guideline quality in psychiatry in particular, are discussed.

Examining the implementation of clinical practice guidelines for the management of adult cancers: A mixed methods study

RATIONALE, AIMS, AND OBJECTIVES

Focusing on the implementation of clinical practice guidelines (CPGs) for the management of adult cancers, the objectives of this study were to (a) describe the intrinsic elements known to influence CPG use; (b) identify the ways in which CPGs are implemented; and (c) explore how CPG characteristics and contextual factors influence implementation and use.

METHODS

We conducted a sequential mixed methods study. First, we performed a content analysis of all CPGs developed and approved for the management of adult cancers in Nova Scotia, Canada, from 2005 to 2015. CPGs were examined for the presence of 22 elements known to influence CPG use. Next, we conducted semistructured interviews with CPG developers and end users. Participants were purposively sampled, based on the findings of the content analysis. All interviews were audiotaped and transcribed verbatim. Data were analysed by two researchers using the Framework Method.

RESULTS

CPGs (n = 20) demonstrated large variation with respect to elements shown to influence CPG use. For example, 85% included content related to individualization and objectives. Yet no CPGs (0%) had journal or patient versions; discussed the education, training, or competencies needed to deliver recommendations; contained an explicit statement on anticipated work changes, or on potential direct or productivity costs; or identified barriers or facilitators that might influence CPG adoption. Interview data from CPG developers (n = 4) and users (n = 6) revealed five themes related to CPG implementation and use: (a) lack of consistency in CPG development; (b) timing and nature of stakeholder engagement; (c) credibility of the CPG development process and final CPGs; (d) limited understanding of implementation as an active process; and (e) factors at organizational and system levels influence CPG implementation and use.

CONCLUSIONS

This mixed methods study provides complementary data that may help inform more effective CPG implementation efforts and optimize their use in practice.

Quality and utilization of the Finnish clinical practice guideline in schizophrenia: evaluation using AGREE II and the vignette approach

PURPOSE

This study aims to evaluate the quality of the Finnish clinical practice guideline related to schizophrenia care and how it can be utilized in psychiatric services.

PARTICIPANTS AND METHODS

The data were collected from one psychiatric ward and seven psychiatric outpatient units situated in two cities in Southern Finland. A total of 49 professionals working in these sites participated in this study. A descriptive study design was adopted. The Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument with six domains including 23 items was used to assess the quality of the guideline, whereas a vignette was implemented to describe how staff would use the guideline in hypothetical cases in daily practice. The analysis of AGREE II was based on rating each of the 23 items on a seven-point scale and calculating a quality score for the six domains. To describe the utilization of the guideline, eight key recommendations of the guideline were deductively rated out of the respondents' answers.

RESULTS

The results showed that the "scope and purpose" of the guideline were well described, but "applicability" was insufficient. The overall quality of the guideline was high (73%). Almost one fifth of the respondents were in agreement with key recommendations.

CONCLUSION

The overall quality of Guideline for Schizophrenia was good, but its "applicability" and utilization should be improved.

Appraising of the Clinical Practice Guidelines Quality in the Non-Pharmacological Management of Chemotherapy-Induced Febrile Neutropenia; A Review

Background:

Febrile neutropenia is a common and serious chemotherapy side effect, is associated with increased mortality, morbidity, and treatment expenditures. Several CPGs (Clinical practice guidelines) have been released for managing chemotherapy-induced febrile neutropenia. The aim of this study is Appraisal of the clinical practice Guidelines quality in the management of chemotherapy-induced febrile neutropenia.

Methods:

A review study with a systematic search of the present CPGs for the management of chemotherapy-induced febrile neutropenia. After screening the CPGs based on eligibility criteria, three CPGs were selected and 5 independent reviewers appraised them for methodological quality by using the AGREE II Instrument.

Results:

Three CPGs were included; all of them were evidence-based guidelines. The clarity of presentation domain scored the highest and the applicability domain has the lowest score among all domains of AGREE instrument and the rest of domains scored as descending respectively; Scope and purpose, stakeholder involvement, editorial independence, rigor of development. In general, the intra-class correlation coefficient (ICC) scores of all domains were very good according to the Landis and Koch’s scale, except the Applicability domain scored as substantial.

Conclusions:

This study showed the quality of appraised CPGs. Three domains of these CPGs based on the AGREE instrument scored less than other domains and were in relatively unfavorable status: applicability, rigor of development, editorial independence. Given the importance of these domains in guideline implementation, it is necessary to take actions for reducing these defects.

Evaluation of Noninvasive Prenatal Testing (NIPT) guidelines using the AGREE II instrument

Objective: The rapid increase of cell-free fetal DNA analysis for Down syndrome screening requires evidence-based clinical practice guidelines for noninvasive prenatal testing (NIPT). Several studies show that the quality of many guidelines is low and there are still many health areas where this quality is not systematically evaluated. Given the absence of research, in the NIPT field, we used an internationally validated tool to evaluate a set of three NIPT practice guidelines and to look at dimensions that can be improved.Methods: Four appraisers, experts in prenatal screening, evaluated three main NIPT guidelines published in the last 2 years using the AGREE II (Appraisal of Guidelines for Research and Evaluation II), a tool specifically designed for guideline quality appraisal.Results: Guidelines scored higher in domains related with scope, purpose, and clarity of presentation, and lower in stakeholder involvement and rigor of development. Intradomain items evaluation showed asymmetries between guidelines. The UK-NSC was the guideline with the best scores.Discussion: Several areas of NIPT guidelines, such as stakeholders involvement, selection of supporting evidence, external reviews, updating processes, and competing interests disclosure, can be improved. Appraisers recommend modifications to all NIPT guidelines that can lead to substantial improvements in their methodological quality and subsequently make a contribution to prenatal screening improvement.

The quality of guidelines in non-pharmacological prevention and management of chemotherapy-induced neuropathy: A review

Chemotherapy-induced peripheral neuropathy is a common adverse effect occurring in patients undergoing neurotoxic chemotherapy. However, there is no FDA-approved treatment option for it. Given the importance of clinical practice guidelines in this area, this study aimed to determine the methodological quality of extant CIPN guidelines. The study was done as part of the adaptation process of CIPN related CPGs at Isfahan University of Medical Sciences, Iran. A systematic search of published CPGs about chemotherapy-induced CIPN in which the AGREE II instrument was applied for appraising CPGs of CIPN was performed. In general, amongst all of the AGREE II Instrument’s domains in the evaluated CPGs, the clarity of presentation and stakeholder involvement domains took favorable scores; and other domains obtained unfavorable and relatively favorable scores. The quality of cancer therapy-induced neuropathy CPGs needs to be improved and designing high-quality CPGs must be considered.

Quality assessment of clinical practice guidelines for infectious diseases in China

AIM

We aimed to appraise the quality of Chinese Society of Infectious Diseases of China Medical Association clinical practice guidelines (CPGs) for infectious diseases in China.

METHODS

We performed the electronic search and hand search on the above-mentioned CPGs for infectious diseases published from January 1997 to October 2017. Each publication was evaluated using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument.

RESULTS

Thirteen guidelines were identified and enrolled for the further evaluation. The median scores of the AGREE II domains across all 13 guidelines were as follows: scope and purpose 73.61% (36.11% to 84.72%), stakeholder involvement 12.50% (8.33% to 20.83%), rigor of development 19.27% (12.50% to 26.04%), clarity of presentation 86.11% (23.61% to 94.44%), applicability 33.33% (26.04% to 50.00%), and editorial independence 8.33% (4.17% to 16.67%). The clarity of presentation domain got the highest score among the six domains. Moreover, when compared the CPGs by the year of publication (2002 to 2006 vs. 2007 to 2017), the quality scores were higher in the CPGs published in the last 10 years than those published before, but with no significant difference.

CONCLUSIONS

The quality score of CPGs for infectious diseases in China from January 1997 to October 2017 was moderate, but had significant shortcomings, especially in the stakeholder involvement and editorial independence domains. However, the AGREE II instrument is useful and feasible for appraising the CPGs and should be applied to develop, implement, and promote the CPGs in China.

Framework of care: communicating the structure and processes of care

AIM

This article attempts to present a 'macro view' of the role and nature of an organization's Framework of Care (FrOC). This 'view' arises from a critical reflection on the available literature and the combined professional experience of the authors, who have worked in a variety of healthcare systems and settings in Australia, North America, United Kingdom, and the Middle East.

BACKGROUND

FrOC can be defined as the systems and processes within an organization that structure the delivery of care. These systems and processes are made evident in a series of documents, such as the Mission and Vision statement, Policies and Procedures, Standards of Care, Clinical Practice Guidelines, Clinical Pathways, and Protocols. These frameworks can provide structure for important organizational activities such as clinical audits, quality management and clinical information system (CIS) 'decision support', thereby supporting clinicians in their efforts to deliver high-quality, evidence-based care. How a healthcare organization structures its systems and processes of care directly impacts the patient and caregiver experience - made evident in patient and staff satisfaction with the services provided.

RECOMMENDATIONS

Mapping out and understanding an organization's FrOC is a critical first step for interprofessional teams attempting to implement evidence into practice and/or accreditation teams and expert consultants critiquing the performance of an organization.

Guideline appraisal with AGREE II: online survey of the potential influence of AGREE II items on overall assessment of guideline quality and recommendation for use

BACKGROUND

The AGREE II instrument is the most commonly used guideline appraisal tool. It includes 23 appraisal criteria (items) organized within six domains. AGREE II also includes two overall assessments (overall guideline quality, recommendation for use). Our aim was to investigate how strongly the 23 AGREE II items influence the two overall assessments.

METHODS

An online survey of authors of publications on guideline appraisals with AGREE II and guideline users from a German scientific network was conducted between 10th February 2015 and 30th March 2015. Participants were asked to rate the influence of the AGREE II items on a Likert scale (0 = no influence to 5 = very strong influence). The frequencies of responses and their dispersion were presented descriptively.

RESULTS

Fifty-eight of the 376 persons contacted (15.4%) participated in the survey and the data of the 51 respondents with prior knowledge of AGREE II were analysed. Items 7-12 of Domain 3 (rigour of development) and both items of Domain 6 (editorial independence) had the strongest influence on the two overall assessments. In addition, Items 15-17 (clarity of presentation) had a strong influence on the recommendation for use. Great variations were shown for the other items. The main limitation of the survey is the low response rate.

CONCLUSIONS

In guideline appraisals using AGREE II, items representing rigour of guideline development and editorial independence seem to have the strongest influence on the two overall assessments. In order to ensure a transparent approach to reaching the overall assessments, we suggest the inclusion of a recommendation in the AGREE II user manual on how to consider item and domain scores. For instance, the manual could include an a-priori weighting of those items and domains that should have the strongest influence on the two overall assessments. The relevance of these assessments within AGREE II could thereby be further specified.

An appraisal of practice guidelines for smoking cessation in people with severe mental illness

OBJECTIVE

To review the quality of current smoking cessation guidelines that include recommendations for people with severe mental illness.

METHODS

A systematic search of scientific databases, central government health authority websites, psychiatry peak bodies, guideline clearing houses and Google was undertaken for relevant smoking cessation guidelines. Three reviewers independently assessed guideline quality using the AGREE II (Appraisal of Guidelines for REsearch and Evaluation II) instrument. Two reviewers extracted recommendations specific to smokers with severe mental illness.

RESULTS

Thirteen guidelines met the inclusion criteria. Seven guidelines scored ⩾60% in at least four domains. Median scores for 'Editorial independence', 'Rigour of development', 'Stakeholder Involvement' and 'Applicability' were less than 60%. The highest median scores were for 'Scope and purpose' (87%, 69-96%) and 'Clarity of presentation' (87%, 56-98%). 'Editorial independence' (33.3%, 0-86%) and 'Rigour of development' (54%, 11-92%) had the lowest median domain scores. The guidelines varied greatly in their recommendations but the majority recommended nicotine replacement therapy, bupropion or varenicline as first-line pharmacotherapy, along with behavioural support.

CONCLUSION

Many guidelines did not adequately report their methods or the competing interests of the authors. Future guidelines development may benefit from more specifically addressing AGREE II criteria and the needs of smokers with severe mental illness.

Systematic review of clinical practice guidelines in kidney transplantation

BACKGROUND

Clinical practice guidelines (CPGs) are widely used to inform the development of protocols for clinical management. Previous work has demonstrated that the quality of CPGs varies widely. This systematic review aimed to determine the quality of CPGs in kidney transplantation in the UK.

METHODS

CPGs in kidney transplantation published between 2010 and 2017 were identified through searches of MEDLINE, NHS NICE Evidence, and websites of relevant UK societies. Using the Appraisal of Guidelines for Research and Evaluation (AGREE) II tool, three appraisers rated the quality of CPGs across six domains, the overall quality of each CPG, and whether it should be recommended for future use. Domain scores were calculated, and inter-rater reliability using the intraclass correlation coefficient (ICC) was reported.

RESULTS

Thirteen CPGs met the inclusion criteria. The domain 'clarity of presentation' scored highest, followed closely by 'scope and purpose'. The poorest scoring domains were 'applicability' and 'editorial independence'. Editorial independence also had the widest range of scores. Of the 13 CPGs, one was not recommended for future use, seven were recommended for use with modifications, and five for future use with no need for modification. Mean overall CPG quality was 5 (range 3-6) of a maximum score of 7, and mean inter-rater reliability was substantial with an ICC of 0·71.

CONCLUSION

UK CPGs scored satisfactorily, although with wide variation in how well each domain scored both within and across CPGs. The quality of UK CPGs can still be improved.

A Systematic Review and Appraisal of Clinical Practice Guidelines for Musculoskeletal Soft Tissue Injuries and Conditions

BACKGROUND

Clinical practice guidelines (CPGs) are published by several sports medicine institutions. A systematic evaluation can help identify the highest quality CPGs for clinical use and identify any deficiencies that remain.

PURPOSE

To identify and appraise CPGs relevant to clinical sports medicine professionals.

STUDY DESIGN

Systematic review.

METHODS

Predetermined selection criteria were utilized by 2 reviewers who independently identified published CPGs before January 1, 2014. CPGs were excluded if they focused on injured workers, radiological criteria, medical pathology, or the axial skeleton (back/neck). The remaining guidelines were scored by 6 reviewers with different clinical backgrounds using the Appraisal of Guidelines for Research and Evaluation II (AGREE II). Scores lower than 50% indicated deficiency. Scores were also stratified by the publishing institution and anatomic location and compared using Kruskal-Wallis tests. The Spearman correlation coefficient was used to assess the range of interobserver agreement between the evaluators.

RESULTS

Seventeen CPGs met the inclusion criteria. The majority of guidelines pertained to the knee, ankle, or shoulder. Interobserver agreement was strong ( r = 0.548-0.740), and mean total scores between nonsurgical (107.8) and surgical evaluators (109.3) were not statistically different. Overall guideline quality was variable but not deficient for 16 of 17 guidelines (>50%), except regarding clinical "applicability" and "editorial independence." No difference was found between CPGs of the knee, shoulder, foot/ankle, or chronic conditions. However, CPG publishing institutions had significantly different scores; the American Academy of Orthopaedic Surgeons (AAOS) guidelines scored significantly higher (141.4) than the total mean score (108.0).

CONCLUSION

The overall quality of sports medicine CPGs was variable but generally not deficient, except regarding applicability and editorial independence. Bias through poor editorial independence is a concern. To improve future guideline quality, authors should pay particular attention to these areas and use existing highest quality guidelines, or the AGREE II instrument, as templates. CPGs dedicated to anatomic areas other than the knee, ankle, and shoulder are needed.

Guideline appraisal with AGREE II: Systematic review of the current evidence on how users handle the 2 overall assessments

INTRODUCTION

The Appraisal of Guidelines for Research & Evaluation (AGREE) II instrument is the most commonly used guideline appraisal tool. It includes 23 appraisal criteria (items) organized within 6 domains and 2 overall assessments (1. overall guideline quality; 2. recommendation for use). The aim of this systematic review was twofold. Firstly, to investigate how often AGREE II users conduct the 2 overall assessments. Secondly, to investigate the influence of the 6 domain scores on each of the 2 overall assessments.

MATERIALS AND METHODS

A systematic bibliographic search was conducted for publications reporting guideline appraisals with AGREE II. The impact of the 6 domain scores on the overall assessment of guideline quality was examined using a multiple linear regression model. Their impact on the recommendation for use (possible answers: "yes", "yes, with modifications", "no") was examined using a multinomial regression model.

RESULTS

118 relevant publications including 1453 guidelines were identified. 77.1% of the publications reported results for at least one overall assessment, but only 32.2% reported results for both overall assessments. The results of the regression analyses showed a statistically significant influence of all domains on overall guideline quality, with Domain 3 (rigour of development) having the strongest influence. For the recommendation for use, the results showed a significant influence of Domains 3 to 5 ("yes" vs. "no") and Domains 3 and 5 ("yes, with modifications" vs. "no").

CONCLUSIONS

The 2 overall assessments of AGREE II are underreported by guideline assessors. Domains 3 and 5 have the strongest influence on the results of the 2 overall assessments, while the other domains have a varying influence. Within a normative approach, our findings could be used as guidance for weighting individual domains in AGREE II to make the overall assessments more objective. Alternatively, a stronger content analysis of the individual domains could clarify their importance in terms of guideline quality. Moreover, AGREE II should require users to transparently present how they conducted the assessments.

Improving the adaptability of WHO evidence-informed guidelines for nutrition actions: results of a mixed methods evaluation

BACKGROUND

Over the past decade, the World Health Organization (WHO) has implemented a standardized, evidence-informed guideline development process to assure technically sound and policy-relevant guidelines. This study is an independent evaluation of the adaptability of the guidelines produced by the Evidence and Programme Guidance unit, at the Department of Nutrition for Health and Development (NHD). The study systematizes the lessons learned by the NHD group at WHO.

METHODS

We used a mixed methods approach to determine the adaptability of the nutrition guidelines. Adaptability was defined as having two components; methodological quality and implementability of guidelines. Additionally, we gathered recommendations to improve future guideline development in nutrition actions for health and development. Data sources for this evaluation were official documentation and feedback (both qualitative and quantitative) from key stakeholders involved in the development of nutrition guidelines. The qualitative data was collected through a desk review and two waves of semi-structured interviews (n = 12) and was analyzed through axial coding. Guideline adaptability was assessed quantitatively using two standardized instruments completed by key stakeholders. The Appraisal Guideline for Research and Evaluation questionnaire, version II was used to assess guideline quality (n = 6), while implementability was assessed with the electronic version of the GuideLine Implementability Appraisal (n = 7).

RESULTS

The nutrition evidence-informed guideline development process has several strengths, among them are the appropriate management of conflicts of interest of guideline developers and the systematic use of high-quality evidence to inform the recommendations. These features contribute to increase the methodological quality of the guidelines. The key areas for improvement are the limited implementability of the recommendations, the lack of explicit and precise implementation advice in the guidelines and challenges related to collaborative work within interdisciplinary groups.

CONCLUSIONS

Overall, our study found that the nutrition evidence-informed guidelines are of good methodological quality but that the implementability requires improvement. The recommendations to improve guideline adaptability address the guideline content, the dynamics shaping interdisciplinary work, and actions for implementation feasibility. As WHO relies heavily on a standardized procedure to develop guidelines, the lessons learned may be applicable to guideline development across the organization and to other groups developing guidelines.

High Reliability of Care in Orthopedic Surgery: Are We There Yet?

As health care reimbursement models shift from volume-based to value-based models, orthopedic surgeons must provide patients with highly reliable care, while consciously minimizing cost, maintaining quality, and providing timely interventions. An established means of achieving these goals is by implementing a highly reliable care model; however, before such a model can be initiated, a safety culture, robust improvement strategies, and committed leadership are needed. This article discusses interdependent and critical changes required to implement a highly reliable care system. Specific operative protocols now mandated are discussed as they pertain to high reliability of orthopedic care and elimination of wrong-site procedures.

Guidelines on vitamin D replacement in bariatric surgery: Identification and systematic appraisal

INTRODUCTION

Bariatric surgery is the most effective therapeutic option to reduce weight in morbidly obese individuals, but it results in a number of mineral and vitamin deficiencies. Clinical Practice Guidelines (CPGs) attempt to balance those benefits and harms to provide guidance to physicians and patients.

OBJECTIVES

We compare and evaluate the quality of the evidence and of the development process of current CPGs that provide recommendations on vitamin D replacement in patients undergoing bariatric surgery, using a validated tool.

METHODS

We searched 4 databases, with no time restriction, to identify relevant and current CPGs. Two reviewers assessed eligibility and abstracted data, in duplicate. They evaluated the quality of CPGs development process using the Appraisal of Guidelines, Research, and Evaluation II (AGREE II) tool that consists of 6 domains. A content expert verified those assessments.

RESULTS

We identified 3 eligible CPGs: (1) the Endocrine Society (ES) guidelines (2010); (2) the American Association of Clinical Endocrinologists (AACE), The Obesity Society (TOS), and the American Society for Metabolic & Bariatric Surgery (ASMBS) guidelines (update 2013); and (3) the Interdisciplinary European (IE) guidelines on Metabolic and Bariatric Surgery (latest update 2014). The ES and the AACE/TOS/ASMBS guidelines recommended high doses of vitamin D, varying from 3000IU daily to 50,000IU 1-3 times weekly. Vitamin D doses were not mentioned in the IE guidelines. The recommendations were based on a low quality of evidence, if any, or limited to a single high quality trial, for some outcomes. In terms of quality, only the IE guidelines described their search methodology but none of the CPGs provided details on evidence selection and appraisal. None of the three CPGs rigorously assessed the preferences of the target population, resource implications, and the applicability of these guidelines. According to the AGREE II tool, we rated the ES guidelines as average in quality, and the other two as low in quality.

CONCLUSION

Current CPGs recommendations on vitamin D supplementation in bariatric surgery differ between societies. They do not fulfill criteria for optimal guideline development, in part possibly due to limited resources, and are based on expert opinion. Thus, the pressing need for high quality randomized trials to inform CPGs, to be developed based on recommended standards.