Appraisal of guidelines developed by the World Health Organization

"Going into the black box": a policy analysis of how the World Health Organization uses evidence to inform guideline recommendations

Background

The World Health Organization (WHO) plays a crucial role in producing global guidelines. In response to previous criticism, WHO has made efforts to enhance the process of guideline development, aiming for greater systematicity and transparency. However, it remains unclear whether these changes have effectively addressed these earlier critiques. This paper examines the policy process employed by WHO to inform guideline recommendations, using the update of the WHO Consolidated HIV Testing Services (HTS) Guidelines as a case study.

Methods

We observed guideline development meetings and conducted semi-structured interviews with key participants involved in the WHO guideline-making process. The interviews were recorded, transcribed, and analysed thematically. The data were deductively coded and analysed in line with the main themes from a published conceptual framework for context-based evidence-based decision making: introduction, interpretation, and application of evidence.

Results

The HTS guideline update was characterized by an inclusive and transparent process, involving a wide range of stakeholders. However, it was noted that not all stakeholders could participate equally due to gaps in training and preparation, particularly regarding the complexity of the Grading Recommendations Assessment Development Evaluation (GRADE) framework. We also found that WHO does not set priorities for which or how many guidelines should be produced each year and does not systematically evaluate the implementation of their recommendations. Our interviews revealed disconnects in the evidence synthesis process, starting from the development of systematic review protocols. While GRADE prioritizes evidence from RCTs, the Guideline Development Group (GDG) heavily emphasized "other" GRADE domains for which little or no evidence was available from the systematic reviews. As a result, expert judgements and opinions played a role in making recommendations. Finally, the role of donors and their presence as observers during GDG meetings was not clearly defined.

Conclusion

We found a need for a different approach to evidence synthesis due to the diverse range of global guidelines produced by WHO. Ideally, the evidence synthesis should be broad enough to capture evidence from different types of studies for all domains in the GRADE framework. Greater structure is required in formulating GDGs and clarifying the role of donors through the process.

Evidence uptake is only part of the process: Stakeholders' insights on WHO treatment guideline recommendation processes for radical cure of P. vivax malaria

Health policy processes should be evidence-informed, transparent and timely, but these processes are often unclear to stakeholders outside the immediate policymaking environment. We spoke to 36 international malaria stakeholders to gain insights on the processes involved in the World Health Organization's Global Malaria Programme's recommendations for their treatment guidelines of P. vivax malaria. Four key themes which drew on the 3i policy framework and Shiffman's four factors that influence global and national policymaking were identified to understand these processes. Triggers for policy change and change prioritisation, evidence types that inform policy, effects of funding on decision-making processes, and transparency and communication of these processes to external stakeholders. Results indicate that more clarity is needed on what triggers global malaria policy change processes, a clearer justification of evidence types used to inform policymaking, better understanding of the impact of the WHO's funding model on policymaking and further transparency and improved communication of these processes to external stakeholders is also needed. We suggest that global malaria policymaking could be improved by using the following strategies: ensuring that identified triggers actually initiate the policy change process, expediting decision-making timelines by developing a priority framework for assessing new evidence, adopting suitable frameworks to assess contextual evidence, and increasing the transparency of the role of non-state funders in policy decision-making processes and when publishing new recommendations.

Assessing the quality and completeness of reporting in health systems guidance for pandemics using the AGREE-HS tool

Background

During health emergencies, leading healthcare organisations, such as the World Health Organization (WHO), the European Centre for Disease Control and Prevention (ECDC), and the United States Centers for Disease Control and Prevention (CDC), provide guidance for public health response. Previous studies have evaluated clinical practice guidelines (CPGs) produced in response to epidemics or pandemics, yet few have focused on public health guidelines and recommendations. To address this gap, we assessed health systems guidance (HSG) produced by the WHO, the ECDC, and the CDC for the 2009 H1N1 and COVID-19 pandemics.

Methods

We extracted HSG for the H1N1 and COVID-19 pandemics from the organisations' dedicated repositories and websites. After screening the retrieved documents for eligibility, five assessors evaluated them using the Appraisal of Guidelines Research & Evaluation - Health Systems (AGREE-HS) tool to assess the completeness and transparency of reporting according to the five AGREE-HS domains: "Topic", "Participants", "Methods", "Recommendations", and "Implementability".

Results

Following the screening process, we included 108 HSG in the analysis. We observed statistically significant differences between the H1N1 and COVID-19 pandemics, with HSG issued during COVID-19 receiving higher AGREE-HS scores. The HSG produced by the CDC had significantly lower overall scores and single-domain scores compared to the WHO and ECDC. However, all HSG scored relatively low, under the median of 40 total points (range = 10-70), indicating incomplete reporting. The HSG produced by all three organisations received a median score <4 (range = 1-7) for the "Participants", "Methods", and "Implementability" domains.

Conclusions

There is still significant progress to be made in the quality and completeness of reporting in HSG issued during pandemics, especially regarding methodological approaches and the composition of the guidance development team. Due to their significant impact and importance for healthcare systems globally, HSG issued during future healthcare crises should adhere to best reporting practices to increase uptake by stakeholders and ensure public trust in healthcare organisations.

Assessing the World Health Organization: What does the academic debate reveal and is it democratic?

The World Health Organization (WHO), the leading global authority in public health, routinely attracts loud calls for reform. Although Member States negotiate reform internally, academic debate is more public, and can generate ideas and provide independent accountability. We investigate why authors advocate for WHO reform so commonly. We wondered if this literature had potentially useful themes for WHO, what methods and evidence were used, and we wanted to analyze the geography of participation. We conducted a systematic review using four databases to identify 139 articles assessing WHO or advocating for reform. We discuss these using categories we derived from the management literature on organizational performance. We also analyzed evidence, country of origin, and topic. The literature we reviewed contained 998 claims about WHO's performance or reform, although there were no standard methods for assessing WHO. We developed a framework to analyze WHO's performance and structure a synthesis of the claims, which find WHO imperiled. Its legitimacy and governance are weakened by disagreements about purpose, unequal Member State influence, and inadequate accountability. Contestation of goals and strategies constrain planning. Structure and workforce deficiencies limit coordination, agility, and competence. WHO has technical and normative authority, but insufficient independence and legal power to influence uncooperative states. WHO's identity claims transparency, independence, and courage, but these aspirations are betrayed in times of need. Most articles (88%) were commentaries without specified methods. More than three-quarters (76%) originated from the US, the UK, or Switzerland. A quarter of papers (25%) focused on international infectious disease outbreaks, and another 25% advocated for WHO reform generally. Many criticisms cite wide-ranging performance problems, some of which may relate to obstructive behavior by Member States. This literature is incomplete in the geographic representation of authors, evidence, methods, and topics. We offer ideas for developing more rigorous and inclusive academic debate on WHO.

Improving WHO's understanding of WHO guideline uptake and use in Member States: a scoping review

Background

WHO publishes public health and clinical guidelines to guide Member States in achieving better health outcomes. Furthermore, WHO’s Thirteenth General Programme of Work for 2019–2023 prioritizes strengthening its normative functional role and uptake of normative and standard-setting products, including guidelines at the country level. Therefore, understanding WHO guideline uptake by the Member States, particularly the low- and middle-income countries (LMICs), is of utmost importance for the organization and scholarship.

Methods

We conducted a scoping review using a comprehensive search strategy to include published literature in English between 2007 and 2020. The review was conducted between May and June 2021. We searched five electronic databases including CINAHL, the Cochrane Library, PubMed, Embase and Scopus. We also searched Google Scholar as a supplementary source. The review adhered to the PRISMA-ScR (PRISMA extension for scoping reviews) guidelines for reporting the searches, screening and identification of evaluation studies from the literature. A narrative synthesis of the evidence around key barriers and challenges for WHO guideline uptake in LMICs is thematically presented.

Results

The scoping review included 48 studies, and the findings were categorized into four themes: (1) lack of national legislation, regulations and policy coherence, (2) inadequate experience, expertise and training of healthcare providers for guideline uptake, (3) funding limitations for guideline uptake and use, and (4) inadequate healthcare infrastructure for guideline compliance. These challenges were situated in the Member States’ health systems. The findings suggest that governance was often weak within the existing health systems amongst most of the LMICs studied, as was the guidance provided by WHO’s guidelines on governance requirements. This challenge was further exacerbated by a lack of accountability and transparency mechanisms for uptake and implementation of guidelines. In addition, the WHO guidelines themselves were either unclear and were technically challenging for some health conditions; however, WHO guidelines were primarily used as a reference by Member States when they developed their national guidelines.

Conclusions

The challenges identified reflect the national health systems’ (in)ability to allocate, implement and monitor the guidelines. Historically this is beyond the remit of WHO, but Member States could benefit from WHO implementation guidance on requirements and needs for successful uptake and use of WHO guidelines.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-022-00899-y.

Guideline for the treatment of no light perception eyes induced by mechanical ocular trauma

Severe mechanical ocular trauma with no light perception (NLP) predicts a poor prognosis of visual acuity and enucleation of the eyeball. Since the innovative treatment concept of exploratory vitreoretinal surgery has developed and treatment technology has advanced, the outcomes of severe ocular trauma treatment in NLP patients have greatly improved. However, there remains a lack of unified standards for the determination, surgical indication, and timing of vitrectomy in NLP eye treatment. To address these problems, we aimed to create evidence-based medical guidelines for the diagnosis, treatment, and prognosis of mechanical ocular trauma with NLP. Sixteen relevant recommendations for mechanical ocular trauma with NLP were obtained, and a consensus was reached. Each recommendation was explained in detail to guide the treatment of mechanical ocular trauma associated with NLP.

The Impact of the Quality of Nutrition and Lifestyle in the Reproductive Years of Women with PKU on the Long-Term Health of Their Children

A woman’s nutritional status before and during pregnancy can affect the health of her progeny. Phenylketonuria (PKU), a rare disorder causing high blood and brain phenylalanine (Phe) concentrations, is associated with neurocognitive disability. Lifelong treatment is mainly dietetic with a Phe-restricted diet, supplemented with a low-Phe protein substitute. Treatment adherence commonly decreases in adolescence, with some adults ceasing dietary treatment. In maternal PKU, elevated blood Phe is harmful to the fetus so a strict Phe-restricted diet must be re-established preconception, and this is particularly difficult to achieve. A woman’s reproductive years introduces an opportunity to adopt healthier behaviours to prepare for successful pregnancies and positive health outcomes for both themselves and their children. Several factors can influence the health status of women with PKU. Political, socioeconomic, and individual food and lifestyle choices affect diet quality, metabolic control, and epigenetics, which then pre-condition the overall maternal health and long-term health of the child. Here, we reflect on a comprehensive approach to treatment and introduce practical recommendations to optimize the wellbeing of women with PKU and the resultant health of their children.

Inclusion of human rights in sexual and reproductive health programming: Facilitators and barriers to implementation

The need to prioritise those furthest behind is well understood in global health circles, and how human rights norms and standards can help often touted. As rights concerns are particularly recognised in sexual and reproductive health (SRH) programming, as part of a larger exercise, a review was conducted to identify documented barriers and facilitators to implementation. Given the role global guidance plays in implementing rights-based approaches to SRH, UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP) guidelines, tools, recommendations and guidance that include the explicit mention of human rights principles served as the basis for this exercise. This was followed by an extensive review of the literature. Sources reviewed confirmed barriers include not only broad structural, policy and health systems barriers but financial, staffing and time constraints, as well as lack of understanding of concretely how to include human rights in these efforts. Facilitators include the existence of human rights champions, leadership, strong civil society participation, training, and funding made available specifically for implementation. Investment in indicators and documentation sensitive to human rights is warranted in sexual and reproductive health, as well as other health topics, to best serve populations who need them most.

[Assessment of Brazilian clinical guidelines in oncology: gaps in drafting, applicability, and editorial independence]

The expansion in the variety of clinical guidelines in oncology is perceptible worldwide, highlighting the need to guarantee the quality of these documents. The study thus aimed to assess the quality of Brazilian national guidelines for treatments of breast, prostate, and colon and rectal cancers. We selected 12 Brazilian guidelines published by four different drafting groups (Ministry of Health, Supplementary Health System, and medical societies and associations), and the AGREE II instrument was applied. In all these guidelines, we identified important weaknesses in more than one Domain, especially low values for "applicability" and "editorial independence". The patterns observed per Domains are more related to the drafting group than the respective clinical conditions. Lower scores in "drafting rigor" and "editorial independence" were obtained by nongovernmental drafting groups, including absence of information or lack of its transparency. Although the "clarity of presentation" in the Ministry of Health guidelines was relatively lower, all the guidelines presented major limitations in "applicability". Consequently, in the overall assessment, none of the guidelines was recommended without modifications, and four were not recommended at all. Finally, it is necessary to upgrade the guidelines according to the underlying evidence ("methodological rigor") and to present the recommended practices in a comprehensible and applicable way ("applicability"), and to mitigate conflicting interests in order to offer cancer patients the best available care in Brazil.

Clinical guidelines for the management of keratoconus patients with gas permeable contact lenses based on expert consensus and available evidence

PURPOSE OF REVIEW

Fitting rigid gas permeable contact lenses (RGP CLs) in keratoconic patients is the most common visual rehabilitation option to improve patients' quality of life, but require long patient and practitioner chair time. The purpose of this review is to provide evidence-based clinical practice guidelines (CPG) in the management of patients with keratoconus with RGP CLs.

RECENT FINDINGS

An extensive literature review from 1990 to 2017 identified 354 potentially relevant publications. Fifty-two articles were reviewed and included in the CPG. An international expert panel of eight contact lens practitioners, with vast experience in keratoconus management reviewed and appraised the CPG following the Appraisal of Guidelines for Research and Evaluation II consortium requirements. The developed CPG clearly outlines a strategy for the successful fitting of RGP CLs in patients with keratoconus. This includes how to calculate parameters of the first diagnostic lens, criteria for assessing good fit and a standardized schedule of wear time and follow-up appointments.

SUMMARY

The current evidence and consensus-based CPG helps guide clinicians in a successful strategy for fitting RGP CLs in patients with keratoconus.

The availability of global guidance for the promotion of women's, newborns', children's and adolescents' health and nutrition in conflicts

BACKGROUND

Significant global gains in sexual, reproductive, maternal, newborn, child and adolescent health and nutrition (SRMNCAH&N) will be difficult unless conflict settings are adequately addressed. We aimed to determine the amount, scope and quality of publically available guidance documents, to characterise the process by which agencies develop their guidance and to identify gaps in guidance on SRMNCAH&N promotion in conflicts.

METHODS

We identified guidance documents published between 2008 and 2018 through English-language Internet sites of humanitarian response organisations, reviewed them for their scope and assessed their quality with the AGREE II (Appraisal of Guidelines for REsearch and Evaluation II) tool. Additionally, we interviewed 22 key informants on guidance development, dissemination processes, perceived guidance gaps and applicability.

FINDINGS

We identified 105 conflict-relevant guidance documents from 75 organisations. Of these, nine were specific to conflicts, others were applicable also to other humanitarian settings. Fifteen documents were technical normative guidelines, others were operational guides (67), descriptive documents (21) or advice on legal, human rights or ethics questions (2). Nutrition was the most addressed health topic, followed by communicable diseases and violence. The documents rated high quality in their 'scope and purpose' and 'clarity of presentation' and low for 'rigour of development' and 'editorial independence'. Key informants reported end user need as the primary driver for guideline development and WHO technical guidelines as their main evidence base. Insufficient local contextualisation, lack of inter-agency coordination and lack of systematic implementation were considered problems in guideline development. Several guidance gaps were noted, including abortion care, newborn care, early child development, mental health, adolescent health beyond sexual and reproductive health and non-communicable diseases.

INTERPRETATION

Organisations are motivated and actively producing guidance for SRMNCAH&N promotion in humanitarian settings, but few documents address conflicts specifically and there are important guidance gaps. Improved inter-organisation collaboration for guidance on SRMNCAH&N promotion in conflicts and other humanitarian settings is needed.

Identifying research questions for HIV, tuberculosis, tuberculosis-HIV, malaria, and neglected tropical diseases through the World Health Organization guideline development process: a retrospective analysis, 2008-2018

OBJECTIVES

World Health Organization (WHO) guidelines for health programmes and healthcare delivery are the foundation of its technical leadership in public health and essential to decision-making globally. A key function of guideline development is to identify areas in which further evidence is needed because filling these gaps will lead to future improvements in population health. The objective of this study was to examine the knowledge gaps and research questions for addressing those gaps generated through the WHO guideline development process, with the goal of informing future strategies for improving and strengthening the guideline development process.

STUDY DESIGN

We did a systematic, retrospective analysis of research questions identified in the published guidelines.

METHODS

We analyzed guidelines published between January 1, 2008, and December 31, 2018, by the Communicable Diseases Cluster in five disease areas: tuberculosis (TB), HIV, malaria, TB-HIV, and neglected tropical diseases (NTDs). Research questions were extracted independently by two researchers. We analyzed the distribution of research questions by disease and by topic category and did a qualitative assessment of optimum practice for research question generation during the guideline development process.

RESULTS

A total of 48 guidelines were included: 26 on HIV, 1 on malaria, 11 on TB, 5 on TB/HIV, and 5 on NTDs. Overall, 36 (75%) guidelines encompassed a total of 360 explicit research questions; the remainder did not contain specific research questions. The number of research questions that focused on TB was 49, TB/HIV was 38, HIV was 250, and NTDs was 23. The number of research questions that focused on diagnosis was 43 (11.9%) of 360, prevention was 62 (17.2%), treatment was 103 (28.6%), good practice was 12 (3.3%), service delivery was 86 (23.8%), and other areas was 54 (15%). Research questions were often not formulated in a specific or actionable way and were hard to identify in the guideline. Examples of good practice identified by the review team involved the generation of specific and narrowly defined research questions, with accompanying recommendations for appropriate study design.

CONCLUSIONS

The WHO must strengthen its approach to identifying and presenting research questions during the guideline development process. Ensuring access to research questions is a key next step in adding value to the guideline development process.

[Methodological quality assessment of guidelines for surveillance and clinical management of dengue and chikungunya]

The study aimed to assess the methodological quality of guidelines by the Brazilian Ministry of Health, Pan American Health Organization (PAHO), and World Health Organization (WHO) on surveillance and clinical management of dengue and chikungunya. This was a descriptive study in which the tool Appraisal of Guidelines for Research & Evaluation Reporting Checklist II (AGREE II) was applied by four evaluators in independent and masked fashion for six guidelines. Each evaluator assigned a score from 1 (disagree completely) to 7 (agree completely) to the 23 items in the AGREE II domains: scope and purpose; stakeholder involvement; rigor in the development; clarity of presentation; applicability; and editorial independence. The dengue guidelines by PAHO (mean = 5.2, SD = 0.8) and WHO (mean = 4.5, SD = 0.5) obtained the highest overall scores and were recommended with modifications by all the evaluators, while the Brazilian Ministry of Health guidelines (mean = 2.7, SD = 0.4) were not recommended by any of them. Meanwhile, the chikungunya guidelines scored low (means from 2.2 to 3.0) for all three agencies. The domains with the greatest conformity were "clarity of presentation" (median 84.7%) and "scope and purpose" (77.1%), while those with the lowest conformity were "editorial independence" (5.2%) and "rigor in development" (9.1%). The study identified gaps in the guidelines' methodological quality, mainly in transparency of the work processes, selection of scientific evidence, and formulation of recommendations, besides lack of clarity in financing and possible conflicts of interest.

Using RIGHT (Reporting Items for Practice Guidelines in Healthcare) to evaluate the reporting quality of WHO guidelines

Background

Without adequate reporting of research, valuable time and resources are wasted. In the same vein, adequate reporting of practice guidelines to optimise patient care is equally important. Our study examines the quality of reporting of published WHO guidelines, over time, using the RIGHT (Reporting Items for Practice Guidelines in HealThcare) reporting checklist.

Methods

We examined English-language guidelines approved by the WHO Guidelines Review Committee from inception of the committee in 2007 until 31 December 2017. Pairs of independent, trained reviewers assessed the reporting quality of these guidelines. Descriptive data were summarised with frequencies and percentages.

Results

We included 182 eligible guidelines. Overall, 25 out of the 34 RIGHT items were reported in 75% or more of the WHO guidelines. The reporting rates improved over time. Further, 90% of the guidelines reported document type in the title. The identification of evidence, the rationale for recommendations and the review process were reported in more than 80% of guidelines. The certainty of the evidence using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) system was assessed in 81% of the guidelines assessed. While 82% of guidelines reported funding sources, only 25% mentioned the role of funders.

Conclusions

WHO guidelines provide adequate reporting of many of the RIGHT items and reporting has improved over time. WHO guidelines compare favourably to guidelines produced by other organisations. However, reporting can be further improved in a number of areas.

[CNGOF Guidelines for Clinical Practice: Revision of the Methodology]

OBJECTIVES

To revise the organization and the methodology of the Practice Clinical Guidelines (PCG) of the French College of Gynecologists and Obstetricians (CNGOF).

METHODS

The different available methods of PCG organization and of scientific evidence grading have been consulted after searching in the Medline database.

RESULTS

The PCG group of the CNGOF has decided to adopt the AGREE II (for Appraisal of Guidelines for REsearch and Evaluation) methology for PCG organization and the GRADE (for Grading of Recommendation Assessment, Development, and Evaluation) system for grading scientific evidence.

CONCLUSION

By adopting the AGREE II consortium criteria and grading scientific evidence according to the GRADE system, the CNGOF will increase the quality of the overall process, will deliver more targeted and easy to assimilate recommendations, to facilitate professional decision making.

An evaluation of WHO emergency guidelines for Zika virus disease

BACKGROUND

In the face of an unclear causal association between Zika virus in utero exposure and congenital abnormalities and urgent demand for guidance, the World Health Organization (WHO) had to produce timely and trustworthy guidelines during the 2016 Public Health Emergency of International Concern (PHEIC).

METHODS

This is a cross-sectional evaluation of WHO emergency guidelines produced during the Zika virus disease PHEIC from 1 February to 18 November 2016. We assessed adherence to WHO publication requirements and the reporting of guideline development processes associated with trustworthiness. In the absence of quality appraisal tools for guidelines developed under compressed timeframes, we applied the Appraisal of Guidelines for Research and Evaluation (AGREE II) tool.

RESULTS

We included 21 guidelines (13 de novo and 8 updates). Six guidelines used a formal evidence review process. Most guidelines involved external experts in the development process and collected declarations of interest. Peer review was reported in six documents. Most emergency guidelines included updating plans. The highest scoring AGREE II domain was clarity of presentation (median score 78%); the lowest scoring domain was applicability (median score 18%).

CONCLUSION

WHO developed moderate- to high-quality emergency guidelines in the challenging context of a PHEIC. We found improvement opportunities for WHO guideline development teams in the use of evidence to formulate recommendations, the collection of declarations of interest, reporting of conflicts of interest, and the use of existing WHO organizational quality assurance processes.

Critical Appraisal of International Clinical Practice Guidelines in Kidney Transplantation Using the Appraisal of Guidelines for Research and Education II Tool: A Systematic Review

BACKGROUND

Although clinical practice guidelines (CPGs) are used for the development of local protocols in kidney transplantation (Ktx), the quality of their methodology is variable. This systematic review aimed to critically appraise international CPGs in all aspects of Ktx using the Appraisal of Guidelines for Research and Evaluation II tool.

METHODS

Clinical Practice Guidelines in Ktx and donation published between 2010 and 2017 were identified from MEDLINE, Embase, National Guideline Clearinghouse, National Health Service and National Institute for Health and Care Excellence Evidence Searches, and the websites of transplant societies. Using Appraisal of Guidelines for Research and Evaluation II, 3 appraisers assessed the quality of CPGs. Interrater reliability was measured using the intraclass correlation coefficient (ICC).

RESULTS

Searches identified 3168 records, and 115 CPGs were included. The highest scoring Appraisal of Guidelines for Research and Evaluation II domain was "scope and purpose" (80%; range, 30%-100%), followed by "clarity of presentation" (77%; range, 43%-98%), "editorial independence" (52%; range, 0%-94%), "rigor of development" (47%; range 6%-97%) and "stakeholder involvement" (41%; range, 11%-85%). The poorest scoring domain was "applicability" (31%; range, 3%-74%). Most CPGs were recommended for future use either with (63%) or without (18%) modifications. A small number (14%) were not recommended for future use or reviewers (5%) did not agree on recommending the CPG. The overall mean CPG quality score was 4 of 7 (range, 2-7). The mean ICC of 0.74 indicated substantial agreement between reviewers.

CONCLUSIONS

The quality of international CPGs in Ktx was variable, and most CPGs lacked key aspects of methodological robustness and transparency. Improvements in methodology, patient involvement, and strategies for implementation are required.

Institutionalising an evidence-informed approach to guideline development: progress and challenges at the World Health Organization

This study explored experiences, perceptions and views among World Health Organization (WHO) staff about the changes, progress and challenges brought by the guideline development reforms initiated in 2007. Thirty-five semistructured interviews were conducted with senior WHO staff. Sixteen of the interviewees had in-depth experience with WHO's formal guideline development process. Thematic analysis was conducted to identify key themes in the qualitative data, and these were interpreted in the context of the existing literature on WHO's guideline development processes. First, the reforms were seen to have transformed and improved the quality of WHO's guidelines. Second, independent evaluation and feedback by the Guidelines Review Committee (GRC) was described to have strengthened the legitimacy of WHO's recommendations. Third, WHO guideline development processes are not yet designed to systematically make use of all types of research evidence needed to inform decisions about health systems and public health interventions. For example, several interviewees expressed dissatisfaction with the insufficient attention paid to qualitative evidence and evidence from programme experience, and how the Grading of Recommendations Assessment, Development and Evaluation (GRADE) process evaluates the quality of evidence from non-randomised study designs, while others believed that GRADE was just not properly understood or applied. Fourth, some staff advocated for a more centralised quality assurance process covering all outputs from WHO's departments and scientific advisory committees, especially to eliminate strategic efforts aimed at bypassing the GRC's requirements. Overall, the 'culture change' senior WHO staff called for over 10 years ago appears to have gradually spread throughout the organisation. However, at least two major challenges remain: (1) ensuring that all issued advice benefits from independent evaluation, monitoring and feedback for quality and (2) designing guideline development processes to better acquire, assess, adapt and apply the full range of evidence that can inform recommendations on health systems and public health interventions.

A Typology of Scientific Advisory Committees

The era of evidence-informed decision-making has seen increased use of the scientific advisory committee (SAC) to provide decision-makers with scientific advice, despite limited evidence of the effectiveness or best strategies for designing these committees. In this study, an in-depth review of academic and gray literature is undertaken to outline the global landscape of SACs. The development of a typology is also undertaken that categorizes SACs along six dimensions: 1) sector, 2) level of operation, 3) permanence, 4) target audience, 5) autonomy, and 6) nature of advice. It is found that SACs differ profoundly in each of these dimensions and provide examples demonstrating this variation. The landscape and typology can help decision-makers understand the key elements of SAC design and reform, and the results will also inform future research on the design and effectiveness of SACs. With SACs expected to promote evidence-informed decision-making, it is imperative that the design of these committees themselves is guided by evidence.

When "Good Evidence" Is Not Enough: A Case of Global Malaria Policy Development

This paper presents findings from a case study of two different policy development processes within the WHO's malaria department. By comparing the policy processes for the interventions of intermittent preventive treatment in infants versus children, the findings suggest that "good evidence" from a technical perspective, though important, is not sufficient to ensure universal agreement and uptake of recommendations. An analysis of 29 key informant interviews finds that evidence also needs to be relevant to the policy question being asked, and that expert actors retain a concern over the legitimacy of the process by which technical evidence is brought to bear in the policy development process. Previous findings from the field of sustainable development, that evidence must be credible, salient, and legitimate to be accepted by the public, appears to apply equally within scientific advisory committees. While the WHO has principally focused on technical criteria for evidence inclusion in its policy development processes, this study suggests that the design and functionality of its advisory bodies must also enable transparent, responsive, and accepted processes of evidence review to ensure that these bodies are effective in producing advice that engenders change in policy and practice.

Systematic Quality Assessment of Published Antishivering Protocols

Shivering is a common side effect of targeted temperature modulation and general anesthesia. Antishivering strategies often employ a stepwise approach involving both pharmacological and physical interventions. However, approaches to treat shivering are often empiric and vary widely across care environments. We evaluated the quality of published antishivering protocols and guidelines with respect to methodological rigor, reliability, and consistency of recommendations.Using 4 medical databases, we identified 4027 publications that addressed shivering therapy, and excluded 3354 due to lack of relevance. After applying predefined eligibility criteria with respect to minimal protocol standards, 18 protocols/guidelines remained. Each was assessed using a modified Appraisal of Guidelines for Research and Evaluation II (mAGREE II) instrument containing 23 quality items within 6 domains (maximal score 23). Among 18 protocols/guidelines, only 3 incorporated systematically reviewed recommendations, whereas 15 merely targeted practice standardization. Fifteen of 18 protocols/guidelines addressed shivering during therapeutic cooling in which skin counterwarming and meperidine were most commonly cited. However, their mAGREE II scores were within the lowest tertile (1 to 7 points) and the median for all 18 protocols was 5. The quality domains most commonly absent were stakeholder involvement, rigor of development, and editorial independence. Three of 18 protocols/guidelines addressed postanesthetic antishivering. Of these, the American Society of Anesthesiologists guidelines recommending forced-air warming and meperidine received the highest mAGREE II score (14 points), whereas the remaining 2 recommendations had low scores (<5 points).Current published antishivering protocols/guidelines lack methodological rigor, reliability, and strength, and even the highest scoring of the 18 protocols/guidelines fulfilled only 60% of quality items. To be consistent with evidence-based protocol/guideline development processes, future antishivering treatment algorithms should increase methodological rigor and transparency.

Guideline appraisal with AGREE II: Systematic review of the current evidence on how users handle the 2 overall assessments

INTRODUCTION

The Appraisal of Guidelines for Research & Evaluation (AGREE) II instrument is the most commonly used guideline appraisal tool. It includes 23 appraisal criteria (items) organized within 6 domains and 2 overall assessments (1. overall guideline quality; 2. recommendation for use). The aim of this systematic review was twofold. Firstly, to investigate how often AGREE II users conduct the 2 overall assessments. Secondly, to investigate the influence of the 6 domain scores on each of the 2 overall assessments.

MATERIALS AND METHODS

A systematic bibliographic search was conducted for publications reporting guideline appraisals with AGREE II. The impact of the 6 domain scores on the overall assessment of guideline quality was examined using a multiple linear regression model. Their impact on the recommendation for use (possible answers: "yes", "yes, with modifications", "no") was examined using a multinomial regression model.

RESULTS

118 relevant publications including 1453 guidelines were identified. 77.1% of the publications reported results for at least one overall assessment, but only 32.2% reported results for both overall assessments. The results of the regression analyses showed a statistically significant influence of all domains on overall guideline quality, with Domain 3 (rigour of development) having the strongest influence. For the recommendation for use, the results showed a significant influence of Domains 3 to 5 ("yes" vs. "no") and Domains 3 and 5 ("yes, with modifications" vs. "no").

CONCLUSIONS

The 2 overall assessments of AGREE II are underreported by guideline assessors. Domains 3 and 5 have the strongest influence on the results of the 2 overall assessments, while the other domains have a varying influence. Within a normative approach, our findings could be used as guidance for weighting individual domains in AGREE II to make the overall assessments more objective. Alternatively, a stronger content analysis of the individual domains could clarify their importance in terms of guideline quality. Moreover, AGREE II should require users to transparently present how they conducted the assessments.

Guidelines in lower-middle income countries

Guidelines include recommendations intended to optimize patient care; used appropriately, they make healthcare consistent and efficient. In most lower-middle income countries (LMICs), there is a paucity of well-designed guidelines; as a result, healthcare workers depend on guidelines developed in Higher Income Countries (HICs). However, local guidelines are more likely to be implemented because they are applicable to the specific environment; and consider factors such as availability of resources, specialized skills and local culture. If guidelines developed in HICs are to be implemented in LMICs, developers need to incorporate local experts in their development. Involvement of local stakeholders may improve the rates of implementation by identifying and removing barriers to implementation in LMICs. Another option is to encourage local experts to adapt them for use in LMICs; these guidelines may recommend strategies different from those used in HICs, but will be aimed at achieving the best practicable standard of care. Infrastructural deficits in LMICs could be improved by learning from and building on the successful response to the human immunodeficiency virus/acquired immunodeficiency syndrome pandemic through interactions between HICs and LMICs. Similarly, collaborations between postgraduate medical colleges in both HICs and LMICs may help specialist doctors training in LMICs develop skills required for guideline development and implementation.

Guidelines of the International Federation of Red Cross and Red Crescent Societies: an overview and quality appraisal using AGREE II

OBJECTIVES

To appraise the quality of guidelines developed by the International Federation of Red Cross and Red Crescent Societies (IFRC) between 2001 and 2015.

STUDY DESIGN

Cross-sectional.

METHODS

2 authors independently assessed the quality of IFRC guidelines using the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument. Average domain scores were calculated and overall quality scores and recommendation for use were determined.

RESULTS

Out of 77 identified guidelines, 27 met the inclusion criteria and were assessed. The domains with the highest average scores across guidelines were 'scope and purpose', 'clarity of presentation' and 'applicability'. The lowest scoring domains were 'rigour of development' and 'editorial independence'. No guideline can be 'recommended for immediate use', 23 guidelines are 'recommended with modifications' and 4 guidelines are 'not recommended'.

CONCLUSIONS

The IFRC produces guidelines that should be adhered to by millions of staff and volunteers in 190 countries. These guidelines should therefore be of high quality. Up until now, the IFRC had no uniform guideline development process. The results of the AGREE II appraisal indicate that the quality of the guidelines needs to be improved.

Uptake of the World Health Organization's trauma care guidelines: a systematic review

OBJECTIVE

To understand the degree to which the trauma care guidelines released by the World Health Organization (WHO) between 2004 and 2009 have been used, and to identify priorities for the future implementation and dissemination of such guidelines.

METHODS

We conducted a systematic review, across 19 databases, in which the titles of the three sets of guidelines - Guidelines for essential trauma care, Prehospital trauma care systems and Guidelines for trauma quality improvement programmes - were used as the search terms. Results were validated via citation analysis and expert consultation. Two authors independently reviewed each record of the guidelines' implementation.

FINDINGS

We identified 578 records that provided evidence of dissemination of WHO trauma care guidelines and 101 information sources that together described 140 implementation events. Implementation evidence could be found for 51 countries - 14 (40%) of the 35 low-income countries, 15 (32%) of the 47 lower-middle income, 15 (28%) of the 53 upper-middle-income and 7 (12%) of the 59 high-income. Of the 140 implementations, 63 (45%) could be categorized as needs assessments, 38 (27%) as endorsements by stakeholders, 20 (14%) as incorporations into policy and 19 (14%) as educational interventions.

CONCLUSION

Although WHO's trauma care guidelines have been widely implemented, no evidence was identified of their implementation in 143 countries. More serial needs assessments for the ongoing monitoring of capacity for trauma care in health systems and more incorporation of the guidelines into both the formal education of health-care providers and health policy are needed.

Preconceptional Nutrition Interventions for Adolescent Girls and Adult Women: Global Guidelines and Gaps in Evidence and Policy with Emphasis on Micronutrients

Much of the global nutrition efforts in recent years have been focused on improving the nutritional status of children during the window of the first 1000 d of life, from conception to 2 y of age. However, as the world transitions from the Millennium Development Goals to the Sustainable Development Goals, women's and adolescent girls' overall health and well-being are being placed at the center of the global agenda. It is also increasingly recognized that a woman's nutritional status before pregnancy affects maternal and child outcomes and thus needs to be improved to ensure optimal outcomes. This article reviews the global picture of preconception nutrition in women and girls, including some of the key factors that influence women's outcomes, as well as their children's outcomes, if they do become pregnant. This article describes the current global guidelines on preconceptional nutrition interventions for girls and women; highlights related gaps in evidence, guidelines, and policy; and discusses research to forward the agenda of improving women's and girls' preconceptional nutrition.

Implementation plans included in World Health Organisation guidelines

Background

The implementation of high-quality guidelines is essential to improve clinical practice and public health. The World Health Organisation (WHO) develops evidence-based public health and other guidelines that are used or adapted by countries around the world. Detailed implementation plans are often necessary for local policymakers to properly use the guidelines developed by WHO. This paper describes the plans for guideline implementation reported in WHO guidelines and indicates which of these plans are evidence-based.

Methods

We conducted a content analysis of the implementation sections of WHO guidelines approved by the WHO guideline review committee between December 2007 and May 2015. The implementation techniques reported in each guideline were coded according to the Cochrane Collaboration’s Effective Practice and Organisation of Care (EPOC) taxonomy and classified as passive, active or policy strategies. The frequencies of implementation techniques are reported.

Results

The WHO guidelines (n = 123) analysed mentioned implementation techniques 800 times, although most mentioned implementation techniques very briefly, if at all. Passive strategies (21 %, 167/800) and general policy strategies (62 %, 496/800) occurred most often. Evidence-based active implementation methods were generally neglected with no guideline mentioning reminders (computerised or paper) and only one mentioning a multifaceted approach. Many guidelines contained implementation sections that were identical to those used in older guidelines produced by the same WHO technical unit.

Conclusions

The prevalence of passive and policy-based implementation techniques as opposed to evidence-based active techniques suggests that WHO guidelines should contain stronger guidance for implementation. This could include structured and increased detail on implementation considerations, accompanying or linked documents that provide information on what is needed to contextualise or adapt a guideline and specific options from among evidence-based implementation strategies.

Strength of recommendations in WHO guidelines using GRADE was associated with uptake in national policy

OBJECTIVES

This study assesses the extent to which the strength of a recommendation in a World Health Organization (WHO) guideline affects uptake of the recommendation in national guidelines.

STUDY DESIGN AND SETTING

The uptake of recommendations included in HIV and TB guidelines issued by WHO from 2009 to 2013 was assessed across guidelines from 20 low- and middle-income countries in Africa and Southeast Asia. Associations between characteristics of recommendations (strength, quality of the evidence, type) and uptake were assessed using logistic regression.

RESULTS

Eight WHO guidelines consisting of 109 strong recommendations and 49 conditional recommendations were included, and uptake assessed across 44 national guidelines (1,255 recommendations) from 20 countries. Uptake of WHO recommendations in national guidelines was 82% for strong recommendations and 61% for conditional recommendations. The odds of uptake comparing strong recommendations and conditional recommendations was 1.9 (95% confidence interval: 1.4, 2.7), after adjustment for quality of evidence. Higher levels of evidence quality were associated with greater uptake, independent of recommendation strength.

CONCLUSION

Guideline developers should be confident that conditional recommendations are frequently adopted. The fact that strong recommendations are more frequently adopted than conditional recommendations underscores the importance of ensuring that such recommendations are justified.