Current experience with applying the GRADE approach to public health interventions: an empirical study

Spinal hyperexcitability in patients with chronic musculoskeletal pain or headache as evidenced by alterations in the nociceptive withdrawal reflex: a systematic review and meta-analysis

ABSTRACT

The nociceptive withdrawal reflex (NWR) is a spinal withdrawal reflex induced by painful stimulation. It is a measure of spinal hyperexcitability, which is believed to contribute to chronic musculoskeletal pain (MSKP) and headache. Previous syntheses of the evidence for alterations in the NWR in patients with chronic MSKP and headache needed a comprehensive update. This systematic review and meta-analysis was performed after the Preferred Items for Systematic reviews and Meta-Analyses guidelines. Studies examining NWR-related outcome measures in patients with chronic MSKP and headache compared to pain-free controls were identified through electronic database searches and included after screening against predefined eligibility criteria. Standardized mean differences or mean differences and 95% confidence intervals (CI) were calculated. Thirty-one studies were included in the systematic review and 25 in the meta-analysis. Moderate-quality evidence was found indicating lower NWR threshold (-3.68; 95% CI, -4.56 to -2.80; P < 0.001), larger NWR area (standardized mean difference = 0.69; 95% CI, 0.37-1.01; P < 0.001), and shorter NWR latency (mean difference = -13.68; 95% CI, -22.69, -4.67; P = 0.003) in patients compared to controls. These findings remained robust when performing meta-regressions based on subgroups (ie, headache, fibromyalgia, whiplash-associated disorder, and osteoarthritis). Low-quality evidence demonstrated facilitated temporal summation of NWR threshold (-2.48; 95% CI, -3.13 to -1.83; P < 0.001) in patients compared to controls. Spinal hyperexcitability as evidenced by lowered NWR threshold values and temporal summation of the NWR is present in patients with chronic MSKP and headache. No evidence was found for alterations in NWR duration and NWR magnitude. Future research is needed to address the gap in research on NWR-related outcome measures other than NWR threshold.

A methodology for determining dosing recommendations for anticancer drugs in patients with reduced kidney function

Reduced kidney function (or kidney dysfunction) is commonly an exclusion criterion for randomised controlled trials (RCTs) in cancer. Consequently, high quality evidence for anticancer drug dosing in reduced kidney function is limited and no internationally agreed guidelines exist to inform prescribing decisions in this population. A methodology for guideline development was applied which did not require availability of RCTs but used critical appraisal of existing observational literature and group consensus. An international multidisciplinary working group (n = 38) established consensus recommendations in two parts to form the International Consensus Guideline on Anticancer Drug Dosing in Kidney Dysfunction (ADDIKD). The approach enabled virtual participation worldwide. In Part 1 we developed a standardised approach for assessment and classification of kidney function in patients with cancer using global nephrology standards and working group expertise. Part 2 involved a comprehensive literature search of 59 anticancer drugs followed by a critical appraisal of the evidence certainty through the Grading of Recommendations Assessment, Development and Evaluation (GRADE) process and development of dosing recommendations in reduced kidney function. Key external stakeholders (n = 9) invited expert contributors (n = 25), and the working group participated in virtual interactive workshops to vote on the acceptability of these recommendations. The participants were provided with evaluation of the literature, and they engaged in several rounds of virtual discussion (involving robustness of the evidence behind recommendations and their real-world application) and anonymous consensus voting. Adapting the ADDIKD guideline development process to a virtual format enabled engagement with a very broad base of specialised international experts especially during the global pandemic. Combining GRADE methodology with consensus-building approaches was an effective method of producing recommendations (in an area lacking RCTs) by merging critical review of the literature with expert opinion and clinical practice.

Funding

Development of the ADDIKD guideline is funded by the Cancer Institute NSW as part of the NSW Government and received no funding from external commercial sources.

Integrating Ethics in Public Health Guideline Development: Case Study of Guidelines on Respiratory Isolation for Persons With Tuberculosis in Community Settings

Public health interventions often involve explicit trade-offs in which the health of the many must be weighed against burdens imposed on individuals. We describe development of public health guidelines for respiratory isolation in community settings for persons with tuberculosis. While stopping the spread of disease is a core moral imperative in public health, the duty to prevent disease transmission does not supersede all other considerations. Community well-being must be balanced with individual well-being, liberty, and social justice. In response to these challenges, the National Tuberculosis Coalition of America's 2024 guidelines for persons with tuberculosis in community settings were developed using a modified GRADE approach supported by a complementary, comprehensive, and context-specific ethical framework. By addressing the distinct roles that evidence (subject to uncertainty), values, justificatory conditions, and procedural legitimacy all play in ethical guideline development, we promote rigor and transparency in the integration of ethics in public health guidelines.

Applying Bradford Hill to assessing causality in systematic reviews: A transparent approach using process tracing

BACKGROUND

Bradford Hill (BH) viewpoints are widely used to assess causality in systematic reviews, but their application has often lacked reproducibility. We describe an approach for assessing causality within systematic reviews ('causal' reviews), illustrating its application to the topic of income inequality and health. Our approach draws on principles of process tracing, a method used for case study research, to harness BH viewpoints to judge evidence for causal claims.

METHODS

In process tracing, a hypothesis may be confirmed by observing highly unique evidence and disconfirmed by observing highly definitive evidence. We drew on these principles to consider the value of finding supportive or contradictory evidence for each BH viewpoint characterised by its uniqueness and definitiveness.

RESULTS

In our exemplar systematic review, we hypothesised that income inequality adversely affects self-rated health and all-cause mortality. BH viewpoints 'analogy' and 'coherence' were excluded from the causal assessment because of their low uniqueness and low definitiveness. The 'experiment' viewpoint was considered highly unique and highly definitive, and thus could be particularly valuable. We propose five steps for using BH viewpoints in a 'causal' review: (1) define the hypothesis; (2) characterise each viewpoint; (3) specify the evidence expected for each BH viewpoint for a true or untrue hypothesis; (4) gather evidence for each viewpoint (e.g., systematic review meta-analyses, critical appraisal, background knowledge); (5) consider if each viewpoint was met (supportive evidence) or unmet (contradictory evidence).

CONCLUSIONS

Incorporating process tracing has the potential to provide transparency and structure when using BH viewpoints in 'causal' reviews.

A Structured Approach to Involve Stakeholders in Prioritising Topics for Systematic Reviews in Public Health

Objectives

This study aimed to develop and apply a structured approach for prioritising topics for systematic reviews in public health, framed according to the readily applicable PICO format, which encourages the involvement of stakeholders' preferences in a transparent matter.

Methods

We developed a multi-stage process, consisting of a scoping and two Delphi stages with web-based surveys and invited public health stakeholders in Switzerland to participate: First, respondents specified topics for different public health domains, which were reformulated in a PICO format by content analysis. Second, respondents rated the topics using five stakeholder-refined assessment criteria. Overall rankings were calculated to assess differences between stakeholder groups and rating criteria.

Results

In total, 215 respondents suggested 728 topics altogether. The response rate in the two Delphi stages was 91.6% and 77.6%, respectively. Most top-rated review topics focused on the effectiveness of interventions providing education to different target groups, followed by interventions to increase access to specific healthcare services.

Conclusion

Our approach encourages involvement of stakeholders in identifying priorities for systematic reviews and highlights disparities between stakeholders and between individual criteria.

Systems for rating bodies of evidence used in systematic reviews of air pollution exposure and reproductive and children's health: a methodological survey

BACKGROUND

Translating findings from systematic reviews assessing associations between environmental exposures and reproductive and children's health into policy recommendations requires valid and transparent evidence grading.

METHODS

We aimed to evaluate systems for grading bodies of evidence used in systematic reviews of environmental exposures and reproductive/ children's health outcomes, by conducting a methodological survey of air pollution research, comprising a comprehensive search for and assessment of all relevant systematic reviews. To evaluate the frameworks used for rating the internal validity of primary studies and for grading bodies of evidence (multiple studies), we considered whether and how specific criteria or domains were operationalized to address reproductive/children's environmental health, e.g., whether the timing of exposure assessment was evaluated with regard to vulnerable developmental stages.

RESULTS

Eighteen out of 177 (9.8%) systematic reviews used formal systems for rating the body of evidence; 15 distinct internal validity assessment tools for primary studies, and nine different grading systems for bodies of evidence were used, with multiple modifications applied to the cited approaches. The Newcastle Ottawa Scale (NOS) and the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) framework, neither developed specifically for this field, were the most commonly used approaches for rating individual studies and bodies of evidence, respectively. Overall, the identified approaches were highly heterogeneous in both their comprehensiveness and their applicability to reproductive/children's environmental health research.

CONCLUSION

Establishing the wider use of more appropriate evidence grading methods is instrumental both for strengthening systematic review methodologies, and for the effective development and implementation of environmental public health policies, particularly for protecting pregnant persons and children.

Efficacy and safety of traditional Chinese medicine adjuvant therapy for severe pneumonia: evidence mapping of the randomized controlled trials, systematic reviews, and meta-analyses

Background and Objective: Severe pneumonia is a critical respiratory disease with high mortality. There is insufficient evidence on the efficacy and safety of traditional Chinese medicine (TCM) adjuvant therapy for severe pneumonia. This study aims to identify, describe, assess, and summarize the currently available high-quality design evidence on TCM adjuvant therapy for severe pneumonia to identify evidence gaps using the evidence mapping approach. Methods: Systematic searches were performed on English and Chinese online databases (PubMed, EMBASE, Cochrane Library, Web of Science, CNKI, WanFang Data, CQVIP, and SinoMed) to identify papers from inception until August 2023 for inclusion into the review. Randomized controlled trials (RCTs), systematic reviews (SRs), and meta-analyses concerning TCM adjuvant therapy for severe pneumonia or its complications in adults were included. The risk of bias in RCTs was evaluated by using the Cochrane Handbook ROB tool. The Assessment of Multiple Systematic Reviews 2 (AMSTAR-2), the Risk of Bias in Systematic Review (ROBIS) tool, and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system were used to assess the methodological quality, risk of bias, and evidence quality of SRs or meta-analyses, respectively. Then, a bubble plot was designed to visually display information in four dimensions. Results: A total of 354 RCTs and 17 SRs or meta-analyses met the inclusion criteria. The published RCTs had several flaws, such as unreasonable design, limited sample size, insufficient attention to non-drug therapy studies and syndrome differentiation, improper selection or use of outcome indicators, and failure to provide high-quality evidence. Sixteen SRs or meta-analyses of methodological quality scored "Critically Low" confidence. Twelve SRs or meta-analyses were rated as "High Risk." Most outcomes were rated as "Low" evidence quality. We found that TCM combined with conventional treatment could improve the clinical total effective rate and the TCM syndromes efficacy. The combined approach could also shorten mechanical ventilation time, infection control time, and length of hospital and ICU stay; significantly reduce temperature, respiratory rate, heart rate, white blood cell counts, levels of C-reactive protein, procalcitonin, blood inflammatory factors, bacteriological response, and D-dimer; decrease CPIS, APACHE II score, and PSI score; improve pulmonary imaging features, arterial blood gas indicators (including arterial oxygen pressure, arterial oxygen saturation, and oxygen index), and lung function (including forced vital capacity and forced expiratory volume in the first second) for severe pneumonia compared with conventional treatment only (p < 0.05). There was no significant difference in adverse reactions and incidence of adverse events (p > 0.05). In addition, compared with conventional treatment only, most SRs or meta-analyses concluded that TCM combined with conventional treatment was "Beneficial" or "Probably beneficial." Conclusion: TCM combined with conventional treatment had advantages in efficacy, clinical signs, laboratory results, and life quality outcomes of severe pneumonia, with no difference in safety outcomes compared with conventional treatment only. QingJin Huatan decoction is the most promising target, and Xuanbai Chengqi decoction has a "Probably beneficial" conclusion. XueBiJing injection and TanReQing injection are two commonly used Chinese herbal injections for treating severe pneumonia, and both are "Probably beneficial." However, there was a need for multicenter RCTs with large sample sizes and high methodological quality in the future. In addition, the methodological design and quality of SRs or meta-analyses should be improved to form high-quality, evidence-based medical evidence and provide evidence for the effectiveness and safety of TCM adjuvant therapy for severe pneumonia.

Characteristics of the sources, evaluation, and grading of the certainty of evidence in systematic reviews in public health: A methodological study

Objectives

To systematically explore how the sources of evidence, types of primary studies, and tools used to assess the quality of the primary studies vary across systematic reviews (SRs) in public health.

Methods

We conducted a methodological survey of SRs in public health by searching the of literature in selected journals from electronic bibliographic databases. We selected a 10% random sample of the SRs that met the explicit inclusion criteria. Two researchers independently extracted data for analysis.

Results

We selected 301 SRs for analysis: 94 (31.2%) of these were pre-registered, and 211 (70.1%) declared to have followed published reporting standard. All SRs searched for evidence in electronic bibliographic databases, and more than half (n = 180, 60.0%) searched also the references of the included studies. The common types of primary studies included in the SRs were primarily cross-sectional studies (n = 132, 43.8%), cohort studies (n = 126, 41.9%), randomized controlled trials (RCTs, n = 89, 29.6%), quasi-experimental studies (n = 83, 27.6%), case-control studies (n = 58, 19.3%) qualitative studies (n = 38, 12.6%) and mixed-methods studies (n = 32, 10.6%). The most frequently used quality assessment tools were the Newcastle-Ottawa Scale (used for 50.0% of cohort studies and 55.6% of case-control studies), Cochrane Collaboration's Risk of Bias tool (50.7% of RCTs) and Critical Appraisal Skills Program (38.5% of qualitative studies). Only 20 (6.6%) of the SRs assessed the certainty of the body of evidence, of which 19 (95.0%) used the GRADE approach. More than 65% of the evidence in the SRs using GRADE was of low or very low certainty.

Conclusions

SRs should always assess the quality both at the individual study level and the body of evidence for outcomes, which will benefit patients, health care practitioners, and policymakers.

A framework for synthesizing intervention evidence from multiple sources into a single certainty of evidence rating: Methodological developments from a US National Academies of Sciences, Engineering, and Medicine Committee

Despite research investment and a growing body of diverse evidence there has been no comprehensive review and grading of evidence for public health emergency preparedness and response practices comparable to those in medicine and other public health fields. The National Academies of Sciences, Engineering, and Medicine convened an ad hoc committee to develop and use methods for grading and synthesizing diverse types of evidence to create a single certainty of intervention-related evidence to support recommendations for Public Health Emergency Preparedness and Response Research. A 13-step consensus building method was used. Experts were first canvassed in public meetings, and a comprehensive review of existing methods was undertaken. Although aspects of existing review methodologies and evidence grading systems were relevant, none adequately covered all requirements for this specific context. Starting with a desire to synthesize diverse sources of evidence not usually included in systematic reviews and using GRADE for assessing certainty and confidence in quantitative and qualitative evidence as the foundation, we developed a mixed-methods synthesis review and grading methodology that drew on (and in some cases adapted) those elements of existing frameworks and methods that were most applicable. Four topics were selected as test cases. The process was operationalized with a suite of method-specific reviews of diverse evidence types for each topic. Further consensus building was undertaken through stakeholder engagement and feedback The NASEM committee's GRADE adaption for mixed-methods reviews will further evolve over time and has yet to be endorsed by the GRADE working group.

Investigating the presentation of uncertainty in an icon array: A randomized trial

BACKGROUND

Clinicians are often advised to use pictographs to communicate risk, but whether they offer benefits when communicating risk imprecision (e.g., 65%-79%) is unknown.

PURPOSE

To test whether any of three approaches to visualizing imprecision would more effectively communicate breast and ovarian cancer risk for BRCA1 pathogenic variant carriers.

METHODS

1,300 UK residents were presented with a genetic report with information about BRCA1-related risks, with random assignment to one of four formats: no visualization (text alone), or a pictograph using shaded icons, a gradient, or arrows marking range endpoints. We also tested pictographs in two layouts. Analysis of variance (ANOVA) and regression was employed.

RESULTS

There was no effect of format. Participants shown pictographs vs. text alone had better uptake of breast cancer risk messages (p < .05, η ² = 0.003). Pictographs facilitated memory for the specific amount of risk (p < 0.001, η ² = 0.019), as did the tabular layout. Individuals not having completed upper secondary education may benefit most.

CONCLUSIONS

We found weak evidence in favor of using simple pictographs with ranges to communicate BRCA risk (versus text alone), and of the tabular layout.

INNOVATION

Testing different ways of communicating imprecision within pictographs is a novel and promising line of research.

Guideline development in harm reduction: Considerations around the meaningful involvement of people who access services

Background

Harm reduction seeks to minimizes the negative effects of drug use while respecting the rights of people with lived and living experience of substance use (PWLLE). Guideline standards ("guidelines for guidelines") provide direction on developing healthcare guidelines. To identify essential considerations for guideline development within harm reduction, we examined whether guideline standards are consistent with a harm reduction approach in their recommendations on involving people who access services.

Methods

We searched the literature from 2011-2021 to identify guideline standards used in harm reduction and publications on involving PWLLE in developing harm reduction services. We used thematic analysis to compare their guidance on involving people who access services. Findings were validated with two organizations of PWLLE.

Results

Six guideline standards and 18 publications met inclusion criteria. We identified three themes related to involving people who access services: Reasons for Involvement, Methods of Involvement, and Factors in Success. Subthemes varied across the literature. We identified five essential considerations for guideline development in harm reduction: establishing a shared understanding of reasons for involving PWLLE; respecting their expertise; partnering with PWLLE to ensure appropriate engagement; incorporating perspectives of populations disproportionately affected by substance use; and securing resources.

Conclusion

Guideline standards and the harm reduction literature approach the involvement of people who access services from different perspectives. Thoughtful integration of the two paradigms can improve guidelines while empowering PWLLE. Our findings can support the development of high-quality guidelines that align with the fundamental principles of harm reduction in their involvement of PWLLE.

Comparison of Ondansetron versus Domperidone for treating vomiting in acute gastroenteritis in children at a resource limited setting of South Punjab, Pakistan

Objectives

To compare the efficacy of Ondansetron versus Domperidone for treating vomiting in acute gastroenteritis (AGE) in children at a resource limited emergency setting of South Punjab, Pakistan.

Methods

This open label randomized controlled trial was conducted at The Pediatric Emergency Department of Tehshil Headquarter Hospital, Liaqatpur, Pakistan, from July 2020 to June 2021. A total of 300 children of both genders aged below 12 years of age having 3 or more non-bilious, non-bloody vomiting episodes within 24 hours and with suggestive signs and symptoms of AGE were enrolled and randomized (150 in each group). Efficacy of both drugs was compared in terms of need of 2nd dose within 15 minutes, cessation of vomiting at 6-hour and 24-hour follow up.

Results

Out of a total of 300 children, 162 (54.0%) were male. Mean age was 4.7±2.3 years. Twenty seven (18.0%) children in Ondansetron group required 2nd dose within 15 minutes while 38 (25.3%) children in Domperidone group required the 2nd dose (p=0.1232). Cessation of vomiting at 6-hour interval was noted among 126 (84.0%) children in Ondansetron group in comparison to 118 (78.7%) in Domperidone group (p=0.2359). It was revealed that 127/142 (89.4%) children in Ondansetron group had cessation of vomiting at 24-hours follow up while this was noted to be among 108/134 (80.6%) children in Domperidone group (p=0.0390).

Conclusion

In comparison to Domperidone, Ondansetron was found to have better efficacy aiming cessation of AGE associated vomiting among children with mild to moderate dehydration.

Individual-, family- and school-based interventions to prevent multiple risk behaviours relating to alcohol, tobacco and drug use in young people aged 8-25 years: a systematic review and meta-analysis

BACKGROUND

Engagement in multiple substance use risk behaviours such as tobacco smoking, alcohol and drug use during adolescence can result in adverse health and social outcomes. The impact of interventions that address multiple substance use risk behaviours, and the differential impact of universal versus targeted approaches, is unclear given findings from systematic reviews have been mixed. Our objective was to assess effects of interventions targeting multiple substance use behaviours in adolescents.

METHODS

Eight databases were searched to October 2019. Individual and cluster randomised controlled trials were included if they addressed two or more substance use behaviours in individuals aged 8-25 years. Data were pooled in random-effects meta-analyses, reported by intervention and setting. Quality of evidence was assessed using GRADE. Heterogeneity was assessed using between-study variance, τ2 and Ι2, and the p-value of between-study heterogeneity statistic Q. Sensitivity analyses were undertaken using the highest and lowest intra-cluster correlation coefficient (ICC).

RESULTS

Of 66 included studies, most were universal (n=52) and school-based (n=41). We found moderate quality evidence that universal school-based interventions are likely to have little or no short-term benefit (up to 12 months) in relation to alcohol use (OR 0.94, 95% CI: 0.84, 1.04), tobacco use (OR 0.98, 95% CI: 0.83, 1.15), cannabis use (OR 1.06, 95% CI: 0.86, 1.31) and other illicit drug use (OR 1.09, 95% CI: 0.85, 1.39). For targeted school-level interventions, there was low quality evidence of no or a small short-term benefit: alcohol use (OR 0.90, 95% CI: 0.74-1.09), tobacco use (OR 0.86, 95% CI: 0.66, 1.11), cannabis use (OR 0.84, 95% CI: 0.66-1.07) and other illicit drug use (OR 0.79, 95% CI 0.62-1.02). There were too few family-level (n=4), individual-level (n=2) and combination level (n=5) studies to draw confident conclusions. Sensitivity analyses of ICC did not change results.

CONCLUSIONS

There is low to moderate quality evidence that universal and targeted school-level interventions have no or a small beneficial effect for preventing substance use multiple risk behaviours in adolescents. Higher quality trials and study reporting would allow better evidence syntheses, which is needed given small benefit of universal interventions can have high public health benefit.

TRIAL REGISTRATION

Cochrane Database of Systematic Reviews 2014, Issue 11. Art. No.: CD011374. DOI: 10.1002/14651858.CD011374.

Effect modification by sex for associations of fine particulate matter (PM2.5) with cardiovascular mortality, hospitalization, and emergency room visits: systematic review and meta-analysis

Particulate matter with aerodynamic diameter no larger than 2.5 μm (PM2.5) has been linked to cardiovascular diseases (CVDs) but evidence for vulnerability by sex remains unclear. We performed systematic review and meta-analysis to synthesize the state of scientific evidence on whether cardiovascular risks from PM2.5 differ for men compared to women. The databases Pubmed, Scopus, Embase, and GreenFILE were searched for studies published Jan. 1995 to Feb. 2020. Observational studies conducting subgroup analysis by sex for impacts of short-term or long-term exposure to PM2.5 on target CVDs were included. Data were independently extracted in duplicate and pooled with random-effects meta-regression. Risk ratios (RRs) for long-term exposure and percent changes in outcomes for short-term exposure were calculated per 10 μg/m3 PM2.5 increase. Quality of evidence of risk differences by sex was rated following Grading of Recommendations Assessment, Development and Evaluation (GRADE). A total of 12,502 articles were screened, with 61 meeting inclusion criteria. An additional 32 studies were added from citation chaining. RRs of all CVD mortality for long-term PM2.5 for men and women were the same (1.14; 95% CI: 1.09, 1.22) indicating no statistically different risks. Men and women did not have statistically different risks of daily CVD mortality, hospitalizations from all CVD, ischemic heart disease, cardiac arrest, acute myocardial infarction, and heart failure from short-term PM2.5 exposure (difference in % change in risk per 10 μg/m3 PM2.5: 0.04 (95% CI, -0.42 to 0.51); -0.05 (-0.47 to 0.38); 0.17 (-0.90, 1.24); 1.42 (-1.06, 3.97); 1.33 (-0.05, 2.73); and -0.48 (-1.94, 1.01), respectively). Analysis using GRADE found low or very low quality of evidence for sex differences for PM2.5-CVD risks. In conclusion, this meta-analysis and quality of evidence assessment of current observational studies found very limited evidence of the effect modification by sex for effects of PM2.5 on CVD outcomes in adults, which can inform clinical approaches and policies.

Recent UK retirees' views about the work-related factors which influenced their decision to retire: a qualitative study within the Health and Employment After Fifty (HEAF) cohort

BACKGROUND

Lower birth rates and increasing longevity have resulted in ageing populations in European countries. These demographic changes place challenges on pension provision as numbers of those who are economically inactive and retired increase relative to those in paid work. Therefore, governments need workers to postpone retirement and work to older ages. Whilst health and wealth are important in retirement decision-making, considerably less is known about the effects of workplace factors. The aim of this study was to explore the views of recent UK retirees about the role that work-related factors played in their decision to retire.

METHODS

This qualitative study was nested within the Health and Employment After Fifty (HEAF) cohort. People who had retired 3-6 years previously (not for health reasons) were purposively sampled to obtain the views of men and women from a range of socio-economic backgrounds and jobs. Semi-structured interviews were carried out by telephone using a pre-defined topic guide. Interviews were audio-recorded, transcribed and analysed thematically.

RESULTS

Seventeen interviews were conducted. Thematic analysis showed that retirement decisions were complex and multi-factorial but that work-related factors contributed to decision-making in two main ways. First, some work factors pushed participants towards retirement. These were perceptions that: workplace change had affected the way they were valued or increased pressure on them; work demands, including commuting, had intruded excessively on personal time, effects that were exacerbated by modern technology; work was draining, isolating or under-appreciated; and /or that work was causing physical strain or discomfort relative to their perception of their capacity. In contrast, work factors could also cause participants to pull back towards work, particularly: autonomy; supportive work colleagues; a sense of being appreciated; and perceived job flexibility.

CONCLUSIONS

Recent retirees explained that their decision to retire was multi-factorial but work-related factors contributed importantly. Potentially, employers could: review workers' perceptions about their work; their capacity in relation to job demands; increase flexibility; and facilitate a supportive work community to encourage longer working lives.

Addressing Challenges When Applying GRADE to Public Health Guidelines: A Scoping Review Protocol and Pilot Analysis

This is a protocol for a scoping review that aims to determine how guideline authors using the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach have addressed previously identified challenges related to public health. The Joanna Briggs Institute (JBI) methodology for scoping reviews will be followed. We will search and screen titles of guidelines for all languages published in 2013-2021 in: the GIN library, BIGG database, Epistemonikos GRADE guidelines repository, GRADEpro Database, MAGICapp, NICE and WHO websites. Two reviewers will independently screen full texts of the documents identified. The following information will be extracted: methods used for identifying different stakeholders and incorporating their perspectives; methods for identification and prioritization of non-health outcomes; methods for determining thresholds for decision-making; methods for incorporating and grading evidence from non-randomized studies; methods for addressing concerns with conditional recommendations in public health; methods for reaching consensus; additional methodological concerns; and any modifications made to GRADE. A combination of directed content analysis and descriptive statistics will be used for data analysis, and the findings presented narratively in a tabular and graphical form. In this protocol, we present the pilot results from 13 identified eligible guidelines issued between January and August 2021. We will publish the full review results when they become available.

Salt and cardiovascular disease: insufficient evidence to recommend low sodium intake

Several blood pressure guidelines recommend low sodium intake (<2.3 g/day, 100 mmol, 5.8 g/day of salt) for the entire population, on the premise that reductions in sodium intake, irrespective of the levels, will lower blood pressure, and, in turn, reduce cardiovascular disease occurrence. These guidelines have been developed without effective interventions to achieve sustained low sodium intake in free-living individuals, without a feasible method to estimate sodium intake reliably in individuals, and without high-quality evidence that low sodium intake reduces cardiovascular events (compared with moderate intake). In this review, we examine whether the recommendation for low sodium intake, reached by current guideline panels, is supported by robust evidence. Our review provides a counterpoint to the current recommendation for low sodium intake and suggests that a specific low sodium intake target (e.g. <2.3 g/day) for individuals may be unfeasible, of uncertain effect on other dietary factors and of unproven effectiveness in reducing cardiovascular disease. We contend that current evidence, despite methodological limitations, suggests that most of the world's population consume a moderate range of dietary sodium (2.3-4.6g/day; 1-2 teaspoons of salt) that is not associated with increased cardiovascular risk, and that the risk of cardiovascular disease increases when sodium intakes exceed 5 g/day. While current evidence has limitations, and there are differences of opinion in interpretation of existing evidence, it is reasonable, based upon observational studies, to suggest a population-level mean target of <5 g/day in populations with mean sodium intake of >5 g/day, while awaiting the results of large randomized controlled trials of sodium reduction on incidence of cardiovascular events and mortality.

Older adult pedestrian trauma: A systematic review, meta-analysis, and GRADE assessment of injury health outcomes from an aggregate study sample of 1 million pedestrians

This systematic review sought to assess older adult pedestrian injury severity, injury by anatomical location and incidence proportions, including comparisons to younger age groups when available and provide an analysis of the quality of the existing evidence. A structured search was conducted in PubMed, Embase, Scopus, CINAHL, PsycInfo, AMED, Web of Science, LILACS and TRID. STROBE was used to assess the reporting quality of the included studies. Random-effect model meta-analysis served to obtain pooled relative risk, incidence proportions and standardized mean differences for different outcomes due to pedestrian crashes comparing older and younger pedestrians, while meta-analyses could not be conducted for pedestrian falls. We screened 7460 records of which 60 studies (1,012,041 pedestrians) were included in the review. Injured pedestrians 60+ compared to those <60 were found to have a higher relative risk of severe injury (pooled relative risk RR 1.6, 95 % CI: 1.4-2.0 p < 0.001), critical care admission (pooled RR 1.5, 95 %CI: 1.3-1.8 p < 0.001), and fatality (pooled RR of 3.7, 95 % CI: 3.0-4.5 p < 0.001). Pedestrians 60+ also had higher incidence rates of pedestrian falls causing higher injury severity. GRADE was used to evaluate evidence quality, with the results suggesting that the overall quality of the evidence supporting these findings was low. Further research is needed to understand health risks associated with older pedestrian trauma and to develop effective risk management strategies.

The International Classification of Health Interventions: an 'epistemic hub' for use in public health

The current lack of a common basis for collecting data on population-level prevention and health promotion interventions causes public health to be relatively invisible within broader health systems, making it vulnerable to funding cuts when there is pressure to reduce spending. Further, the inconsistent use of terms for describing interventions hinders knowledge translation and building an evidence base for public health practice and policy. The International Classification of Health Interventions (ICHI), being developed by the World Health Organization, is a standard statistical classification for interventions across the full scope of health systems. ICHI has potential to meet the need for a common language and structure for describing and capturing information about prevention and health promotion interventions. We report on a developmental appraisal conducted to examine the strengths and limitations of ICHI for coding interventions delivered for public health purposes. Our findings highlight classification challenges in relation to: consistently identifying separate components within multi-component interventions; operationalizing the ICHI concept of intervention target when there are intermediary targets as well as an ultimate target; coding an intervention component that involves more than one ICHI target or action; and standardising what is being counted. We propose that, alongside its purpose as a statistical classification, ICHI can play a valuable role as an 'epistemic hub', to be used flexibly by public health actors to meet a range of information needs, and as a basis for improved communication and exchange.

Challenges in applying the GRADE approach in public health guidelines and systematic reviews: a concept article from the GRADE Public Health Group

BACKGROUND AND OBJECTIVE

This article explores the need for conceptual advances and practical guidance in the application of the GRADE approach within public health contexts.

METHODS

We convened an expert workshop and conducted a scoping review to identify challenges experienced by GRADE users in public health contexts. We developed this concept article through thematic analysis and an iterative process of consultation and discussion conducted with members electronically and at three GRADE Working Group meetings.

RESULTS

Five priority issues can pose challenges for public health guideline developers and systematic reviewers when applying GRADE: (1) incorporating the perspectives of diverse stakeholders; (2) selecting and prioritizing health and "nonhealth" outcomes; (3) interpreting outcomes and identifying a threshold for decision-making; (4) assessing certainty of evidence from diverse sources, including nonrandomized studies; and (5) addressing implications for decision makers, including concerns about conditional recommendations. We illustrate these challenges with examples from public health guidelines and systematic reviews, identifying gaps where conceptual advances may facilitate the consistent application or further development of the methodology and provide solutions.

CONCLUSION

The GRADE Public Health Group will respond to these challenges with solutions that are coherent with existing guidance and can be consistently implemented across public health decision-making contexts.

Variations in processes for guideline adaptation: a qualitative study of World Health Organization staff experiences in implementing guidelines

BACKGROUND

The World Health Organisation (WHO) publishes a large number of clinical practice and public health guidelines to promote evidence-based practice across the world. Due to the variety of health system capacities and contextual issues in different regions and countries, adapting the recommendations in the guidelines to the local situation is vital for the success of their implementation. We aim to understand the range of experiences with guideline adaptation from the perspectives of those working in WHO regional and country offices. Our findings will inform development of guidance on how to improve adaptability of WHO guidelines.

METHODS

A grounded theory-informed, qualitative study was carried out between March 2018 and December 2018. Seventeen semi-structured interviews were conducted with participants who included WHO guideline developers and staff in the headquarters, regional and country offices recruited from a sample of published WHO guidelines. Participants were eligible for recruitment if they had recent experience in clinical practice or public health guideline implementation. Deidentified transcripts of these interview were analysed through three cycles of coding.

RESULTS

We categorised the adaptation processes described by the participants into two dominant models along a spectrum of guideline adaptation processes. First, the Copy or Customise Model is a pragmatic approach of either copying or customising WHO guidelines to suit local needs. This is done by local health authorities and/or clinicians directly through consultations with WHO staff. Selections and adjustments of guideline recommendations are made according to what the implementers deemed important, feasible and applicable through the consensus discussions. Second, the Capacity Building Model focuses on WHO building local capacity in evidence synthesis methods and adaptation frameworks to support local development of a national guideline informed by international guidelines.

CONCLUSIONS

In comparing and contrasting these two models of guideline adaptation, we outline the different kinds of support from WHO that may be necessary to improve the effectiveness and efficiency of the respective models. We also suggest clarifications in the descriptions of the process of guideline adaptation in WHO and academic literature, to help guideline adaptors and implementers decide on the appropriate course of action according to their specific circumstances.

ETHICS

This project was conducted with ethics approval from The University of Sydney (Project number: 2017/723) and WHO (Protocol ID: 00001).

"It's Not Smooth Sailing": Bridging the Gap Between Methods and Content Expertise in Public Health Guideline Development

BACKGROUND

The development of reliable, high quality health-related guidelines depends on explicit and transparent processes, methods aimed at minimising risks of bias and the inclusion of all relevant expertise and perspectives. While the methodological aspects of guidelines have been a focus to improve their quality, less is known about the social processes involved, for example, how guideline group members interact and communicate with one another, and how the evidence is considered in informing recommendations. With this in in mind, we aimed to empirically examine the perspectives and experiences of the key participants involved in developing public health guidelines for the Australian National Health and Medical Research Council (NHMRC).

DESIGN

This study was conducted using constructivist grounded theory as described by Charmaz, which informed our sampling, data collection, coding and analysis of interviews with key participants involved in developing public health guidelines.

SETTING

Australian public health guidelines commissioned by the NHMRC.

PARTICIPANTS

Twenty experts that were involved in Australian NHMRC public health guideline development, including working committee members with content topic expertise (n=16) and members of evidence review groups responsible for evaluating the evidence (n=4).

RESULTS

Public health guideline development in Australia is a divided process. The division is driven by 3 related factors: the divergent disciplinary background and expertise that each group brings to the process; the methodological limitations of the framework, inherited from clinical medicine, that is used to assess the evidence; and barriers to communication between content experts and evidence reviewers around respective roles and methodological limitations.

CONCLUSION

Our findings suggest several improvements for a more functional and unified guideline development process: greater education of the working committee on the methodological process employed to evaluate evidence, improved communication on the role of the evidence review groups and better facilitation of the process so that the evidence review groups feel their contribution is valued.

Use of the GRADE approach in health policymaking and evaluation: a scoping review of nutrition and physical activity policies

BACKGROUND

Nutrition and physical activity policies have the potential to influence lifestyle patterns and reduce the burden of non-communicable diseases. In the world of health-related guidelines, GRADE (Grading of Recommendations, Assessment, Development and Evaluation) is the most widely used approach for assessing the certainty of evidence and determining the strength of recommendations. Thus, it is relevant to explore its usefulness also in the process of nutrition and physical activity policymaking and evaluation. The purpose of this scoping review was (i) to generate an exemplary overview of documents using the GRADE approach in the process of nutrition and physical activity policymaking and evaluation, (ii) to find out how the GRADE approach has been applied, and (iii) to explore which facilitators of and barriers to the use of GRADE have been described on the basis of the identified documents. The overarching aim of this work is to work towards improving the process of evidence-informed policymaking in the areas of dietary behavior, physical activity, and sedentary behavior.

METHODS

A scoping review was conducted according to current reporting standards. MEDLINE via Ovid, the Cochrane Library, and Web of Science were systematically searched up until 4 July 2019. Documents describing a body of evidence which was assessed for the development or evaluation of a policy, including documents labeled as "guidelines," or systematic reviews used to inform policymaking were included.

RESULTS

Thirty-six documents were included. Overall, 313 GRADE certainty of evidence ratings were identified in systematic reviews and guidelines; the strength of recommendations/policies was assessed in four documents, and six documents mentioned facilitators or barriers for the use of GRADE. The major reported barrier was the initial low starting level of a body of evidence from non-randomized studies when assessing the certainty of evidence.

CONCLUSION

This scoping review found that the GRADE approach has been used for policy evaluations, in the evaluation of the effectiveness of policy-relevant interventions (policymaking), as well as in the development of guidelines intended to guide policymaking. Several areas for future research were identified to explore the use of GRADE in health policymaking and evaluation.

Interventions to reduce ambient air pollution and their effects on health: An abridged Cochrane systematic review

BACKGROUND

A broad range of interventions have been implemented to improve ambient air quality, and many of these have been evaluated. Yet to date no systematic review has been conducted to identify and synthesize these studies. In this systematic review, we assess the effectiveness of interventions in reducing ambient particulate matter air pollution and improving adverse health outcomes.

METHODS

We searched a range of electronic databases across multiple disciplines, as well as grey literature databases, trial registries, reference lists of included studies and the contents of relevant journals, through August 2016. Eligible for inclusion were randomized and cluster randomized controlled trials, as well as several non-randomized study designs often used for evaluating air quality interventions. We included studies that evaluated interventions targeting industrial, residential, vehicular and multiple sources, with respect to their effect on mortality, morbidity and the concentrations of particulate matter (PM - including PM₁₀, PM_2.5, coarse particulate matter and combustion-related PM), as well as several criteria pollutants, including ozone, carbon monoxide, nitrogen oxides, nitrogen dioxide, nitric oxide and sulphur dioxide. We did not restrict studies based on the population, setting or comparison. Two authors independently assessed studies for inclusion, extracted data and assessed risk of bias. We assessed risk of bias using the Graphic Appraisal Tool for Epidemiological studies (GATE) for correlation studies, as modified and employed by the UK National Institute for Health and Care Excellence. We synthesized evidence narratively, as well as graphically using harvest plots. We assessed the certainty of evidence using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) system.

RESULTS

We included 42 studies assessing 38 unique interventions. These comprised a heterogeneous mix of interventions, including those aiming to address industrial sources (n = 5; e.g. the closure of a factory), residential sources (n = 7; e.g. coal ban), vehicular sources (n = 22; e.g. low emission zones), and multiple sources (n = 4; e.g. tailored measures that target both local traffic and industrial polluters). Evidence for effectiveness was mixed. Most included studies observed either no significant association or an association favoring the intervention, with little evidence that the assessed interventions might be harmful.

CONCLUSIONS

Given the heterogeneity across interventions, outcomes, and methods, it was difficult to derive overall conclusions regarding the effectiveness of interventions in terms of improved air quality or health. Some evidence suggests that interventions are associated with improvements in air quality and human health, with very little evidence suggesting interventions were harmful. The evidence base highlights the challenges related to establishing the effectiveness of specific air pollution interventions on outcomes. It also points to the need for improved study design and analysis methods, as well as more uniform evaluations. The prospective planning of evaluations and an evaluation component built into the design and implementation of interventions may also be particularly beneficial.

Interventions to reduce ambient particulate matter air pollution and their effect on health

BACKGROUND

Ambient air pollution is associated with a large burden of disease in both high-income countries (HICs) and low- and middle-income countries (LMICs). To date, no systematic review has assessed the effectiveness of interventions aiming to reduce ambient air pollution.

OBJECTIVES

To assess the effectiveness of interventions to reduce ambient particulate matter air pollution in reducing pollutant concentrations and improving associated health outcomes.

SEARCH METHODS

We searched a range of electronic databases with diverse focuses, including health and biomedical research (CENTRAL, Cochrane Public Health Group Specialised Register, MEDLINE, Embase, PsycINFO), multidisciplinary research (Scopus, Science Citation Index), social sciences (Social Science Citation Index), urban planning and environment (Greenfile), and LMICs (Global Health Library regional indexes, WHOLIS). Additionally, we searched grey literature databases, multiple online trial registries, references of included studies and the contents of relevant journals in an attempt to identify unpublished and ongoing studies, and studies not identified by our search strategy. The final search date for all databases was 31 August 2016.

SELECTION CRITERIA

Eligible for inclusion were randomized and cluster randomized controlled trials, as well as several non-randomized study designs, including controlled interrupted time-series studies (cITS-EPOC), interrupted time-series studies adhering to EPOC standards (ITS-EPOC), interrupted time-series studies not adhering to EPOC standards (ITS), controlled before-after studies adhering to EPOC standards (CBA-EPOC), and controlled before-after studies not adhering to EPOC standards (CBA); these were classified as main studies. Additionally, we included uncontrolled before-after studies (UBA) as supporting studies. We included studies that evaluated interventions to reduce ambient air pollution from industrial, residential, vehicular and multiple sources, with respect to their effect on mortality, morbidity and several air pollutant concentrations. We did not restrict studies based on the population, setting or comparison.

DATA COLLECTION AND ANALYSIS

After a calibration exercise among the author team, two authors independently assessed studies for inclusion, extracted data and assessed risk of bias. We conducted data extraction, risk of bias assessment and evidence synthesis only for main studies; we mapped supporting studies with regard to the types of intervention and setting. To assess risk of bias, we used the Graphic Appraisal Tool for Epidemiological studies (GATE) for correlation studies, as modified and employed by the Centre for Public Health Excellence at the UK National Institute for Health and Care Excellence (NICE). For each intervention category, i.e. those targeting industrial, residential, vehicular and multiple sources, we synthesized evidence narratively, as well as graphically using harvest plots.

MAIN RESULTS

We included 42 main studies assessing 38 unique interventions. These were heterogeneous with respect to setting; interventions were implemented in countries across the world, but most (79%) were implemented in HICs, with the remaining scattered across LMICs. Most interventions (76%) were implemented in urban or community settings.We identified a heterogeneous mix of interventions, including those aiming to address industrial (n = 5), residential (n = 7), vehicular (n = 22), and multiple sources (n = 4). Some specific interventions, such as low emission zones and stove exchanges, were assessed by several studies, whereas others, such as a wood burning ban, were only assessed by a single study.Most studies assessing health and air quality outcomes used routine monitoring data. Studies assessing health outcomes mostly investigated effects in the general population, while few studies assessed specific subgroups such as infants, children and the elderly. No identified studies assessed unintended or adverse effects.The judgements regarding the risk of bias of studies were mixed. Regarding health outcomes, we appraised eight studies (47%) as having no substantial risk of bias concerns, five studies (29%) as having some risk of bias concerns, and four studies (24%) as having serious risk of bias concerns. Regarding air quality outcomes, we judged 11 studies (31%) as having no substantial risk of bias concerns, 16 studies (46%) as having some risk of bias concerns, and eight studies (23%) as having serious risk of bias concerns.The evidence base, comprising non-randomized studies only, was of low or very low certainty for all intervention categories and primary outcomes. The narrative and graphical synthesis showed that evidence for effectiveness was mixed across the four intervention categories. For interventions targeting industrial, residential and multiple sources, a similar pattern emerged for both health and air quality outcomes, with essentially all studies observing either no clear association in either direction or a significant association favouring the intervention. The evidence base for interventions targeting vehicular sources was more heterogeneous, as a small number of studies did observe a significant association favouring the control. Overall, however, the evidence suggests that the assessed interventions do not worsen air quality or health.

AUTHORS' CONCLUSIONS

Given the heterogeneity across interventions, outcomes, and methods, it was difficult to derive overall conclusions regarding the effectiveness of interventions in terms of improved air quality or health. Most included studies observed either no significant association in either direction or an association favouring the intervention, with little evidence that the assessed interventions might be harmful. The evidence base highlights the challenges related to establishing a causal relationship between specific air pollution interventions and outcomes. In light of these challenges, the results on effectiveness should be interpreted with caution; it is important to emphasize that lack of evidence of an association is not equivalent to evidence of no association.We identified limited evidence for several world regions, notably Africa, the Middle East, Eastern Europe, Central Asia and Southeast Asia; decision-makers should prioritize the development and implementation of interventions in these settings. In the future, as new policies are introduced, decision-makers should consider a built-in evaluation component, which could facilitate more systematic and comprehensive evaluations. These could assess effectiveness, but also aspects of feasibility, fidelity and acceptability.The production of higher quality and more uniform evidence would be helpful in informing decisions. Researchers should strive to sufficiently account for confounding, assess the impact of methodological decisions through the conduct and communication of sensitivity analyses, and improve the reporting of methods, and other aspects of the study, most importantly the description of the intervention and the context in which it is implemented.

Considerations of complexity in rating certainty of evidence in systematic reviews: a primer on using the GRADE approach in global health

Public health interventions and health technologies are commonly described as 'complex', as they involve multiple interacting components and outcomes, and their effects are largely influenced by contextual interactions and system-level processes. Systematic reviewers and guideline developers evaluating the effects of these complex interventions and technologies report difficulties in using existing methods and frameworks, such as the Grading of Recommendations Assessment, Development and Evaluation (GRADE). As part of a special series of papers on implications of complexity in the WHO guideline development, this paper serves as a primer on how to consider sources of complexity when using the GRADE approach to rate certainty of evidence. Relevant sources of complexity in systematic reviews, health technology assessments and guidelines of public health are outlined and mapped onto the reported difficulties in rating the estimates of the effect of these interventions. Recommendations on how to address these difficulties are further outlined, and the need for an integrated use of GRADE from the beginning of the review or guideline development is emphasised. The content of this paper is informed by the existing GRADE guidance, an ongoing research project on considering sources of complexity when applying the GRADE approach to rate certainty of evidence in systematic reviews and the review authors' own experiences with using GRADE.

The WHO-INTEGRATE evidence to decision framework version 1.0: integrating WHO norms and values and a complexity perspective

INTRODUCTION

Evidence-to-decision (EtD) frameworks intend to ensure that all criteria of relevance to a health decision are systematically considered. This paper, part of a series commissioned by the WHO, reports on the development of an EtD framework that is rooted in WHO norms and values, reflective of the changing global health landscape, and suitable for a range of interventions and complexity features. We also sought to assess the value of this framework to decision-makers at global and national levels, and to facilitate uptake through suggestions on how to prioritise criteria and methods to collect evidence.

METHODS

In an iterative, principles-based approach, we developed the framework structure from WHO norms and values. Preliminary criteria were derived from key documents and supplemented with comprehensive subcriteria obtained through an overview of systematic reviews of criteria employed in health decision-making. We assessed to what extent the framework can accommodate features of complexity, and conducted key informant interviews among WHO guideline developers. Suggestions on methods were drawn from the literature and expert consultation.

RESULTS

The new WHO-INTEGRATE (INTEGRATe Evidence) framework comprises six substantive criteria-balance of health benefits and harms, human rights and sociocultural acceptability, health equity, equality and non-discrimination, societal implications, financial and economic considerations, and feasibility and health system considerations-and the meta-criterion quality of evidence. It is intended to facilitate a structured process of reflection and discussion in a problem-specific and context-specific manner from the start of a guideline development or other health decision-making process. For each criterion, the framework offers a definition, subcriteria and example questions; it also suggests relevant primary research and evidence synthesis methods and approaches to assessing quality of evidence.

CONCLUSION

The framework is deliberately labelled version 1.0. We expect further modifications based on focus group discussions in four countries, example applications and input across concerned disciplines.

Taking account of context in systematic reviews and guidelines considering a complexity perspective

Systematic review teams and guideline development groups face considerable challenges when considering context within the evidence production process. Many complex interventions are context-dependent and are frequently evaluated within considerable contextual variation and change. This paper considers the extent to which current tools used within systematic reviews and guideline development are suitable in meeting these challenges. The paper briefly reviews strengths and weaknesses of existing approaches to specifying context. Illustrative tools are mapped to corresponding stages of the systematic review process. Collectively, systematic review and guideline production reveals a rich diversity of frameworks and tools for handling context. However, current approaches address only specific elements of context, are derived from primary studies which lack information or have not been tested within systematic reviews. A hypothetical example is used to illustrate how context could be integrated throughout the guideline development process. Guideline developers and evidence synthesis organisations should select an appropriate level of contextual detail for their specific guideline that is parsimonious and yet sensitive to health systems contexts and the values, preferences and needs of their target populations.

Individual-, family-, and school-level interventions targeting multiple risk behaviours in young people

BACKGROUND

Engagement in multiple risk behaviours can have adverse consequences for health during childhood, during adolescence, and later in life, yet little is known about the impact of different types of interventions that target multiple risk behaviours in children and young people, or the differential impact of universal versus targeted approaches. Findings from systematic reviews have been mixed, and effects of these interventions have not been quantitatively estimated.

OBJECTIVES

To examine the effects of interventions implemented up to 18 years of age for the primary or secondary prevention of multiple risk behaviours among young people.

SEARCH METHODS

We searched 11 databases (Australian Education Index; British Education Index; Campbell Library; Cumulative Index to Nursing and Allied Health Literature (CINAHL); Cochrane Central Register of Controlled Trials (CENTRAL), in the Cochrane Library; Embase; Education Resource Information Center (ERIC); International Bibliography of the Social Sciences; MEDLINE; PsycINFO; and Sociological Abstracts) on three occasions (2012, 2015, and 14 November 2016)). We conducted handsearches of reference lists, contacted experts in the field, conducted citation searches, and searched websites of relevant organisations.

SELECTION CRITERIA

We included randomised controlled trials (RCTs), including cluster RCTs, which aimed to address at least two risk behaviours. Participants were children and young people up to 18 years of age and/or parents, guardians, or carers, as long as the intervention aimed to address involvement in multiple risk behaviours among children and young people up to 18 years of age. However, studies could include outcome data on children > 18 years of age at the time of follow-up. Specifically,we included studies with outcomes collected from those eight to 25 years of age. Further, we included only studies with a combined intervention and follow-up period of six months or longer. We excluded interventions aimed at individuals with clinically diagnosed disorders along with clinical interventions. We categorised interventions according to whether they were conducted at the individual level; the family level; or the school level.

DATA COLLECTION AND ANALYSIS

We identified a total of 34,680 titles, screened 27,691 articles and assessed 424 full-text articles for eligibility. Two or more review authors independently assessed studies for inclusion in the review, extracted data, and assessed risk of bias.We pooled data in meta-analyses using a random-effects (DerSimonian and Laird) model in RevMan 5.3. For each outcome, we included subgroups related to study type (individual, family, or school level, and universal or targeted approach) and examined effectiveness at up to 12 months' follow-up and over the longer term (> 12 months). We assessed the quality and certainty of evidence using the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) approach.

MAIN RESULTS

We included in the review a total of 70 eligible studies, of which a substantial proportion were universal school-based studies (n = 28; 40%). Most studies were conducted in the USA (n = 55; 79%). On average, studies aimed to prevent four of the primary behaviours. Behaviours that were most frequently addressed included alcohol use (n = 55), drug use (n = 53), and/or antisocial behaviour (n = 53), followed by tobacco use (n = 42). No studies aimed to prevent self-harm or gambling alongside other behaviours.Evidence suggests that for multiple risk behaviours, universal school-based interventions were beneficial in relation to tobacco use (odds ratio (OR) 0.77, 95% confidence interval (CI) 0.60 to 0.97; n = 9 studies; 15,354 participants) and alcohol use (OR 0.72, 95% CI 0.56 to 0.92; n = 8 studies; 8751 participants; both moderate-quality evidence) compared to a comparator, and that such interventions may be effective in preventing illicit drug use (OR 0.74, 95% CI 0.55 to 1.00; n = 5 studies; 11,058 participants; low-quality evidence) and engagement in any antisocial behaviour (OR 0.81, 95% CI 0.66 to 0.98; n = 13 studies; 20,756 participants; very low-quality evidence) at up to 12 months' follow-up, although there was evidence of moderate to substantial heterogeneity (I² = 49% to 69%). Moderate-quality evidence also showed that multiple risk behaviour universal school-based interventions improved the odds of physical activity (OR 1.32, 95% CI 1.16 to 1.50; I² = 0%; n = 4 studies; 6441 participants). We considered observed effects to be of public health importance when applied at the population level. Evidence was less certain for the effects of such multiple risk behaviour interventions for cannabis use (OR 0.79, 95% CI 0.62 to 1.01; P = 0.06; n = 5 studies; 4140 participants; I² = 0%; moderate-quality evidence), sexual risk behaviours (OR 0.83, 95% CI 0.61 to 1.12; P = 0.22; n = 6 studies; 12,633 participants; I² = 77%; low-quality evidence), and unhealthy diet (OR 0.82, 95% CI 0.64 to 1.06; P = 0.13; n = 3 studies; 6441 participants; I² = 49%; moderate-quality evidence). It is important to note that some evidence supported the positive effects of universal school-level interventions on three or more risk behaviours.For most outcomes of individual- and family-level targeted and universal interventions, moderate- or low-quality evidence suggests little or no effect, although caution is warranted in interpretation because few of these studies were available for comparison (n ≤ 4 studies for each outcome).Seven studies reported adverse effects, which involved evidence suggestive of increased involvement in a risk behaviour among participants receiving the intervention compared to participants given control interventions.We judged the quality of evidence to be moderate or low for most outcomes, primarily owing to concerns around selection, performance, and detection bias and heterogeneity between studies.

AUTHORS' CONCLUSIONS

Available evidence is strongest for universal school-based interventions that target multiple- risk behaviours, demonstrating that they may be effective in preventing engagement in tobacco use, alcohol use, illicit drug use, and antisocial behaviour, and in improving physical activity among young people, but not in preventing other risk behaviours. Results of this review do not provide strong evidence of benefit for family- or individual-level interventions across the risk behaviours studied. However, poor reporting and concerns around the quality of evidence highlight the need for high-quality multiple- risk behaviour intervention studies to further strengthen the evidence base in this field.

Institutionalising an evidence-informed approach to guideline development: progress and challenges at the World Health Organization

This study explored experiences, perceptions and views among World Health Organization (WHO) staff about the changes, progress and challenges brought by the guideline development reforms initiated in 2007. Thirty-five semistructured interviews were conducted with senior WHO staff. Sixteen of the interviewees had in-depth experience with WHO's formal guideline development process. Thematic analysis was conducted to identify key themes in the qualitative data, and these were interpreted in the context of the existing literature on WHO's guideline development processes. First, the reforms were seen to have transformed and improved the quality of WHO's guidelines. Second, independent evaluation and feedback by the Guidelines Review Committee (GRC) was described to have strengthened the legitimacy of WHO's recommendations. Third, WHO guideline development processes are not yet designed to systematically make use of all types of research evidence needed to inform decisions about health systems and public health interventions. For example, several interviewees expressed dissatisfaction with the insufficient attention paid to qualitative evidence and evidence from programme experience, and how the Grading of Recommendations Assessment, Development and Evaluation (GRADE) process evaluates the quality of evidence from non-randomised study designs, while others believed that GRADE was just not properly understood or applied. Fourth, some staff advocated for a more centralised quality assurance process covering all outputs from WHO's departments and scientific advisory committees, especially to eliminate strategic efforts aimed at bypassing the GRC's requirements. Overall, the 'culture change' senior WHO staff called for over 10 years ago appears to have gradually spread throughout the organisation. However, at least two major challenges remain: (1) ensuring that all issued advice benefits from independent evaluation, monitoring and feedback for quality and (2) designing guideline development processes to better acquire, assess, adapt and apply the full range of evidence that can inform recommendations on health systems and public health interventions.

Facilitating evidence uptake: development and user testing of a systematic review summary format to inform public health decision-making in German-speaking countries

BACKGROUND

Systematic reviews are an important source of evidence for public health decision-making, but length and technical jargon tend to hinder their use. In non-English speaking countries, inaccessibility of information in the native language often represents an additional barrier. In line with our vision to strengthen evidence-based public health in the German-speaking world, we developed a German language summary format for systematic reviews of public health interventions and undertook user-testing with public health decision-makers in Germany, Austria and Switzerland.

METHODS

We used several guiding principles and core elements identified from the literature to produce a prototype summary format and applied it to a Cochrane review on the impacts of changing portion and package sizes on selection and consumption of food, alcohol and tobacco. Following a pre-test in each of the three countries, we carried out 18 user tests with public health decision-makers in Germany, Austria and Switzerland using the 'think-aloud' method. We analysed participants' comments according to the facets credibility, usability, understandability, usefulness, desirability, findability, identification and accessibility. We also identified elements that hindered the facile and satisfying use of the summary format, and revised it based on participants' feedback.

RESULTS

The summary format was well-received; participants particularly appreciated receiving information in their own language. They generally found the summary format useful and a credible source of information, but also signalled several barriers to a positive user experience such as an information-dense structure and difficulties with understanding statistical terms. Many of the identified challenges were addressed through modifications of the summary format, in particular by allowing for flexible length, placing more emphasis on key messages and relevance for public health practice, expanding the interpretation aid for statistical findings, providing a glossary of technical terms, and only including graphical GRADE ratings. Some barriers to uptake, notably the participants' wish for actionable recommendations and contextual information, could not be addressed.

CONCLUSIONS

Participants welcomed the initiative, but user tests also revealed their problems with understanding and interpreting the findings summarised in our prototype format. The revised summary format will be used to communicate the results of Cochrane reviews of public health interventions.

Rating the quality of a body of evidence on the effectiveness of health and social interventions: A systematic review and mapping of evidence domains

INTRODUCTION

Rating the quality of a body of evidence is an increasingly common component of research syntheses on intervention effectiveness. This study sought to identify and examine existing systems for rating the quality of a body of evidence on the effectiveness of health and social interventions.

METHODS

We used a multicomponent search strategy to search for full-length reports of systems for rating the quality of a body of evidence on the effectiveness of health and social interventions published in English from 1995 onward. Two independent reviewers extracted data from each eligible system on the evidence domains included, as well as the development and dissemination processes for each system.

RESULTS

Seventeen systems met our eligibility criteria. Across systems, we identified 13 discrete evidence domains: study design, study execution, consistency, measures of precision, directness, publication bias, magnitude of effect, dose-response, plausible confounding, analogy, robustness, applicability, and coherence. We found little reporting of rigorous procedures in the development and dissemination of evidence rating systems.

CONCLUSION

We identified 17 systems for rating the quality of a body of evidence on intervention effectiveness across health and social policy. Existing systems vary greatly in the domains they include and how they operationalize domains, and most have important limitations in their development and dissemination. The construct of the quality of the body of evidence was defined in a few systems largely extending the Grading of Recommendations Assessment, Development, and Evaluation approach. Grading of Recommendations Assessment, Development, and Evaluation was found to be unique in its comprehensive guidance, rigorous development, and dissemination strategy.

Application of the GRADE Approach in the Development of Guidelines and Recommendations in Genomic Medicine

A great deal of ambiguity exists in the development of guidelines for genomic applications used in clinical practice. The GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach has the potential to be applied in the guidelines and recommendations development process in genomics. Here, we discuss whether and how GRADE can be applied to address the challenges posed by the evidence-based guidelines and recommendations development process in genomics. To see how GRADE can complement to the current guidelines development in genomics, we compare and contrast GRADE with other approaches. GRADE differed from other methods by incorporating patient values and preferences and balance of consequences. We conclude that the groups trying to implement genomics into practice may gleam more information from applying the GRADE framework. However, it is not clear yet whether GRADE can address the issue of timeliness in terms of the differences between the time required for guidelines development and the rapid pace of genomics.

Meta-Analysis of Complex Interventions

Meta-analysis is a prominent method for estimating the effects of public health interventions, yet these interventions are often complex in ways that pose challenges to using conventional meta-analytic methods. This article discusses meta-analytic techniques that can be used in research syntheses on the effects of complex public health interventions. We first introduce the use of complexity frameworks to conceptualize public health interventions. We then present a menu of meta-analytic procedures for addressing various sources of complexity when answering questions about the effects of public health interventions in research syntheses. We conclude with a review of important practices and key resources for conducting meta-analyses on complex interventions, as well as future directions for research synthesis more generally. Overall, we argue that it is possible to conduct meaningful quantitative syntheses of research on the effects of public health interventions, though these meta-analyses may require the use of advanced techniques to properly consider and attend to issues of complexity.

Systematic review of interventions for improving the performance of community health workers in low-income and middle-income countries

OBJECTIVE

To systematically review and critically appraise the evidence for the effects of interventions to improve the performance of community health workers (CHWs) for community-based primary healthcare in low- and middle-income countries.

DESIGN

Systematic review following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

METHODS

19 electronic databases were searched with a highly sensitive prespecified strategy and the grey literature examined, completed July 2016. Randomised controlled trials evaluating interventions to improve CHW performance in low- and middle-income countries were included and appraised for risk of bias. Outcomes were biological and behavioural patient outcomes (primary), use of health services, quality of care provided by CHWs and CHW retention (secondary).

RESULTS

Two reviewers screened 8082 records; 14 evaluations were included. Due to heterogeneity and lack of clear outcome data, no meta-analysis was conducted. Results were presented in a narrative summary. The review found one study showing no effect on the biological outcomes of interest, though these moderate quality data may not be indicative of all biological outcomes. It also found moderate quality evidence of the efficacy of performance improvement interventions for (1) improving behavioural outcomes for patients, (2) improving use of services by increasing the absolute number of patients who access services and, perhaps, better identifying those who would benefit from such services and (3) improving CHW quality of care in terms of upstream measures like completion of prescribed activities and downstream measures like adherence to treatment protocols. Nearly half of studies were compound interventions, making it difficult to isolate the effects of individual performance improvement intervention components, though four specific strategies pertaining to recruitment, supervision, incentivisation and equipment were identified.

CONCLUSIONS

Variations in recruitment, supervision, incentivisation and equipment may improve CHW performance. Practitioners should, however, assess the relevance and feasibility of these strategies in their health setting prior to implementation. Component selection experiments on a greater range of interventions to improve performance ought to be conducted.

Model-based projection of health and economic effects of screening for hepatitis C in Canada

BACKGROUND

Because most hepatitis C virus (HCV) infections are asymptomatic and often unrecognized, screening for hepatitis C has been proposed as a plausible public health strategy. We examined the health and economic consequences of a selective one-time hepatitis C screening program for specific populations in the context of current treatment patterns.

METHODS

We used a state-transition model to evaluate 2 general strategies: no screening, and screen and treat with direct-acting antiviral agents. We examined these strategies for 4 different target populations (scenarios): 1) asymptomatic people not at high risk for HCV infection, 2) immigrant populations with high prevalence, 3) a birth cohort of people aged 25-64 years and 4) a birth cohort of people aged 45-64 years of age. We obtained model data from the published literature and expert opinions. We used a payer perspective, a lifetime time horizon and a 5% discount rate.

RESULTS

Screening would prevent 49.7%, 57.4%, 64.1% and 49.6% of HCV-related deaths over the lifetime of the cohort for scenarios 1, 2, 3 and 4, respectively. Screening would produce incremental-cost-effectiveness ratios between $31 468/quality-adjusted life-year and $50 490/quality-adjusted life-year. Probabilistic sensitivity analyses indicated that the chance that screening would be cost-effective at $50 000 willingness-to-pay threshold was 39.5%, 63.2%, 58.4% and 58.1% for scenarios 1, 2, 3 and 4, respectively.

INTERPRETATION

Our analyses suggest that a one-time hepatitis C screening and treatment program in Canada is likely to be cost-effective for scenarios 2, 3 and 4. The screening programs we have evaluated would identify asymptomatic people with chronic HCV infection and would enable medical treatment to be offered if needed before the development of advanced liver disease.

Towards a taxonomy of logic models in systematic reviews and health technology assessments: A priori, staged, and iterative approaches

The complexity associated with how interventions result-or fail to result-in outcomes and how context matters is increasingly recognised. Logic models provide an important tool for handling complexity, with contrasting uses in programme evaluation and evidence synthesis. To reconcile these, we developed an approach that combines the strengths of both traditions, propose a taxonomy of logic models, and provide guidance on how to choose between approaches and types of logic models in systematic reviews and health technology assessments (HTA). The taxonomy distinguishes 3 approaches (a priori, staged, and iterative) and 2 types (systems-based and process-orientated) of logic models. An a priori logic model is specified at the start of the systematic review/HTA and remains unchanged. With a staged logic model, the reviewer prespecifies several points, at which major data inputs require a subsequent version. An iterative logic model is continuously modified throughout the systematic review/HTA process. System-based logic models describe the system, in which the interaction between participants, intervention, and context takes place; process-orientated models display the causal pathways leading from the intervention to multiple outcomes. The proposed taxonomy of logic models offers an improved understanding of the advantages and limitations of logic models across the spectrum from a priori to fully iterative approaches. Choice of logic model should be informed by scope of evidence synthesis, presence/absence of clearly defined population, intervention, comparison, outcome (PICO) elements, and feasibility considerations. Applications across distinct interventions and methodological approaches will deliver good practice case studies and offer further insights on the choice and implementation of logic modelling approaches.

Critique of the review of 'Water fluoridation for the prevention of dental caries' published by the Cochrane Collaboration in 2015

The Cochrane Review on water fluoridation for the prevention of dental caries was published in 2015 and attracted considerable interest and comment, especially in countries with extensive water fluoridation programmes. The Review had two objectives: (i) to evaluate the effects of water fluoridation (artificial or natural) on the prevention of dental caries, and (ii) to evaluate the effects of water fluoridation (artificial or natural) on dental fluorosis. The authors concluded, inter alia, that there was very little contemporary evidence, meeting the Review's inclusion criteria, that evaluated the effectiveness of water fluoridation for the prevention of dental caries. The purpose of this critique is to examine the conduct of the above Review, and to put it into context in the wider body of evidence regarding the effectiveness of water fluoridation. While the overall conclusion that water fluoridation is effective in caries prevention agrees with previous reviews, many important public health questions could not be answered by the Review because of the restrictive criteria used to judge adequacy of study design and risk of bias. The potential benefits of using wider criteria in order to achieve a fuller understanding of the effectiveness of water fluoridation are discussed.

Including values in evidence-based policy making for breast screening: An empirically grounded tool to assist expert decision makers

Values are an important part of evidence-based decision making for health policy: they guide the type of evidence that is collected, how it is interpreted, and how important the conclusions are considered to be. Experts in breast screening (including clinicians, researchers, consumer advocates and senior administrators) hold differing values in relation to what is important in breast screening policy and practice, and committees may find it difficult to incorporate the complexity and variety of values into policy decisions. The decision making tool provided here is intended to assist with this process. The tool is modified from more general frameworks that are intended to assist with ethical decision making in public health, and informed by data drawn from previous empirical studies on values amongst Australian breast screening experts. It provides a structured format for breast screening committees to consider and discuss the values of themselves and others, suggests relevant topics for further inquiry and highlights areas of need for future research into the values of the public. It enables committees to publicly explain and justify their decisions with reference to values, improving transparency and accountability. It is intended to act alongside practices that seek to accommodate the values of individual women in the informed decision making process for personal decision making about participation in breast screening.

[Prevention among migrants: Participation, migrant sensitive strategies and programme characteristics]

Health promotion and prevention can contribute to a long, healthy life in populations both with and without migrant background. This paper provides an overview on migrant participation in prevention programmes in Germany. Furthermore, we describe migrant sensitive prevention strategies and characteristics of prevention programmes for migrants in Germany. With regard to participation in prevention programmes, lower vaccination rates are found among children and adolescents who migrated to Germany after birth. Among adults with a migrant background, we found lower participation in general health check-ups, oral health check-ups, cancer screening programs and influenza vaccination. Migrant sensitive prevention strategies address the visual style of the material, a target group specific risk communication, language requirements, a systematic involvement of the target group, and the recognition of deeply rooted sociocultural practices and beliefs. On analyzing a large database on prevention programs in Germany, we found only a few programmes that were exclusively targeted to migrant groups (0.6%). In 16.6% of the programs migrants were addressed as the target group among others. Compared to general population programs, programs for migrants were more often exclusively directed towards girls or women. Moreover, programs for migrants used community-based approaches more often and addressed different age groups. Although information on migrant participation in prevention programs and utilization of migrant sensitive strategies is still incomplete, we can assume that there is a need for diversity-oriented, migrant sensitive prevention.

Population-level interventions in government jurisdictions for dietary sodium reduction

BACKGROUND

Excess dietary sodium consumption is a risk factor for high blood pressure, stroke and cardiovascular disease. Currently, dietary sodium consumption in almost every country is too high. Excess sodium intake is associated with high blood pressure, which is common and costly and accounts for significant burden of disease. A large number of jurisdictions worldwide have implemented population-level dietary sodium reduction initiatives. No systematic review has examined the impact of these initiatives.

OBJECTIVES

• To assess the impact of population-level interventions for dietary sodium reduction in government jurisdictions worldwide.• To assess the differential impact of those initiatives by social and economic indicators.

SEARCH METHODS

We searched the following electronic databases from their start date to 5 January 2015: the Cochrane Central Register of Controlled Trials (CENTRAL); Cochrane Public Health Group Specialised Register; MEDLINE; MEDLINE In Process & Other Non-Indexed Citations; EMBASE; Effective Public Health Practice Project Database; Web of Science; Trials Register of Promoting Health Interventions (TRoPHI) databases; and Latin American Caribbean Health Sciences Literature (LILACS). We also searched grey literature, other national sources and references of included studies.This review was conducted in parallel with a comprehensive review of national sodium reduction efforts under way worldwide (Trieu 2015), through which we gained additional information directly from country contacts.We imposed no restrictions on language or publication status.

SELECTION CRITERIA

We included population-level initiatives (i.e. interventions that target whole populations, in this case, government jurisdictions, worldwide) for dietary sodium reduction, with at least one pre-intervention data point and at least one post-intervention data point of comparable jurisdiction. We included populations of all ages and the following types of study designs: cluster-randomised, controlled pre-post, interrupted time series and uncontrolled pre-post. We contacted study authors at different points in the review to ask for missing information.

DATA COLLECTION AND ANALYSIS

Two review authors extracted data, and two review authors assessed risk of bias for each included initiative.We analysed the impact of initiatives by using estimates of sodium consumption from dietary surveys or urine samples. All estimates were converted to a common metric: salt intake in grams per day. We analysed impact by computing the mean change in salt intake (grams per day) from pre-intervention to post-intervention.

MAIN RESULTS

We reviewed a total of 881 full-text documents. From these, we identified 15 national initiatives, including more than 260,000 people, that met the inclusion criteria. None of the initiatives were provided in lower-middle-income or low-income countries. All initiatives except one used an uncontrolled pre-post study design.Because of high levels of study heterogeneity (I2 > 90%), we focused on individual initiatives rather than on pooled results.Ten initiatives provided sufficient data for quantitative analysis of impact (64,798 participants). As required by the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) method, we graded the evidence as very low due to the risk of bias of the included studies, as well as variation in the direction and size of effect across the studies. Five of these showed mean decreases in average daily salt intake per person from pre-intervention to post-intervention, ranging from 1.15 grams/day less (Finland) to 0.35 grams/day less (Ireland). Two initiatives showed mean increase in salt intake from pre-intervention to post-intervention: Canada (1.66) and Switzerland (0.80 grams/day more per person. The remaining initiatives did not show a statistically significant mean change.Seven of the 10 initiatives were multi-component and incorporated intervention activities of a structural nature (e.g. food product reformulation, food procurement policy in specific settings). Of those seven initiatives, four showed a statistically significant mean decrease in salt intake from pre-intervention to post-intervention, ranging from Finland to Ireland (see above), and one showed a statistically significant mean increase in salt intake from pre-intervention to post-intervention (Switzerland; see above).Nine initiatives permitted quantitative analysis of differential impact by sex (men and women separately). For women, three initiatives (China, Finland, France) showed a statistically significant mean decrease, four (Austria, Netherlands, Switzerland, United Kingdom) showed no significant change and two (Canada, United States) showed a statistically significant mean increase in salt intake from pre-intervention to post-intervention. For men, five initiatives (Austria, China, Finland, France, United Kingdom) showed a statistically significant mean decrease, three (Netherlands, Switzerland, United States) showed no significant change and one (Canada) showed a statistically significant mean increase in salt intake from pre-intervention to post-intervention.Information was insufficient to indicate whether a differential change in mean salt intake occurred from pre-intervention to post-intervention by other axes of equity included in the PROGRESS framework (e.g. education, place of residence).We identified no adverse effects of these initiatives.The number of initiatives was insufficient to permit other subgroup analyses, including stratification by intervention type, economic status of country and duration (or start year) of the initiative.Many studies had methodological strengths, including large, nationally representative samples of the population and rigorous measurement of dietary sodium intake. However, all studies were scored as having high risk of bias, reflecting the observational nature of the research and the use of an uncontrolled study design. The quality of evidence for the main outcome was low. We could perform a sensitivity analysis only for impact.

AUTHORS' CONCLUSIONS

Population-level interventions in government jurisdictions for dietary sodium reduction have the potential to result in population-wide reductions in salt intake from pre-intervention to post-intervention, particularly if they are multi-component (more than one intervention activity) and incorporate intervention activities of a structural nature (e.g. food product reformulation), and particularly amongst men. Heterogeneity across studies was significant, reflecting different contexts (population and setting) and initiative characteristics. Implementation of future initiatives should embed more effective means of evaluation to help us better understand the variation in the effects.