Seeing the forests and the trees--innovative approaches to exploring heterogeneity in systematic reviews of complex interventions to enhance health system decision-making: a protocol

Evidence Synthesis for Complex Interventions Using Meta-Regression Models

A goal of evidence synthesis for trials of complex interventions is to inform the design or implementation of novel versions of complex interventions by predicting expected outcomes with each intervention version. Conventional aggregate data meta-analyses of studies comparing complex interventions have limited ability to provide such information. We argue that evidence synthesis for trials of complex interventions should forgo aspirations of estimating causal effects and instead model the response surface of study results to 1) summarize the available evidence and 2) predict the average outcomes of future studies or in new settings. We illustrate this modeling approach using data from a systematic review of diabetes quality improvement (QI) interventions involving at least 1 of 12 QI strategy components. We specify a series of meta-regression models to assess the association of specific components with the posttreatment outcome mean and compare the results to conventional meta-analysis approaches. Compared with conventional approaches, modeling the response surface of study results can better reflect the associations between intervention components and study characteristics with the posttreatment outcome mean. Modeling study results using a response surface approach offers a useful and feasible goal for evidence synthesis of complex interventions that rely on aggregate data.

New horizons in evidence synthesis for older adults

Evidence synthesis, embedded within a systematic review of the literature, is a well-established approach for collating and combining all the relevant information on a particular research question. A robust synthesis can establish the evidence base, which underpins best practice guidance. Such endeavours are frequently used by policymakers and practitioners to inform their decision making. Traditionally, an evidence synthesis of interventions consisted of a meta-analysis of quantitative data comparing two treatment alternatives addressing a specific and focussed clinical question. However, as the methods in the field have evolved, especially in response to the increasingly complex healthcare questions, more advanced evidence synthesis techniques have been developed. These can deal with extended data structures considering more than two treatment alternatives (network meta-analysis) and complex multicomponent interventions. The array of questions capable of being answered has also increased with specific approaches being developed for different evidence types including diagnostic, prognostic and qualitative data. Furthermore, driven by a desire for increasingly up-to-date evidence summaries, living systematic reviews have emerged. All of these methods can potentially have a role in informing older adult healthcare decisions. The aim of this review is to increase awareness and uptake of the increasingly comprehensive array of newer synthesis methods available and highlight their utility for answering clinically relevant questions in the context of older adult research, giving examples of where such techniques have already been effectively applied within the field. Their strengths and limitations are discussed, and we suggest user-friendly software options to implement the methods described.

Data visualisation approaches for component network meta-analysis: visualising the data structure

BACKGROUND

Health and social care interventions are often complex and can be decomposed into multiple components. Multicomponent interventions are often evaluated in randomised controlled trials. Across trials, interventions often have components in common which are given alongside other components which differ across trials. Multicomponent interventions can be synthesised using component NMA (CNMA). CNMA is limited by the structure of the available evidence, but it is not always straightforward to visualise such complex evidence networks. The aim of this paper is to develop tools to visualise the structure of complex evidence networks to support CNMA.

METHODS

We performed a citation review of two key CNMA methods papers to identify existing published CNMA analyses and reviewed how they graphically represent intervention complexity and comparisons across trials. Building on identified shortcomings of existing visualisation approaches, we propose three approaches to standardise visualising the data structure and/or availability of data: CNMA-UpSet plot, CNMA heat map, CNMA-circle plot. We use a motivating example to illustrate these plots.

RESULTS

We identified 34 articles reporting CNMAs. A network diagram was the most common plot type used to visualise the data structure for CNMA (26/34 papers), but was unable to express the complex data structures and large number of components and potential combinations of components associated with CNMA. Therefore, we focused visualisation development around representing the data structure of a CNMA more completely. The CNMA-UpSet plot presents arm-level data and is suitable for networks with large numbers of components or combinations of components. Heat maps can be utilised to inform decisions about which pairwise interactions to consider for inclusion in a CNMA model. The CNMA-circle plot visualises the combinations of components which differ between trial arms and offers flexibility in presenting additional information such as the number of patients experiencing the outcome of interest in each arm.

CONCLUSIONS

As CNMA becomes more widely used for the evaluation of multicomponent interventions, the novel CNMA-specific visualisations presented in this paper, which improve on the limitations of existing visualisations, will be important to aid understanding of the complex data structure and facilitate interpretation of the CNMA results.

Quality improvement strategies for diabetes care: Effects on outcomes for adults living with diabetes

BACKGROUND

There is a large body of evidence evaluating quality improvement (QI) programmes to improve care for adults living with diabetes. These programmes are often comprised of multiple QI strategies, which may be implemented in various combinations. Decision-makers planning to implement or evaluate a new QI programme, or both, need reliable evidence on the relative effectiveness of different QI strategies (individually and in combination) for different patient populations.

OBJECTIVES

To update existing systematic reviews of diabetes QI programmes and apply novel meta-analytical techniques to estimate the effectiveness of QI strategies (individually and in combination) on diabetes quality of care.

SEARCH METHODS

We searched databases (CENTRAL, MEDLINE, Embase and CINAHL) and trials registers (ClinicalTrials.gov and WHO ICTRP) to 4 June 2019. We conducted a top-up search to 23 September 2021; we screened these search results and 42 studies meeting our eligibility criteria are available in the awaiting classification section.

SELECTION CRITERIA

We included randomised trials that assessed a QI programme to improve care in outpatient settings for people living with diabetes. QI programmes needed to evaluate at least one system- or provider-targeted QI strategy alone or in combination with a patient-targeted strategy. - System-targeted: case management (CM); team changes (TC); electronic patient registry (EPR); facilitated relay of clinical information (FR); continuous quality improvement (CQI). - Provider-targeted: audit and feedback (AF); clinician education (CE); clinician reminders (CR); financial incentives (FI). - Patient-targeted: patient education (PE); promotion of self-management (PSM); patient reminders (PR). Patient-targeted QI strategies needed to occur with a minimum of one provider or system-targeted strategy.

DATA COLLECTION AND ANALYSIS

We dual-screened search results and abstracted data on study design, study population and QI strategies. We assessed the impact of the programmes on 13 measures of diabetes care, including: glycaemic control (e.g. mean glycated haemoglobin (HbA1c)); cardiovascular risk factor management (e.g. mean systolic blood pressure (SBP), low-density lipoprotein cholesterol (LDL-C), proportion of people living with diabetes that quit smoking or receiving cardiovascular medications); and screening/prevention of microvascular complications (e.g. proportion of patients receiving retinopathy or foot screening); and harms (e.g. proportion of patients experiencing adverse hypoglycaemia or hyperglycaemia). We modelled the association of each QI strategy with outcomes using a series of hierarchical multivariable meta-regression models in a Bayesian framework. The previous version of this review identified that different strategies were more or less effective depending on baseline levels of outcomes. To explore this further, we extended the main additive model for continuous outcomes (HbA1c, SBP and LDL-C) to include an interaction term between each strategy and average baseline risk for each study (baseline thresholds were based on a data-driven approach; we used the median of all baseline values reported in the trials). Based on model diagnostics, the baseline interaction models for HbA1c, SBP and LDL-C performed better than the main model and are therefore presented as the primary analyses for these outcomes. Based on the model results, we qualitatively ordered each QI strategy within three tiers (Top, Middle, Bottom) based on its magnitude of effect relative to the other QI strategies, where 'Top' indicates that the QI strategy was likely one of the most effective strategies for that specific outcome. Secondary analyses explored the sensitivity of results to choices in model specification and priors.  Additional information about the methods and results of the review are available as Appendices in an online repository. This review will be maintained as a living systematic review; we will update our syntheses as more data become available.

MAIN RESULTS

We identified 553 trials (428 patient-randomised and 125 cluster-randomised trials), including a total of 412,161 participants. Of the included studies, 66% involved people living with type 2 diabetes only. Participants were 50% female and the median age of participants was 58.4 years. The mean duration of follow-up was 12.5 months. HbA1c was the commonest reported outcome; screening outcomes and outcomes related to cardiovascular medications, smoking and harms were reported infrequently. The most frequently evaluated QI strategies across all study arms were PE, PSM and CM, while the least frequently evaluated QI strategies included AF, FI and CQI. Our confidence in the evidence is limited due to a lack of information on how studies were conducted.  Four QI strategies (CM, TC, PE, PSM) were consistently identified as 'Top' across the majority of outcomes. All QI strategies were ranked as 'Top' for at least one key outcome. The majority of effects of individual QI strategies were modest, but when used in combination could result in meaningful population-level improvements across the majority of outcomes. The median number of QI strategies in multicomponent QI programmes was three.  Combinations of the three most effective QI strategies were estimated to lead to the below effects:  - PR + PSM + CE: decrease in HbA1c by 0.41% (credibility interval (CrI) -0.61 to -0.22) when baseline HbA1c < 8.3%; - CM + PE + EPR: decrease in HbA1c by 0.62% (CrI -0.84 to -0.39) when baseline HbA1c > 8.3%;  - PE + TC + PSM: reduction in SBP by 2.14 mmHg (CrI -3.80 to -0.52) when baseline SBP < 136 mmHg; - CM + TC + PSM: reduction in SBP by 4.39 mmHg (CrI -6.20 to -2.56) when baseline SBP > 136 mmHg;  - TC + PE + CM: LDL-C lowering of 5.73 mg/dL (CrI -7.93 to -3.61) when baseline LDL < 107 mg/dL; - TC + CM + CR: LDL-C lowering by 5.52 mg/dL (CrI -9.24 to -1.89) when baseline LDL > 107 mg/dL. Assuming a baseline screening rate of 50%, the three most effective QI strategies were estimated to lead to an absolute improvement of 33% in retinopathy screening (PE + PR + TC) and 38% absolute increase in foot screening (PE + TC + Other).

AUTHORS' CONCLUSIONS

There is a significant body of evidence about QI programmes to improve the management of diabetes. Multicomponent QI programmes for diabetes care (comprised of effective QI strategies) may achieve meaningful population-level improvements across the majority of outcomes. For health system decision-makers, the evidence summarised in this review can be used to identify strategies to include in QI programmes. For researchers, this synthesis identifies higher-priority QI strategies to examine in further research regarding how to optimise their evaluation and effects. We will maintain this as a living systematic review.

Involving men and boys in family planning: A systematic review of the effective components and characteristics of complex interventions in low- and middle-income countries

BACKGROUND

Involving men and boys as both users and supporters of Family Planning (FP) is now considered essential for optimising maternal and child health outcomes. Evidence on how to engage men and boys to meet FP needs is therefore important.

OBJECTIVES

The main objective of this review was to assess the strength of evidence in the area and uncover the effective components and critical process- and system-level characteristics of successful interventions.

SEARCH METHODS

We searched nine electronic databases, seven grey literature databases, organisational websites, and the reference lists of systematic reviews relating to FP. To identify process evaluations and qualitative papers associated with the included experimental studies, we used Connected Papers and hand searches of reference lists.

SELECTION CRITERIA

Experimental and quasi-experimental studies of behavioural and service-level interventions involving males aged 10 years or over in low- and middle-income countries to increase uptake of FP methods were included in this review.

DATA COLLECTION AND ANALYSIS

Methodology was a causal chain analysis involving the development and testing of a logic model of intervention components based on stakeholder consultation and prior research. Qualitative and quantitative data relating to the evaluation studies and interventions were extracted based on the principles of 'effectiveness-plus' reviews. Quantitative analysis was undertaken using r with robust variance estimation (RVE), meta-analysis and meta-regression. Qualitative analysis involved 'best fit' framework synthesis.

RESULTS

We identified 8885 potentially relevant records and included 127 in the review. Fifty-nine (46%) of these were randomised trials, the remainder were quasi-experimental studies with a comparison group. Fifty-four percent of the included studies were assessed as having a high risk of bias. A meta-analysis of 72 studies (k = 265) showed that the included group of interventions had statistically significantly higher odds of improving contraceptive use when compared to comparison groups (odds ratio = 1.38, confidence interval = 1.21 to 1.57, prediction interval = 0.36 to 5.31, p < 0.0001), but there were substantial variations in the effect sizes of the studies (Q = 40,647, df = 264, p < 0.0001; I ² = 98%) and 73% was within cluster/study. Multi-variate meta-regression revealed several significant intervention delivery characteristics that moderate contraceptive use. These included community-based educational FP interventions, interventions delivered to women as well as men and interventions delivered by trained facilitators, professionals, or peers in community, home and community, or school settings. None of the eight identified intervention components or 33 combinations of components were significant moderators of effects on contraceptive use. Qualitative analysis highlighted some of the barriers and facilitators of effective models of FP that should be considered in future practice and research.

AUTHORS' CONCLUSIONS

FP interventions that involve men and boys alongside women and girls are effective in improving uptake and use of contraceptives. The evidence suggests that policy should continue to promote the involvement of men and boys in FP in ways that also promote gender equality. Recommendations for research include the need for evaluations during conflict and disease outbreaks, and evaluation of gender transformative interventions which engage men and boys as contraceptive users and supporters in helping to achieve desired family size, fertility promotion, safe conception, as well as promoting equitable family planning decision-making for women and girls.

The impact of hospital accreditation on the quality of healthcare: a systematic literature review

BACKGROUND

Accreditation is viewed as a reputable tool to evaluate and enhance the quality of health care. However, its effect on performance and outcomes remains unclear. This review aimed to identify and analyze the evidence on the impact of hospital accreditation.

METHODS

We systematically searched electronic databases (PubMed, CINAHL, PsycINFO, EMBASE, MEDLINE (OvidSP), CDSR, CENTRAL, ScienceDirect, SSCI, RSCI, SciELO, and KCI) and other sources using relevant subject headings. We included peer-reviewed quantitative studies published over the last two decades, irrespective of its design or language. Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, two reviewers independently screened initially identified articles, reviewed the full-text of potentially relevant studies, extracted necessary data, and assessed the methodological quality of the included studies using a validated tool. The accreditation effects were synthesized and categorized thematically into six impact themes.

RESULTS

We screened a total of 17,830 studies, of which 76 empirical studies that examined the impact of accreditation met our inclusion criteria. These studies were methodologically heterogeneous. Apart from the effect of accreditation on healthcare workers and particularly on job stress, our results indicate a consistent positive effect of hospital accreditation on safety culture, process-related performance measures, efficiency, and the patient length of stay, whereas employee satisfaction, patient satisfaction and experience, and 30-day hospital readmission rate were found to be unrelated to accreditation. Paradoxical results regarding the impact of accreditation on mortality rate and healthcare-associated infections hampered drawing firm conclusions on these outcome measures.

CONCLUSION

There is reasonable evidence to support the notion that compliance with accreditation standards has multiple plausible benefits in improving the performance in the hospital setting. Despite inconclusive evidence on causality, introducing hospital accreditation schemes stimulates performance improvement and patient safety. Efforts to incentivize and modernize accreditation are recommended to move towards institutionalization and sustaining the performance gains. PROSPERO registration number CRD42020167863.

Imputing intracluster correlation coefficients from a posterior predictive distribution is a feasible method of dealing with unit of analysis errors in a meta-analysis of cluster RCTs

BACKGROUND

Incorporating cluster randomized trials (CRTs) into meta-analyses is challenging because appropriate standard errors of study estimates accounting for clustering are not always reported. Systematic reviews of CRTs often use a single constant external estimate of the intraclass correlation coefficient (ICC) to adjust study estimate standard errors and facilitate meta-analyses; an approach that fails to account for possible variation of ICCs among studies and the imprecision with which they are estimated. Using a large systematic review of the effects of diabetes quality improvement interventions, we investigated whether we could better account for ICC variation and uncertainty in meta-analyzed effect estimates by imputing missing ICCs from a posterior predictive distribution constructed from a database of relevant ICCs.

METHODS

We constructed a dataset of ICC estimates from applicable studies. For outcomes with two or more available ICC estimates, we constructed posterior predictive ICC distributions in a Bayesian framework. For a selected continuous outcome, glycosylated hemoglobin (HbA1c), we compared the impact of incorporating a single constant ICC versus imputing ICCs drawn from the posterior predictive distribution when estimating the effect of intervention components on post treatment mean in a case study of diabetes quality improvement trials.

RESULTS

Using internal and external ICC estimates, we were able to construct a database of 59 ICCs for 12 of the 13 review outcomes (range 1-10 per outcome) and estimate the posterior predictive ICC distribution for 11 review outcomes. Synthesized results were not markedly changed by our approach for HbA1c.

CONCLUSION

Building posterior predictive distributions to impute missing ICCs is a feasible approach to facilitate principled meta-analyses of cluster randomized trials using prior data. Further work is needed to establish whether the application of these methods leads to improved review inferences for different reviews based on different factors (e.g., proportion of CRTs and CRTs with missing ICCs, different outcomes, variation and precision of ICCs).

Trialists perspectives on sustaining, spreading, and scaling-up of quality improvement interventions

BACKGROUND

Quality improvement (QI) evaluations rarely consider how a successful intervention can be sustained long term, nor how to spread or scale to other locations. A survey of authors of randomized trials of diabetes QI interventions included in an ongoing systematic review found that 78% of trials reported improved quality of care, but 40% of these trials were not sustained. This study explores why and how the effective interventions were sustained, spread, or scaled.

METHODS

A qualitative approach was used, focusing on case examples. Diabetes QI program trial authors were purposefully sampled and recruited for telephone interviews. Authors were eligible if they had completed the author survey, agreed to follow-up, and had a completed a diabetes QI trial they deemed "effective." Snowball sampling was used if the participant identified someone who could provide a different perspective on the same trial. Interviews were transcribed verbatim. Inductive thematic analysis was conducted to identify barriers and facilitators to sustainability, spread, and/or scale of the QI program, using case examples to show trajectories across projects and people.

RESULTS

Eleven of 44 eligible trialists participated in an interview. Four reported that the intervention was "sustained" and nine were "spread," however, interviews highlighted that these terms were interpreted differently over time and between participants. Participant stories highlighted the varied trajectories of how projects evolved and how some research careers adapted to increase impact. Three interacting themes, termed the "3C's," helped explain the variation in sustainability, spread, and scale: (i) understanding the concepts of implementation, sustainability, sustainment, spread, and scale; (ii) having the appropriate competencies; and (iii) the need for individual, organizational, and system capacity.

CONCLUSIONS

Challenges in defining sustainability, spread and scale make it difficult to fully understand impact. However, it is clear that from the beginning of intervention design, trialists need to understand the concepts and have the competency and capacity to plan for feasible and sustainable interventions that have potential to be sustained, spread and/or scaled if found to be effective.

Enhancing the translation of health behaviour change research into practice: a selective conceptual review of the synergy between implementation science and health psychology

Health psychology is at the forefront of developing and disseminating evidence, theories, and methods that have improved the understanding of health behaviour change. However, current dissemination approaches may be insufficient for promoting broader application and impact of this evidence to benefit the health of patients and the public. Nevertheless, behaviour change theory/methods typically directed towards health behaviours are now used in implementation science to understand and support behaviour change in individuals at different health system levels whose own behaviour impacts delivering evidence-based health behaviour change interventions. Despite contributing to implementation science, health psychology is perhaps doing less to draw from it. A redoubled focus on implementation science in health psychology could provide novel prospects for enhancing the impact of health behaviour change evidence. We report a Health Psychology Review-specific review-of-reviews of trials of health behaviour change interventions published from inception to April 2020. We identified 34 reviews and assessed whether implementation readiness of behaviour change interventions was discussed. We then narratively review how implementation science has integrated theory/methods from health psychology and related discipline. Finally, we demonstrate how greater synergy between implementation science and health psychology could promote greater follow-through on advances made in the science of health behaviour change.

Publication and related bias in quantitative health services and delivery research: a multimethod study

Background

Bias in the publication and reporting of research findings (referred to as publication and related bias here) poses a major threat in evidence synthesis and evidence-based decision-making. Although this bias has been well documented in clinical research, little is known about its occurrence and magnitude in health services and delivery research.

Objectives

To obtain empirical evidence on publication and related bias in quantitative health services and delivery research; to examine current practice in detecting/mitigating this bias in health services and delivery research systematic reviews; and to explore stakeholders’ perception and experiences concerning such bias.

Methods

The project included five distinct but interrelated work packages. Work package 1 was a systematic review of empirical and methodological studies. Work package 2 involved a survey (meta-epidemiological study) of randomly selected systematic reviews of health services and delivery research topics (n = 200) to evaluate current practice in the assessment of publication and outcome reporting bias during evidence synthesis. Work package 3 included four case studies to explore the applicability of statistical methods for detecting such bias in health services and delivery research. In work package 4 we followed up four cohorts of health services and delivery research studies (total n = 300) to ascertain their publication status, and examined whether publication status was associated with statistical significance or perceived ‘positivity’ of study findings. Work package 5 involved key informant interviews with diverse health services and delivery research stakeholders (n = 24), and a focus group discussion with patient and service user representatives (n = 8).

Results

We identified only four studies that set out to investigate publication and related bias in health services and delivery research in work package 1. Three of these studies focused on health informatics research and one concerned health economics. All four studies reported evidence of the existence of this bias, but had methodological weaknesses. We also identified three health services and delivery research systematic reviews in which findings were compared between published and grey/unpublished literature. These reviews found that the quality and volume of evidence and effect estimates sometimes differed significantly between published and unpublished literature. Work package 2 showed low prevalence of considering/assessing publication (43%) and outcome reporting (17%) bias in health services and delivery research systematic reviews. The prevalence was lower among reviews of associations than among reviews of interventions. The case studies in work package 3 highlighted limitations in current methods for detecting these biases due to heterogeneity and potential confounders. Follow-up of health services and delivery research cohorts in work package 4 showed positive association between publication status and having statistically significant or positive findings. Diverse views concerning publication and related bias and insights into how features of health services and delivery research might influence its occurrence were uncovered through the interviews with health services and delivery research stakeholders and focus group discussion conducted in work package 5.

Conclusions

This study provided prima facie evidence on publication and related bias in quantitative health services and delivery research. This bias does appear to exist, but its prevalence and impact may vary depending on study characteristics, such as study design, and motivation for conducting the evaluation. Emphasis on methodological novelty and focus beyond summative assessments may mitigate/lessen the risk of such bias in health services and delivery research. Methodological and epistemological diversity in health services and delivery research and changing landscape in research publication need to be considered when interpreting the evidence. Collection of further empirical evidence and exploration of optimal health services and delivery research practice are required.

Study registration

This study is registered as PROSPERO CRD42016052333 and CRD42016052366.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 33. See the NIHR Journals Library website for further project information.

What works to increase attendance for diabetic retinopathy screening? An evidence synthesis and economic analysis

BACKGROUND

Diabetic retinopathy screening (DRS) is effective but uptake is suboptimal.

OBJECTIVES

To determine the effectiveness of quality improvement (QI) interventions for DRS attendance; describe the interventions in terms of QI components and behaviour change techniques (BCTs); identify theoretical determinants of attendance; investigate coherence between BCTs identified in interventions and determinants of attendance; and determine the cost-effectiveness of QI components and BCTs for improving DRS.

DATA SOURCES AND REVIEW METHODS

Phase 1 - systematic review of randomised controlled trials (RCTs) evaluating interventions to increase DRS attendance (The Cochrane Library, MEDLINE, EMBASE and trials registers to February 2017) and coding intervention content to classify QI components and BCTs. Phase 2 - review of studies reporting factors influencing attendance, coded to theoretical domains (MEDLINE, EMBASE, PsycINFO and sources of grey literature to March 2016). Phase 3 - mapping BCTs (phase 1) to theoretical domains (phase 2) and an economic evaluation to determine the cost-effectiveness of BCTs or QI components.

RESULTS

Phase 1 - 7277 studies were screened, of which 66 RCTs were included in the review. Interventions were multifaceted and targeted patients, health-care professionals (HCPs) or health-care systems. Overall, interventions increased DRS attendance by 12% [risk difference (RD) 0.12, 95% confidence interval (CI) 0.10 to 0.14] compared with usual care, with substantial heterogeneity in effect size. Both DRS-targeted and general QI interventions were effective, particularly when baseline attendance levels were low. All commonly used QI components and BCTs were associated with significant improvements, particularly in those with poor attendance. Higher effect estimates were observed in subgroup analyses for the BCTs of 'goal setting (outcome, i.e. consequences)' (RD 0.26, 95% CI 0.16 to 0.36) and 'feedback on outcomes (consequences) of behaviour' (RD 0.22, 95% CI 0.15 to 0.29) in interventions targeting patients and of 'restructuring the social environment' (RD 0.19, 95% CI 0.12 to 0.26) and 'credible source' (RD 0.16, 95% CI 0.08 to 0.24) in interventions targeting HCPs. Phase 2 - 3457 studies were screened, of which 65 non-randomised studies were included in the review. The following theoretical domains were likely to influence attendance: 'environmental context and resources', 'social influences', 'knowledge', 'memory, attention and decision processes', 'beliefs about consequences' and 'emotions'. Phase 3 - mapping identified that interventions included BCTs targeting important barriers to/enablers of DRS attendance. However, BCTs targeting emotional factors around DRS were under-represented. QI components were unlikely to be cost-effective whereas BCTs with a high probability (≥ 0.975) of being cost-effective at a societal willingness-to-pay threshold of £20,000 per QALY included 'goal-setting (outcome)', 'feedback on outcomes of behaviour', 'social support' and 'information about health consequences'. Cost-effectiveness increased when DRS attendance was lower and with longer screening intervals.

LIMITATIONS

Quality improvement/BCT coding was dependent on descriptions of intervention content in primary sources; methods for the identification of coherence of BCTs require improvement.

CONCLUSIONS

Randomised controlled trial evidence indicates that QI interventions incorporating specific BCT components are associated with meaningful improvements in DRS attendance compared with usual care. Interventions generally used appropriate BCTs that target important barriers to screening attendance, with a high probability of being cost-effective. Research is needed to optimise BCTs or BCT combinations that seek to improve DRS attendance at an acceptable cost. BCTs targeting emotional factors represent a missed opportunity to improve attendance and should be tested in future studies.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42016044157 and PROSPERO CRD42016032990.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

Synthesising quantitative evidence in systematic reviews of complex health interventions

Public health and health service interventions are typically complex: they are multifaceted, with impacts at multiple levels and on multiple stakeholders. Systematic reviews evaluating the effects of complex health interventions can be challenging to conduct. This paper is part of a special series of papers considering these challenges particularly in the context of WHO guideline development. We outline established and innovative methods for synthesising quantitative evidence within a systematic review of a complex intervention, including considerations of the complexity of the system into which the intervention is introduced. We describe methods in three broad areas: non-quantitative approaches, including tabulation, narrative and graphical approaches; standard meta-analysis methods, including meta-regression to investigate study-level moderators of effect; and advanced synthesis methods, in which models allow exploration of intervention components, investigation of both moderators and mediators, examination of mechanisms, and exploration of complexities of the system. We offer guidance on the choice of approach that might be taken by people collating evidence in support of guideline development, and emphasise that the appropriate methods will depend on the purpose of the synthesis, the similarity of the studies included in the review, the level of detail available from the studies, the nature of the results reported in the studies, the expertise of the synthesis team and the resources available.

IMP2ART systematic review of education for healthcare professionals implementing supported self-management for asthma

Despite a robust evidence base for its effectiveness, implementation of supported self-management for asthma is suboptimal. Professional education is an implementation strategy with proven effectiveness, though the specific features linked with effectiveness are often unclear. We performed a systematic review of randomised controlled trials and controlled clinical trials (published from 1990 and updated to May 2017 using forward citation searching) to determine the effectiveness of professional education on asthma self-management support and identify features of effective initiatives. Primary outcomes reflected professional behaviour change (provision of asthma action plans) and patient outcomes (asthma control; unscheduled care). Data were coded using the Effective Practice and Organisation of Care Taxonomy, the Theoretical Domains Framework (TDF), and Bloom's Taxonomy and synthesised narratively. Of 15,637 articles identified, 18 (reporting 15 studies including 21 educational initiatives) met inclusion criteria. Risk of bias was high for five studies, and unclear for 10. Three of 6 initiatives improved action plan provision; 1/2 improved asthma control; and 2/7 reduced unscheduled care. Compared to ineffective initiatives, effective initiatives were more often coded as being guideline-based; involving local opinion leaders; including inter-professional education; and addressing the TDF domains 'social influences'; 'environmental context and resources'; 'behavioural regulation'; 'beliefs about consequences'; and 'social/professional role and identity'. Findings should be interpreted cautiously as many strategies were specified infrequently. However, identified features warrant further investigation as part of implementation strategies aiming to improve the provision of supported self-management for asthma.

Interventions using pediatric diabetes registry data for quality improvement: A systematic review

BACKGROUND

Diabetes registries contain vast amounts of data that can be used for quality improvement (QI) and are foundational elements of learning health systems; infrastructure to share data, create knowledge rapidly and inform decisions to improve health outcomes. QI interventions using adult diabetes registries are associated with improved glycemic control, complication screening rates, and reduced hospitalizations; pediatric data are limited.

OBJECTIVE

To evaluate the effects of QI strategies that use pediatric diabetes registry data on care processes, organization of care, and patient outcomes.

METHODS

We searched MEDLINE, EMBASE, Web of Science, Cochrane Central Register of Controlled Trials, Google, Google Scholar, Directory of Open Access Journals, and diabetes registry websites for studies that evaluated the impact of QI interventions on diabetes care processes, care organization, or patient outcomes, using pediatric diabetes registry data. Two reviewers independently assessed eligibility, extracted data and assessed the risk of bias.

RESULTS

Twelve studies were included. Most interventions targeted health-care providers and evaluated effects on patient outcomes. Five of nine studies that evaluated hemoglobin A1c found improvements of 0.26% to 0.85% (2.8-9.3 mmol/mol) while four found no difference. Many report positive effects on care processes or organization. Study data could not be combined because of variable study design and outcome measures. Included studies represent a minority of existing registries.

CONCLUSIONS

Pediatric diabetes registries are underused for QI and may facilitate improved care and outcomes. Existing vast amount of pediatric registry data could be used to foster the development of learning health systems and to improve diabetes care and outcomes.

Economic Evaluation of Quality Improvement Interventions Designed to Improve Glycemic Control in Diabetes: A Systematic Review and Weighted Regression Analysis

OBJECTIVE

Quality improvement (QI) interventions can improve glycemic control, but little is known about their value. We systematically reviewed economic evaluations of QI interventions for glycemic control among adults with type 1 or type 2 diabetes.

RESEARCH DESIGN AND METHODS

We used English-language studies from high-income countries that evaluated organizational changes and reported program and utilization-related costs, chosen from PubMed, EconLit, Centre for Reviews and Dissemination, New York Academy of Medicine's Grey Literature Report, and WorldCat (January 2004 to August 2016). We extracted data regarding intervention, study design, change in HbA1c, time horizon, perspective, incremental net cost (studies lasting ≤3 years), incremental cost-effectiveness ratio (ICER) (studies lasting ≥20 years), and study quality. Weighted least-squares regression analysis was used to estimate mean changes in HbA1c and incremental net cost.

RESULTS

Of 3,646 records, 46 unique studies were eligible. Across 19 randomized controlled trials (RCTs), HbA1c declined by 0.26% (95% CI 0.17-0.35) or 3 mmol/mol (2 to 4) relative to usual care. In 8 RCTs lasting ≤3 years, incremental net costs were $116 (95% CI -$612 to $843) per patient annually. Long-term ICERs were $100,000-$115,000/quality-adjusted life year (QALY) in 3 RCTs, $50,000-$99,999/QALY in 1 RCT, $0-$49,999/QALY in 4 RCTs, and dominant in 1 RCT. Results were more favorable in non-RCTs. Our limitations include the fact that the studies had diverse designs and involved moderate risk of bias.

CONCLUSIONS

Diverse multifaceted QI interventions that lower HbA1c appear to be a fair-to-good value relative to usual care, depending on society's willingness to pay for improvements in health.

Do quality improvement initiatives for diabetes care address social inequities? Secondary analysis of a systematic review

BACKGROUND

Socially disadvantaged populations carry a disproportionate burden of diabetes-related morbidity and mortality. There is an emerging interest in quality improvement (QI) strategies in the care of patients with diabetes, however, the effect of these interventions on disadvantaged groups remains unclear.

OBJECTIVE

This is a secondary analysis of a systematic review that seeks to examine the extent of equity considerations in diabetes QI studies, specifically quantifying the proportion of studies that target interventions toward disadvantaged populations and conduct analyses on the impact of interventions on disadvantaged groups.

RESEARCH DESIGN AND METHODS

Studies were identified using Medline, HealthStar and the Cochrane Effective Practice and Organisation of Care database. Randomised controlled trials assessing 12 QI strategies targeting health systems, healthcare professionals and/or patients for the management of adult outpatients with diabetes were eligible. The place of residence, race/ethnicity/culture/language, occupational status, gender/sexual identity, religious affiliations, education level, socioeconomic status, social capital, plus age, disability, sexual preferences and relationships (PROGRESS-Plus) framework was used to identify trials that focused on disadvantaged patient populations, to examine the types of equity-relevant factors that are being considered and to explore temporal trends in equity-relevant diabetes QI trials.

RESULTS

Of the 278 trials that met the inclusion criteria, 95 trials had equity-relevant considerations. These include 64 targeted trials that focused on a disadvantaged population with the aim to improve the health status of that population and 31 general trials that undertook subgroup analyses to assess the extent to which their interventions may have had differential impacts on disadvantaged subgroups. Trials predominantly focused on race/ethnicity, socioeconomic status and place of residence as potential factors for disadvantage in patients receiving diabetes care.

CONCLUSIONS

Less than one-third of diabetes QI trials included equity-relevant considerations, limiting the relevance and applicability of their data to disadvantaged populations. There is a need for better data collection, reporting, analysis and interventions on the social determinants of health that may influence the health outcomes of patients with diabetes.

PROSPERO REGISTRATION NUMBER

CRD42013005165.

Interventions to increase attendance for diabetic retinopathy screening

BACKGROUND

Despite evidence supporting the effectiveness of diabetic retinopathy screening (DRS) in reducing the risk of sight loss, attendance for screening is consistently below recommended levels.

OBJECTIVES

The primary objective of the review was to assess the effectiveness of quality improvement (QI) interventions that seek to increase attendance for DRS in people with type 1 and type 2 diabetes.Secondary objectives were:To use validated taxonomies of QI intervention strategies and behaviour change techniques (BCTs) to code the description of interventions in the included studies and determine whether interventions that include particular QI strategies or component BCTs are more effective in increasing screening attendance;To explore heterogeneity in effect size within and between studies to identify potential explanatory factors for variability in effect size;To explore differential effects in subgroups to provide information on how equity of screening attendance could be improved;To critically appraise and summarise current evidence on the resource use, costs and cost effectiveness.

SEARCH METHODS

We searched the Cochrane Library, MEDLINE, Embase, PsycINFO, Web of Science, ProQuest Family Health, OpenGrey, the ISRCTN, ClinicalTrials.gov, and the WHO ICTRP to identify randomised controlled trials (RCTs) that were designed to improve attendance for DRS or were evaluating general quality improvement (QI) strategies for diabetes care and reported the effect of the intervention on DRS attendance. We searched the resources on 13 February 2017. We did not use any date or language restrictions in the searches.

SELECTION CRITERIA

We included RCTs that compared any QI intervention to usual care or a more intensive (stepped) intervention versus a less intensive intervention.

DATA COLLECTION AND ANALYSIS

We coded the QI strategy using a modification of the taxonomy developed by Cochrane Effective Practice and Organisation of Care (EPOC) and BCTs using the BCT Taxonomy version 1 (BCTTv1). We used Place of residence, Race/ethnicity/culture/language, Occupation, Gender/sex, Religion, Education, Socioeconomic status, and Social capital (PROGRESS) elements to describe the characteristics of participants in the included studies that could have an impact on equity of access to health services.Two review authors independently extracted data. One review author entered the data into Review Manager 5 and a second review author checked them. Two review authors independently assessed risks of bias in the included studies and extracted data. We rated certainty of evidence using GRADE.

MAIN RESULTS

We included 66 RCTs conducted predominantly (62%) in the USA. Overall we judged the trials to be at low or unclear risk of bias. QI strategies were multifaceted and targeted patients, healthcare professionals or healthcare systems. Fifty-six studies (329,164 participants) compared intervention versus usual care (median duration of follow-up 12 months). Overall, DRS attendance increased by 12% (risk difference (RD) 0.12, 95% confidence interval (CI) 0.10 to 0.14; low-certainty evidence) compared with usual care, with substantial heterogeneity in effect size. Both DRS-targeted (RD 0.17, 95% CI 0.11 to 0.22) and general QI interventions (RD 0.12, 95% CI 0.09 to 0.15) were effective, particularly where baseline DRS attendance was low. All BCT combinations were associated with significant improvements, particularly in those with poor attendance. We found higher effect estimates in subgroup analyses for the BCTs 'goal setting (outcome)' (RD 0.26, 95% CI 0.16 to 0.36) and 'feedback on outcomes of behaviour' (RD 0.22, 95% CI 0.15 to 0.29) in interventions targeting patients, and 'restructuring the social environment' (RD 0.19, 95% CI 0.12 to 0.26) and 'credible source' (RD 0.16, 95% CI 0.08 to 0.24) in interventions targeting healthcare professionals.Ten studies (23,715 participants) compared a more intensive (stepped) intervention versus a less intensive intervention. In these studies DRS attendance increased by 5% (RD 0.05, 95% CI 0.02 to 0.09; moderate-certainty evidence).Fourteen studies reporting any QI intervention compared to usual care included economic outcomes. However, only five of these were full economic evaluations. Overall, we found that there is insufficient evidence to draw robust conclusions about the relative cost effectiveness of the interventions compared to each other or against usual care.With the exception of gender and ethnicity, the characteristics of participants were poorly described in terms of PROGRESS elements. Seventeen studies (25.8%) were conducted in disadvantaged populations. No studies were carried out in low- or middle-income countries.

AUTHORS' CONCLUSIONS

The results of this review provide evidence that QI interventions targeting patients, healthcare professionals or the healthcare system are associated with meaningful improvements in DRS attendance compared to usual care. There was no statistically significant difference between interventions specifically aimed at DRS and those which were part of a general QI strategy for improving diabetes care. This is a significant finding, due to the additional benefits of general QI interventions in terms of improving glycaemic control, vascular risk management and screening for other microvascular complications. It is likely that further (but smaller) improvements in DRS attendance can also be achieved by increasing the intensity of a particular QI component or adding further components.

Using Large-Scale Linkage Data to Evaluate the Effectiveness of a National Educational Program on Antithrombotic Prescribing and Associated Stroke Prevention in Primary Care

BACKGROUND

The National Prescribing Service (NPS) MedicineWise Stroke Prevention Program, which was implemented nationally in 2009-2010 in Australia, sought to improve antithrombotic prescribing in stroke prevention using dedicated interventions that target general practitioners. This study evaluated the impact of the NPS MedicineWise Stroke Prevention Program on antithrombotic prescribing and primary stroke hospitalizations.

METHOD AND RESULTS

This population-based time series study used administrative health data linked to 45 and Up Study participants with a high risk of cardiovascular disease (CVD) to assess the possible impact of the NPS MedicineWise program on first-time aspirin prescriptions and primary stroke-related hospitalizations. Time series analysis showed that the NPS MedicineWise program was significantly associated with increased first-time prescribing of aspirin (P=0.03) and decreased hospitalizations for primary ischemic stroke (P=0.03) in the at-risk study population (n=90 023). First-time aspirin prescription was correlated with a reduction in the rate of hospitalization for primary stroke (P=0.02). Following intervention, the number of first-time aspirin prescriptions increased by 19.8% (95% confidence interval, 1.6-38.0), while the number of first-time stroke hospitalizations decreased by 17.3% (95% confidence interval, 1.8-30.0).

CONCLUSIONS

Consistent with NPS MedicineWise program messages for the high-risk CVD population, the NPS MedicineWise Stroke Prevention Program (2009) was associated with increased initiation of aspirin and a reduced rate of hospitalization for primary stroke. The findings suggest that the provision of evidence-based multifaceted large-scale educational programs in primary care can be effective in changing prescriber behavior and positively impacting patient health outcomes.