Is reporting on interventions a weak link in understanding how and why they work? A preliminary exploration using community heart health exemplars

Informing the development of the SUCCEED reporting guideline for studies on the scaling of health interventions: A systematic review

BACKGROUND

Quality reporting contributes to effective translation of health research in practice and policy. As an initial step in the development of a reporting guideline for scaling, the Standards for reporting stUdies of sCaling evidenCEd-informED interventions (SUCCEED), we performed a systematic review to identify relevant guidelines and compile a list of potential items.

METHODS

We conducted a systematic review according to Cochrane method guidelines. We searched the following databases: MEDLINE, Embase, PsycINFO, Cochrane Library, CINAHL, Web of Science, from their respective inceptions. We also searched websites of relevant organizations and Google. We included any document that provided instructions or recommendations, e.g., reporting guideline, checklist, guidance, framework, standard; could inform the design or reporting of scaling interventions; and related to the health sector. We extracted characteristics of the included guidelines and assessed their methodological quality using a 3-item internal validity assessment tool. We extracted all items from the guidelines and classified them according to the main sections of reporting guidelines (title, abstract, introduction, methods, results, discussion and other information). We performed a narrative synthesis based on descriptive statistics.

RESULTS

Of 7704 records screened (published between 1999 and 2019), we included 39 guidelines, from which data were extracted from 57 reports. Of the 39 guidelines, 17 were for designing scaling interventions and 22 for reporting implementation interventions. At least one female author was listed in 31 guidelines, and 21 first authors were female. None of the authors belonged to the patient stakeholder group. Only one guideline clearly identified a patient as having participated in the consensus process. More than half the guidelines (56%) had been developed using an evidence-based process. In total, 750 items were extracted from the 39 guidelines and distributed into the 7 main sections.

CONCLUSION

Relevant items identified could inform the development of a reporting guideline for scaling studies of evidence-based health interventions. This and our assessment of guidelines could contribute to better reporting in the science and practice of scaling.

Opportunistic health screening for cardiovascular and diabetes risk factors in primary care dental practices: experiences from a service evaluation and a call to action

Introduction Substantial evidence has established associations between oral health and chronic diseases, mediated by common risk factors and inflammatory processes. Dental professionals are aptly positioned to screen for cardiovascular disease and diabetes high-risk indicators. This article presents a service evaluation of two dental practices delivering health screening in dental practice, and a call to action for the wider profession.Methods Following training, two general dental practices implemented a suite of health screens, including blood pressure, blood glucose, cholesterol, body mass index (BMI) and waist-to-height ratio. A service evaluation was undertaken to review and improve service provision.Results Most patients (78.4%) had blood pressure values above normal range. More than half (55.8%) were outside of the healthy range for BMI. Out-of-range results for cholesterol were observed in less than one-fifth (16.7%). High blood glucose values were observed for few patients (3.3%).Conclusion Dental professionals can be successfully trained to deliver health screening interventions. Out-of-range health screening results offer an opportunity to provide targeted health advice for both the oral and general health. Clear protocols and careful interpretation of screening tests are required to minimise patients' confusion. More robust alliances between dental and general medical care are recommended.

Web-Based Structured Education for Type 2 Diabetes: Interdisciplinary User-Centered Design Approach

Background

Digital health research encompasses methods from human-computer interaction and health research.

Objective

This paper aims to describe how these methods were combined to develop HeLP-Diabetes: Starting Out, a web-based structured education program for people newly diagnosed with type 2 diabetes.

Methods

The development process consisted of three phases: initial design for effectiveness, optimization for usability, and in the wild testing in the National Health Service with people newly diagnosed with type 2 diabetes, and further revisions. We adopted an iterative user-centered approach and followed steps from the human-computer interaction design life cycle and the Medical Research Council guidelines on developing and evaluating complex interventions.

Results

The initial design process resulted in an 8-session program containing information and behavior change techniques targeting weight loss, being more active, and taking medication. The usability testing was highlighted at an early stage, where changes needed to be made to the language and layout of the program. The in the wild testing provided data on uptake of and barriers to use. The study suggested low uptake and completion of the program, but those who used it seemed to benefit from it. The qualitative findings suggested that barriers to use included an expectation that the program would take too long. This informed refinements to the program.

Conclusions

The use of interdisciplinary methods resulted in an iterative development process and refinements to the program that were based on user needs and data on uptake. The final intervention was more suitable for a definitive evaluation than the initial version. The description of our approach informs other digital health researchers on how to make interventions more sensitive to user needs.

Revisiting the Innovation Strategy performance measurement process: insights from practice

Performance measurement (PM) aims to ensure transparency and effectiveness in public spending. More specifically, it provides a foundation that allows funders to determine whether an intervention has achieved its stated objective, thus improving understanding of what interventions should be funded in the future. Ideally, performance measurement should help us understand "what works, for whom, and in what context." Unfortunately, more often than not, performance reports are limited to a collection of indicators that make it difficult to answer this question. Based on our work with the Public Health Agency of Canada Innovation Strategy (PHAC-IS), we offer several recommendations to address this issue and support the performance measurement process, especially for complex interventions. We discuss the importance of contextualizing results to better understand impact and associating these results to a well-defined intervention. We also suggest using a validated tool to improve data collection and analysis and critically appraise the methods used to collect impact information. The integration of these key considerations will save time in data analysis and ensure funded recipients are not overburdened by the data collection process. Although this commentary is presented within the context of a complex multi-year population health funding program, we believe our approach can be applied to any performance management process and ultimately improve decisions such as whether an intervention should be continued, scaled up, or adapted to different contexts.

Weight gain prevention content delivered to mothers: Lessons from an intervention embedded in a community organization

BACKGROUND

Transparent reporting of the intervention content study participants receive is particularly important for dissemination and implementation of evidence-based interventions to address obesity. This study explores intervention delivery and perceptions of content when an intervention is embedded within an organization's usual practice and associations with weight outcomes.

METHODS

This is a secondary analysis from the Healthy Eating and Active Living Taught at Home (HEALTH) randomized trial. HEALTH is a weight gain prevention intervention embedded within a national home visiting programme (Parents as Teachers, PAT). To be consistent with PAT practice, HEALTH has a suggested but also flexible home visit structure. Therefore, the number and frequency of home visits are determined by the family's needs and preferences based on the parent educator's professional judgement. The proportion of participants who received each curriculum core lesson was explored among the 105 families randomized to the HEALTH intervention group, as were mean parent educator perception ratings of the visits (5-point Likert-type scale). A chi-square test was used to examine the association between utilization (ie, low: 1-6, middle: 7-12, or high: 13-18 visits) category and weight maintenance.

RESULTS

Mothers received on average 13.3 (standard deviation 6.2) core visits. Mean parent educator perception of the mothers' response (out of 5) ranged from 3.99 to 4.27; educators' perception of their own experience with the lesson ranged from 4.13 to 4.34. Among mothers who maintained their weight, 13% were low, 22% were middle, and 65% were high utilizers, while among mothers who gained weight, 3%, 31%, and 67% were low, middle, and high utilizers, respectively; this difference was not statistically significant.

CONCLUSIONS

This study identified wide variation in the visits families received from the curriculum but minimal variation in visit perceptions. Future studies could explore whether there are optimal patterns for visit content, associated with successful outcomes.

Effectiveness of physical activity interventions for overweight and obesity during pregnancy: a systematic review of the content of behaviour change interventions

BACKGROUND

Behaviour change techniques (BCTs) employed within PA intervention for pregnant women with a healthy body mass index (BMI) have been previously identified, however, these BCTS may differ for other weight profiles during pregnancy. The aim of this current review was to identify and summarise the evidence for effectiveness of PA interventions on PA levels for pregnant women with overweight and obesity, with an emphasis on the BCTs employed.

METHODS

A systematic review and meta-analysis of PA intervention studies using the PRISMA statement was conducted. Searches were conducted of eight databases in January 2019. Strict inclusion/exclusion criteria were employed. The validity of each included study was assessed using the Cochrane Collaboration's tool for assessing risk of bias. The primary outcome measure was change in PA levels, subjectively or objectively measured, with physical fitness as a secondary outcome. All intervention descriptions were double coded by two authors using Michie's et al's BCT taxonomy V1. Meta-analyses using random effect models assessed the intervention effects on PA. Other PA outcomes were summarised in a narrative synthesis.

RESULTS

From 8389 studies, 19 met the inclusion criteria 13 of which were suitable for inclusion in a meta-analysis. The remaining 6 studies were described narratively due to insufficient data and different outcome measures reported. In the meta-analysis, comparing interventions to a control group, significant increases were found in the intervention group for metabolic equivalent (SMD 0.39 [0.14, 0.64], Z = 3.08 P = 0.002) and physical fitness (VO₂ max) (SMD 0.55 [0.34, 0.75], Z = 5.20 P = < 0.001). Of the other six, five studies reported an increase in PA for the intervention group versus the control with the other study reporting a significant decrease for women in their 3rd trimester (p = 0.002). 'Self-monitoring of behaviour' was the most frequently used BCTs (76.5%), with 'social support' being newly identified for this pregnant population with overweight or obesity.

CONCLUSIONS

This review identified a slight increase in PA for pregnant women with overweight and obesity participating in interventions. However, due to the high risk of bias of the included studies, the results should be interpreted with caution. PA measures should be carefully selected so that studies can be meaningfully compared and standardised taxonomies should be used so that BCTs can be accurately assessed.

Implementation science and stigma reduction interventions in low- and middle-income countries: a systematic review

BACKGROUND

Interventions to alleviate stigma are demonstrating effectiveness across a range of conditions, though few move beyond the pilot phase, especially in low- and middle-income countries (LMICs). Implementation science offers tools to study complex interventions, understand barriers to implementation, and generate evidence of affordability, scalability, and sustainability. Such evidence could be used to convince policy-makers and donors to invest in implementation. However, the utility of implementation research depends on its rigor and replicability. Our objectives were to systematically review implementation studies of health-related stigma reduction interventions in LMICs and critically assess the reporting of implementation outcomes and intervention descriptions.

METHODS

PubMed, CINAHL, PsycINFO, and EMBASE were searched for evaluations of stigma reduction interventions in LMICs reporting at least one implementation outcome. Study- and intervention-level characteristics were abstracted. The quality of reporting of implementation outcomes was assessed using a five-item rubric, and the comprehensiveness of intervention description and specification was assessed using the 12-item Template for Intervention Description and Replication (TIDieR).

RESULTS

A total of 35 eligible studies published between 2003 and 2017 were identified; of these, 20 (57%) used qualitative methods, 32 (91%) were type 1 hybrid effectiveness-implementation studies, and 29 (83%) were evaluations of once-off or pilot implementations. No studies adopted a formal theoretical framework for implementation research. Acceptability (20, 57%) and feasibility (14, 40%) were the most frequently reported implementation outcomes. The quality of reporting of implementation outcomes was low. The 35 studies evaluated 29 different interventions, of which 18 (62%) were implemented across sub-Saharan Africa, 20 (69%) focused on stigma related to HIV/AIDS, and 28 (97%) used information or education to reduce stigma. Intervention specification and description was uneven.

CONCLUSION

Implementation science could support the dissemination of stigma reduction interventions in LMICs, though usage to date has been limited. Theoretical frameworks and validated measures have not been used, key implementation outcomes like cost and sustainability have rarely been assessed, and intervention processes have not been presented in detail. Adapted frameworks, new measures, and increased LMIC-based implementation research capacity could promote the rigor of future stigma implementation research, helping the field deliver on the promise of stigma reduction interventions worldwide.

Specifying sickle cell disease interventions: a study protocol of the Sickle Cell Disease Implementation Consortium (SCDIC)

BACKGROUND

Sickle cell disease (SCD) is an inherited blood disorder that results in a lifetime of anemia, severe pain, and end-organ damage that can lead to premature mortality. While the SCD field has made major medical advances, much needs to be done to improve the quality of care for people with SCD. This study capitalizes on the Sickle Cell Disease Implementation Consortium (SCDIC), a consortium of eight academic sites aiming to test implementation strategies that could lead to more accelerated application of the NHLBI guidelines for treating SCD. This report documents the process to support the consortium by specifying the interventions being developed.

METHODS

This study consists of three steps. The Principal Investigator of each site and two site representatives who are knowledgeable of the intervention (e.g., study coordinator or the person delivering the intervention) will answer an online survey aiming to capture components of the interventions. This survey will be completed by the site representatives three times during the study: during the development of the interventions, after one year of the interventions being implemented, and at the end of this study (after 2 years). A site visit and semi-structured interview (Step 2) in the first year of the process will capture the context of the sites. Step 3 comprises of the development of a framework with the details of the multi-component SCDIC interventions at the sites.

DISCUSSION

The outcome of this study, a framework of the SCDIC, will enable accurate replication and extension of published research, facilitating the translation of SCD studies to diverse populations and settings and allowing for theory testing of the effects of the intervention components across studies in different contexts and for different populations.

TRIAL REGISTRATION

ClinicalTrial.Gov (# NCT03380351 ). Registered December 21, 2017.

Using the Replicating Effective Programs Framework to Adapt a Heart Health Intervention

Heart disease is the leading cause of death in the United States. African Americans and people of low socioeconomic status suffer disproportionately from heart disease-related morbidity and mortality. In Detroit, Michigan, a primarily African American and low-income urban area, heart disease mortality is at twice the national rate. Despite evidence for the effectiveness of self-management support interventions in reducing chronic disease burden for older adults, few are adapted for communities most in need. This article describes the process of adapting Take PRIDE, an evidence-based heart disease self-management intervention, for older adults in Detroit via the Replicating Effective Programs (REP) framework. Working within a community-academic partnership, we found REP useful in facilitating the identification of diverse stakeholders, core versus adaptable elements of the intervention and barriers to implementation. We also made several modifications to the REP framework in order to better fit our project needs. Overall, we found REP to be an effective, flexible tool that allowed us to successfully adapt a disease-management intervention for this setting. Processes, lessons learned, and recommendations offered in this article may help researchers and practitioners working to expand access to self-management support for populations most affected by chronic disease.

Are emergency department clinical pathway interventions adequately described, and are they delivered as intended? A systematic review

Introduction

The accurate reproduction of clinical interventions and the evaluation of provider adherence in research publications improve the evaluation and implementation of research findings into clinical practice. We sought to examine the proportion of clinical pathway publications in an emergency department setting that adequately reported the following: (1) the exact reproduction of the clinical pathway that was implemented in the study, (2) the adherence to and correct execution of the clinical pathway intervention, and (3) the presence of a pre-implementation education phase.

Methods

We performed a descriptive systematic review of the literature from 2006 to 2015 using MEDLINE, EMBASE, CENTRAL, and CINAHL. All types of prospective trial designs were eligible. Validated clinical pathway criteria were used to identify relevant publications. Two reviewers independently collected data using a piloted data abstraction tool. Risk of bias was assessed using the Cochrane Effective Practice and Organization of Care Group Risk of Bias Tool and the Newcastle-Ottawa Scale.

Results

We identified 5947 publications, 44 of which met our inclusion criteria. The formal clinical pathway was fully reproduced in 27 (61%) publications, partially reproduced in 9 (21%), and not reproduced in 8 (18%). Only 14 (32%) studies reported whether at least one decision step was executed correctly. The presence of a pre-implementation education phase was reported in 33 (75%) studies.

Conclusion

The underreporting of intervention elements may present a barrier to both the evaluation and accurate replication of clinical pathway interventions. These finding may be useful for the elaboration of complex intervention reporting guidelines, improved reporting in future clinical pathway publications, and improved knowledge translation and exchange of clinical pathway interventions.

The next phase in professional services research: From implementation to sustainability

The provision of professional pharmacy services has been heralded as the professional and the economic future of pharmacy. There are different phases involved in a service creation including service design, impact evaluation, implementation and sustainability. The two first phases have been subject to extensive research. In the last years the principles of Implementation science have been applied in pharmacy to study the initial uptake and integration of evidence-based services into routine practice. However, little attention has been paid to the sustainability of those services, during which there is a continued use of the service previously implemented to achieve and sustain long-term outcomes. The objective of this commentary is to describe the differences and common characteristics between the implementation and the sustainability phase and to propose a definition for pharmacy. A literature search was performed. Four critical elements were identified: 1. The aim of the implementation phase is to incorporate new services into practice, the sustainability phase's aim is to make the services routine to achieve and sustain long-term benefits 2. At the implementation phase planned activities are used as a process to integrate the new service, at the sustainability phase there is a continuous improvement of the service 3. The implementation phase occurs during the period of time between the adoption of a service and its integration. Some authors suggest the sustainability phase is a concomitant phase with the implementation phase and others suggest it is independent 4. There is a lack of consensus regarding the duration of each phase. The following definition of sustainability for pharmacy services is proposed: "Sustainability is a phase in the process of a professional pharmacy service, in which the service previously integrated into practice during the implementation phase is routinized and institutionalized over time to achieve and sustain the expected service outcomes". An agreement on a definition will facilitate an understanding of when the profession has reached this ultimate goal.

A systematic review of electronic audit and feedback: intervention effectiveness and use of behaviour change theory

BACKGROUND

Audit and feedback is a common intervention for supporting clinical behaviour change. Increasingly, health data are available in electronic format. Yet, little is known regarding if and how electronic audit and feedback (e-A&F) improves quality of care in practice.

OBJECTIVE

The study aimed to assess the effectiveness of e-A&F interventions in a primary care and hospital context and to identify theoretical mechanisms of behaviour change underlying these interventions.

METHODS

In August 2016, we searched five electronic databases, including MEDLINE and EMBASE via Ovid, and the Cochrane Central Register of Controlled Trials for published randomised controlled trials. We included studies that evaluated e-A&F interventions, defined as a summary of clinical performance delivered through an interactive computer interface to healthcare providers. Data on feedback characteristics, underlying theoretical domains, effect size and risk of bias were extracted by two independent review authors, who determined the domains within the Theoretical Domains Framework (TDF). We performed a meta-analysis of e-A&F effectiveness, and a narrative analysis of the nature and patterns of TDF domains and potential links with the intervention effect.

RESULTS

We included seven studies comprising of 81,700 patients being cared for by 329 healthcare professionals/primary care facilities. Given the extremely high heterogeneity of the e-A&F interventions and five studies having a medium or high risk of bias, the average effect was deemed unreliable. Only two studies explicitly used theory to guide intervention design. The most frequent theoretical domains targeted by the e-A&F interventions included 'knowledge', 'social influences', 'goals' and 'behaviour regulation', with each intervention targeting a combination of at least three. None of the interventions addressed the domains 'social/professional role and identity' or 'emotion'. Analyses identified the number of different domains coded in control arm to have the biggest role in heterogeneity in e-A&F effect size.

CONCLUSIONS

Given the high heterogeneity of identified studies, the effects of e-A&F were found to be highly variable. Additionally, e-A&F interventions tend to implicitly target only a fraction of known theoretical domains, even after omitting domains presumed not to be linked to e-A&F. Also, little evaluation of comparative effectiveness across trial arms was conducted. Future research should seek to further unpack the theoretical domains essential for effective e-A&F in order to better support strategic individual and team goals.

Alcohol Prevention and Evaluation in the Era of Evidence Based Practice – the Need for a Systematic Approach to Evaluation

AIMS

This article focuses on the shortcomings of experimental outcome evaluations.

MATERIAL

The study uses two studies of the alcohol prevention program Prime for Life (PFL) to illustrate problems associated with a ‘one dimensional’ goal focus and how implementation may affect the efficiency of an intervention.

Results

As evidence based practice often fails to acknowledge the importance of context and implementation quality in its quest to find ‘what works’, it is argued in the article that a wider perspective on efficiency is needed in evaluations of alcohol prevention.

CONSLUSIONS

To be able to find out what works and during what circumstances, evaluators need to look for a wider set of traits that constitute effective interventions.

Swimming against the tide: A Canadian qualitative study examining the implementation of a province-wide public health initiative to address health equity

BACKGROUND

Effectively addressing the social determinants of health and health equity are critical yet still-emerging areas of public health practice. This is significant for contemporary practice as the egregious impacts of health inequities on health outcomes continue to be revealed. More public health organizations seek to augment internal organizational capacity to address health equity while the evidence base to inform such leadership is in its infancy. The purpose of this paper is to report on findings of a study examining key factors influencing the development and implementation of the social determinants of health public health nurse (SDH-PHN) role in Ontario, Canada.

METHODS

A descriptive qualitative case study approach examined the first Canadian province-wide initiative to add SDH-PHNs to each public health unit. Data sources were documents and staff from public health units (i.e., SDH-PHNs, Managers, Directors, Chief Nursing Officers, Medical Officers of Health) as well as external stakeholders. Data were collected through 42 individual interviews and 226 documents. Interview data were analyzed using framework analysis methods; Prior's approach guided document analysis.

RESULTS

Three themes related to the SDH-PHN role implementation were identified: (1) 'Swimming against the tide' to lead change as staff navigated ideological tensions, competency development, and novel collaborations; (2) Shifting organizational practice environments impacted by initial role placement and action to structurally embed health equity priorities; and (3) Bridging policy implementation gaps related to local-provincial implementation and reporting expectations.

CONCLUSIONS

This study extends our understanding of the dynamic interplay among leadership, change management, ideological tensions, and local-provincial public health policy impacting health equity agendas. Given that the social determinants of health lie outside public health, collaboration with communities, health partners and non-health partners is essential to public health practice for health equity. The study findings have implications for increasing our knowledge and capacity for effective system-wide intervention towards health equity as a critical strategic priority for public health and for broader public policy and community engagement. Appropriate and effective public health leadership at multiple levels and by multiple actors is tantamount to adequately making inroads for health equity.

The IDEFICS intervention: what can we learn for public policy?

INTRODUCTION

As considered in the rest of this volume, the effects of the IDEFICS intervention on obesity rates were not encouraging. This paper considers how far findings from the IDEFICS study and similar intervention studies are relevant to the policy process and political decision-making.

METHODS

The paper offers theoretical and policy-level arguments concerning the evaluation of evidence and its implications for policymaking. The paper is divided into three parts. The first considers problems in the nature and applicability of evidence gained from school- and community-level obesity interventions. The second part considers whether such interventions present a model that policymakers could implement. The third part considers how we should think about policy measures given the limited evidence we can obtain and the many different goals that public policy must take account of.

RESULTS

The paper argues that (1) there are clear reasons why we are not obtaining good evidence for effective school- and community-level interventions; (2) public policy is not in a good position to mandate larger-scale, long-term versions of these interventions; and (3) there are serious problems in obtaining 'evidence' for most public policy options, but this should not deter us from pursuing options that tackle systemic problems and have a good likelihood of delivering benefits on several dimensions.

CONCLUSIONS

Research on school- and community-level obesity interventions has not produced much evidence that is directly relevant to policymaking. Instead, it shows how difficult it is to affect obesity rates without changing wider social and economic factors. Public policy should focus on these.

An Internet-Based Intervention (Mamma Mia) for Postpartum Depression: Mapping the Development from Theory to Practice

BACKGROUND

As much as 10-15% of new mothers experience depression postpartum. An Internet-based intervention (Mamma Mia) was developed with the primary aims of preventing depressive symptoms and enhancing subjective well-being among pregnant and postpartum women. A secondary aim of Mamma Mia was to ease the transition of becoming a mother by providing knowledge, techniques, and support during pregnancy and after birth.

OBJECTIVE

The aim of the paper is to provide a systematic and comprehensive description of the intervention rationale and the development of Mamma Mia.

METHODS

For this purpose, we used the intervention mapping (IM) protocol as descriptive tool, which consists of the following 6 steps: (1) a needs assessment, (2) definition of change objectives, (3) selection of theoretical methods and practical strategies, (4) development of program components, (5) planning adoption and implementation, and (6) planning evaluation.

RESULTS

Mamma Mia is a fully automated Internet intervention available for computers, tablets, and smartphones, intended for individual use by the mother. It starts in gestational week 18-24 and lasts up to when the baby becomes 6 months old. This intervention applies a tunneled design to guide the woman through the program in a step-by-step fashion in accordance with the psychological preparations of becoming a mother. The intervention is delivered by email and interactive websites, combining text, pictures, prerecorded audio files, and user input. It targets risk and protective factors for postpartum depression such as prepartum and postpartum attachment, couple satisfaction, social support, and subjective well-being, as identified in the needs assessment. The plan is to implement Mamma Mia directly to users and as part of ordinary services at well-baby clinics, and to evaluate the effectiveness of Mamma Mia in a randomized controlled trial and assess users' experiences with the program.

CONCLUSIONS

The IM of Mamma Mia has made clear the rationale for the intervention, and linked theories and empirical evidence to the contents and materials of the program. This meets the recent calls for intervention descriptions and may inform future studies, development of interventions, and systematic reviews.

[An intervention research program (AAPRISS) to reduce social inequalities in health: methods and validation]

OBJECTIVE

The AAPRISS intervention-research program (Apprendre et Agir Pour Réduire les Inégalités Sociales de Santé) aims to change prevention projects to improve the extent to which they take health inequalities into account. The goal is to assess a project's feasibility and its acceptability, from the viewpoint of the sponsors of the project in question, and to present its tools and potentially-transferable elements.

PARTICIPANTS

Five cancer-prevention projects focused on nutrition have been included in the AAPRISS program: two projects conducted in school, one community-health project conducted in a disadvantaged neighbourhood and two hospital therapeutic-education programes. For each one, a workgroup was created bringing together the project sponsors and their team as well as researchers from various disciplines.

LOCATION

The study involves projects currently underway in the Midi- Pyrénées region of France.

INTERVENTION

AAPRISS is based on the exchange of knowledge among project sponsors and researchers and the co-construction of the analysis, reorientations and the assessment of the prevention projects included in the program. The validation of the program and its feasibility from the project sponsors' point of view is based on a questionnaire and presentations carried out during a symposium that took place in May 2014 in Toulouse.

RESULTS

Twelve key program functions have been described. They have been shown to be feasible and acceptable and concrete project modifications towards a better consideration of social inequalities of health (SIH) have been realized in most of the projects. Co-construction among the stakeholders raises many issues and difficulties that could have been overcome by the establishment of a relationship of trust between project sponsors and researchers.

CONCLUSION

The complexity of the determinants leading to SIH calls for programs that revisit existing projects rather than a new intervention aimed at reducing them.

Understanding effects in reviews of implementation interventions using the Theoretical Domains Framework

BACKGROUND

Behavioural theory can be used to better understand the effects of behaviour change interventions targeting healthcare professional behaviour to improve quality of care. However, the explicit use of theory is rarely reported despite interventions inevitably involving at least an implicit idea of what factors to target to implement change. There is a quality of care gap in the post-fracture investigation (bone mineral density (BMD) scanning) and management (bisphosphonate prescription) of patients at risk of osteoporosis. We aimed to use the Theoretical Domains Framework (TDF) within a systematic review of interventions to improve quality of care in post-fracture investigation. Our objectives were to explore which theoretical factors the interventions in the review may have been targeting and how this might be related to the size of the effect on rates of BMD scanning and osteoporosis treatment with bisphosphonate medication.

METHODS

A behavioural scientist and a clinician independently coded TDF domains in intervention and control groups. Quantitative analyses explored the relationship between intervention effect size and total number of domains targeted, and as number of different domains targeted.

RESULTS

Nine randomised controlled trials (RCTs) (10 interventions) were analysed. The five theoretical domains most frequently coded as being targeted by the interventions in the review included "memory, attention and decision processes", "knowledge", "environmental context and resources", "social influences" and "beliefs about consequences". Each intervention targeted a combination of at least four of these five domains. Analyses identified an inverse relationship between both number of times and number of different domains coded and the effect size for BMD scanning but not for bisphosphonate prescription, suggesting that the more domains the intervention targeted, the lower the observed effect size.

CONCLUSIONS

When explicit use of theory to inform interventions is absent, it is possible to retrospectively identify the likely targeted factors using theoretical frameworks such as the TDF. In osteoporosis management, this suggested that several likely determinants of healthcare professional behaviour appear not yet to have been considered in implementation interventions. This approach may serve as a useful basis for using theory-based frameworks such as the TDF to retrospectively identify targeted factors within systematic reviews of implementation interventions in other implementation contexts.

An analysis of the adaptability of a professional development program in public health: results from the ALPS Study

Background

Professional development is a key component of effective public health infrastructures. To be successful, professional development programs in public health and health promotion must adapt to practitioners’ complex real-world practice settings while preserving the core components of those programs’ models and theoretical bases. An appropriate balance must be struck between implementation fidelity, defined as respecting the core nature of the program that underlies its effects, and adaptability to context to maximize benefit in specific situations. This article presents a professional development pilot program, the Health Promotion Laboratory (HPL), and analyzes how it was adapted to three different settings while preserving its core components. An exploratory analysis was also conducted to identify team and contextual factors that might have been at play in the emergence of implementation profiles in each site.

Methods

This paper describes the program, its core components and adaptive features, along with three implementation experiences in local public health teams in Quebec, Canada. For each setting, documentary sources were analyzed to trace the implementation of activities, including temporal patterns throughout the project for each program component. Information about teams and their contexts/settings was obtained through documentary analysis and semi-structured interviews with HPL participants, colleagues and managers from each organization.

Results

While each team developed a unique pattern of implementing the activities, all the program’s core components were implemented. Differences of implementation were observed in terms of numbers and percentages of activities related to different components of the program as well as in the patterns of activities across time. It is plausible that organizational characteristics influencing, for example, work schedule flexibility or learning culture might have played a role in the HPL implementation process.

Conclusions

This paper shows how a professional development program model can be adapted to different contexts while preserving its core components. Capturing the heterogeneity of the intervention’s exposure, as was done here, will make possible in-depth impact analyses involving, for example, the testing of program–context interactions to identify program outcomes predictors. Such work is essential to advance knowledge on the action mechanisms of professional development programs.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-015-0903-3) contains supplementary material, which is available to authorized users.

A RE-AIM evaluation of evidence-based multi-level interventions to improve obesity-related behaviours in adults: a systematic review (the SPOTLIGHT project)

BACKGROUND

This systematic literature review describes the potential public health impact of evidence-based multi-level interventions to improve obesity-related behaviours in adults, using the Reach, Efficacy, Adoption, Implementation and Maintenance (RE-AIM) framework.

METHODS

Electronic databases (PubMed, Embase, and The Cochrane Library) were searched to identify intervention studies published between January 2000 and October 2013. The following inclusion criteria were used: (1) the study included at least one outcome measure assessing obesity-related behaviours (i.e. diet, physical activity or sedentary behaviour), (2) the study collected data over at least one year and (3) the study's intervention targeted adults, was conducted in a specified geographical area or worksite, and was multi-level (i.e. targeting both individual and environmental level). Evidence of RE-AIM of the selected interventions was assessed. Potential public health impact of an intervention was evaluated if information was provided on at least four of the five RE-AIM dimensions.

RESULTS

Thirty-five multi-level interventions met the inclusion criteria. RE-AIM evaluation revealed that the included interventions generally had the potential to: reach a large number of people (on average 58% of the target population was aware of the intervention); achieve the assumed goals (89% found positive outcomes); be broadly adopted (the proportion of intervention deliverers varied from 9% to 92%) and be sustained (sixteen interventions were maintained). The highest potential public health impact was found in multi-level interventions that: 1) focused on all levels at the beginning of the planning process, 2) guided the implementation process using diffusion theory, and 3) used a website to disseminate the intervention.

CONCLUSIONS

Although most studies underreported results within the RE-AIM dimensions, the reported Reach, Effectiveness, Adoption, Implementation and Maintenance were positively evaluated. However, more information on external validity and sustainability is needed in order to take informed decisions on the choice of interventions that should be implemented in real-world settings to accomplish long-term changes in obesity-related behaviours.

Influence of behavioral theory on fruit and vegetable intervention effectiveness among children: a meta-analysis

OBJECTIVE

To test the hypotheses that interventions clearly based on theory, multiple theories, or a formal intervention planning process will be more effective in changing fruit and vegetable consumption among children than interventions with no behavioral theoretical foundation.

DESIGN

Systematic review and meta-analysis.

SETTING

Identification of articles in PubMed, PsycInfo, Medline, Cochrane Collaborative database, and existing literature reviews and meta-analyses.

PARTICIPANTS

Children aged 2-18 years.

INTERVENTIONS

Change in fruit and/or vegetable consumption in dietary change interventions.

METHODS

Meta-analysis, meta-regression analysis, and summary reporting for articles.

CONCLUSIONS AND IMPLICATIONS

Predicating an intervention on behavioral theory had a small to moderate enhancement (P < .001) of outcome effectiveness. Differences in mean Hedges' g effect sizes between theory and non-theory interventions were 0.232 for fruit, 0.043 for vegetables, and 0.333 for fruit and vegetables combined. There was mixed support, however, for enhanced dietary change with multiple theories or a formal planning process. After controlling for study quality, theory use was related only to vegetable consumption (β = 0.373; P < .001). More research is needed on theory's influences on dietary behaviors to guide future interventions among children. More research is also needed to identify what may be effective practical- or experience-based procedures that complement theory, to incorporate into interventions.

Europe needs a central, transparent, and evidence-based approval process for behavioural prevention interventions

Fabrizio Faggiano and colleagues discuss how a central, transparent, and evidence-based approval process is needed for behavioral prevention interventions in Europe and propose a way forward. Please see later in the article for the Editors' Summary.

Strengthening chronic disease prevention programming: the Toward Evidence-Informed Practice (TEIP) Program Assessment Tool

Best practices identified solely on the strength of research evidence may not be entirely relevant or practical for use in community-based public health and the practice of chronic disease prevention. Aiming to bridge the gap between best practices literature and local knowledge and expertise, the Ontario Public Health Association, through the Toward Evidence-Informed Practice initiative, developed a set of resources to strengthen evidence-informed decision making in chronic disease prevention programs. A Program Assessment Tool, described in this article, emphasizes better processes by incorporating review criteria into the program planning and implementation process. In a companion paper, "Strengthening Chronic Disease Prevention Programming: The Toward Evidence-Informed Practice (TEIP) Program Evidence Tool," we describe another tool, which emphasizes better evidence by providing guidelines and worksheets to identify, synthesize, and incorporate evidence from a range of sources (eg, peer-reviewed literature, gray literature, local expertise) to strengthen local programs.The Program Assessment Tool uses 19 criteria derived from literature on best and promising practices to assess and strengthen program planning and implementation. We describe the benefits, strengths, and challenges in implementing the tool in 22 community-based chronic disease prevention projects in Ontario, Canada. The Program Assessment Tool helps put best processes into operation to complement adoption and adaptation of evidence-informed practices for chronic disease prevention.

Managing symptoms during cancer treatments: evaluating the implementation of evidence-informed remote support protocols

BACKGROUND

Management of cancer treatment-related symptoms is an important safety issue given that symptoms can become life-threatening and often occur when patients are at home. With funding from the Canadian Partnership Against Cancer, a pan-Canadian steering committee was established with representation from eight provinces to develop symptom protocols using a rigorous methodology (CAN-IMPLEMENT©). Each protocol is based on a systematic review of the literature to identify relevant clinical practice guidelines. Protocols were validated by cancer nurses from across Canada. The aim of this study is to build an effective and sustainable approach for implementing evidence-informed protocols for nurses to use when providing remote symptom assessment, triage, and guidance in self-management for patients experiencing symptoms while undergoing cancer treatments.

METHODS

A prospective mixed-methods study design will be used. Guided by the Knowledge to Action Framework, the study will involve (a) establishing an advisory knowledge user team in each of three targeted settings; (b) assessing factors influencing nurses' use of protocols using interviews/focus groups and a standardized survey instrument; (c) adapting protocols for local use, ensuring fidelity of the content; (d) selecting intervention strategies to overcome known barriers and implementing the protocols; (e) conducting think-aloud usability testing; (f) evaluating protocol use and outcomes by conducting an audit of 100 randomly selected charts at each of the three settings; and (g) assessing satisfaction with remote support using symptom protocols and change in nurses' barriers to use using survey instruments. The primary outcome is sustained use of the protocols, defined as use in 75% of the calls. Descriptive analysis will be conducted for the barriers, use of protocols, and chart audit outcomes. Content analysis will be conducted on interviews/focus groups and usability testing with comparisons across settings.

DISCUSSION

Given the importance of patient safety, patient-centered care, and delivery of quality services, learning how to effectively implement evidence-informed symptom protocols in oncology healthcare services is essential for ensuring safe, consistent, and effective care for individuals with cancer. This study is likely to have a significant contribution to the delivery of remote oncology services, as well as influence symptom management by patients at home.

Time issues in multilevel interventions for cancer treatment and prevention

The concept of time introduces important complexities in estimating intervention effects, program and evaluation design, and measurement and analysis of individual change in multilevel interventions (MLIs). Despite growing recognition that time is a critical element for assessing both individual-level outcomes and higher-level changes in organizational, community, and policy contexts, most MLI designs and evaluations have not addressed these issues. In this chapter we discuss 1) conceptualizing disease life-course and treatment theory in MLIs, 2) approaches to incorporating time in research and program design for MLIs in cancer treatment and prevention, 3) analysis of time-varying multilevel data in the context of cancer treatment and prevention, and 4) resource considerations and trade-offs of incorporating time as a dimension of MLIs and analysis. Although analytic techniques for analyzing time-related phenomena are becoming more available and powerful, there has not been corresponding progress made in the development of theory to guide the application of these techniques in program design and implementation.

Using documents to investigate links between implementation and sustainability in a complex community intervention: the PRISM study

The increasing imperative to find what works in health services has meant a rise in research trialing interventions deemed 'complex'. While the strength of these interventions comes from taking a 'whole of problem' approach using multiple and inter-linking strategies, ways of examining implementation are under-explored. Building sustainability is an important part of implementing complex intervention research, but this too has received little exploration in the implementation literature. This paper explores issues of implementation and sustainability by examining the case of PRISM (Program of Resources, Information and Support for Mothers), a community randomised trial in Victoria, Australia aimed at improving maternal health and wellbeing. It examines documents placed on the project website. Three groups of documents relating to implementation of the intervention were examined - implementation reports, media reports and community newsletters. Analysing these documents allowed a focus on the 'work' of the intervention - who does the work and what activities comprise the work - in order to examine implementation as it relates to sustainability. Document analysis provides a useful way of considering implementation and sustainability of complex intervention research. It can 'value add' to findings from process evaluation and extend our understanding of an intervention beyond outcome measures. Analysis of the documents in this case provides insights into why sustainability of an intervention may be difficult to achieve during implementation.

Methods and metrics challenges of delivery-system research

Background

Many delivery-system interventions are fundamentally about change in social systems (both planned and unplanned). This systems perspective raises a number of methodological challenges for studying the effects of delivery-system change--particularly for answering questions related to whether the change will work under different conditions and how the change is integrated (or not) into the operating context of the delivery system.

Methods

The purpose of this paper is to describe the methodological and measurement challenges posed by five key issues in delivery-system research: (1) modeling intervention context; (2) measuring readiness for change; (3) assessing intervention fidelity and sustainability; (4) assessing complex, multicomponent interventions; and (5) incorporating time in delivery-system models to discuss recommendations for addressing these issues. For each issue, we provide recommendations for how research may be designed and implemented to overcome these challenges.

Results and conclusions

We suggest that a more refined understanding of the mechanisms underlying delivery-system interventions (treatment theory) and the ways in which outcomes for different classes of individuals change over time are fundamental starting points for capturing the heterogeneity in samples of individuals exposed to delivery-system interventions. To support the research recommendations outlined in this paper and to advance understanding of the "why" and "how" questions of delivery-system change and their effects, funding agencies should consider supporting studies with larger organizational sample sizes; longer duration; and nontraditional, mixed-methods designs.

A version of this paper was prepared under contract with the Agency for Healthcare Research and Quality (AHRQ), US Department of Health and Human Services for presentation and discussion at a meeting on "The Challenge and Promise of Delivery System Research," held in Sterling, VA, on February 16-17, 2011. The opinions in the paper are those of the author and do not represent the views or recommendations of AHRQ or the US Department of Health and Human Services.¹

Specifying and reporting complex behaviour change interventions: the need for a scientific method

Complex behaviour change interventions are not well described; when they are described, the terminology used is inconsistent. This constrains scientific replication, and limits the subsequent introduction of successful interventions. Implementation Science is introducing a policy of initially encouraging and subsequently requiring the scientific reporting of complex behaviour change interventions.

Specifying and reporting complex behaviour change interventions: the need for a scientific method

Complex behaviour change interventions are not well described; when they are described, the terminology used is inconsistent. This constrains scientific replication, and limits the subsequent introduction of successful interventions. Implementation Science is introducing a policy of initially encouraging and subsequently requiring the scientific reporting of complex behaviour change interventions.

Steps in the design, development and formative evaluation of obesity prevention-related behavior change trials

Obesity prevention interventions through dietary and physical activity change have generally not been effective. Limitations on possible program effectiveness are herein identified at every step in the mediating variable model, a generic conceptual framework for understanding how interventions may promote behavior change. To minimize these problems, and thereby enhance likely intervention effectiveness, four sequential types of formative studies are proposed: targeted behavior validation, targeted mediator validation, intervention procedure validation, and pilot feasibility intervention. Implementing these studies would establish the relationships at each step in the mediating variable model, thereby maximizing the likelihood that an intervention would work and its effects would be detected. Building consensus among researchers, funding agencies, and journal editors on distinct intervention development studies should avoid identified limitations and move the field forward.