A conceptual framework for implementation fidelity

Evaluating the feasibility of complex interventions in mental health services: standardised measure and reporting guidelines

BACKGROUND

The feasibility of implementation is insufficiently considered in clinical guideline development, leading to human and financial resource wastage.

AIMS

To develop (a) an empirically based standardised measure of the feasibility of complex interventions for use within mental health services and (b) reporting guidelines to facilitate feasibility assessment.

METHOD

A focused narrative review of studies assessing implementation blocks and enablers was conducted with thematic analysis and vote counting used to determine candidate items for the measure. Twenty purposively sampled studies (15 trial reports, 5 protocols) were included in the psychometric evaluation, spanning different interventions types. Cohen's kappa (κ) was calculated for interrater reliability and test-retest reliability.

RESULTS

In total, 95 influences on implementation were identified from 299 references. The final measure - Structured Assessment of FEasibility (SAFE) - comprises 16 items rated on a Likert scale. There was excellent interrater (κ = 0.84, 95% CI 0.79-0.89) and test-retest reliability (κ = 0.89, 95% CI 0.85-0.93). Cost information and training time were the two influences least likely to be reported in intervention papers. The SAFE reporting guidelines include 16 items organised into three categories (intervention, resource consequences, evaluation).

CONCLUSIONS

A novel approach to evaluating interventions, SAFE, supplements efficacy and health economic evidence. The SAFE reporting guidelines will allow feasibility of an intervention to be systematically assessed.

Assessment of fidelity in an educational workshop designed to increase the uptake of a primary care alcohol screening recommendation

RATIONALE, AIMS AND OBJECTIVES

Educational workshops are a commonly used quality improvement intervention. Often delivered by credible local health professionals who do not necessarily have skills in pedagogy, it can be challenging to achieve high intervention fidelity. This paper summarizes the fidelity assessment of a workshop designed to increase the uptake of a primary care alcohol screening recommendation.

METHOD

Delivered in a single health region, the workshop comprised separate sessions delivered by three local health professionals, plus two role plays delivered by a commercial company. Sessions were tailored to local barriers. Meetings were held with presenters and an outline of the barriers was provided. Two researchers attended the workshop, rating the number of specified barriers targeted by presenters and their quality of delivery. Participant responsiveness was measured through attendees' feedback and intervention dose was calculated as the proportion of health professionals who attended and proportion of general practices represented.

RESULTS

Exposure was low, with 62 of 545 health professionals from 30 of a possible 80 practices attending. Sixty-five per cent of the specified barriers were targeted. There was variability in quality of delivery and participant responsiveness; challenges included potential mixed messages, overreliance on didactic methods and certain barriers appearing easier to target than others.

CONCLUSIONS

The framework provided a rounded assessment of intervention fidelity: intervention coverage was low, adherence was moderate and there was variability in the quality of delivery across presenters. Future studies testing the effectiveness of interventions delivered by local experts with and without brief training in pedagogy/behaviour change would be beneficial.

Analyzing barriers and facilitators to the implementation of an action plan to strengthen the midwifery professional role: a Moroccan case study

BACKGROUND

As part of a national strategy for reaching Millennium Development Goals 4 and 5 in Morocco, an action plan covering three systems (sociocultural, educational and professional) was developed to strengthen midwives' professional role in order to contribute to high quality maternity care. This study aimed to understand the implementation process by identifying the characteristics of this intervention and the dimensions of the three-systems which could act as barriers to/facilitators of the implementation process. We used a conceptual framework that builds on Hatem-Asmar's model that describes change in a health professional role; and on the Consolidated Framework for Implementation Research for our analysis.

METHODS

An embedded case study with three levels of analysis was conducted during June and July 2010. Data were collected through 11 semi-structured interviews, 20 focus groups, training session observations and documents. A purposive sample of 106 multi-stakeholders from two Moroccan regions (health professionals, academic staff, students, medical administrative officers and health programmers) and one international consultant were recruited. A thematic analysis was conducted using QDA Miner.

RESULTS

Data showed a failure to carry out the plan as intended. Seventeen barriers and seven facilitators were identified. Misalignment of the values, methods, actors and targets of the sociocultural system with the values, methods and actors of the educational and professional systems, on one hand, and with the intervention, on the other hand, were likely the greatest impediments to implementing the plan. The bureaucratic structure and lack of readiness of the sociocultural system were among the most influential barriers to: dissemination of information, involvement of key actors in the process and readiness of the educational system. The main facilitators were the values promoted related to human rights, and the national and international policies to strengthen midwifery and reduce maternal mortality. The plan was perceived as beneficial, but complex and externally driven.

CONCLUSIONS

The findings suggest that successful implementation requires redesigning the implementation strategy to adapt to the factors identified in our study. The results would be very useful to health planners seeking the expansion of such an intervention to other developing countries looking to strengthen midwives' role and to improve maternity health care services.

What is the extent and quality of documentation and reporting of fidelity to implementation strategies: a scoping review

BACKGROUND

Implementation fidelity is critical to the internal and external validity of implementation research. Much of what is written about implementation fidelity addresses fidelity of evidence-informed interventions rather than fidelity of implementation strategies. The documentation and reporting of fidelity to implementation strategies requires attention. Therefore, in this scoping review, we identify the extent and quality of documentation and reporting of fidelity of implementation strategies that were used to implement evidence-informed interventions.

METHODS

A six-stage methodological framework for scoping studies guided our work. Studies were identified from the outputs of the Effective Practice and Organization of Care (EPOC) review group within the Cochrane Database of Systematic Reviews. EPOC's primary focus, implementation strategies influencing provider behavior change, optimized our ability to identify articles for inclusion. We organized the retrieved articles from the systematic reviews by journal and selected the three journals with the largest number of retrieved articles. Using a data extraction tool, we organized retrieved article data from these three journals. In addition, we summarized implementation strategies using the EPOC categories. Data extraction pertaining to the quality of reporting the fidelity of implementation strategies was facilitated with an "Implementation Strategy Fidelity Checklist" based on definitions adapted from Dusenbury et al. We conducted inter-rater reliability checks for all of the independently scored articles. Using linear regression, we assessed the fidelity scores in relation to the publication year.

RESULTS

Seventy-two implementation articles were included in the final analysis. Researchers reported neither fidelity definitions nor conceptual frameworks for fidelity in any articles. The most frequently employed implementation strategies included distribution of education materials (n = 35), audit and feedback (n = 32), and educational meetings (n = 25). Fidelity of implementation strategies was documented in 51 (71 %) articles. Inter-rater reliability coefficients of the independent reviews for each component of fidelity were as follows: adherence = 0.85, dose = 0.89, and participant responsiveness = 0.96. The mean fidelity score was 2.6 (SD = 2.25). We noted a statistically significant decline in fidelity scores over time.

CONCLUSIONS

In addition to identifying the under-reporting of fidelity of implementation strategies in the health literature, we developed and tested a simple checklist to assess the reporting of fidelity of implementation strategies. More research is indicated to assess the definitions and scoring schema of this checklist. Careful reporting of details about fidelity of implementation strategies will make an important contribution to implementation science.

Development, inter-rater reliability and feasibility of a checklist to assess implementation (Ch-IMP) in systematic reviews: the case of provider-based prevention and treatment programs targeting children and youth

BACKGROUND

Several papers report deficiencies in the reporting of information about the implementation of interventions in clinical trials. Information about implementation is also required in systematic reviews of complex interventions to facilitate the translation and uptake of evidence of provider-based prevention and treatment programs. To capture whether and how implementation is assessed within systematic effectiveness reviews, we developed a checklist for implementation (Ch-IMP) and piloted it in a cohort of reviews on provider-based prevention and treatment interventions for children and young people. This paper reports on the inter-rater reliability, feasibility and reasons for discrepant ratings.

METHODS

Checklist domains were informed by a framework for program theory; items within domains were generated from a literature review. The checklist was pilot-tested on a cohort of 27 effectiveness reviews targeting children and youth. Two raters independently extracted information on 47 items. Inter-rater reliability was evaluated using percentage agreement and unweighted kappa coefficients. Reasons for discrepant ratings were content analysed.

RESULTS

Kappa coefficients ranged from 0.37 to 1.00 and were not influenced by one-sided bias. Most kappa values were classified as excellent (n = 20) or good (n = 17) with a few items categorised as fair (n = 7) or poor (n = 1). Prevalence-adjusted kappa coefficients indicate good or excellent agreement for all but one item. Four areas contributed to scoring discrepancies: 1) clarity or sufficiency of information provided in the review; 2) information missed in the review; 3) issues encountered with the tool; and 4) issues encountered at the review level. Use of the tool demands time investment and it requires adjustment to improve its feasibility for wider use.

CONCLUSIONS

The case of provider-based prevention and treatment interventions showed relevancy in developing and piloting the Ch-IMP as a useful tool for assessing the extent to which systematic reviews assess the quality of implementation. The checklist could be used by authors and editors to improve the quality of systematic reviews, and shows promise as a pedagogical tool to facilitate the extraction and reporting of implementation characteristics.

Interventions in randomised controlled trials in surgery: issues to consider during trial design

Until recently, insufficient attention has been paid to the fact that surgical interventions are complex. This complexity has several implications, including the way in which surgical interventions are described and delivered in trials. In order for surgeons to adopt trial findings, interventions need to be described in sufficient detail to enable accurate replication; however, it may be permissible to allow some aspects to be delivered according to local practice. Accumulating work in this area has identified the need for general guidance on the design of surgical interventions in trial protocols and reports. Key issues to consider when designing surgical interventions include the identification of each surgical intervention and their components, who will deliver the interventions, and where and how the interventions will be standardised and monitored during the trial. The trial design (pragmatic and explanatory), comparator and stage of innovation may also influence the extent of detail required. Thoughtful consideration of surgical interventions in this way may help with the interpretation of trial results and the adoption of successful interventions into clinical practice.

Desirability and feasibility of wireless electronic monitoring of medications in clinical trials

Medication nonadherence is a vexing problem in health care necessitating patients and health professionals' efforts to prevent, minimize, or reverse it. Research participants' inconsistent medication taking obscures treatment efficacy and adds costs to biomedical research. Electronic monitoring devices (EMDs), like the Medication Event Monitoring System (MEMS), have grown in sophistication, providing precise, timely insights into individuals' medication-taking patterns across clinical populations. This article reports on the desirability and feasibility study of using a wireless EMD in clinical research to promote adherence to clinical regimens and research protocols. Nonadherence in transplant patients has been linked to late acute rejection and graft loss. High levels of adherence (97.7 %) were documented for six renal transplant recipients for a mean of 6 months (M = 196.1 ± 71.2 days) who indicated acceptance of the technology. MEMS data confirmed the feasibility of using wireless EMDs to monitor medication use. Monitoring provides greater assurance that research studies reflect the biological impact of medications and provide a basis for targeting adherence enhancement efforts within research investigations.

Implementation science: issues of fidelity to consider in community nursing

Programme implementation is a complex process, and literature around the area of implementation science is growing, particularly with respect to programme fidelity (Dusenbury et al, 2003; Carroll et al, 2007). This paper draws on the findings from an evaluation in one area involving restructuring of the Public Health Nursing service from a generalist, geographically based service, to a team-based specialist approach. From this evaluation, it is clear that careful consideration must be given to a number of key issues that could arise in moving from a pilot project to a more mainstream or widescale implementation. These issues could relate to workforce planning; the role of the team leader; formation, maintenance, and development of teams; expertise and knowledge, including knowledge of local communities; active caseload management; continuity across a range of areas; alignment with primary care teams; co-location; and the need for support, particularly in the IT infrastructure.

Assessing fidelity to treatment delivery in the ICONS (Identifying Continence OptioNs after Stroke) cluster randomised feasibility trial

Background

The implementation of strategies to monitor and enhance treatment fidelity is of paramount importance in trials of complex interventions. A recent framework published by the National Institutes of Health Behavior Change Consortium recommends addressing five areas of treatment fidelity, one of which is delivery of treatment. This study aimed to explore fidelity to treatment delivery of the ICONS intervention (a systematic voiding programme [SVP]). This included exploring the feasibility of a method to assess fidelity to treatment delivery and collecting preliminary evidence of the level of fidelity to SVP delivery in order to inform strategies for improving fidelity in a future trial.

Methods

Delivery of treatment was recorded by nurses through completion of daily clinical logs, which included: a voiding interval, proposed voiding times and actual voiding times. The a priori method for assessment of fidelity – comparing actual voiding times with proposed voiding times – was trialled on a small amount of data. Due to errors in documentation of the voiding intervals and proposed voiding times it was not possible to assess fidelity directly as planned. A new method was devised, which included identification of ‘key quality indicators’.

Results

This new approach to assessing fidelity used key quality indicators based upon presence of the data needed to make the comparison between proposed and actual voiding times. The proportion of clinical logs with correct documentation of voiding intervals and proposed voiding times was less than 40 %. For clinical logs with correct documentation, an actual voiding time within 30 min of the proposed voiding time was identified on approximately 55 % of occasions.

Conclusions

Lessons learnt from this study have implications for the future ICONS definitive trial and for other trials of complex interventions. Implementation of a complex intervention may often deviate from what is intended. While careful consideration should be given to the best method of fidelity assessment, an iterative approach allowing flexibility to adapt pre-planned methods is recommended within feasibility trials. As fidelity to treatment delivery in the ICONS feasibility trial appeared to be relatively low, more attention to implementation strategies will be required in the definitive trial.

Trial registration

Identifier: ISRCTN08609907; date registered: 07/07/2010.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-015-0051-9) contains supplementary material, which is available to authorized users.

Implementation of geriatric assessment-based recommendations in older patients with cancer: A multicentre prospective study

PURPOSE

The main objective of this study was to describe geriatric recommendations based on a geriatric assessment (GA) and to evaluate the implementation of these recommendations.

PATIENTS AND METHODS

A two-step approach of screening followed by a GA was implemented in nine hospitals in Belgium. Patients ≥ 70 years were included at diagnosis or at disease progression/relapse. Concrete geriatric recommendations were systematically documented and reported to the treating physicians and consisted of referrals to professional health care workers. Patient charts were reviewed after one month to verify which geriatric recommendations have been performed.

RESULTS

From August 2011 to July 2012, 1550 patients were included for analysis. The median age was 77 (range: 70-97) and 57.0% were female. A solid tumour was diagnosed in 91.4% and a haematological malignancy in 8.6%. Geriatric screening with the G8 identified 63.6% of the patients for GA (n=986). A median of two geriatric recommendations (range: 1-6) were given for 76.2% (95%CI: 73.4-78.8) of the evaluable patients (n=710). A median of one geriatric recommendation (range: 1-5) was performed in 52.1% (95%CI: 48.4-55.8) of the evaluable patients (n=689). In general, 460 or 35.3% (95%CI: 32.8-38.0) of all the geriatric recommendations were performed. Geriatric recommendations most frequently consisted of referrals to the dietician (60.4%), social worker (40.3%), and psychologist (28.9%).

CONCLUSION

This implementation study provides insight into GA-based recommendations/interventions in daily oncology practice. Geriatric recommendations were given in about three-fourths of patients. About one-third of all geriatric recommendations were performed in approximately half of these patients.

Evaluating the impact of implementation factors on family-based prevention programming: methods for strengthening causal inference

Despite growing recognition of the important role implementation plays in successful prevention efforts, relatively little work has sought to demonstrate a causal relationship between implementation factors and participant outcomes. In turn, failure to explore the implementation-to-outcome link limits our understanding of the mechanisms essential to successful programming. This gap is partially due to the inability of current methodological procedures within prevention science to account for the multitude of confounders responsible for variation in implementation factors (i.e., selection bias). The current paper illustrates how propensity and marginal structural models can be used to improve causal inferences involving implementation factors not easily randomized (e.g., participant attendance). We first present analytic steps for simultaneously evaluating the impact of multiple implementation factors on prevention program outcome. Then, we demonstrate this approach for evaluating the impact of enrollment and attendance in a family program, over and above the impact of a school-based program, within PROSPER, a large-scale real-world prevention trial. Findings illustrate the capacity of this approach to successfully account for confounders that influence enrollment and attendance, thereby more accurately representing true causal relations. For instance, after accounting for selection bias, we observed a 5% reduction in the prevalence of 11th grade underage drinking for those who chose to receive a family program and school program compared to those who received only the school program. Further, we detected a 7% reduction in underage drinking for those with high attendance in the family program.

Enriching Patient-Centered Medical Homes Through Peer Support

Peer supporters are recognized by various designations-community health workers, promotores de salud, lay health advisers-and are community members who work for pay or as volunteers in association with health care systems or nonprofit community organizations and often share ethnicity, language, and socioeconomic status with the mentees that they serve. Although emerging evidence demonstrates the efficacy of peer support at the community level, the adoption and implementation of this resource into patient-centered medical homes (PCMHs) is still under development. To accelerate that integration, this article addresses three major elements of peer support interventions: the functions and features of peer support, a framework and programmatic strategies for implementation, and fiscal models that would support the sustained viability of peer support programs within PCMHs. Key functions of peer support include assistance in daily management of health-related behaviors, social and emotional support, linkage to clinical care, and longitudinal or ongoing support. An organizational model of innovation implementation provides a useful framework for determining how to implement and evaluate peer support programs in PCMHs. Programmatic strategies that can be useful in developing peer support programs within PCMHs include peer coaching or mentoring, group self-management training, and programs designed around the telephone and information technology. Fiscal models for peer support programs include linkages with hospital or health care systems, service- or community-based nonprofit organizations, and partnerships between health care systems and community groups. Peer support promises to enrich PCMHs by activating patients in their self-care, providing culturally sensitive outreach, and opening the way for partnerships with community-based organizations.

Comparative efficacy of interventions to promote hand hygiene in hospital: systematic review and network meta-analysis

OBJECTIVE

To evaluate the relative efficacy of the World Health Organization 2005 campaign (WHO-5) and other interventions to promote hand hygiene among healthcare workers in hospital settings and to summarize associated information on use of resources.

DESIGN

Systematic review and network meta-analysis.

DATA SOURCES

Medline, Embase, CINAHL, NHS Economic Evaluation Database, NHS Centre for Reviews and Dissemination, Cochrane Library, and the EPOC register (December 2009 to February 2014); studies selected by the same search terms in previous systematic reviews (1980-2009).

REVIEW METHODS

Included studies were randomised controlled trials, non-randomised trials, controlled before-after trials, and interrupted time series studies implementing an intervention to improve compliance with hand hygiene among healthcare workers in hospital settings and measuring compliance or appropriate proxies that met predefined quality inclusion criteria. When studies had not used appropriate analytical methods, primary data were re-analysed. Random effects and network meta-analyses were performed on studies reporting directly observed compliance with hand hygiene when they were considered sufficiently homogeneous with regard to interventions and participants. Information on resources required for interventions was extracted and graded into three levels.

RESULTS

Of 3639 studies retrieved, 41 met the inclusion criteria (six randomised controlled trials, 32 interrupted time series, one non-randomised trial, and two controlled before-after studies). Meta-analysis of two randomised controlled trials showed the addition of goal setting to WHO-5 was associated with improved compliance (pooled odds ratio 1.35, 95% confidence interval 1.04 to 1.76; I(2)=81%). Of 22 pairwise comparisons from interrupted time series, 18 showed stepwise increases in compliance with hand hygiene, and all but four showed a trend for increasing compliance after the intervention. Network meta-analysis indicated considerable uncertainty in the relative effectiveness of interventions, but nonetheless provided evidence that WHO-5 is effective and that compliance can be further improved by adding interventions including goal setting, reward incentives, and accountability. Nineteen studies reported clinical outcomes; data from these were consistent with clinically important reductions in rates of infection resulting from improved hand hygiene for some but not all important hospital pathogens. Reported costs of interventions ranged from $225 to $4669 (£146-£3035; €204-€4229) per 1000 bed days.

CONCLUSION

Promotion of hand hygiene with WHO-5 is effective at increasing compliance in healthcare workers. Addition of goal setting, reward incentives, and accountability strategies can lead to further improvements. Reporting of resources required for such interventions remains inadequate.

Approaches for informing optimal dose of behavioral interventions

BACKGROUND

There is little guidance about to how select dose parameter values when designing behavioral interventions.

PURPOSE

The purpose of this study is to present approaches to inform intervention duration, frequency, and amount when (1) the investigator has no a priori expectation and is seeking a descriptive approach for identifying and narrowing the universe of dose values or (2) the investigator has an a priori expectation and is seeking validation of this expectation using an inferential approach.

METHODS

Strengths and weaknesses of various approaches are described and illustrated with examples.

RESULTS

Descriptive approaches include retrospective analysis of data from randomized trials, assessment of perceived optimal dose via prospective surveys or interviews of key stakeholders, and assessment of target patient behavior via prospective, longitudinal, observational studies. Inferential approaches include nonrandomized, early-phase trials and randomized designs.

CONCLUSIONS

By utilizing these approaches, researchers may more efficiently apply resources to identify the optimal values of dose parameters for behavioral interventions.

Evaluation of the fidelity of an interactive face-to-face educational intervention to improve general practitioner management of back pain

OBJECTIVES

Implementation intervention effects can only be fully realised and understood if they are faithfully delivered. However the evaluation of implementation intervention fidelity is not commonly undertaken. The IMPLEMENT intervention was designed to improve the management of low back pain by general medical practitioners. It consisted of a two-session interactive workshop, including didactic presentations and small group discussions by trained facilitators. This study aimed to evaluate the fidelity of the IMPLEMENT intervention by assessing: (1) observed facilitator adherence to planned behaviour change techniques (BCTs); (2) comparison of observed and self-reported adherence to planned BCTs and (3) variation across different facilitators and different BCTs.

DESIGN

The study compared planned and actual, and observed versus self-assessed delivery of BCTs during the IMPLEMENT workshops.

METHOD

Workshop sessions were audiorecorded and transcribed verbatim. Observed adherence of facilitators to the planned intervention was assessed by analysing the workshop transcripts in terms of BCTs delivered. Self-reported adherence was measured using a checklist completed at the end of each workshop session and was compared with the 'gold standard' of observed adherence using sensitivity and specificity analyses.

RESULTS

The overall observed adherence to planned BCTs was 79%, representing moderate-to-high intervention fidelity. There was no significant difference in adherence to BCTs between the facilitators. Sensitivity of self-reported adherence was 95% (95% CI 88 to 98) and specificity was 30% (95% CI 11 to 60).

CONCLUSIONS

The findings suggest that the IMPLEMENT intervention was delivered with high levels of adherence to the planned intervention protocol.

TRIAL REGISTRATION NUMBER

The IMPLEMENT trial was registered in the Australian New Zealand Clinical Trials Registry, ACTRN012606000098538 (http://www.anzctr.org.au/trial_view.aspx?ID=1162).

Implementation of a school-based social and emotional learning intervention: understanding diffusion processes within complex systems

Sporadic and inconsistent implementation remains a significant challenge for social and emotional learning (SEL) interventions. This may be partly explained by the dearth of flexible, causative models that capture the multifarious determinants of implementation practices within complex systems. This paper draws upon Rogers (2003) Diffusion of Innovations Theory to explain the adoption, implementation and discontinuance of a SEL intervention. A pragmatic, formative process evaluation was conducted in alignment with phase 1 of the UK Medical Research Council's framework for Developing and Evaluating Complex Interventions. Employing case-study methodology, qualitative data were generated with four socio-economically and academically contrasting secondary schools in Wales implementing the Student Assistance Programme. Semi-structured interviews were conducted with 15 programme stakeholders. Data suggested that variation in implementation activity could be largely attributed to four key intervention reinvention points, which contributed to the transformation of the programme as it interacted with contextual features and individual needs. These reinvention points comprise the following: intervention training, which captures the process through which adopters acquire knowledge about a programme and delivery expertise; intervention assessment, which reflects adopters' evaluation of an intervention in relation to contextual needs; intervention clarification, which comprises the cascading of knowledge through an organisation in order to secure support in delivery; and intervention responsibility, which refers to the process of assigning accountability for sustainable delivery. Taken together, these points identify opportunities to predict and intervene with potential implementation problems. Further research would benefit from exploring additional reinvention activity.

The Effect of a Meditative Movement Intervention on Quality of Sleep in the Elderly: A Systematic Review and Meta-Analysis

BACKGROUND/OBJECTIVES

Sleep disorders are one of the most common difficulties facing older people. Meditative movement interventions (MMIs), a new category of exercise integrating physical activity and meditation (e.g., t'ai chi, yoga, and qigong), may benefit older people with sleep problems. This systematic review synthesized the evidence on the effect of MMIs on older people's quality of sleep.

METHODS

PubMed, EMBASE, CINAHL, PsycINFO, Scopus, the Cochrane Library, the China Science and Technology Journal Database (CSTJ) and the China National Knowledge Infrastructure (CNKI) were searched-from 1950 to March 2014-for randomized controlled trials. Articles were screened and selected by two researchers. Data were extracted from the included studies using specified forms. The same researchers independently evaluated the quality of each article. A meta-analysis was conducted to examine the pooled effect of MMI on sleep quality compared with the control groups.

RESULTS

Fourteen of 1049 studies were included, involving 1225 participants. Interventions included t'ai chi, yoga, qigong, and multicomponent MMI and lasted 12-24 weeks. Ten studies had a low risk of bias. The interventions resulted in significantly better sleep quality scores than either active therapy or usual care/wait-list controls (standardized mean difference, -0.70;95% confidence interval, -0.96 to -0.43). Subgroup analyses revealed that the effect of MMI on sleep quality was not influenced by the type or duration of the intervention. However, pooled results of studies with intervention frequency of fewer than 3 times per week did not show a positive effect on sleep quality. Lower-quality studies had larger effect sizes than high-quality studies. No adverse events were reported.

CONCLUSION

The current review demonstrated that MMI had a moderate effect on improving the quality of sleep for older people with sleep complaints. However, the evidence is inconclusive because of the varying study quality and MMI modalities. This study highlights the need for higher-quality randomized, controlled trials and more sufficient fidelity of intervention implementation to confirm the results. It also highlights the need for comparing the effectiveness of MMI on different subgroups of the elderly population.

Translation of the Risk Avoidance Partnership (RAP) for Implementation in Outpatient Drug Treatment Clinics

Scientific literature increasingly calls for studies to translate evidence-based interventions into real-world contexts balancing fidelity to the original design and fit to the new setting. The Risk Avoidance Partnership (RAP) is a health promotion intervention originally designed to train active drug users to become Peer Health Advocates. A theoretically driven approach was used to adapt RAP to fit implementation in outpatient methadone treatment clinics and pilot it with clinic patients. Ethnographic observations and process tracking documented the RAP translation and pilot experience, and clinic and community characteristics relevant to program implementation. Clinic administrators, staff, and patients were interviewed on their values, capacities, interest in RAP, perceived challenges of implementing RAP in drug treatment clinics, and experiences during the pilot. Findings indicated that RAP core components can be met when implemented in these settings and RAP can fit with the goals, interests, and other programs of the clinic. Balancing fidelity and fit requires recognition of the mutual impacts RAP and the clinic have on each other, which generate new interactions among staff and require ongoing specification of RAP to keep abreast of clinic and community changes. Collaboration of multiple stakeholders significantly benefited translation and pilot processes.

Shaping the Social: design of a settings-based intervention study to improve well-being and reduce smoking and dropout in Danish vocational schools

Background

The social environment at schools is an important setting to promote educational attainment, and health and well-being of young people. However, within upper secondary education there is a need for evidence-based school intervention programmes. The Shaping the Social intervention is a comprehensive programme integrating social and educational activities to promote student well-being and reduce smoking and dropout in upper secondary vocational education. The evaluation design is reported here.

Methods/design

The evaluation employed a non-randomised cluster controlled design, and schools were selected to either implement the intervention or continue with normal practice for comparison. In the baseline survey conducted 2011–2012, 2,329 students from four intervention schools and 3,371 students from six comparison schools answered a computer-based questionnaire during class, representing 73 % and 81 % of eligible students, and 22 % of all technical/agricultural vocational schools in Denmark. Follow-up assessment was conducted 10 weeks after baseline and at the same time teachers of the intervention classes answered a questionnaire about implementation. School dropout rates will be tracked via national education registers through a 2-year follow-up period.

Discussion

Shaping the Social was designed to address that students at Danish vocational schools constitute a high risk population concerning health behaviour as well as school dropout by modifying the school environment, alongside developing appropriate evaluation strategies. To address difficulties in implementing settings-based interventions, as highlighted in prior research, the strategy was to involve intervention schools in the development of the intervention. Baseline differences will be included in the effectiveness analysis, so will the impact of likely mediators and moderators of the intervention.

Trials registration

ISRCTN57822968. Date of registration: 16/01/2013

Can municipality-based post-discharge follow-up visits including a general practitioner reduce early readmission among the fragile elderly (65+ years old)? A randomized controlled trial

OBJECTIVE

To evaluate how municipality-based post-discharge follow-up visits including a general practitioner and municipal nurse affect early readmission among high-risk older people discharged from a hospital department of internal medicine.

DESIGN AND SETTING

Centrally randomized single-centre pragmatic controlled trial comparing intervention and usual care with investigator-blinded outcome assessment.

INTERVENTION

The intervention was home visits with a general practitioner and municipal nurse within seven days of discharge focusing on medication, rehabilitation plan, functional level, and need for further health care initiatives. The visit was concluded by planning one or two further visits. Controls received standard health care services.

PATIENTS

People aged 65 + years discharged from Holbæk University Hospital, Denmark, in 2012 considered at high risk of readmission.

MAIN OUTCOME MEASURES

The primary outcome was readmission within 30 days. Secondary outcomes at 30 and 180 days included readmission, primary health care, and municipal services. Outcomes were register-based and analysis used the intention-to-treat principle.

RESULTS

A total of 270 and 261 patients were randomized to intervention and control groups, respectively. The groups were similar in baseline characteristics. In all 149 planned discharge follow-up visits were carried out (55%). Within 30 days, 24% of the intervention group and 23% of the control group were readmitted (p = 0.93). No significant differences were found for any other secondary outcomes except that the intervention group received more municipal nursing services.

CONCLUSION

This municipality-based follow-up intervention was only feasible in half the planned visits. The intervention as delivered had no effect on readmission or subsequent use of primary or secondary health care services.

Multicentre cluster randomised trial comparing a community group exercise programme and home-based exercise with usual care for people aged 65 years and over in primary care

BACKGROUND

Regular physical activity (PA) reduces the risk of falls and hip fractures, and mortality from all causes. However, PA levels are low in the older population and previous intervention studies have demonstrated only modest, short-term improvements.

OBJECTIVE

To evaluate the impact of two exercise promotion programmes on PA in people aged ≥ 65 years.

DESIGN

The ProAct65+ study was a pragmatic, three-arm parallel design, cluster randomised controlled trial of class-based exercise [Falls Management Exercise (FaME) programme], home-based exercise [Otago Exercise Programme (OEP)] and usual care among older people (aged ≥ 65 years) in primary care.

SETTING

Forty-three UK-based general practices in London and Nottingham/Derby.

PARTICIPANTS

A total of 1256 people ≥ 65 years were recruited through their general practices to take part in the trial.

INTERVENTIONS

The FaME programme and OEP. FaME included weekly classes plus home exercises for 24 weeks and encouraged walking. OEP included home exercises supported by peer mentors (PMs) for 24 weeks, and encouraged walking.

MAIN OUTCOME MEASURES

The primary outcome was the proportion that reported reaching the recommended PA target of 150 minutes of moderate to vigorous physical activity (MVPA) per week, 12 months after cessation of the intervention. Secondary outcomes included functional assessments of balance and falls risk, the incidence of falls, fear of falling, quality of life, social networks and self-efficacy. An economic evaluation including participant and NHS costs was embedded in the clinical trial.

RESULTS

In total, 20,507 patients from 43 general practices were invited to participate. Expressions of interest were received from 2752 (13%) and 1256 (6%) consented to join the trial; 387 were allocated to the FaME arm, 411 to the OEP arm and 458 to usual care. Primary outcome data were available at 12 months after the end of the intervention period for 830 (66%) of the study participants. The proportions reporting at least 150 minutes of MVPA per week rose between baseline and 12 months after the intervention from 40% to 49% in the FaME arm, from 41% to 43% in the OEP arm and from 37.5% to 38.0% in the usual-care arm. A significantly higher proportion in the FaME arm than in the usual-care arm reported at least 150 minutes of MVPA per week at 12 months after the intervention [adjusted odds ratio (AOR) 1.78, 95% confidence interval (CI) 1.11 to 2.87; p = 0.02]. There was no significant difference in MVPA between OEP and usual care (AOR 1.17, 95% CI 0.72 to 1.92; p = 0.52). Participants in the FaME arm added around 15 minutes of MVPA per day to their baseline physical activity level. In the 12 months after the close of the intervention phase, there was a statistically significant reduction in falls rate in the FaME arm compared with the usual-care arm (incidence rate ratio 0.74, 95% CI 0.55 to 0.99; p = 0.042). Scores on the Physical Activity Scale for the Elderly showed a small but statistically significant benefit for FaME compared with usual care, as did perceptions of benefits from exercise. Balance confidence was significantly improved at 12 months post intervention in both arms compared with the usual-care arm. There were no statistically significant differences between intervention arms and the usual-care arm in other secondary outcomes, including quality-adjusted life-years. FaME is more expensive than OEP delivered with PMs (£269 vs. £88 per participant in London; £218 vs. £117 in Nottingham). The cost per extra person exercising at, or above, target was £1919.64 in London and £1560.21 in Nottingham (mean £1739.93).

CONCLUSION

The FaME intervention increased self-reported PA levels among community-dwelling older adults 12 months after the intervention, and significantly reduced falls. Both the FaME and OEP interventions appeared to be safe, with no significant differences in adverse reactions between study arms.

TRIAL REGISTRATION

This trial is registered as ISRCTN43453770.

FUNDING

This project was funded by the NIHR Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 18, No. 49. See the NIHR Journals Library website for further project information.

The Impact of Implementation Fidelity on Mortality Under a CD4-Stratified Timing Strategy for Antiretroviral Therapy in Patients With Tuberculosis

Among patients with tuberculosis and human immunodeficiency virus type 1, CD4-stratified initiation of antiretroviral therapy (ART) is recommended, with earlier ART in those with low CD4 counts. However, the impact of implementation fidelity to this recommendation is unknown. We examined a prospective cohort study of 395 adult patients diagnosed with tuberculosis and human immunodeficiency virus between August 2007 and November 2009 in Kinshasa, Democratic Republic of the Congo. ART was to be initiated after 1 month of tuberculosis treatment at a CD4 count of <100 cells/mm(3) or World Health Organization stage 4 (other than extrapulmonary tuberculosis) and after 2 months of tuberculosis treatment at a CD4 count of 100-350 cells/mm(3). We used the parametric g-formula to estimate the impact of implementation fidelity on 6-month mortality. Observed implementation fidelity was low (46%); 54% of patients either experienced delays in ART initiation or did not initiate ART, which could be avoided under perfect implementation fidelity. The observed mortality risk was 12.0% (95% confidence interval (CI): 8.2, 15.7); under complete (counterfactual) implementation fidelity, the mortality risk was 7.8% (95% CI: 2.4, 12.3), corresponding to a risk reduction of 4.2% (95% CI: 0.3, 8.1) and a preventable fraction of 35.1% (95% CI: 2.9, 67.9). Strategies to achieve high implementation fidelity to CD4-stratified ART timing are needed to maximize survival benefit.

Do problem-solving interventions improve psychosocial outcomes in vision impaired adults: a systematic review and meta-analysis

OBJECTIVE

To evaluate the effectiveness of problem-solving interventions on psychosocial outcomes in vision impaired adults.

METHODS

A systematic search of randomised controlled trials (RCTs), published between 1990 and 2013, that investigated the impact of problem-solving interventions on depressive symptoms, emotional distress, quality of life (QoL) and functioning was conducted. Two reviewers independently selected and appraised study quality. Data permitting, intervention effects were statistically pooled and meta-analyses were performed, otherwise summarised descriptively.

RESULTS

Eleven studies (reporting on eight trials) met inclusion criteria. Pooled analysis showed problem-solving interventions improved vision-related functioning (standardised mean change [SMC]: 0.15; 95% CI: 0.04-0.27) and emotional distress (SMC: -0.36; 95% CI: -0.54 to -0.19). There was no evidence to support improvements in depressive symptoms (SMC: -0.27, 95% CI: -0.66 to 0.12) and insufficient evidence to determine the effectiveness of problem-solving interventions on QoL.

CONCLUSION

The small number of well-designed studies and narrow inclusion criteria limit the conclusions drawn from this review. However, problem-solving skills may be important for nurturing daily functioning and reducing emotional distress for adults with vision impairment.

PRACTICE IMPLICATIONS

Given the empirical support for the importance of effective problem-solving skills in managing chronic illness, more well-designed RCTs are needed with diverse vision impaired samples.

Medical Crises in Older People: cohort study of older people attending acute medical units, developmental work and randomised controlled trial of a specialist geriatric medical intervention for high-risk older people; cohort study of older people with mental health problems admitted to hospital, developmental work and randomised controlled trial of a specialist medical and mental health unit for general hospital patients with delirium and dementia; and cohort study of residents of care homes and interview study of health-care provision to residents of care homes

Background

This programme of research addressed shortcomings in the care of three groups of older patients: patients discharged from acute medical units (AMUs), patients with dementia and delirium admitted to general hospitals, and care home residents.

Methods

In the AMU workstream we undertook literature reviews, performed a cohort study of older people discharged from AMU (Acute Medical Unit Outcome Study; AMOS), developed an intervention (interface geriatricians) and evaluated the intervention in a randomised controlled trial (Acute Medical Unit Comprehensive Geriatric Assessment Intervention Study; AMIGOS). In the second workstream we undertook a cohort study of older people with mental health problems in a general hospital, developed a specialist unit to care for them and tested the unit in a randomised controlled trial (Trial of an Elderly Acute care Medical and mental health unit; TEAM). In the third workstream we undertook a literature review, a cohort study of a representative sample of care home residents and a qualitative study of the delivery of health care to care home residents.

Results

Although 222 of the 433 (51%) patients recruited to the AMIGOS study were vulnerable enough to be readmitted within 3 months, the trial showed no clinical benefit of interface geriatricians over usual care and they were not cost-effective. The TEAM study recruited 600 patients and there were no significant benefits of the specialist unit over usual care in terms of mortality, institutionalisation, mental or functional outcomes, or length of hospital stay, but there were significant benefits in terms of patient experience and carer satisfaction with care. The medical and mental health unit was cost-effective. The care home workstream found that the organisation of health care for residents in the UK was variable, leaving many residents, whose health needs are complex and unpredictable, at risk of poor health care. The variability of health care was explained by the variability in the types and sizes of homes, the training of care home staff, the relationships between care home staff and the primary care doctors and the organisation of care and training among primary care doctors.

Discussion

The interface geriatrician intervention was not sufficient to alter clinical outcomes and this might be because it was not multidisciplinary and well integrated across the secondary care–primary care interface. The development and evaluation of multidisciplinary and better-integrated models of care is justified. The specialist unit improved the quality of experience of patients with delirium and dementia in general hospitals. Despite the need for investment to develop such a unit, the unit was cost-effective. Such units provide a model of care for patients with dementia and delirium in general hospitals that requires replication. The health status of, and delivery of health care to, care home residents is now well understood. Models of care that follow the principles of comprehensive geriatric assessment would seem to be required, but in the UK these must be sufficient to take account of the current provision of primary health care and must recognise the importance of the care home staff in the identification of health-care needs and the delivery of much of that care.

Trial registration

Current Controlled Trials ISRCTN21800480 (AMIGOS); ClinicalTrials.gov NCT01136148 (TEAM).

Funding

This project was funded by the NIHR Programme Grants for Applied Research programme and will be published in full inProgramme Grants for Applied Research; Vol. 3, No. 4. See the NIHR Journals Library website for further project information.

Implementation of the partograph in India's JSY cash transfer programme for facility births: a mixed methods study in Madhya Pradesh province

OBJECTIVES

To study implementation of partograph use to monitor labour in facilities providing the JSY (Janani Suraksha Yojana) cash transfer programme for facility births in India by determining (1) adherence to partograph use, (2) staff abilities at partograph use and (3) staff responsiveness to the policy on partograph use.

DESIGN

A mixed methods study using Carroll's framework for implementation fidelity. Methods include (1) obstetric case record review, (2) a vignette-based survey among nurse midwives and (3) interviews with staff.

SETTING

Routine use of the partograph is recommended to monitor progress of labour in most low-and middle-income countries (LMICs), including India, although currently available evidence in this regard is insufficient. This study was conducted in the context of the highly successful JSY programme in three districts of Madhya Pradesh province.

PARTICIPANTS

73 different level JSY programme facilities participated in the record review, 233 nurse midwives at these facilities participated in the vignette survey and a total of 11 doctors and midwives participated in the interviews.

RESULTS

The partograph was used in 6% of the 1466 records reviewed. The staff obtained a median score of 1.08 (maximum of 10) at competence in plotting a partograph. Three themes emerged from the qualitative data: (1) partographs are used rarely and retrospectively; (2) training does not support correct use of the partograph; and (3) partographs can be useful but are not feasible.

CONCLUSIONS

Implementation fidelity of partograph use in the JSY programme is low. Successful implementation of the partograph can result in improved quality of care in the JSY programme only if potential moderators to its adherence, such as training, supervision, staff 'buy in' and practice environment are addressed so that staff find a conducive practice environment in which to use the partograph and women find it beneficial to present early in labour.

Improving occupational physicians' adherence to a practice guideline: feasibility and impact of a tailored implementation strategy

BACKGROUND

Although practice guidelines are important tools to improve quality of care, implementation remains challenging. To improve adherence to an evidence-based guideline for the management of mental health problems, we developed a tailored implementation strategy targeting barriers perceived by occupational physicians (OPs). Feasibility and impact on OPs' barriers were evaluated.

METHODS

OPs received 8 training-sessions in small peer-learning groups, aimed at discussing the content of the guideline and their perceived barriers to adhere to guideline recommendations; finding solutions to overcome these barriers; and implementing solutions in practice. The training had a plan-do-check-act (PDCA) structure and was guided by a trainer. Protocol compliance and OPs' experiences were qualitatively and quantitatively assessed. Using a questionnaire, impact on knowledge, attitude, and external barriers to guideline adherence was investigated before and after the training.

RESULTS

The training protocol was successfully conducted; guideline recommendations and related barriers were discussed with peers, (innovative) solutions were found and implemented in practice. The participating 32 OPs were divided into 6 groups and all OPs attended 8 sessions. Of the OPs, 90% agreed that the peer-learning groups and the meetings spread over one year were highly effective training components. Significant improvements (p < .05) were found in knowledge, self-efficacy, motivation to use the guideline and its applicability to individual patients. After the training, OPs did not perceive any barriers related to knowledge and self-efficacy. Perceived adherence increased from 48.8% to 96.8% (p < .01).

CONCLUSIONS

The results imply that an implementation strategy focusing on perceived barriers and tailor-made implementation interventions is a feasible method to enhance guideline adherence. Moreover, the strategy contributed to OPs' knowledge, attitudes, and skills in using the guideline. As a generic approach to overcome barriers perceived in specific situations, this strategy provides a useful method to guideline implementation for other health care professionals too.

Handling a challenging context: experiences of facilitating evidence-based elderly care

AIM

To explore improvement facilitators' experiences of handling their commission to implement evidence-based practice in elderly care for frail older persons.

BACKGROUND

Improvement facilitators were put in place across Sweden in a time-limited project by the government, with one part of the project being to evaluate the model before establishing this facilitation of evidence-based practice in elderly care.

METHOD

Two focus groups were interviewed twice. Each group comprised three respondents. The interviews were analysed using qualitative content analysis.

FINDINGS

A main theme, 'Moving forward by adjusting to the circumstances', described how the improvement facilitators handle their commitment. Five subthemes emerged: identifying barriers, keeping focus, maintaining motivation, building bridges and finding balance.

CONCLUSION

The improvement facilitators' commitment is ambiguous because of unclear leadership of, and responsibility for the national investment. They have to handle leaders' different approaches and justify the need for evidence-based practice. The improvement facilitators did not reflect on the impact of programme adaptations on evidence-based practice.

IMPLICATIONS FOR NURSING MANAGEMENT

The findings emphasise the need for collaboration between the improvement facilitator and the nurse manager. To fully implement evidence-based practice, negotiations with current practitioners for adaptation to local conditions are necessary. Furthermore, the value of improving organisational performance needs to be rigorously communicated throughout the organisation.

Steps for implementing delayed cord clamping in a hospital setting

Background

Delayed umbilical cord clamping (DCC) permits placental-to-newborn transfusion and results in an increased neonatal blood volume at birth. Despite endorsement by numerous medical governing bodies, DCC in preterm newborns has been slow to be adopted into practice. The purpose of this article is to provide a framework to guide medical providers interested in implementing DCC in a hospital setting. A descriptive implementation guideline is presented based on the author’s personal experiences and the steps taken at the University of Washington (UW) to implement DCC in premature newborns <37 weeks’ gestational age. Quality improvement data was obtained to assess compliance with DCC performance over the initial six months following initiation of the treatment protocol in July 2014. An anonymous electronic survey was administered to obstetrical providers in January 2015 to assess DCC policy awareness and adherence.

Results

Important steps to consider regarding implementation of DCC in a hospital settings include applying a multidisciplinary educational approach aimed at motivating potential stakeholders potentially impacted by DCC, addressing safety concerns regarding DCC, and developing a standardized DCC treatment protocol. In the first month following DCC protocol implementation at UW, 79.2% (19/24) of premature newborns admitted to the neonatal intensive care unit received DCC, but compliance decreased over time, with DCC documented in only 40.1% (61/150) of newborns during the 6-month period following implementation. The majority of obstetrician survey respondents (90.9%, 20/22) were aware of the UW DCC policy for preterm deliveries, had performed DCC in the past 6 months (95.5%, 21/22), felt that they had sufficient understanding of the risks and benefits of DCC (90.9%, 20/22) and cited concerns for maternal hemorrhage and the need to resuscitate the baby as the main reasons to perform immediate cord clamping instead of DCC.

Conclusion

Healthcare providers interested in implementing DCC may benefit from a procedural practice plan that includes an assessment of organizational readiness to adopt a DCC protocol, methods to measure and encourage staff compliance, and ways to track outcome data of infants who underwent DCC. Strategies to improve protocol awareness after DCC has been implemented are recommended since compliance may decrease over time.

The effectiveness of toolkits as knowledge translation strategies for integrating evidence into clinical care: a systematic review

OBJECTIVES

The aim of this systematic review was to evaluate the effectiveness of toolkits as a knowledge translation (KT) strategy for facilitating the implementation of evidence into clinical care. Toolkits include multiple resources for educating and/or facilitating behaviour change.

DESIGN

Systematic review of the literature on toolkits.

METHODS

A search was conducted on MEDLINE, EMBASE, PsycINFO and CINAHL. Studies were included if they evaluated the effectiveness of a toolkit to support the integration of evidence into clinical care, and if the KT goal(s) of the study were to inform, share knowledge, build awareness, change practice, change behaviour, and/or clinical outcomes in healthcare settings, inform policy, or to commercialise an innovation. Screening of studies, assessment of methodological quality and data extraction for the included studies were conducted by at least two reviewers.

RESULTS

39 relevant studies were included for full review; 8 were rated as moderate to strong methodologically with clinical outcomes that could be somewhat attributed to the toolkit. Three of the eight studies evaluated the toolkit as a single KT intervention, while five embedded the toolkit into a multistrategy intervention. Six of the eight toolkits were partially or mostly effective in changing clinical outcomes and six studies reported on implementation outcomes. The types of resources embedded within toolkits varied but included predominantly educational materials.

CONCLUSIONS

Future toolkits should be informed by high-quality evidence and theory, and should be evaluated using rigorous study designs to explain the factors underlying their effectiveness and successful implementation.

Staff's views on delivering patient-led therapy during inpatient stroke rehabilitation: a focus group study with lessons for trial fidelity

BACKGROUND

Fidelity to the treatment protocol is key to successful trials but often problematic. This article reports the staff's views on delivering a complex rehabilitation intervention: patient-led therapy during inpatient stroke care.

METHODS

An exploratory qualitative study using focus groups with staff involved in a multicenter (n = 12) feasibility trial of patient-led therapy (the MAESTRO trial) was undertaken as part of the evaluation process. Purposive sampling ensured that participants represented all recruiting sites, relevant professions and levels of seniority. Data analysis used a Framework Approach.

RESULTS

Five focus groups were held involving 30 participants. Five main themes emerged: the effect of the interventions, practical problems, patient-related factors, professional dilemmas, and skills. Staff felt the main effect of the therapies was on patients' autonomy and occupation; the main practical problems were the patients' difficulties in achieving the correct position and a lack of space. Staff clearly identified characteristics that made patient-led therapy unsuitable for some patients. Most staff experienced dilemmas over how to prioritize the trial interventions compared to their usual therapy and other clinical demands. Staff also lacked confidence about how to deliver the interventions, particularly when adapting the interventions to individual needs. For each barrier to implementation, possible solutions were identified. Of these, involving other people and establishing a routine were the most common.

CONCLUSIONS

Delivering rehabilitation interventions within a trial is complex. Staff require time and support to develop the skills, strategies and confidence to identify suitable patients, deliver new treatments, adapt the new treatments to individuals' needs and balance the demands of delivering the trial intervention according to the treatment protocol with other clinical and professional priorities.

TRIAL REGISTRATION

ISRCTN

ISRCTN29533052 . October 2011.

The management of individuals with enduring moderate to severe mental health needs: a participatory evaluation of client journeys and the interface of mental health services with the criminal justice system in Cornwall

Background

Existing research identified substantial gaps between NHS mental health services and the criminal justice system for individuals with enduring moderate to severe mental health needs (EMHN). A pilot study in Cornwall echoed these findings, identifying deficiencies in provision at the interface of police and mental health services.

Aim

To explore the interagency management of individuals with EMHN as they come into contact with the police.

Design

A mixed-methods approach within a community psychology framework to enhance the implementation of findings. Stage 1: policy review and clinical audit to identify a sample of mental health service users who were in contact with the police. Stage 2: case-linkage study of 80 service user journeys through services at the time of three types of police contact (Section 136 detention; arrest for criminal offence and contact that did not result in detention); and a health economics component including analysis of the actual cost of 55 service user journeys and enhanced service scenarios. Stage 3: local stakeholder consultation to validate and contextualise case-linkage findings, including a national event.

Setting

The research site was the county of Cornwall within the organisational contexts of Cornwall Partnership NHS Foundation Trust and Devon & Cornwall Police.

Sample

Proportionate stratified random sampling identified a sample of 80 cases examined in the case-linkage study from the 538 linked cases identified by the clinical audit.

Data sources

Case-linkage and health economics data involved individuals’ police and mental health records; stakeholder consultation data involved focus groups and interviews.

Results

Of the sample of 80 cases examined, 23 individuals had been detained under Section 136 of the Mental Health Act (1983: Great Britain.Mental Health Act 1983.Chapter 20. London: The Stationery Office; 1983) (accounting for 32 detentions), 52 had been detained in custody on suspicion of an offence (accounting for 126 arrests) and 15 had non-detention contact with the police. Findings showed that where police were aware of mental health needs and individuals were on caseload of a Mental Health Team, there was increased interaction and enhanced outcomes for service users and organisations. The health economics scenario modelling suggests that enhancing services has minimal effects on individual level costs compared with current practice.

Conclusions

The research revealed discrepancy in police and mental health professionals’ assessment of risk and interpretation of protocol and highlighted the need for joint interagency protocols and training to improve information sharing between agencies to enhance the management of individuals with enduring moderate to severe mental health needs.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Evidence-based practice implementation: case report of the evolution of a quality improvement program in a multicenter physical therapy organization

BACKGROUND AND PURPOSE

Our nation's suboptimal health care quality and unsustainable costs can be linked to the failure to implement evidence-based interventions. Implementation is the bridge between the decision to adopt a strategy and its sustained use in practice. The purpose of this case report is threefold: (1) to outline the historical implementation of an evidence-based quality improvement project, (2) to describe the program's future direction using a systems perspective to identify implementation barriers, and (3) to provide implications for the profession as it works toward closing the evidence-to-practice gap.

CASE DESCRIPTION

The University of Pittsburgh Medical Center (UPMC) Centers for Rehab Services is a large, multicenter physical therapy organization. In 2005, they implemented a Low Back Initiative utilizing evidence-based protocols to guide clinical decision making.

OUTCOMES

The initial implementation strategy used a multifaceted approach. Formative evaluations were used repeatedly to identify barriers to implementation. Barriers may exist outside the organization, they can be created internally, they may result from personnel, or they may be a direct function of the research evidence. Since the program launch, 3 distinct improvement cycles have been utilized to address identified implementation barriers.

DISCUSSION

Implementation is an iterative process requiring evaluation, measurement, and refinement. During this period, behavior change is actualized as clinicians become increasingly proficient and committed to their use of new evidence. Successfully incorporating evidence into routine practice requires a systems perspective to account for the complexity of the clinical setting. The value the profession provides can be enhanced by improving the implementation of evidence-based strategies. Achieving this outcome will require a concerted effort in all areas of the profession. New skills will be needed by leaders, researchers, managers, and clinicians.

Psychotherapeutic benefits of compassion-focused therapy: an early systematic review

BACKGROUND

Compassion-focused therapy (CFT) is a relatively novel form of psychotherapy that was developed for people who have mental health problems primarily linked to high shame and self-criticism. The aim of this early systematic review was to draw together the current research evidence of the effectiveness of CFT as a psychotherapeutic intervention, and to provide recommendations that may inform the development of further trials.

METHOD

A comprehensive search of electronic databases was undertaken to systematically identify literature relating to the effectiveness of CFT as a psychotherapeutic intervention. Reference lists of key journals were hand searched and contact with experts in the field was made to identify unpublished data.

RESULTS

Fourteen studies were included in the review, including three randomized controlled studies. The findings from the included studies were, in the most part, favourable to CFT, and in particular seemed to be effective for people who were high in self-criticism.

CONCLUSIONS

CFT shows promise as an intervention for mood disorders, particularly those high in self-criticism. However, more large-scale, high-quality trials are needed before it can be considered evidence-based practice. The review highlights issues from the current evidence that may be used to inform such trials.

Factors associated with physical therapists' implementation of physical activity interventions in The Netherlands

BACKGROUND

Physical therapists play an important role in the promotion of physical activity (PA) and the effectiveness of PA interventions. However, little is known about the extent to which they implement PA interventions following the intervention protocol and about the factors influencing their implementation behaviors.

OBJECTIVE

The study objective was to investigate physical therapists' implementation fidelity regarding PA interventions, including completeness and quality of delivery, and influencing factors with a Theoretical Domains Framework-based questionnaire.

DESIGN

The study was based on a cross-sectional design.

METHODS

A total of 268 physical therapists completed the Determinants of Implementation Behavior Questionnaire. Questions about completeness and quality of delivery were based on components and tasks of PA interventions as described by the Royal Dutch Society for Physical Therapy. Multilevel regression analyses were used to identify factors associated with completeness and quality of delivery.

RESULTS

High implementation fidelity was found for the physical therapists, with higher scores for completeness of delivery than for quality of delivery. Physical therapists' knowledge, skills, beliefs about capabilities and consequences, positive emotions, behavioral regulation, and the automaticity of PA intervention delivery were the most important predictors of implementation fidelity. Together, the Theoretical Domains Framework accounted for 23% of the variance in both total completeness and total quality scores.

LIMITATIONS

The cross-sectional design precluded the determination of causal relationships. Also, the use of a self-report measure to assess implementation fidelity could have led to socially desirable responses, possibly resulting in more favorable ratings for completeness and quality.

CONCLUSIONS

This study enhances the understanding of how physical therapists implement PA interventions and which factors influence their behaviors. Knowledge about these factors may assist in the development of strategies to improve physical therapists' implementation behaviors.

Assessing implementation fidelity of a community-based infant and young child feeding intervention in Ethiopia identifies delivery challenges that limit reach to communities: a mixed-method process evaluation study

Background

Program effectiveness is influenced by the degree and quality of implementation, thus requiring careful examination of delivery processes and how the program is or is not being implemented as intended. Implementation fidelity is defined by adherence to intervention design, exposure or dose, quality of delivery, and participant responsiveness. As part of the process evaluation (PE) of Alive & Thrive in Ethiopia, a large-scale initiative to improve infant and young child feeding (IYCF), we assessed these four fidelity elements along three components of its community-based intervention: training of frontline workers (FLWs), delivery of program tools and messages, and supportive supervision.

Methods

Data from a qualitative study among three levels of FLWs (n = 54), i.e. supervisors, health extension workers (HEWs), and community volunteers, and among mothers with children under two years of age (n = 60); and cross-sectional PE surveys with FLWs (n = 504) and mothers (n = 750) in two regions (Tigray and SNNPR) were analyzed to examine program fidelity.

Results

There was strong adherence to the intended cascading design (i.e. transfer of knowledge and information from higher to lower FLW levels) and high exposure to training (95% HEWs and 94% volunteers in Tigray, 68% and 81% respectively in SNNPR). Training quality, assessed by IYCF knowledge and self-reported capacity, was high and increased since baseline. Job aids were used regularly by most supervisors and HEWs, but only 54% of volunteers in Tigray and 39% in SNNPR received them. Quality of program message delivery was lower among volunteers, and aided recall of key messages among mothers was also low. Although FLW supervision exposure was high, content and frequency were irregular.

Conclusions

There is evidence of strong fidelity in training and delivery of program tools and messages at higher FLW levels, but gaps in the reach of these to community volunteers and mothers and variability between regions could limit the potential for impact. Strengthening the linkages between HEWs and volunteers further can help to reach the target households and deliver IYCF results at scale.

A Process Evaluation of an Efficacious Family-Based Intervention to Promote Healthy Eating

Entre Familia: Reflejos de Salud was a successful family-based randomized controlled trial designed to improve dietary behaviors and intake among U.S. Latino families, specifically fruit and vegetable intake. The novel intervention design merged a community health worker ( promotora) model with an entertainment-education component. This process evaluation examined intervention implementation and assessed relationships between implementation factors and dietary change. Participants included 180 mothers randomized to an intervention condition. Process evaluation measures were obtained from participant interviews and promotora notes and included fidelity, dose delivered (i.e., minutes of promotora in-person contact with families, number of promotora home visits), and dose received (i.e., participant use of and satisfaction with intervention materials). Outcome variables included changes in vegetable intake and the use of behavioral strategies to increase dietary fiber and decrease dietary fat intake. Participant satisfaction was high, and fidelity was achieved; 87.5% of families received the planned number of promotora home visits. In the multivariable model, satisfaction with intervention materials predicted more frequent use of strategies to increase dietary fiber ( p ≤ .01). Trends suggested that keeping families in the prescribed intervention timeline and obtaining support from other social network members through sharing of program materials may improve changes. Study findings elucidate the relationship between specific intervention processes and dietary changes.

Identifying Continence OptioNs after Stroke (ICONS): an evidence synthesis, case study and exploratory cluster randomised controlled trial of the introduction of a systematic voiding programme for patients with urinary incontinence after stroke in secondary care

Background

Urinary incontinence (UI) following acute stroke is common, affecting between 40% and 60% of people in hospital, but is often poorly managed.

Aim

To develop, implement and evaluate the preliminary effectiveness and potential cost-effectiveness of a systematic voiding programme (SVP), with or without supported implementation, for the management of UI after stroke in secondary care.

Design

Structured in line with the Medical Research Council framework for the evaluation of complex interventions, the programme comprised two phases: Phase I, evidence synthesis of combined approaches to manage UI post stroke, case study of the introduction of the SVP in one stroke service; Phase II, cluster randomised controlled exploratory trial incorporating a process evaluation and testing of health economic data collection methods.

Setting

One English stroke service (case study) and 12 stroke services in England and Wales (randomised trial).

Participants

Case study, 43 patients; randomised trial, 413 patients admitted to hospital with stroke and UI.

Interventions

A SVP comprising assessment, individualised conservative interventions and weekly review. In the supported implementation trial arm, facilitation was used as an implementation strategy to support and enable people to change their practice.

Main outcome measures

Participant incontinence (presence/absence) at 12 weeks post stroke. Secondary outcomes were quality of life, frequency and severity of incontinence, urinary symptoms, activities of daily living and death, at discharge, 6, 12 and 52 weeks post stroke.

Results

There was no suggestion of a beneficial effect on outcome at 12 weeks post stroke [intervention vs. usual care: odds ratio (OR) 1.02, 95% confidence interval (CI) 0.54 to 1.93; supported implementation vs. usual care: OR 1.06, 95% CI 0.54 to 2.09]. There was weak evidence of better outcomes on the Incontinence Impact Questionnaire in supported implementation (OR 1.22, 95% CI 0.72 to 2.08) but the CI is wide and includes both clinically relevant benefit and harm. Both intervention arms had a higher estimated odds of continence for patients with urge incontinence than usual care (intervention: OR 1.58, 95% CI 0.83 to 2.99; supported implementation: OR 1.73, 95% CI 0.88 to 3.43). The process evaluation showed that the SVP increased the visibility of continence management through greater evaluation of patients’ trajectories and outcomes, and closer attention to workload. In-hospital resource use had to be based on estimates provided by staff. The response rates for the postal questionnaires were 73% and 56% of eligible patients at 12 and 52 weeks respectively. Completion of individual data items varied between 67% and 100%.

Conclusions

The trial was exploratory and did not set out to establish effectiveness; however, there are indications the intervention may be effective in patients with urge and stress incontinence. A definitive trial is now warranted.

Study registration

This study is registered as ISRCTN08609907.

Funding details

The National Institute for Health Research Programme Grants for Applied Research programme. Excess treatment costs and research support costs were funded by participating NHS trusts and health boards, Lancashire and Cumbria and East Anglia Comprehensive Local Research Networks and the Welsh National Institute for Social Care and Health Research.

The challenge of implementing peer-led interventions in a professionalized health service: a case study of the national health trainers service in England

Policy Points: In 2004, England's National Health Service introduced health trainer services to help individuals adopt healthier lifestyles and to redress national health inequalities. Over time these anticipated community-focused services became more NHS-focused, delivering "downstream" lifestyle interventions. At the same time, individuals' lifestyle choices were abstracted from the wider social determinants of health and the potential to address inequalities was diminished. While different service models are needed to engage hard-to-reach populations, the long-term sustainability of any new service model depends on its aligning with the established medical system's characteristics.

CONTEXT

In 2004, the English Public Health White Paper Choosing Health introduced "health trainers" as new members of the National Health Service (NHS) workforce. Health trainers would offer one-to-one peer-support to anyone who wished to adopt and maintain a healthier lifestyle. Choosing Health implicitly envisaged health trainers working in community settings in order to engage "hard-to-reach" individuals and other groups who often have the poorest health but who engage the least with traditional health promotion and other NHS services.

METHODS

During longitudinal case studies of 6 local health trainer services, we conducted in-depth interviews with key stakeholders and analyzed service activity data.

FINDINGS

Rather than an unproblematic and stable implementation of community-focused services according to the vision in Choosing Health, we observed substantial shifts in the case studies' configuration and delivery as the services embedded themselves in the local NHS systems. To explain these observations, we drew on a recently proposed conceptual framework to examine and understand the adoption and diffusion of innovations in health care systems.

CONCLUSIONS

The health trainer services have become more "medicalized" over time, and in doing so, the original theory underpinning the program has been threatened. The paradox is that policymakers and practitioners recognize the need to have a different service model for traditional NHS services if they want hard-to-reach populations to engage in preventive actions as a first step to redress health inequalities. The long-term sustainability of any new service model, however, depends on its aligning with the established medical system's (ie, the NHS's) characteristics.

Context and implementation: A concept analysis towards conceptual maturity

Context and implementation of health interventions have received increasing attention over the past decade, in particular with respect to their influence on the effectiveness and reach of complex interventions. The underlying concepts are both considered partially mature, limiting their operationalization in research and practice. We conducted systematic literature searches and pragmatic utility (PU) concept analyses to provide a state-of-the-art assessment of the concepts of "context" and "implementation" in the health sciences to create a common understanding for their use within systematic reviews and HTA. We performed two separate searches, one for context (EMBASE, MEDLINE) and the other for implementation (Google Scholar) to identify relevant models, theories and frameworks. 17 publications on context and 35 articles on implementation met our inclusion criteria. PU concept analysis comprises three guiding principles: selection of the literature, organization and structuring of the literature, and asking analytic questions of the literature. Both concepts were analyzed according to four features of conceptual maturity, i.e., consensual definitions, clear characteristics, fully described preconditions and outcomes, and delineated boundaries. Context and implementation are highly intertwined, with both concepts influencing and interacting with each other. Context is defined as a set of characteristics and circumstances that surround the implementation effort. Implementation is conceptualized as a planned and deliberately initiated effort with the intention to put an intervention into practice. The concept of implementation presents largely consensual definitions and relatively well-defined boundaries, while distinguishing features, preconditions and outcomes are not yet fully articulated. In contrast, definitions of context vary widely, and boundaries with neighbouring concepts, such as setting and environment, are blurred; characteristics, preconditions and outcomes are ill-defined. Therefore, the maturity of both concepts should be further improved to facilitate operationalization in systematic reviews and HTAs.

Implementation of strategies to increase adolescents' access to fruit and vegetables at school: process evaluation findings from the Boost study

Background

Access to fruit and vegetables (FV) is associated with adolescents’ FV consumption. However, little is known about implementation of strategies to increase access to FV at schools. We examined the implementation of two environmental components designed to increase access to FV at Danish schools.

Methods

We used data from 20 intervention schools involved in the school-based multicomponent Boost trial targeting 13-year-olds’ FV consumption. The environmental components at school included daily provision of free FV and promotion of a pleasant eating environment.

Questionnaire data was collected by the end of the nine-month intervention period among 1,121 pupils (95%), from all school principals (n = 20) and half way through the intervention period and by the end of the intervention among 114 teachers (44%).

The implementation of the components was examined descriptively using the following process evaluation measures; fidelity, dose delivered, dose received and reach. Schools with stable high implementation levels over time were characterised by context, intervention appreciation and implementation of other components.

Results

For all process evaluation measures, the level of implementation varied by schools, classes and over time. Dose received: 45% of pupils (school range: 13-72%, class range: 7-77%) ate the provided FV daily; 68% of pupils (school range: 40-93%, class range: 24-100%) reported that time was allocated to eating FV in class. Reach: The intake of FV provided did not differ by SEP nor gender, but more girls and low SEP pupils enjoyed eating FV together. Dose delivered: The proportion of teachers offering FV at a daily basis decreased over time, while the proportion of teachers cutting up FV increased over time. Schools in which high proportions of teachers offered FV daily throughout the intervention period were characterized by being: small; having a low proportion of low SEP pupils; having a school food policy; high teacher- and pupil intervention appreciation; having fewer teachers who cut up FV; and having high implementation of educational components.

Conclusions

The appliance of different approaches and levels of analyses to describe data provided comprehension and knowledge of the implementation process. This knowledge is crucial for the interpretation of intervention effect.

Trial registration

Current Controlled Trials ISRCTN11666034

From concept to content: assessing the implementation fidelity of a chronic care model for frail, older people who live at home

Background

Implementation fidelity, the degree to which a care program is implemented as intended, can influence program impact. Since results of trials that aim to implement comprehensive care programs for frail, older people have been conflicting, assessing implementation fidelity alongside these trials is essential to differentiate between flaws inherent to the program and implementation issues. This study demonstrates how a theory-based assessment of fidelity can increase insight in the implementation process of a complex intervention in primary elderly care.

Methods

The Geriatric Care Model was implemented among 35 primary care practices in the Netherlands. During home visits, practice nurses conducted a comprehensive geriatric assessment and wrote a tailored care plan. Multidisciplinary team consultations were organized with the aim to enhance the coordination between professionals caring for a single patient with complex needs. To assess fidelity, we identified 5 key intervention components and formulated corresponding research questions using Carroll’s framework for fidelity. Adherence (coverage, frequency, duration, content) was assessed per intervention component during and at the end of the intervention period. Two moderating factors (participant responsiveness and facilitation strategies) were assessed at the end of the intervention.

Results

Adherence to the geriatric assessments and care plans was high, but decreased over time. Adherence to multidisciplinary consultations was initially poor, but increased over time. We found that individual differences in adherence between practice nurses and primary care physicians were moderate, while differences in participant responsiveness (satisfaction, involvement) were more distinct. Nurses deviated from protocol due to contextual factors and personal work routines.

Conclusions

Adherence to the Geriatric Care Model was high for most of the essential intervention components. Study limitations include the limited number of assessed moderating factors. We argue that a longitudinal investigation of adherence per intervention component is essential for a complete understanding of the implementation process, but that such investigations may be complicated by practical and methodological challenges.

Trial registration

The Netherlands National Trial Register (NTR). Trial number: 2160.

Sobriety Treatment and Recovery Teams: Implementation Fidelity and Related Outcomes

BACKGROUND

Although integrated programs between child welfare and substance abuse treatment are recommended for families with co-occurring child maltreatment and substance use disorders, implementing integrated service delivery strategies with fidelity is a challenging process.

OBJECTIVE

This study of the first five years of the Sobriety Treatment and Recovery Team (START) program examines implementation fidelity using a model proposed by Carroll et al. (2007). The study describes the process of strengthening moderators of implementation fidelity, trends in adherence to START service delivery standards, and trends in parent and child outcomes.

METHODS

Qualitative and quantitative measures were used to prospectively study three START sites serving 341 families with 550 parents and 717 children.

RESULTS

To achieve implementation fidelity to service delivery standards required a pre-service year and two full years of operation, persistent leadership, and facilitative actions that challenged the existing paradigm. Over four years of service delivery, the time from the child protective services report to completion of five drug treatment sessions was reduced by an average of 75 days. This trend was associated with an increase in parent retention, parental sobriety, and parent retention of child custody. Conclusions/Importance: Understanding the implementation processes necessary to establish complex integrated programs may support realistic allocation of resources. Although implementation fidelity is a moderator of program outcome, complex inter-agency interventions may benefit from innovative measures of fidelity that promote improvement without extensive cost and data collection burden. The implementation framework applied in this study was useful in examining implementation processes, fidelity, and related outcomes.

HIV self-testing among key populations: an implementation science approach to evaluating self-testing

OBJECTIVES

To review methods for measuring HIV self-testing (HIVST) among key populations, including both conventional approaches and implementation science approaches.

METHODS

We reviewed the literature on evaluating HIVST among key populations.

RESULTS

Simple HIV self-tests have already entered markets in several regions, but metrics required to demonstrate the benefits and costs of HIVST remain simplistic. Conventional measurements of sensitivity, specificity, acceptability, and behavioural preferences must be supplemented with richer implementation science measurement tools and innovative research designs in order to capture data on the following components: how self-testing affects subsequent linkage to confirmatory testing, preventive services and onward steps in the HIV continuum of care; how self-testing can be marketed to reach untested subpopulations; and how self-testing can be sustained based on overarching organisational and financial models. We outline an implementation science research agenda that incorporates these components, drawing from evaluation study designs focused on HIVST and testing in general.

CONCLUSION

HIVST holds great promise for key populations, but must be guided by implementation research to inform programmes and scale up.