A conceptual framework for implementation fidelity

A narrative review of generic intervention fidelity measures

To increase the rigor of pediatric rehabilitation research, there is a need to evaluate the degree to which an intervention is conducted as planned (i.e., fidelity). Generic fidelity measures evaluate more than one intervention and often include nonspecific attributes of the therapy process common to both interventions. The objective of this study was to describe the characteristics of generic fidelity measures and examine how these attributes fit with pediatric rehabilitation. A review of generic fidelity measures was conducted utilizing health and education databases. Five generic fidelity measures are described and examined for their applicability to pediatric rehabilitation. The measures were used in nine studies meeting the inclusion criteria, involving people ages from 11 years to >65 years undergoing psychotherapy. From the 76 nonspecific items, 37 items were judged to be applicable to pediatric rehabilitation. Common characteristics of nonspecific attributes with pediatric rehabilitation are discussed, and investigator plans to conduct future testing.

Evaluation of Active Living by Design: implementation patterns across communities

BACKGROUND

Twenty-five cross-sector, multidisciplinary community partnerships received funding through the Active Living by Design (ALbD) national program to design, plan, and implement innovative initiatives to support active living.

PURPOSE

This paper examines implementation patterns across ALbD community partnerships related to community characteristics; preparation efforts; and policy, environmental, programmatic, and promotional strategies.

METHODS

Investigators used a mixed-methods, participatory evaluation design, triangulating multiple qualitative and quantitative data sources collected from 2007 to 2009. Configural frequency analysis facilitated detection of variables as well as configurations of variables occurring more (types) or less (anti-types) frequently than patterns expected by chance alone.

RESULTS

Overall, community partnerships with more preparation activities (assessment, sustainability) implemented a larger number of active living promotions, programs, policy influences, and physical projects, cumulatively (type). Yet, community partnerships working in communities with >40% of the population from a non-Caucasian racial and ethnic background and >40% of the population in poverty implemented fewer active living promotions, programs, policy influences, and physical projects, cumulatively (type).

CONCLUSIONS

The resulting types and anti-types provide insight into patterns across communities that may be ascribed to varying configurations of community contexts, resources, and strategies implemented. Rigorous, systematic examination of the underlying causal structures related to the configurations of community characteristics, preparation efforts, and implementation strategies is needed.

Factors influencing the implementation of a lifestyle counseling program in patients with venous leg ulcers: a multiple case study

Background

Implementation of lifestyle interventions in patient care is a major challenge. Understanding factors that influence implementation is a first step in programs to enhance uptake of these interventions. A lifestyle-counseling intervention, Lively Legs, delivered by trained nurses, can effectively improve the lifestyle in patients with venous leg ulcers. The aim of this study was to identify factors that hindered or facilitated implementation of this intervention in outpatient dermatology clinics and in home care.

Methods

A mixed-methods multiple case study in five purposefully selected healthcare settings in the Netherlands was conducted. Measurements to identify influencing factors before and after implementation of Lively Legs included interviews, focus groups, questionnaires, and nurses’ registration. Analyses focused on qualitative data as the main data source. All data were compared across multiple cases to draw conclusions from the study as a whole.

Results

A total of 53 patients enrolled in the Lively Legs program, which was delivered by 12 trained nurses. Barriers for implementation were mainly organizational. It was difficult to effectively organize reaching and recruiting patients for the program, especially in home care. Main barriers were a lack of a standardized healthcare delivery process, insufficient nursing time, and a lack of motivated nurses to deliver the program. Facilitating factors were nurse-driven coordination of care and a standardized care process to tie Lively Legs into, as this resulted in better patient recruitment and better program implementation.

Conclusions

This study identified a range of factors influencing the implementation of a lifestyle-counseling program, mainly related to the organization of healthcare. Using a case study method proved valuable in obtaining insight into influencing factors for implementation. This study also shed light on a more general issue, which is that leg ulcer care is often fragmented, indicating that quality improvement is needed.

Literature review to identify factors that support implementation of evidence-based practice in residential aged care

The aim was to undertake a review of the literature on change management, quality improvement, evidence-based practice and diffusion of innovations to identify key factors that might influence the uptake and continued use of evidence in residential aged care. The key factors will be used to shape and inform the evaluation of the Encouraging Best Practice in Residential Aged Care Program which commenced in Australia in 2007. MEDLINE, CINAHL and the Cochrane Database of Systematic Reviews were searched using combinations of search terms. Searching focused on existing literature reviews, discussions of relevant conceptual and theoretical frameworks and primary studies that have examined the implementation of evidence-based practice in residential aged care. Keyword searching was supplemented with snowball searching (following up on the references cited in the papers identified by the search), searching by key authors in the field and hand searching of a small number of journals. In general, the period covered by the searches was from 2002 to 2008. The findings from the literature are often equivocal. Analysis and consolidation of factors derived from the literature that might influence the implementation of evidence-based practice resulted in the identification of eight factors: (i) a receptive context for change; (ii) having a model of change to guide implementation; (iii) adequate resources; (iv) staff with the necessary skills; (v) stakeholder engagement, participation and commitment; (vi) the nature of the change in practice; (vii) systems in place to support the use of evidence; and (viii) demonstrable benefits of the change. Most of the literature included in the review is from studies in healthcare and hence the generalisability to residential aged care is largely unknown. However, the focus of this research is on clinical care, within the context of residential aged care, hence the healthcare literature is relevant. The factors are relatively broad and cover the evidence itself, the process of implementation, the context within which evidence will be implemented and the systems and resources to support implementation. It is likely that the factors are not independent of each other. The set of factors will be refined over the course of the evaluation.

A systematic review of hand hygiene improvement strategies: a behavioural approach

BACKGROUND

Many strategies have been designed and evaluated to address the problem of low hand hygiene (HH) compliance. Which of these strategies are most effective and how they work is still unclear. Here we describe frequently used improvement strategies and related determinants of behaviour change that prompt good HH behaviour to provide a better overview of the choice and content of such strategies.

METHODS

Systematic searches of experimental and quasi-experimental research on HH improvement strategies were conducted in Medline, Embase, CINAHL, and Cochrane databases from January 2000 to November 2009. First, we extracted the study characteristics using the EPOC Data Collection Checklist, including study objectives, setting, study design, target population, outcome measures, description of the intervention, analysis, and results. Second, we used the Taxonomy of Behavioural Change Techniques to identify targeted determinants.

RESULTS

We reviewed 41 studies. The most frequently addressed determinants were knowledge, awareness, action control, and facilitation of behaviour. Fewer studies addressed social influence, attitude, self-efficacy, and intention. Thirteen studies used a controlled design to measure the effects of HH improvement strategies on HH behaviour. The effectiveness of the strategies varied substantially, but most controlled studies showed positive results. The median effect size of these strategies increased from 17.6 (relative difference) addressing one determinant to 49.5 for the studies that addressed five determinants.

CONCLUSIONS

By focussing on determinants of behaviour change, we found hidden and valuable components in HH improvement strategies. Addressing only determinants such as knowledge, awareness, action control, and facilitation is not enough to change HH behaviour. Addressing combinations of different determinants showed better results. This indicates that we should be more creative in the application of alternative improvement activities addressing determinants such as social influence, attitude, self-efficacy, or intention.

Assessing fidelity of treatment delivery in group and individual 12-step facilitation

Twelve step facilitation (TSF) is an emerging, empirically supported treatment, the study of which will be strengthened by rigorous fidelity assessment. This report describes the development, reliability and concurrent validity of the Twelve Step Facilitation Adherence Competence Empathy Scale (TSF ACES), a comprehensive fidelity rating scale for group and individual TSF treatment developed for the National Drug Abuse Treatment Clinical Trials Network study, Stimulant Abuser Groups to Engage in 12-Step. Independent raters used TSF ACES to rate treatment delivery fidelity of 966 (97% of total) TSF group and individual sessions. TSF ACES summary measures assessed therapist treatment adherence, competence, proscribed behaviors, empathy and overall session performance. TSF ACES showed fair to good overall reliability; weighted kappa coefficients for 59 co-rated sessions ranged from .31 to 1.00, with a mean of .69. Reliability ratings for session summary measures were good to excellent (.69-.91). Internal consistency for the instrument was variable (.47-.71). Relationships of the TSF ACES summary measures with each other, as well as relationships of the summary measures with a measure of therapeutic alliance provided support for concurrent and convergent validity. Implications and future directions for the use of TSF ACES in clinical trials and community treatment implementation are discussed.

Translating knowledge to practice: optimizing the use of guidelines

Guidelines support health care decision-making but continue to be underused, therefore more research is needed on how they can be better developed and implemented. The same is true of mental health care, for which there is recent growing interest in improving care delivery and associated outcomes by optimizing the use of mental health care guidelines. This editorial describes the key concepts from accumulated research on guideline implementation to suggest a number of avenues for research on implementation of mental health care guidelines.

Examining the relationship between completeness of teachers’ implementation of the Krachtvoer healthy diet programme and changes in students’ dietary intakes

Objective

The goal of the current study was to examine if the completeness of programme implementation and the completeness of implementation of specific programme elements of the Dutch school-based healthy diet promotion programme Krachtvoer are related to short- and longer-term changes in students’ fruit, sweets and breakfast intakes.

Design

Data on students’ dietary intakes were collected 1–4 weeks and 6 months after programme implementation. Teachers filled in a logbook on programme implementation after each lesson. The relationships between changes in students’ dietary intakes and completeness of implementation of the programme and of specific programme elements were tested using mixed linear regression analyses.

Setting

Thirteen Dutch prevocational schools.

Subjects

Eight hundred and seventy-six of the 1117 participating students and eighteen of the twenty-two participating teachers.

Results

Completeness of programme implementation was positively related to an increase in fruit consumption in the short term. Completeness of implementation of food exposure activities and a practical lesson on advertisements were related to an increase in fruit consumption in the short as well as the longer term. No such relationships were found for sweets and breakfast consumption.

Conclusions

Our results indicate that efforts should be made to help teachers implement the programme as fully as possible.

Implementing a surgical checklist: more than checking a box

BACKGROUND

Perioperative checklists are mandated by many hospitals as determined by the reduction in morbidity and mortality seen with the use of the World Health Organization's Surgical Safety Checklist. An adapted perioperative checklist was implemented within our hospital system, and compliance with the checklist was reported to be 100%. We hypothesized that compliance does not measure the fidelity of implementation.

METHODS

During a 7-week period, a prospective study was performed to evaluate the completion of all preincision components of the surgical checklist. Pediatric surgical operations were selected for direct observation. In addition, a poststudy survey was used to assess perception and understanding of the checklist process.

RESULTS

A total of 142 pediatric surgical cases were observed. Hospital reported data demonstrated 100% compliance with the preincision phase of the checklist for these cases. None of the cases completely executed all items on the checklist, and the average number of checklist items performed in the observed cases was 4 of 13. The most commonly performed checkpoint were the confirmation of patient name and procedure (99%) and the "timeout" at the start of the checklist (97%). The rest of the checkpoints were performed in less than 60% of cases. Adherence did not increase during the observation period.

CONCLUSION

These data show that despite the 100% documented completion of the preincision phase of the checklist; most of the individual checkpoints are either not executed as designed or not executed at all. These findings demonstrate lack of checklist implementation fidelity, which may be a reflection a poor implementation and dissemination strategy.

Clarifying differences between review designs and methods

This paper argues that the current proliferation of types of systematic reviews creates challenges for the terminology for describing such reviews. Terminology is necessary for planning, describing, appraising, and using reviews, building infrastructure to enable the conduct and use of reviews, and for further developing review methodology. There is insufficient consensus on terminology for a typology of reviews to be produced and any such attempt is likely to be limited by the overlapping nature of the dimensions along which reviews vary. It is therefore proposed that the most useful strategy for the field is to develop terminology for the main dimensions of variation. Three such main dimensions are proposed: (1) aims and approaches (including what the review is aiming to achieve, the theoretical and ideological assumptions, and the use of theory and logics of aggregation and configuration in synthesis); (2) structure and components (including the number and type of mapping and synthesis components and how they relate); and (3) breadth and depth and the extent of 'work done' in addressing a research issue (including the breadth of review questions, the detail with which they are addressed, and the amount the review progresses a research agenda). This then provides an overarching strategy to encompass more detailed descriptions of methodology and may lead in time to a more overarching system of terminology for systematic reviews.

Qualitative evaluation of a local coronary heart disease treatment pathway: practical implications and theoretical framework

Background

Coronary heart disease (CHD) is a common medical problem in general practice. Due to its chronic character, shared care of the patient between general practitioner (GP) and cardiologist (C) is required. In order to improve the cooperation between both medical specialists for patients with CHD, a local treatment pathway was developed. The objective of this study was first to evaluate GPs’ opinions regarding the pathway and its practical implications, and secondly to suggest a theoretical framework of the findings by feeding the identified key factors influencing the pathway implementation into a multi-dimensional model.

Methods

The evaluation of the pathway was conducted in a qualitative design on a sample of 12 pathway developers (8 GPs and 4 cardiologists) and 4 pathway users (GPs). Face-to face interviews, which were aligned with previously conducted studies of the department and assumptions of the theory of planned behaviour (TPB), were performed following a semi-structured interview guideline. These were audio-taped, transcribed verbatim, coded, and analyzed according to the standards of qualitative content analysis.

Results

We identified 10 frequently mentioned key factors having an impact on the implementation success of the CHD treatment pathway. We thereby differentiated between pathway related (pathway content, effort, individual flexibility, ownership), behaviour related (previous behaviour, support), interaction related (patient, shared care/colleagues), and system related factors (context, health care system). The overall evaluation of the CHD pathway was positive, but did not automatically lead to a change of clinical behaviour as some GPs felt to have already acted as the pathway recommends.

Conclusions

By providing an account of our experience creating and implementing an intersectoral care pathway for CHD, this study contributes to our knowledge of factors that may influence physicians’ decisions regarding the use of a local treatment pathway. An improved adaptation of the pathway in daily practice might be best achieved by a combined implementation strategy addressing internal and external factors. A simple, direct adaptation regards the design of the pathway material (e.g. layout, PC version), or the embedding of the pathway in another programme, like a Disease Management Programme (DMP). In addition to these practical implications, we propose a theoretical framework to understand the key factors’ influence on the pathway implementation, with the identified factors along the microlevel (pathway related factors), the mesolevel (interaction related factors), and system- related factors along the macrolevel.

What happened to the no-wait hospital? A case study of implementation of operational plans for reduced waits

BACKGROUND

Both research and practice show that waiting lists are hard to reduce. Implementing complex interventions for reduced waits is an intricate and challenging process that requires special attention for surrounding factors helping and hindering the implementation. This article reports a case study of a hospital implementation of operational plans for reduced waits, with an emphasis on the process of change.

METHODS

A case study research design, theoretically informed by the Pettigrew and Whipp model of strategic change, was applied. Data were gathered from individual and focus group interviews with informants from different organizational levels at different times and from documents and plans.

FINDINGS

The findings revealed arrangements both helping and hindering the implementation work. Helping factors were the hospital's contemporary savings requirements and experiences from similar change initiatives. Those hindering the actions to plan and agree the changes were unclear support functions and unclear task prioritization.

CONCLUSIONS AND PRACTICE IMPLICATIONS

One contribution of this study is to demonstrate the advantages, disadvantages, and challenges of a contextualized case study for increased understanding of factors influencing organizational change implementation. One lesson for current policy is to regard context factors that are critical for successful implementation.

Contingency management in substance abuse treatment: a structured review of the evidence for its transportability

AIMS

Extant literature on contingency management (CM) transportability, or its transition from academia to community practice, is reviewed. The Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009) guides the examination of this material.

METHODS

PsychInfo and Medline database searches identified 27 publications, with reviewed reference lists garnering 22 others. These 49 sources were examined according to CFIR domains of the intervention, outer setting, inner setting, clinicians, and implementation processes.

RESULTS

Intervention characteristics were focal in 59% of the identified literature, with less frequent focus on clinicians (34%), inner setting (32%), implementation processes (18%), and outer setting (8%). As intervention characteristics, adaptability and trialability most facilitate transportability whereas non-clinical origin, perceived inefficacy or disadvantages, and costs are impediments. Clinicians with a managerial focus and greater clinic tenure and CM experience are candidates to curry organizational readiness for implementation, and combat staff disinterest or philosophical objection. A clinic's technology comfort, staff continuity, and leadership advocacy are inner setting characteristics that prompt effective implementation. Implementation processes in successful demonstration projects include careful fiscal/logistical planning, role-specific staff engagement, practical adaptation in execution, and evaluation via fidelity-monitoring and cost-effectiveness analyses. Outer setting characteristics-like economic policies and inter-agency networking or competition-are salient, often unrecognized influences.

CONCLUSIONS

As most implementation constructs are still moving targets, CM transportability is in its infancy and warrants further scientific attention. More effective dissemination may necessitate that future research weight emphasis on external validity, and utilize models of implementation science.

Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care

Background

Prior studies measuring fidelity of complex interventions have mainly evaluated adherence, and not taken factors affecting adherence into consideration. A need for studies that clarify the concept of fidelity and the function of factors moderating fidelity has been emphasized. The aim of the study was to systematically evaluate implementation fidelity and possible factors influencing fidelity of a complex care continuum intervention for frail elderly people.

Methods

The intervention was a systematization of the collaboration between a nurse with geriatric expertise situated at the emergency department, the hospital ward staff, and a multi-professional team with a case manager in the municipal care services for older people. Implementation was evaluated between September 2008 and May 2010 with observations of work practices, stakeholder interviews, and document analysis according to a modified version of The Conceptual Framework for Implementation Fidelity.

Results

A total of 16 of the 18 intervention components were to a great extent delivered as planned, while some new components were added to the model. No changes in the frequency or duration of the 18 components were observed, but the dose of the added components varied over time. Changes in fidelity were caused in a complex, interrelated fashion by all the moderating factors in the framework, i.e., context, staff and participant responsiveness, facilitation, recruitment, and complexity.

Discussion

The Conceptual Framework for Implementation Fidelity was empirically useful and included comprehensive measures of factors affecting fidelity. Future studies should focus on developing the framework with regard to how to investigate relationships between the moderating factors and fidelity over time.

Trial registration

ClinicalTrials.gov, NCT01260493.

The Role of the Mentor in Supporting New Teachers: Associations with Self-Efficacy, Reflection, and Quality

In our study, we examined variation in mentoring aspects of an induction program for 77 novice teachers and associations with self-efficacy, reflection, and quality of student-teacher interactions. Mentors' previous experience and full- vs. part-time status predicted novices' perception of support, reflection, and observed student-teacher interactions. Time spent with a mentor, participation in mentor-facilitated professional development activities and the quality of mentors' interactions with novice teachers were related to novice teachers' perceptions of mentoring success, self-reflection, and efficacy. The data in our study add to the growing research suggesting the need to look within the mentoring experience to more fully understand the working mechanisms and important contributors to their success.

Therapist competence and treatment adherence for a brief intervention addressing alcohol and violence among adolescents

This study examines therapist competency and treatment adherence for a brief intervention addressing alcohol misuse and violent behaviors among adolescents aged 14-18 years. Three observational measures of fidelity were used by independent raters to evaluate 60 therapist-delivered sessions (M = 32.5 minutes). Individual items from the Content Adherence scale, the Global Rating of Competence (Global Rating of Motivational Interviewing Therapist [GROMIT]), and the Self Exploration and Change Talk (SECT) demonstrated fair to excellent interrater reliability (intraclass correlations ranged from .40 to 1.0). Principal components analysis was used to identify the underlying factor structure of the Content Adherence and the GROMIT. Parallel analysis suggested the extraction of three components for the Content Adherence reflecting the three distinct goals for each segment of the intervention. Two components were identified for the GROMIT representing the general spirit of motivational interviewing and empowerment. Findings provide support for the fidelity instruments adapted for this study and offer direction for future training and clinical supervision.

An intervention fidelity framework for technology-based behavioral interventions

BACKGROUND

Despite the proliferation of health technologies, descriptions of the unique considerations and practical guidance for evaluating the intervention fidelity of technology-based behavioral interventions are lacking.

OBJECTIVES

The aims of this study were to (a) discuss how technology-based behavioral interventions challenge conventions about how intervention fidelity is conceptualized and evaluated, (b) propose an intervention fidelity framework that may be more appropriate for technology-based behavioral interventions, and (c) present a plan for operationalizing each concept in the framework using the intervention fidelity monitoring plan for Pocket PATH (Personal Assistant for Tracking Health), a mobile health technology designed to promote self-care behaviors after lung transplantation, as an exemplar.

METHOD

The literature related to intervention fidelity and technology acceptance was used to identify the issues that are unique to the fidelity of technology-based behavioral interventions and thus important to include in a proposed intervention fidelity framework. An intervention fidelity monitoring plan for technology-based behavioral interventions was developed as an example.

RESULTS

The intervention fidelity monitoring plan was deemed feasible and practical to implement and showed utility in operationalizing the concepts such as assessing interventionists' delivery and participants' acceptance of the technology-based behavioral intervention.

DISCUSSION

The framework has the potential to guide the development of implementation fidelity monitoring tools for other technology-based behavioral interventions. Further application and testing of this framework will allow for a better understanding of the role that technology acceptance plays in the adoption and enactment of the behaviors that technology-based behavioral interventions are intended to promote.

Reducing depression in older home care clients: design of a prospective study of a nurse-led interprofessional mental health promotion intervention

BACKGROUND

Very little research has been conducted in the area of depression among older home care clients using personal support services. These older adults are particularly vulnerable to depression because of decreased cognition, comorbid chronic conditions, functional limitations, lack of social support, and reduced access to health services. To date, research has focused on collaborative, nurse-led depression care programs among older adults in primary care settings. Optimal management of depression among older home care clients is not currently known. The objective of this study is to evaluate the feasibility, acceptability and effectiveness of a 6-month nurse-led, interprofessional mental health promotion intervention aimed at older home care clients with depressive symptoms using personal support services.

METHODS/DESIGN

This one-group pre-test post-test study aims to recruit a total of 250 long-stay (> 60 days) home care clients, 70 years or older, with depressive symptoms who are receiving personal support services through a home care program in Ontario, Canada. The nurse-led intervention is a multi-faceted 6-month program led by a Registered Nurse that involves regular home visits, monthly case conferences, and evidence-based assessment and management of depression using an interprofessional approach. The primary outcome is the change in severity of depressive symptoms from baseline to 6 months using the Centre for Epidemiological Studies in Depression Scale. Secondary outcomes include changes in the prevalence of depressive symptoms and anxiety, health-related quality of life, cognitive function, and the rate and appropriateness of depression treatment from baseline to 12 months. Changes in the costs of use of health services will be assessed from a societal perspective. Descriptive and qualitative data will be collected to examine the feasibility and acceptability of the intervention and identify barriers and facilitators to implementation.

DISCUSSION

Data collection began in May 2010 and is expected to be completed by July 2012. A collaborative nurse-led strategy may provide a feasible, acceptable and effective means for improving the health of older home care clients by improving the prevention, recognition, and management of depression in this vulnerable population. The challenges involved in designing a practical, transferable and sustainable nurse-led intervention in home care are also discussed.

TRIAL REGISTRATION

ClinicalTrials.gov: NCT01407926.

Methodological quality in randomised controlled trials of transcutaneous electric nerve stimulation for pain: low fidelity may explain negative findings

The benefits of transcutaneous electrical nerve stimulation (TENS) for pain relief have not been reliably established, as most systematic reviews find poor methodological quality in many studies. The paradox within the evidence base for TENS is that despite identified sources of bias that may lead to an overestimation of treatment effects, no benefits for TENS can be clearly demonstrated. Conventional assessments of quality assume a single direction of bias, and little work has been undertaken examining other directions of bias. Our hypothesis was that low fidelity in studies (bias leading to an underestimation of treatment effects) may account for inconclusive findings. We included 38 studies from 3 recently published Cochrane systematic reviews that examined TENS for acute, chronic, and cancer pain. We extracted data relating to treatment allocation, application of TENS and to the assessment of outcomes. We quantified these data and judged this against standardised assessment criteria using a "traffic light" approach based on the number of studies reaching the standard. We identified significant sources of potential bias in both directions in relation to study design and implementation fidelity that have not been quantified previously. Suboptimal dosing of TENS and inappropriate outcome assessment were particularly prevalent weaknesses indicating low fidelity. We propose criteria for judging directions of bias in future studies of TENS that may be adapted to assess other trials in which implementation fidelity is important, such as other nonpharmacological interventions for pain. Poor implementation fidelity was identified as a significant source of bias in systematic reviews of TENS studies and might explain lack of consistent treatment effects of TENS in pain. Here, criteria for assessing methodology are proposed for use in designing future clinical trials of TENS.

Navigating patient navigation: crossing health services research and clinical boundaries

As health care delivery becomes increasingly focused on patient-centered models, interventions such as patient navigation that have the potential to improve care coordination garner interest from health care managers and clinicians. The ability to understand how and to what extent patient navigation is successful in addressing coordination issues, however, is hampered by multiple definitions, vague boundaries, and different contextual implementations of patient navigation. Using a systematic review strategy and classification method, we review both the conceptual and empirical literature regarding navigation in multiple clinical contexts. We then describe and conceptualize variation in how patient navigation has been defined, implemented, and theorized to affect outcomes. This review suggests that patient navigation varies along multiple dimensions and that the variation is related to differing resources, constraints, and goals. We propose a conceptual model to frame further research and suggest that research in this area must carefully account for this variation in order to accurately assess the benefits of patient navigation and provide actionable knowledge for managers.

Fidelity of implementation: development and testing of a measure

BACKGROUND

Along with the increasing prevalence of chronic illness has been an increase in interventions, such as nurse case management programs, to improve outcomes for patients with chronic illness. Evidence supports the effectiveness of such interventions in reducing patient morbidity, mortality, and resource utilization, but other studies have produced equivocal results. Often, little is known about how implementation of an intervention actually occurs in clinical practice. While studies often assume that interventions are used in clinical practice exactly as originally designed, this may not be the case. Thus, fidelity of an intervention's implementation reflects how an intervention is, or is not, used in clinical practice and is an important factor in understanding intervention effectiveness and in replicating the intervention in dissemination efforts. The purpose of this paper is to contribute to the understanding of implementation science by (a) proposing a methodology for measuring fidelity of implementation (FOI) and (b) testing the measure by examining the association between FOI and intervention effectiveness.

METHODS

We define and measure FOI based on organizational members' level of commitment to using the distinct components that make up an intervention as they were designed. Semistructured interviews were conducted among 18 organizational members in four medical centers, and the interviews were analyzed qualitatively to assess three dimensions of commitment to use--satisfaction, consistency, and quality--and to develop an overall rating of FOI. Mixed methods were used to explore the association between FOI and intervention effectiveness (inpatient resource utilization and mortality).

RESULTS

Predictive validity of the FOI measure was supported based on the statistical significance of FOI as a predictor of intervention effectiveness. The strongest relationship between FOI and intervention effectiveness was found when an alternative measure of FOI was utilized based on individual intervention components that had the greatest variation across medical centers.

CONCLUSIONS

In addition to contextual factors, implementation research needs to consider FOI as an important factor in influencing intervention effectiveness. Our proposed methodology offers a systematic means for understanding organizational members' use of distinct intervention components, assessing the reasons for variation in use across components and organizations, and evaluating the impact of FOI on intervention effectiveness.

Fidelity: an essential component of evidence-based practice in speech-language pathology

PURPOSE

To provide a primer regarding treatment fidelity as it affects evidence-based practice (EBP) for speech-language pathologists.

METHOD

This tutorial defines treatment fidelity, examines the role of treatment fidelity for speech-language pathologists, provides examples of fidelity measurement, and describes approaches for assessing treatment fidelity.

CONCLUSION

Treatment fidelity is a neglected construct in the EBP literature; however, fidelity is a crucial construct for documenting intervention effectiveness and engaging in EBP.

Systematic evaluation of implementation fidelity of complex interventions in health and social care

Background

Evaluation of an implementation process and its fidelity can give insight into the 'black box' of interventions. However, a lack of standardized methods for studying fidelity and implementation process have been reported, which might be one reason for the fact that few prior studies in the field of health service research have systematically evaluated interventions' implementation processes.

The aim of this project is to systematically evaluate implementation fidelity and possible factors influencing fidelity of complex interventions in health and social care.

Methods

A modified version of The Conceptual Framework for Implementation Fidelity will be used as a conceptual model for the evaluation. The modification implies two additional moderating factors: context and recruitment. A systematic evaluation process was developed. Multiple case study method is used to investigate implementation of three complex health service interventions. Each case will be investigated in depth and longitudinally, using both quantitative and qualitative methods.

Discussion

This study is the first attempt to empirically test The Conceptual Framework for Implementation Fidelity. The study can highlight mechanism and factors of importance when implementing complex interventions. Especially the role of the moderating factors on implementation fidelity can be clarified.

Trial Registration

Supported Employment, SE, among people with severe mental illness -- a randomized controlled trial: NCT00960024.

Increasing delivery of an outdoor journey intervention to people with stroke: A feasibility study involving five community rehabilitation teams

Background

Contrary to recommendations in a national clinical guideline, baseline audits from five community-based stroke rehabilitation teams demonstrated an evidence-practice gap; only 17% of eligible people with stroke were receiving targeted rehabilitation by occupational therapists and physiotherapists to increase outdoor journeys. The primary aim of this feasibility study was to design, test, and evaluate the impact of an implementation program intended to change the behaviour of community rehabilitation teams. A secondary aim was to measure the impact of this change on client outcomes.

Methods

A before-and-after study design was used. The primary data collection method was a medical record audit. Five community rehabilitation teams and a total of 12 professionals were recruited, including occupational therapists, physiotherapists, and a therapy assistant. A medical record audit was conducted twice over 12 months (total of 77 records pre-intervention, 53 records post-intervention) against a guideline recommendation about delivering outdoor journey sessions to people with stroke. A behavioural intervention (the 'Out-and-About Implementation Program') was used to help change team practice. Active components of the intervention included feedback about the audit, barrier identification, and tailored education to target known barriers. The primary outcome measure was the proportion of medical records containing evidence of multiple outdoor journey sessions. Other outcomes of interest included the proportion of medical records that contained evidence of screening for outdoor journeys and driving by team members, and changes in patient outcomes. A small sample of community-dwelling people with stroke (n = 23) provided pre-post outcome data over three months. Data were analysed using descriptive statistics and t-tests.

Results

Medical record audits found that teams were delivering six or more outdoor journeys to 17% of people with stroke pre-intervention, rising to 32% by 12 months post-intervention. This change represents a modest increase in practice behaviour (15%) across teams. More people with stroke (57%) reported getting out of the house as often as they wanted after receiving the outdoor journey intervention compared to 35% one year earlier; other quality of life outcomes also improved.

Conclusions

The 'Out-and-About Implementation Program' helped rehabilitation teams to change their practice, implement evidence, and improve client outcomes. This behavioural intervention requires more rigorous evaluation using a cluster randomised trial design.

The implementation of a translational study involving a primary care based behavioral program to improve blood pressure control: The HTN-IMPROVE study protocol (01295)

BACKGROUND

Despite the impact of hypertension and widely accepted target values for blood pressure (BP), interventions to improve BP control have had limited success.

OBJECTIVES

We describe the design of a 'translational' study that examines the implementation, impact, sustainability, and cost of an evidence-based nurse-delivered tailored behavioral self-management intervention to improve BP control as it moves from a research context to healthcare delivery. The study addresses four specific aims: assess the implementation of an evidence-based behavioral self-management intervention to improve BP levels; evaluate the clinical impact of the intervention as it is implemented; assess organizational factors associated with the sustainability of the intervention; and assess the cost of implementing and sustaining the intervention.

METHODS

The project involves three geographically diverse VA intervention facilities and nine control sites. We first conduct an evaluation of barriers and facilitators for implementing the intervention at intervention sites. We examine the impact of the intervention by comparing 12-month pre/post changes in BP control between patients in intervention sites versus patients in the matched control sites. Next, we examine the sustainability of the intervention and organizational factors facilitating or hindering the sustained implementation. Finally, we examine the costs of intervention implementation. Key outcomes are acceptability and costs of the program, as well as changes in BP. Outcomes will be assessed using mixed methods (e.g., qualitative analyses--pattern matching; quantitative methods--linear mixed models).

DISCUSSION

The study results will provide information about the challenges and costs to implement and sustain the intervention, and what clinical impact can be expected.

Individual and setting level predictors of the implementation of a skin cancer prevention program: a multilevel analysis

BACKGROUND

To achieve widespread cancer control, a better understanding is needed of the factors that contribute to successful implementation of effective skin cancer prevention interventions. This study assessed the relative contributions of individual- and setting-level characteristics to implementation of a widely disseminated skin cancer prevention program.

METHODS

A multilevel analysis was conducted using data from the Pool Cool Diffusion Trial from 2004 and replicated with data from 2005. Implementation of Pool Cool by lifeguards was measured using a composite score (implementation variable, range 0 to 10) that assessed whether the lifeguard performed different components of the intervention. Predictors included lifeguard background characteristics, lifeguard sun protection-related attitudes and behaviors, pool characteristics, and enhanced (i.e., more technical assistance, tailored materials, and incentives are provided) versus basic treatment group.

RESULTS

The mean value of the implementation variable was 4 in both years (2004 and 2005; SD = 2 in 2004 and SD = 3 in 2005) indicating a moderate implementation for most lifeguards. Several individual-level (lifeguard characteristics) and setting-level (pool characteristics and treatment group) factors were found to be significantly associated with implementation of Pool Cool by lifeguards. All three lifeguard-level domains (lifeguard background characteristics, lifeguard sun protection-related attitudes and behaviors) and six pool-level predictors (number of weekly pool visitors, intervention intensity, geographic latitude, pool location, sun safety and/or skin cancer prevention programs, and sun safety programs and policies) were included in the final model. The most important predictors of implementation were the number of weekly pool visitors (inverse association) and enhanced treatment group (positive association). That is, pools with fewer weekly visitors and pools in the enhanced treatment group had significantly higher program implementation in both 2004 and 2005.

CONCLUSIONS

More intense, theory-driven dissemination strategies led to higher levels of implementation of this effective skin cancer prevention program. Issues to be considered by practitioners seeking to implement evidence-based programs in community settings, include taking into account both individual-level and setting-level factors, using active implementation approaches, and assessing local needs to adapt intervention materials.

Measuring implementation fidelity in a community-based parenting intervention

BACKGROUND

Establishing the feasibility and validity of implementation fidelity monitoring strategies is an important methodological step in implementing evidence-based interventions on a large scale.

OBJECTIVES

The objective of the study was to examine the reliability and validity of the Fidelity Checklist, a measure designed to assess group leader adherence and competence delivering a parent training intervention (the Chicago Parent Program) in child care centers serving low-income families.

METHOD

The sample included 9 parent groups (12 group sessions each), 12 group leaders, and 103 parents. Independent raters reviewed 106 audiotaped parent group sessions and coded group leaders' fidelity on the Adherence and Competence Scales of the Fidelity Checklist. Group leaders completed self-report adherence checklists and a measure of parent engagement in the intervention. Parents completed measures of consumer satisfaction and child behavior.

RESULTS

High interrater agreement (Adherence Scale = 94%, Competence Scale = 85%) and adequate intraclass correlation coefficients (Adherence Scale = .69, Competence Scale = .91) were achieved for the Fidelity Checklist. Group leader adherence changed over time, but competence remained stable. Agreement between group leader self-report and independent ratings on the Adherence Scale was 85%; disagreements were more frequently due to positive bias in group leader self-report. Positive correlations were found between group leader adherence and parent attendance and engagement in the intervention and between group leader competence and parent satisfaction. Although child behavior problems improved, improvements were not related to fidelity.

DISCUSSION

The results suggest that the Fidelity Checklist is a feasible, reliable, and valid measure of group leader implementation fidelity in a group-based parenting intervention. Future research will be focused on testing the Fidelity Checklist with diverse and larger samples and generalizing to other group-based interventions using a similar intervention model.

Workplace involvement improves return to work rates among employees with back pain on long-term sick leave: a systematic review of the effectiveness and cost-effectiveness of interventions

PURPOSE

Long-term sickness absence among workers is a major problem in industrialised countries. The aim of the review is to determine whether interventions involving the workplace are more effective and cost-effective at helping employees on sick leave return to work than those that do not involve the workplace at all.

METHODS

A systematic review of controlled intervention studies and economic evaluations. Sixteen electronic databases and grey literature sources were searched, and reference and citation tracking was performed on included publications. A narrative synthesis was performed.

RESULTS

Ten articles were found reporting nine trials from Europe and Canada, and four articles were found evaluating the cost-effectiveness of interventions. The population in eight trials suffered from back pain and related musculoskeletal conditions. Interventions involving employees, health practitioners and employers working together, to implement work modifications for the absentee, were more consistently effective than other interventions. Early intervention was also found to be effective. The majority of trials were of good or moderate quality. Economic evaluations indicated that interventions with a workplace component are likely to be more cost effective than those without.

CONCLUSION

Stakeholder participation and work modification are more effective and cost effective at returning to work adults with musculoskeletal conditions than other workplace-linked interventions, including exercise.

Implementation fidelity in community-based interventions

Implementation fidelity is the degree to which an intervention is delivered as intended and is critical to successful translation of evidence-based interventions into practice. Diminished fidelity may be why interventions that work well in highly controlled trials may fail to yield the same outcomes when applied in real life contexts. The purpose of this paper is to define implementation fidelity and describe its importance for the larger science of implementation, discuss data collection methods and current efforts in measuring implementation fidelity in community-based prevention interventions, and present future research directions for measuring implementation fidelity that will advance implementation science.

Motivational intervention to reduce rapid subsequent births to adolescent mothers: a community-based randomized trial

PURPOSE

One-quarter of adolescent mothers bear another child within 2 years, compounding their risk of poorer medical, educational, economic, and parenting outcomes. Most efforts to prevent rapid subsequent birth to teenagers have been unsuccessful but have seldom addressed motivational processes.

METHODS

We conducted a randomized trial to determine the effectiveness of a computer-assisted motivational intervention (CAMI) in preventing rapid subsequent birth to adolescent mothers. Pregnant teenagers (N = 235), aged 18 years and older who were at more than 24 weeks' gestation, were recruited from urban prenatal clinics serving low-income, predominantly African American communities. After completing baseline assessments, they were randomly assigned to 3 groups: (1) those in CAMI plus enhanced home visit (n = 80) received a multi-component home-based intervention (CAMI+); (2) those in CAMI-only (n = 87) received a single component home-based intervention; (3) and those in usual-care control (n = 68) received standard usual care. Teens in both intervention groups received CAMI sessions at quarterly intervals until 2 years' postpartum. Those in the CAMI+ group also received monthly home visits with parenting education and support. CAMI algorithms, based on the transtheoretical model, assessed sexual relationships and contraception-use intentions and behaviors, and readiness to engage in pregnancy prevention. Trained interventionists used CAMI risk summaries to guide motivational interviewing. Repeat birth by 24 months' postpartum was measured with birth certificates.

RESULTS

Intent-to-treat analysis indicated that the CAMI+ group compared with the usual-care control group exhibited a trend toward lower birth rates (13.8% vs 25.0%; P = .08), whereas the CAMI-only group did not (17.2% vs 25.0%; P = .32). Controlling for baseline group differences, the hazard ratio (HR) for repeat birth was significantly lower for the CAMI+ group than it was with the usual-care group (HR = 0.45; 95% CI, 0.21-0.98). We developed complier average causal effects models to produce unbiased estimates of intervention effects accounting for variable participation. Completing 2 or more CAMI sessions significantly reduced the risk of repeat birth in both groups: CAMI+ (HR = 0.40; 95% CI, 0.16-0.98) and CAMI-only (HR = 0.19; 95% CI, 0.05-0.69).

CONCLUSIONS

Receipt of 2 or more CAMI sessions, either alone or within a multicomponent home-based intervention, reduced the risk of rapid subsequent birth to adolescent mothers.

Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science

BACKGROUND

Many interventions found to be effective in health services research studies fail to translate into meaningful patient care outcomes across multiple contexts. Health services researchers recognize the need to evaluate not only summative outcomes but also formative outcomes to assess the extent to which implementation is effective in a specific setting, prolongs sustainability, and promotes dissemination into other settings. Many implementation theories have been published to help promote effective implementation. However, they overlap considerably in the constructs included in individual theories, and a comparison of theories reveals that each is missing important constructs included in other theories. In addition, terminology and definitions are not consistent across theories. We describe the Consolidated Framework For Implementation Research (CFIR) that offers an overarching typology to promote implementation theory development and verification about what works where and why across multiple contexts.

METHODS

We used a snowball sampling approach to identify published theories that were evaluated to identify constructs based on strength of conceptual or empirical support for influence on implementation, consistency in definitions, alignment with our own findings, and potential for measurement. We combined constructs across published theories that had different labels but were redundant or overlapping in definition, and we parsed apart constructs that conflated underlying concepts.

RESULTS

The CFIR is composed of five major domains: intervention characteristics, outer setting, inner setting, characteristics of the individuals involved, and the process of implementation. Eight constructs were identified related to the intervention (e.g., evidence strength and quality), four constructs were identified related to outer setting (e.g., patient needs and resources), 12 constructs were identified related to inner setting (e.g., culture, leadership engagement), five constructs were identified related to individual characteristics, and eight constructs were identified related to process (e.g., plan, evaluate, and reflect). We present explicit definitions for each construct.

CONCLUSION

The CFIR provides a pragmatic structure for approaching complex, interacting, multi-level, and transient states of constructs in the real world by embracing, consolidating, and unifying key constructs from published implementation theories. It can be used to guide formative evaluations and build the implementation knowledge base across multiple studies and settings.

Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science

BACKGROUND

Many interventions found to be effective in health services research studies fail to translate into meaningful patient care outcomes across multiple contexts. Health services researchers recognize the need to evaluate not only summative outcomes but also formative outcomes to assess the extent to which implementation is effective in a specific setting, prolongs sustainability, and promotes dissemination into other settings. Many implementation theories have been published to help promote effective implementation. However, they overlap considerably in the constructs included in individual theories, and a comparison of theories reveals that each is missing important constructs included in other theories. In addition, terminology and definitions are not consistent across theories. We describe the Consolidated Framework For Implementation Research (CFIR) that offers an overarching typology to promote implementation theory development and verification about what works where and why across multiple contexts.

METHODS

We used a snowball sampling approach to identify published theories that were evaluated to identify constructs based on strength of conceptual or empirical support for influence on implementation, consistency in definitions, alignment with our own findings, and potential for measurement. We combined constructs across published theories that had different labels but were redundant or overlapping in definition, and we parsed apart constructs that conflated underlying concepts.

RESULTS

The CFIR is composed of five major domains: intervention characteristics, outer setting, inner setting, characteristics of the individuals involved, and the process of implementation. Eight constructs were identified related to the intervention (e.g., evidence strength and quality), four constructs were identified related to outer setting (e.g., patient needs and resources), 12 constructs were identified related to inner setting (e.g., culture, leadership engagement), five constructs were identified related to individual characteristics, and eight constructs were identified related to process (e.g., plan, evaluate, and reflect). We present explicit definitions for each construct.

CONCLUSION

The CFIR provides a pragmatic structure for approaching complex, interacting, multi-level, and transient states of constructs in the real world by embracing, consolidating, and unifying key constructs from published implementation theories. It can be used to guide formative evaluations and build the implementation knowledge base across multiple studies and settings.

Implementation Fidelity of MyTeachingPartner Literacy and Language Activities: Association with Preschoolers' Language and Literacy Growth

There is surprisingly little empirical research examining issues of fidelity of implementation within the early childhood education literature. In the MyTeachingPartner project, 154 teachers were provided with materials to implement a supplemental classroom curriculum addressing six aspects of literacy and language development. The present study examines the degree of variability in three aspects of implementation fidelity - dosage, adherence, and quality of delivery - and whether these components of fidelity were associated with children's growth in language and literacy skills across the preschool year. Findings indicate that teachers reported using the curriculum fairly often (dosage) and that they were observed to generally follow curricular lesson plans (adherence). In contrast, the quality of delivery, defined as the use of evidence-based teacher-child interactions for teaching literacy and language, was much lower. Children in classrooms in which activities were observed to last for longer (dosage) and in which teachers exhibited higher quality of delivery of literacy lessons made significantly greater gains in early literacy skills across the preschool year. Also, teachers' use of higher quality language interactions was associated with gains for children who did not speak English at home. Results have implications for teacher professional development and the supports provided to ensure that curricula are delivered most effectively.

Teacher Characteristics Associated with Responsiveness and Exposure to Consultation and On-line Professional Development Resources

This paper is a natural follow-up to intent-to-treat findings indicating that the MyTeachingPartner Consultancy, inclusive of on-line video resources and web-mediated consultation, improved the quality of pre-k teachers' interactions with children. This study takes a close look at implementation fidelity within the effective MTP Consultancy condition over both years of implementation, in order to learn more about the ingredients of professional development that may have contributed to the success of the intervention. Variation in teachers' responsiveness (e.g., ratings of Consultancy worth) and exposure to the intervention (e.g., number of consultation cycles completed) are examined, with particular interest in the identification of teacher factors that may serve as supports or barriers to successfully implementing consultation supports and on-line professional development resources.

Fidelity versus flexibility: translating evidence-based research into practice

BACKGROUND

Understanding the process by which research is translated into practice is limited. This study sought to examine how interventions change during implementation.

METHODS

Data were collected from July 2005 to September 2007. A real-time and cross-case comparison was conducted, examining ten interventions designed to improve health promotion in primary care practices in practice-based research networks. An iterative group process was used to analyze qualitative data (survey data, interviews, site visits, and project diary entries made by grantees approximately every 2 weeks) and to identify intervention adaptations reported during implementation.

RESULTS

All interventions required changes as they were integrated into practice. Modifications differed by project and by practice, and were often unanticipated. Three broad categories of changes were identified and include modifications undertaken to accommodate practices' and patients' circumstances as well as personnel costs. In addition, research teams played a crucial role in fostering intervention uptake through their use of personal influence and by providing motivation, retraining, and instrumental assistance to practices. These efforts by the research teams, although rarely considered an essential component of the intervention, were an active ingredient in successful implementation and translation.

CONCLUSIONS

Changes are common when interventions are implemented into practice settings. The translation of evidence into practice will be improved when research design and reporting standards are modified to help quality-improvement teams understand both these adaptations and the effort required to implement interventions in practice.

A cluster randomized trial to improve adherence to evidence-based guidelines on diabetes and reduce clinical inertia in primary care physicians in Belgium: study protocol [NTR 1369]

BACKGROUND

Most quality improvement programs in diabetes care incorporate aspects of clinician education, performance feedback, patient education, care management, and diabetes care teams to support primary care physicians. Few studies have applied all of these dimensions to address clinical inertia.

AIM

To evaluate interventions to improve adherence to evidence-based guidelines for diabetes and reduce clinical inertia in primary care physicians.

DESIGN

Two-arm cluster randomized controlled trial.

PARTICIPANTS

Primary care physicians in Belgium.

INTERVENTIONS

Primary care physicians will be randomly allocated to 'Usual' (UQIP) or 'Advanced' (AQIP) Quality Improvement Programs. Physicians in the UQIP will receive interventions addressing the main physician, patient, and office system factors that contribute to clinical inertia. Physicians in the AQIP will receive additional interventions that focus on sustainable behavior changes in patients and providers.

OUTCOMES

Primary endpoints are the proportions of patients within targets for three clinical outcomes: 1) glycosylated hemoglobin < 7%; 2) systolic blood pressure differences < or =130 mmHg; and 3) low density lipoprotein/cholesterol < 100 mg/dl. Secondary endpoints are individual improvements in 12 validated parameters: glycosylated hemoglobin, low and high density lipoprotein/cholesterol, total cholesterol, systolic blood pressure, diastolic blood pressure, weight, physical exercise, healthy diet, smoking status, and statin and anti-platelet therapy. PRIMARY AND SECONDARY ANALYSIS: Statistical analyses will be performed using an intent-to-treat approach with a multilevel model. Linear and generalized linear mixed models will be used to account for the clustered nature of the data, i.e., patients clustered withinimary care physicians, and repeated assessments clustered within patients. To compare patient characteristics at baseline and between the intervention arms, the generalized estimating equations (GEE) approach will be used, taking the clustered nature of the data within physicians into account. We will also use the GEE approach to test for differences in evolution of the primary and secondary endpoints for all patients, and for patients in the two interventions arms, accounting for within-patient clustering.