Common Elements Approaches to Implementation Research and Practice: Methods and Integration with Intervention Science

Education strategies are the most commonly used in pediatric rehabilitation implementation research: a scoping review

BACKGROUND

Approximately one in six children has a disability, and effective, evidence-based rehabilitation can ameliorate the impact of these conditions over the lifespan. However, implementing interventions in real-world settings remains a challenge. This scoping review aimed to summarize the characteristics, implementation strategies, and outcomes of implementation studies in pediatric rehabilitation.

METHODS

A comprehensive search was conducted in PubMed/MEDLINE, EMBASE, CINAHL, SCOPUS, and Web of Science from the database inception to December 2, 2022. Studies testing implementation strategies in pediatric rehabilitation interventions were included. Data extracted included study characteristics (e.g., country, intervention type, field of rehabilitation), implementation strategies characterized using the Expert Recommendations for Implementing Change taxonomy, and outcomes based on the Implementation Outcomes Framework.

RESULTS

Of the 11,740 studies identified, 44 met the inclusion criteria. Most studies were conducted in the United States (n = 15, 34%) or Canada (n = 10, 23%) and used a mixed-methods design (n = 13, 30%). Interventions primarily targeted motor skills (n = 19, 43%) and were conducted in outpatient settings (n = 14, 32%) or homes (n = 11, 23%). The most commonly used implementation strategies were "train and educate key informant" (n = 21, 48%) and "use evaluative/iterative strategies" (n = 19, 43%). Feasibility (n = 19, 43%) and acceptability (n = 16, 36%) were the most frequently targeted implementation outcomes.

CONCLUSIONS

Reporting implementation strategies and outcomes in pediatric rehabilitation studies is limited and highly variable. Most strategies focused on developing and sharing educational materials, while administrative and systems-level interventions were largely absent. Standardized documentation of implementation strategies and outcomes could advance the field's understanding of the effective development of interventions designed for implementation, encouraging faster uptake of effective interventions.

Process and Implementation Elements of Measurement Feedback Systems: A Systematic Review

Measurement feedback systems (MFS) can help guide treatment and improve clinical outcomes. Studies of MFS are heterogeneous both in execution and results, and the effects of MFS seem restricted by limited attention to process and implementation elements and by limited adoption by health professionals. The current systematic review mapped the use of process and implementation elements in MFS studies. An overview of therapists' use of and attitudes toward MFS is provided. Three-level meta-analyses were used to test theoretically informed process and implementation elements as moderators of the effects of MFS. Hypotheses and general propositions from Clinical Performance Feedback Intervention Theory (CP-FIT) were used to organize the elements of the studies and were used as moderator variables. Previous studies on MFS interventions have had a limited focus on implementation efforts and process elements that may increase the effects of MFS and their use among therapists. Efforts have sparsely been made to reduce barriers to MFS use, and several studies have reported limited engagement with MFS among therapists. Therapists' attitudes toward MFS, feedback, or standardized measures were heterogeneously reported, making data synthesis challenging. Identified process and implementation elements were not significantly associated with effect sizes in the studies and the results did not support the propositions of CP-FIT. The lack of statistically significant associations may be due to limited reporting of details about process and implementation aspects. More research designed to test hypotheses regarding process and implementation elements is needed to improve the use and effects of MFS. Future studies should aspire to report findings in a manner that allows for an understanding of the implementation process and therapists' adoption of these systems.

A Novel Approach to Research Synthesis with the Distillation and Matching Model: Application to the Prevention of Youth Social, Emotional, and Behavioral Problems

It is difficult for consumers to access the evidence base for prevention programs to determine which models or practices have the strongest empirical support for improving youth social, emotional, and behavioral (SEB) outcomes within their specific service contexts. Researchers can address this evidence-to-practice gap through innovations in research synthesis. The Distillation and Matching Model (Chorpita et al., 2005), an approach to research synthesis developed for the mental health field, is designed to identify what works for whom and under what conditions via three steps. In this paper, we describe the Distillation and Matching Model and suggest that applying this approach to the prevention literature for youth SEB problems may help bridge the evidence-to-practice gap. The first step, distillation, involves identifying "practice elements," defined as the goal or general principle guiding a discrete practice (e.g., praise) targeting a specific domain of SEB outcomes. This step produces a standard set of terms for the individual practices used across the literature that are studied in isolation and comprise comprehensive intervention models. The second step involves identifying "common elements," or the practice elements found in studies that meet standards of methodological rigor and report significant improvements in youth SEB outcomes. The third step, "matching," is a method for matching common element profiles (combinations of common elements) to intervention and personal characteristics to identify what combinations of common elements work for whom and under what conditions. The Distillation and Matching Model can provide a method for researchers to generate actionable information about common elements that can be used to develop and evaluate tailored interventions.

A systematic review of experimentally tested implementation strategies across health and human service settings: evidence from 2010-2022

BACKGROUND

Studies of implementation strategies range in rigor, design, and evaluated outcomes, presenting interpretation challenges for practitioners and researchers. This systematic review aimed to describe the body of research evidence testing implementation strategies across diverse settings and domains, using the Expert Recommendations for Implementing Change (ERIC) taxonomy to classify strategies and the Reach Effectiveness Adoption Implementation and Maintenance (RE-AIM) framework to classify outcomes.

METHODS

We conducted a systematic review of studies examining implementation strategies from 2010-2022 and registered with PROSPERO (CRD42021235592). We searched databases using terms "implementation strategy", "intervention", "bundle", "support", and their variants. We also solicited study recommendations from implementation science experts and mined existing systematic reviews. We included studies that quantitatively assessed the impact of at least one implementation strategy to improve health or health care using an outcome that could be mapped to the five evaluation dimensions of RE-AIM. Only studies meeting prespecified methodologic standards were included. We described the characteristics of studies and frequency of implementation strategy use across study arms. We also examined common strategy pairings and cooccurrence with significant outcomes.

FINDINGS

Our search resulted in 16,605 studies; 129 met inclusion criteria. Studies tested an average of 6.73 strategies (0-20 range). The most assessed outcomes were Effectiveness (n=82; 64%) and Implementation (n=73; 56%). The implementation strategies most frequently occurring in the experimental arm were Distribute Educational Materials (n=99), Conduct Educational Meetings (n=96), Audit and Provide Feedback (n=76), and External Facilitation (n=59). These strategies were often used in combination. Nineteen implementation strategies were frequently tested and associated with significantly improved outcomes. However, many strategies were not tested sufficiently to draw conclusions.

CONCLUSION

This review of 129 methodologically rigorous studies built upon prior implementation science data syntheses to identify implementation strategies that had been experimentally tested and summarized their impact on outcomes across diverse outcomes and clinical settings. We present recommendations for improving future similar efforts.

Getting to Fidelity: Consensus Development Process to Identify Core Activities of Implementation Facilitation

Transferring successful implementation strategies from research to practice requires approaches for assessing fidelity to the strategy's core components. Implementation facilitation (IF) is a strategy involving an interactive process of problem-solving, enabling, and supporting individuals in efforts to implement clinical innovations that occurs in the context of a recognized need for improvement and supportive interpersonal relationships. Because IF is a dynamic strategy involving numerous activities, our objective was to conduct a rigorous consensus development process to identify core activities for monitoring fidelity to IF when applied in clinical settings. We first conducted a scoping literature review to identify the range of activities used when IF has been applied in clinical settings, searching multiple citation databases for English-language articles including "facilitation" or other commonly-used terms for the strategy published from 1996-2015. Through multi-stage screening, 135 articles (from 94 studies) were identified for data extraction on IF activities, frequency with which IF activities were identified as 'core' by study authors, and study outcomes. From the literature review, we identified 32 distinct IF activities and developed definitions/examples for each. Next, we conducted a 3-stage, modified-Delphi expert panel consensus development process to identify core IF activities across three implementation phases (i.e., Pre-Implementation, Implementation, Sustainment). The expert panel identified 8 core activities for the Pre-Implementation Phase, 8 core activities for the Implementation Phase, and 4 core activities for the Sustainment Phase. This work provides an important foundation for developing measures/tools to assess use of core IF activities to ensure the strategy is delivered with fidelity.

The core functions and forms paradigm throughout EPIS: designing and implementing an evidence-based practice with function fidelity

There are numerous frameworks for implementing evidence-based practices (EBPs) in novel settings to achieve "fidelity." However, identifying appropriate referents for fidelity poses a challenge. The Core Functions and Forms paradigm offers a model that can inform adaptation decisions throughout all phases of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. We applied the Core Functions-Forms paradigm throughout the Exploration and Preparation phases of EPIS in the design of two EBPs targeting family protective factors among Latinos in San Diego, as well as describe plans for its use in Implementation and Sustainment. We employed a distinct approach for each intervention element to contrast adaptation decisions that prioritize adherence to either form or function fidelity. We describe our application of the functions-forms paradigm within the EPIS framework, focusing on the Preparation phase. We also provide functions-forms matrices that map out the relationship between individual intervention components (forms) and the essential processes (functions) by which components are theorized to exert their impact. This case study of how the core functions-forms framework can be mapped onto EPIS can support a conceptual shift from prioritizing form fidelity to also focusing on function fidelity. This might allow interventionists to target appropriate fidelity referents when adapting an EBP, rather than defaulting to maintaining fidelity to forms as described in the protocol. We see great promise for using this framework for guiding actions throughout all EPIS phases and informing future applications of this paradigm to foster more robust fidelity to function.

Guideline evaluation and implementation mechanisms in school health services (GuideMe): protocol for a hybrid randomized factorial trial

BACKGROUND

Norwegian school health services received a national best-practice guideline in 2017. To promote healthy life skills and identify adolescents needing support, the guideline includes strong recommendations for individual consultations with all 8th graders and increased collaboration with schools. To help implement the recommendations, a blended implementation strategy (SchoolHealth) was co-created with school nurses, students, and stakeholders. SchoolHealth consists of three implementation elements: Digital dialog and administration tool (audit and feedback +), Dialog support (external consultation), and Collaboration materials (targeted dissemination). This hybrid study will test the main and combined effects of the elements on guideline fidelity and effectiveness.

METHODS

The GuideMe study is a factorial cluster randomized controlled trial examining SchoolHealth's effectiveness on guideline fidelity and guideline effectiveness goals. Forty Norwegian secondary schools will be randomized to eight different combinations of the elements in SchoolHealth. Participants will include school nurses and school personnel from these schools, and 8th grade students (n = 1200). Primary outcomes are school nurses' fidelity to the guidelines and student's ability to cope with their life (i.e., health literacy, positive health behaviors and self-efficacy). Quantitative methods will be used to test effects and mechanisms, while mixed- and qualitative methods will be used to explore mechanisms, experiences, and other phenomena in depth. Participants will complete digital questionnaires at the start and end of the schoolyear, and after the consultation during the schoolyear. The study will run in two waves, each lasting for one school year. The multifactorial design allows testing of interactions and main effects due to equal distribution of all factors within each main effect. Sustainment and scale-up of optimized SchoolHealth elements using national infrastructure are simultaneously prepared.

DISCUSSION

The study will investigate possible effects of the implementation elements in isolation and in combination, and hypothesized implementation mechanisms. In-depth study of user experiences will inform improvements to elements in SchoolHealth. The results will yield causal knowledge about implementation strategies and the mechanisms through which they assert effects. Mixed-methods will provide insights into how and when the elements work. Optimizing guideline implementation elements can support adolescents in a crucial life phase.

TRAIL REGISTRATION

ISRCTN24173836. Registration date 8 August 2022.