Measuring implementation climate: psychometric properties of the Implementation Climate Scale (ICS) in Norwegian mental health care services

Guideline evaluation and implementation mechanisms in school health services (GuideMe): protocol for a hybrid randomized factorial trial

BACKGROUND

Norwegian school health services received a national best-practice guideline in 2017. To promote healthy life skills and identify adolescents needing support, the guideline includes strong recommendations for individual consultations with all 8th graders and increased collaboration with schools. To help implement the recommendations, a blended implementation strategy (SchoolHealth) was co-created with school nurses, students, and stakeholders. SchoolHealth consists of three implementation elements: Digital dialog and administration tool (audit and feedback +), Dialog support (external consultation), and Collaboration materials (targeted dissemination). This hybrid study will test the main and combined effects of the elements on guideline fidelity and effectiveness.

METHODS

The GuideMe study is a factorial cluster randomized controlled trial examining SchoolHealth's effectiveness on guideline fidelity and guideline effectiveness goals. Forty Norwegian secondary schools will be randomized to eight different combinations of the elements in SchoolHealth. Participants will include school nurses and school personnel from these schools, and 8th grade students (n = 1200). Primary outcomes are school nurses' fidelity to the guidelines and student's ability to cope with their life (i.e., health literacy, positive health behaviors and self-efficacy). Quantitative methods will be used to test effects and mechanisms, while mixed- and qualitative methods will be used to explore mechanisms, experiences, and other phenomena in depth. Participants will complete digital questionnaires at the start and end of the schoolyear, and after the consultation during the schoolyear. The study will run in two waves, each lasting for one school year. The multifactorial design allows testing of interactions and main effects due to equal distribution of all factors within each main effect. Sustainment and scale-up of optimized SchoolHealth elements using national infrastructure are simultaneously prepared.

DISCUSSION

The study will investigate possible effects of the implementation elements in isolation and in combination, and hypothesized implementation mechanisms. In-depth study of user experiences will inform improvements to elements in SchoolHealth. The results will yield causal knowledge about implementation strategies and the mechanisms through which they assert effects. Mixed-methods will provide insights into how and when the elements work. Optimizing guideline implementation elements can support adolescents in a crucial life phase.

TRAIL REGISTRATION

ISRCTN24173836. Registration date 8 August 2022.

Individual-level associations between implementation leadership, climate, and anticipated outcomes: a time-lagged mediation analysis

BACKGROUND

Leaders can improve implementation outcomes by developing an organizational climate conducive to the implementation of evidence-based practices (EBP). This study tested the lagged associations between individual-level perceptions of implementation leadership, implementation climate, and three anticipated implementation outcomes, that is EBP acceptability, appropriateness, and feasibility.

METHODS

Screening tools and treatment methods for posttraumatic stress disorder were implemented in 43 Norwegian mental health services. A sample of 494 child and adult mental health care professionals (M = 43 years, 78% female) completed surveys addressing perceptions of first-level leaders' (n = 47) implementation leadership and their clinics' implementation climate. Single-level structural equation models estimating both direct, indirect, and total effects were used to investigate whether perceived implementation climate mediated the association between perceived implementation leadership and perceived acceptability, appropriateness, and feasibility of screening tools and treatment methods.

RESULTS

Regarding the treatment methods, implementation leadership was associated with therapists' perceptions of acceptability, appropriateness, and feasibility. Implementation climate also mediated between implementation leadership and the outcomes. Regarding the screening tools, implementation leadership was not associated with the outcomes. However, implementation climate mediated between implementation leadership and therapists' perceptions of acceptability and feasibility, but not appropriateness. Analyses with the implementation climate subscales showed stronger associations for therapists' perceptions of the treatment methods than of screening tools.

CONCLUSIONS

Leaders may promote positive implementation outcomes, both directly and through implementation climate. With regard to the effect sizes and explained variance, results indicated that both implementation leadership and implementation climate were more strongly associated with the therapists' perceptions of the treatment methods, implemented by one group of therapists, than the screening tools, implemented by all therapists. This may imply that implementation leadership and climate may have stronger effects for smaller implementation teams within a larger system than for system-wide implementations or when the clinical interventions being implemented are more complex rather than simple ones.

TRIAL REGISTRATION

ClinicalTrials NCT03719651, 25 October 2018.

Dynamic interplay between available resources and implementation climate across phases of implementation: a qualitative study of a VA national population health tool

BACKGROUND

Available resources within an organization can determine the implementation success of an intervention. However, few studies have investigated how the required resources change over the phases of implementation. Using stakeholder interviews, we examined the changes in and interactions between available resources and implementation climate in the implementation and sustainment phases of a national implementation effort for a population health tool.

METHODS

We conducted a secondary analysis of the interviews with 20 anticoagulation professionals at 17 clinical sites in the Veterans Health Administration health system about their experiences with a population health dashboard for anticoagulant management. Interview transcripts were coded using constructs from the Consolidated Framework for Implementation Research (CFIR) and according to the phase of implementation (pre-implementation, implementation, and sustainment) as defined by the VA Quality Enhancement Research Initiative (QUERI) Roadmap. We analyzed the factors that may determine successful implementation by examining the co-occurrence patterns between available resources and implementation climate across different implementation phases. To illustrate the variations in these determinants across phases, we aggregated and scored coded statements using a previously published CFIR scoring system (- 2 to + 2). Key relationships between available resources and implementation climate were identified and summarized using thematic analysis.

RESULTS

The resources necessary to support the successful implementation of an intervention are not static; both the quantity and types of resources shift based on the phases of the intervention. Furthermore, increased resource availability does not guarantee the sustainment of intervention success. Users need different types of support beyond the technical aspects of an intervention, and this support varies over time. Specifically, available resources in the form of technological support and social/emotional support help users establish trust in a new technological-based intervention during the implementation phase. Resources that foster and maintain collaboration between users and other stakeholders help them stay motivated during sustainment.

CONCLUSIONS

Our findings highlight the dynamic nature of available resources and their impacts on the implementation climate across different phases of implementation. A better understanding of the dynamics of available resources over time from the users' perspectives will allow the adaptation of resources to better meet the needs of the intervention stakeholders.

Climate, Health, and Nursing Tool (CHANT): A confirmatory factor analysis

OBJECTIVE

To confirm the factor structure of the Climate, Health, and Nursing Tool (CHANT) tool via confirmatory factor analysis.

DESIGN AND SAMPLE

This is a cross-sectional analysis of voluntary, anonymous responses collected online in 2019, from a non-representative sample of 489 nurses from 12 nations with 95% of the respondents from the United States.

MEASUREMENTS

A confirmatory factor analysis (CFA) was conducted to test a five-factor measurement model of the 22-item CHANT. Reliability was examined via Cronbach's α coefficient.

RESULTS

The five CHANT subscales demonstrated acceptable reliability with Cronbach's α ranging from 0.67 to 0.91. The five-factor model of CHANT demonstrated good fit, x2 (199) = 582.747, p < .001, CFI = 0.94, RMSEA = 0.06, and SRMR = 0.04 with statistically significant item-factor loadings.

CONCLUSION

CHANT is a reliable and robust instrument to measure nurses' awareness, concern, motivation, and home and work behaviors regarding climate change and health, and is ready to be utilized in research, policy, professional settings, and among educators.

A stepped-wedge randomized trial investigating the effect of the Leadership and Organizational Change for Implementation (LOCI) intervention on implementation and transformational leadership, and implementation climate

BACKGROUND

This study evaluates the Leadership and Organizational Change for Implementation (LOCI) strategy and its effect on implementation leadership, transformational leadership, and implementation climate.

METHODS

A stepped wedge cluster randomized study design enrolling 47 first-level leaders from child- and adult-specialized mental health clinics within Norwegian health trusts across three cohorts. All therapists (n = 790) received training in screening of trauma exposure and posttraumatic stress, and a subgroup of therapists (n = 248) received training in evidence-based treatment methods for posttraumatic stress disorder (PTSD). First-level leaders and therapists completed surveys at baseline, 4, 8-, 12-, 16-, and 20-months assessing leadership and implementation climate. General linear mixed-effects models were used to investigate whether the LOCI strategy would lead to greater therapist-rated scores on implementation leadership, transformational leadership, and implementation climate.

RESULTS

After introducing the LOCI strategy, there was a significant increase in therapist-rated implementation and transformational leadership and implementation climate. The increase was sustained at all measurement time points compared to non-LOCI conditions, which demonstrated a steady decrease in scores before LOCI.

CONCLUSIONS

The LOCI strategy can develop better transformational and implementation leadership skills and contribute to a more positive implementation climate, which may enhance successful EBP implementation. Thus, LOCI can help leaders create an organizational context conducive for effective EBP implementation.

TRIAL REGISTRATION

Retrospectively registered: ClinicalTrials NCT03719651 , 25th of October 2018. The trial protocol can be accessed from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6417075/ .