Individual-level associations between implementation leadership, climate, and anticipated outcomes: a time-lagged mediation analysis

BACKGROUND

Leaders can improve implementation outcomes by developing an organizational climate conducive to the implementation of evidence-based practices (EBP). This study tested the lagged associations between individual-level perceptions of implementation leadership, implementation climate, and three anticipated implementation outcomes, that is EBP acceptability, appropriateness, and feasibility.

METHODS

Screening tools and treatment methods for posttraumatic stress disorder were implemented in 43 Norwegian mental health services. A sample of 494 child and adult mental health care professionals (M = 43 years, 78% female) completed surveys addressing perceptions of first-level leaders' (n = 47) implementation leadership and their clinics' implementation climate. Single-level structural equation models estimating both direct, indirect, and total effects were used to investigate whether perceived implementation climate mediated the association between perceived implementation leadership and perceived acceptability, appropriateness, and feasibility of screening tools and treatment methods.

RESULTS

Regarding the treatment methods, implementation leadership was associated with therapists' perceptions of acceptability, appropriateness, and feasibility. Implementation climate also mediated between implementation leadership and the outcomes. Regarding the screening tools, implementation leadership was not associated with the outcomes. However, implementation climate mediated between implementation leadership and therapists' perceptions of acceptability and feasibility, but not appropriateness. Analyses with the implementation climate subscales showed stronger associations for therapists' perceptions of the treatment methods than of screening tools.

CONCLUSIONS

Leaders may promote positive implementation outcomes, both directly and through implementation climate. With regard to the effect sizes and explained variance, results indicated that both implementation leadership and implementation climate were more strongly associated with the therapists' perceptions of the treatment methods, implemented by one group of therapists, than the screening tools, implemented by all therapists. This may imply that implementation leadership and climate may have stronger effects for smaller implementation teams within a larger system than for system-wide implementations or when the clinical interventions being implemented are more complex rather than simple ones.

TRIAL REGISTRATION

ClinicalTrials NCT03719651, 25 October 2018.

Psychometric Properties of the General Organizational Index (GOI): A Measure of Individualization and Quality Improvement to Complement Program Fidelity

To assess the implementation of effective practices, mental health programs need standardized measures. The General Organizational Index (GOI), although widely used for this purpose, has received minimal psychometric research. For this study, we assessed psychometric properties of the GOI scale administered four times over 18 months during the implementation of a new program in 11 sites. The GOI scale demonstrated high levels of interrater reliability (.97), agreement between assessors on item ratings (86% overall), internal consistency (.77–.80 at three time points), sensitivity to change, and feasibility. We conclude that the GOI scale has acceptable psychometric properties, and its use may enhance implementation and research on evidence-based mental health practices.

Trial registration: REK2015/2169. ClinicalTrials.gov Identifier: NCT03271242

Psychometric Properties of a Fidelity Scale for Illness Management and Recovery

This study examined the psychometric properties and feasibility of the Illness Management and Recovery (IMR) Fidelity scale. Despite widespread use of the scale, the psychometric properties have received limited attention. Trained fidelity assessors conducted assessments four times over 18 months at 11 sites implementing IMR. The IMR Fidelity scale showed excellent interrater reliability (.99), interrater item agreement (94%), internal consistency (.91–.95 at three time points), and sensitivity to change. Frequency distributions generally showed that item ratings included the entire range. The IMR Fidelity scale has excellent psychometric properties and should be used to evaluate and guide the implementation of IMR.

Trial registration: ClinicalTrials.gov Identifier: NCT03271242.

How to implement Illness Management and Recovery (IMR) in mental health service settings: evaluation of the implementation strategy

Background

The purpose of this study was to evaluate the implementation strategy used in the first-phase of implementation of the Illness Management and Recovery (IMR) programme, an intervention for adults with severe mental illnesses, in nine mental health service settings in Norway.

Methods

A total of 9 clinical leaders, 31 clinicians, and 44 consumers at 9 service settings participated in the implementation of IMR. Implementation was conducted by an external team of researchers and an experienced trainer. Data were gathered on fidelity to the intervention and implementation strategy, feasibility, and consumer outcomes.

Results

Although the majority of clinicians scored within the acceptable range of high intervention fidelity, their participation in the implementation strategy appeared to moderate anticipated future use of IMR. No service settings reached high intervention fidelity scores for organizational quality improvement after 12 months of implementation. IMR implementation seemed feasible, albeit with some challenges. Consumer outcomes indicated significant improvements in illness self-management, severity of problems, functioning, and hope. There were nonsignificant positive changes in symptoms and quality of life.

Conclusions

The implementation strategy appeared adequate to build clinician competence over time, enabling clinicians to provide treatment that increased functioning and hope for consumers. Additional efficient strategies should be incorporated to facilitate organizational change and thus secure the sustainability of the implemented practice.

Trial registration ClinicalTrials.gov NCT02077829. Registered 25 February 2014