Building a Program Theory of Implementation Using Process Evaluation of a Complex Quality Improvement Trial in Nursing Homes

Implementation of the Cognitive Daisy (COG-D) for improving care planning and delivery for residents with dementia in care homes: results of a feasibility randomised controlled trial

BACKGROUND

Many residents in care homes for older adults live with dementia. Understanding the unique profiles of cognitive impairments for each resident is important for person-centred care, yet information about specific cognitive problems is limited, and knowledge varies. This study explored the feasibility of implementing the Cognitive Daisy (COG-D) intervention, which provide a visual summary in the shape of a 15-petal flower derived from the scores on a neuropsychological assessment battery, in care homes for older adults.

METHODS

A parallel-group feasibility cluster randomised controlled trial (cRCT) was conducted over 24 months. Eight care homes were randomised in a 1:1 ratio to either usual care plus the Cognitive Daisy intervention (COG-D) or usual care (control). Care staff were trained on how to use Cognitive Daisies and/or on how to conduct the COG-D assessments with residents. Cognitive Daisies were displayed in residents' rooms and included in care plans. COG-D assessments were repeated after 6 months. The primary objective was to explore areas of uncertainty for a future large-scale trial including recruitment rates and intervention implementation and adherence. Secondary objectives were to explore signals of effects in candidate outcome measures for residents and staff, obtained at baseline and 6- and 9-month post-randomisation. A process evaluation explored barriers and facilitators to intervention implementation through care-plan audits (to explore recommendations in response to COG-D assessments), interviews and focus groups with staff, residents and relatives.

RESULTS

Resident recruitment (n = 115) and staff recruitment (n = 99) in 8 care homes exceeded targets (100 and 50, respectively, in 8-10 care homes). Staff training was perceived positively with high completion rates (77.1% and 83.3% for basic and advanced training, respectively). Completion rates were also high for COG-D assessments (75.5% for assessment 1, and 72.5% of these residents completed assessment 2), and COG-D scores remained stable across the two assessment points. No clear signals of effects were found for candidate outcome measures. Number of recommendations in care plans varied across care homes, and interviews/focus groups highlighted several barriers to staff's use of the Cognitive Daisies in daily practice.

CONCLUSION

Findings indicate trial delivery was feasible. However, the COG-D requires modification if it is to be feasibly implemented in care home settings.

TRIAL REGISTRATION

This trial was registered on (date) (ISRCTN15208844).

'We listened and supported and depended on each other': a qualitative study of how leadership influences implementation of QI interventions

BACKGROUND

There is growing recognition in the literature of the 'Herculean' efforts required to bring about change in healthcare processes and systems. Leadership is recognised as a critical lever for implementation of quality improvement (QI) and other complex team-level interventions; however, the processes by which leaders facilitate change are not well understood. The aim of this study is to examine 'how' leadership influences implementation of QI interventions.

METHODS

We drew on the leadership literature and used secondary data collected as part of a process evaluation of the Safer Care for Older Persons in residential Environments (SCOPE) QI intervention to gain insights regarding the processes by which leadership influences QI implementation. Specifically, using detailed process evaluation data from 31 unit-based nursing home teams we conducted a thematic analysis with a codebook developed a priori based on the existing literature to identify leadership processes.

RESULTS

Effective leaders (ie, those who care teams felt supported by and who facilitated SCOPE implementation) successfully developed and reaffirmed teams' commitment to the SCOPE QI intervention (theme 1), facilitated learning capacity by fostering follower participation in SCOPE and empowering care aides to step into team leadership roles (theme 2) and actively supported team-oriented processes where they developed and nurtured relationships with their followers and supported them as they navigated relationships with other staff (theme 3). Together, these were the mechanisms by which care aides were brought on board with the intervention, stayed on board and, ultimately, transplanted the intervention into the facility. Building learning capacity and creating a culture of improvement are thought to be the overarching processes by which leadership facilitates implementation of complex interventions like SCOPE.

CONCLUSIONS

Results highlight important, often overlooked, relational and sociocultural aspects of successful QI leadership in nursing homes that can guide the design, implementation and scaling of complex interventions and can guide future research.

Building a communication and support network among quality improvement teams in nursing homes: a longitudinal study of the SCOPE trial

BACKGROUND

We applied a longitudinal network analysis approach to assess the formation of knowledge sharing and collaboration networks among care aide-led quality improvement (QI) teams in Canadian nursing homes participating in the Safer Care for Older Persons (in residential) Environments (SCOPE) trial which aimed to support unregulated front-line staff to lead unit-based quality improvement (QI) teams in nursing homes. We hypothesized that SCOPE's communicative and participatory nature would provide opportunities for peer support, knowledge sharing, and collaboration building among teams.

METHODS

Fourteen QI teams in Alberta (AB) and seventeen QI teams in British Columbia (BC) participated in the study. Communications across nursing homes occurred through a series of 4 collaborative Learning Congresses (training sessions) over a 1-year period. The senior leaders of QI teams participated in two online network surveys about the communication/collaboration between teams in their province, 1 month after the first, and 6 months later, after the fourth Learning Congress. We developed communication and collaboration network maps pertaining to three time points: before SCOPE, at 2 months, and at 9 months.

RESULTS

Over time, teams made significantly more new connections and strengthened existing ones, within and across regions. Geographic proximity and co-membership in organizational chains were important predictors of connectivity before and during SCOPE. Teams whose members were well connected at baseline disproportionately improved connectivity over time. On the other hand, teams that did not have prior opportunities to connect appeared to use SCOPE to build new ties.

CONCLUSIONS

Our findings suggest the importance of network-altering activities to the formation of collaboration networks among QI teams across nursing homes. Active strategies could be used to better connect less connected teams and facilitate collaboration among geographically proximate teams. These findings may inform the development of interventions to leverage existing networks and provide new networking opportunities to develop and sustain organizational improvements.

Development and validation of the Overall Fidelity Enactment Scale for Complex Interventions (OFES-CI)

BACKGROUND

In many quality improvement (QI) and other complex interventions, assessing the fidelity with which participants 'enact' intervention activities (ie, implement them as intended) is underexplored. Adapting the evaluative approach used in objective structured clinical examinations, we aimed to develop and validate a practical approach to assessing fidelity enactment-the Overall Fidelity Enactment Scale for Complex Interventions (OFES-CI).

METHODS

We developed the OFES-CI to evaluate enactment of the SCOPE QI intervention, which teaches nursing home teams to use plan-do-study-act (PDSA) cycles. The OFES-CI was piloted and revised early in SCOPE with good inter-rater reliability, so we proceeded with a single rater. An intraclass correlation coefficient (ICC) was used to assess inter-rater reliability. For 27 SCOPE teams, we used ICC to compare two methods for assessing fidelity enactment: (1) OFES-CI ratings provided by one of five trained experts who observed structured 6 min PDSA progress presentations made at the end of SCOPE, (2) average rating of two coders' deductive content analysis of qualitative process evaluation data collected during the final 3 months of SCOPE (our gold standard).

RESULTS

Using Cicchetti's classification, inter-rater reliability between two coders who derived the gold standard enactment score was 'excellent' (ICC=0.93, 95% CI=0.85 to 0.97). Inter-rater reliability between the OFES-CI and the gold standard was good (ICC=0.71, 95% CI=0.46 to 0.86), after removing one team where open-text comments were discrepant with the rating. Rater feedback suggests the OFES-CI has strong face validity and positive implementation qualities (acceptability, easy to use, low training requirements).

CONCLUSIONS

The OFES-CI provides a promising novel approach for assessing fidelity enactment in QI and other complex interventions. It demonstrates good reliability against our gold standard assessment approach and addresses the practicality problem in fidelity assessment by virtue of its suitable implementation qualities. Steps for adapting the OFES-CI to other complex interventions are offered.