Developing an evaluation framework for assessing the impact of recovery colleges: protocol for a participatory stakeholder engagement process and cocreated scoping review

Evaluating the impact of a UK recovery college on mental well-being: pre- and post-intervention study

BACKGROUND

Recovery colleges provide personalised educational mental health support for people who self-refer. The research evidence supporting them is growing, with key components and the positive experiences of attendees reported. However, the quantitative outcome evidence and impact on economic outcomes is limited.

AIMS

To evaluate the impact of attending a UK recovery college for students who receive a full educational intervention.

METHOD

This is a pre- and post-intervention study, with predominantly quantitative methods. Participants recruited over an 18-month period (01.2020-07.2021) completed self-reported well-being (Short Warwick-Edinburgh Mental Wellbeing Scale (SWEMWBS)) and recovery (Process of Recovery (QPR)) surveys, and provided details and evidence of employment and educational status. Descriptive statistics for baseline data and Shapiro-Wilk, Wilcoxon signed-rank and paired t-tests were used to compare pre- and post-intervention scores, with Hedges' g-statistic as a measure of effect size. Medical records were reviewed and a brief qualitative assessment of changes reported by students was conducted.

RESULTS

Of 101 student research participants, 84 completed the intervention. Well-being (mean SWEMWBS scores 17.3 and 21.9; n = 80) and recovery (mean QPR scores 27.2 and 38.8; n = 75) improved significantly (P < 0.001; Hedges' g of 1.08 and 1.03). The number of economically inactive students reduced from 53 (69%) to 19 (24.4%). No research participants were referred for specialist mental health support while students. 'Within-self' and 'practical' changes were described by students following the intervention.

CONCLUSIONS

Findings detail the largest self-reported pre-post data-set for students attending a recovery college, and the first data detailing outcomes of remote delivery of a recovery college.

A balancing act: navigating the nuances of co-production in mental health research

BACKGROUND

In the context of mental health research, co-production involves people with lived expertise, those with professional or academic expertise, and people with both of these perspectives collaborating to design and actualize research initiatives. In the literature, two dominant perspectives on co-production emerge. The first is in support of co-production, pointing to the transformative value of co-production for those involved, the quality of services developed through this process, as well as to broader system-level impacts (e.g. influencing changes in health system decision making, care practices, government policies, etc.). The second stance expresses scepticism about the capacity of co-production to engender genuine collaboration given the deeply ingrained power imbalances in the systems in which we operate. While some scholars have explored the intersections of these two perspectives, this body of literature remains limited.

MAIN TEXT

This paper contributes to the literature base by exploring the nuances of co-production in health research. Using our mental health participatory action research project as a case example, we explore the nuances of co-production through four key values that we embraced: 1. Navigating power relations together 2. Multi-directional learning 3. Slow and steady wins the race 4. Connecting through vulnerability CONCLUSIONS: By sharing these values and associated principles and practices, we invite readers to consider the complexities of co-production and explore how our experiences may inform their practice of co-production. Despite the inherent complexity of co-production, we contend that pursuing authentic and equitable collaborations is integral to shaping a more just and inclusive future in mental health research and the mental health system at large.

Evaluating recovery colleges: a co-created scoping review

BACKGROUND

Recovery Colleges (RCs) are education-based centres providing information, networking, and skills development for managing mental health, well-being, and daily living. A central principle is co-creation involving people with lived experience of mental health/illness and/or addictions (MHA). Identified gaps are RCs evaluations and information about whether such evaluations are co-created.

AIMS

We describe a co-created scoping review of how RCs are evaluated in the published and grey literature. Also assessed were: the frameworks, designs, and analyses used; the themes/outcomes reported; the trustworthiness of the evidence; and whether the evaluations are co-created.

METHODS

We followed Arksey and O'Malley's methodology with one important modification: "Consultation" was re-conceptualised as "co-creator engagement" and was the first, foundational step rather than the last, optional one.

RESULTS

Seventy-nine percent of the 43 included evaluations were peer-reviewed, 21% grey literature. These evaluations represented 33 RCs located in the UK (58%), Australia (15%), Canada (9%), Ireland (9%), the USA (6%), and Italy (3%).

CONCLUSION

Our findings depict a developing field that is exploring a mix of evaluative approaches. However, few evaluations appeared to be co-created. Although most studies referenced co-design/co-production, few described how much or how meaningfully people with lived experience were involved in the evaluation.