Impact of receiving recorded mental health recovery narratives on quality of life in people experiencing non-psychosis mental health problems (NEON-O Trial): updated randomised controlled trial protocol

The Implementation of Recommender Systems for Mental Health Recovery Narratives: Evaluation of Use and Performance

BACKGROUND

Recommender systems help narrow down a large range of items to a smaller, personalized set. NarraGive is a first-in-field hybrid recommender system for mental health recovery narratives, recommending narratives based on their content and narrator characteristics (using content-based filtering) and on narratives beneficially impacting other similar users (using collaborative filtering). NarraGive is integrated into the Narrative Experiences Online (NEON) intervention, a web application providing access to the NEON Collection of recovery narratives.

OBJECTIVE

This study aims to analyze the 3 recommender system algorithms used in NarraGive to inform future interventions using recommender systems for lived experience narratives.

METHODS

Using a recently published framework for evaluating recommender systems to structure the analysis, we compared the content-based filtering algorithm and collaborative filtering algorithms by evaluating the accuracy (how close the predicted ratings are to the true ratings), precision (the proportion of the recommended narratives that are relevant), diversity (how diverse the recommended narratives are), coverage (the proportion of all available narratives that can be recommended), and unfairness (whether the algorithms produce less accurate predictions for disadvantaged participants) across gender and ethnicity. We used data from all participants in 2 parallel-group, waitlist control clinical trials of the NEON intervention (NEON trial: N=739; NEON for other [eg, nonpsychosis] mental health problems [NEON-O] trial: N=1023). Both trials included people with self-reported mental health problems who had and had not used statutory mental health services. In addition, NEON trial participants had experienced self-reported psychosis in the previous 5 years. Our evaluation used a database of Likert-scale narrative ratings provided by trial participants in response to validated narrative feedback questions.

RESULTS

Participants from the NEON and NEON-O trials provided 2288 and 1896 narrative ratings, respectively. Each rated narrative had a median of 3 ratings and 2 ratings, respectively. For the NEON trial, the content-based filtering algorithm performed better for coverage; the collaborative filtering algorithms performed better for accuracy, diversity, and unfairness across both gender and ethnicity; and neither algorithm performed better for precision. For the NEON-O trial, the content-based filtering algorithm did not perform better on any metric; the collaborative filtering algorithms performed better on accuracy and unfairness across both gender and ethnicity; and neither algorithm performed better for precision, diversity, or coverage.

CONCLUSIONS

Clinical population may be associated with recommender system performance. Recommender systems are susceptible to a wide range of undesirable biases. Approaches to mitigating these include providing enough initial data for the recommender system (to prevent overfitting), ensuring that items can be accessed outside the recommender system (to prevent a feedback loop between accessed items and recommended items), and encouraging participants to provide feedback on every narrative they interact with (to prevent participants from only providing feedback when they have strong opinions).

Perception and appropriation of a web-based recovery narratives intervention: qualitative interview study

Introduction

Mental health recovery narratives are widely available to the public, and can benefit people affected by mental health problems. The NEON Intervention is a novel web-based digital health intervention providing access to the NEON Collection of recovery narratives. The NEON Intervention was found to be effective and cost-effective in the NEON-O Trial for people with nonpsychosis mental health problems (ISRCTN63197153), and has also been evaluated in the NEON Trial for people with psychosis experience (ISRCTN11152837). We aimed to document NEON Intervention experiences, through an integrated process evaluation.

Methods

Analysis of interviews with a purposive sample of intervention arm participants who had completed trial participation.

Results

We interviewed 34 NEON Trial and 20 NEON-O Trial participants (mean age 40.4 years). Some users accessed narratives through the NEON Intervention almost daily, whilst others used it infrequently or not at all. Motivations for trial participation included: exploring the NEON Intervention as an alternative or addition to existing mental health provision; searching for answers about mental health experiences; developing their practice as a mental health professional (for a subset who were mental health professionals); claiming payment vouchers. High users (10 + narrative accesses) described three forms of appropriation: distracting from difficult mental health experiences; providing an emotional boost; sustaining a sense of having a social support network. Most participants valued the scale of the NEON Collection (n = 659 narratives), but some found it overwhelming. Many felt they could describe the characteristics of a desired narrative that would benefit their mental health. Finding a narrative meeting their desires enhanced engagement, but not finding one reduced engagement. Narratives in the NEON Collection were perceived as authentic if they acknowledged the difficult reality of mental health experiences, appeared to describe real world experiences, and described mental health experiences similar to those of the participant.

Discussion

We present recommendations for digital health interventions incorporating collections of digital narratives: (1) make the scale and diversity of the collection visible; (2) provide delivery mechanisms that afford appropriation; (3) enable contributors to produce authentic narratives; (4) enable learning by healthcare professionals; (5) consider use to address loneliness.

Effectiveness and cost-effectiveness of online recorded recovery narratives in improving quality of life for people with non-psychotic mental health problems: a pragmatic randomized controlled trial

Narratives describing first-hand experiences of recovery from mental health problems are widely available. Emerging evidence suggests that engaging with mental health recovery narratives can benefit people experiencing mental health problems, but no randomized controlled trial has been conducted as yet. We developed the Narrative Experiences Online (NEON) Intervention, a web application providing self-guided and recommender systems access to a collection of recorded mental health recovery narratives (n=659). We investigated whether NEON Intervention access benefited adults experiencing non-psychotic mental health problems by conducting a pragmatic parallel-group randomized trial, with usual care as control condition. The primary endpoint was quality of life at week 52 assessed by the Manchester Short Assessment (MANSA). Secondary outcomes were psychological distress, hope, self-efficacy, and meaning in life at week 52. Between March 9, 2020 and March 26, 2021, we recruited 1,023 participants from across England (the target based on power analysis was 994), of whom 827 (80.8%) identified as White British, 811 (79.3%) were female, 586 (57.3%) were employed, and 272 (26.6%) were unemployed. Their mean age was 38.4±13.6 years. Mood and/or anxiety disorders (N=626, 61.2%) and stress-related disorders (N=152, 14.9%) were the most common mental health problems. At week 52, our intention-to-treat analysis found a significant baseline-adjusted difference of 0.13 (95% CI: 0.01-0.26, p=0.041) in the MANSA score between the intervention and control groups, corresponding to a mean change of 1.56 scale points per participant, which indicates that the intervention increased quality of life. We also detected a significant baseline-adjusted difference of 0.22 (95% CI: 0.05-0.40, p=0.014) between the groups in the score on the "presence of meaning" subscale of the Meaning in Life Questionnaire, corresponding to a mean change of 1.1 scale points per participant. We found an incremental gain of 0.0142 quality-adjusted life years (QALYs) (95% credible interval: 0.0059 to 0.0226) and a £178 incremental increase in cost (95% credible interval: -£154 to £455) per participant, generating an incremental cost-effectiveness ratio of £12,526 per QALY compared with usual care. This was lower than the £20,000 per QALY threshold used by the National Health Service in England, indicating that the intervention would be a cost-effective use of health service resources. In the subgroup analysis including participants who had used specialist mental health services at baseline, the intervention both reduced cost (-£98, 95% credible interval: -£606 to £309) and improved QALYs (0.0165, 95% credible interval: 0.0057 to 0.0273) per participant as compared to usual care. We conclude that the NEON Intervention is an effective and cost-effective new intervention for people experiencing non-psychotic mental health problems.

Pragmatic, feasibility randomized controlled trial of a recorded mental health recovery narrative intervention: narrative experiences online intervention for informal carers (NEON-C)

Introduction

Informal carers of people with mental health problems often have unmet support needs. Mental health recovery narratives are increasingly accessible, but their relevance to and effect on informal carers have been minimally investigated. The Narrative Experiences Online (NEON) Intervention is a first-in-field intervention that provides informal carers with access to a diverse collection of recorded mental health recovery narratives. This trial aimed to examine the feasibility and acceptability of the NEON Intervention for informal carers.

Methods

This study involved a two-arm feasibility randomized controlled trial. Carers were randomly assigned to receiving versus not receiving the NEON Intervention. The feasibility aspects investigated included the acceptability of the intervention and of randomization, trial processes, engagement rates, recruitment procedures, attrition, sample size estimation, identification of candidate primary and secondary outcomes, and the feasibility of conducting a definitive trial. A qualitative process evaluation was conducted.

Findings

A total of 121 carers were eligible, of whom 54 were randomized (intervention: 27, control: 27). Twelve-month follow-up data were available for 36 carers. Carers accessed a mean of 25 narratives over a 12-month period, and the intervention group, compared with the control group, reported a small effect on hope and a moderate effect on the presence of meaning in life. Five modifications were recommended to improve the user experience, applicability, and trial processes.

Discussion

The NEON Intervention is feasible and acceptable. Significant refinement of the NEON Intervention and trial processes is required to personalize and ensure applicability to carers. Further feasibility testing is recommended prior to a definitive trial.

Self-Compassion during COVID-19 in Non-WEIRD Countries: A Narrative Review

The coronavirus disease 2019 (COVID-19) pandemic impacted people's mental health negatively worldwide, including in non-WEIRD (Western, Educated, Industrialised, Rich and Democratic) countries. Self-compassion, kindness and understanding towards oneself in difficult times have received increasing attention in the field of mental health. Self-compassion is strongly associated with good mental health in various populations. This narrative review aimed to synthesise the evidence on self-compassion and mental health in non-WEIRD countries during the COVID-19 pandemic. MEDLINE and PsycINFO were searched for empirical studies. Self-compassion was consistently associated with positive mental health in non-WEIRD countries too. However, how, and to what degree, each component of self-compassion impacts mental health remains to be evaluated across different cultures. Future research such as multi-national intervention studies, or component network meta-analysis, is needed to advance our understanding of how self-compassion improves mental health in different populations.

Statistical analysis plans for two randomised controlled trials of the Narrative Experiences Online (NEON) Intervention: impact of receiving recorded mental health recovery narratives on quality of life in people experiencing psychosis (NEON) and people experiencing non-psychosis mental health problems (NEON-O)

BACKGROUND

Mental health recovery narratives are a first-hand account of an individual's recovery from mental health distress, access to narratives can aid recovery. The NEON Intervention is a web-application providing access to a managed collection of narratives. We present the statistical analysis plan for assessing the effectiveness of the NEON Intervention in improving quality of life at 1-year post-randomisation. We pay particular focus on the statistical challenges encountered due to the online nature of this trial.

METHODS AND DESIGN

The NEON Intervention is assessed in two trial populations, one for people with experience of psychosis in the last 5 years, and mental health distress in the last six months (NEON Trial) and one for people with experience of non-psychosis mental health problems (NEON-O Trial). Both NEON trials are two-arm randomised controlled superiority trials comparing the effectiveness of the NEON Intervention with usual care. The target sample size is 684 randomised participants for NEON and 994 for NEON-O. Participants were randomised centrally in a 1:1 ratio.

RESULTS

The primary outcome is the mean score of subjective items on the Manchester Short Assessment of Quality-of-Life questionnaire (MANSA) at 52 weeks. Secondary outcomes are scores from the Herth Hope Index, Mental Health Confidence Scale, Meaning of Life questionnaire, CORE-10 questionnaire and Euroqol 5-Dimension 5-Level (EQ-5D-5L).

CONCLUSION

This manuscript is the statistical analysis plan (SAP) for the NEON trials. Any post hoc analysis, such as those requested by journal reviewers will be clearly labelled as such in the final trial reporting. Trial registration Both trials were prospectively registered. NEON Trial: ISRCTN11152837, registered on 13 August 2018. NEON-O Trial: ISRCTN63197153, registered on 9 January 2020.

Development and delivery cost of digital health technologies for mental health: Application to the Narrative Experiences Online Intervention

BACKGROUND

The increasing development and use of digital health interventions requires good quality costing information to inform development and commissioning choices about resource allocation decisions. The Narrative Experiences Online (NEON) Intervention is a web-application that delivers recorded mental health recovery narratives to its users. Two randomized controlled trials are testing the NEON Intervention in people with experience of psychosis (NEON) and people experiencing non-psychosis mental health problems (NEON-O).

AIM

This study describes and estimates the cost components and total cost of developing and delivering the NEON Intervention.

MATERIALS AND METHODS

Total costs for the NEON Trial (739 participants) and NEON-O Trial (1,024 participants) were estimated by: identifying resource use categories involved in intervention development and delivery; accurate measurement or estimation of resource use; and a valuation of resource use to generate overall costs, using relevant unit costs. Resource use categories were identified through consultation with literature, costing reporting standards and iterative consultation with health researchers involved in NEON Intervention development and delivery. Sensitivity analysis was used to test assumptions made.

RESULTS

The total cost of developing the NEON Intervention was £182,851. The largest cost components were software development (27%); Lived Experience Advisory Panel workshops (23%); coding the narratives (9%); and researchers' time to source narratives (9%). The total cost of NEON Intervention delivery during the NEON Trial was £118,663 (£349 per NEON Intervention user). In the NEON-O Trial, the total delivery cost of the NEON Intervention was £123,444 (£241 per NEON Intervention user). The largest cost components include updating the narrative collection (50%); advertising (19%); administration (14%); and software maintenance (11%). Uncertainty in the cost of administration had the largest effect on delivery cost estimates.

CONCLUSION

Our work shows that developing and delivering a digital health intervention requires expertise and time commitment from a range of personnel. Teams developing digital narrative interventions need to allocate substantial resources to curating narrative collections.

IMPLICATIONS FOR PRACTICE

This study identifies the development and delivery resource use categories of a digital health intervention to promote the consistent reporting of costs and informs future decision-making about the costs of delivering the NEON Intervention at scale.

TRIAL REGISTRATION

NEON Trial: ISRCTN11152837, registered 13 August 2018, http://www.isrctn.com/ISRCTN11152837. NEON-O Trial: ISRCTN63197153, registered 9 January 2020, http://www.isrctn.com/ISRCTN63197153.