A Multidimensional Examination of the Measurement of Treatment Engagement: Implications for Children's Mental Health Services and Research

Barriers and Facilitators to User Engagement and Moderation for Web-Based Peer Support Among Young People: Qualitative Study Using the Behavior Change Wheel Framework

BACKGROUND

Peer support groups or web-based chats for young people offer anonymous peer support in judgment-free spaces, where users may share their thoughts and feelings with others who may have experienced similar situations. User engagement is crucial for effective web-based peer support; however, levels of engagement vary. While moderation of peer support groups can have a positive impact on the engagement of young people, effective moderation can be challenging to implement.

OBJECTIVE

This study aimed to identify barriers and facilitators to user engagement with, and moderation of, web-based peer support groups among young people aged 16 to 25 years and to provide recommendations for enhancing this service.

METHODS

Drawing upon the Theoretical Domains Framework (TDF) and the Behavior Change Wheel (BCW), this study conducted qualitative interviews and gathered open-ended questionnaires from service users and moderators of The Mix, the United Kingdom's leading web-based mental health platform providing peer support groups for young people. Semistructured interviews were conducted with 2 service users and 8 moderators, and open-ended questionnaires were completed by 7 service users. Themes were coded using the Capability, Opportunity, Motivation, and Behavior (COM-B) model and the TDF. The BCW tools were then used to identify relevant behavior change techniques to improve user engagement in, and moderation of, the service.

RESULTS

Thematic analysis revealed a total of 20 inductive themes within 10 TDF domains-9 (45%) for engagement and 11 (55%) for moderation. Of these 20 themes, 3 (15%) were facilitators of engagement, 7 (35%) were facilitators of moderation, 4 (20%) were barriers to moderation, and 6 (30%) barriers to engagement. Results suggest that skills, knowledge, beliefs about consequences, intentions, emotions, and the social and physical environment are important factors influencing service users and moderators of group chats. In particular, supporting the improvement of memory, attention, and decision-making skills of those involved; adapting the physical environment to facilitate effective interactions; and reducing negative emotions are suggested to optimize the value and effectiveness of peer support groups for young people's mental health for both the service users and moderators of these services.

CONCLUSIONS

The study demonstrates the effectiveness of the BCW approach and the use of the TDF and COM-B model to understand the influences on behavior in a systematic manner, especially for mental health and well-being interventions. The findings can be applied to design structured interventions to change behaviors related to the engagement with, and moderation of, web-based peer support groups and, in turn, improve mental health outcomes for young people.

Ethnic Identity and Mechanisms of Mental Health Service Engagement Among Young Adults with Serious Mental Illnesses

PURPOSE

Young adults from minoritized racial and ethnic groups have lower rates of engagement in treatment for serious mental illnesses (SMI). Previous research suggests a relationship between ethnic identity development and engagement in mental health services, but it remains unclear how a sense of belonging and attachment to one's racial and ethnic group influences participation in treatment among young adults with SMI.

METHODS

Bivariate analyses and structural equation modeling (SEM) were used to examine whether ethnic identity was associated with treatment engagement (attendance and investment in treatment) and how ethnic identity might influence engagement through theoretical proximal mediators. Eighty-three young adults with SMI (95% from minoritized racial and ethnic groups) were recruited from four outpatient psychiatric rehabilitation programs and assessed at least 3 months after initiating services.

RESULTS

Stronger ethnic identity was associated with greater investment in treatment but not with treatment attendance. The SEM analysis indicated that stronger ethnic identity may improve investment in treatment by enhancing hope (0.53, p < .05) and beliefs that mental health providers are credible (0.32, p < .05), and by increasing self-efficacy (-0.09, p < .05). Proximal mediators of engagement were associated with investment in treatment (hope and credibility, p < .05, and self-efficacy p = 0.055).

CONCLUSIONS

Findings provide preliminary evidence of an empirical and theoretical relationship between ethnic identity development and engagement in treatment among young adults with SMI. Assessment and strengthening of a young person's ethnic identity may be a promising approach for improving their engagement in services and reducing inequities in their care.

Young People's Barriers and Facilitators of Engagement with Web-Based Mental Health Interventions for Anxiety and Depression: A Qualitative Study

BACKGROUND

The prevalence of anxiety and depressive symptoms in young people have increased in many countries around the world. Web-based mental health interventions (or W-MHIs) have the potential to reduce anxiety and depression symptoms for young people. Although W-MHIs have become more widely used by young people since the coronavirus disease 2019 (COVID-19) pandemic, real-world engagement in these W-MHIs has remained low compared with engagement reported in research studies. Moreover, there are limited studies examining factors influencing engagement with W-MHIs in the post-COVID-19 pandemic years.

OBJECTIVE

This study aims to explore barriers and facilitators of engagement with W-MHIs for anxiety and depression among young people.

METHOD

Seventeen semi-structured interviews and one focus group with three participants were conducted online via Zoom between February and March 2023. Participants were young people aged 18-25 years who had self-reported experience of anxiety and/or depression in the past 6 months, lived in Australia, and considered using W-MHIs to manage their anxiety and/or depression symptoms. Inductive thematic analysis was performed to understand the key barriers and facilitators of young people's engagement with W-MHIs.

RESULTS

Both individual- and intervention-related factors influenced young people's engagement with W-MHIs. Facilitators of engagement included personal trust and beliefs in W-MHIs, ability to contact a health professional, programme suitability (e.g., affordability, content aligning with user needs), programme usability (e.g., user interface), and accessibility of the online platform. Barriers included concerns about online security, lack of human interaction and immediate responses from health professionals (if any), and negative experience with mental health programmes. Participants expressed greater willingness to pay if they could contact health professionals during the programme.

CONCLUSION

Better promotion strategies for mental health and W-MHI awareness are needed to increase the perceived importance and priority of mental health interventions among young people. Young people should be involved in the W-MHI co-design to enhance the programme suitability and usability for young people, fostering their engagement with W-MHIs.

Engaging Youth Voice and Family Partnerships to Improve Children's Mental Health Outcomes

Promoting active participation of families and youth in mental health systems of care is the cornerstone of creating a more inclusive, effective, and responsive care network. This article focuses on the inclusion of parent and youth voice in transforming our mental health care system to promote increased engagement at all levels of service delivery. Youth and parent peer support delivery models, digital innovation, and technology not only empower the individuals involved, but also have the potential to enhance the overall efficacy of the mental health care system.

Caregiver Engagement in Outpatient Treatment for Adolescents in Community Settings: Construct and Predictive Validity of Family Therapy Techniques

Caregiver engagement and participation in community-based outpatient treatment services for adolescents is generally poor, which is problematic given the integral role of caregivers prescribed by evidence-based treatments across orientations. The current study explores the psychometric and predictive properties of a set of caregiver engagement techniques distilled from family therapy, used by community clinicians in routine care. It highlights relational engagement interventions and adds to growing work distilling core elements of family therapy. The study examined caregiver engagement techniques observed in 320 recorded sessions and outcome data from 152 cases treated by 45 therapists participating in one of three randomized trials investigating delivery of family therapy for adolescent behavior problems in community settings. Construct and predictive validity of caregiver engagement coding items were analyzed to understand the degree to which they cohered as a single factor and predicted outcomes in predictable ways. Results demonstrated item reliability and construct validity of a Caregiver Engagement Techniques factor. Greater use of these techniques was associated with decreased adolescent substance use. Unexpected results suggested greater use of techniques was associated with worsening internalizing symptoms and family cohesion per youth-report only. Post-hoc analyses revealed additional complexities in the association between engagement techniques and outcomes. Caregiver engagement practices tested in the current study represent a unified treatment factor that may contribute to positive therapeutic outcomes for adolescents in some clinical domains. Further research is needed to understand predictive effects.

Future Directions in Youth and Family Treatment Engagement: Finishing the Bridge Between Science and Service

The field has spent more than 50 years investing in the quality of youth mental healthcare, with intervention science yielding roughly 1,300 efficacious treatments. In the latter half of this period, concurrent efforts in implementation science have developed effective methods for supporting front-line service organizations and therapists to begin to bridge the science to service gap. However, many youths and families still do not benefit fully from these strategic investments due to low treatment engagement: nearly half of youths in need of services pursue them, and among those who do, roughly another half terminate prematurely. The negative impact of low engagement is substantial, and is disproportionally and inequitably so for many. We contend that to build a robust and "finished" bridge connecting science and service, the field must go beyond its two historical foci of designing interventions and preparing therapists to deliver them, to include an intentional focus on the youths and families who participate in these interventions and who work with those therapists. In this paper, we highlight the significance of treatment engagement in youth mental healthcare and discuss the current state of the literature related to four priorities: conceptualization, theory, measurement, and interventions. Next, we offer an example from our own program of research as one illustration for advancing these priorities. Finally, we propose recommendations to act on these priorities.

The Operations Triad Model and Youth Mental Health Assessments: Catalyzing a Paradigm Shift in Measurement Validation

Researchers strategically assess youth mental health by soliciting reports from multiple informants. Typically, these informants (e.g., parents, teachers, youth themselves) vary in the social contexts where they observe youth. Decades of research reveal that the most common data conditions produced with this approach consist of discrepancies across informants' reports (i.e., informant discrepancies). Researchers should arguably treat these informant discrepancies as domain-relevant information: data relevant to understanding youth mental health domains (e.g., anxiety, depression, aggression). Yet, historically, in youth mental health research as in many other research areas, one set of paradigms has guided interpretations of informant discrepancies: Converging Operations and the Multi-Trait Multi-Method Matrix (MTMM). These paradigms (a) emphasize shared or common variance observed in multivariate data, and (b) inspire research practices that treat unique variance (i.e., informant discrepancies) as measurement confounds, namely random error and/or rater biases. Several yearsw ago, the Operations Triad Model emerged to address a conceptual problem that Converging Operations does not address: Some informant discrepancies might reflect measurement confounds, whereas others reflect domain-relevant information. However, addressing this problem requires more than a conceptual paradigm shift beyond Converging Operations. This problem necessitates a paradigm shift in measurement validation. We advance a paradigm (Classifying Observations Necessitates Theory, Epistemology, and Testing [CONTEXT]) that addresses problems with using the MTMM in youth mental health research. CONTEXT optimizes measurement validity by guiding researchers to leverage (a) informants that produce domain-relevant informant discrepancies, (b) analytic procedures that retain domain-relevant informant discrepancies, and (c) study designs that facilitate detecting domain-relevant informant discrepancies.

Impact of a brief intervention to improve engagement in a recovery program for young adults with serious mental illness

OBJECTIVE

Serious mental illnesses (SMI) commonly emerge during young adulthood. Effective treatments for this population exist; however, engagement in treatment is a persistent challenge. This study examines the impact of Just Do You (JDY), an innovative intake-focused intervention designed to improve engagement in treatment and enhance personal recovery.

METHODS

The study used a parallel group randomized trial to examine if and how JDY improved recovery among 121 young adults with SMI from low-resourced communities referred to personalized recovery-oriented services (PROS). Measures of engagement (buy-in and attendance) and personal recovery in this pilot study were assessed at baseline and 3-month follow-up.

RESULTS

Participants in JDY reported more positive engagement outcomes; that is, relative to the control group they reported higher past two week attendance (b = 0.72, p < 0.05, Cohen's d = 0.56) and higher levels of buy-in to treatment (b = 2.42, p < 0.05, Cohen's d = 0.50). JDY also impacted young adults' personal recovery (b = 0.99, p < 0.05, Cohen's d = 1.15) and did so largely by increasing their level of buy-in to the treatment program.

CONCLUSION

This study suggests that an engagement intervention for young adults that orients, prepares, and empowers them to be active and involved in the larger treatment program makes a difference by improving engagement and enhancing recovery. Data also support conceptualizing and examining engagement beyond treatment attendance; in this study what mattered most for recovery was the level of buy-in to treatment among young adults.

A Systematic Review of Music-Based Interventions to Improve Treatment Engagement and Mental Health Outcomes for Adolescents and Young Adults

Mental health disorders and suicidality are rising among adolescents and young adults (A-YA) while rates of treatment engagement remain notoriously low. Emerging research supports the potential of music-based interventions to improve mental health, but their efficacy remains unclear for A-YA. This systematic review evaluates the evidence on music-based psychosocial interventions to improve engagement in treatment and/or mental health outcomes among A-YA. This review was prospectively registered with PROSPERO and followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Twenty-six studies were extracted. There was heterogeneity of designs, populations, measures, and outcomes. A typology of music-based interventions emerged, which is characterized by combinations of three broad categories: 1) Somatosensory, 2) Social-Emotional, and 3) Cognitive-Reflective. Most interventions are Socio-Cognitive and Holistically Integrated (combines all three) followed by Socio-Somatic. All interventions involved Social-Emotional processes. Results indicate that most studies report significant effects for mental health outcomes related to social and emotional improvements and reductions of internalizing symptoms for adolescents. Few studies targeted young adults and effects on engagement were rarely measured. There is a need for more studies that use rigorous methods. This review illuminated a need for interventions that are developmentally and culturally tailored to subgroups. Finally, the field is ripe from more studies that apply experimental therapeutics to conceptualize, operationalize, and test mechanisms of change to improve the understanding of how and for whom music-based interventions work. Recommendations for embedding these innovative strategies into research and practice for A-YA are discussed.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10560-022-00893-x.

How Do Treatment Protocols Affect the Use of Engagement Practices in Youth Mental Health Services?

PURPOSE

Treatment engagement poses challenges for youth mental health providers. With the expansion of evidence-based treatments (EBTs), providers face complex decisions regarding how to engage youth and families using available information sources. This study investigated how EBT protocols are associated with the selection and delivery of engagement practices.

METHOD

Twenty engagement practices were coded in a sample of digital recordings of early treatment sessions (N = 193) from the Child STEPs in California study, a randomized trial testing modular treatment and community-implemented treatment for youth mental health problems. Data were collected on which protocols mental health providers reportedly used to guide their sessions and the protocols in which they had received training. We examined which information sources (i.e., the guiding protocol, other protocols in training history, unspecified source) were associated with observed engagement practices.

RESULTS

In sessions guided by a protocol, most observed engagement practices were accounted for by the guiding protocol (p < .001), rather than protocols in training history or unspecified sources (p < .001). In sessions not guided by a protocol, most observed practices were accounted for by training history (p < .001). Practice frequency and extensiveness was generally greater when a protocol guided the session.

CONCLUSIONS

Inclusion in protocols is associated with the selection and delivery of engagement practices, but this strategy might be insufficient for supporting the use of the full range of engagement practices supported by evidence. Supports are needed that leverage the engagement evidence base to ensure that selected practices empirically fit the engagement needs of youth and families.