Implementing a digital health model of care in Australian youth mental health services: protocol for impact evaluation

Experiences from youth advisors in chronic disease prevention research

Engaging young people in research is a promising approach to tackling issues like chronic disease prevention. Our involvement as youth advisors provided valuable experiences, including being at the forefront of change and learning to work within a research team. Furthermore, our experience provides greater insight and learnings for future youth engagement in research.

A Multilingual Digital Mental Health and Well-Being Chatbot (ChatPal): Pre-Post Multicenter Intervention Study

BACKGROUND

In recent years, advances in technology have led to an influx of mental health apps, in particular the development of mental health and well-being chatbots, which have already shown promise in terms of their efficacy, availability, and accessibility. The ChatPal chatbot was developed to promote positive mental well-being among citizens living in rural areas. ChatPal is a multilingual chatbot, available in English, Scottish Gaelic, Swedish, and Finnish, containing psychoeducational content and exercises such as mindfulness and breathing, mood logging, gratitude, and thought diaries.

OBJECTIVE

The primary objective of this study is to evaluate a multilingual mental health and well-being chatbot (ChatPal) to establish if it has an effect on mental well-being. Secondary objectives include investigating the characteristics of individuals that showed improvements in well-being along with those with worsening well-being and applying thematic analysis to user feedback.

METHODS

A pre-post intervention study was conducted where participants were recruited to use the intervention (ChatPal) for a 12-week period. Recruitment took place across 5 regions: Northern Ireland, Scotland, the Republic of Ireland, Sweden, and Finland. Outcome measures included the Short Warwick-Edinburgh Mental Well-Being Scale, the World Health Organization-Five Well-Being Index, and the Satisfaction with Life Scale, which were evaluated at baseline, midpoint, and end point. Written feedback was collected from participants and subjected to qualitative analysis to identify themes.

RESULTS

A total of 348 people were recruited to the study (n=254, 73% female; n=94, 27% male) aged between 18 and 73 (mean 30) years. The well-being scores of participants improved from baseline to midpoint and from baseline to end point; however, improvement in scores was not statistically significant on the Short Warwick-Edinburgh Mental Well-Being Scale (P=.42), the World Health Organization-Five Well-Being Index (P=.52), or the Satisfaction With Life Scale (P=.81). Individuals that had improved well-being scores (n=16) interacted more with the chatbot and were significantly younger compared to those whose well-being declined over the study (P=.03). Three themes were identified from user feedback, including "positive experiences," "mixed or neutral experiences," and "negative experiences." Positive experiences included enjoying exercises provided by the chatbot, while most of the mixed, neutral, or negative experiences mentioned liking the chatbot overall, but there were some barriers, such as technical or performance errors, that needed to be overcome.

CONCLUSIONS

Marginal improvements in mental well-being were seen in those who used ChatPal, albeit nonsignificant. We propose that the chatbot could be used along with other service offerings to complement different digital or face-to-face services, although further research should be carried out to confirm the effectiveness of this approach. Nonetheless, this paper highlights the need for blended service offerings in mental health care.

Engagement, Use, and Impact of Digital Mental Health Resources for Diverse Populations in COVID-19: Community-Partnered Evaluation

BACKGROUND

The COVID-19 pandemic increased disparities for communities burdened by structural barriers such as reduced affordable housing, with mental health consequences. Limited data are available on digital resources for public mental health prevention during the COVID-19 pandemic.

OBJECTIVE

The study aim was to evaluate engagement in and impact of free digital resources on the Together for Wellness/Juntos por Nuestro Bienestar (T4W/Juntos) website during COVID-19 in California.

METHODS

A pilot evaluation of T4W/Juntos was performed, with partner agencies inviting providers, clients, and partners to visit the website and complete surveys at baseline (September 20, 2021, to April 4, 2022) and at 4-6-week follow-up (October 22, 2021, to May 17, 2022). Website use was assessed by three engagement items (ease of use, satisfaction, relevance), comfort in use, and use of six resource categories. Primary outcomes at follow-up were depression and anxiety (scores≥3 on Patient Health Questionnaire-2 item [PHQ2] and Generalized Anxiety Disorder-2 item [GAD2] scales). Secondary outcomes were post-pre differences in PHQ2 and GAD2 scores, and use of behavioral health hotlines and services the month before follow-up.

RESULTS

Of 366 eligible participants, 315 (86.1%) completed baseline and 193 (61.3%) completed follow-up surveys. Of baseline participants, 72.6% identified as female, and 21.3% identified as lesbian, gay, bisexual, transgender, queer/questioning, and others (LGBTQ+). In terms of ethnicity, 44.0% identified as Hispanic, 17.8% as African American, 26.9% as non-Hispanic white, and 11.4% as other ethnicity. Overall, 32.7% had moderate anxiety or depression (GAD2/PHQ2≥3) at baseline. Predictors of baseline website engagement included being Hispanic versus other race/ethnicity (β=.27, 95% CI .10-.44; P=.002) and number of COVID-19-related behavior changes (β=.09, 95% CI .05-.13; P<.001). Predictors of comfort using the website were preferring English for website use (odds ratio [OR] 5.57, 95% CI 2.22-13.96; P<.001) and COVID-19-related behavior changes (OR 1.37, 95% CI 1.12-1.66; P=.002); receiving overnight behavioral health treatment in the prior 6 months (OR 0.15, 95% CI 0.03-0.69, P=.015) was associated with less comfort in website use. The main predictor of depression at follow-up (PHQ2≥3) was baseline depression (OR 6.24, 95% CI 2.77-14.09; P<.001). Engagement in T4W/Juntos was associated with lower likelihood of depression (OR 0.54, 95% CI 0.34-0.86; P=.01). Website use the month before follow-up was associated with a post-pre reduction in PHQ2 score (β=-.62, 95% CI -1.04 to -0.20; P=.004). The main predictor of GAD2≥3 at follow-up was baseline GAD2≥3 (OR 13.65, 95% CI 6.06-30.72; P<.001). Greater baseline website engagement predicted reduced hotline use (OR 0.36, 95% CI 0.18-0.71; P=.004).

CONCLUSIONS

Ethnicity/language and COVID-19-related behavior changes were associated with website engagement; engagement and use predicted reduced follow-up depression and behavioral hotline use. Findings are based on participants recommended by community agencies with moderate follow-up rates; however, significance was similar when weighting for nonresponse. This study may inform research and policy on digital mental health prevention resources.

Linking music streaming platform advertisements with a digital mental health assessment and interventions

Accessibility issues and low rates of help-seeking hinder engagement with mental health resources and treatment. Pragmatic, (cost-)effective solutions are required to increase engagement with efficacious digital mental health interventions (DMHIs) including for hard-to-reach individuals. As an example, music-based interventions have been positively used in health care to reduce stress, anxiety and depression through music medicine, music therapy and recreational use. Although, enhanced mental health awareness from music listening has yet to be converted into engagement with a DMH assessment (DMHA) and DMHIs. Therefore, a new study is proposed to place linked advertisements on Spotify, the most used music streaming platform. MindSpot's vetted DMHA is suitable to use as an example for linking unto because it measures depression, anxiety, general mental well-being problems and psychological distress in Australian adults and provides access to DMHIs. The primary aim is to provide a convenient, robust and scalable consumer pathway to reduce engagement barriers and maximize facilitation to a vetted DMHA and DMHIs. The proposed study is important because it addresses notorious help-seeking difficulties in the adult population (e.g., young people and men). It also expands outreach to the underserved and the unserved and streamlines the integration of digital solutions with mental health services.

Ownership, Use of, and Interest in Digital Mental Health Technologies Among Clinicians and Young People Across a Spectrum of Clinical Care Needs: Cross-sectional Survey

BACKGROUND

There is currently an increased interest in and acceptance of technology-enabled mental health care. To adequately harness this opportunity, it is critical that the design and development of digital mental health technologies be informed by the needs and preferences of end users. Despite young people and clinicians being the predominant users of such technologies, few studies have examined their perspectives on different digital mental health technologies.

OBJECTIVE

This study aims to understand the technologies that young people have access to and use in their everyday lives and what applications of these technologies they are interested in to support their mental health. The study also explores the technologies that youth mental health clinicians currently use within their practice and what applications of these technologies they are interested in to support their clients' mental health.

METHODS

Youth mental health service users (aged 12-25 years) from both primary and specialist services, young people from the general population (aged 16-25 years), and youth mental health clinicians completed a web-based survey exploring technology ownership, use of, and interest levels in using different digital interventions to support their mental health or that of their clients.

RESULTS

A total of 588 young people and 73 youth mental health clinicians completed the survey. Smartphone ownership or private access among young people within mental health services and the general population was universal (611/617, 99%), with high levels of access to computers and social media. Youth technology use was frequent, with 63.3% (387/611) using smartphones several times an hour. Clinicians reported using smartphones (61/76, 80%) and video chat (69/76, 91%) commonly in clinical practice and found them to be helpful. Approximately 50% (296/609) of the young people used mental health apps, which was significantly less than the clinicians (χ²₃=28.8, n=670; P<.001). Similarly, clinicians were significantly more interested in using technology for mental health support than young people (H₃=55.90; P<.001), with 100% (73/73) of clinicians being at least slightly interested in technology to support mental health compared with 88% (520/591) of young people. Follow-up tests revealed no difference in interest between young people from the general population, primary mental health services, and specialist mental health services (all P>.23). Young people were most interested in web-based self-help, mobile self-help, and blended therapy.

CONCLUSIONS

Technology access is pervasive among young people within and outside of youth mental health services; clinicians are already using technology to support clinical care, and there is widespread interest in digital mental health technologies among these groups of end users. These findings provide important insights into the perspectives of young people and clinicians regarding the value of digital mental health interventions in supporting youth mental health.

Supporting Clinicians to Use Technology to Deliver Highly Personalized and Measurement-Based Mental Health Care to Young People: Protocol for an Evaluation Study

BACKGROUND

Australia's mental health care system has long been fragmented and under-resourced, with services falling well short of demand. In response, the World Economic Forum has recently called for the rapid deployment of smarter, digitally enhanced health services to facilitate effective care coordination and address issues of demand. The University of Sydney's Brain and Mind Centre (BMC) has developed an innovative digital health solution that incorporates 2 components: a highly personalized and measurement-based (data-driven) model of youth mental health care and a health information technology (HIT) registered on the Australian Register of Therapeutic Goods. Importantly, research into implementation of such solutions considers education and training of clinicians to be essential to adoption and optimization of use in standard clinical practice. The BMC's Youth Mental Health and Technology Program has subsequently developed a comprehensive education and training program to accompany implementation of the digital health solution.

OBJECTIVE

This paper describes the protocol for an evaluation study to assess the effectiveness of the education and training program on the adoption and optimization of use of the digital health solution in service delivery. It also describes the proposed tools to assess the impact of training on knowledge and skills of mental health clinicians.

METHODS

The evaluation study will use the Kirkpatrick Evaluation Model as a framework with 4 levels of analysis: Reaction (to education and training), Learning (knowledge acquired), Behavior (practice change), and Results (client outcomes). Quantitative and qualitative data will be collected using a variety of tools, including evaluation forms, pre- and postknowledge questionnaires, skill development and behavior change scales, as well as a real-time clinical practice audit.

RESULTS

This project is funded by philanthropic funding from Future Generation Global. Ethics approval has been granted via Sydney Local Health District's Human Research Ethics Committee. At the time of this publication, clinicians and their services were being recruited to this study. The first results are expected to be submitted for publication in 2021.

CONCLUSIONS

The education and training program teaches clinicians the necessary knowledge and skills to assess, monitor, and manage complex needs; mood and psychotic syndromes; and trajectories of youth mental ill-health using a HIT that facilitates a highly personalized and measurement-based model of care. The digital health solution may therefore guide clinicians to help young people recover low functioning associated with subthreshold diagnostic presentations and prevent progression to more serious mental ill-health.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/24697.

Using Digital Technologies to Facilitate Care Coordination Between Youth Mental Health Services: A Guide for Implementation

Enhanced care coordination is essential to improving access to and navigation between youth mental health services. By facilitating better communication and coordination within and between youth mental health services, the goal is to guide young people quickly to the level of care they need and reduce instances of those receiving inappropriate care (too much or too little), or no care at all. Yet, it is often unclear how this goal can be achieved in a scalable way in local regions. We recommend using technology-enabled care coordination to facilitate streamlined transitions for young people across primary, secondary, more specialised or hospital-based care. First, we describe how technology-enabled care coordination could be achieved through two fundamental shifts in current service provisions; a model of care which puts the person at the centre of their care; and a technology infrastructure that facilitates this model. Second, we detail how dynamic simulation modelling can be used to rapidly test the operational features of implementation and the likely impacts of technology-enabled care coordination in a local service environment. Combined with traditional implementation research, dynamic simulation modelling can facilitate the transformation of real-world services. This work demonstrates the benefits of creating a smart health service infrastructure with embedded dynamic simulation modelling to improve operational efficiency and clinical outcomes through participatory and data driven health service planning.