Surveying the Role of Analytics in Evaluating Digital Mental Health Interventions for Transition-Aged Youth: Scoping Review

User journey method: a case study for improving digital intervention use measurement

BACKGROUND

Many digital mental health interventions meet low levels of use. However, current use measurement methods do not necessarily help identify which intervention elements are associated with dropout, despite this information potentially facilitating iterative intervention development. Here, we suggest improving the comprehensiveness of intervention use measurement with the user journey method, which evaluates every intervention element to identify intervention-specific use barriers.

METHODS

We applied user journey method in a clinical trial that investigated the efficacy of a novel game-based intervention, Meliora, for adult Major Depressive Disorder. We modelled the intervention for its four technological (Recruitment, Website, Questionnaires, Intervention Software) and two interpersonal elements (Assessment, Support). We then applied the user journey method to measure how many users proceeded from one element to the next combining social media analytics, website use data, signup data, clinical subject coordinator interview data, symptom questionnaire data, and behavioral intervention use data. These measurements were complemented with the qualitative analysis of the study discovery sources and email support contacts.

RESULTS

Recruitment: The intervention recruitment reached at least 145,000 Finns, with social media, word-of-mouth, and news and web sources being the most effective recruitment channels. Website: The study website received 16,243 visitors, which led to 1,007 sign-ups.

ASSESSMENT

895 participants were assessed and 735 were accepted. Intervention Software: 498 participants were assigned to the active intervention or comparator, of whom 457 used them at least once: on average, for 17.3 h (SD = 20.4 h) on 19.7 days (SD = 20.7 d) over a period of 38.9 days (SD = 31.2 d). The 28 intervention levels were associated with an average dropout rate of 2.6%, with two sections exhibiting an increase against this baseline. 150 participants met the minimum adherence goal of 24 h use. Questionnaires: 116 participants completed the post-intervention questionnaire.

SUPPORT

313 signed-up participants contacted the researchers via email.

CONCLUSION

The user journey method allowed for the comprehensive evaluation of the six intervention elements, and enabled identifying use barriers expediting iterative intervention development and implementation.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT05426265. Registered 28 June 2022, https://clinicaltrials.gov/ct2/show/NCT05426265 .

Artificial Intelligence-Driven Analysis of Telehealth Effectiveness in Youth Mental Health Services: Insights from SAMHSA Data

Background: The rapid adoption of telehealth services for youth mental health care necessitates a comprehensive evaluation of its effectiveness. This study aimed to analyze the impact of telehealth on youth mental health outcomes using artificial intelligence techniques applied to large-scale public health data. Methods: We conducted an AI-driven analysis of data from the National Survey on Drug Use and Health (NSDUH) and other SAMHSA datasets. Machine learning techniques, including random forest models, K-means clustering, and time series analysis, were employed to evaluate telehealth adoption patterns, predictors of effectiveness, and comparative outcomes with traditional in-person care. Natural language processing was used to analyze sentiment in user feedback. Results: Telehealth adoption among youth increased significantly, with usage rising from 2.3 sessions per year in 2019 to 8.7 in 2022. Telehealth showed comparable effectiveness to in-person care for depressive disorders and superior effectiveness for anxiety disorders. Session frequency, age, and prior diagnosis were identified as key predictors of telehealth effectiveness. Four distinct user clusters were identified, with socioeconomic status and home environment strongly associated with positive outcomes. States with favorable reimbursement policies saw a 15% greater increase in youth telehealth utilization and 7% greater improvement in mental health outcomes. Conclusions: Telehealth demonstrates significant potential in improving access to and effectiveness of mental health services for youth. However, addressing technological barriers and socioeconomic disparities is crucial to maximize its benefits.

A Process Evaluation of a Mobile App for Medical Students Aimed at Increasing Resilience and Decreasing Stigma in Mental Health

OBJECTIVE

The purpose of this pilot study was to conduct a process evaluation of a mental health and wellness mobile health (mHealth) application for medical students designed to increase resilience and decrease mental health stigma.

METHODS

The customized application, MindfulMEDS, was developed with peer-focused interactive modules specific to medical students within an existing system called Sharpen®. The Sharpen® system contains an extensive library of didactic and experiential mental health and wellness content built specifically to promote evidence-based protective factors for resilience. A mixed-methods approach including surveys and focus groups assessed participant resiliency, perception of mental health within the context of medical school, and evaluation of the app. Assessments were conducted at baseline (n = 66), six months (n = 30), and one year (n = 24). Demographic information was collected once at baseline as a part of the initial survey.  Results: A total of 215 users were registered in MindfulMEDS, consumed 83 courses, and engaged in 1,428 "connect clicks" to community resources and crisis-response supports. Resilience levels did not change significantly between surveys; however, a significant decrease in the perception of mental health stigma associated with utilizing mental health resources was observed. Focus group participants (n = 11) reported the screening tools to be useful, encouraged expansion, and suggested additional reminders to access the app to increase engagement.

CONCLUSION

Findings of this pilot study demonstrate the feasibility of implementing MindfulMEDS (an mHealth app focused on mental health and wellness) among medical students. Students found the app experience valuable, accessed mental health screeners embedded within the app, utilized the app to seek help, and engaged with the app to learn more about mental health. There was also a decrease in mental health stigma observed during the course of the study. Based on these results, we propose that medical schools incorporate mobile-based technology into their mental health support programs.

The Use of Mobile Assessments for Monitoring Mental Health in Youth: Umbrella Review

BACKGROUND

Improving mental health in youth is a major concern. Future approaches to monitor and intervene in youth mental health problems should rely on mobile tools that allow for the daily monitoring of mental health both actively (eg, using ecological momentary assessments [EMAs]) and passively (eg, digital phenotyping) by capturing individuals' data.

OBJECTIVE

This umbrella review aims to (1) report the main characteristics of existing reviews on mental health and young people, including mobile approaches to mental health; (2) describe EMAs and trace data and the mental health conditions investigated; (3) report the main results; and (4) outline promises, limitations, and directions for future research.

METHODS

A systematic literature search was carried out in 9 scientific databases (Communication & Mass Media Complete, Psychology and Behavioral Sciences Collection, PsycINFO, CINAHL, ERIC, MEDLINE, the ProQuest Sociology Database, Web of Science, and PubMed) on January 30, 2022, coupled with a hand search and updated in July 2022. We included (systematic) reviews of EMAs and trace data in the context of mental health, with a specific focus on young populations, including children, adolescents, and young adults. The quality of the included reviews was evaluated using the AMSTAR (Assessment of Multiple Systematic Reviews) checklist.

RESULTS

After the screening process, 30 reviews (published between 2016 and 2022) were included in this umbrella review, of which 21 (70%) were systematic reviews and 9 (30%) were narrative reviews. The included systematic reviews focused on symptoms of depression (5/21, 24%); bipolar disorders, schizophrenia, or psychosis (6/21, 29%); general ill-being (5/21, 24%); cognitive abilities (2/21, 9.5%); well-being (1/21, 5%); personality (1/21, 5%); and suicidal thoughts (1/21, 5%). Of the 21 systematic reviews, 15 (71%) summarized studies that used mobile apps for tracing, 2 (10%) summarized studies that used them for intervention, and 4 (19%) summarized studies that used them for both intervention and tracing. Mobile tools used in the systematic reviews were smartphones only (8/21, 38%), smartphones and wearable devices (6/21, 29%), and smartphones with other tools (7/21, 33%). In total, 29% (6/21) of the systematic reviews focused on EMAs, including ecological momentary interventions; 33% (7/21) focused on trace data; and 38% (8/21) focused on both. Narrative reviews mainly focused on the discussion of issues related to digital phenotyping, existing theoretical frameworks used, new opportunities, and practical examples.

CONCLUSIONS

EMAs and trace data in the context of mental health assessments and interventions are promising tools. Opportunities (eg, using mobile approaches in low- and middle-income countries, integration of multimodal data, and improving self-efficacy and self-awareness on mental health) and limitations (eg, absence of theoretical frameworks, difficulty in assessing the reliability and effectiveness of such approaches, and need to appropriately assess the quality of the studies) were further discussed.

TRIAL REGISTRATION

PROSPERO CRD42022347717; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=347717.

Identifying essential factors that influence user engagement with digital mental health tools in clinical care settings: Protocol for a Delphi study

Introduction

Improving effective user engagement with digital mental health tools has become a priority in enabling the value of digital health. With increased interest from the mental health community in embedding digital health tools as part of care delivery, there is a need to examine and identify the essential factors in influencing user engagement with digital mental health tools in clinical care. The current study will use a Delphi approach to gain consensus from individuals with relevant experience and expertise (e.g. patients, clinicians and healthcare administrators) on factors that influence user engagement (i.e. an essential factor).

Methods

Participants will be invited to complete up to four rounds of online surveys. The first round of the Delphi study comprises of reviewing existing factors identified in literature and commenting on whether any factors they believe are important are missing from the list. Subsequent rounds will involve asking participants to rate the perceived impact of each factor in influencing user engagement with digital mental health tools in clinical care contexts. This work is expected to consolidate the perspectives from relevant stakeholders and the academic literature to identify a core set of factors considered essential in influencing user engagement with digital mental health tools in clinical care contexts.

An Evaluation of 5-Year Web Analytics for HeadsUpGuys: A Men's Depression E-Mental Health Resource

The present study reports an evaluation of web analytics, over a 5-year period, for HeadsUpGuys.org, an eHealth resource for men with depression. Google Analytics, Search Console, and Tag Manager were used to monitor user activity over the course of the website's first 5 years (June 15, 2015-June 15, 2020). Through this period, HeadsUpGuys had a total of 1,665,356 unique users, amounting to 1,948,481 sessions and 3,328,258 page views. Organic traffic accounted for the highest proportion (53.44%; n = 1,041,277) of website sessions. Four of the top 10 Google search queries that brought users to the website related to suicidality. Three countries (United States, United Kingdom, Canada) accounted for almost three-quarters (71.10%; n = 1,385,485) of the site's traffic. Nearly three-quarters (73.35%; n = 1,429,285) of sessions occurred on a mobile device. The goal conversion rate for the Self Check was 60.27%. The average time on page was 2 min 53 s, with a bounce rate of 65.92%, and an exit rate of 57.20%. The goal conversion rate for the Stress Test was 52.89%. The average time on page was 4 min 8 s, with a bounce rate of 72.40% and an exit rate of 48.88%. The conversion rate for the final goal was 11.53%, indicating that approximately one in 10 visitors to the site had a session of at least 3 min. The findings illustrate the potential of eHealth resources to support men's mental health and provide a real-world benchmark to help advance the men's eHealth field.

Learning About the Current State of Digital Mental Health Interventions for Canadian Youth to Inform Future Decision-Making: Mixed Methods Study

Background

The COVID-19 pandemic has increased the demand for youth mental health services in Canada as disruptions to clinical care continue to persist due to the risk of transmission and exposure to the virus. Digital mental health interventions, including web-based resources and mobile apps, have provided opportunities to support youth mental health remotely across Canada. There is a need to better understand how these digital interventions are being selected, recommended, and used in various regions across Canada.

Objective

A national jurisdictional scan was completed to (1) determine what web-based programs, apps, and websites are promoted and licensed in Canada for youth mental health; (2) identify criteria and decision-making processes that Canadian jurisdictions use to select web-based programs, apps, and websites for youth mental health; and (3) identify upcoming trends, innovations, and digital mental health possibilities that are emerging in the youth sector.

Methods

The aims of the jurisdictional scan were addressed through a review of related academic and grey literature; stakeholder interviews, including individuals involved in various areas of the youth mental health sector; and a social media review of pertinent Twitter content.

Results

A total of 66 web-based resources and apps were identified for use by youth in Canada. 16 stakeholder interviews were completed and included discussions with researchers, clinicians, youth organizations, and others involved in digital interventions for youth mental health. These discussions identified a limited use of frameworks used to guide decision-making processes when selecting digital interventions. Many clinicians agreed on a similar set of eligibility requirements for youth mental health apps and digital resources, such as the evidence base and cultural relevance of the intervention. Stakeholders also identified upcoming trends and innovations in the youth digital mental health space, including artificial intelligence, digital phenotyping, and personalized therapy. Over 4 weeks, 2184 tweets were reviewed to identify and compare global and national trends and innovations involving digital mental health and youth. Key trends included the promotion of regional chat services as well as the effects of the COVID-19 pandemic on youth mental health and access to care.

Conclusions

As organizations begin to plan for the delivery of mental health care following the pandemic, there are concerns about the sustainability of these digital mental health interventions as well as a need for services to be more informed by the experiences and preferences of youth.

An Iterative and Collaborative End-to-End Methodology Applied to Digital Mental Health

Artificial intelligence (AI) algorithms together with advances in data storage have recently made it possible to better characterize, predict, prevent, and treat a range of psychiatric illnesses. Amid the rapidly growing number of biological devices and the exponential accumulation of data in the mental health sector, the upcoming years are facing a need to homogenize research and development processes in academia as well as in the private sector and to centralize data into federalizing platforms. This has become even more important in light of the current global pandemic. Here, we propose an end-to-end methodology that optimizes and homogenizes digital research processes. Each step of the process is elaborated from project conception to knowledge extraction, with a focus on data analysis. The methodology is based on iterative processes, thus allowing an adaptation to the rate at which digital technologies evolve. The methodology also advocates for interdisciplinary (from mathematics to psychology) and intersectoral (from academia to the industry) collaborations to merge the gap between fundamental and applied research. We also pinpoint the ethical challenges and technical and human biases (from data recorded to the end user) associated with digital mental health. In conclusion, our work provides guidelines for upcoming digital mental health studies, which will accompany the translation of fundamental mental health research to digital technologies.

Assessing the usability and user engagement of Thought Spot - A digital mental health help-seeking solution for transition-aged youth

OBJECTIVE

To evaluate the perceived usability of and user engagement with a digital platform (Thought Spot) designed to enhance mental health and wellness help-seeking among transition-aged youth (TAY; 17-29-years old).

MATERIALS AND METHODS

Survey responses and usage patterns were collected as part of a randomized controlled trial evaluating the efficacy of Thought Spot. Participants given Thought Spot completed an adapted Usefulness, Satisfaction, and Ease of Use (USE) Questionnaire to measure perceived usability of the platform. User engagement patterns on Thought Spot were examined using analytics data collected throughout the study (March 2018-June 2019).

RESULTS

A total of 131 transition-aged participants completed the USE questionnaire and logged on to Thought Spot at least once. Ease of learning scored higher than ease of use, usefulness and satisfaction. Participants identified numerous strengths and challenges related to usability, visual appeal, functionality and usefulness of the content. In terms of user engagement, most participants stopped using the platform after 3 weeks. Participants searched and were interested in a variety of resources, including mental health, counselling and social services.

DISCUSSION

Participants reported mixed experiences while using Thought Spot and exhibited low levels of long-term user engagement. User satisfaction, the willingness to recommend Thought Spot to others, and the willingness for future use appeared to be influenced by content relevance, ease of learning, available features, and other contextual factors. Analysis of the types of resources viewed and searches conducted by TAY end-users provided insight into their behaviour and needs.

CONCLUSION

Users had mixed perceptions about the usability of Thought Spot, which may have contributed to the high attrition rate. User satisfaction and engagement appears to be influenced by content relevance, ease of learning, and the types of features available. Further investigation to understand the contextual factors that affect TAYs' adoption and engagement with digital mental health tools is required.