The effect of depression and anxiety symptom severity on clinical outcomes and app use in digital mental health treatments: Meta-regression of three trials

Nudges and Prompts Increase Engagement in Self-Guided Digital Health Treatment for Depression and Anxiety: Results From a 3-Arm Randomized Controlled Trial

BACKGROUND

Accessible and effective approaches to mental health treatment are important because of common barriers such as cost, stigma, and provider shortage. The effectiveness of self-guided treatment is well established, and its use has intensified because of the COVID-19 pandemic. Engagement remains important as dose-response relationships have been observed. Platforms such as Facebook (Meta Platform, Inc), LinkedIn (Microsoft Corp), and X Corp (formerly known as Twitter, Inc) use principles of behavioral economics to increase engagement. We hypothesized that similar concepts would increase engagement in self-guided digital health.

OBJECTIVE

This 3-arm randomized controlled trial aimed to test whether members of 2 digital self-health courses for anxiety and depression would engage with behavioral nudges and prompts. Our primary hypothesis was that members would click on 2 features: tips and a to-do checklist. Our secondary hypothesis was that members would prefer to engage with directive tips in arm 2 versus social proof and present bias tips in arm 3. Our tertiary hypothesis was that rotating tips and a to-do checklist would increase completion rates. The results of this study will form a baseline for future artificial intelligence-directed research.

METHODS

Overall, 13,224 new members registered between November 2021 and May 2022 for Evolution Health's self-guided treatment courses for anxiety and depression. The control arm featured a member home page without nudges or prompts. Arm 2 featured a home page with a tip-of-the-day section. Arm 3 featured a home page with a tip-of-the-day section and a to-do checklist. The research protocol for this study was published in JMIR Research Protocols on August 15, 2022.

RESULTS

Arm 3 had significantly younger members (F2,4564=40.97; P<.001) and significantly more female members (χ24=92.2; P<.001) than the other 2 arms. Control arm members (1788/13,224, 13.52%) completed an average of 1.5 course components. Arm 2 members (865/13,224, 6.54%) clicked on 5% of tips and completed an average of 1.8 course components. Arm 3 members (1914/13,224, 14.47%) clicked on 5% of tips, completed 2.7 of 8 to-do checklist items, and completed an average of 2.11 course components. Completion rates in arm 2 were greater than those in arm 1 (z score=3.37; P<.001), and completion rates in arm 3 were greater than those in arm 1 (z score=12.23; P<.001). Engagement in all 8 components in arm 3 was higher than that in arm 2 (z score=1.31; P<.001).

CONCLUSIONS

Members engaged with behavioral nudges and prompts. The results of this study may be important because efficacy is related to increased engagement. Due to its novel approach, the outcomes of this study should be interpreted with caution and used as a guideline for future research in this nascent field.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/37231.

Determinants and outcome correlates of engagement with a mobile mental health intervention for depression and anxiety in middle-aged and older adults

OBJECTIVES

To examine baseline factors (i.e., age, gender, mobile device proficiency, sensory impairment) associated with app engagement in a 12-week mental health app intervention and to explore whether app engagement predicts changes in depression and anxiety symptoms among middle-aged and older adults.

METHOD

Mobile device proficiency, sensory impairment, depression, and anxiety symptoms were measured using questionnaires. App engagement was defined by metrics characterizing the core intervention features (i.e., messages sent to therapist, mindfulness meditation minutes, action tasks completed). Multiple regressions and multilevel models were conducted.

RESULTS

Forty-nine participants (M age = 57.40, SD = 11.09 years) enrolled. Women (β = .35, p < .05) and participants with less sensory impairment completed more action tasks (β = -.40, p < .05). Depressive and anxiety symptoms measured within the app declined significantly across treatment. Clinical significant improvements were observed for depression in 48.9% and for anxiety in 40% of participants. App engagement metrics were not predictive of depression or anxiety symptoms, either incrementally in time-lagged models or cumulatively in hierarchical linear regression analyses.

CONCLUSION

App engagement is multifaceted; participants engaged differently by gender and ability. Participation in this digital mental health intervention reduced depression and anxiety symptoms, but these findings should be interpreted with caution as the study did not include a control condition. Our findings underscore the importance of considering individual factors that may influence use of a digital mental health intervention.

Demographic and clinical characteristics associated with anxiety and depressive symptom outcomes in users of a digital mental health intervention incorporating a relational agent

BACKGROUND

Digital mental health interventions (DMHIs) may reduce treatment access issues for those experiencing depressive and/or anxiety symptoms. DMHIs that incorporate relational agents may offer unique ways to engage and respond to users and to potentially help reduce provider burden. This study tested Woebot for Mood & Anxiety (W-MA-02), a DMHI that employs Woebot, a relational agent that incorporates elements of several evidence-based psychotherapies, among those with baseline clinical levels of depressive or anxiety symptoms. Changes in self-reported depressive and anxiety symptoms over 8 weeks were measured, along with the association between each of these outcomes and demographic and clinical characteristics.

METHODS

This exploratory, single-arm, 8-week study of 256 adults yielded non-mutually exclusive subsamples with either clinical levels of depressive or anxiety symptoms at baseline. Week 8 Patient Health Questionnaire-8 (PHQ-8) changes were measured in the depressive subsample (PHQ-8 ≥ 10). Week 8 Generalized Anxiety Disorder-7 (GAD-7) changes were measured in the anxiety subsample (GAD-7 ≥ 10). Demographic and clinical characteristics were examined in association with symptom changes via bivariate and multiple regression models adjusted for W-MA-02 utilization. Characteristics included age, sex at birth, race/ethnicity, marital status, education, sexual orientation, employment status, health insurance, baseline levels of depressive and anxiety symptoms, and concurrent psychotherapeutic or psychotropic medication treatments during the study.

RESULTS

Both the depressive and anxiety subsamples were predominantly female, educated, non-Hispanic white, and averaged 38 and 37 years of age, respectively. The depressive subsample had significant reductions in depressive symptoms at Week 8 (mean change =-7.28, SD = 5.91, Cohen's d = -1.23, p < 0.01); the anxiety subsample had significant reductions in anxiety symptoms at Week 8 (mean change = -7.45, SD = 5.99, Cohen's d = -1.24, p < 0.01). No significant associations were found between sex at birth, age, employment status, educational background and Week 8 symptom changes. Significant associations between depressive and anxiety symptom outcomes and sexual orientation, marital status, concurrent mental health treatment, and baseline symptom severity were found.

CONCLUSIONS

The present study suggests early promise for W-MA-02 as an intervention for depression and/or anxiety symptoms. Although exploratory in nature, this study revealed potential user characteristics associated with outcomes that can be investigated in future studies.

TRIAL REGISTRATION

This study was retrospectively registered on ClinicalTrials.gov (#NCT05672745) on January 5th, 2023.

The Effect of a Digital Mental Health Program on Anxiety and Depression Symptoms: Retrospective Analysis of Clinical Severity

BACKGROUND

Evidence-based digital health programs have shown efficacy in being primary tools to improve emotional and mental health, as well as offering supplementary support to individuals undergoing psychotherapy for anxiety, depression, and other mental health disorders. However, information is lacking about the dose response to digital mental health interventions.

OBJECTIVE

The objective of the study was to examine the effect of time in program and program usage on symptom change among individuals enrolled in a real-world comprehensive digital mental health program (myStrength) who are experiencing severe anxiety or depression.

METHODS

Eligible participants (N=18,626) were adults aged 18 years and older who were enrolled in myStrength for at least four weeks as part of their employee wellness benefit program, who completed baseline, the 2-week, 2-month, and 6-month surveys querying symptoms of anxiety (Generalized Anxiety Disorder-7 [GAD-7]) and depression (Patient Health Questionnaire-9 [PHQ-9]). Linear growth curve models were used to analyze the effect of average weekly program usage on subsequent GAD-7 and PHQ-9 scores for participants with scores indicating severe anxiety (GAD-7≥15) or depression (PHQ-9≥15). All models were adjusted for baseline score and demographics.

RESULTS

Participants in the study (N=1519) were 77.4% female (1176/1519), had a mean age of 45 years (SD 14 years), and had an average enrollment time of 3 months. At baseline, participants reported an average of 9.39 (SD 6.04) on the GAD-7 and 11.0 (SD 6.6) on the PHQ-9. Those who reported 6-month results had an average of 8.18 (SD 6.15) on the GAD-7 and 9.18 (SD 6.79) on the PHQ-9. Participants with severe scores (n=506) experienced a significant improvement of 2.97 (SE 0.35) and 3.97 (SE 0.46) at each time point for anxiety and depression, respectively (t=-8.53 and t=-8.69, respectively; Ps<.001). Those with severe baseline scores also saw a reduction of 0.27 (SE 0.08) and 0.25 (SE 0.09) points in anxiety and depression, respectively, for each additional program activity per week (t=-3.47 and t=-2.66, respectively; Ps<.05).

CONCLUSIONS

For participants with severe baseline scores, the study found a clinically significant reduction of approximately 9 points for anxiety and 12 points for depression after 6 months of enrollment, suggesting that interventions targeting mental health must maintain active, ongoing engagement when symptoms are present and be available as a continuous resource to maximize clinical impact, specifically in those experiencing severe anxiety or depression. Moreover, a dosing effect was shown, indicating improvement in outcomes among participants who engaged with the program every other day for both anxiety and depression. This suggests that digital mental health programs that provide both interesting and evidence-based activities could be more successful in further improving mental health outcomes.

The impact of guided versus supportive coaching on mental health app engagement and clinical outcomes

Although mobile mental health apps have the unique potential to increase access to care, evidence reveals engagement is low unless coupled with coaching. However, most coaching protocols are limited in their scalability. This study assesses how human support and guidance from a Digital Navigator (DN), a scalable coach, can impact mental health app engagement and effectiveness on anxiety and depressive symptoms. This study aims to detach components of coaching, specifically personalized recommendations versus general support, to inform scalability of coaching models for mental health apps. 156 participants were split into the DN Guide versus DN Support groups for the 6-week study. Both groups utilized the mindLAMP app for the duration of the study and had equal time with the DN, but the Guide group received personalized app recommendations. The Guide group completed significantly more activities than the Support group. 34% (49/139) of all participants saw a 25% decrease in PHQ-9 scores and 38% (53/141) saw a 25% decrease in GAD-7 scores. These findings show mental health apps, especially when supported by DNs, can reduce depression and anxiety symptoms when coupled with coaching, suggesting a feasible path for large-scale deployment.

Who benefits from guided internet-based interventions? A systematic review of predictors and moderators of treatment outcome

To our knowledge, no systematic review has been conducted on predictors or moderators of treatment outcome across diagnoses in guided internet-based interventions (IBIs) for adults. To identify who benefits from this specific format and therein inform future research on improving patient-treatment fit, we aimed to aggregate results of relevant studies. 2100 articles, identified by searching the databases PsycInfo, Ovid Medline, and Pubmed and through snowballing, were screened in April/May 2021 and October 2022. Risk of bias and intra- and interrater reliability were assessed. Variables were grouped by predictor category, then synthesized using vote counting based on direction of effect. N = 60 articles were included in the review. Grouping resulted in 88 predictors/moderators, of which adherence, baseline symptoms, education, age, and gender were most frequently assessed. Better adherence, treatment credibility, and working alliance emerged as conclusive predictors/moderators for better outcome, whereas higher baseline scores predicted more reliable change but higher post-treatment symptoms. Results of all other predictors/moderators were inconclusive or lacked data. Our review highlights that it is currently difficult to predict, across diagnoses, who will benefit from guided IBIs. Further rigorous research is needed to identify predictors and moderators based on a sufficient number of studies. PROSPERO registration: CRD42021242305.

Prognostic factors and effect modifiers for personalisation of internet-based cognitive behavioural therapy among university students with subthreshold depression: A secondary analysis of a factorial trial

BACKGROUND

Internet-cognitive behavioural therapy (iCBT) for depression can include multiple components. This study explored depressive symptom improvement prognostic factors (PFs) and effect modifiers (EMs) for five common iCBT components including behavioural activation, cognitive restructuring, problem solving, self-monitoring, and assertion training.

METHODS

We used data from a factorial trial of iCBT for subthreshold depression among Japanese university students (N = 1093). The primary outcome was the change in PHQ-9 scores at 8 weeks from baseline. Interactions between each component and various baseline characteristics were estimated using a mixed-effects model for repeated measures. We calculated multiplicity-adjusted p-values at 5 % false discovery rate using the Benjamini-Hochberg procedure.

RESULTS

After multiplicity adjustment, the baseline PHQ-9 total score emerged as a PF and exercise habits as an EM for self-monitoring (adjusted p-values <0.05). The higher the PHQ-9 total score at baseline (range: 5-14), the greater the decrease after 8 weeks. For each 5-point increase at baseline, the change from baseline to 8 weeks was bigger by 2.8 points. The more frequent the exercise habits (range: 0-2 points), the less effective the self-monitoring component. The difference in PHQ-9 change scores between presence or absence of self-monitoring was smaller by 0.94 points when the participant exercised one level more frequently. Additionally, the study suggested seven out of 36 PFs and 14 out of 160 EMs examined were candidates for future research.

LIMITATIONS

Generalizability is limited to university students with subthreshold depression.

CONCLUSIONS

These results provide some helpful information for the future development of individualized iCBT algorithms for depression.

Smartphone app engagement and clinical outcomes in a hybrid clinic

Despite the growing prevalence of mental health-related smartphone apps, low real-world engagement has prevented these apps from transforming the mental health landscape. Integrating mental health apps into more traditional therapeutic models appears to support better clinical outcomes, but also raises questions about the relationship between app engagement, the app itself, and the coach or clinician. This study explores patient app engagement patterns and the associated clinical outcomes gathered from piloting a digital clinic. Patients with anxiety or depression completed eight clinical visits and coach visits over a median of 83 days with a standard deviation of 17.25 days. Between clinical visits, patients completed therapeutic activities on the mindLAMP app. Mean PHQ-9 and GAD-7 scores decreased from the intake visit to both visit 4 and visit 8. Patients had high app engagement, but engagement did not correlate with outcomes. From intake visit to visit 4, the interaction effects indicate significant differences in the change of both PHQ-9 and GAD-7 depending on participants' average app satisfaction and clinician/coach satisfaction (as measured by WAI-SR) with engagement. Overall, results support the feasibility of incorporating an app into a hybrid clinic.

Comparing Professional and Consumer Ratings of Mental Health Apps: Mixed Methods Study

Background

As the number of mental health apps has grown, increasing efforts have been focused on establishing quality tailored reviews. These reviews prioritize clinician and academic views rather than the views of those who use them, particularly those with lived experiences of mental health problems. Given that the COVID-19 pandemic has increased reliance on web-based and mobile mental health support, understanding the views of those with mental health conditions is of increasing importance.

Objective

This study aimed to understand the opinions of people with mental health problems on mental health apps and how they differ from established ratings by professionals.

Methods

A mixed methods study was conducted using a web-based survey administered between December 2020 and April 2021, assessing 11 mental health apps. We recruited individuals who had experienced mental health problems to download and use 3 apps for 3 days and complete a survey. The survey consisted of the One Mind PsyberGuide Consumer Review Questionnaire and 2 items from the Mobile App Rating Scale (star and recommendation ratings from 1 to 5). The consumer review questionnaire contained a series of open-ended questions, which were thematically analyzed and using a predefined protocol, converted into binary (positive or negative) ratings, and compared with app ratings by professionals and star ratings from app stores.

Results

We found low agreement between the participants’ and professionals’ ratings. More than half of the app ratings showed disagreement between participants and professionals (198/372, 53.2%). Compared with participants, professionals gave the apps higher star ratings (3.58 vs 4.56) and were more likely to recommend the apps to others (3.44 vs 4.39). Participants’ star ratings were weakly positively correlated with app store ratings (r=0.32, P=.01). Thematic analysis found 11 themes, including issues of user experience, ease of use and interactivity, privacy concerns, customization, and integration with daily life. Participants particularly valued certain aspects of mental health apps, which appear to be overlooked by professional reviewers. These included functions such as the ability to track and measure mental health and providing general mental health education. The cost of apps was among the most important factors for participants. Although this is already considered by professionals, this information is not always easily accessible.

Conclusions

As reviews on app stores and by professionals differ from those by people with lived experiences of mental health problems, these alone are not sufficient to provide people with mental health problems with the information they desire when choosing a mental health app. App rating measures must include the perspectives of mental health service users to ensure ratings represent their priorities. Additional work should be done to incorporate the features most important to mental health service users into mental health apps.

Predictors of engagement with remote sensing technologies for symptom measurement in Major Depressive Disorder

BACKGROUND

Remote sensing for the measurement and management of long-term conditions such as Major Depressive Disorder (MDD) is becoming more prevalent. User-engagement is essential to yield any benefits. We tested three hypotheses examining associations between clinical characteristics, perceptions of remote sensing, and objective user engagement metrics.

METHODS

The Remote Assessment of Disease and Relapse - Major Depressive Disorder (RADAR-MDD) study is a multicentre longitudinal observational cohort study in people with recurrent MDD. Participants wore a FitBit and completed app-based assessments every two weeks for a median of 18 months. Multivariable random effects regression models pooling data across timepoints were used to examine associations between variables.

RESULTS

A total of 547 participants (87.8% of the total sample) were included in the current analysis. Higher levels of anxiety were associated with lower levels of perceived technology ease of use; increased functional disability was associated with small differences in perceptions of technology usefulness and usability. Participants who reported higher system ease of use, usefulness, and acceptability subsequently completed more app-based questionnaires and tended to wear their FitBit activity tracker for longer. All effect sizes were small and unlikely to be of practical significance.

LIMITATIONS

Symptoms of depression, anxiety, functional disability, and perceptions of system usability are measured at the same time. These therefore represent cross-sectional associations rather than predictions of future perceptions.

CONCLUSIONS

These findings suggest that perceived usability and actual use of remote measurement technologies in people with MDD are robust across differences in severity of depression, anxiety, and functional impairment.

Testing Behavioral Nudges and Prompts in Digital Courses for Self-Guided Treatment of Depression and Anxiety, Protocol for 3-Arm Randomized Controlled Trial (Preprint)

Background

Despite showing strong evidence of positive outcomes, a common problem in the field of digital health is poor engagement and adherence. Non–health care, for-profit digital ventures, such as Facebook, LinkedIn, and Twitter, conduct behavioral experiments to increase user engagement. To our knowledge, digital health organizations have not published similar types of experiments in ad libitum environments, and there are limited published data indicating whether nudges and prompts can be leveraged to increase engagement with digital health interventions.

Objective

The main objective of our 3-arm randomized controlled trial is to test whether registered members in two well-established digital health courses for anxiety and depression will engage with four different types of nudges and prompts, and whether engaging with nudges and prompts increases engagement within the courses.

Methods

New members who register for the self-guided anxiety and depression courses on the Evolution Health platform will be randomized into 1 of 3 arms. The first control arm will feature a member home page without any behavioral nudges or prompts. The second arm will feature a member home page with a Tip-of-the-Day section containing directive content. Arm 3 will feature a member home page with a Tip-of-the-Day section containing social proof and present bias content. The third arm will also feature a to-do item checklist.

Results

The experiment was designed in August 2021 and was launched in November 2021. Initially, we will measure engagement with the tips and the to-do checklist by calculating the frequency of use by age and gender. If members do engage, we will then, according to age and gender, examine whether nudges and prompts result in higher course completion rates and whether specific types of prompts and nudges are more popular than others.

Conclusions

Our 3-arm randomized controlled trial will be the first to compare four distinct types of behavioral prompts and nudges in two self-guided digital health courses that were designed to treat mental health issues. We expect the results to generate insights into which types of behavioral prompts and nudges work best in the population. If they are shown to increase engagement, the insights will then be used to apply prompts and nudges to the platform’s addiction-focused courses. Based on the results of the experiment, the insights will be applied to using artificial intelligence to train the platform to recognize different usage patterns and provide specific engagement recommendations to stratified users.

International Registered Report Identifier (IRRID)

DERR1-10.2196/37231