Developing an explanatory theoretical model for engagement with a web-based mental health platform: results of a mixed methods study

Application of a Sociotechnical Framework to Uncover Factors That Influence Effective User Engagement With Digital Mental Health Tools in Clinical Care Contexts: Scoping Review

BACKGROUND

Digital health tools such as mobile apps and patient portals continue to be embedded in clinical care pathways to enhance mental health care delivery and achieve the quintuple aim of improving patient experience, population health, care team well-being, health care costs, and equity. However, a key issue that has greatly hindered the value of these tools is the suboptimal user engagement by patients and families. With only a small fraction of users staying engaged over time, there is a great need to better understand the factors that influence user engagement with digital mental health tools in clinical care settings.

OBJECTIVE

This review aims to identify the factors relevant to user engagement with digital mental health tools in clinical care settings using a sociotechnical approach.

METHODS

A scoping review methodology was used to identify the relevant factors from the literature. Five academic databases (MEDLINE, Embase, CINAHL, Web of Science, and PsycINFO) were searched to identify pertinent articles using key terms related to user engagement, mental health, and digital health tools. The abstracts were screened independently by 2 reviewers, and data were extracted using a standardized data extraction form. Articles were included if the digital mental health tool had at least 1 patient-facing component and 1 clinician-facing component, and at least one of the objectives of the article was to examine user engagement with the tool. An established sociotechnical framework developed by Sittig and Singh was used to inform the mapping and analysis of the factors.

RESULTS

The database search identified 136 articles for inclusion in the analysis. Of these 136 articles, 84 (61.8%) were published in the last 5 years, 47 (34.6%) were from the United States, and 23 (16.9%) were from the United Kingdom. With regard to examining user engagement, the majority of the articles (95/136, 69.9%) used a qualitative approach to understand engagement. From these articles, 26 factors were identified across 7 categories of the established sociotechnical framework. These ranged from technology-focused factors (eg, the modality of the tool) and the clinical environment (eg, alignment with clinical workflows) to system-level issues (eg, reimbursement for physician use of the digital tool with patients).

CONCLUSIONS

On the basis of the factors identified in this review, we have uncovered how the tool, individuals, the clinical environment, and the health system may influence user engagement with digital mental health tools for clinical care. Future work should focus on validating and identifying a core set of essential factors for user engagement with digital mental health tools in clinical care environments. Moreover, exploring strategies for improving user engagement through these factors would be useful for health care leaders and clinicians interested in using digital health tools in care.

The Effect of Predicted Compliance With a Web-Based Intervention for Anxiety and Depression Among Latin American University Students: Randomized Controlled Trial

BACKGROUND

Web-based cognitive behavioral therapy (wb-CBT) is a scalable way to reach distressed university students. Guided wb-CBT is typically superior to self-guided wb-CBT over short follow-up periods, but evidence is less clear over longer periods.

OBJECTIVE

This study aimed to compare short-term (3 months) and longer-term (12 months) aggregate effects of guided and self-guided wb-CBT versus treatment as usual (TAU) in a randomized controlled trial of Colombian and Mexican university students and carry out an initially unplanned secondary analysis of the role of differential predicted compliance in explaining these differences.

METHODS

The 1319 participants, recruited either through email and social media outreach invitations or from waiting lists of campus mental health clinics, were undergraduates (1038/1319, 78.7% female) with clinically significant baseline anxiety (Generalized Anxiety Disorder-7 score≥10) or depression (Patient Health Questionnaire-9 score≥10). The intervention arms comprised guided wb-CBT with weekly asynchronous written human feedback, self-guided wb-CBT with the same content as the guided modality, and TAU as provided at each university. The prespecified primary outcome was joint remission (Generalized Anxiety Disorder-7 score=0-4 and Patient Health Questionnaire-9 score=0-4). The secondary outcome was joint symptom reduction (mean scores on the Patient Health Questionnaire Anxiety and Depression Scale) at 3 and 12 months after randomization.

RESULTS

As reported previously, 3-month outcomes were significantly better with guided wb-CBT than self-guided wb-CBT (P=.02) or TAU (P=.02). However, subsequent follow-up showed that 12-month joint remission (adjusted risk differences=6.0-6.5, SE 0.4-0.5, and P<.001 to P=.007; adjusted mean differences=2.70-2.69, SE 0.7-0.8, and P<.001 to P=.001) was significantly better with self-guided wb-CBT than with the other interventions. Participants randomly assigned to the guided wb-CBT arm spent twice as many minutes logged on as those in the self-guided wb-CBT arm in the first 12 weeks (mean 12.5, SD 36.9 vs 5.9, SD 27.7; χ21=107.1, P<.001), whereas participants in the self-guided wb-CBT arm spent twice as many minutes logged on as those in the guided wb-CBT arm in weeks 13 to 52 (mean 0.4, SD 7.5 vs 0.2, SD 4.4; χ21=10.5, P=.001). Subgroup analysis showed that this longer-term superiority of self-guided wb-CBT was confined to the 40% (528/1319) of participants with high predicted self-guided wb-CBT compliance beyond 3 months based on a counterfactual nested cross-validated machine learning model. The 12-month outcome differences were nonsignificant across arms among other participants (all P>.05).

CONCLUSIONS

The results have important practical implications for precision intervention targeting to maximize longer-term wb-CBT benefits. Future research needs to investigate strategies to increase sustained guided wb-CBT use once guidance ends.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04780542; https://www.clinicaltrials.gov/study/NCT04780542.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1186/s13063-022-06255-3.

Barriers and facilitators to the implementation of digital technologies in mental health systems: a qualitative systematic review to inform a policy framework

BACKGROUND

Despite the potential for improved population mental health and wellbeing, the integration of mental health digital interventions has been difficult to achieve. In this qualitative systematic review, we aimed to identify barriers and facilitators to the implementation of digital technologies in mental healthcare systems, and map these to an implementation framework to inform policy development.

METHODS

We searched Medline, Embase, Scopus, PsycInfo, Web of Science, and Google Scholar for primary research articles published between January 2010 and 2022. Studies were considered eligible if they reported barriers and/or facilitators to the integration of any digital mental healthcare technologies. Data were extracted using EPPI-Reviewer Web and analysed thematically via inductive and deductive cycles.

RESULTS

Of 12,525 references identified initially, 81 studies were included in the final analysis. Barriers and facilitators were grouped within an implementation (evidence-practice gap) framework across six domains, organised by four levels of mental healthcare systems. Broadly, implementation was hindered by the perception of digital technologies as impersonal tools that add additional burden of care onto both providers and patients, and change relational power asymmetries; an absence of resources; and regulatory complexities that impede access to universal coverage. Facilitators included person-cantered approaches that consider patients' intersectional features e.g., gender, class, disability, illness severity; evidence-based training for providers; collaboration among colleagues; appropriate investment in human and financial resources; and policy reforms that tackle universal access to digital health.

CONCLUSION

It is important to consider the complex and interrelated nature of barriers across different domains and levels of the mental health system. To facilitate the equitable, sustainable, and long-term digital transition of mental health systems, policymakers should consider a systemic approach to collaboration between public and private sectors to inform evidence-based planning and strengthen mental health systems.

PROTOCOL REGISTRATION

The protocol is registered on PROSPERO, CRD42021276838.

The engagement problem: A review of engagement with digital mental health interventions and recommendations for a path forward

Purpose of the review

Digital mental health interventions (DMHIs) are an effective and accessible means of addressing the unprecedented levels of mental illness worldwide. Currently, however, patient engagement with DMHIs in real-world settings is often insufficient to see clinical benefit. In order to realize the potential of DMHIs, there is a need to better understand what drives patient engagement.

Recent findings

We discuss takeaways from the existing literature related to patient engagement with DMHIs and highlight gaps to be addressed through further research. Findings suggest that engagement is influenced by patient-, intervention- and systems-level factors. At the patient-level, variables such as sex, education, personality traits, race, ethnicity, age and symptom severity appear to be associated with engagement. At the intervention-level, integrating human support, gamification, financial incentives and persuasive technology features may improve engagement. Finally, although systems-level factors have not been widely explored, the existing evidence suggests that achieving engagement will require addressing organizational and social barriers and drawing on the field of implementation science.

Summary

Future research clarifying the patient-, intervention- and systems-level factors that drive engagement will be essential. Additionally, to facilitate improved understanding of DMHI engagement, we propose the following: (a) widespread adoption of a minimum necessary 5-element engagement reporting framework; (b) broader application of alternative clinical trial designs; and (c) directed efforts to build upon an initial parsimonious conceptual model of DMHI engagement.

A Fully Automated Self-help Biopsychosocial Transdiagnostic Digital Intervention to Reduce Anxiety and/or Depression and Improve Emotional Regulation and Well-being: Pre-Follow-up Single-Arm Feasibility Trial

BACKGROUND

Anxiety disorders and depression are prevalent disorders with high comorbidity, leading to greater chronicity and severity of symptoms. Given the accessibility to treatment issues, more evaluation is needed to assess the potential benefits of fully automated self-help transdiagnostic digital interventions. Innovating beyond the current transdiagnostic one-size-fits-all shared mechanistic approach may also lead to further improvements.

OBJECTIVE

The primary objective of this study was to explore the preliminary effectiveness and acceptability of a new fully automated self-help biopsychosocial transdiagnostic digital intervention (Life Flex) aimed at treating anxiety and/or depression, as well as improving emotional regulation; emotional, social, and psychological well-being; optimism; and health-related quality of life.

METHODS

This was a real-world pre-during-post-follow-up feasibility trial design evaluation of Life Flex. Participants were assessed at the preintervention time point (week 0), during intervention (weeks 3 and 5), at the postintervention time point (week 8), and at 1- and 3-month follow-ups (weeks 12 and 20, respectively).

RESULTS

The results provided early support for the Life Flex program in reducing anxiety (Generalized Anxiety Disorder 7), depression (Patient Health Questionnaire 9), psychological distress (Kessler 6), and emotional dysregulation (Difficulties in Emotional Regulation 36) and increasing emotional, social, and psychological well-being (Mental Health Continuum-Short Form); optimism (Revised Life Orientation Test); and health-related quality of life (EQ-5D-3L Utility Index and Health Rating; all false discovery rate [FDR]<.001). Large within-group treatment effect sizes (range |d|=0.82 to 1.33) were found for most variables from pre- to postintervention assessments and at the 1- and 3-month follow-up. The exceptions were medium treatment effect sizes for EQ-5D-3L Utility Index (range Cohen d=-0.50 to -0.63) and optimism (range Cohen d=-0.72 to -0.79) and small-to-medium treatment effect size change for EQ-5D-3L Health Rating (range Cohen d=-0.34 to -0.58). Changes across all outcome variables were generally strongest for participants with preintervention clinical comorbid anxiety and depression presentations (range |d|=0.58 to 2.01) and weakest for participants presenting with nonclinical anxiety and/or depressive symptoms (|d|=0.05 to 0.84). Life Flex was rated as acceptable at the postintervention time point, and participants indicated that they enjoyed the transdiagnostic program and biological, wellness, and lifestyle-focused content and strategies.

CONCLUSIONS

Given the paucity of evidence on fully automated self-help transdiagnostic digital interventions for anxiety and/or depressive symptomatology and general treatment accessibility issues, this study provides preliminary support for biopsychosocial transdiagnostic interventions, such as Life Flex, as a promising future mental health service delivery gap filler. Following large-scale, randomized controlled trials, the potential benefits of fully automated self-help digital health programs, such as Life Flex, could be considerable.

TRIAL REGISTRATION

Australian and New Zealand Clinical Trials Registry ACTRN12615000480583; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=368007.

Measuring Engagement with Mental Health and Behavior Change Interventions: an Integrative Review of Methods and Instruments

BACKGROUND

Engagement is a complex construct consisting of behavioral, cognitive, and affective dimensions, making engagement a difficult construct to measure. This integrative review aims to (1) present a multidisciplinary overview of measurement methods that are currently used to measure engagement with adult mental health and behavior change interventions, delivered in-person, blended, or digitally, and (2) provide a set of recommendations and considerations for researchers wishing to study engagement.

METHODS

We used an integrative approach and identified original studies and reviews on engagement with mental health or behavior change interventions that were delivered in-person, digitally, or blended.

RESULTS

Forty articles were analyzed in this review. Common methods to assess engagement were through objective usage data, questionnaire-based data, and qualitative data, with objective usage data being used most frequently. Based on the synthesis of engagement measures, we advise researchers to (1) predefine the operationalization of engagement for their specific research context, (2) measure behavioral, cognitive, and affective dimensions of engagement in all cases, and (3) measure engagement over time.

CONCLUSIONS

Current literature shows a bias towards behavioral measures of engagement in research, as most studies measured engagement exclusively through objective usage data, without including cognitive and affective measures of engagement. We hope that our recommendations will help to reduce this bias and to steer engagement research towards an integrated approach.

Predictors of adolescent engagement and outcomes - A cross-sectional study using the togetherall (formerly Big White Wall) digital mental health platform

BACKGROUND

Online mental health platforms can improve access to, and use of, mental health support for young people who may find it difficult to engage with face-to-face delivery.

OBJECTIVE

We modelled predictors of engagement and symptom change in adolescent users of the Togetherall (formerly "Big White Wall") anonymous digital mental health peer-support platform.

METHODS

We report a retrospective analysis of longitudinal user data from UK 16-18 year Togetherall users, referred from mental health services (N = 606). Baseline demographics were reported for participants who logged anxiety and depression measures. Number of log-ins, mean session duration, total usage time, number of guided support courses and self-help materials accessed were our usage metrics. Participant characteristics and symptoms were used to predict engagement. For n = 245 users with symptom measures at >1 timepoint we modelled the effect of predictors on symptom scores.

RESULTS

Mean logins was 5.11 and mean usage time was 64.22 mins. Participants with one log-in represented 33.5% of the sample. Total time accessing Togetherall predicated greater usage of self-help materials and courses. Females made greater use of materials and courses than males. In a subsample, higher baseline depression and anxiety, longer total usage time and mean session duration predicted final depression scores, whereas higher baseline depression and anxiety and greater accessed self-help materials predicted lower final anxiety scores.

LIMITATIONS

A naturalistic design was used and symptom modelling should be interpreted with caution.

CONCLUSIONS

Findings suggest adolescents can engage with the Togetherall platform. Baseline symptoms and characteristics can inform user engagement with digital platforms.