Considerations of diversity, equity, and inclusion in mental health apps: A scoping review of evaluation frameworks

Analyzing metaverse-based digital therapies, their effectiveness, and potential risks in mental healthcare

The metaverse, defined as a collective virtual shared space created by the convergence of augmented reality (AR), virtual reality (VR), and the Internet, offers new opportunities for mental healthcare by delivering immersive and engaging digital therapies. This study examines the current landscape of metaverse-based mental healthcare applications, analyzing their effectiveness and potential risks. Using a systematic literature review (SLR) and case study research, four digital therapeutic applications-NightWare, Freespira, EndeavorRx, and Sleepio-were evaluated for their ability to address conditions such as PTSD, anxiety, and ADHD. The results indicate that metaverse-based therapies can provide significant benefits, with clinical validation supporting their effectiveness. However, concerns around user privacy, accessibility, and long-term efficacy remain challenges. Overall, metaverse-based digital therapies represent a promising shift in mental healthcare, offering innovative, personalized, and scalable solutions. Further research is needed to address ethical issues, improve accessibility, and confirm the long-term impact of these interventions.

Developing Digital Mental Health Tools With Culturally Diverse Parents and Young People: Qualitative User-Centered Design Study

BACKGROUND

Approximately 39% of young people (aged 16-24 y) experience mental ill health, but only 23% seek professional help. Early intervention is essential for reducing the impacts of mental illness, but young people, particularly those from culturally diverse communities, report experiencing shame and stigma, which can deter them from engaging with face-to-face services. Digital mental health (DMH) tools promise to increase access, but there is a lack of literature exploring the suitability of DMH tools for culturally diverse populations.

OBJECTIVE

The project was conducted in partnership with a large-scale national DMH organization that promotes evidence-based early intervention, treatment, and support of mental health in young people and their families. The organization wanted to develop a self-directed web-based platform for parents and young people that integrates psychological assessments and intervention pathways via a web-based "check-in" tool. Our project explored the views of culturally diverse parents and young people on the opportunities and barriers to engagement with a web-based DMH screening tool.

METHODS

We conducted a 2-phase qualitative study aiming to identify potential issues faced by culturally diverse communities when engaging with DMH tools designed for the Australian public. We worked with 18 culturally diverse participants (parents: n=8, 44%; young people: n=10, 56%) in a series of design-led workshops drawing on methods from speculative design and user experience to understand the opportunities and barriers that organizations might face when implementing population-level DMH tools with culturally diverse communities. NVivo was used to conduct thematic analyses of the audio-recorded and transcribed workshop data.

RESULTS

Five themes were constructed from the workshops: (1) trust in the use and application of a DMH tool, (2) data management and sharing, (3) sociocultural influences on mental health, (4) generational differences in mental health and digital literacy, and (5) stigma and culturally based discrimination in mental health support.

CONCLUSIONS

The emergent themes have important considerations for researchers wishing to develop more inclusive DMH tools. The study found that healthy parent-child relationships will increase engagement in mental health support for young persons from culturally diverse backgrounds. Barriers to engagement with DMH tools included culturally based discrimination, the influence of culture on mental health support, and the potential impact of a diagnostic label on help seeking. The study's findings suggest a need for culturally safe psychoeducation for culturally diverse end users that fosters self-determination with tailored resources. They also highlight important key challenges when working with culturally diverse populations.

Evaluating User Engagement With a Real-Time, Text-Based Digital Mental Health Support App: Cross-Sectional, Retrospective Study

BACKGROUND

Approximately 20% of US adults identify as having a mental illness. Structural and other barriers prevent many people from receiving mental health services. Digital mental health apps that provide 24-hour, real-time access to human support may improve access to mental health services. However, information is needed regarding how and why people engage with licensed counselors through a digital, real-time, text-based mental health support app in nonexperimental settings.

OBJECTIVE

This study aimed to evaluate how people engage with Counslr, a 24-hour, digital, mental health support app where users communicate in real time with human counselors through text messaging. Specifically, access patterns (eg, day of the week and time of session) and reasons for accessing the platform were examined. Furthermore, whether differences existed between session types (on-demand or scheduled) and membership types (education or noneducation) in regard to access patterns and why people accessed the platform were evaluated.

METHODS

The study population (users) consisted of students whose schools, universities, or colleges partnered with Counslr and employees whose organizations also partnered with Counslr. Users participated in text-based mental health support sessions. In these sessions, users engaged with licensed counselors through digital, text-based messaging in real time. Users could initiate an on-demand session or schedule a session 24 hours a day. User engagement patterns were evaluated through session length, session day, session time, and self-reported reasons for initiating the session. The data were stratified by membership type (education [students] or noneducation [employees]) and session type (on-demand or scheduled) to evaluate whether differences existed in usage patterns and self-reported reasons for initiating sessions by membership and session types.

RESULTS

Most students (178/283, 62.9%) and employees (28/44, 63.6%) accessed Counslr through on-demand sessions. The average and median session times were 40 (SD 15.3) and 45 minutes. On-demand sessions (37.9 minutes) were shorter (P=.001) than scheduled sessions (43.5 minutes). Most users (262/327, 80.1%) accessed Counslr between 7 PM and 5 AM. The hours that users accessed Counslr did not statistically differ by membership type (P=.19) or session type (P=.10). Primary self-reported reasons for accessing Counslr were relationship reasons, depression, and anxiety; however, users initiated sessions for a variety of reasons. Statistically significant differences existed between membership and session types (P<.05) for some of the reasons why people initiated sessions.

CONCLUSIONS

The novel findings of this study illustrate that real-time, digital mental health support apps, which offer people the opportunity to engage with licensed counselors outside of standard office hours for a variety of mental health conditions, may help address structural barriers to accessing mental health support services. Additional research is needed to evaluate the effectiveness of human-based apps such as Counslr and whether such apps can also address disparities in access to mental health support services among different demographic groups.

Developing a standardized framework for evaluating health apps using natural language processing

Despite regulatory efforts, many smartphone health applications remain unregulated, raising concerns about privacy, security, and evidence-based effectiveness. The lack of standardized regulation has led to the proliferation of over 130 frameworks, introducing new criteria and methodologies for app evaluation. The sheer number of frameworks, coupled with their varying approaches to app evaluation, create challenges for comparison. Our study aims to synthesize existing knowledge and propose a standardized app evaluation framework. We conducted a synthesis of reviews on health app evaluation frameworks. Using natural language processing (NLP), we analyzed evaluation domains and grouped them into clusters based on semantic similarities. Standardized definitions for these clusters were developed. We identified eight review articles that met the inclusion criteria, each proposing between six and 17 app evaluation domains. Using NLP, we identified five clusters of app evaluation: Effectiveness & Development, Technology & Functionality, Validity & Legal, Safety & Privacy, and Implementation & Ethics, each of which was assigned a standardized definition. The clusters align with but expand on the American Psychiatric Association's evaluation domains, incorporating critical aspects such as inclusivity, safety, engagement, and ethical principles. Temporal analysis revealed an increasing focus on Effectiveness & Development, while Safety & Privacy showed a stagnation in attention over time.

A scoping review of Spanish language pediatric digital health interventions

OBJECTIVE

Despite the growing number of Hispanic/Latino families in the United States, major concerns are reported when navigating the healthcare system. Monolingual Spanish-speaking families may experience compounded barriers given the inconsistent availability of Spanish resources and services in traditional healthcare settings. Digital health interventions have the potential to alleviate some barriers in healthcare for these individuals. This scoping review summarizes the state of the literature on the development, adaptation, and implementation of pediatric Spanish-language digital health interventions offered to Spanish-speaking families in the United States to better understand current cultural-sensitivity practices and strategies implemented by researchers.

METHODS

A search in major databases was completed in May 2024. Articles that discussed the development, implementation, or outcome of any digital health intervention primarily oriented to a Spanish-speaking pediatric population in the United States were included. Telephone- and telehealth-only interventions were excluded.

RESULTS

A total of 44 articles were reviewed, representing 30 unique digital health interventions. Most covered preventive health topics, utilized SMS texting, and were intended primarily for parents/caregivers. Only 22 articles discussed specific methods to culturally tailor the intervention. The most common methods implemented were advisory boards and collecting qualitative data from parents/caregivers and youth. About 50% of articles reported results related to efficacy, acceptability, and feasibility.

CONCLUSION

While similar methods are implemented to develop and adapt these interventions, there is ample variation throughout the process. Including and learning directly from intended users in the adaptation and development phases of digital health interventions can help create quality and culturally appropriate digital health programs for families.

Initial Requirements for the Prototyping of an App for a Psychosocial Rehabilitation Project: An Integrative Review

The Psychosocial Rehabilitation Project (PRP) is a tool designed to structure and organize mental health care, guided by the theoretical and practical principles of Psychosocial Rehabilitation (PR). This article aims to identify the initial requirements for the prototyping of a "Psychosocial Rehabilitation Project App". To achieve this, an integrative review was conducted with the research question: what initial requirements are important to compose the prototype of the "Psychosocial Rehabilitation Project App" in mental health? In the search process, 834 articles were identified and exported to the online systematic review application Rayyan QCRI, resulting in 36 eligible articles for this study, along with one app. The reading of this material allowed the elicitation of three themes: privacy and data protection policy; design; and software and programming. The prototyping of the "Psychosocial Rehabilitation Project App" should prioritize data security and protection, simplicity in design, and the integration of technological resources that facilitate the management, construction, monitoring, and evaluation of psychosocial rehabilitation projects by mental health professionals.

Pre-implementation determinants for digital mental health integration in Chicago pediatric primary care

OBJECTIVE

Pediatric primary care (PPC) is a common treatment site for pediatric mental health, but it is currently unable to meet the needs of all teen patients, particularly those with minoritized identities and/or marginalized experiences. Digital mental health (DMH) low-intensity treatments (LITs) can increase mental health screening and care capacity in PPC, but how this is done successfully without burdening providers, patients, or families is unclear. This paper presents a pre-implementation study aimed at understanding the implementation context (PPCs in Chicago, IL) for a specific DMH LIT.

METHOD

Using a mixed-methods design, quantitative data from an online survey of providers assessed current DMH practices in PPC, and qualitative interviews with Pediatricians and Pediatric Psychologists examined implementation determinants for a specific DMH LIT. Quantitative data were analyzed using descriptive statistics, and interviews were analyzed using rapid qualitative assessment.

RESULTS

Survey reports (n = 105) and interviews (n = 6) indicated low current use of DMH. Providers in PPC clinics voiced multiple reasons for low usage and low perceived feasibility, including: Consolidated Framework for Implementation Research (CFIR) Inner Setting Domain (PPC clinic workflow, responsibility and ethical considerations, patient privacy and confidentiality), CFIR Outer Setting Domain (hospital and healthcare system factors), CFIR Innovation Domain (DMH design), and a cross-cutting theme of safety.

CONCLUSIONS

Provider-reported low feasibility for integrating DMH in PPC is a call to action to partner with interdisciplinary colleagues and identify how such settings can ethically and seamlessly deliver digital evidence-based and accessible screening and care prior to implementation.

Real-World Adoption of Mental Health Support Among Adolescents: Cross-Sectional Analysis of the California Health Interview Survey

OBJECTIVE

We aim to examine: (a) the extent to which patterns of adoption of counseling services and digital mental health interventions (DMHIs) shifted in recent years (2019-2021); (b) the impact of distress on adoption of mental health support; and (c) reasons related to adolescents' low adoption of DMHIs when experiencing distress.

METHODS

Data were from three cohorts of adolescents aged 12-17 years (n = 847 in 2019; n = 1,365 in 2020; n = 1,169 in 2021) recruited as part of the California Health Interview Survey. We estimated logistic regression models to examine the likelihood of using mental health support as a function of psychological distress, sociodemographic characteristics, and cohorts. We also analyzed adolescents' self-reported reasons for not trying DMHIs as a function of distress.

RESULTS

The proportion of adolescents reporting elevated psychological distress (∼50%) was higher than those adopting counseling services (<20%) or DMHIs (<10%). A higher level of distress was associated with a greater likelihood of receiving counseling (OR = 1.15), and using DMHIs to connect with a professional (Odds ratio (OR) = 1.11) and for self-help (OR = 1.17). Among those experiencing high distress, adolescents' top reason for not adopting an online tool was a lack of perceived need (19.2%).

CONCLUSION

Adolescents' main barriers to DMHI adoption included a lack of perceived need, which may be explained by a lack of mental health literacy. Thoughtful marketing and dissemination efforts are needed to increase mental health awareness and normalize adoption of counseling services and DMHIs.

State of the Science: Using Digital Mental Health Interventions to Extend the Impact of Psychological Services

In recent years, digital mental health interventions (DMHIs) have emerged as a paradigm shift in care delivery that could expand the scale, efficiency, and effectiveness of psychological services. However, DMHI impact is constrained by issues related to limited reach, poor adoption, implementation barriers, and insufficient long-term maintenance. Organized by the Reach, Effectiveness, Adoption, Implementation, and Maintenance(RE-AIM) framework, this paper surveys the current state of DMHIs, highlighting research and practice gaps as well as potential strategies to move the field forward. Similarly, we discuss the role that emerging technologies and changes in the profession will play in shaping DMHIs in years to come. Finally, concrete and actionable steps to advance equity in the DMHI field are provided, with an emphasis on strategies to increase the representativeness of marginalized populations in DMHI research, the inclusion of these groups in the design and testing of DMHIs, and how to improve the contextual and cultural fit of DMHIs.

Centering equity, diversity, and inclusion in youth digital mental health: findings from a research, policy, and practice knowledge exchange workshop

Background

Youth mental health service organizations continue to rapidly broaden their use of virtual care and digital mental health interventions as well as leverage artificial intelligence and other technologies to inform care decisions. However, many of these digital services have failed to alleviate persistent mental health disparities among equity-seeking populations and in some instances have exacerbated them. Transdisciplinary and intersectional knowledge exchange is greatly needed to address structural barriers to digital mental health engagement, develop and evaluate interventions with historically underserved communities, and ultimately promote more accessible, useful, and equitable care.

Methods

To that end, the Digital, Inclusive, Virtual, and Equitable Research Training in Mental Health Platform (DIVERT), the Maritime Strategy for Patient Oriented Research (SPOR) SUPPORT (Support for People and Patient-Oriented Research and Trials) Unit and IWK Mental Health Program invited researchers, policymakers, interprofessional mental health practitioners, trainees, computer scientists, health system administrators, community leaders and youth advocates to participate in a knowledge exchange workshop. The workshop aimed to (a) highlight local research and innovation in youth-focused digital mental health services; (b) learn more about current policy and practice issues in inclusive digital mental health for youth in Canada, (c) participate in generating action recommendations to address challenges to inclusive, diverse and equitable digital mental health services, and (d) to synthesize cross-sector feedback to inform future training curriculum, policy, strategic planning and to stimulate new lines of patient-oriented research.

Results

Eleven challenge themes emerged related to white-colonial normativity, lack of cultural humility, inaccessibility and affordability of participating in the digital world, lack of youth and community involvement, risks of too much digital time in youth's lives, and lack of scientific evidence derived from equity-deserving communities. Nine action recommendations focused on diversifying research and development funding, policy and standards, youth and community led promotion, long-term trust-building and collaboration, and needing to callout and advocate against unsafe digital services and processes.

Conclusion

Key policy, training and practice implications are discussed.

Digital Health Readiness: Making Digital Health Care More Inclusive

This paper proposes an approach to assess digital health readiness in clinical settings to understand how prepared, experienced, and equipped individual people are to participate in digital health activities. Existing digital health literacy and telehealth prediction tools exist but do not assess technological aptitude for particular tasks or incorporate available electronic health record data to improve efficiency and efficacy. As such, we propose a multidomain digital health readiness assessment that incorporates a person's stated goals and motivations for use of digital health, a focused digital health literacy assessment, passively collected data from the electronic health record, and a focused aptitude assessment for critical skills needed to achieve a person's goals. This combination of elements should allow for easy integration into clinical workflows and make the assessment as actionable as possible for health care providers and in-clinic digital health navigators. Digital health readiness profiles could be used to match individuals with support interventions to promote the use of digital tools like telehealth, mobile apps, and remote monitoring, especially for those who are motivated but do not have adequate experience. Moreover, while effective and holistic digital health readiness assessments could contribute to increased use and greater equity in digital health engagement, they must also be designed with inclusivity in mind to avoid worsening known disparities in digital health care.

Equity in Digital Mental Health Interventions in the United States: Where to Next?

Health care technologies have the ability to bridge or hinder equitable care. Advocates of digital mental health interventions (DMHIs) report that such technologies are poised to reduce the documented gross health care inequities that have plagued generations of people seeking care in the United States. This is due to a multitude of factors such as their potential to revolutionize access; mitigate logistical barriers to in-person mental health care; and leverage patient inputs to formulate tailored, responsive, and personalized experiences. Although we agree with the potential of DMHIs to advance health equity, we articulate several steps essential to mobilize and sustain meaningful forward progression in this endeavor, reflecting on decades of research and learnings drawn from multiple fields of expertise and real-world experience. First, DMHI manufacturers must build diversity, equity, inclusion, and belonging (DEIB) processes into the full spectrum of product evolution itself (eg, product design, evidence generation) as well as into the fabric of internal company practices (eg, talent recruitment, communication principles, and advisory boards). Second, awareness of the DEIB efforts-or lack thereof-in DMHI research trials is needed to refine and optimize future study design for inclusivity as well as proactively address potential barriers to doing so. Trials should incorporate thoughtful, inclusive, and creative approaches to recruitment, enrollment, and measurement of social determinants of health and self-identity, as well as a prioritization of planned and exploratory analyses examining outcomes across various groups of people. Third, mental health care advocacy, research funding policies, and local and federal legislation can advance these pursuits, with directives from the US Preventive Services Taskforce, National Institutes of Health, and Food and Drug Administration applied as poignant examples. For products with artificial intelligence/machine learning, maintaining a "human in the loop" as well as prespecified and adaptive analytic frameworks to monitor and remediate potential algorithmic bias can reduce the risk of increasing inequity. Last, but certainly not least, is a call for partnership and transparency within and across ecosystems (academic, industry, payer, provider, regulatory agencies, and value-based care organizations) to reliably build health equity into real-world DMHI product deployments and evidence-generation strategies. All these considerations should also extend into the context of an equity-informed commercial strategy for DMHI manufacturers and health care organizations alike. The potential to advance health equity in innovation with DMHI is apparent. We advocate the field's thoughtful and evergreen advancement in inclusivity, thereby redefining the mental health care experience for this generation and those to come.

Is There a Place for Cognitive Restructuring in Brief, Self-Guided Interventions? Randomized Controlled Trial of a Single-Session, Digital Program for Adolescents

OBJECTIVE

Self-guided digital mental health interventions (DMHIs) teaching empirically supported skills (e.g. behavioral activation) have demonstrated efficacy for improving youth mental health, but we lack evidence for the complex skill of cognitive restructuring (CR).

METHOD

We conducted the first-ever RCT testing a CR DMHI ("Project Think") against an active control (supportive therapy; "Project Share") in collaboration with public schools. Pre-registered outcomes were DMHI acceptability and helpfulness post-intervention, as well as internalizing symptoms and CR skills use from baseline to seven-month follow-up, in the full sample and the subsample with elevated symptoms.

RESULTS

Participants (N = 597; MAge = 11.99; 48% female; 68% White) rated both programs highly on acceptability and helpfulness. Both conditions were associated with significant internalizing symptom reductions across time in both samples, with no significant condition differences. CR skills use declined significantly across time for Project Share youths but held steady across time for Project Think youths in both samples; this pattern produced a significant condition difference favoring Project Think within the elevated sample at seven-month follow-up.

CONCLUSION

Internalizing symptoms declined comparably for Think and Share participants. Consequently, future research should examine whether encouraging youths to share their feelings produces symptom improvements, and whether a single-session, self-guided CR DMHI produces beneficial effects relative to more inert control conditions. Further, the decline in CR skills use for Project Share youths versus sustained CR use by Project Think youths raises questions about the natural time course of youths' CR use and the impact of these DMHIs on that course. ClinicalTrials.gov Registration: NCT04806321.

The Acceptability, Engagement, and Feasibility of Mental Health Apps for Marginalized and Underserved Young People: Systematic Review and Qualitative Study

BACKGROUND

Smartphone apps may provide an opportunity to deliver mental health resources and interventions in a scalable and cost-effective manner. However, young people from marginalized and underserved groups face numerous and unique challenges to accessing, engaging with, and benefiting from these apps.

OBJECTIVE

This study aims to better understand the acceptability (ie, perceived usefulness and satisfaction with an app) and feasibility (ie, the extent to which an app was successfully used) of mental health apps for underserved young people. A secondary aim was to establish whether adaptations can be made to increase the accessibility and inclusivity of apps for these groups.

METHODS

We conducted 2 sequential studies, consisting of a systematic literature review of mental health apps for underserved populations followed by a qualitative study with underserved young male participants (n=20; age: mean 19). Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, an electronic search of 5 databases was conducted in 2021. The search yielded 18,687 results, of which 14 articles met the eligibility criteria.

RESULTS

The included studies comprised a range of groups, including those affected by homelessness, having physical health conditions, living in low- and middle-income countries, and those with sexual and gender minority identities. Establishing and maintaining user engagement was a pervasive challenge across mental health apps and populations, and dropout was a reported problem among nearly all the included studies. Positive subjective reports of usability, satisfaction, and acceptability were insufficient to determine users' objective engagement.

CONCLUSIONS

Despite the significant amount of funding directed to the development of mental health apps, juxtaposed with only limited empirical evidence to support their effectiveness, few apps have been deliberately developed or adapted to meet the heterogeneous needs of marginalized and underserved young people. Before mental health apps are scaled up, a greater understanding is needed of the types of services that more at-risk young people and those in limited-resource settings prefer (eg, standard vs digital) followed by more rigorous and consistent demonstrations of acceptability, effectiveness, and cost-effectiveness. Adopting an iterative participatory approach by involving young people in the development and evaluation process is an essential step in enhancing the adoption of any intervention, including apps, in "real-world" settings and will support future implementation and sustainability efforts to ensure that marginalized and underserved groups are reached.

TRIAL REGISTRATION

PROSPERO CRD42021254241; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=254241.

Advancing equitable access to digital mental health in the Asia-Pacific region in the context of the COVID-19 pandemic and beyond: A modified Delphi consensus study

The COVID-19 pandemic had an unprecedented impact on global mental health and well-being, including across the Asia-Pacific. Efforts to mitigate virus spread led to far-reaching disruption in the delivery of health and social services. In response, there was a rapid shift to the use of digital mental health (DMH) approaches. Though these technologies helped to improve access to care for many, there was also substantial risk of access barriers leading to increased inequities in access to mental health care, particularly among at-risk and equity-deserving populations. The objective of this study was to conduct a needs assessment and identify priorities related to equitable DMH access among at-risk and equity-deserving populations in the Asia Pacific region during the first year of the COVID-19 pandemic. The study consisted of a modified Delphi consensus methodology including two rounds of online surveys and online consultations with stakeholders from across the region. Study participants included policy makers, clinicians and service providers, and people with lived experience of mental health conditions. Results demonstrate that vulnerabilities to negative mental health impacts and access barriers were compounded during the pandemic. Access barriers included a lack of linguistically and culturally appropriate DMH options, low mental health literacy and poor access to technological infrastructure and devices, low levels of awareness and trust of DMH options, and lack of policies and guidelines to support effective and equitable delivery of DMH. Recommendations to improve equitable access include ensuring that diverse people with lived experience are engaged in research, co-design and policy development, the development and implementation of evidence-based and equity-informed guidelines and frameworks, clear communication about DMH evidence and availability, and the integration of DMH into broader health systems. Study results can inform the development and implementation of equitable DMH as its use becomes more widespread across health systems.

Facilitators of and barriers to County Behavioral Health System Transformation and Innovation: an interview study

BACKGROUND

Inadequate and inequitable access to quality behavioral health services and high costs within the mental health systems are long-standing problems. System-level (e.g., fee-for-service payment model, lack of a universal payor) and individual factors (e.g., lack of knowledge of existing resources) contribute to difficulties in accessing resources and services. Patients are underserved in County behavioral health systems in the United States. Orange County's (California) Behavioral Health System Transformation project sought to improve access by addressing two parts of their system: developing a template for value-based contracts that promote payor-agnostic care (Part 1); developing a digital platform to support resource navigation (Part 2). Our aim was to evaluate facilitators of and barriers to each of these system changes.

METHODS

We collected interview data from County or health care agency leaders, contracted partners, and community stakeholders. Themes were informed by the Consolidated Framework for Implementation Research.

RESULTS

Five themes were identified related to behavioral health system transformation, including 1) aligning goals and values, 2) addressing fit, 3) fostering engagement and partnership, 4) being aware of implementation contexts, and 5) promoting communication. A lack of fit into incentive structures and changing state guidelines and priorities were barriers to contract development. Involving diverse communities to inform design and content facilitated the process of developing digital tools.

CONCLUSIONS

The study highlights the multifaceted factors that help facilitate or hinder behavioral health system transformation, such as the need for addressing systematic and process behaviors, leveraging the knowledge of leadership and community stakeholders, fostering collaboration, and adapting to implementation contexts.

Diversity, equity and inclusion considerations in mental health apps for young people: protocol for a scoping review

INTRODUCTION

After COVID-19, a global mental health crisis affects young people, with one in five youth experiencing mental health problems worldwide. Delivering mental health interventions via mobile devices is a promising strategy to address the treatment gap. Mental health apps are effective for adolescent and young adult samples, but face challenges such as low real-world reach and under-representation of minoritised youth. To increase digital health uptake, including among minoritised youth, there is a need for diversity, equity and inclusion (DEI) considerations in the development and evaluation of mental health apps. How well DEI is integrated into youth mental health apps has not been comprehensively assessed. This scoping review aims to examine to what extent DEI considerations are integrated into the design and evaluation of youth mental health apps and report on youth, caregiver and other stakeholder involvement.

METHODS AND ANALYSIS

We will identify studies published in English from 2009 to 29 September 2023 on apps for mental health in youth. We will use PubMed, Global Health, APA PsycINFO, SCOPUS, CINAHL PLUS and the Cochrane Database and will report according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Scoping Review Extension guidelines. Papers eligible for inclusion must be peer-reviewed publications in English involving smartphone applications used by adolescents or young adults aged 10-25, with a focus on depression, anxiety or suicidal ideation. Two independent reviewers will review and extract articles using a template developed by the authors. We will analyse the data using narrative synthesis and descriptive statistics. This study will identify gaps in the literature and provide a roadmap for equitable and inclusive mental health apps for youth.

ETHICS AND DISSEMINATION

Ethics approval is not required. Findings will be disseminated through academic, industry, community networks and scientific publications.

Race and Socioeconomic Status as Predictors of Willingness to Use Digital Mental Health Interventions or One-On-One Psychotherapy: National Survey Study

BACKGROUND

There is an ongoing debate about whether digital mental health interventions (DMHIs) can reduce racial and socioeconomic inequities in access to mental health care. A key factor in this debate involves the extent to which racial and ethnic minoritized individuals and socioeconomically disadvantaged individuals are willing to use, and pay for, DMHIs.

OBJECTIVE

This study examined racial and ethnic as well as socioeconomic differences in participants' willingness to pay for DMHIs versus one-on-one therapy (1:1 therapy).

METHODS

We conducted a national survey of people in the United States (N=423; women: n=204; mean age 45.15, SD 16.19 years; non-Hispanic White: n=293) through Prolific. After reading descriptions of DMHIs and 1:1 therapy, participants rated their willingness to use each treatment (1) for free, (2) for a small fee, (3) as a maximum dollar amount, and (4) as a percentage of their total monthly income. At the end of the study, there was a decision task to potentially receive more information about DMHIs and 1:1 therapy.

RESULTS

Race and ethnicity was associated with willingness to pay more of one's income, as a percent or in dollar amounts, and was also associated with information-seeking for DMHIs in the behavioral task. For most outcomes, race and ethnicity was not associated with willingness to try 1:1 therapy. Greater educational attainment was associated to willingness to try DMHIs for free, the decision to learn more about DMHIs, and willingness to pay for 1:1 therapy. Income was inconsistently associated to willingness to try DMHIs or 1:1 therapy.

CONCLUSIONS

If they are available for free or at very low costs, DMHIs may reduce inequities by expanding access to mental health care for racial and ethnic minoritized individuals and economically disadvantaged groups.

Modeling Long-Term Budgetary Impacts of Prevention: An Overview of Meta-analyses of Relationships Between Key Health Outcomes Across the Life-Course

Budget analysis entities often cannot capture the full downstream impacts of investments in prevention services, programs, and interventions. This study describes and applies an approach to synthesizing existing literature to more fully account for these effects. This study reviewed meta-analyses in PubMed published between Jan 1, 2010 and Dec 31, 2019. The initial search included meta-analyses on the association between health risk factors, including maternal behavioral health, intimate partner violence, child maltreatment, depression, and obesity, with a later health condition. Through a snowball sampling-type approach, the endpoints of the meta-analyses identified became search terms for a subsequent search, until each health risk was connected to one of the ten costliest health conditions. These results were synthesized to create a path model connecting the health risks to the high-cost health conditions in a cascade. Thirty-seven meta-analyses were included. They connected early-life health risk factors with six high-cost health conditions: hypertension, diabetes, asthma and chronic obstructive pulmonary disorder, mental disorders, heart conditions, and trauma-related disorders. If confounders could be controlled for and causality inferred, the cascading associations could be used to more fully account for downstream impacts of preventive interventions. This would support budget analysis entities to better include potential savings from investments in chronic disease prevention and promote greater implementation at scale.

Assessing the Influence of Patient Empowerment Gained Through Mental Health Apps on Patient Trust in the Health Care Provider and Patient Compliance With the Recommended Treatment: Cross-sectional Study

BACKGROUND

In chronic mental illness, noncompliance with treatment significantly worsens the illness course and outcomes for patients. Considering that nearly 1 billion people worldwide experience mental health issues, including 1 of 5 Canadians in any given year, finding tools to lower noncompliance in these populations is critical for health care systems. A promising avenue is apps that make mental health services more accessible to patients. However, little is known regarding the impact of the empowerment gained from mental health apps on patient compliance with recommended treatment.

OBJECTIVE

This study aimed to investigate the impact of patient empowerment gained through mental health apps on patient trust in the health care provider and patient compliance with the recommended treatment.

METHODS

A cross-sectional web-based survey was conducted in Canada. Eligible participants were Canadian adults diagnosed with chronic mental health disorders who were using at least one of the following apps: Dialogue, MindBeacon, Deprexis, Ginger, Talkspace, BetterHelp, MindStrong, Mindshift, Bloom, Headspace, and Calm. A total of 347 valid questionnaires were collected and analyzed using partial least-squares structural equation modeling. Trust in the health care provider and patient compliance were measured with multiple-item scales adapted from existing scales. Patient empowerment was conceived and measured as a higher-order construct encompassing the following 2 dimensions: patient process and patient outcome. All the items contributing to the constructs in the model were measured with 7-point Likert scales. The reliability and validity of the measurement model were assessed, and the path coefficients of the structural model were estimated.

RESULTS

The results clearly show that patient empowerment gained through mental health apps positively influenced patient trust in the health care provider (β=.306; P<.001). Patient trust in the health care provider had a positive effect on patient compliance (β=.725; P<.001). The direct relationship between patient empowerment and patient compliance was not significant (β=.061, P=.23). Interestingly, the data highlight that the effect of patient empowerment on patient compliance was fully mediated by trust in the health care provider (β=.222; P<.001). The results show that patient empowerment gained through the mental health app involves 2 dimensions: a process and an outcome.

CONCLUSIONS

This study shows that for individuals living with mental health disorders, empowerment gained through mental health apps enhances trust in the health care provider. It reveals that patient empowerment impacts patient compliance but only through the full mediating effect of patient trust in the health care provider, indicating that patient trust is a critical variable to enhance patient compliance. Hence, our results confirm that health care systems could encourage the use of mental health apps to favor a climate that facilitates patients' trust in health care provider recommendations, possibly leading to better compliance with the recommended treatment.

Mental Health and Well-Being in Racial or Ethnic Minority Individuals After Using a Faith and Prayer Mobile App (Pray.com): Feasibility and Preliminary Efficacy Trial

BACKGROUND

Research is needed on how faith and prayer apps fit within the values of racial and ethnic minority (REM) groups, as well as whether such apps are effective in promoting mental health and well-being.

OBJECTIVE

This study aims to determine the feasibility and preliminary effectiveness of using the mobile app Pray.com on mental health and well-being among REM participants.

METHODS

This study was a single-group (N=77), 4-week feasibility trial in REM groups (65/77, 84% Black or African American). Participants were asked to use the Pray.com app at no cost for at least 5 times per week for 5 minutes per day. Participants completed questionnaires at the baseline and postintervention time points. Feasibility questionnaires were only completed at the postintervention time point, including qualitative interviews (n=15). The feasibility questions included acceptability (ie, satisfaction, intent to continue use, perceived appropriateness, and fit within culture), demand (ie, self-reported app use, expressed interest, and perceived demand), and practicality (ie, ease or difficulty of use, ability to use the app, and cost-effectiveness). Frequency and descriptive statistics were used to analyze feasibility outcomes. Changes in dependent variables were analyzed using paired-sample 2-tailed t tests. Partial correlations were conducted to explore the association between app use and outcomes, controlling for baseline scores.

RESULTS

Participants reported (54/72, 75% responded with "very likely" or "likely" to the feasibility questions) that they perceived the Pray.com app as acceptable. These findings were supported by qualitative interviews (n=15). Most participants (62/72, 86%) did not meet the app use prescription but expressed interest in using the app in the future and perceived demand for it in their communities. In addition, participants reported that the app was easy to use and perceived it to be inexpensive (US $7.99). Participants reported improved mental health (ie, stress and depressive and anxiety symptoms) and well-being (ie, satisfaction with life, spiritual well-being, religious commitment, and racial or ethnic identity development) at postintervention despite relatively low average levels and high variability of app use (average total of 45.83, SD 111.90 min over the course of the study). Greater app use was significantly associated with improvements in mental health and spiritual well-being. However, app use and study methodology limitations suggest that the study results may not accurately capture the full impact of Pray.com use.

CONCLUSIONS

This is the first study to assess the feasibility of a faith and prayer app for mental health and well-being in a sample of REM individuals. Our findings suggest that the use of a faith and prayer app (ie, Pray.com) could be feasible and significantly impactful for the improvement of mental health symptoms and well-being in REM individuals and their communities, especially Black and African American individuals with a Christian affiliation. Further research is warranted.

Race, Ethnicity, and Other Cultural Background Factors in Trials of Internet-Based Cognitive Behavioral Therapy for Depression: Systematic Review

BACKGROUND

There is a growing interest in developing scalable interventions, including internet-based cognitive behavioral therapy (iCBT), to meet the increasing demand for mental health services. Given the growth in diversity worldwide, it is essential that the clinical trials of iCBT for depression include diverse samples or, at least, report information on the race, ethnicity, or other background indicators of their samples. Unfortunately, the field lacks data on how well diversity is currently reported and represented in the iCBT literature.

OBJECTIVE

Thus, the main objective of this systematic review was to examine the overall reporting of racial and ethnic identities in published clinical trials of iCBT for depression. We also aimed to review the representation of specific racial and ethnic minoritized groups and the inclusion of alternative background indicators such as migration status or country of residence.

METHODS

Studies were included if they were randomized controlled trials in which iCBT was compared to a waiting list, care-as-usual, active control, or another iCBT. The included papers also had to have a focus on acute treatment (eg, 4 weeks to 6 months) of depression, be delivered via the internet on a website or a smartphone app and use guided or unguided self-help. Studies were initially identified from the METAPSY database (n=59) and then extended to include papers up to 2022, with papers retrieved from Embase, PubMed, PsycINFO, and Cochrane (n=3). Risk of bias assessment suggested that reported studies had at least some risk of bias due to use of self-report outcome measures.

RESULTS

A total of 62 iCBT randomized controlled trials representing 17,210 participants are summarized in this study. Out of those 62 papers, only 17 (27%) of the trials reported race, and only 12 (19%) reported ethnicity. Reporting outside of the United States was very poor, with the United States accounting for 15 (88%) out of 17 of studies that reported race and 9 (75%) out of 12 for ethnicity. Out of 3,623 participants whose race was reported in the systematic review, the racial category reported the most was White (n=2716, 74.9%), followed by Asian (n=209, 5.8%) and Black (n=274, 7.6%). Furthermore, only 25 (54%) out of the 46 papers conducted outside of the United States reported other background demographics.

CONCLUSIONS

It is important to note that the underreporting observed in this study does not necessarily indicate an underrepresentation in the actual study population. However, these findings highlight the poor reporting of race and ethnicity in iCBT trials for depression found in the literature. This lack of diversity reporting may have significant implications for the scalability of these interventions.

Longitudinal outcomes of a therapist-supported digital mental health intervention for depression and anxiety symptoms: A retrospective cohort study

PURPOSE

This study examined treatment outcomes (depression and anxiety symptoms) up to 24 months after completion of a therapist-supported digital mental health intervention (DMHI).

METHODS

The sample consisted of 380 participants who participated in an eight-week DMHI from February 6, 2017 to May 20, 2019. Participants reported depression and anxiety symptoms at eight timepoints from baseline to 24 months. Mixed-effects modelling was used to investigate symptom changes over time. The proportion of participants meeting criteria for treatment response, clinically significant change, and remission of depression and anxiety symptoms were calculated, including proportions demonstrating each outcome sustained up to each timepoint.

RESULTS

Multivariate analyses yielded statistically significant reductions in depression (β = -5.40) and anxiety (β = -3.31) symptoms from baseline to end of treatment (8 weeks). Symptom levels remained significantly reduced from baseline through 24 months. The proportion of participants meeting criteria for clinical treatment outcomes remained constant over 24 months, although there were linear decreases in the proportions experiencing sustained clinical outcomes.

CONCLUSIONS

Treatment gains were made for depression and anxiety symptoms at the end of treatment and up to 24 months. Future studies should determine the feasibility of integrating post-treatment programmes into DMHIs to address symptom deterioration.

Digital therapeutics in the hospital for suicide crisis - content and design recommendations from young people and hospital staff

Objective

Hospital emergency departments lack the resources to adequately support young people who present for suicidal crisis. Digital therapeutics could fill this service gap by providing psychological support without creating additional burden on hospital staff. However, existing research on what is needed for successful integration of digital therapeutics in hospital settings is scant. Thus, this study sought to identify key considerations for implementing digital therapeutics to manage acute suicidal distress in hospitals.

Method

Participants were 17 young people who recently presented at the hospital for suicide-related crisis, and 12 hospital staff who regularly interacted with young people experiencing mental ill-health in their day-to-day work. Interviews were conducted via videoconference. Framework analysis and reflexive thematic analysis were used to interpret the data obtained.

Results

Qualitative insights were centred around three major themes: hospital-specific content, therapeutic content, and usability. Digital therapeutics were seen as a useful means for facilitating hospital-based assessment and treatment planning, and for conducting post-discharge check-ins. Therapeutic content should be focused on helping young people self-manage suicide-related distress while they wait for in-person services. Features to promote usability, such as the availability of customisable features and the use of inclusive design or language, should be considered in the design of digital therapeutics.

Conclusions

Digital therapeutics in hospital settings need to benefit both patients and staff. Given the unique context of the hospital setting and acute nature of suicidal distress, creating specialty digital therapeutics may be more viable than integrating existing ones.

Exploring Generation Z and Young Millennials' Perspectives of a Spiritual Self-Care App and Their Spiritual Identity (Skylight): Qualitative Semistructured Interview Study

BACKGROUND

Generation Z and young millennials (ages 18-35 years), collectively referred to as GenZennials, are connected to technology and the internet like no other generation before them. This has mental health implications, such as increased rates of anxiety and stress. Recent research has shown that app-based mental health interventions can be useful to address such mental health concerns. However, spirituality is an untapped resource, especially since GenZennials largely identify as spiritual and already integrate spiritual practices into their self-care.

OBJECTIVE

There were four objectives to this study: (1) comprehensively explore reasons why GenZennials use a spiritual self-care app (ie, Skylight; Radiant Foundation), (2) understand how GenZennials identify spiritually, (3) understand the app's relevance to GenZennials, and (4) gather feedback and suggestions to improve the app.

METHODS

Semistructured interviews were conducted with 23 GenZennials (ages 18-35 years; mean 28.7, SD 5.0 years; n=20, 87% female) who used the Skylight app. Interviews were 30 to 60 minutes and conducted on Zoom. Thematic analysis was used to analyze interviews.

RESULTS

Five major categories emerged from the analysis, each encompassing one to several themes: (1) reasons for using the Skylight app, (2) content favorites, (3) defining spiritual identity, (4) relevance to GenZennials, and (5) overall improvement recommendations. Participants used the app for various reasons including to relax, escape, or ground themselves; improve mood; and enhance overall health and wellness. Participants also cited the app's variety of content offerings and its free accessibility as their primary reasons for using it. Most participants identified themselves as solely spiritual (8/23/35%) among the options provided (ie, spiritual or religious or both), and they appreciated the app's inclusive content. Participants felt that the app was relevant to their generation as it offered modern content (eg, spiritual self-care activities and short content). Participants recommended adding more personalization capabilities, content, and representation to the app.

CONCLUSIONS

This is the first study to qualitatively explore GenZennials' perspectives and the use of a spiritual self-care app. Our findings should inform the future creation and improvement of spiritual self-care apps aimed at cultivating GenZennials' spiritual and mental well-being. Future research is warranted to examine the effects of using a spiritual self-care app on GenZennial mental health.

Self-Guided Mental Health Apps Targeting Racial and Ethnic Minority Groups: Scoping Review

BACKGROUND

The use of mental health apps (MHAs) is increasing rapidly. However, little is known about the use of MHAs by racial and ethnic minority groups.

OBJECTIVE

In this review, we aimed to examine the acceptability and effectiveness of MHAs among racial and ethnic minority groups, describe the purposes of using MHAs, identify the barriers to MHA use in racial and ethnic minority groups, and identify the gaps in the literature.

METHODS

A systematic search was conducted on August 25, 2023, using Web of Science, Embase, PsycINFO, PsycArticles, PsycExtra, and MEDLINE. Articles were quality appraised using the Mixed Methods Appraisal Tool, and data were extracted and summarized to form a narrative synthesis.

RESULTS

A total of 15 studies met the inclusion criteria. Studies were primarily conducted in the United States, and the MHAs designed for racial and ethnic minority groups included ¡Aptívate!, iBobbly, AIMhi- Y, BRAVE, Build Your Own Theme Song, Mindful You, Sanadak, and 12 more MHAs used in 1 study. The MHAs were predominantly informed by cognitive behavioral therapy and focused on reducing depressive symptoms. MHAs were considered acceptable for racial and ethnic minority groups; however, engagement rates dropped over time. Only 2 studies quantitatively reported the effectiveness of MHAs among racial and ethnic minority groups. Barriers to use included the repetitiveness of the MHAs, stigma, lack of personalization, and technical issues.

CONCLUSIONS

Considering the growing interest in MHAs, the available evidence for MHAs for racial and ethnic minority groups appears limited. Although the acceptability seems consistent, more research is needed to support the effectiveness of MHAs. Future research should also prioritize studies to explore the specific needs of racial and ethnic minority groups if MHAs are to be successfully adopted.

A Digital Mental Health Support Program for Depression and Anxiety in Populations With Attention-Deficit/Hyperactivity Disorder: Feasibility and Usability Study

BACKGROUND

A total of 1 in 2 adults with attention-deficit/hyperactivity disorder (ADHD) struggles with major depressive or anxiety disorders. The co-occurrence of these disorders adds to the complexity of finding utility in as well as adherence to a treatment option. Digital therapeutic solutions may present a promising alternative treatment option that could mitigate these challenges and alleviate symptoms.

OBJECTIVE

This study aims to investigate (1) the feasibility and acceptance of a digital mental health intervention, (2) participants' engagement and retention levels, and (3) the potential efficacy with respect to anxiety and depression symptoms in a population with ADHD. Our main hypothesis was that a digital, data-driven, and personalized intervention for adults with coexisting ADHD and depressive or anxiety symptoms would show high engagement and adherence, which would be accompanied by a decrease in depressive and anxiety symptoms along with an increase in quality of life and life satisfaction levels.

METHODS

This real-world data, single-arm study included 30 adult participants with ADHD symptomatology and coexisting depressive or anxiety symptoms who joined a 16-week digital, data-driven mental health support program. This intervention is based on a combination of evidence-based approaches such as cognitive behavioral therapy, mindfulness, and positive psychology techniques. The targeted symptomatology was evaluated using the Patient Health Questionnaire-9, Generalized Anxiety Disorder-7, and Barkley Adult ADHD Rating Scale-IV. Quality of life aspects were evaluated using the Satisfaction With Life Scale and the Life Satisfaction Questionnaire, and user feedback surveys were used to assess user experience and acceptability.

RESULTS

The study retention rate was 97% (29/30), and high engagement levels were observed, as depicted by the 69 minutes spent on the app per week, 5 emotion logs per week, and 11.5 mental health actions per week. An average decrease of 46.2% (P<.001; r=0.89) in depressive symptoms and 46.4% (P<.001; r=0.86) in anxiety symptoms was observed, with clinically significant improvement for more than half (17/30, 57% and 18/30, 60%, respectively) of the participants. This was followed by an average increase of 23% (P<.001; r=0.78) and 20% (P=.003; r=0.8) in Satisfaction With Life Scale and Life Satisfaction Questionnaire scores, respectively. The overall participant satisfaction level was 4.3 out of 5.

CONCLUSIONS

The findings support the feasibility, acceptability, and value of the examined digital program for adults with ADHD symptomatology to address the coexisting depressive or anxiety symptoms. However, controlled trials with larger sample sizes and more diverse participant profiles are required to provide further evidence of clinical efficacy.

Cultural Responsivity in Technology-Enabled Services: Integrating Culture Into Technology and Service Components

Technology-enabled services (TESs) are clinical interventions that combine technological and human components to provide health services. TESs for mental health are efficacious in the treatment of anxiety and depression and are currently being offered as frontline treatments around the world. It is hoped that these interventions will be able to reach diverse populations across a range of identities and ultimately decrease disparities in mental health treatment. However, this hope is largely unrealized. TESs include both technology and human service components, and we argue that cultural responsivity must be considered in each of these components to help address existing treatment disparities. To date, there is limited guidance on how to consider cultural responsivity within these interventions, including specific targets for the development, tailoring, or design of the technologies and services within TESs. In response, we propose a framework that provides specific recommendations for targets based on existing models, both at the technological component level (informed by the Behavioral Intervention Technology Model) and the human support level (informed by the Efficiency Model of Support). We hope that integrating culturally responsive considerations into these existing models will facilitate increased attention to cultural responsivity within TESs to ensure they are ethical and responsive for everyone.

A Call to Action on Assessing and Mitigating Bias in Artificial Intelligence Applications for Mental Health

Advances in computer science and data-analytic methods are driving a new era in mental health research and application. Artificial intelligence (AI) technologies hold the potential to enhance the assessment, diagnosis, and treatment of people experiencing mental health problems and to increase the reach and impact of mental health care. However, AI applications will not mitigate mental health disparities if they are built from historical data that reflect underlying social biases and inequities. AI models biased against sensitive classes could reinforce and even perpetuate existing inequities if these models create legacies that differentially impact who is diagnosed and treated, and how effectively. The current article reviews the health-equity implications of applying AI to mental health problems, outlines state-of-the-art methods for assessing and mitigating algorithmic bias, and presents a call to action to guide the development of fair-aware AI in psychological science.

Longitudinal Patterns of Engagement and Clinical Outcomes: Results From a Therapist-Supported Digital Mental Health Intervention

OBJECTIVE

Digital mental health interventions (DMHIs) are an effective treatment modality for common mental disorders like depression and anxiety; however, the role of intervention engagement as a longitudinal "dosing" factor is poorly understood in relation to clinical outcomes.

METHODS

We studied 4978 participants in a 12-week therapist-supported DMHI (June 2020-December 2021), applying a longitudinal agglomerative hierarchical cluster analysis to the number of days per week of intervention engagement. The proportion of people demonstrating remission in depression and anxiety symptoms during the intervention was calculated for each cluster. Multivariable logistic regression models were fit to examine associations between the engagement clusters and symptom remission, adjusting for demographic and clinical characteristics.

RESULTS

Based on clinical interpretability and stopping rules, four clusters were derived from the hierarchical cluster analysis (in descending order): a) sustained high engagers (45.0%), b) late disengagers (24.1%), c) early disengagers (22.5%), and d) immediate disengagers (8.4%). Bivariate and multivariate analyses supported a dose-response relationship between engagement and depression symptom remission, whereas the pattern was partially evident for anxiety symptom remission. In multivariable logistic regression models, older age groups, male participants, and Asians had increased odds of achieving depression and anxiety symptom remission, whereas higher odds of anxiety symptom remission were observed among gender-expansive individuals.

CONCLUSIONS

Segmentation based on the frequency of engagement performs well in discerning timing of intervention disengagement and a dose-response relationship with clinical outcomes. The findings among the demographic subpopulations indicate that therapist-supported DMHIs may be effective in addressing mental health problems among patients who disproportionately experience stigma and structural barriers to care. Machine learning models can enable precision care by delineating how heterogeneous patterns of engagement over time relate to clinical outcomes. This empirical identification may help clinicians personalize and optimize interventions to prevent premature disengagement.

Increasing digital mental health reach and uptake via youth partnerships

Youth in the United States are facing an unprecedented mental health crisis. Yet, brick-and-mortar mental healthcare, such as face-to-face therapy, is overwhelmingly inaccessible to youth despite research advances in youth mental health. Digital Mental Health tools (DMH), the use of technologies to deliver mental health assessments and interventions, may help to increase mental healthcare accessibility. However, for a variety of reasons, evidence-based DMH have not been successful in reaching youth in real-world settings, particularly those who are most encumbered with access barriers to mental healthcare. This Comment therefore focuses on increasing DMH reach and uptake by young people, particularly among minoritized youth, by engaging in community-based youth partnerships. This idea recognizes and grows from decades' worth of community-based participatory research and youth partnerships successfully conducted by other disciplines (e.g., social work, public health, urban planning, education). Increasing uptake and engagement is an issue that is unlikely to be solved by adult-driven theory and design. As such, we emphasize the necessity of reframing youth input into DMH design and deployment from one-time participants to integral community-based partners. Indeed, recognizing and valuing their expertise to equitably address DMH implementation challenges, youth should help to pose the very questions that they will help to answer throughout the design and implementation planning for DMH moving forward.

Effectiveness and implementation of a text messaging intervention to reduce depression and anxiety symptoms among Latinx and Non-Latinx white users during the COVID-19 pandemic

Text messaging interventions are increasingly used to help people manage depression and anxiety. However, little is known about the effectiveness and implementation of these interventions among U.S. Latinxs, who often face barriers to using mental health tools. The StayWell at Home (StayWell) intervention, a 60-day text messaging program based on cognitive behavioral therapy (CBT), was developed to help adults cope with depressive and anxiety symptoms during the COVID-19 pandemic. StayWell users (n = 398) received daily mood inquiries and automated skills-based text messages delivering CBT-informed coping strategies from an investigator-generated message bank. We conduct a Hybrid Type 1 mixed-methods study to compare the effectiveness and implementation of StayWell for Latinx and Non-Latinx White (NLW) adults using the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. Effectiveness was measured using the PHQ-8 depression and GAD-7 anxiety scales, assessed before starting and after completing StayWell. Guided by RE-AIM, we conducted a thematic text analysis of responses to an open-ended question about user experiences to help contextualize quantitative findings. Approximately 65.8% (n = 262) of StayWell users completed pre-and-post surveys. On average, depressive (-1.48, p = 0.001) and anxiety (-1.38, p = 0.001) symptoms decreased from pre-to-post StayWell. Compared to NLW users (n = 192), Latinx users (n = 70) reported an additional -1.45 point (p < 0.05) decline in depressive symptoms, adjusting for demographics. Although Latinxs reported StayWell as relatively less useable (76.8 vs. 83.9, p = 0.001) than NLWs, they were more interested in continuing the program (7.5 vs. 6.2 out of 10, p = 0.001) and recommending it to a family member/friend (7.8 vs. 7.0 out of 10, p = 0.01). Based on the thematic analysis, both Latinx and NLW users enjoyed responding to mood inquiries and sought bi-directional, personalized text messages and texts with links to more information to resources. Only NLW users stated that StayWell provided no new information than they already knew from therapy or other sources. In contrast, Latinx users suggested that engagement with a behavioral provider through text or support groups would be beneficial, highlighting this group's unmet need for behavioral health care. mHealth interventions like StayWell are well-positioned to address population-level disparities by serving those with the greatest unmet needs if they are culturally adapted and actively disseminated to marginalized groups. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT04473599.

Methods for Navigating the Mobile Mental Health App Landscape for Clinical Use

There are over 10,000 mental health and wellness apps on the market. Apps offer the opportunity to increase access to mental health care. However, with many apps to choose from and an app landscape that is largely unregulated, it can be difficult to incorporate this technology into clinical practice. The first step towards achieving this goal is to identify clinically relevant and appropriate apps. The purpose of this review is to discuss app evaluation, raise awareness of considerations involved in implementing mental health apps into clinical care, and provide an example of how apps can be used effectively in the clinical space. We discuss the current regulatory environment for health apps, how to evaluate apps, and implement them into clinical practice. We also showcase a digital clinic where apps are integrated into the clinical workflow and discuss barriers to app implementation. Mental health apps have the potential to increase access to care if they are clinically effective, easy to use, and protect patient privacy. Learning how to find, evaluate, and implement quality apps into practice is key in harnessing this technology for the benefit of patients.

Facilitators of and Barriers to Integrating Digital Mental Health Into County Mental Health Services: Qualitative Interview Analyses

BACKGROUND

Digital mental health interventions (DMHIs) represent a promising solution to address the growing unmet mental health needs and increase access to care. Integrating DMHIs into clinical and community settings is challenging and complex. Frameworks that explore a wide range of factors, such as the Exploration, Preparation, Implementation, Sustainment (EPIS) framework, can be useful for examining multilevel factors related to DMHI implementation efforts.

OBJECTIVE

This paper aimed to identify the barriers to, facilitators of, and best practice recommendations for implementing DMHIs across similar organizational settings, according to the EPIS domains of inner context, outer context, innovation factors, and bridging factors.

METHODS

This study stems from a large state-funded project in which 6 county behavioral health departments in California explored the use of DMHIs as part of county mental health services. Our team conducted interviews with clinical staff, peer support specialists, county leaders, project leaders, and clinic leaders using a semistructured interview guide. The development of the semistructured interview guide was informed by expert input regarding relevant inner context, outer context, innovation factors, and bridging factors in the exploration, preparation, and implementation phases of the EPIS framework. We followed a recursive 6-step process to conduct qualitative analyses using inductive and deductive components guided by the EPIS framework.

RESULTS

On the basis of 69 interviews, we identified 3 main themes that aligned with the EPIS framework: readiness of individuals, readiness of innovations, and readiness of organizations and systems. Individual-level readiness referred to the extent to which clients had the necessary technological tools (eg, smartphones) and knowledge (digital literacy) to support the DMHI. Innovation-level readiness pertained to the accessibility, usefulness, safety, and fit of the DMHI. Organization- and system-level readiness concerned the extent to which providers and leadership collectively held positive views about DMHIs as well as the extent to which infrastructure (eg, staffing and payment model) was appropriate.

CONCLUSIONS

The successful implementation of DMHIs requires readiness at the individual, innovation, and organization and system levels. To improve individual-level readiness, we recommend equitable device distribution and digital literacy training. To improve innovation readiness, we recommend making DMHIs easier to use and introduce, clinically useful, and safe and adapting them to fit into the existing client needs and clinical workflow. To improve organization- and system-level readiness, we recommend supporting providers and local behavioral health departments with adequate technology and training and exploring potential system transformations (eg, integrated care model). Conceptualizing DMHIs as services allows the consideration of both the innovation characteristics of DMHIs (eg, efficacy, safety, and clinical usefulness) and the ecosystem around DMHIs, such as individual and organizational characteristics (inner context), purveyors and intermediaries (bridging factor), client characteristics (outer context), as well as the fit between the innovation and implementation settings (innovation factor).

Clinical Considerations for Digital Resources in Care for Patients With Suicidal Ideation

Smartphone apps offer accessible new tools that may help prevent suicide and that offer support for individuals with active suicidal ideation. Numerous smartphone apps for mental health conditions exist; however, their functionality is limited, and evidence is nascent. A new generation of apps using smartphone sensors and integrating real-time data on evolving risk offers the potential of more personalized support, but these apps present ethical risks and currently remain more in the research domain than in the clinical domain. Nevertheless, clinicians can use apps to benefit patients. This article outlines practical strategies to select safe and effective apps for the creation of a digital toolkit that can augment suicide prevention and safety plans. By creating a unique digital toolkit for each patient, clinicians can help ensure that the apps selected will be most relevant, engaging, and effective.

Is the uptake, engagement, and effectiveness of exclusively mobile interventions for the promotion of weight-related behaviors equal for all? A systematic review

Mobile health interventions are promising behavior change tools. However, there is a concern that they may benefit some populations less than others and thus widen inequalities in health. This systematic review investigated differences in uptake of, engagement with, and effectiveness of mobile interventions for weight-related behaviors (i.e., diet, physical activity, and sedentary behavior) based on a range of inequality indicators including age, gender, race/ethnicity, and socioeconomic status. The protocol was registered on PROSPERO (CRD42020192473). Six databases (CINAHL, EMBASE, ProQuest, PsycINFO, Pubmed, and Web of Science) were searched from inception to July 2021. Publications were eligible for inclusion if they reported the results of an exclusively mobile intervention and examined outcomes by at least one inequality indicator. Sixteen publications reporting on 13 studies were included with most reporting on multiple behaviors and inequality indicators. Uptake was investigated in one study with no differences reported by the inequality indicators studied. Studies investigating engagement (n = 7) reported differences by age (n = 1), gender (n = 3), ethnicity (n = 2), and education (n = 2), while those investigating effectiveness (n = 9) reported differences by age (n = 3), gender (n = 5), education (n = 2), occupation (n = 1), and geographical location (n = 1). Given the limited number of studies and their inconsistent findings, evidence of the presence of a digital divide in mobile interventions targeting weight-related behaviors is inconclusive. Therefore, we recommend that inequality indicators are specifically addressed, analyzed, and reported when evaluating mobile interventions.

A Digital Single-Session Intervention Platform for Youth Mental Health: Cultural Adaptation, Evaluation, and Dissemination

BACKGROUND

Despite the proliferation of evidence-based digital mental health programs for young people, their low uptake and inconsistent implementation preclude them from benefiting youths at scale. Identifying effective implementation strategies for evidence-based supports is especially critical in regions where treatment access is lowest owing to mental health provider shortages.

OBJECTIVE

The goal of this academic-community partnership, funded by the City of San Antonio Metropolitan Health District, was to culturally adapt, disseminate, and gauge the acceptability and utility of an evidence-based digital mental health platform-Project Youth Empowerment and Support (YES)-among English- and Spanish-speaking youths living in south Texas.

METHODS

Project YES is an open-access, anonymous platform containing 3 evidence-based, self-guided interventions for youth mental health. Project YES was culturally adapted via focus groups and co-design sessions with San Antonio youths with lived experience of depression and anxiety; translated into Spanish; and disseminated throughout San Antonio, Texas, via community and school partnerships.

RESULTS

During the project period (April 2021 to December 2021), 1801 San Antonio youths began and 894 (49.64%) of them completed a 30-minute, single-session intervention within Project YES (aged 11-17 years; n=718, 39.87% male; n=961, 53.36% female; and n=3, 0.17% intersex; n=1477, 82.01% Hispanic; n=77, 4.28% non-Hispanic White; n=113, 6.27% Black; n=28, 1.55% Asian; and n=93, 5.16% other). This completion rate (49.64%) surpassed those previously observed for Project YES (eg, 34% when disseminated via social media). San Antonio youths rated Project YES as highly acceptable across all metrics, both in English and Spanish. In addition, the youths who completed Project YES-ENGLISH reported significant improvements in hopelessness (Cohen d=0.33; P<001), self-hate (Cohen d=0.27; P<001), and perceived agency (Cohen d=0.25; P<001) from before to after the intervention, and the youths who completed Project YES-SPANISH reported significant improvements in self-hate (Cohen d=0.37; P=.049) from before to after the intervention.

CONCLUSIONS

The results indicate that Project YES-an open-access, free, and anonymous web-based single-session intervention platform-is an acceptable, accessible, and applicable mental health support for English- and Spanish-speaking San Antonio youths.

Assessing the Dynamics of the Mental Health Apple and Android App Marketplaces

Barriers to mental health care, including stigma, costs, and mental health professional shortages, have been exacerbated by the COVID-19 pandemic. Smartphone apps have the potential to increase scalability and improve access to mental health information, support, and interventions. However, evaluating these apps and selecting ones for use in care remain challenging, especially as apps are often updating and changing. Recommending apps requires knowledge of how stable apps are as the experience of one user several months ago may or may not be the same. A sample of 347 apps of the 650 apps on the M-health Index and Navigation Database (MIND) https://mindapps.org were reviewed between September 1, 2021, and January 5, 2022. Apps were selected by time since their last review, with updates occurring on average approximately 4 months from the last review. Eleven trained app evaluators reviewed apps across 105 evaluation criteria in 9 categories. Results were compared to initial ratings, identifying the changes that occurred. The average app updates every 433 days, though 19% were updated in the last 3 months and some nearly weekly. Changes in privacy and features made up the highest percentage of changes, both at 38%. The most frequently observed privacy-related change was increased privacy policy reading level. Functionality parameters changed in 28% of apps. The most common functionality change was the removal of an accessibility feature. Clinical foundations changed in 18% of apps and 9% added supporting studies. Cost structure changed in 17% of apps, with 10% adding a fee for use of the app. Engagement features changed in 17% of the apps, with additions and removals of validated assessments or screeners most common. The dynamic nature of the app stores is reflected in app privacy, features, and functionality. These changes, reflected by the increased reading levels required to understand privacy policies, the decrease in accessibility features, and the additions of fees to access mobile apps, reflect the need to constantly review apps and understand how they are evolving. Patient and clinicians should use the most recent and updated possible when evaluating apps.

Examining Equity in Access and Utilization of a Freely Available Meditation App

Digital interventions have the potential to alleviate mental health disparities for marginalized and minoritized communities. The current study examined whether disparities in access and utilization of meditation in the United States (US) were reduced for a freely available meditation app. We analyzed demographic and usage data from US-based users of the Healthy Minds Program (HMP; N = 66,482) between October 2019 and July 2022. College education was associated with a greater likelihood of accessing (65.0% of users vs. 32.9% of the US population) and continuing to utilize the app (β = .11-.17). Conversely, identifying as African American was associated lower likelihood of accessing (5.3% vs. 13.4% of US population) and continuing to utilize the app (β = -.02-.03). African Americans were more likely to access content from an African American meditation teacher, but this did not appear to increase utilization. Additional efforts are warranted to identify factors which might reduce disparities.

Current challenges for evaluating mobile health applications

As mobile health applications continue to proliferate without clear regulation, the need for app evaluation frameworks to offer guidance to patients and clinicians also expands. However, this expanding number of app evaluation frameworks itself can be a source of confusion and often contradictory recommendations. In pursuit of better frameworks that offer innovation for app evaluation, we present 4 challenges that app evaluation frameworks must overcome as well as examples from our own experience toward overcoming them. The recommendations are applicable to all health apps from any field of medicine, although we use examples from mental health as they are illustrative.

Assessment of Mental Health Services Available Through Smartphone Apps

IMPORTANCE

As more patients and clinicians are turning to mental health smartphone apps to expand access to services, little is known about the current state of the app marketplaces and what these apps are actually offering in terms of features, privacy, price, and services.

OBJECTIVE

To assess the current state of mental health apps, explore the association between app privacy scores and popularity as measured by star ratings and downloads, and to understand opportunities and challenges facing the commercial app landscape.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study had trained raters using the public-facing M-Health Index and Navigation Database (MIND) to assess and review 578 mental health apps. The sample of apps used in this analysis were pulled from MIND and include apps across various conditions including schizophrenia, eating disorders, sleep, and more. Analysis of these apps was conducted in June 2022.

EXPOSURES

There were 578 mental health apps rated across 105 dimensions derived from the American Psychiatric Association's app evaluation framework.

MAIN OUTCOMES AND MEASURES

App raters assessed each app across 6 categories: (1) app origin and accessibility, (2) privacy and security, (3) clinical foundation, (4) features and engagement, (5) inputs and outputs, and (6) interoperability. Privacy scores were determined by 5 MIND criteria, including (1) having a privacy policy, (2) reporting security measures in place, (3) declaring data use and purpose, (4) allowing for the deletion of data, and (5) allowing users to opt out of data collection. Correlations between privacy scores and popularity metrics (star ratings and number of downloads) were measured.

RESULTS

This study included 578 mental health apps that were identified, assessed, and analyzed across 105 MIND dimensions. Psychoeducation, goal setting, and mindfulness were among the top app features. Of the 578 apps analyzed, 443 (77%) had a privacy policy. This analysis of apps with a privacy policy revealed that there was no statistically significant correlation between privacy scores and Apple App Store (r = 0.058, P = .29) or Google Play Store star ratings (r = 0.041; P = .48). The number of app downloads on the Google Play Store, however, was weakly correlated with privacy scores (χ25 = 22.1; P < .001).

CONCLUSIONS AND RELEVANCE

In this cross-sectional study of mental health apps, findings indicate that the current app marketplaces primarily offered basic features such as psychoeducation, goal tracking, and mindfulness but fewer innovative features such as biofeedback or specialized therapies. Privacy challenges remained common, and app popularity metrics provided little help in identifying apps with more privacy.

Determination of Markers of Successful Implementation of Mental Health Apps for Young People: Systematic Review

Background

Smartphone apps have the potential to address some of the current issues facing service provision for young people’s mental health by improving the scalability of evidence-based mental health interventions. However, very few apps have been successfully implemented, and consensus on implementation measurement is lacking.

Objective

This review aims to determine the proportion of evidence-based mental health and well-being apps that have been successfully adopted and sustained in real-world settings. A secondary aim is to establish if key implementation determinants such as coproduction, acceptability, feasibility, appropriateness, and engagement contribute toward successful implementation and longevity.

Methods

Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, an electronic search of 5 databases in 2021 yielded 18,660 results. After full-text screening, 34 articles met the full eligibility criteria, providing data on 29 smartphone apps studied with individuals aged 15 to 25 years.

Results

Of 34 studies, only 10 (29%) studies were identified that were evaluating the effectiveness of 8 existing, commercially available mental health apps, and the remaining 24 (71%) studies reported the development and evaluation of 21 newly developed apps, of which 43% (9/21) were available, commercially or otherwise (eg, in mental health services), at the time of enquiry. Most studies addressed some implementation components including adoption, acceptability, appropriateness, feasibility, and engagement. Factors including high cost, funding constraints, and lengthy research processes impeded implementation.

Conclusions

Without addressing common implementation drivers, there is considerable redundancy in the translation of mobile mental health research findings into practice. Studies should embed implementation strategies from the outset of the planned research, build collaborations with partners already working in the field (academic and commercial) to capitalize on existing interventions and platforms, and modify and evaluate them for local contexts or target problems and populations.

Trial Registration

PROSPERO CRD42021224365; https://tinyurl.com/4umpn85f

Participant retention in a fully remote trial of digital psychotherapy: Comparison of incentive types

Numerous studies have found that long term retention is very low in remote clinical studies (&gt;4 weeks) and to date there is limited information on the best methods to ensure retention. The ability to retain participants in the completion of key assessments periods is critical to all clinical research, and to date little is known as to what methods are best to encourage participant retention. To study incentive-based retention methods we randomized 215 US adults (18+ years) who agreed to participate in a sequential, multiple assignment randomized trial to either high monetary incentive (HMI, $125 USD) and combined low monetary incentive ($75 USD) plus alternative incentive (LMAI). Participants were asked to complete daily and weekly surveys for a total of 12 weeks, which included a tailoring assessment around week 5 to determine who should be stepped up and rerandomized to one of two augmentation conditions. Key assessment points were weeks 5 and 12. There was no difference in participant retention at week 5 (tailoring event), with approximately 75% of the sample completing the week-5 survey. By week 10, the HMI condition retained approximately 70% of the sample, compared to 60% of the LMAI group. By week 12, all differences were attenuated. Differences in completed measures were not significant between groups. At the end of the study, participants were asked the impressions of the incentive condition they were assigned and asked for suggestions for improving engagement. There were no significant differences between conditions on ratings of the fairness of compensation, study satisfaction, or study burden, but study burden, intrinsic motivation and incentive fairness did influence participation. Men were also more likely to drop out of the study than women. Qualitative analysis from both groups found the following engagement suggestions: desire for feedback on survey responses and an interest in automated sharing of individual survey responses with study therapists to assist in treatment. Participants in the LMAI arm indicated that the alternative incentives were engaging and motivating. In sum, while we were able to increase engagement above what is typical for such study, more research is needed to truly improve long term retention in remote trials.

The Promise of Digital Mental Health for LGBTQ+ Youths

Lesbian, gay, bisexual, transgender, queer, plus (LGBTQ+) youths face increased risk for psychiatric morbidity, and the stress of being in a minority group drives this disparity. Affirmative treatments can improve mental health among LGBTQ+ youths, but barriers are encountered in accessing care. Digital mental health offers the opportunity to increase access, therefore potentially reducing mental health disparities. However, insufficient attention has been devoted to addressing the needs of LGBTQ+ youths through digital tools. In this column, the authors outline strategies to improve the inclusiveness of existing digital mental health content and to move toward equitable care with the development of new content.

Developing an mHealth App for Empowering Cancer Survivors With Disabilities: Co-design Study

BACKGROUND

The transition from active treatment to long-term cancer survivorship leaves the needs of many cancer survivors unaddressed as they struggle with physical, cognitive, psychological, and social consequences of cancer and its treatment. The lack of guidance after treatment has forced cancer survivors to manage long-term effects on their own, which has an impact on their overall health, quality of life, and social participation. Mobile health (mHealth) interventions can be used to promote self-management and evidence-informed education.

OBJECTIVE

This study aims to design an mHealth app for cancer survivors with disabilities that will offer interventions to improve their quality of life and increase their self-efficacy to manage cancer as a chronic condition.

METHODS

We organized 3 co-design workshops with cancer survivors (n=5). These workshops included persona development based on data from 25 interviews with cancer survivors with disabilities; prototype ideation, where we sketched ideas for the prototype; and prototype development, where participants critiqued, and suggested improvements for, the wireframes.

RESULTS

These workshops helped us to define the challenges that cancer survivors with disabilities face as well as important considerations when designing an mHealth app for cancer survivors with disabilities, such as the need for including flexibility, engagement, socialization, and a minimalistic design. We also outline guidelines for other researchers to follow when planning their own co-design workshops, which include allowing more time for discussion among participants, having small participant groups, keeping workshops engaging and inclusive, and letting participants dream big.

CONCLUSIONS

Using a co-design process aided us in developing a prototype of an mHealth app for cancer survivors with disabilities as well as a list of guidelines that other researchers can use to develop their own co-design workshops and design their app. Furthermore, working together with cancer survivors ensured that the design team had a deeper sense of empathy toward the target users and kept the focus on our ultimate goal: creating something that cancer survivors would want to use and benefit from. Future work will include usability testing of a high-fidelity prototype based on the results of these workshops.

Digital Intervention Barriers Scale–7 (DIBS-7): Development, Evaluation, and Preliminary Validation (Preprint)

BACKGROUND

The translation of mental health services into digital formats, deemed digital mental health interventions (DMHIs), has the potential to address long-standing obstacles to accessing care. However, DMHIs have barriers of their own that impact enrollment, adherence, and attrition in these programs. Unlike in traditional face-to-face therapy, there is a paucity of standardized and validated measures of barriers in DMHIs.

OBJECTIVE

In this study, we describe the preliminary development and evaluation of such a scale, the Digital Intervention Barriers Scale-7 (DIBS-7).

METHODS

Following an iterative QUAN → QUAL mixed methods approach, item generation was guided by qualitative analysis of feedback from participants (n=259) who completed a DMHI trial for anxiety and depression and identified barriers related to self-motivation, ease of use, acceptability, and comprehension of tasks. Item refinement was achieved through DMHI expert review. A final item pool was administered to 559 treatment completers (mean age 23.02 years; 438/559, 78.4% female; 374/559, 69.9% racially or ethnically minoritized). Exploratory factor analyses and confirmatory factor analyses were estimated to determine the psychometric properties of the measure. Finally, criterion-related validity was examined by estimating partial correlations between the DIBS-7 mean score and constructs related to treatment engagement in DMHIs.

RESULTS

Statistical analyses estimated a 7-item unidimensional scale with high internal consistency (α=.82, ω=0.89). Preliminary criterion-related validity was supported by significant partial correlations between the DIBS-7 mean score and treatment expectations (pr=-0.25), number of modules with activity (pr=-0.55), number of weekly check-ins (pr=-0.28), and treatment satisfaction (pr=-0.71).

CONCLUSIONS

Overall, these results provide preliminary support for the use of the DIBS-7 as a potentially useful short scale for clinicians and researchers interested in measuring an important variable often associated with treatment adherence and outcomes in DMHIs.