Exploring how informed mental health app selection may impact user engagement and satisfaction

The evolving field of digital mental health: current evidence and implementation issues for smartphone apps, generative artificial intelligence, and virtual reality

The expanding domain of digital mental health is transitioning beyond traditional telehealth to incorporate smartphone apps, virtual reality, and generative artificial intelligence, including large language models. While industry setbacks and methodological critiques have highlighted gaps in evidence and challenges in scaling these technologies, emerging solutions rooted in co-design, rigorous evaluation, and implementation science offer promising pathways forward. This paper underscores the dual necessity of advancing the scientific foundations of digital mental health and increasing its real-world applicability through five themes. First, we discuss recent technological advances in digital phenotyping, virtual reality, and generative artificial intelligence. Progress in this latter area, specifically designed to create new outputs such as conversations and images, holds unique potential for the mental health field. Given the spread of smartphone apps, we then evaluate the evidence supporting their utility across various mental health contexts, including well-being, depression, anxiety, schizophrenia, eating disorders, and substance use disorders. This broad view of the field highlights the need for a new generation of more rigorous, placebo-controlled, and real-world studies. We subsequently explore engagement challenges that hamper all digital mental health tools, and propose solutions, including human support, digital navigators, just-in-time adaptive interventions, and personalized approaches. We then analyze implementation issues, emphasizing clinician engagement, service integration, and scalable delivery models. We finally consider the need to ensure that innovations work for all people and thus can bridge digital health disparities, reviewing the evidence on tailoring digital tools for historically marginalized populations and low- and middle-income countries. Regarding digital mental health innovations as tools to augment and extend care, we conclude that smartphone apps, virtual reality, and large language models can positively impact mental health care if deployed correctly.

Developing a standardized framework for evaluating health apps using natural language processing

Despite regulatory efforts, many smartphone health applications remain unregulated, raising concerns about privacy, security, and evidence-based effectiveness. The lack of standardized regulation has led to the proliferation of over 130 frameworks, introducing new criteria and methodologies for app evaluation. The sheer number of frameworks, coupled with their varying approaches to app evaluation, create challenges for comparison. Our study aims to synthesize existing knowledge and propose a standardized app evaluation framework. We conducted a synthesis of reviews on health app evaluation frameworks. Using natural language processing (NLP), we analyzed evaluation domains and grouped them into clusters based on semantic similarities. Standardized definitions for these clusters were developed. We identified eight review articles that met the inclusion criteria, each proposing between six and 17 app evaluation domains. Using NLP, we identified five clusters of app evaluation: Effectiveness & Development, Technology & Functionality, Validity & Legal, Safety & Privacy, and Implementation & Ethics, each of which was assigned a standardized definition. The clusters align with but expand on the American Psychiatric Association's evaluation domains, incorporating critical aspects such as inclusivity, safety, engagement, and ethical principles. Temporal analysis revealed an increasing focus on Effectiveness & Development, while Safety & Privacy showed a stagnation in attention over time.

Digital Interventions for People Waitlisted for Mental Health Services: A Needs Assessment and Preference Survey

This study aimed to characterize the needs and preferences for digital health of individuals waitlisted for services. 135 people who sought mental health services in the past year and who were not receiving the service they sought completed questionnaires assessing their experiences during this period as well as their preferences related to digital interventions. Participants had sought services on average nearly eight months prior to completing the survey (M = 7.74 months, SD = 8.51). most commonly for anxiety (N = 118, 87.4%) and depression (N = 101, 74.8%). Relative to the date they first sought services, participants reported reduced interest (d = -0.45) and motivation to engage (d = -0.43) in treatment, as well as reduced conviction in the belief that they would get better (d = -0.23) or that treatment would help (d = -0.30), although they also reported reduced symptom distress (d = -0.39). Very few (8%) reported satisfaction with the materials they were given by providers when seeking services. Participants reported high interest in a digital tool providing information about therapies (94.7% reporting moderate or greater interest), mental illnesses (87.4%), and places to find treatment (91.1%), as well as one that would provide interactive cognitive practices (85.2%) and strategies to improve sleep (85.8%). Over half of all participants reported that a mobile app would be their first choice for mode of delivery. This study highlights the need for and potential of digital tools optimized for waitlisted individuals, particularly interactive and informational resources provided through mobile devices.

The Trifecta of Industry, Academic, and Health System Partnership to Improve Mental Health Care Through Smartphone-Based Remote Patient Monitoring: Development and Usability Study

BACKGROUND

Mental health treatment is hindered by the limited number of mental health care providers and the infrequency of care. Digital mental health technology can help supplement treatment by remotely monitoring patient symptoms and predicting mental health crises in between clinical visits. However, the feasibility of digital mental health technologies has not yet been sufficiently explored. Rhythms, from the company Health Rhythms, is a smartphone platform that uses passively acquired smartphone data with artificial intelligence and predictive analytics to alert patients and providers to an emerging mental health crisis.

OBJECTIVE

The objective of this study was to test the feasibility and acceptability of Rhythms among patients attending an academic psychiatric outpatient clinic.

METHODS

Our group embedded Rhythms into the electronic health record of a large health system. Patients with a diagnosis of major depressive disorder, bipolar disorder, or other mood disorder were contacted online and enrolled for a 6-week trial of Rhythms. Participants provided data by completing electronic surveys as well as by active and passive use of Rhythms. Emergent and urgent alerts were monitored and managed according to passively collected data and patient self-ratings. A purposively sampled group of participants also participated in qualitative interviews about their experience with Rhythms at the end of the study.

RESULTS

Of the 104 participants, 89 (85.6%) completed 6 weeks of monitoring. The majority of the participants were women (72/104, 69.2%), White (84/104, 80.8%), and non-Hispanic (100/104, 96.2%) and had a diagnosis of major depressive disorder (71/104, 68.3%). Two emergent alerts and 19 urgent alerts were received and managed according to protocol over 16 weeks. More than two-thirds (63/87, 72%) of those participating continued to use Rhythms after study completion. Comments from participants indicated appreciation for greater self-awareness and provider connection, while providers reported that Rhythms provided a more nuanced understanding of patient experience between clinical visits.

CONCLUSIONS

Rhythms is a user-friendly, electronic health record-adaptable, smartphone-based tool that provides patients and providers with a greater understanding of patient mental health status. Integration of Rhythms into health systems has the potential to facilitate mental health care and improve the experience of both patients and providers.

Validation of a Questionnaire to Assess the Usability of and User Experience with Mobile Health Applications

BACKGROUND/OBJECTIVES

The growing use of mobile health (mHealth) applications needs reliable tools to assess their usability and user experience in clinical practice to improve the digital health (eHealth) interventions and ensure engagement, as higher engagement is often linked to increased efficacy of healthcare interventions. This study aimed to validate the patient Satisfaction and Usability with APPs questionnaire (pSUAPP), a multidimensional tool designed for the comprehensive assessment of mHealth applications, particularly for the integrated follow-up of patients with chronic diseases.

METHODS

A validation study was conducted between August and December 2022 with 85 participants from two hospitals in Spain, who completed the pSUAPP questionnaire, comprising 27 Likert-like items across four dimensions: first contact, registration, features and overall experience, and 1 open question. The questionnaire was validated by a panel of 11 experts and further assessed for psychometric properties.

RESULTS

The mean pSUAPP score was 79.0 (SD = 12.0), indicating high usability and positive user experience, with the highest scores in the 'features' dimension. The pSUAPP demonstrated moderate correlation with the System Usability Scale (SUS) and high reliability (Cronbach's alpha and omega t > 0.9). A reproducibility analysis showed negligible changes between repeated measures.

CONCLUSIONS

The pSUAPP questionnaire was found to be a robust tool for evaluating mHealth app usability and user experience, with potential application across various clinical settings.

Assessing Digital Phenotyping for App Recommendations and Sustained Engagement: Cohort Study

BACKGROUND

Low engagement with mental health apps continues to limit their impact. New approaches to help match patients to the right app may increase engagement by ensuring the app they are using is best suited to their mental health needs.

OBJECTIVE

This study aims to pilot how digital phenotyping, using data from smartphone sensors to infer symptom, behavioral, and functional outcomes, could be used to match people to mental health apps and potentially increase engagement.

METHODS

After 1 week of collecting digital phenotyping data with the mindLAMP app (Beth Israel Deaconess Medical Center), participants were randomly assigned to the digital phenotyping arm, receiving feedback and recommendations based on those data to select 1 of 4 predetermined mental health apps (related to mood, anxiety, sleep, and fitness), or the control arm, selecting the same apps but without any feedback or recommendations. All participants used their selected app for 4 weeks with numerous metrics of engagement recorded, including objective screentime measures, self-reported engagement measures, and Digital Working Alliance Inventory scores.

RESULTS

A total of 82 participants enrolled in the study; 17 (21%) dropped out of the digital phenotyping arm and 18 (22%) dropped out from the control arm. Across both groups, few participants chose or were recommended the insomnia or fitness app. The majority (39/47, 83%) used a depression or anxiety app. Engagement as measured by objective screen time and Digital Working Alliance Inventory scores were higher in the digital phenotyping arm. There was no correlation between self-reported and objective metrics of app use. Qualitative results highlighted the importance of habit formation in sustained app use.

CONCLUSIONS

The results suggest that digital phenotyping app recommendation is feasible and may increase engagement. This approach is generalizable to other apps beyond the 4 apps selected for use in this pilot, and practical for real-world use given that the study was conducted without any compensation or external incentives that may have biased results. Advances in digital phenotyping will likely make this method of app recommendation more personalized and thus of even greater interest.

Secure Messaging Use Among Patients with Depression: An Analysis Using Real-World Data

Background: Although depression is one of the most common mental health disorders outpacing other diseases and conditions, poor access to care and limited resources leave many untreated. Secure messaging (SM) offers patients an online means to bridge this gap by communicating nonurgent medical questions. We focused on self-care health management behaviors and delved into SM initiation as the initial act of engagement and SM exchanges as continuous engagement patterns. This study examined whether those with depression might be using SM more than those without depression. Methods: Patient portal data were obtained from a large academic medical center's electronic health records spanning 5 years, from January 2018 to December 2022. We organized and analyzed SM initiations and exchanges using the linear mixed-effects modeling technique. Results: Our predictors correlated with SM initiations, accounting for 25.1% of variance explained. In parallel, 24.9% of SM exchanges were attributable to these predictors. Overall, our predictors demonstrate stronger associations with SM exchanges. Discussion: We examined patients with and without depression across 2,629 zip codes over five years. Our findings reveal that the predictors affecting SM initiations and exchanges are multifaceted, with certain predictors enhancing its utilization and others impeding it. Conclusions: SM telehealth service provided support to patients with mental health needs to a greater extent than those without. By increasing access, fostering better communication, and efficiently allocating resources, telehealth services not only encourage patients to begin using SM but also promote sustained interaction through ongoing SM exchanges.

The PICOTS-ComTeC Framework for Defining Digital Health Interventions: An ISPOR Special Interest Group Report

OBJECTIVES

Digital health definitions are abundant, but often lack clarity and precision. We aimed to develop a minimum information framework to define patient-facing digital health interventions (DHIs) for outcomes research.

METHODS

Definitions of digital-health-related terms (DHTs) were systematically reviewed, followed by a content analysis using frameworks, including PICOTS (population, intervention, comparator, outcome, timing, and setting), Shannon-Weaver Model of Communication, Agency for Healthcare Research and Quality Measures, and the World Health Organization's Classification of Digital Health Interventions. Subsequently, we conducted an online Delphi study to establish a minimum information framework, which was pilot tested by 5 experts using hypothetical examples.

RESULTS

After screening 2610 records and 545 full-text articles, we identified 101 unique definitions of 67 secondary DHTs in 76 articles, resulting in 95 different patterns of concepts among the definitions. World Health Organization system (84.5%), message (75.7%), intervention (58.3%), and technology (52.4%) were the most frequently covered concepts. For the Delphi survey, we invited 47 members of the ISPOR Digital Health Special Interest Group, 18 of whom became the Delphi panel. The first, second, and third survey rounds were completed by 18, 11, and 10 respondents, respectively. After consolidating results, the PICOTS-ComTeC acronym emerged, involving 9 domains (population, intervention, comparator, outcome, timing, setting, communication, technology, and context) and 32 optional subcategories.

CONCLUSIONS

Patient-facing DHIs can be specified using PICOTS-ComTeC that facilitates identification of appropriate interventions and comparators for a given decision. PICOTS-ComTeC is a flexible and versatile tool, intended to assist authors in designing and reporting primary studies and evidence syntheses, yielding actionable results for clinicians and other decision makers.