Engagement and Usability of a Cognitive Behavioral Therapy Mobile App Compared With Web-Based Cognitive Behavioral Therapy Among College Students: Randomized Heuristic Trial

Impact of Conversational and Animation Features of a Mental Health App Virtual Agent on Depressive Symptoms and User Experience Among College Students: Randomized Controlled Trial

Background

Numerous mental health apps purport to alleviate depressive symptoms. Strong evidence suggests that brief cognitive behavioral therapy (bCBT)-based mental health apps can decrease depressive symptoms, yet there is limited research elucidating the specific features that may augment its therapeutic benefits. One potential design feature that may influence effectiveness and user experience is the inclusion of virtual agents that can mimic realistic, human face-to-face interactions.

Objective

The goal of the current experiment was to determine the effect of conversational and animation features of a virtual agent within a bCBT-based mental health app on depressive symptoms and user experience in college students with and without depressive symptoms.

Methods

College students (N=209) completed a 2-week intervention in which they engaged with a bCBT-based mental health app with a customizable therapeutic virtual agent that varied in conversational and animation features. A 2 (time: baseline vs 2-week follow-up) × 2 (conversational vs non-conversational agent) × 2 (animated vs non-animated agent) randomized controlled trial was used to assess mental health symptoms (Patient Health Questionnaire-8, Perceived Stress Scale-10, and Response Rumination Scale questionnaires) and user experience (mHealth App Usability Questionnaire, MAUQ) in college students with and without current depressive symptoms. The mental health app usability and qualitative questions regarding users' perceptions of their therapeutic virtual agent interactions and customization process were assessed at follow-up.

Results

Mixed ANOVA (analysis of variance) results demonstrated a significant decrease in symptoms of depression (P=.002; mean [SD]=5.5 [4.86] at follow-up vs mean [SD]=6.35 [4.71] at baseline), stress (P=.005; mean [SD]=15.91 [7.67] at follow-up vs mean [SD]=17.02 [6.81] at baseline), and rumination (P=.03; mean [SD]=40.42 [12.96] at follow-up vs mean [SD]=41.92 [13.61] at baseline); however, no significant effect of conversation or animation was observed. Findings also indicate a significant increase in user experience in animated conditions. This significant increase in animated conditions is also reflected in the user's ease of use and satisfaction (F(1, 201)=102.60, P<.001), system information arrangement (F(1, 201)=123.12, P<.001), and usefulness of the application (F(1, 201)=3667.62, P<.001).

Conclusions

The current experiment provides support for bCBT-based mental health apps featuring customizable, humanlike therapeutic virtual agents and their ability to significantly reduce negative symptomology over a brief timeframe. The app intervention reduced mental health symptoms, regardless of whether the agent included conversational or animation features, but animation features enhanced the user experience. These effects were observed in both users with and without depressive symptoms.

Mining User Reviews for Key Design Features in Cognitive Behavioral Therapy-Based Mobile Mental Health Apps

Background: Cognitive behavioral therapy (CBT)-based mobile apps have been shown to improve CBT-based interventions effectiveness. Despite the proliferation of these apps, user-centered guidelines pertaining to their design remain limited. The study aims to identify design features of CBT-based apps using online app reviews. Methods: We used 4- and 5-star reviews, preprocessed the reviews, and represented the reviews using word-level bigrams. Then, we leveraged latent Dirichlet allocation (LDA) and visualization techniques using python library for interactive topic model visualization to analyze the review and identify design features that contribute to the success and effectiveness of the app. Results: A total of 24,902 reviews were analyzed. LDA optimization resulted in 86 topics that were labeled by two independent researchers, with an interrater Cohen's kappa value of 0.86. The labeling and grouping process resulted in a total of six main design features for effective CBT-based mobile apps, namely, mental health management and support, credibility support, self-understanding and personality insights, therapeutic approaches and tools, beneficial rescue sessions, and personal growth and development. Conclusions: The high-level design features identified in this study could evidently serve as the backbone of successful CBT-based mobile apps for mental health.

The Reach, Effectiveness, Adoption, Implementation, and Maintenance of Digital Mental Health Interventions for College Students: A Systematic Review

PURPOSE OF REVIEW

We evaluated the impact of digital mental health interventions (DMHIs) for college students. We organized findings using the RE-AIM framework to include reach, effectiveness, adoption, implementation, and maintenance.

RECENT FINDINGS

We conducted a systematic literature review of recent findings from 2019-2024. Our search identified 2,701 articles, of which 95 met inclusion criteria. In the reach domain, student samples were overwhelmingly female and White. In the effectiveness domain, over 80% of DMHIs were effective or partially effective at reducing their primary outcome. In the adoption domain, studies reported modest uptake for DMHIs. In the implementation and maintenance domains, studies reported high adherence rates to DMHI content. While recruitment methods were commonly reported, adaptations and costs of implementation and maintenance were rarely reported. DMHIs for college students are effective for many psychological outcomes. Future work should address diversifying samples and considering implementation in a variety of college settings.

Do engagement and behavioural mechanisms underpin the effectiveness of the Drink Less app?

This is a process evaluation of a large UK-based randomised controlled trial (RCT) (n = 5602) evaluating the effectiveness of recommending an alcohol reduction app, Drink Less, compared with usual digital care in reducing alcohol consumption in increasing and higher risk drinkers. The aim was to understand whether participants' engagement ('self-reported adherence') and behavioural characteristics were mechanisms of action underpinning the effectiveness of Drink Less. Self-reported adherence with both digital tools was over 70% (Drink Less: 78.0%, 95% CI = 77.6-78.4; usual digital care: 71.5%, 95% CI = 71.0-71.9). Self-reported adherence to the intervention (average causal mediation effect [ACME] = -0.250, 95% CI = -0.42, -0.11) and self-monitoring behaviour (ACME = -0.235, 95% CI = -0.44, -0.03) both partially mediated the effect of the intervention (versus comparator) on alcohol reduction. Following the recommendation (self-reported adherence) and the tracking (self-monitoring behaviour) feature of the Drink Less app appear to be important mechanisms of action for alcohol reduction among increasing and higher risk drinkers.

The NeoRoo mobile app: Initial design and prototyping of an Android-based digital health tool to support Kangaroo Mother Care in low/middle-income countries (LMICs)

Premature birth and neonatal mortality are significant global health challenges, with 15 million premature births annually and an estimated 2.5 million neonatal deaths. Approximately 90% of preterm births occur in low/middle income countries, particularly within the global regions of sub-Saharan Africa and South Asia. Neonatal hypothermia is a common and significant cause of morbidity and mortality among premature and low birth weight infants, particularly in low/middle-income countries where rates of premature delivery are high, and access to health workers, medical commodities, and other resources is limited. Kangaroo Mother Care/Skin-to-Skin care has been shown to significantly reduce the incidence of neonatal hypothermia and improve survival rates among premature infants, but there are significant barriers to its implementation, especially in low/middle-income countries (LMICs). The paper proposes the use of a multidisciplinary approach to develop an integrated mHealth solution to overcome the barriers and challenges to the implementation of Kangaroo Mother Care/Skin-to-skin care (KMC/STS) in LMICs. The innovation is an integrated mHealth platform that features a wearable biomedical device (NeoWarm) and an Android-based mobile application (NeoRoo) with customized user interfaces that are targeted specifically to parents/family stakeholders and healthcare providers, respectively. This publication describes the iterative, human-centered design and participatory development of a high-fidelity prototype of the NeoRoo mobile application. The aim of this study was to design and develop an initial ("A") version of the Android-based NeoRoo mobile app specifically to support the use case of KMC/STS in health facilities in Kenya. Key functions and features are highlighted. The proposed solution leverages the promise of digital health to overcome identified barriers and challenges to the implementation of KMC/STS in LMICs and aims to equip parents and healthcare providers of prematurely born infants with the tools and resources needed to improve the care provided to premature and low birthweight babies. It is hoped that, when implemented and scaled as part of a thoughtful, strategic, cross-disciplinary approach to reduction of global rates of neonatal mortality, NeoRoo will prove to be a useful tool within the toolkit of parents, health workers, and program implementors.

Effects of a Smartphone-Based, Multisession Interpretation-Bias Modification for Anxiety: Positive Intervention Effects and Low Attrition

While interpretation-bias modification (IBM) is an effective intervention for treating anxiety, it is not broadly used in clinical or daily practice. To this end, this study developed and tested a smartphone-based IBM application. We adopted the ambiguous situation paradigm as an intervention task in conjunction with robust training materials that broadly covered situations encountered in daily life. We recruited participants with high-trait anxiety and divided them into three groups: (1) positive training; (2) 50% positive-50% negative training; and (3) no-training control. The first two groups completed 28 days of smartphone-based training (IBM in positive cases), and all groups completed six rounds of assessments. The smartphone-based IBM training changed positive and negative endorsements and more specific measures of interpretation bias, thus reducing anxiety. The results also showed that changes in the number of negative interpretations played a mediating role in anxiety reduction. It is notable that the attrition rate was extremely low across the experiment. Our follow-up showed that positive gains persisted throughout the intervening period. Smartphone-based IBM can help individuals with anxiety shift negative biases, broaden their thoughts, enhance their information processing, and effectively target the clinical features of anxiety.

Operationalizing Engagement With an Interpretation Bias Smartphone App Intervention: Case Series

BACKGROUND

Engagement with mental health smartphone apps is an understudied but critical construct to understand in the pursuit of improved efficacy.

OBJECTIVE

This study aimed to examine engagement as a multidimensional construct for a novel app called HabitWorks. HabitWorks delivers a personalized interpretation bias intervention and includes various strategies to enhance engagement such as human support, personalization, and self-monitoring.

METHODS

We examined app use in a pilot study (n=31) and identified 5 patterns of behavioral engagement: consistently low, drop-off, adherent, high diary, and superuser.

RESULTS

We present a series of cases (5/31, 16%) from this trial to illustrate the patterns of behavioral engagement and cognitive and affective engagement for each case. With rich participant-level data, we emphasize the diverse engagement patterns and the necessity of studying engagement as a heterogeneous and multifaceted construct.

CONCLUSIONS

Our thorough idiographic exploration of engagement with HabitWorks provides an example of how to operationalize engagement for other mental health apps.

REDCap Delivery of a Web-Based Intervention for Patients With Voice Disorders: Usability Study

BACKGROUND

Web-based health interventions are increasingly common and are promising for patients with voice disorders because web-based participation does not require voice use. To address needs such as Health Insurance Portability and Accountability Act compliance, unique user access, the ability to send automated reminders, and a limited development budget, we used the Research Electronic Data Capture (REDCap) data management platform to deliver a patient-facing psychological intervention designed for patients with voice disorders. This was a novel use of REDCap.

OBJECTIVE

We aimed to evaluate the usability of the intervention, with this intervention serving as a use case for REDCap-based patient-facing interventions.

METHODS

We used REDCap survey instruments to develop the web-based voice intervention modules, then conducted usability evaluations using (1) heuristic evaluations by 2 evaluators, and (2) formal usability testing with 7 participants, consisting of predetermined tasks, a think-aloud protocol, ease-of-use measurements, a product reaction card, and a debriefing interview.

RESULTS

Heuristic evaluations found strengths in visibility of system status and real-world match, and weaknesses in user control and help documentation. Based on this feedback, changes to the intervention were made before usability testing. Overall, usability testing participants found the intervention useful and easy to use, although testing revealed some concerns with design, content, and terminology. Some concerns were readily addressed, and others required adaptations within REDCap.

CONCLUSIONS

The REDCap version of a complex web-based patient-facing intervention performed well in heuristic evaluation and formal usability testing. REDCap can effectively be used for patient-facing intervention delivery, particularly if the limitations of the platform are anticipated and mitigated.

Implementation of Cognitive Behavioral Therapy in e-Mental Health Apps: Literature Review

Background

To address the matter of limited resources for treating individuals with mental disorders, e–mental health has gained interest in recent years. More specifically, mobile health (mHealth) apps have been suggested as electronic mental health interventions accompanied by cognitive behavioral therapy (CBT).

Objective

This study aims to identify the therapeutic aspects of CBT that have been implemented in existing mHealth apps and the technologies used. From these, we aim to derive research gaps that should be addressed in the future.

Methods

Three databases were screened for studies on mHealth apps in the context of mental disorders that implement techniques of CBT: PubMed, IEEE Xplore, and ACM Digital Library. The studies were independently selected by 2 reviewers, who then extracted data from the included studies. Data on CBT techniques and their technical implementation in mHealth apps were synthesized narratively.

Results

Of the 530 retrieved citations, 34 (6.4%) studies were included in this review. mHealth apps for CBT exploit two groups of technologies: technologies that implement CBT techniques for cognitive restructuring, behavioral activation, and problem solving (exposure is not yet realized in mHealth apps) and technologies that aim to increase user experience, adherence, and engagement. The synergy of these technologies enables patients to self-manage and self-monitor their mental state and access relevant information on their mental illness, which helps them cope with mental health problems and allows self-treatment.

Conclusions

There are CBT techniques that can be implemented in mHealth apps. Additional research is needed on the efficacy of the mHealth interventions and their side effects, including inequalities because of the digital divide, addictive internet behavior, lack of trust in mHealth, anonymity issues, risks and biases for user groups and social contexts, and ethical implications. Further research is also required to integrate and test psychological theories to improve the impact of mHealth and adherence to the e–mental health interventions.

Challenges in Participant Engagement and Retention using Mobile Health Apps: A Literature Review (Preprint)

Background

Mobile health (mHealth) apps are revolutionizing the way clinicians and researchers monitor and manage the health of their participants. However, many studies using mHealth apps are hampered by substantial participant dropout or attrition, which may impact the representativeness of the sample and the effectiveness of the study. Therefore, it is imperative for researchers to understand what makes participants stay with mHealth apps or studies using mHealth apps.

Objective

This study aimed to review the current peer-reviewed research literature to identify the notable factors and strategies used in adult participant engagement and retention.

Methods

We conducted a systematic search of PubMed, MEDLINE, and PsycINFO databases for mHealth studies that evaluated and assessed issues or strategies to improve the engagement and retention of adults from 2015 to 2020. We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Notable themes were identified and narratively compared among different studies. A binomial regression model was generated to examine the factors affecting retention.

Results

Of the 389 identified studies, 62 (15.9%) were included in this review. Overall, most studies were partially successful in maintaining participant engagement. Factors related to particular elements of the app (eg, feedback, appropriate reminders, and in-app support from peers or coaches) and research strategies (eg, compensation and niche samples) that promote retention were identified. Factors that obstructed retention were also identified (eg, lack of support features, technical difficulties, and usefulness of the app). The regression model results showed that a participant is more likely to drop out than to be retained.

Conclusions

Retaining participants is an omnipresent challenge in mHealth studies. The insights from this review can help inform future studies about the factors and strategies to improve participant retention.

Opportunities to Integrate Mobile App-Based Interventions Into Mental Health and Substance Use Disorder Treatment Services in the Wake of COVID-19

The COVID-19 pandemic has heightened concerns about the impact of depression, anxiety, alcohol, and drug use on public health. Mobile apps to address these problems were increasingly popular even before the pandemic, and may help reach people who otherwise have limited treatment access. In this review, we describe pandemic-related substance use and mental health problems, the growing evidence for mobile app efficacy, how health systems can integrate apps into patient care, and future research directions. If equity in access and effective implementation can be addressed, mobile apps are likely to play an important role in mental health and substance use disorder treatment.

Digital Health Innovations, Tools, and Resources to Support Helping Babies Survive Programs

The Helping Babies Survive (HBS) initiative features a suite of evidence-based curricula and simulation-based training programs designed to provide health workers in low- and middle-income countries (LMICs) with the knowledge, skills, and competencies to prevent, recognize, and manage leading causes of newborn morbidity and mortality. Global scale-up of HBS initiatives has been rapid. As HBS initiatives rolled out across LMIC settings, numerous bottlenecks, gaps, and barriers to the effective, consistent dissemination and implementation of the programs, across both the pre- and in-service continuums, emerged. Within the first decade of expansive scale-up of HBS programs, mobile phone ownership and access to cellular networks have also concomitantly surged in LMICs. In this article, we describe a number of HBS digital health innovations and resources that have been developed from 2010 to 2020 to support education and training, data collection for monitoring and evaluation, clinical decision support, and quality improvement. Helping Babies Survive partners and stakeholders can potentially integrate the described digital tools with HBS dissemination and implementation efforts in a myriad of ways to support low-dose high-frequency skills practice, in-person refresher courses, continuing medical and nursing education, on-the-job training, or peer-to-peer learning, and strengthen data collection for key newborn care and quality improvement indicators and outcomes. Thoughtful integration of purpose-built digital health tools, innovations, and resources may assist HBS practitioners to more effectively disseminate and implement newborn care programs in LMICs, and facilitate progress toward the achievement of Sustainable Development Goal health goals, targets, and objectives.