Dose-Response Effects of Patient Engagement on Health Outcomes in an mHealth Intervention: Secondary Analysis of a Randomized Controlled Trial

Dose‒Response Effects of Patient Engagement on Anxiety and Depression in a Cognitive-Behavioral Intervention: Secondary Analysis of a Pilot Randomized Controlled Trial and a Clinical Controlled Trial

Understanding the dose‒response relationship between patient engagement in cognitive behavioral therapy (CBT) and health outcomes is critical for developing and implementing effective CBT programs. In studies of CBT interventions, patient engagement is measured only at a single time point, and outcomes are typically assessed before and after the intervention. Examination of the dose‒response relationship between patient engagement in CBT and outcomes is limited. It is unclear whether a dose‒response relationship exists between patient engagement in on-site CBT intervention and anxiety and depression in people living with HIV (PLWH). If present, does this dose‒response relationship occur early or later in the intervention? This study aimed to address this gap by examining the dose‒response relationships between patient engagement and anxiety and depression in CBT interventions among PLWH. Utilizing data from a pilot randomized trial (10 participants) and a clinical controlled trial (70 participants), our secondary analysis spans baseline, 3-month, and 6-month assessments. Both trials implemented the nurse-led CBT intervention. Cluster analysis identified two groups based on on-site attendance and WeChat activity. Patients with good adherence (6-10 times) of on-site attendance exhibited significantly lower anxiety and depression scores at 3 months (β = 1.220, P = 0.047; β = 1.270, P = 0.019), with no significant differences observed at 6 months. WeChat activity did not significantly influence anxiety or depression scores. The findings highlight a significant short-term dose‒response relationship, endorsing nurse-led CBT interventions for mental health in PLWH. Organizational strategies should focus on incentivizing and facilitating patient engagement, particularly through enhancing WeChat features.

The Goldilocks Dilemma on Balancing User Response and Reflection in mHealth Interventions: Observational Study

Background

Mobile health (mHealth) has the potential to radically improve health behaviors and quality of life; however, there are still key gaps in understanding how to optimize mHealth engagement. Most engagement research reports only on system use without consideration of whether the user is reflecting on the content cognitively. Although interactions with mHealth are critical, cognitive investment may also be important for meaningful behavior change. Notably, content that is designed to request too much reflection could result in users' disengagement. Understanding how to strike the balance between response burden and reflection burden has critical implications for achieving effective engagement to impact intended outcomes.

Objective

In this observational study, we sought to understand the interplay between response burden and reflection burden and how they impact mHealth engagement. Specifically, we explored how varying the response and reflection burdens of mHealth content would impact users' text message response rates in an mHealth intervention.

Methods

We recruited support persons of people with diabetes for a randomized controlled trial that evaluated an mHealth intervention for diabetes management. Support person participants assigned to the intervention (n=148) completed a survey and received text messages for 9 months. During the 2-year randomized controlled trial, we sent 4 versions of a weekly, two-way text message that varied in both reflection burden (level of cognitive reflection requested relative to that of other messages) and response burden (level of information requested for the response relative to that of other messages). We quantified engagement by using participant-level response rates. We compared the odds of responding to each text and used Poisson regression to estimate associations between participant characteristics and response rates.

Results

The texts requesting the most reflection had the lowest response rates regardless of response burden (high reflection and low response burdens: median 10%, IQR 0%-40%; high reflection and high response burdens: median 23%, IQR 0%-51%). The response rate was highest for the text requesting the least reflection (low reflection and low response burdens: median 90%, IQR 61%-100%) yet still relatively high for the text requesting medium reflection (medium reflection and low response burdens: median 75%, IQR 38%-96%). Lower odds of responding were associated with higher reflection burden (P<.001). Younger participants and participants who had a lower socioeconomic status had lower response rates to texts with more reflection burden, relative to those of their counterparts (all P values were <.05).

Conclusions

As reflection burden increased, engagement decreased, and we found more disparities in engagement across participants' characteristics. Content encouraging moderate levels of reflection may be ideal for achieving both cognitive investment and system use. Our findings provide insights into mHealth design and the optimization of both engagement and effectiveness.

Hospital Readmissions by Variation in Engagement in the Health Care Hotspotting Trial: A Secondary Analysis of a Randomized Clinical Trial

Importance

Variability in intervention participation within care management programs can complicate standard analysis strategies.

Objective

To evaluate whether care management was associated with reduced hospital readmissions among individuals with higher participation probabilities.

Design, Setting, and Participants

A total of 800 hospitalized patients aged 18 years and older were randomized as part of the Health Care Hotspotting randomized clinical trial, which was conducted in Camden, New Jersey, from June 2014 to September 2017. Data were collected through October 2018. In this new analysis performed between April 6, 2022, and April 23, 2023, the distillation method was applied to account for variable intervention participation. A gradient-boosting machine learning model produced predicted probabilities of engaged participation using baseline covariates only. Predicted probabilities were used to trim both intervention and control populations in an equivalent manner, and intervention effects were reevaluated within study population subsets that were increasingly concentrated with patients having higher participation probabilities. Patients had 2 or more hospitalizations in the 6-month preenrollment period and documented evidence of chronic illness and social complexity.

Intervention

Multidisciplinary teams provided services to patients in the intervention arm for a mean 120 days after hospital discharge. Patients in the control group received usual postdischarge care.

Main Outcomes and Measures

Hospital readmission rates and counts 30, 90, and 180 days postdischarge.

Results

Of 800 eligible patients, 782 had complete discharge information and were included in this analysis (mean [SD] age, 56.6 [12.7] years; 395 [50.5%] female). In the intent-to-treat analysis, the unadjusted 180-day readmission rate for treatment and control groups was 60.1% vs 61.7% (adjusted odds ratio, 0.95; 95% CI, 0.71-1.28; P = .73) and the mean (SD) number of 180-day readmissions was 1.45 (1.89) vs 1.48 (1.94) (adjusted incidence rate ratio, 0.99, 95% CI, 0.88-1.12; P = .86). Among the population with the highest participation probabilities, the mean (SD) 180-day readmission count was 1.22 (1.74) vs 1.57 (1.74) and the incidence rate ratio attained statistical significance (adjusted incidence rate ratio, 0.74; 95% CI, 0.56-0.99; P = .045). Adjusted odds ratios and adjusted incidence rate ratios for 30- and 90-day outcomes reached statistical significance after population distillation.

Conclusions and Relevance

This secondary analysis of a randomized clinical trial found that care management was associated with reduced readmissions among patients with higher participation probabilities, suggesting that program operation could be improved by addressing barriers to participation and refining inclusion criteria to identify patients most likely to benefit.

Trial Registration

ClinicalTrials.gov Identifier: NCT02090426.

Assessing Patient Adherence to and Engagement With Digital Interventions for Depression in Clinical Trials: Systematic Literature Review

BACKGROUND

New approaches to the treatment of depression are necessary for patients who do not respond to current treatments or lack access to them because of barriers such as cost, stigma, and provider shortage. Digital interventions for depression are promising; however, low patient engagement could limit their effectiveness.

OBJECTIVE

This systematic literature review (SLR) assessed how participant adherence to and engagement with digital interventions for depression have been measured in the published literature, what levels of adherence and engagement have been reported, and whether higher adherence and increased engagement are linked to increased efficacy.

METHODS

We focused on a participant population of adults (aged ≥18 years) with depression or major depressive disorder as the primary diagnosis and included clinical trials, feasibility studies, and pilot studies of digital interventions for treating depression, such as digital therapeutics. We screened 756 unique records from Ovid MEDLINE, Embase, and Cochrane published between January 1, 2000, and April 15, 2022; extracted data from and appraised the 94 studies meeting the inclusion criteria; and performed a primarily descriptive analysis. Otsuka Pharmaceutical Development & Commercialization, Inc (Princeton, New Jersey, United States) funded this study.

RESULTS

This SLR encompassed results from 20,111 participants in studies using 47 unique web-based interventions (an additional 10 web-based interventions were not described by name), 15 mobile app interventions, 5 app-based interventions that are also accessible via the web, and 1 CD-ROM. Adherence was most often measured as the percentage of participants who completed all available modules. Less than half (44.2%) of the participants completed all the modules; however, the average dose received was 60.7% of the available modules. Although engagement with digital interventions was measured differently in different studies, it was most commonly measured as the number of modules completed, the mean of which was 6.4 (means ranged from 1.0 to 19.7) modules. The mean amount of time participants engaged with the interventions was 3.9 (means ranged from 0.7 to 8.4) hours. Most studies of web-based (34/45, 76%) and app-based (8/9, 89%) interventions found that the intervention group had substantially greater improvement for at least 1 outcome than the control group (eg, care as usual, waitlist, or active control). Of the 14 studies that investigated the relationship between engagement and efficacy, 9 (64%) found that increased engagement with digital interventions was significantly associated with improved participant outcomes. The limitations of this SLR include publication bias, which may overstate engagement and efficacy, and low participant diversity, which reduces the generalizability.

CONCLUSIONS

Patient adherence to and engagement with digital interventions for depression have been reported in the literature using various metrics. Arriving at more standardized ways of reporting adherence and engagement would enable more effective comparisons across different digital interventions, studies, and populations.

Translating Research Evidence Into Marketplace Application: Cohort Study of Internet-Based Intervention Platforms for Perinatal Depression

BACKGROUND

Internet-based intervention platforms may improve access to mental health care for women with perinatal depression (PND). Though the majority of platforms in the market lack an evidence base, a small number of them are supported by research evidence.

OBJECTIVE

This study aims to assess the current status of internet-based PND intervention platforms supported by published evidence, understand the reasons behind the disappearance of any of these previously accessible platforms, examine adjustments made by those active platforms between research trials and market implementation, and evaluate their current quality.

METHODS

A cohort of internet-based PND intervention platforms was first identified by systematic searches in multiple academic databases from database inception until March 26, 2021. We searched on the World Wide Web and the iOS and Android app stores to assess which of these were available in the marketplace between April and May 2021. The basic characteristics of all platforms were collected. For inaccessible platforms, inquiries were made via email to the authors of publications to determine the reasons for their unavailability. We compared the intervention-related information of accessible platforms in the marketplace with that reported in original publications and conducted quality assessments using the App Evaluation Model of the American Psychiatric Association. Fisher exact tests were used to compare the functional characteristics in publications of available and unavailable platforms and to investigate potential associations between functional adjustments or quality indices and platform survival time.

RESULTS

Out of 35 platforms supported by research evidence, only 19 (54%) were still accessible in the marketplace. The main reason for platforms disappearing was the termination of research projects. No statistically significant differences were found in functional characteristics between available and unavailable platforms. A total of 18 (95%) platforms adapted their core functions from what was reported in related publications. The adjustments included changes to intervention methods (11/19, 58%), target population (10/19, 53%), human resources for intervention support (9/19, 47%), mood assessment and monitoring (8/19, 42%), communication modality (4/19, 21%), and platform type (2/19, 11%). Quality issues across platforms included low frequency of update, lack of crisis management mechanism, poor user interactivity, and weak evidence base or absence of citation of supporting evidence. Platforms that survived longer than 10 years had a higher tendency to use external resources from third parties compared to those that survived less than 10 years (P=.04). No significant differences were found for functional adjustments or other quality indices.

CONCLUSIONS

Internet-based platforms supported by evidence were not effectively translated into real-world practice. It is unclear if adjustments to accessible platforms made during actual operation may undermine the proven validity of the original research. Future research to explore the reasons behind the success of the implementation of evidence-based platforms in the marketplace is warranted.

A Social Media-Based Intervention for Chinese American Caregivers of Persons With Dementia: Protocol Development

BACKGROUND

Racial/ethnic minority and immigrant caregivers of persons with dementia experience high rates of psychosocial stress and adverse health outcomes. Few culturally tailored mobile health (mHealth) programs were designed for these vulnerable populations.

OBJECTIVE

This study reports the development of a culturally tailored mHealth program called Wellness Enhancement for Caregivers (WECARE) to improve caregiving skills, reduce distress, and improve the psychosocial well-being of Chinese American family caregivers of persons with dementia.

METHODS

Community-based user-centered design principles were applied in the program development. First, the structure and curriculum of the WECARE program were crafted based on existing evidence-based interventions for caregivers with input from 4 experts. Second, through working closely with 8 stakeholders, we culturally adapted evidence-based programs into multimedia program components. Lastly, 5 target users tested the initial WECARE program; their experience and feedback were used to further refine the program.

RESULTS

The resulting WECARE is a 7-week mHealth program delivered via WeChat, a social media app highly popular in Chinese Americans. By subscribing to the official WECARE account, users can receive 6 interactive multimedia articles pushed to their WeChat accounts each week for 7 weeks. The 7 major themes include (1) facts of dementia and caregiving; (2) the enhancement of caregiving skills; (3) effective communication with health care providers, care partners, and family members; (4) problem-solving skills for caregiving stress management; (5) stress reduction and depression prevention; (6) the practice of self-care and health behaviors; and (7) social support and available resources. Users also have the option of joining group chats for peer support. The WECARE program also includes a back-end database that manages intervention delivery and tracks user engagement.

CONCLUSIONS

The WECARE program represents one of the first culturally tailored social media-based interventions for Chinese American caregivers of persons with dementia. It demonstrates the use of community-based user-centered design principles in developing an mHealth intervention program in underserved communities. We call for more cultural adaptation and development of mHealth interventions for immigrant and racial/ethnic minority caregivers of persons with dementia.

The Effect of Periodic Email Prompts on Participant Engagement with a Behavior Change mHealth App: Longitudinal Study (Preprint)

BACKGROUND

Following the need for the prevention of noncommunicable diseases, mobile health (mHealth) apps are increasingly used for promoting lifestyle behavior changes. Although mHealth apps have the potential to reach all population segments, providing accessible and personalized services, their effectiveness is often limited by low participant engagement and high attrition rates.

OBJECTIVE

This study concerns a large-scale, open-access mHealth app, based in the Netherlands, focused on improving the lifestyle behaviors of its participants. The study examines whether periodic email prompts increased participant engagement with the mHealth app and how this effect evolved over time. Points gained from the activities in the app were used as an objective measure of participant engagement with the program. The activities considered were physical workouts tracked through the mHealth app and interactions with the web-based coach.

METHODS

The data analyzed covered 22,797 unique participants over a period of 78 weeks. A hidden Markov model (HMM) was used for disentangling the overtime effects of periodic email prompts on participant engagement with the mHealth app. The HMM accounted for transitions between latent activity states, which generated the observed measure of points received in a week.

RESULTS

The HMM indicated that, on average, 70% (15,958/22,797) of the participants were in the inactivity state, gaining 0 points in total per week; 18% (4103/22,797) of the participants were in the average activity state, gaining 27 points per week; and 12% (2736/22,797) of the participants were in the high activity state, gaining 182 points per week. Receiving and opening a generic email was associated with a 3 percentage point increase in the likelihood of becoming active in that week, compared with the weeks when no email was received. Examining detailed email categories revealed that the participants were more likely to increase their activity level following emails that were in line with the program's goal, such as emails regarding health campaigns, while being resistant to emails that deviated from the program's goal, such as emails regarding special deals.

CONCLUSIONS

Participant engagement with a behavior change mHealth app can be positively influenced by email prompts, albeit to a limited extent. Given the relatively low costs associated with emails and the high population reach that mHealth apps can achieve, such instruments can be a cost-effective means of increasing participant engagement in the stride toward improving program effectiveness.

Testing Behavioral Nudges and Prompts in Digital Courses for Self-Guided Treatment of Depression and Anxiety, Protocol for 3-Arm Randomized Controlled Trial (Preprint)

Background

Despite showing strong evidence of positive outcomes, a common problem in the field of digital health is poor engagement and adherence. Non–health care, for-profit digital ventures, such as Facebook, LinkedIn, and Twitter, conduct behavioral experiments to increase user engagement. To our knowledge, digital health organizations have not published similar types of experiments in ad libitum environments, and there are limited published data indicating whether nudges and prompts can be leveraged to increase engagement with digital health interventions.

Objective

The main objective of our 3-arm randomized controlled trial is to test whether registered members in two well-established digital health courses for anxiety and depression will engage with four different types of nudges and prompts, and whether engaging with nudges and prompts increases engagement within the courses.

Methods

New members who register for the self-guided anxiety and depression courses on the Evolution Health platform will be randomized into 1 of 3 arms. The first control arm will feature a member home page without any behavioral nudges or prompts. The second arm will feature a member home page with a Tip-of-the-Day section containing directive content. Arm 3 will feature a member home page with a Tip-of-the-Day section containing social proof and present bias content. The third arm will also feature a to-do item checklist.

Results

The experiment was designed in August 2021 and was launched in November 2021. Initially, we will measure engagement with the tips and the to-do checklist by calculating the frequency of use by age and gender. If members do engage, we will then, according to age and gender, examine whether nudges and prompts result in higher course completion rates and whether specific types of prompts and nudges are more popular than others.

Conclusions

Our 3-arm randomized controlled trial will be the first to compare four distinct types of behavioral prompts and nudges in two self-guided digital health courses that were designed to treat mental health issues. We expect the results to generate insights into which types of behavioral prompts and nudges work best in the population. If they are shown to increase engagement, the insights will then be used to apply prompts and nudges to the platform’s addiction-focused courses. Based on the results of the experiment, the insights will be applied to using artificial intelligence to train the platform to recognize different usage patterns and provide specific engagement recommendations to stratified users.

International Registered Report Identifier (IRRID)

DERR1-10.2196/37231