ewrapper: Operationalizing engagement strategies in mHealth

How Notifications Affect Engagement With a Behavior Change App: Results From a Micro-Randomized Trial

BACKGROUND

Drink Less is a behavior change app to help higher-risk drinkers in the United Kingdom reduce their alcohol consumption. The app includes a daily notification asking users to "Please complete your drinks and mood diary," yet we did not understand the causal effect of the notification on engagement nor how to improve this component of Drink Less. We developed a new bank of 30 new messages to increase users' reflective motivation to engage with Drink Less. This study aimed to determine how standard and new notifications affect engagement.

OBJECTIVE

Our objective was to estimate the causal effect of the notification on near-term engagement, to explore whether this effect changed over time, and to create an evidence base to further inform the optimization of the notification policy.

METHODS

We conducted a micro-randomized trial (MRT) with 2 additional parallel arms. Inclusion criteria were Drink Less users who consented to participate in the trial, self-reported a baseline Alcohol Use Disorders Identification Test score of ≥8, resided in the United Kingdom, were aged ≥18 years, and reported interest in drinking less alcohol. Our MRT randomized 350 new users to test whether receiving a notification, compared with receiving no notification, increased the probability of opening the app in the subsequent hour, over the first 30 days since downloading Drink Less. Each day at 8 PM, users were randomized with a 30% probability of receiving the standard message, a 30% probability of receiving a new message, or a 40% probability of receiving no message. We additionally explored time to disengagement, with the allocation of 60% of eligible users randomized to the MRT (n=350) and 40% of eligible users randomized in equal number to the 2 parallel arms, either receiving the no notification policy (n=98) or the standard notification policy (n=121). Ancillary analyses explored effect moderation by recent states of habituation and engagement.

RESULTS

Receiving a notification, compared with not receiving a notification, increased the probability of opening the app in the next hour by 3.5-fold (95% CI 2.91-4.25). Both types of messages were similarly effective. The effect of the notification did not change significantly over time. A user being in a state of already engaged lowered the new notification effect by 0.80 (95% CI 0.55-1.16), although not significantly. Across the 3 arms, time to disengagement was not significantly different.

CONCLUSIONS

We found a strong near-term effect of engagement on the notification, but no overall difference in time to disengagement between users receiving the standard fixed notification, no notification at all, or the random sequence of notifications within the MRT. The strong near-term effect of the notification presents an opportunity to target notifications to increase "in-the-moment" engagement. Further optimization is required to improve the long-term engagement.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/18690.

Moderation Analysis of a couple-based HIV/STI Intervention Among Heterosexual Couples in the Criminal Legal System Experiencing Intimate Partner Violence: Results from a Randomized Controlled Trial

Since the COVID-19 pandemic, intimate partner violence (IPV) rates have increased in the United States. Although accumulating research has documented the effectiveness of couple-based interventions in reducing HIV/STIs, it remains unclear whether they are effective and safe for couples experiencing IPV. We used moderation analysis from a randomized clinical trial to evaluate whether a couples-based HIV/STI intervention may have differential effectiveness in reducing HIV/STI risks among couples where one or both partners reported experiencing IPV compared to couples without such IPV among a sample of 230 men at risk for HIV/STIs who reported using drugs and were mandated to community supervision settings in New York City and their main female sexual partners. The findings of this study suggest that the effectiveness of this evidence-based couple HIV intervention in reducing condomless sex and other HIV/STI risks did not differ between couples with IPV compared to couples without IPV. Intimate partners who use drugs and are involved in the criminal legal system are disproportionately impacted by both HIV/STIs and IPV, underscoring the importance of couple-level interventions that may be scaled up to address the dyadic HIV risks and IPV together in community supervision settings.

The Intersection of Persuasive System Design and Personalization in Mobile Health: Statistical Evaluation

Background

Persuasive technology is an umbrella term that encompasses software (eg, mobile apps) or hardware (eg, smartwatches) designed to influence users to perform preferable behavior once or on a long-term basis. Considering the ubiquitous nature of mobile devices across all socioeconomic groups, user behavior modification thrives under the personalized care that persuasive technology can offer. However, there is no guidance for developing personalized persuasive technologies based on the psychological characteristics of users.

Objective

This study examined the role that psychological characteristics play in interpreted mobile health (mHealth) screen perceived persuasiveness. In addition, this study aims to explore how users’ psychological characteristics drive the perceived persuasiveness of digital health technologies in an effort to assist developers and researchers of digital health technologies by creating more engaging solutions.

Methods

An experiment was designed to evaluate how psychological characteristics (self-efficacy, health consciousness, health motivation, and the Big Five personality traits) affect the perceived persuasiveness of digital health technologies, using the persuasive system design framework. Participants (n=262) were recruited by Qualtrics International, Inc, using the web-based survey system of the XM Research Service. This experiment involved a survey-based design with a series of 25 mHealth app screens that featured the use of persuasive principles, with a focus on physical activity. Exploratory factor analysis and linear regression were used to evaluate the multifaceted needs of digital health users based on their psychological characteristics.

Results

The results imply that an individual user’s psychological characteristics (self-efficacy, health consciousness, health motivation, and extraversion) affect interpreted mHealth screen perceived persuasiveness, and combinations of persuasive principles and psychological characteristics lead to greater perceived persuasiveness. The F test (ie, ANOVA) for model 1 was significant (F9,6540=191.806; P<.001), with an adjusted R2 of 0.208, indicating that the demographic variables explained 20.8% of the variance in perceived persuasiveness. Gender was a significant predictor, with women having higher perceived persuasiveness (P=.008) relative to men. Age was a significant predictor of perceived persuasiveness with individuals aged 40 to 59 years (P<.001) and ≥60 years (P<.001). Model 2 was significant (F13,6536=341.035; P<.001), with an adjusted R2 of 0.403, indicating that the demographic variables self-efficacy, health consciousness, health motivation, and extraversion together explained 40.3% of the variance in perceived persuasiveness.

Conclusions

This study evaluates the role that psychological characteristics play in interpreted mHealth screen perceived persuasiveness. Findings indicate that self-efficacy, health consciousness, health motivation, extraversion, gender, age, and education significantly influence the perceived persuasiveness of digital health technologies. Moreover, this study showed that varying combinations of psychological characteristics and demographic variables affected the perceived persuasiveness of the primary persuasive technology category.

Understanding Adolescent and Young Adult 6-Mercaptopurine Adherence and mHealth Engagement During Cancer Treatment: Protocol for Ecological Momentary Assessment

BACKGROUND

Adolescents and young adults (AYAs) with cancer demonstrate suboptimal oral chemotherapy adherence, increasing their risk of cancer relapse. It is unclear how everyday time-varying contextual factors (eg, mood) affect their adherence, stalling the development of personalized mobile health (mHealth) interventions. Poor engagement is also a challenge across mHealth trials; an effective adherence intervention must be engaging to promote uptake.

OBJECTIVE

This protocol aims to determine the temporal associations between daily contextual factors and 6-mercaptopurine (6-MP) adherence and explore the proximal impact of various engagement strategies on ecological momentary assessment survey completion.

METHODS

At the Children's Hospital of Philadelphia, AYAs with acute lymphoblastic leukemia or lymphoma who are prescribed prolonged maintenance chemotherapy that includes daily oral 6-MP are eligible, along with their matched caregivers. Participants will use an ecological momentary assessment app called ADAPTS (Adherence Assessments and Personalized Timely Support)-a version of an open-source app that was modified for AYAs with cancer through a user-centered process-and complete surveys in bursts over 6 months. Theory-informed engagement strategies will be microrandomized to estimate the causal effects on proximal survey completion.

RESULTS

With funding from the National Cancer Institute and institutional review board approval, of the proposed 30 AYA-caregiver dyads, 60% (18/30) have been enrolled; of the 18 enrolled, 15 (83%) have completed the study so far.

CONCLUSIONS

This protocol represents an important first step toward prescreening tailoring variables and engagement components for a just-in-time adaptive intervention designed to promote both 6-MP adherence and mHealth engagement.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/32789.

Engaging youth in mHealth: what works and how can we be sure?

BACKGROUND

Youth participating in mobile health (mHealth) intervention trials often engage with the technologies [e.g., applications (app) or mobile-optimized websites] only partially, often prematurely discontinuing use altogether. Limited engagement can impact the interventions effect on behavior change and compromise researchers' ability to test and estimate the true efficacy of their interventions. While mHealth interventions have been shown to be feasible and acceptable to youth, across diverse health conditions, strategies to increase engagement have been less well studied. Specifically, within HIV prevention and care mHealth interventions, there is not consensus as to which components represent the "key ingredients" to support maximal engagement of youth. Further, successful intervention evaluation requires the ability to systematically track users' engagement with intervention components (i.e., paradata) to evaluate its effects on behavior change.

METHODS

As part of the Adolescent Medicine Trials Network UNC/Emory Center for Innovative Technology (iTech) portfolio of HIV/AIDS Interventions, we present diverse strategies used across five mHealth protocols seeking to promote youth engagement, track and measure engagement through paradata, and incorporate these components into mHealth intervention evaluations.

RESULTS

We describe the importance of defining and measuring engagement using case studies from iTech to illustrate how different research teams select mHealth features to promote youth engagement over time, taking into account features embedded in the technology design, key mechanisms of change and trial outcomes (e.g., HIV testing, pre-exposure prophylaxis uptake and adherence, HIV treatment adherence). Finally, we discuss how the research teams plan to evaluate engagement's role on their intervention's outcomes.

CONCLUSIONS

Based on this synthesis, we discuss strategies to enhance mHealth engagement during intervention development and design, ensure its monitoring and reporting throughout the trial, and evaluate its impact on trial outcomes.

National Institutes of Health Support of Digital Health Behavior Research

The National Institutes of Health (NIH) has increasingly supported research in digital health technologies to advance research and deliver behavior change interventions. We highlight some of the research supported by the NIH in eHealth, mHealth, and social media as well as research resources supported by the NIH to accelerate research in this area. We also describe some of the challenges and opportunities in the digital health field and the need to balance the promise of these technologies with rigorous scientific evidence.

The application of digital health to the assessment and treatment of substance use disorders: The past, current, and future role of the National Drug Abuse Treatment Clinical Trials Network

The application of digital technologies to better assess, understand, and treat substance use disorders (SUDs) is a particularly promising and vibrant area of scientific research. The National Drug Abuse Treatment Clinical Trials Network (CTN), launched in 1999 by the U.S. National Institute on Drug Abuse, has supported a growing line of research that leverages digital technologies to glean new insights into SUDs and provide science-based therapeutic tools to a diverse array of persons with SUDs. This manuscript provides an overview of the breadth and impact of research conducted in the realm of digital health within the CTN. This work has included the CTN's efforts to systematically embed digital screeners for SUDs into general medical settings to impact care models across the nation. This work has also included a pivotal multi-site clinical trial conducted on the CTN platform, whose data led to the very first "prescription digital therapeutic" authorized by the U.S. Food and Drug Administration (FDA) for the treatment of SUDs. Further CTN research includes the study of telehealth to increase capacity for science-based SUD treatment in rural and under-resourced communities. In addition, the CTN has supported an assessment of the feasibility of detecting cocaine-taking behavior via smartwatch sensing. And, the CTN has supported the conduct of clinical trials entirely online (including the recruitment of national and hard-to-reach/under-served participant samples online, with remote intervention delivery and data collection). Further, the CTN is supporting innovative work focused on the use of digital health technologies and data analytics to identify digital biomarkers and understand the clinical trajectories of individuals receiving medications for opioid use disorder (OUD). This manuscript concludes by outlining the many potential future opportunities to leverage the unique national CTN research network to scale-up the science on digital health to examine optimal strategies to increase the reach of science-based SUD service delivery models both within and outside of healthcare.

Engagement in mHealth behavioral interventions for HIV prevention and care: making sense of the metrics

BACKGROUND

Engagement is the primary metric by which researchers can assess whether participants in a mHealth intervention used and interacted with the intervention's content as intended over a pre-specified period to result in behavior change. Paradata, defined as the process data documenting users' access, participation, and navigation through a mHealth intervention, have been associated with differential treatment outcomes in mHealth interventions. Within behavioral mHealth interventions, there has been an increase in the number of studies addressing the HIV prevention and care continuum in recent years, yet few have presented engagement metrics or examined how these data could inform design modifications, promote continued engagement, and supplement primary intervention efficacy and scale-up efforts.

METHODS

We review common paradata metrics in mHealth interventions (e.g., amount, frequency, duration and depth of use), using case studies from four technology-driven HIV interventions to illustrate their utility in evaluating mHealth behavioral interventions for HIV prevention and care. Across the four case studies, participants' ages ranged between 15 and 30 years and included a racially and ethnically diverse sample of youth. The four case studies had different approaches for engaging young men who have sex with men: a tailored brief intervention, an interactive modular program, a daily tool to monitor and self-regulate treatment adherence, and an online platform promoting social engagement and social support. Each focused on key outcomes across the HIV prevention and care continuum [e.g., safer sex behaviors, HIV testing, antiretroviral therapy (ART) adherence] and collected paradata metrics systematically.

RESULTS

Across the four interventions, paradata was utilized to identify patterns of use, create user profiles, and determine a minimum engagement threshold for future randomized trials based on initial pilot trial data. Evidence of treatment differences based on paradata analyses were also observed in between-arm and within-arm analyses, indicating that intervention exposure and dosage might influence the strength of the observed intervention effects. Paradata reflecting participants' engagement with intervention content was used to suggest modifications to intervention design and navigation, to understand what theoretically-driven content participants chose to engage with in an intervention, and to illustrate how engagement was linked to HIV-related outcomes.

CONCLUSIONS

Paradata monitoring and reporting can enhance the rigor of mHealth trials. Metrics of engagement must be systematically collected, analyzed and interpreted to meaningfully understand a mHealth intervention's efficacy. Future mHealth trials should work to identify suitable engagement metrics during intervention development, ensure their collection throughout the trial, and evaluate their impact on trial outcomes.

Optimizing Digital Integrated Care via Micro-Randomized Trials

Mobile health (mHealth) interventions are a promising tool in providing digitally mediated integrative care. They can extend care outside of the clinic by providing reminders to take medications, assisting in managing symptoms, and supporting healthy behaviors including physical activity, healthy eating, and stress management. mHealth interventions can adapt the delivery of care across time in order to optimize treatment effectiveness. Yet there exists limited empirical evidence useful to the development of adaptive mHealth interventions. This article describes a new randomized trial design, the Micro-Randomized Trial (MRT), for informing the development of mHealth interventions. We provide examples of scientific questions important to the development of an mHealth intervention, and describe how these questions can be answered using an MRT.