Engagement Within a Mobile Phone-Based Smoking Cessation Intervention for Adolescents and its Association With Participant Characteristics and Outcomes

Measuring Engagement in Provider-Guided Digital Health Interventions With a Conceptual and Analytical Framework Using Nurse WRITE as an Exemplar: Exploratory Study With an Iterative Approach

BACKGROUND

Limited guidance exists for analyzing participant engagement in provider-guided digital health interventions (DHIs). System usage is commonly assessed, with acknowledged limitations in measuring socio-affective and cognitive aspects of engagement. Nurse WRITE, an 8-week web-based nurse-guided DHI for managing symptoms among women with recurrent ovarian cancer, offers an opportunity to develop a framework for assessing multidimensional engagement.

OBJECTIVE

This study aims to develop a conceptual and analytic framework to measure socio-affective, cognitive, and behavioral engagement with provider-guided DHIs. We then illustrate the framework's ability to describe and categorize engagement using Nurse WRITE as an example.

METHODS

A sample of 68 participants from Nurse WRITE who posted on the message boards were included. We adapted a prior framework for conceptualizing and operationalizing engagement across 3 dimensions and finalized a set of 6 distinct measures. Using patients' posts, we created 2 socio-affective engagement measures-total count of socio-affective engagement classes (eg, sharing personal experience) and total word count-and 2 cognitive engagement measures-total count of cognitive engagement classes (eg, asking information-seeking questions) and average question completion percentage. Additionally, we devised behavioral engagement measures using website data-the total count of symptom care plans and plan reviews. k-Means clustering categorized the participants into distinct groups based on levels of engagement across 3 dimensions. Descriptive statistics and narratives were used to describe engagement in 3 dimensions.

RESULTS

On average, participants displayed socio-affective engagement 34.7 times, writing 14,851 words. They showed cognitive engagement 19.4 times, with an average of 78.3% completion of nurses' inquiries. Participants also submitted an average of 1.6 symptom care plans and 0.7 plan reviews. Participants were clustered into high (n=13), moderate (n=17), and low engagers (n=38) based on the 6 measures. High engagers wrote a median of 36,956 (IQR 26,199-46,265) words. They demonstrated socio-affective engagement approximately 81 times and cognitive engagement around 46 times, approximately 6 times that of the low engagers and twice that of the moderate engagers. High engagers had a median of 91.7% (IQR 82.2%-93.7%) completion of the nurses' queries, whereas moderate engagers had 86.4% (IQR 80%-96.4%), and low engagers had 68.3% (IQR 60.1%-79.6%). High engagers completed a median of 3 symptom care plans and 2 reviews, while moderate engagers completed 2 plans and 1 review. Low engagers completed a median of 1 plan with no reviews.

CONCLUSIONS

This study developed and reported an engagement framework to guide behavioral intervention scientists in understanding and analyzing participants' engagement with provider-guided DHIs. Significant variations in engagement levels across 3 dimensions highlight the importance of measuring engagement with provider-guided DHIs in socio-affective, cognitive, and behavioral dimensions. Future studies should validate the framework with other DHIs, explore the influence of patient and provider factors on engagement, and investigate how engagement influences intervention efficacy.

Feasibility, usability, and acceptability of MobileCoach-Teen: A smartphone app-based preventative intervention for risky adolescent drinking behavior

BACKGROUND

Older adolescence (ages 15-18) is a critical period for experimentation with substance use, especially alcohol. Adolescent drinking poses hazards to physical and mental health, amplifies risk associated with other activities typically initiated during this life stage (e.g., driving, sexual activity), and is associated with adverse outcomes in adolescence and adulthood. Existing preventative interventions are expensive and have questionable long-term efficacy. Digital interventions may represent an accessible and personalized approach to providing preventative intervention content to youth.

METHODS

This study recruited 29 adolescents aged 16-18 (M = 17.24, SD = 0.74) for a pilot feasibility trial of the MobileCoach-Teen (MC-Teen) smartphone app-based intervention. The study team randomized participants to receive either the alcohol intervention (MC-Teen) or attention control pseudo-intervention (MC-Fit). MC-Teen participants received 12 weeks of content adapted from a prior Swiss-based trial of a preventative alcohol intervention. Participants provided qualitative and quantitative feedback at baseline, via six biweekly surveys during and post-intervention.

RESULTS

Both groups rated the application as easy to download (M = 4.31, SD = 0.93; 5-point Likert). All participants completed the baseline survey in less than the estimated time of 10 min (M = 7:42, SD = 2:15) and rated the survey as easy to complete (M = 4.69, SD = 0.60; 5-point Likert). MC-Teen participants favorably assessed application user experience, message user experience, and digital working alliance with application. Qualitative themes included a desire for increased rate/amount and diversity of content, greater representation via coach options, user interface/user experience improvements, and additional features.

CONCLUSION

The MC-Teen intervention is feasible and acceptable based on a pilot feasibility trial with a sample of U.S. adolescents.

Strengths and Limitations of Web-Based Cessation Support for Individuals Who Smoke, Dual Use, or Vape: Qualitative Interview Study

BACKGROUND

Tobacco use has shifted in recent years, especially with the introduction of e-cigarettes. Despite the current variable and intersecting tobacco product use among tobacco users, most want to quit, which necessitates cessation programs to adapt to these variable trends (vs focusing on combustible cigarettes alone). The use of web-based modalities for cessation support has become quite popular in recent years and has been compounded by the COVID-19 pandemic. Therefore, understanding the current strengths and limitations of existing programs to meet the needs of current various tobacco users is critical for ensuring the saliency of such programs moving forward.

OBJECTIVE

The purpose of this study was to understand the strengths and limitations of web-based cessation support offered through QuitNow to better understand the needs of a variety of end users who smoke, dual use, or vape.

METHODS

Semistructured interviews were conducted with 36 nicotine product users in British Columbia. Using conventional content analysis methods, we inductively derived descriptive categories and themes related to the strengths and limitations of QuitNow for those who smoke, dual use, or vape. We analyzed the data with the support of NVivo (version 12; QSR International) and Excel (Microsoft Corporation).

RESULTS

Participants described several strengths and limitations of QuitNow and provided suggestions for improvement, which fell under 2 broad categories: look and feel and content and features. Shared strengths included the breadth of information and the credible nature of the website. Individuals who smoke were particularly keen about the site having a nonjudgmental feeling. Moreover, compared with individuals who smoke, individuals who dual use and individuals who vape were particularly keen about access to professional quit support (eg, quit coach). Shared limitations included the presence of too much text and the need to create an account. Individuals who dual use and individuals who vape thought that the content was geared toward older adults and indicated that there was a lack of information about vaping and personalized content. Regarding suggestions for improvement, participants stated that the site needed more interaction, intuitive organization, improved interface esthetics, a complementary smartphone app, forum discussion tags, more information for different tobacco user profiles, and user testimonials. Individuals who vape were particularly interested in website user reviews. In addition, individuals who vape were more interested in an intrinsic approach to quitting (eg, mindfulness) compared with extrinsic approaches (eg, material incentives), the latter of which was endorsed by more individuals who dual use and individuals who smoke.

CONCLUSIONS

The findings of this study provide directions for enhancing the saliency of web-based cessation programs for a variety of tobacco use behaviors that hallmark current tobacco use.

Examining engagement effects in an adaptive preventive intervention for college student drinking

OBJECTIVE

This study determined the characteristics of engagement and whether engagement in an adaptive preventive intervention (API) was associated with reduced binge drinking and alcohol-related consequences.

METHOD

Incoming students were recruited for a sequential multiple assignment randomized trial (SMART; N = 891, 62.4% female, 76.8% non-Hispanic White) with an assessment-only control group. The API occurred during the first semester of college, with outcomes assessed at the end of the semester. The API involved two stages. Stage 1 included universal intervention components (personalized normative feedback [PNF] and self-monitoring). Stage 2 bridged heavy drinkers to access additional resources. We estimated the effect of engagement in Stage 1 only and in the whole API (Stages 1 and 2) among the intervention group, and the effect of the API versus control had all students assigned an API engaged, on alcohol-related outcomes.

RESULTS

Precollege binge drinking, intention to pledge a fraternity/sorority, and higher conformity motives were most associated with lower odds of Stage 1 engagement. Action (readiness to change) and PNF engagement were associated with Stage 2 engagement. API engagement was associated with significant reductions in alcohol-related consequences among heavy drinkers. Compared to the control, we estimated the API would reduce the relative increase in alcohol-related consequences from baseline to follow-up by 25%, had all API students engaged.

CONCLUSIONS

Even partial engagement in each component of the "light-touch" API rendered benefits. Analyses suggested that had all students in the intervention group engaged, the API would significantly reduce the change in alcohol-related consequences over the first semester in college. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Trajectory classes of engagement with an alcohol text message intervention and predictors of intervention engagement

Digital interventions have increased our capacity to reach young adults who have hazardous alcohol use. Alcohol text message interventions have shown small effects in reducing hazardous drinking, leaving room for improvement. An important challenge to address in improving digital interventions is maintaining engagement, which reflects the "dose" of intervention received. This study aimed to identify trajectory classes of engagement with an alcohol text message intervention, and baseline predictors of the trajectory classes to determine "for whom" the digital intervention was more versus less engaging, to guide further intervention tailoring. This secondary analysis examined data from a study that compared five 12-week alcohol text message interventions designed to reduce hazardous drinking in young adults (aged 18-25; N = 1,131, 68% female) recruited from Emergency Departments in Western Pennsylvania. Engagement with the intervention was based on response (present/absent) to text message queries delivered twice per week during 2-week run-in and 12-week intervention. Repeated measures latent profile analysis identified five latent trajectory classes as having the best fit to the data: "High engagement" (55.1%), "Slow decrease, moderate engagement" (23.2%); "Mid-way decrease in engagement" (8.9%), "Steadily decreasing engagement" (8.1%); and "Fluctuating, moderate engagement" (4.6%). Females and individuals enrolled in college were overrepresented in the high engagement trajectory class, whereas individuals higher in impulsivity were more likely to be in decreasing (versus high) engagement trajectory classes. Methods to boost engagement using, for example, motivational enhancement for young adults with higher levels of impulsivity, at specific time points, such as the mid-point of the intervention, warrant consideration.

Predictors of Youth Accessibility for a Mobile Phone-Based Life Skills Training Program for Addiction Prevention

BACKGROUND

Digital interventions are an emerging and promising avenue for addiction prevention and mental health promotion, but their reach and use are often limited, and little is known about the factors associated with youth accessibility. SmartCoach is a life skills training program for addiction prevention where adolescents are proactively invited for program participation in secondary school classes. The mobile phone-based program provides individualized coaching for a period of 4 months and addresses self-management skills, social skills, and substance use resistance skills. This study examined sociodemographic and other predictors of program participation and program use.

METHODS

A total of 476 adolescents in 28 secondary and upper secondary school classes in the German-speaking part of Switzerland were proactively invited for participation in the SmartCoach program. Using generalized linear mixed models (GLMMs), we examined predictors of both program participation and program use at the individual and school class levels.

RESULTS

In total, 315 (66.2%) of the present 476 adolescents gave their active consent and provided the necessary information to be included in the program. None of the individual sociodemographic characteristics significantly predicted program participation, however, the participation rate was significantly higher in upper secondary school classes (84%) than secondary school classes (59%). The mean number of interactions with the program was 15.9, i.e., participants took part in almost half of the 34 possible interactions with the SmartCoach program. None of the baseline characteristics on the level of the school class significantly predicted program use. On the level of the individual, the univariate models showed that, compared to the reference category of 14-year-old students, program use was significantly lower for students who were 16 or older. Furthermore, participants with a migration background or an origin from a non-German-speaking country showed significantly lower program use. Finally, students with a medium level of perceived stress showed higher program use compared to those with a low level of stress. Within the final multivariate model for program use, only the variable "origin from a non-German-speaking country" remained significant.

CONCLUSIONS

SmartCoach is an attractive offer for young people, in which two out of three young people who are invited in the classroom to participate do so. Among the program participants, the use of the program is acceptable, with an average of almost half of the content being worked on. There is potential for improvement in terms of recruitment, especially in school classes with a lower level of education. The most important starting point for improving program use lies in taking greater account of needs and wishes of students with non-German-speaking countries of origin.

Effectiveness of a Social Marketing Mix Intervention on Changing the Smoking Behavior of Santri in Traditional Islamic Boarding Schools in Indonesia

OBJECTIVES

This study investigated the effectiveness of the social marketing mix approach in increasing students' knowledge about smoking, promoting positive attitudes toward smoking cessation, and decreasing smoking behavior.

METHODS

This quantitative research study incorporated a quasi-experimental method with a pretest-posttest non-equivalent group design. Using the purposive sampling technique, 152 smoking students were selected as participants. They were divided into 2 equal groups, with 76 students in the control group and 76 in the intervention group. The data were collected using questionnaires and analyzed with the chi-square test, independent t-test, Wilcoxon signed-rank test, and Mann-Whitney U-test.

RESULTS

The social marketing mix intervention was effective in increasing the students' knowledge about smoking (p<0.001), improving their attitude toward smoking cessation (p<0.001), and reducing their smoking behavior (p=0.014).

CONCLUSIONS

This approach should be implemented by local governments to reduce smoking behavior in the community, especially among teenagers, in addition to instituting a smoking ban and applying fines.

Designing, Developing, Evaluating, and Implementing a Smartphone-Delivered, Rule-Based Conversational Agent (DISCOVER): Development of a Conceptual Framework

BACKGROUND

Conversational agents (CAs), also known as chatbots, are computer programs that simulate human conversations by using predetermined rule-based responses or artificial intelligence algorithms. They are increasingly used in health care, particularly via smartphones. There is, at present, no conceptual framework guiding the development of smartphone-based, rule-based CAs in health care. To fill this gap, we propose structured and tailored guidance for their design, development, evaluation, and implementation.

OBJECTIVE

The aim of this study was to develop a conceptual framework for the design, evaluation, and implementation of smartphone-delivered, rule-based, goal-oriented, and text-based CAs for health care.

METHODS

We followed the approach by Jabareen, which was based on the grounded theory method, to develop this conceptual framework. We performed 2 literature reviews focusing on health care CAs and conceptual frameworks for the development of mobile health interventions. We identified, named, categorized, integrated, and synthesized the information retrieved from the literature reviews to develop the conceptual framework. We then applied this framework by developing a CA and testing it in a feasibility study.

RESULTS

The Designing, Developing, Evaluating, and Implementing a Smartphone-Delivered, Rule-Based Conversational Agent (DISCOVER) conceptual framework includes 8 iterative steps grouped into 3 stages, as follows: design, comprising defining the goal, creating an identity, assembling the team, and selecting the delivery interface; development, including developing the content and building the conversation flow; and the evaluation and implementation of the CA. They were complemented by 2 cross-cutting considerations-user-centered design and privacy and security-that were relevant at all stages. This conceptual framework was successfully applied in the development of a CA to support lifestyle changes and prevent type 2 diabetes.

CONCLUSIONS

Drawing on published evidence, the DISCOVER conceptual framework provides a step-by-step guide for developing rule-based, smartphone-delivered CAs. Further evaluation of this framework in diverse health care areas and settings and for a variety of users is needed to demonstrate its validity. Future research should aim to explore the use of CAs to deliver health care interventions, including behavior change and potential privacy and safety concerns.

Engagement With a Mobile Phone-Based Life Skills Intervention for Adolescents and Its Association With Participant Characteristics and Outcomes: Tree-Based Analysis

Background

Mobile phone–delivered life skills programs are an emerging and promising way to promote mental health and prevent substance use among adolescents, but little is known about how adolescents actually use them.

Objective

The aim of this study is to determine engagement with a mobile phone–based life skills program and its different components, as well as the associations of engagement with adolescent characteristics and intended substance use and mental health outcomes.

Methods

We performed secondary data analysis on data from the intervention group (n=750) from a study that compared a mobile phone–based life skills intervention for adolescents recruited in secondary and upper secondary school classes with an assessment-only control group. Throughout the 6-month intervention, participants received 1 SMS text message prompt per week that introduced a life skills topic or encouraged participation in a quiz or individual life skills training or stimulated sharing messages with other program participants through a friendly contest. Decision trees were used to identify predictors of engagement (use and subjective experience). The stability of these decision trees was assessed using a resampling method and by graphical representation. Finally, associations between engagement and intended substance use and mental health outcomes were examined using logistic and linear regression analyses.

Results

The adolescents took part in half of the 50 interactions (mean 23.6, SD 15.9) prompted by the program, with SMS text messages being the most used and contests being the least used components. Adolescents who did not drink in a problematic manner and attended an upper secondary school were the ones to use the program the most. Regarding associations between engagement and intended outcomes, adolescents who used the contests more frequently were more likely to be nonsmokers at follow-up than those who did not (odds ratio 0.86, 95% CI 0.76-0.98; P=.02). In addition, adolescents who read the SMS text messages more attentively were less likely to drink in a problematic manner at follow-up (odds ratio 0.43, 95% CI 1.29-3.41; P=.003). Finally, participants who used the program the most and least were more likely to increase their well-being from baseline to 6-month follow-up compared with those with average engagement (βs=.39; t₅₈₆=2.66; P=.008; R²=0.24).

Conclusions

Most of the adolescents participating in a digital life skills program that aimed to prevent substance use and promote mental health engaged with the intervention. However, measures to increase engagement in problem drinkers should be considered. Furthermore, efforts must be made to ensure that interventions are engaging and powerful across different educational levels. First results indicate that higher engagement with digital life skills programs could be associated with intended outcomes. Future studies should apply further measures to improve the reach of lower-engaged participants at follow-up to establish such associations with certainty.

Understanding health disparities affecting utilization of tobacco treatment in low-income patients in an urban health center in Southern California

Tobacco use disproportionately affects low-income communities. Prevalence among patients in Federally Qualified Health Centers (FQHCs) is higher (29.3%) than the general population (20%). Little is known about the rates of referrals to cessation services and cessation pharmacotherapy practices in FQHCs. This study will examine referral and prescribing patterns based on patient characteristics at a large FQHC in Southern California. We conducted a retrospective analysis of EHR data from 2019. Patients who were ≥ 18 years old and had "tobacco use" as an active problem were included in analyses. We characterized the proportion of 1) those who were referred to California Smokers' Helpline (CSH), 2) referred to smoking cessation counseling (SCC) at the FQHC clinic, or 3) received pharmacotherapy. Associations of demographic characteristics and comorbidities with referral types and uptake of services were evaluated using mixed-effects multinomial and logistic regressions. Of the 20,119 tobacco users identified, 87% had some cessation intervention: 66% were advised to quit and given information to contact CSH, while 21% were referred to SCC. Patients were least likely to get referred to cessation services if they had more medical, psychiatric, or substance use comorbidities, were in the lowest income group, were uninsured or were Hispanic. Although EHR systems have enhanced the ease of screening, most patients do not receive more than brief advice to quit during a PCP visit. Most (70%) low-income smokers see their PCPs at least once a year, making FQHCs excellent settings to promote smoking cessation initiatives in low-income populations.

Contextual Predictors of Engagement in a Tailored mHealth Intervention for Adolescent and Young Adult Cancer Survivors

BACKGROUND

Despite the promise of mobile health (mHealth), engagement is often too low for durable health behavior change, and little is known regarding why certain individuals abandon mHealth tools.

PURPOSE

Guided by a mHealth engagement framework, we evaluated contextual predictors of objective engagement with an app for adolescents and young adults (AYA) who survived cancer.

METHODS

One hundred and ten AYA survivors (M age = 20.5, 43% female, 30% racial/ethnic minority) were randomized to receive a disease self-management app that delivered 1-2 tailored messages/day for 16 weeks, and contained a survivorship care plan (SCP). Demographic, disease, psychosocial, and setting characteristics were examined as predictors of three objective engagement outcomes: (a) % of active app days, (b) % of messages read, and (c) viewed SCP in the app versus not. A subsample (n = 10) completed qualitative interviews to further assess engagement barriers.

RESULTS

Self-reported uninterrupted app access (β = -0.56, p < .001), iPhone (vs. Android) ownership (β = 0.30, p < .001), and receiving the intervention in the summer (β = -0.20, p = .01) predicted more active days. Lower depressed mood (β = -0.30, p = .047) and uninterrupted app access (β = -0.50, p < .001) predicted more messages read. Qualitatively, technical glitches and competing priorities were described as engagement barriers, whereas certain types of messages (e.g., health goal messages) were perceived as engaging. Among participants who had uninterrupted app access (n = 76), higher baseline motivation to change, better health perceptions, using the app during the summer, and iPhone ownership predicted higher engagement.

CONCLUSIONS

Findings demonstrate the importance of comprehensively assessing and planning for multi-level ecological determinants of mHealth engagement in future trials.

CLINICALTRIALS.GOV IDENTIFIER

NCT03363711.

A Mobile Phone-Based Life-Skills Training Program for Substance Use Prevention Among Adolescents: Cluster-Randomized Controlled Trial

Background

Life skills are abilities for adaptive and positive behavior that enable individuals to deal effectively with the demands and challenges of everyday life. Life-skills training programs conducted within the school curriculum are effective in preventing the onset and escalation of substance use among adolescents. However, their dissemination is impeded due to their large resource requirements. Life-skills training provided via mobile phones may provide a more economic and scalable approach.

Objective

The goal of this study was to test the appropriateness (ie, acceptance, use, and evaluation) and short-term efficacy of a mobile phone–based life-skills training program to prevent substance use among adolescents within a controlled trial.

Methods

The study design was a two-arm, parallel-group, cluster-randomized controlled trial with assessments at baseline and follow-up assessments after 6 and 18 months. This report includes outcomes measured up to the 6-month follow-up. The efficacy of the intervention was tested in comparison to an assessment-only control group. The automated intervention program SmartCoach included online feedback and individually tailored text messages provided over 22 weeks. The contents were based on social cognitive theory and addressed self-management skills, social skills, and substance use resistance skills. Linear mixed models and generalized linear mixed models, as well as logistic or linear regressions, were used to investigate changes between baseline and 6-month follow-up in the following outcomes: 30-day prevalence rates of problem drinking, tobacco use, and cannabis use as well as quantity of alcohol use, quantity of cigarettes smoked, cannabis use days, perceived stress, well-being, and social skills.

Results

A total of 1759 students from 89 Swiss secondary and upper secondary school classes were invited to participate in the study. Of these, 1473 (83.7%) students participated in the study; the mean age was 15.4 years (SD 1.0) and 55.2% (813/1473) were female. Follow-up assessments at 6 months were completed by 1233 (83.7%) study participants. On average, program participants responded to half (23.6 out of 50) of the prompted activities. Program evaluations underlined its appropriateness for the target group of secondary school students, with the majority rating the program as helpful and individually tailored. The results concerning the initial effectiveness of this program based on 6-month follow-up data are promising, with three of nine outcomes of the intention-to-treat analyses showing beneficial developments of statistical significance (ie, quantity of alcohol use, quantity of tobacco use, and perceived stress; P<.05) and another three outcomes (ie, problem drinking prevalence, cannabis use days, and social skills) showing beneficial developments of borderline significance (P<.10).

Conclusions

The results showed good acceptance of this intervention program that could be easily and economically implemented in school classes. Initial results on program efficacy indicate that it might be effective in both preventing or reducing substance use and fostering life skills; however, data from the final 18-month follow-up assessments will be more conclusive.

Trial Registration

ISRCTN Registry ISRCTN41347061; https://doi.org/10.1186/ISRCTN41347061

Identifying Content-Based Engagement Patterns in a Smoking Cessation Website and Associations With User Characteristics and Cessation Outcomes: A Sequence and Cluster Analysis

INTRODUCTION

Using WebQuit as a case study, a smoking cessation website grounded in Acceptance and Commitment Therapy, we aimed to identify sequence clusters of content usage and examine their associations with baseline characteristics, change to a key mechanism of action, and smoking cessation.

METHODS

Participants were adult smokers allocated to the WebQuit arm in a randomized controlled trial (n = 1,313). WebQuit contains theory-informed content including goal setting, self-monitoring and feedback, and values- and acceptance-based exercises. Sequence analysis was used to temporally order 30-s website usage segments for each participant. Similarities between sequences were assessed with the optimal matching distance algorithm and used as input in an agglomerative hierarchical clustering analysis. Associations between sequence clusters and baseline characteristics, acceptance of cravings at 3 months and self-reported 30-day point prevalence abstinence at 12 months were examined with linear and logistic regression.

RESULTS

Three qualitatively different sequence clusters were identified. "Disengagers" (576/1,313) almost exclusively used the goal-setting feature. "Tryers" (375/1,313) used goal setting and two of the values- and acceptance-based components ("Be Aware," "Be Willing"). "Committers" (362/1,313) primarily used two of the values- and acceptance-based components ("Be Willing," "Be Inspired"), goal setting, and self-monitoring and feedback. Compared with Disengagers, Committers demonstrated greater increases in acceptance of cravings (p = .01) and 64% greater odds of quit success (ORadj = 1.64, 95% CI = 1.18, 2.29, p = .003).

DISCUSSION

WebQuit users were categorized into Disengagers, Tryers, and Committers based on their qualitatively different content usage patterns. Committers saw increases in a key mechanism of action and greater odds of quit success.

IMPLICATIONS

This case study demonstrates how employing sequence and cluster analysis of usage data can help researchers and practitioners gain a better understanding of how users engage with a given eHealth intervention over time and use findings to test theory and/or to improve future iterations to the intervention. Future WebQuit users may benefit from being directed to the values- and acceptance-based and the self-monitoring and feedback components via reminders over the course of the program.

Engaging youth in mHealth: what works and how can we be sure?

BACKGROUND

Youth participating in mobile health (mHealth) intervention trials often engage with the technologies [e.g., applications (app) or mobile-optimized websites] only partially, often prematurely discontinuing use altogether. Limited engagement can impact the interventions effect on behavior change and compromise researchers' ability to test and estimate the true efficacy of their interventions. While mHealth interventions have been shown to be feasible and acceptable to youth, across diverse health conditions, strategies to increase engagement have been less well studied. Specifically, within HIV prevention and care mHealth interventions, there is not consensus as to which components represent the "key ingredients" to support maximal engagement of youth. Further, successful intervention evaluation requires the ability to systematically track users' engagement with intervention components (i.e., paradata) to evaluate its effects on behavior change.

METHODS

As part of the Adolescent Medicine Trials Network UNC/Emory Center for Innovative Technology (iTech) portfolio of HIV/AIDS Interventions, we present diverse strategies used across five mHealth protocols seeking to promote youth engagement, track and measure engagement through paradata, and incorporate these components into mHealth intervention evaluations.

RESULTS

We describe the importance of defining and measuring engagement using case studies from iTech to illustrate how different research teams select mHealth features to promote youth engagement over time, taking into account features embedded in the technology design, key mechanisms of change and trial outcomes (e.g., HIV testing, pre-exposure prophylaxis uptake and adherence, HIV treatment adherence). Finally, we discuss how the research teams plan to evaluate engagement's role on their intervention's outcomes.

CONCLUSIONS

Based on this synthesis, we discuss strategies to enhance mHealth engagement during intervention development and design, ensure its monitoring and reporting throughout the trial, and evaluate its impact on trial outcomes.

Efficacy of a smartphone-based coaching program for addiction prevention among apprentices: study protocol of a cluster-randomised controlled trial

BACKGROUND

A large proportion of apprentices shows addictive behaviours like cigarette smoking, alcohol, cannabis, or compulsive Internet use, others do not show such behaviours at all. ready4life is a smartphone application-based coaching program for apprentices, which takes into account the heterogeneity of adolescent addictive behaviour by promoting life skills and reducing risk behaviours. The main objective of the planned study is to test the efficacy of ready4life for addiction prevention among apprentices in Switzerland within a controlled trial.

METHODS/DESIGN

The efficacy of the ready4life coaching program will be tested in comparison to an assessment only control group, within a cluster-randomised controlled trial with one follow-up assessment after 6 months. At the beginning of the program, participants of the intervention group will receive an individual profile, showing areas in which they have sufficient resources and in which there is a need for coaching. Based on this feedback, they can select two out of the following six program modules: stress, social skills, Internet use, tobacco/e-cigarettes, cannabis, and alcohol. Participants of the intervention group will receive individualised coaching by a conversational agent (chatbot) for a period of four months. The coaching relies on motivational and social-cognitive principles of behaviour change. Within weekly dialogues, the coach provides individually tailored information in different formats, such as videoclips, texts, or pictures. Study participants will be 1318 apprentices with a minimum age of 15, recruited in approximately 100 vocational school classes in Switzerland. Primary outcome will be a composite measure for addictive behaviours including (1) at risk-drinking, (2) tobacco/e-cigarette smoking, (3) cannabis use, and (4) problematic Internet use.

DISCUSSION

The study will reveal whether this universally implementable but individually tailored intervention approach is effective in preventing the onset and escalation of addictive behaviors among apprentices.

TRIAL REGISTRATION

ISRCTN59908406 (registration date: 21/10/2020).

The Put It Out Project (POP) Facebook Intervention for Young Sexual and Gender Minority Smokers: Outcomes of a Pilot, Randomized, Controlled Trial

INTRODUCTION

This trial investigated whether a Facebook smoking cessation intervention culturally tailored to young sexual and gender minority (SGM) smokers (versus non-tailored) would increase smoking abstinence.

METHODS

Participants were 165 SGM young adult US smokers (age 18-25) recruited from Facebook in April 2018 and randomized to an SGM-tailored (POP; N = 84) or non-tailored (TSP-SGM; N = 81) intervention. Interventions delivered weekly live counseling sessions and 90 daily Facebook posts to participants in Facebook groups. Primary analyses compared POP and TSP-SGM on biochemically verified smoking abstinence (yes/no; primary outcome), self-reported 7-day point prevalence abstinence (yes/no), reduction in cigarettes per week by 50+% from baseline (yes/no), making a quit attempt during treatment (yes/no), and stage of change (precontemplation/contemplation vs. preparation/action). Supplemental analyses compared POP to two historical control groups.

RESULTS

POP participants were more likely than TSP-SGM participants to report smoking abstinence at 3 (23.8% vs. 12.3%; OR = 2.50; p = .03) and 6 months (34.5% vs. 12.3%; OR = 4.06; p < .001) and reduction in smoking at 3 months (52.4% vs. 39.5%; OR = 2.11; p = .03). Biochemically verified smoking abstinence did not significantly differ between POP and TSP-SGM at 3 (OR = 2.00; p = .33) or 6 months (OR = 3.12; p = .08), potentially due to challenges with remote biochemical verification. In supplemental analyses, POP participants were more likely to report abstinence at 3 (OR = 6.82, p = .01) and 6 (OR = 2.75, p = .03) months and reduced smoking at 3 months (OR = 2.72, p = .01) than participants who received a referral to Smokefree.gov.

CONCLUSIONS

This pilot study provides preliminary support for the effectiveness of a Facebook smoking cessation intervention tailored to SGM young adults.

IMPLICATIONS

SGM individuals have disproportionately high smoking prevalence. It is unclear whether smoking cessation interventions culturally tailored to the SGM community are more effective than non-tailored interventions. This pilot trial found preliminary evidence that an SGM-tailored Facebook smoking cessation intervention increased reported abstinence from smoking, compared to a non-tailored intervention.

TRIAL REGISTRATION

NCT03259360.

Predicting Outcomes from Engagement With Specific Components of an Internet-Based Physical Activity Intervention With Financial Incentives: Process Analysis of a Cluster Randomized Controlled Trial

Background

Investigating participant engagement and nonusage attrition can help identify the likely active ingredients of electronic health interventions. Research on engagement can identify which intervention components predict health outcomes. Research on nonusage attrition is important to make recommendations for retaining participants in future studies.

Objective

This study aimed to investigate engagement and nonusage attrition in the Physical Activity Loyalty (PAL) scheme, a 6-month complex physical activity intervention in workplaces in Northern Ireland. The intervention included financial incentives with reward redemption and self-regulation techniques. Specific objectives were (1) to determine whether engagement in specific intervention components predicted physical activity at 6 months, (2) to determine whether engagement in specific intervention components predicted targeted mediators at 6 months, and (3) to investigate predictors of nonusage attrition for participants recording daily activity via the PAL scheme physical activity monitoring system and logging onto the website.

Methods

Physical activity was assessed at baseline and 6 months using pedometers (Yamax Digiwalker CW-701, Japan). Markers of engagement and website use, monitoring system use, and reward redemption were collected throughout the scheme. Random-effects generalized least-squares regressions determined whether engagement with specific intervention components predicted 6-month physical activity and mediators. Cox proportional hazards regressions were used to investigate predictors of nonusage attrition (days until first 2-week lapse).

Results

A multivariable generalized least-squares regression model (n=230) showed that the frequency of hits on the website’s monitoring and feedback component (regression coefficient [b]=50.2; SE=24.5; P=.04) and the percentage of earned points redeemed for financial incentives (b=9.1; SE=3.3; P=.005) were positively related to 6-month pedometer steps per day. The frequency of hits on the discussion forum (b=−69.3; SE=26.6; P=.009) was negatively related to 6-month pedometer steps per day. Reward redemption was not related to levels of more internal forms of motivation. Multivariable Cox proportional hazards regression models identified several baseline predictors associated with nonusage attrition. These included identified regulation (hazard ratio [HR] 0.88, 95% CI 0.81-0.97), recovery self-efficacy (HR 0.88, 95% CI 0.80-0.98), and perceived workplace environment safety (HR 1.07, 95% CI 1.02-1.11) for using the physical activity monitoring system. The EuroQoL health index (HR 0.33, 95% CI 0.12-0.91), financial motivation (HR 0.93, 95% CI 0.87-0.99), and perceived availability of physical activity opportunities in the workplace environment (HR 0.96, 95% CI 0.93-0.99) were associated with website nonusage attrition.

Conclusions

Our results provide evidence opposing one of the main hypotheses of self-determination theory by showing that financial rewards are not necessarily associated with decreases in more internal forms of motivation when offered as part of a complex multicomponent intervention. Identifying baseline predictors of nonusage attrition can help researchers to develop strategies to ensure maximum intervention adherence.

Trial Registration

ISRCTN Registry ISRCTN17975376; http://www.isrctn.com/ISRCTN17975376 (Archived by WebCite at http://www.webcitation.org/76VGZsZug)

Text Message Responsivity in a 2-Way Short Message Service Pilot Intervention With Adolescent and Young Adult Survivors of Cancer

OBJECTIVE

Within a 2-way text messaging study in AYAs who recently completed treatment for cancer, we sought to evaluate text message responsivity across different types of text messages.

METHODS

AYAs who recently completed treatment for cancer (n=26; mean age=16 years; 62% female, 16/26 participants) received 2-way text messages about survivorship health topics over a 16-week period. Using participants' text message log data, we coded responsivity to text messages and evaluated trends in responsivity to unprompted text messages and prompted text messages of varying content (eg, medication reminders, appointment reminders, and texts about personal experiences as a cancer survivor).

RESULTS

Across prompted and unprompted text messages, responsivity rapidly decreased (P ≤.001 and =.01, respectively) and plateaued by the third week of the intervention. However, participants were more responsive to prompted text messages (mean responsivity=46% by week 16) than unprompted messages (mean responsivity=10% by week 16). They also demonstrated stable responsivity to certain prompted content: medication reminders, appointment reminders, goal motivation, goal progress, and patient experience texts.

CONCLUSIONS

Our methodology of evaluating text message responsivity revealed important patterns of engagement in a 2-way text message intervention for AYA cancer survivors.

A Framework for Analyzing and Measuring Usage and Engagement Data (AMUsED) in Digital Interventions: Viewpoint

Trials of digital interventions can yield extensive, in-depth usage data, yet usage analyses tend to focus on broad descriptive summaries of how an intervention has been used by the whole sample. This paper proposes a novel framework to guide systematic, fine-grained usage analyses that better enables understanding of how an intervention works, when, and for whom. The framework comprises three stages to assist in the following: (1) familiarization with the intervention and its relationship to the captured data, (2) identification of meaningful measures of usage and specifying research questions to guide systematic analyses of usage data, and (3) preparation of datasheets and consideration of available analytical methods with which to examine the data. The framework can be applied to inform data capture during the development of a digital intervention and/or in the analysis of data after the completion of an evaluation trial. We will demonstrate how the framework shaped preparation and aided efficient data capture for a digital intervention to lower transmission of cold and flu viruses in the home, as well as how it informed a systematic, in-depth analysis of usage data collected from a separate digital intervention designed to promote self-management of colds and flu. The Analyzing and Measuring Usage and Engagement Data (AMUsED) framework guides systematic and efficient in-depth usage analyses that will support standardized reporting with transparent and replicable findings. These detailed findings may also enable examination of what constitutes effective engagement with particular interventions.

Investigating Intervention Components and Exploring States of Receptivity for a Smartphone App to Promote Physical Activity: Protocol of a Microrandomized Trial

Background

Smartphones enable the implementation of just-in-time adaptive interventions (JITAIs) that tailor the delivery of health interventions over time to user- and time-varying context characteristics. Ideally, JITAIs include effective intervention components, and delivery tailoring is based on effective moderators of intervention effects. Using machine learning techniques to infer each user’s context from smartphone sensor data is a promising approach to further enhance tailoring.

Objective

The primary objective of this study is to quantify main effects, interactions, and moderators of 3 intervention components of a smartphone-based intervention for physical activity. The secondary objective is the exploration of participants’ states of receptivity, that is, situations in which participants are more likely to react to intervention notifications through collection of smartphone sensor data.

Methods

In 2017, we developed the Assistant to Lift your Level of activitY (Ally), a chatbot-based mobile health intervention for increasing physical activity that utilizes incentives, planning, and self-monitoring prompts to help participants meet personalized step goals. We used a microrandomized trial design to meet the study objectives. Insurees of a large Swiss insurance company were invited to use the Ally app over a 12-day baseline and a 6-week intervention period. Upon enrollment, participants were randomly allocated to either a financial incentive, a charity incentive, or a no incentive condition. Over the course of the intervention period, participants were repeatedly randomized on a daily basis to either receive prompts that support self-monitoring or not and on a weekly basis to receive 1 of 2 planning interventions or no planning. Participants completed a Web-based questionnaire at baseline and postintervention follow-up.

Results

Data collection was completed in January 2018. In total, 274 insurees (mean age 41.73 years; 57.7% [158/274] female) enrolled in the study and installed the Ally app on their smartphones. Main reasons for declining participation were having an incompatible smartphone (37/191, 19.4%) and collection of sensor data (35/191, 18.3%). Step data are available for 227 (82.8%, 227/274) participants, and smartphone sensor data are available for 247 (90.1%, 247/274) participants.

Conclusions

This study describes the evidence-based development of a JITAI for increasing physical activity. If components prove to be efficacious, they will be included in a revised version of the app that offers scalable promotion of physical activity at low cost.

Trial Registration

ClinicalTrials.gov NCT03384550; https://clinicaltrials.gov/ct2/show/NCT03384550 (Archived by WebCite at http://www.webcitation.org/74IgCiK3d)

International Registered Report Identifier (IRRID)

DERR1-10.2196/11540

Is More Always Better?: Discovering Incentivized mHealth Intervention Engagement Related to Health Behavior Trends

Behavioral medicine is devoting increasing attention to the topic of participant engagement and its role in effective mobile health (mHealth) behavioral interventions. Several definitions of the term "engagement" have been proposed and discussed, especially in the context of digital health behavioral interventions. We consider that engagement refers to specific interaction and use patterns with the mHealth tools such as smartphone applications for intervention, whereas adherence refers to compliance with the directives of the health intervention, independent of the mHealth tools. Through our analysis of participant interaction and self-reported behavioral data in a college student health study with incentives, we demonstrate an example of measuring "effective engagement" as engagement behaviors that can be linked to the goals of the desired intervention. We demonstrate how clustering of one year of weekly health behavior self-reports generate four interpretable clusters related to participants' adherence to the desired health behaviors: healthy and steady, unhealthy and steady, decliners, and improvers. Based on the intervention goals of this study (health promotion and behavioral change), we show that not all app usage metrics are indicative of the desired outcomes that create effective engagement. As such, mHealth intervention design might consider eliciting not just more engagement or use overall, but rather, effective engagement defined by use patterns related to the desired behavioral outcome.

Effective Behavioral Changes through a Digital mHealth App: Exploring the Impact of Hedonic Well-Being, Psychological Empowerment and Inspiration

Background

New mobile health (mHealth) software apps are emerging and are providing the foundation to radically transform the practice and reach of medical research and care. For this study we collaborated with Quit Genius, a cognitive behavioral therapy (CBT) based mHealth app that helps users quit smoking, to explore the effective design of a digital mHealth app; one that delivers important benefits to its users and helps them change their behaviors for a healthier lifestyle.

Objective

The specific aims of this study were to (1) explore the key role of CBT program progress, (2) examine the gamification design app elements that deliver significant benefits (ie, empowerment, well-being, inspiration) to users, (3) explore the effectiveness of these app elements to help users quit smoking or reduce the number of cigarettes smoked, and (4) identify and describe any potential drivers and hindrances arising from the app design elements.

Methods

We developed an online survey and sent an email invitation to 4144 individuals, who had previously or were at the time using the Quit Genius mHealth app, to encourage participation in the study. We matched the online survey data with objective app usage data of the study participants.

Results

A dataset of 190 completed responses was used. At the time of the survey, respondents had completed an average of 60% of the CBT program in the Quit Genius mHealth app. Of the respondents, 36.3% (69/190) noted to have quit smoking successfully after using the Quit Genius app. As for those who remained smokers after using the app (N=121), the number of cigarettes smoked per day was significantly reduced by 59.6%. The ability of the app to enhance users’ hedonic well-being and psychologically empower them in their daily lives was identified as being key in helping users quit smoking. Specifically, the results show that users whose well-being was enhanced through the app were 1.72 times more likely to quit smoking successfully. Moreover, a one-unit increase on a 7-point Likert scale in the app’s ability to empower smokers in their daily lives led to a reduction of cigarettes smoked per day of 53%. The app’s inspiration to users, however, was negatively associated with quitting success and the reduction in cigarette smoked per day.

Conclusions

The findings offer important insights for the effective design of digital mHealth apps. Specifically, we find that perceived psychological empowerment and enhanced hedonic well-being from the mobile solution may be a more impactful way to support the effectiveness of mobile cognitive behavioral therapy for smoking cessation than eliciting strong inspiration.