Predicting Success of a Digital Self-Help Intervention for Alcohol and Substance Use With Machine Learning

Effectiveness of Machine Learning-Based Adjustments to an eHealth Intervention Targeting Mild Alcohol Use

INTRODUCTION

This study aimed to evaluate effects of three machine learning based adjustments made to an eHealth intervention for mild alcohol use disorder, regarding (a) early dropout, (b) participation duration, and (c) success in reaching personal alcohol use goals. Additionally, we aimed to replicate earlier machine learning analyses.

METHODS

We used three cohorts of observational log data from the Jellinek Digital Self-help intervention. First, a cohort before implementation of adjustments (T0; n = 320); second, a cohort after implementing two adjustments (i.e., sending daily emails in the first week and nudging participants towards a "no alcohol use" goal; T1; n = 146); third, a cohort comprising the prior adjustments complemented with eliminated time constraints to reaching further in the intervention (T2; n = 236).

RESULTS

We found an increase in participants reaching further in the intervention, yet an increase in early dropout after implementing all adjustments. Moreover, we found that more participants aimed for a quit goal, whilst participation duration declined at T2. Intervention success increased, yet not significantly. Lastly, machine learning demonstrated reliability for outcome prediction in smaller datasets of an eHealth intervention.

CONCLUSION

Strong correlates as indicated by machine learning analyses were found to affect goal setting and use of an eHealth program for alcohol use problems.

Discovering what young adults want in electronic interventions aimed at reducing alcohol-related consequences

BACKGROUND

Despite intervention efforts, negative alcohol-related consequences continue to impact young adults. Most alcohol interventions focus on reducing alcohol consumption; however, previous research indicates that focusing solely on alcohol use may not decrease consequences. Additionally, many alcohol interventions have diminishing engagement, and few are designed with young adults involved in the development process. Drawing on user-centered design, this study sought to understand young adult perceptions, preferences, and needs for electronic interventions specifically aimed at reducing alcohol consequences.

METHODS

Using semi-structured qualitative interviews, 21 young adult drinkers (ages 18-24; 57.1% female) shared their opinions regarding the need for electronic interventions (i.e., mobile or web-delivered) to reduce alcohol consequences as well as their preferences for content, features, and ways to increase engagement. Interviews were coded and analyzed using a multi-step thematic analysis approach.

RESULTS

As part of our discovery phase of intervention development, content coding revealed four main themes. Participants perceived several benefits of interventions focused on alcohol consequences, such as promoting mindful alcohol use and reducing alcohol-related harms. Participants also discussed perceived limitations of such programs, including believing consequences from drinking are unavoidable, necessary for learning, and associated with peer pressure. Preferences for features included real-time tracking, personalized feedback, and psychoeducation along with preferences for design including non-judgmental framing, interactive content, and a user-friendly platform.

CONCLUSIONS

Engaging end users early in the development process is a valuable approach to increase intervention relevancy with the target population. This can also inform intervention content and design to maximize engagement and satisfaction (e.g., framing, features, and interactivity) while also reducing barriers identified early on (e.g., peer pressure).

Predicting cannabis use moderation among a sample of digital self-help subscribers: A machine learning study

BACKGROUND

For individuals who wish to reduce their cannabis use without formal help, there are a variety of self-help tools available. Although some are proven to be effective in reducing cannabis use, effect sizes are typically small. More insight into predictors of successful reduction of use among individuals who frequently use cannabis and desire to reduce/quit could help identify factors that contribute to successful cannabis use moderation.

METHODS

We analyzed data taken from a randomized controlled trial comparing the effectiveness of the digital cannabis intervention ICan to four online modules of educational information on cannabis. For the current study, we included 253 participants. Success was defined as reducing the grams of cannabis used in the past 7 days at baseline by at least 50 % at 6-month follow-up. To train and evaluate the machine learning models we used a nested k-fold cross-validation procedure.

RESULTS

The results show that the two models applied had comparable low AUROC values of .61 (Random Forest) and .57 (Logistic Regression). Not identifying oneself as a cannabis user, not using tobacco products, high levels of depressive symptoms, high levels of psychological distress and high initial cannabis use values were the relatively most important predictors for success, although overall the associations were not strong.

CONCLUSIONS

Our study found only modest prediction accuracy when using machine learning models to predict success among individuals who use cannabis and desire to reduce/quit and show interest in digital self-help tools.

Evaluation and Future Challenges in a Self-Guided Web-Based Intervention With and Without Chat Support for Depression and Anxiety Symptoms During the COVID-19 Pandemic: Randomized Controlled Trial

BACKGROUND

The COVID-19 pandemic has had an impact on mental health worldwide. Low- and middle-income countries were largely affected by it. Mexico was one of the most affected countries. Extended periods of lockdowns, isolation, and social distancing, among other factors, highlighted the need to introduce web-based psychological interventions to the Mexican population. In this context, Mental Health COVID-19 emerged as a self-guided web-based intervention (SGWI) aimed at adults to improve mental health during the COVID-19 pandemic.

OBJECTIVE

This study aims to assess the efficacy of 2 modalities of a self-guided intervention (with and without chat support) in reducing depression symptoms, generalized anxiety, community posttraumatic stress, widespread fear, anxiety, sleep quality, physiological and affective coping, and suicide ideation. In addition, it aimed to compare the moderating role of coping strategies, acceptance, and satisfaction in participants' symptom reduction. We hypothesize that the self-guided, chat-supported modality will show higher efficacy than the modality without chat support in achieving clinical change and better performance as a moderator of depression symptoms, generalized anxiety, community posttraumatic stress, widespread fear, anxiety, sleep quality, physiological and affective coping, and suicide ideation, as well as an increase in participants' satisfaction and acceptability.

METHODS

A randomized controlled trial was conducted. Data were collected from May 2020 to June 2022. We performed intrasubject measures at 4 evaluation periods: pretest, posttest, and follow-up measurements at 3 and 6 months. Differences between intervention groups were assessed through the Mann-Whitney U test for continuous variables and the chi-square test for categorical variables. Changes due to intervention were analyzed using Wilcoxon W test. Moderated regression analysis was performed to test the hypothesized moderating role of coping strategies, usability, and opinion about treatment on clinical change.

RESULTS

A total of 36 participants completed the intervention; of these, 5 (14%) were part of the SGWI group, and 31 (86%) were on the SGWI plus chat support (SGWI+C) group, which included a chat service with therapists. The perceived high complexity of the system for the SGWI group had a moderating effect associated with a lack of efficacy of the intervention regarding depression, but not when controlled for sociodemographic variables. A perception of lower helpfulness of the intervention was associated with poorer outcomes. Coping strategies did not show moderating effects.

CONCLUSIONS

Enhancing the utility of web-based interventions for reducing clinical symptoms by incorporating a support chat to boost treatment adherence seemed to improve the perception of the intervention's usefulness. Web-based interventions face several challenges, such as eliminating complexities in platform use and increasing the users' perceived utility of the intervention, among other issues identified in the study.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04468893; https://clinicaltrials.gov/study/NCT04468893?tab=results.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/23117.

Digital Smoking Cessation Intervention for Cancer Survivors: Analysis of Predictors and Moderators of Engagement and Outcome Alongside a Randomized Controlled Trial

BACKGROUND

Recent studies have shown positive, though small, clinical effects of digital smoking cessation (SC) interventions for cancer survivors. However, research on associations among participant characteristics, intervention engagement, and outcomes is limited.

OBJECTIVE

This study aimed to explore the predictors and moderators of engagement and outcome of MyCourse-Quit Smoking (in Dutch: "MijnKoers-Stoppen met Roken"), a digital minimally guided intervention for cancer survivors.

METHODS

A secondary analysis of data from the randomized controlled trial was performed. The number of cigarettes smoked in the past 7 days at 6-month follow-up was the primary outcome measure. We analyzed interactions among participant characteristics (11 variables), intervention engagement (3 variables), and outcome using robust linear (mixed) modeling.

RESULTS

In total, 165 participants were included in this study. Female participants accessed the intervention less often than male participants (B=-11.12; P=.004). A higher Alcohol Use Disorders Identification Test score at baseline was associated with a significantly higher number of logins (B=1.10; P<.001) and diary registrations (B=1.29; P<.001). A higher Fagerström Test for Nicotine Dependence score at baseline in the intervention group was associated with a significantly larger reduction in tobacco use after 6 months (B=-9.86; P=.002). No other associations and no moderating effects were found.

CONCLUSIONS

Overall, a limited number of associations was found between participant characteristics, engagement, and outcome, except for gender, problematic alcohol use, and nicotine dependence. Future studies are needed to shed light on how this knowledge can be used to improve the effects of digital SC programs for cancer survivors.

TRIAL REGISTRATION

Netherlands Trial register NTR6011/NL5434; https://onderzoekmetmensen.nl/nl/trial/22832.

Artificial intelligence in preventive cardiology

Artificial intelligence (AI) is a field of study that strives to replicate aspects of human intelligence into machines. Preventive cardiology, a subspeciality of cardiovascular (CV) medicine, aims to target and mitigate known risk factors for CV disease (CVD). AI's integration into preventive cardiology may introduce novel treatment interventions and AI-centered clinician assistive tools to reduce the risk of CVD. AI's role in nutrition, weight loss, physical activity, sleep hygiene, blood pressure, dyslipidemia, smoking, alcohol, recreational drugs, and mental health has been investigated. AI has immense potential to be used for the screening, detection, and monitoring of the mentioned risk factors. However, the current literature must be supplemented with future clinical trials to evaluate the capabilities of AI interventions for preventive cardiology. This review discusses present examples, potentials, and limitations of AI's role for the primary and secondary prevention of CVD.

Evaluating the impact of motivational interviewing on engagement and outcomes in a web-based self-help intervention for gambling disorder: A randomised controlled trial

Background

Self-paced internet interventions for gambling problems offer cost-effective, accessible, and private alternatives to traditional psychotherapy for a population that rarely seeks help. However, these interventions have been relatively slow to develop, evaluate, and deploy at scale relative to those for other addictive behaviors. Moreover, user engagement remains low despite the high interest. Motivational interviews have improved the effectiveness gambling bibliotherapy but have not been augmented with an analogous web-based self-guided program.

Objectives

This trial aimed to replicate and extend prior work by translating a paperback workbook to the internet and pairing it with a single motivational interview. It was hypothesized that the motivational interview would enhance program engagement and gambling outcomes.

Methods

A two-arm randomised controlled trial was conducted. Treatment-seeking Canadian adults recruited solely via social media received one year of access to a web-based self-guided program, either alone (N = 158) or in combination with a virtual motivational interview completed upon enrolment (N = 155). The program was based on principles of cognitive-behavioral therapy and motivational interviewing. Gambling severity, expenditures, frequency, and duration were assessed via online questionnaires at baseline and 3-, 6-, and 12-months post-baseline, along with secondary outcomes (i.e., depression, anxiety, nonspecific psychological distress, alcohol consumption).

Results

Baseline characteristics were indicative of severe gambling problems and concurrent mental health problems but not problematic alcohol consumption in this sample. Both treatment groups demonstrated roughly equal improvements across all gambling outcomes and most secondary outcomes over time, except alcohol consumption, which did not meaningfully change. Changes were most prominent by 3 months, followed by more gradual change by 6 and 12 months. Only 57 % of gamblers who were assigned to receive a motivational interview completed that interview. About 40 % of users did not complete any program modules and 11 % completed all four. No group differences in program engagement were observed, although the number of modules completed was associated with greater reductions in gambling behaviors in both groups.

Discussion

The problem of user engagement with web-based self-help programs remains. There is a dose-response relationship between engagement and outcomes when engagement is measured in terms of therapeutic content completed.

Conclusions

The addition of a motivational interview to a web-based self-help program for gambling problems was unsuccessful in improving engagement or outcomes. Future work should aim to make self-guided programs more engaging rather than solely making users more engaged.

Trial registration

Registered on 7 July 2020 (ISRCTN13009468).

Predicting Adherence to Behavior Change Support Systems Using Machine Learning: Systematic Review

BACKGROUND

There is a dearth of knowledge on reliable adherence prediction measures in behavior change support systems (BCSSs). Existing reviews have predominately focused on self-reporting measures of adherence. These measures are susceptible to overestimation or underestimation of adherence behavior.

OBJECTIVE

This systematic review seeks to identify and summarize trends in the use of machine learning approaches to predict adherence to BCSSs.

METHODS

Systematic literature searches were conducted in the Scopus and PubMed electronic databases between January 2011 and August 2022. The initial search retrieved 2182 journal papers, but only 11 of these papers were eligible for this review.

RESULTS

A total of 4 categories of adherence problems in BCSSs were identified: adherence to digital cognitive and behavioral interventions, medication adherence, physical activity adherence, and diet adherence. The use of machine learning techniques for real-time adherence prediction in BCSSs is gaining research attention. A total of 13 unique supervised learning techniques were identified and the majority of them were traditional machine learning techniques (eg, support vector machine). Long short-term memory, multilayer perception, and ensemble learning are currently the only advanced learning techniques. Despite the heterogeneity in the feature selection approaches, most prediction models achieved good classification accuracies. This indicates that the features or predictors used were a good representation of the adherence problem.

CONCLUSIONS

Using machine learning algorithms to predict the adherence behavior of a BCSS user can facilitate the reinforcement of adherence behavior. This can be achieved by developing intelligent BCSSs that can provide users with more personalized, tailored, and timely suggestions.

Key Risk Factors Associated With Electronic Nicotine Delivery Systems Use Among Adolescents

Importance

The prevalence of electronic nicotine delivery systems (ENDS) use among US youths has increased significantly during the past decade. Identifying key factors highly associated with ENDS use is essential in monitoring and preventing this harmful behavior among youths.

Objective

To identify the most important risk factors in wave 4.5 (ie, December 2017 to December 2018) of the Population Assessment of Tobacco and Health Study (PATH) data that are associated with ENDS use in wave 5 (ie, December 2018 to November 2019) among adolescents who were tobacco-naive at baseline.

Design, Setting, and Participants

This prognostic study examined data from waves 4.5 and 5 of the PATH youth data set using machine learning techniques. The PATH study is a nationally representative longitudinal cohort study of tobacco use and health in the United States among individuals aged 12 years and older. The data analysis was carried out between January and April 2023.

Main Outcomes and Measures

Wave 5 current ENDS use status of wave 4.5 adolescents who were tobacco-naive.

Results

The analyzed data set comprised 7943 individuals who were tobacco-naive in wave 4.5. Among this group, 332 participants (4.2%) indicated their present use of ENDS in wave 5, 5047 (63.5%) were aged 12 to 14 years, 4066 (51.2%) were male, and 2455 (30.9%) were Hispanic. The most important risk factors of ENDS use in wave 5 among adolescents who were tobacco-naive in wave 4.5 were the likelihood of using ENDS if offered by a best friend (mean SHAP value, 0.184), the number of best friends using e-cigarettes (mean SHAP value, 0.167), household tobacco usage (mean SHAP value, 0.161), curiosity about ENDS use (mean SHAP value, 0.088), future intention to use ENDS (mean SHAP value, 0.068), youth's total average weekly earnings (mean SHAP value, 0.060), and perceptions of tobacco product safety (mean SHAP value, 0.026).

Conclusions and Relevance

The findings of this study suggest that family and friends play an important role in ENDS use among adolescents. The top-ranking factors associated with ENDS use in this study are areas for further exploration, given the increasing prevalence of ENDS use among youths in recent years. Additionally, these findings highlight the important role of families and schools in shaping adolescents' tobacco-related knowledge, which can protect them from using ENDS.

Identifying factors associated with user retention and outcomes of a digital intervention for substance use disorder: a retrospective analysis of real-world data

Objectives

Successful delivery of digital health interventions is affected by multiple real-world factors. These factors may be identified in routinely collected, ecologically valid data from these interventions. We propose ideas for exploring these data, focusing on interventions targeting complex, comorbid conditions.

Materials and Methods

This study retrospectively explores pre-post data collected between 2016 and 2019 from users of digital cognitive behavioral therapy (CBT)-containing psychoeducation and practical exercises-for substance use disorder (SUD) at UK addiction services. To identify factors associated with heterogenous user responses to the technology, we employed multivariable and multivariate regressions and random forest models of user-reported questionnaire data.

Results

The dataset contained information from 14 078 individuals of which 12 529 reported complete data at baseline and 2925 did so again after engagement with the CBT. Ninety-three percent screened positive for dependence on 1 of 43 substances at baseline, and 73% screened positive for anxiety or depression. Despite pre-post improvements independent of user sociodemographics, women reported more frequent and persistent symptoms of SUD, anxiety, and depression. Retention-minimum 2 use events recorded-was associated more with deployment environment than user characteristics. Prediction accuracy of post-engagement outcomes was acceptable (Area Under Curve [AUC]: 0.74-0.79), depending non-trivially on user characteristics.

Discussion

Traditionally, performance of digital health interventions is determined in controlled trials. Our analysis showcases multivariate models with which real-world data from these interventions can be explored and sources of user heterogeneity in retention and symptom reduction uncovered.

Conclusion

Real-world data from digital health interventions contain information on natural user-technology interactions which could enrich results from controlled trials.

Preferences of young adults on the development of a new digital add-on alcohol intervention for depression treatment: A qualitative study

AIM

To explore the preferences of young adults with regard to the development of a new digital add-on alcohol intervention to complement depression treatment.

METHODS

This qualitative study included young adults (18-35 years) with experience of either problematic alcohol use or depression or both (n = 29). Two rounds of focus groups were conducted, with two focus groups in each round. All focus groups were recorded, transcribed and analysed deductively and inductively on the basis of qualitative content analysis of the intervention type, features and design.

RESULTS

Young adults preferred a mobile health application with a clear and simple objective and navigation which was also accessible on a computer. With regard to intervention features, participants indicated a preference for in-depth, gain-framed information on alcohol use and a main feature enabling them to record their alcohol use and mood, which would be rewarded. Other preferences included personal goal-setting and monitoring, an activity list, experience stories, peer contact, guidance from experts by experience or volunteers and receiving notifications from the application. In terms of design, participants preferred short, animated videos and animation figure illustrations to complement written text. Moreover, participants rated the design of the intervention as highly important, yet very personal. Generally, participants preferred a light pastel colour scheme. Once again, participants indicated a need for a clear dashboard using pictograms to reduce the amount of text and fast, easy-to-use navigation.

CONCLUSION

The preferences indicated by young adults with regard to the intervention type, features and design may enhance the development of a new digital add-on alcohol intervention to complement depression treatment.

Predictors of smoking cessation outcomes identified by machine learning: A systematic review

This systematic review aims to characterize the utility of machine learning to identify the predictors of smoking cessation outcomes and identify the machine learning methods applied in this area. In the current study, multiple searches occurred through December 9, 2022 in MEDLINE, Science Citation Index, Social Science Citation Index, EMBASE, CINAHL Plus, APA PsycINFO, PubMed, Cochrane Central Register of Controlled Trials, and the IEEE Xplore were performed. Inclusion criteria included various machine learning techniques, studies reporting cigarette smoking cessation outcomes (smoking status and the number of cigarettes), and various experimental designs (e.g., cross-sectional and longitudinal). Predictors of smoking cessation outcomes were assessed, including behavioral markers, biomarkers, and other predictors. Our systematic review identified 12 papers fitting our inclusion criteria. In this review, we identified gaps in knowledge and innovation opportunities for machine learning research in the field of smoking cessation.

Can a Single Variable Predict Early Dropout From Digital Health Interventions? Comparison of Predictive Models From Two Large Randomized Trials

BACKGROUND

A single generalizable metric that accurately predicts early dropout from digital health interventions has the potential to readily inform intervention targets and treatment augmentations that could boost retention and intervention outcomes. We recently identified a type of early dropout from digital health interventions for smoking cessation, specifically, users who logged in during the first week of the intervention and had little to no activity thereafter. These users also had a substantially lower smoking cessation rate with our iCanQuit smoking cessation app compared with users who used the app for longer periods.

OBJECTIVE

This study aimed to explore whether log-in count data, using standard statistical methods, can precisely predict whether an individual will become an iCanQuit early dropout while validating the approach using other statistical methods and randomized trial data from 3 other digital interventions for smoking cessation (combined randomized N=4529).

METHODS

Standard logistic regression models were used to predict early dropouts for individuals receiving the iCanQuit smoking cessation intervention app, the National Cancer Institute QuitGuide smoking cessation intervention app, the WebQuit.org smoking cessation intervention website, and the Smokefree.gov smoking cessation intervention website. The main predictors were the number of times a participant logged in per day during the first 7 days following randomization. The area under the curve (AUC) assessed the performance of the logistic regression models, which were compared with decision trees, support vector machine, and neural network models. We also examined whether 13 baseline variables that included a variety of demographics (eg, race and ethnicity, gender, and age) and smoking characteristics (eg, use of e-cigarettes and confidence in being smoke free) might improve this prediction.

RESULTS

The AUC for each logistic regression model using only the first 7 days of log-in count variables was 0.94 (95% CI 0.90-0.97) for iCanQuit, 0.88 (95% CI 0.83-0.93) for QuitGuide, 0.85 (95% CI 0.80-0.88) for WebQuit.org, and 0.60 (95% CI 0.54-0.66) for Smokefree.gov. Replacing logistic regression models with more complex decision trees, support vector machines, or neural network models did not significantly increase the AUC, nor did including additional baseline variables as predictors. The sensitivity and specificity were generally good, and they were excellent for iCanQuit (ie, 0.91 and 0.85, respectively, at the 0.5 classification threshold).

CONCLUSIONS

Logistic regression models using only the first 7 days of log-in count data were generally good at predicting early dropouts. These models performed well when using simple, automated, and readily available log-in count data, whereas including self-reported baseline variables did not improve the prediction. The results will inform the early identification of people at risk of early dropout from digital health interventions with the goal of intervening further by providing them with augmented treatments to increase their retention and, ultimately, their intervention outcomes.

Advances in problematic usage of the internet research - A narrative review by experts from the European network for problematic usage of the internet

Global concern about problematic usage of the internet (PUI), and its public health and societal costs, continues to grow, sharpened in focus under the privations of the COVID-19 pandemic. This narrative review reports the expert opinions of members of the largest international network of researchers on PUI in the framework of the European Cooperation in Science and Technology (COST) Action (CA 16207), on the scientific progress made and the critical knowledge gaps remaining to be filled as the term of the Action reaches its conclusion. A key advance has been achieving consensus on the clinical definition of various forms of PUI. Based on the overarching public health principles of protecting individuals and the public from harm and promoting the highest attainable standard of health, the World Health Organisation has introduced several new structured diagnoses into the ICD-11, including gambling disorder, gaming disorder, compulsive sexual behaviour disorder, and other unspecified or specified disorders due to addictive behaviours, alongside naming online activity as a diagnostic specifier. These definitions provide for the first time a sound platform for developing systematic networked research into various forms of PUI at global scale. Progress has also been made in areas such as refining and simplifying some of the available assessment instruments, clarifying the underpinning brain-based and social determinants, and building more empirically based etiological models, as a basis for therapeutic intervention, alongside public engagement initiatives. However, important gaps in our knowledge remain to be tackled. Principal among these include a better understanding of the course and evolution of the PUI-related problems, across different age groups, genders and other specific vulnerable groups, reliable methods for early identification of individuals at risk (before PUI becomes disordered), efficacious preventative and therapeutic interventions and ethical health and social policy changes that adequately safeguard human digital rights. The paper concludes with recommendations for achievable research goals, based on longitudinal analysis of a large multinational cohort co-designed with public stakeholders.