Evaluating the effectiveness of the smartphone app, Drink Less, compared with the NHS alcohol advice webpage, for the reduction of alcohol consumption among hazardous and harmful adult drinkers in the UK at 6-month follow-up: protocol for a randomised controlled trial

Effectiveness of a smartphone app (Drink Less) versus usual digital care for reducing alcohol consumption among increasing-and-higher-risk adult drinkers in the UK: a two-arm, parallel-group, double-blind, randomised controlled trial

Background

Digital interventions, including apps and websites, can be effective for reducing alcohol consumption. However, many are not evidence- or theory-informed and have not been evaluated. We tested the effectiveness of the Drink Less app for reducing alcohol consumption compared with usual digital care in the UK.

Methods

In this two-arm, parallel group, double-blind, randomised controlled trial, we enrolled increasing-and-higher-risk drinkers (AUDIT ≥ 8) in the UK, who were motivated to reduce their alcohol consumption and willing to use a digital intervention to do so, via online methods. Participants were randomly assigned (1:1), using an online algorithm, to receive a web link to download the Drink Less app (intervention) or to the NHS alcohol advice webpage (usual digital care). Researchers were masked to group allocation. Participants were followed up at one, three and six months. The primary outcome was self-reported weekly alcohol consumption at six months, adjusting for baseline consumption. The full analytic sample was used in most analyses, though missing data was treated in different ways. The primary, pre-registered intention-to-treat analysis assumed baseline-carried-forwards. Secondary pre-registered analyses also focused on the full analytic sample and used alternatives including multiple imputation and last observation carried forwards. This trial is registered with the ISRCTN registry, ISRCTN64052601.

Findings

Between 07/13/2020 and 03/29/2022, 5602 people were randomly assigned to the Drink Less app (n = 2788) or comparator (n = 2814) groups. Six-month follow-up rates were 79% and 80%, respectively. The primary pre-registered conservative intention-to-treat approach assuming non-responders were drinking at baseline levels of consumption, found a non-significant greater reduction of 0.98 units in weekly alcohol consumption in the intervention group at 6-month follow-up (95% CI -2.67 to 0.70). The data were insensitive to detect the hypothesised effect (Bayes factor = 1.17). Data were not missing completely at random, with 6-month follow-up rates differing in terms of education, occupation, and income. We therefore conducted the pre-registered sensitivity analysis using multiple imputation, showing that the Drink Less app resulted in a 2.00-unit greater weekly reduction at 6-month follow-up compared with the NHS alcohol advice webpage (95% CI -3.76 to -0.24). Fewer than 0.1% of participants in both arms who responded to one, three or six-month follow-up reported adverse events linked to participation in the trial.

Interpretation

The Drink Less app may be effective in reducing the alcohol consumption in increasing-and-higher-risk drinkers motivated to reduce their consumption.

Funding

NIHR Public Health Research Programme.

Methodological Insights on Recruitment and Retention From a Remote Randomized Controlled Trial Examining the Effectiveness of an Alcohol Reduction App: Descriptive Analysis Study

BACKGROUND

Randomized controlled trials (RCTs) with no in-person contact (ie, remote) between researchers and participants offer savings in terms of cost and time but present unique challenges.

OBJECTIVE

The goal of this study is to examine the differences between different forms of remote recruitment (eg, National Health Service [NHS] website, social media, and radio advertising) in the proportion of participants recruited, demographic diversity, follow-up rates, and cost. We also examine the cost per participant of sequential methods of follow-up (emails, phone calls, postal surveys, and postcards). Finally, our experience with broader issues around study advertising and participant deception is discussed.

METHODS

We conducted a descriptive analysis of 5602 increasing-and-higher-risk drinkers (Alcohol Use Disorders Identification Test score ≥8), taking part in a 2-arm, parallel group, remote RCT with a 1:1 allocation, comparing the intervention (Drink Less app) with usual digital care (NHS alcohol advice web page). Participants were recruited between July 2020 and March 2022 and compensated with gift vouchers of up to £36 (a currency exchange rate of £1=US $1.26988 is applicable) for completing follow-up surveys, with 4 stages of follow-up: email reminders, phone calls, postal survey, and postcard.

RESULTS

The three main recruitment methods were advertisements on (1) social media (2483/5602, 44.32%), (2) the NHS website (1961/5602, 35.01%), and (3) radio and newspapers (745/5602, 13.3%), with the remaining methods of recruitment accounting 7.37% (413/5602) of the sample. The overall recruitment cost per participant varied from £0 to £11.01. Costs were greater when recruiting participants who were men (£0-£28.85), from an ethnic minority group (£0-£303.81), and more disadvantaged (£0-£49.12). Targeted approaches were useful for recruiting more men but less useful in achieving diversity in ethnicity and socioeconomic status. Follow-up at 6 months was 79.58% (4458/5602). Of those who responded, 92.4% (4119/4458) responded by email. Each additional stage of follow-up resulted in an additional 2-3 percentage points of the overall sample being followed up, although phone calls, postal surveys, and postcards were more resource intensive than email reminders.

CONCLUSIONS

For remote RCTs, researchers could benefit from using a range of recruitment methods and cost-targeted approaches to achieve demographic diversity. Automated emails with substantial financial incentives for prompt completion can achieve good follow-up rates, and sequential, offline follow-up options, such as phone calls and postal surveys, can further increase follow-up rates but are comparatively expensive. We also make broader recommendations focused on striking the right balance when designing remote RCTs. Careful planning, ongoing maintenance, and dynamic decision-making are required throughout a trial to balance the competing demands of participation among those eligible, deceptive participation among those who are not eligible, and ensuring no postrandomization bias is introduced by data-checking protocols.

Bot or Not? Detecting and Managing Participant Deception When Conducting Digital Research Remotely: Case Study of a Randomized Controlled Trial

BACKGROUND

Evaluating digital interventions using remote methods enables the recruitment of large numbers of participants relatively conveniently and cheaply compared with in-person methods. However, conducting research remotely based on participant self-report with little verification is open to automated "bots" and participant deception.

OBJECTIVE

This paper uses a case study of a remotely conducted trial of an alcohol reduction app to highlight and discuss (1) the issues with participant deception affecting remote research trials with financial compensation; and (2) the importance of rigorous data management to detect and address these issues.

METHODS

We recruited participants on the internet from July 2020 to March 2022 for a randomized controlled trial (n=5602) evaluating the effectiveness of an alcohol reduction app, Drink Less. Follow-up occurred at 3 time points, with financial compensation offered (up to £36 [US $39.23]). Address authentication and telephone verification were used to detect 2 kinds of deception: "bots," that is, automated responses generated in clusters; and manual participant deception, that is, participants providing false information.

RESULTS

Of the 1142 participants who enrolled in the first 2 months of recruitment, 75.6% (n=863) of them were identified as bots during data screening. As a result, a CAPTCHA (Completely Automated Public Turing Test to Tell Computers and Humans Apart) was added, and after this, no more bots were identified. Manual participant deception occurred throughout the study. Of the 5956 participants (excluding bots) who enrolled in the study, 298 (5%) were identified as false participants. The extent of this decreased from 110 in November 2020, to a negligible level by February 2022 including a number of months with 0. The decline occurred after we added further screening questions such as attention checks, removed the prominence of financial compensation from social media advertising, and added an additional requirement to provide a mobile phone number for identity verification.

CONCLUSIONS

Data management protocols are necessary to detect automated bots and manual participant deception in remotely conducted trials. Bots and manual deception can be minimized by adding a CAPTCHA, attention checks, a requirement to provide a phone number for identity verification, and not prominently advertising financial compensation on social media.

TRIAL REGISTRATION

ISRCTN Number ISRCTN64052601; https://doi.org/10.1186/ISRCTN64052601.

The association of alcohol dependence and consumption during adolescence with depression in young adulthood, in England: a prospective cohort study

BACKGROUND

The role of alcohol use in the development of depression is unclear. We aimed to investigate whether alcohol dependence, but not high frequency or quantity of consumption, during adolescence increased the risk of depression in young adulthood.

METHODS

In this prospective cohort study, we included adolescents who were born to women recruited to the Avon Longitudinal Study of Parents and Children in Avon, UK, with delivery dates between April 1, 1991, and Dec 31, 1992. Alcohol dependence and consumption were measured at about age 16 years, 18 years, 19 years, 21 years, and 23 years using the self-reported Alcohol Use Disorders Identification Test, and at about age 18 years, 21 years, and 23 years using items corresponding to DSM-IV symptoms. The primary outcome was depression at age 24 years, assessed using the Clinical Interview Schedule Revised. Analyses were probit regressions between growth factors for alcohol dependence and consumption and depression, before and after adjustments for confounders: sex, housing tenure, maternal education, maternal depressive symptoms, parents' alcohol use, conduct problems at age 4 years, being bullied from age 12-16 years, and frequency of smoking cigarettes or cannabis. Adolescents were included in analyses if they had data from at least one timepoint for alcohol use and confounders.

FINDINGS

We included 3902 adolescents (2264 [58·0%] female; 1638 [42·0%] male) in our analysis, and 3727 (96·7%) of 3853 participants with data on ethnicity were White. After adjustments, we found a positive association between alcohol dependence at 18 years of age (latent intercept) and depression at 24 years of age (probit coefficient 0·13 [95% CI 0·02 to 0·25]; p=0·019), but no association between rate of change (linear slope) and depression (0·10 [-0·82 to 1·01]; p=0·84). There was no evidence of an association between alcohol consumption and depression (latent intercept probit coefficient -0·01 [-0·06 to 0·03]; p=0·60; linear slope 0·01 [-0·40 to 0·42]; p=0·96) after adjustments.

INTERPRETATION

Psychosocial or behavioural interventions that reduce the risk of alcohol dependence during adolescence could contribute to preventing depression in young adulthood.

FUNDING

UK Medical Research Council and Alcohol Research UK (grant number MR/L022206/1).

How Notifications Affect Engagement With a Behavior Change App: Results From a Micro-Randomized Trial

BACKGROUND

Drink Less is a behavior change app to help higher-risk drinkers in the United Kingdom reduce their alcohol consumption. The app includes a daily notification asking users to "Please complete your drinks and mood diary," yet we did not understand the causal effect of the notification on engagement nor how to improve this component of Drink Less. We developed a new bank of 30 new messages to increase users' reflective motivation to engage with Drink Less. This study aimed to determine how standard and new notifications affect engagement.

OBJECTIVE

Our objective was to estimate the causal effect of the notification on near-term engagement, to explore whether this effect changed over time, and to create an evidence base to further inform the optimization of the notification policy.

METHODS

We conducted a micro-randomized trial (MRT) with 2 additional parallel arms. Inclusion criteria were Drink Less users who consented to participate in the trial, self-reported a baseline Alcohol Use Disorders Identification Test score of ≥8, resided in the United Kingdom, were aged ≥18 years, and reported interest in drinking less alcohol. Our MRT randomized 350 new users to test whether receiving a notification, compared with receiving no notification, increased the probability of opening the app in the subsequent hour, over the first 30 days since downloading Drink Less. Each day at 8 PM, users were randomized with a 30% probability of receiving the standard message, a 30% probability of receiving a new message, or a 40% probability of receiving no message. We additionally explored time to disengagement, with the allocation of 60% of eligible users randomized to the MRT (n=350) and 40% of eligible users randomized in equal number to the 2 parallel arms, either receiving the no notification policy (n=98) or the standard notification policy (n=121). Ancillary analyses explored effect moderation by recent states of habituation and engagement.

RESULTS

Receiving a notification, compared with not receiving a notification, increased the probability of opening the app in the next hour by 3.5-fold (95% CI 2.91-4.25). Both types of messages were similarly effective. The effect of the notification did not change significantly over time. A user being in a state of already engaged lowered the new notification effect by 0.80 (95% CI 0.55-1.16), although not significantly. Across the 3 arms, time to disengagement was not significantly different.

CONCLUSIONS

We found a strong near-term effect of engagement on the notification, but no overall difference in time to disengagement between users receiving the standard fixed notification, no notification at all, or the random sequence of notifications within the MRT. The strong near-term effect of the notification presents an opportunity to target notifications to increase "in-the-moment" engagement. Further optimization is required to improve the long-term engagement.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/18690.

Promoting Hand Hygiene During the COVID-19 Pandemic: Parallel Randomized Trial for the Optimization of the Soapp App

BACKGROUND

Hand hygiene is an effective behavior for preventing the spread of the respiratory disease COVID-19 and was included in public health guidelines worldwide. Behavior change interventions addressing hand hygiene have the potential to support the adherence to public health recommendations and, thereby, prevent the spread of COVID-19. However, randomized trials are largely absent during a pandemic; therefore, there is little knowledge about the most effective strategies to promote hand hygiene during an ongoing pandemic. This study addresses this gap by presenting the results of the optimization phase of a Multiphase Optimization Strategy of Soapp, a smartphone app for promoting hand hygiene in the context of the COVID-19 pandemic.

OBJECTIVE

This study aimed to identify the most effective combination and sequence of 3 theory- and evidence-based intervention modules (habit, motivation, and social norms) for promoting hand hygiene. To this end, 9 versions of Soapp were developed (conditions), and 2 optimization criteria were defined: the condition with the largest increase in hand hygiene at follow-up and condition with the highest engagement, usability, and satisfaction based on quantitative and qualitative analyses.

METHODS

This study was a parallel randomized trial with 9 intervention conditions defined by the combination of 2 intervention modules and their sequence. The trial was conducted from March to August 2021 with interested participants from the Swiss general population (N=232; randomized). Randomization was performed using Qualtrics (Qualtrics International Inc), and blinding was ensured. The duration of the intervention was 34 days. The primary outcome was self-reported hand hygiene at follow-up, which was assessed using an electronic diary. The secondary outcomes were user engagement, usability, and satisfaction assessed at follow-up. Nine participants were further invited to participate in semistructured exit interviews. A set of ANOVAs was performed to test the main hypotheses, whereas a thematic analysis was performed to analyze the qualitative data.

RESULTS

The results showed a significant increase in hand hygiene over time across all conditions. There was no interaction effect between time and intervention condition. Similarly, no between-group differences in engagement, usability, and satisfaction emerged. Seven themes (eg, "variety and timeliness of the task load" and "social interaction") were found in the thematic analysis.

CONCLUSIONS

The effectiveness of Soapp in promoting hand hygiene laid the foundation for the next evaluation phase of the app. More generally, the study supported the value of digital interventions in pandemic contexts. The findings showed no differential effect of intervention conditions involving different combinations and sequences of the habit, motivation, and social norms modules on hand hygiene, engagement, usability, and satisfaction. In the absence of quantitative differences, we relied on the results from the thematic analysis to select the best version of Soapp for the evaluation phase.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04830761; https://clinicaltrials.gov/ct2/show/NCT04830761.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1136/bmjopen-2021-055971.

Promoting benzodiazepine cessation through an electronically-delivered patient self-management intervention (EMPOWER-ED): Randomized controlled trial protocol

Background

Long-term benzodiazepine dependence carries significant health risks which might be reduced with low-cost patient self-management interventions. A booklet version of one such intervention (Eliminating Medications Through Patient Ownership of End Results; EMPOWER) proved effective in a Canadian clinical trial with older adults. Digitizing such an intervention for electronic delivery and tailoring it to different populations could expand its reach. Accordingly, this article describes the protocol for a randomized controlled trial to test the effectiveness of an electronically-delivered, direct-to-patient benzodiazepine cessation intervention tailored to U.S. military veterans.

Methods

Design: Two-arm individually randomized controlled trial.

Setting

US Veterans Health Administration primary care clinics.

Participants

Primary care patients taking benzodiazepines for three or more months and having access to a smartphone, tablet or desktop computer.

Intervention and comparator

Participants will be randomized to receive either the electronically-delivered EMPOWER (EMPOWER-ED) protocol or asked to continue to follow provider recommendations regarding their benzodiazepine use (treatment-as-usual).

Measurements

The primary outcomes are complete benzodiazepine cessation and 25% dose reduction, assessed using administrative and self-report data, between baseline and six-month follow-up. Secondary outcomes are self-reported anxiety symptoms, sleep quality, and overall health and quality of life, measured at baseline and 6-month follow-up, and benzodiazepine cessation at 12-month follow-up.

Comments

This randomized controlled trial will evaluate whether the accessibility and effectiveness of a promising intervention for benzodiazepine cessation can be improved through digitization and population tailoring.

Effectiveness and cost-effectiveness of face-to-face and electronic brief interventions versus screening alone to reduce alcohol consumption among high-risk adolescents presenting to emergency departments: three-arm pragmatic randomized trial (SIPS Junior high risk trial)

BACKGROUND AND AIMS

Alcohol use increases throughout adolescence. Emergency department (ED) attendance is an opportunity for alcohol screening and brief intervention (ASBI), which is effective for adults. This trial evaluated the effectiveness and cost-effectiveness of ASBI compared with screening alone (SA) in high-risk adolescents.

DESIGN, SETTING AND PARTICIPANTS

Multi-centre, three-group, single-blind, individually randomized trial with follow-ups after 6 and 12 months in 10 ED settings in England. From October 2014 to May 2015 we screened 3327 adolescents aged 14 to 18 years, of whom 756 (22.7%) scored at least 3 on the Alcohol Use Disorders Identification Test: consumption (AUDIT-C) and consented to participate in this trial. Mean age was 16.1 years; 50.2% were female and 84.9% were white.

INTERVENTIONS

Interventions were personalized feedback and brief advice (PFBA), personalized feedback plus electronic brief intervention (eBI) and SA.

MEASURES

The primary outcome was the weekly alcohol consumed in standard UK units (8 g ethanol) at 12 months post-randomization, derived from extended AUDIT-C. Economic outcomes included quality of life and service use, from perspectives of both the National Health Service and personal social services (NHS&PSS) and society.

FINDINGS

At 12 months, mean weekly consumption was 2.99 [95% confidence interval (CI) = 2.38-3.70] standard units for the SA group, 3.56 (95% CI = 2.90, 4.32) for PFBA and 3.18 (95% CI = 2.50, 3.97) for eBI, showing no significant differences. The PFBA group consumed mean 0.57 (-0.36, 1.70) units more than SA; and eBIs consumed 0.19 (-0.71, 1.30) more. Bayes factors suggested lack of effectiveness explained non-significance. From the NHS&PSS perspective, economic analysis showed that PFBA and eBI were not cost-effective compared with SA: PFBA yielded incremental cost-effectiveness ratio of £6213 (-£736 843, £812 884), with the intervention having 54% probability of being cost-effective compared with SA at the £20 000 WTP threshold.

CONCLUSIONS

In emergency departments in England, neither personalized feedback and brief advice nor personalized feedback plus electronic brief intervention showed evidence of being effective or cost-effective when compared with screening alone in reducing alcohol consumption among adolescents.

Digital Assessment Tools Using Animation Features to Quantify Alcohol Consumption: Systematic App Store and Literature Review

Background

Accurate and user-friendly assessment tools for quantifying alcohol consumption are a prerequisite for effective interventions to reduce alcohol-related harm. Digital assessment tools (DATs) that allow the description of consumed alcoholic drinks through animation features may facilitate more accurate reporting than conventional approaches.

Objective

This review aims to identify and characterize freely available DATs in English or Russian that use animation features to support the quantitative assessment of alcohol consumption (alcohol DATs) and determine the extent to which such tools have been scientifically evaluated in terms of feasibility, acceptability, and validity.

Methods

Systematic English and Russian searches were conducted in iOS and Android app stores and via the Google search engine. Information on the background and content of eligible DATs was obtained from app store descriptions, websites, and test completions. A systematic literature review was conducted in Embase, MEDLINE, PsycINFO, and Web of Science to identify English-language studies reporting the feasibility, acceptability, and validity of animation-using alcohol DATs. Where possible, the evaluated DATs were accessed and assessed. Owing to the high heterogeneity of study designs, results were synthesized narratively.

Results

We identified 22 eligible alcohol DATs in English, 3 (14%) of which were also available in Russian. More than 95% (21/22) of tools allowed the choice of a beverage type from a visually displayed selection. In addition, 36% (8/22) of tools enabled the choice of a drinking vessel. Only 9% (2/22) of tools allowed the simulated interactive pouring of a drink. For none of the tools published evaluation studies were identified in the literature review. The systematic literature review identified 5 exploratory studies evaluating the feasibility, acceptability, and validity of 4 animation-using alcohol DATs, 1 (25%) of which was available in the searched app stores. The evaluated tools reached moderate to high scores on user rating scales and showed fair to high convergent validity when compared with established assessment methods.

Conclusions

Animation-using alcohol DATs are available in app stores and on the web. However, they often use nondynamic features and lack scientific background information. Explorative study data suggest that such tools might enable the user-friendly and valid assessment of alcohol consumption and could thus serve as a building block in the reduction of alcohol-attributable health burden worldwide.

Trial Registration

PROSPERO International Prospective Register of Systematic Reviews CRD42020172825; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020172825

Rethink My Drink: study protocol for a 12-month randomised controlled trial comparing a brief internet-delivered intervention to an online patient information booklet in reducing risky alcohol consumption among older adults in Australia

BACKGROUND AND AIMS

Alcohol consumption is increasing among older adults. Rethink My Drink is a brief internet-delivered intervention to reduce alcohol consumption and related harms, adapted specifically for older adults. This protocol for a large-scale randomised controlled trial will evaluate whether Rethink My Drink is effective in reducing alcohol consumption and cognitive decline in a sample of older risky drinkers, compared with an active control.

DESIGN

1:1 parallel group, randomised controlled trial.

SETTING

Online trial in Australia.

PARTICIPANTS

Hazardous or harmful drinkers (defined as those scoring ≥5 on the Alcohol Use Disorders Identification Test [AUDIT]) age 60 to 75 years old (n = 842). Participants will be recruited from August 2021 to August 2022 through online social media advertisements and community networks.

INTERVENTION AND COMPARATOR

Participants will be randomly allocated to receive access to Rethink My Drink (intervention) or Alcohol: The Facts (comparator), an online patient information booklet provided by New South Wales (NSW) Health.

MEASUREMENTS

Primary outcomes include (i) average weekly standard drinks and (ii) rate of cognitive decline. Secondary outcomes include (i) typical quantity of drinks per drinking day; (ii) heavy episodic drinking; (iii) age-specific risky drinking; (iv) alcohol-related harms; (v) subjective cognitive complaints; and (vi) quality of life. All primary and secondary outcomes will be assessed at baseline, post-intervention (4 weeks) and 12 months. Effectiveness will be evaluated using multilevel linear regression, adjusting for baseline demographic differences. Bonferroni adjustments will be used to control for multiple comparisons. Multiple imputation, regression weighting and sensitivity analyses will assess the effect of attrition.

COMMENTS

This will be the first large-scale trial, internationally, to examine whether a brief internet-delivered intervention is effective in reducing alcohol consumption and cognitive decline among older adults. If successful, the intervention will provide an accessible and highly scalable treatment to reduce risky alcohol consumption in older adulthood.

Refining the content and design of an alcohol reduction app, Drink Less, to improve its usability and effectiveness: a mixed methods approach

Background: Digital interventions have the potential to reduce alcohol consumption, although evidence on the effectiveness of apps is lacking. Drink Less is a popular, evidence-informed app with good usability, putting it in a strong position to be improved upon prior to conducting a confirmatory evaluation. This paper describes the process of refining Drink Less to improve its usability and likely effectiveness. Methods: The refinement consisted of three phases and involved qualitative and quantitative (mixed) methods: i) identifying changes to app content, based on findings from an initial evaluation of Drink Less, an updated review of digital alcohol interventions and a content analysis of user feedback; ii) designing new app modules with public input and a consultation with app developers and researchers; and iii) improving the app's usability through user testing. Results: As a result of the updated review of digital alcohol interventions and user feedback analysis in Phase 1, three new modules: 'Behaviour Substitution', 'Information about Antecedents' and 'Insights', were added to the app. One existing module - 'Identity Change' - was removed based on the initial evaluation of Drink Less. Phases 2 and 3 resulted in changes to existing features, such as improving the navigational structure and onboarding process, and clarifying how to edit drinks and goals. Conclusions: A mixed methods approach was used to refine the content and design of Drink Less, providing insights into how to improve its usability and likely effectiveness. Drink Less is now ready for a confirmatory evaluation.

Refining the content and design of an alcohol reduction app, Drink Less, to improve its usability and effectiveness: a mixed methods approach

Background: Digital interventions have the potential to reduce alcohol consumption, although evidence on the effectiveness of apps is lacking. Drink Less is a popular, evidence-informed app with good usability, putting it in a strong position to be improved upon prior to conducting a confirmatory evaluation. This paper describes the process of refining Drink Less to improve its usability and likely effectiveness. Methods: The refinement consisted of three phases and involved qualitative and quantitative (mixed) methods: i) identifying changes to app content, based on findings from an initial evaluation of Drink Less, an updated review of digital alcohol interventions and a content analysis of user feedback; ii) designing new app modules with public input and a consultation with app developers and researchers; and iii) improving the app's usability through user testing. Results: As a result of the updated review of digital alcohol interventions and user feedback analysis in Phase 1, three new modules: 'Behaviour Substitution', 'Information about Antecedents' and 'Insights', were added to the app. One existing module - 'Identity Change' - was removed based on the initial evaluation of Drink Less. Phases 2 and 3 resulted in changes to existing features, such as improving the navigational structure and onboarding process, and clarifying how to edit drinks and goals. Conclusions: A mixed methods approach was used to refine the content and design of Drink Less, providing insights into how to improve its usability and likely effectiveness. Drink Less is now ready for a confirmatory evaluation.