How big is the elephant in the room? Estimated and actual IT costs in an online behaviour change trial

Economic Evaluation of Digital Health Interventions: Methodological Issues and Recommendations for Practice

Health care interventions are increasingly being delivered through digital technologies, offering major opportunities for delivering more health gains from scarce health care resources. Digital health interventions (DHIs) raise distinct challenges for economic evaluations compared with drugs and medical devices, not least due to their interacting, evolving features. The implications of the distinctive nature of DHIs for the methodological choices underpinning their economic evaluation is not well understood. This paper provides an in-depth discussion of distinct features of DHIs and how they might impact the design, measurement, analysis and reporting of cost-effectiveness analysis conducted alongside both randomised and non-randomised studies. These include aspects related to choice of comparator, costs and benefits assessment, study perspective and type of economic analysis. We argue that typical methodological standpoints, such as taking a health service perspective, focusing on health-related benefits and adopting cost-utility analyses, as typically adopted in the economic evaluation of non-digital technologies (pharmaceutical drugs and medical devices), are unlikely to be appropriate for DHIs. We illustrate how these methodological aspects can be appropriately addressed in an evaluation of a digitally supported, remote rehabilitation programme for patients with Long Covid in England. We highlight several methodological considerations for improving practice and areas where further methodological work is required.

Factors affecting implementation of digital health interventions for people with psychosis or bipolar disorder, and their family and friends: a systematic review

Digital health interventions present an important opportunity to improve health care for people with psychosis or bipolar disorder, but despite their potential, integrating and implementing them into clinical settings has been difficult worldwide. This Review aims to identify factors affecting implementation of digital health interventions for people affected by psychosis or bipolar disorder. We searched seven databases and synthesised data from 26 studies using the Consolidated Framework for Implementation Research. Attitudes and beliefs about interventions were crucial factors for both staff and service users, with negative attitudes and scepticism resulting in a lack of motivation to engage with interventions or complete them. The complexity of the interventions was a barrier for people with psychiatric symptoms, low premorbid intelligence quotient, or minimal information technology skills. The accessibility and adaptability of interventions were key facilitators, but insufficient resources, finances, and staff time were barriers to implementation. Interventions need to be user friendly and adaptable to the needs and capabilities of people with psychosis or bipolar disorder, and the staff who support their implementation. Service users and staff should cofacilitate the process of developing and implementing the interventions.

Designing and Undertaking a Health Economics Study of Digital Health Interventions

This paper introduces and discusses key issues in the economic evaluation of digital health interventions. The purpose is to stimulate debate so that existing economic techniques may be refined or new methods developed. The paper does not seek to provide definitive guidance on appropriate methods of economic analysis for digital health interventions. This paper describes existing guides and analytic frameworks that have been suggested for the economic evaluation of healthcare interventions. Using selected examples of digital health interventions, it assesses how well existing guides and frameworks align to digital health interventions. It shows that digital health interventions may be best characterized as complex interventions in complex systems. Key features of complexity relate to intervention complexity, outcome complexity, and causal pathway complexity, with much of this driven by iterative intervention development over time and uncertainty regarding likely reach of the interventions among the relevant population. These characteristics imply that more-complex methods of economic evaluation are likely to be better able to capture fully the impact of the intervention on costs and benefits over the appropriate time horizon. This complexity includes wider measurement of costs and benefits, and a modeling framework that is able to capture dynamic interactions among the intervention, the population of interest, and the environment. The authors recommend that future research should develop and apply more-flexible modeling techniques to allow better prediction of the interdependency between interventions and important environmental influences.

Web-based alcohol screening and brief intervention for university students: a randomized trial

IMPORTANCE

Unhealthy alcohol use is a leading contributor to the global burden of disease, particularly among young people. Systematic reviews suggest efficacy of web-based alcohol screening and brief intervention and call for effectiveness trials in settings where it could be sustainably delivered.

OBJECTIVE

To evaluate a national web-based alcohol screening and brief intervention program.

DESIGN, SETTING, AND PARTICIPANTS

A multisite, double-blind, parallel-group, individually randomized trial was conducted at 7 New Zealand universities. In April and May of 2010, invitations containing hyperlinks to the Alcohol Use Disorders Identification Test-Consumption (AUDIT-C) screening test were e-mailed to 14,991 students aged 17 to 24 years.

INTERVENTIONS

Participants who screened positive (AUDIT-C score ≥4) were randomized to undergo screening alone or to 10 minutes of assessment and feedback (including comparisons with medical guidelines and peer norms) on alcohol expenditure, peak blood alcohol concentration, alcohol dependence, and access to help and information.

MAIN OUTCOMES AND MEASURES

A fully automated 5-month follow-up assessment was conducted that measured 6 primary outcomes: consumption per typical occasion, drinking frequency, volume of alcohol consumed, an academic problems score, and whether participants exceeded medical guidelines for acute harm (binge drinking) and chronic harm (heavy drinking). A Bonferroni-corrected significance threshold of .0083 was used to account for the 6 comparisons and a sensitivity analysis was used to assess possible attrition bias.

RESULTS

Of 5135 students screened, 3422 scored 4 or greater and were randomized, and 83% were followed up. There was a significant effect on 1 of the 6 prespecified outcomes. Relative to control participants, those who received intervention consumed less alcohol per typical drinking occasion (median 4 drinks [interquartile range {IQR}, 2-8] vs 5 drinks [IQR 2-8]; rate ratio [RR], 0.93 [99.17% CI, 0.86-1.00]; P = .005) but not less often (RR, 0.95 [99.17% CI, 0.88-1.03]; P = .08) or less overall (RR, 0.95 [99.17% CI, 0.81-1.10]; P = .33). Academic problem scores were not lower (RR, 0.91 [99.17% CI, 0.76-1.08]; P = .14) and effects on the risks of binge drinking (odds ratio [OR], 0.84 [99.17% CI, 0.67-1.05]; P = .04) and heavy drinking (OR, 0.77 [99.17% CI, 0.56-1.05]; P = .03) were not significantly significant. In a sensitivity analysis accounting for attrition, the effect on alcohol per typical drinking occasion was no longer statistically significant.

CONCLUSIONS AND RELEVANCE

A national web-based alcohol screening and brief intervention program produced no significant reductions in the frequency or overall volume of drinking or academic problems. There remains a possibility of a small reduction in the amount of alcohol consumed per typical drinking occasion.

TRIAL REGISTRATION

anzctr.org.au Identifier: ACTRN12610000279022.

Alcohol assessment and feedback by email for university students: main findings from a randomised controlled trial

BACKGROUND

Brief interventions can be efficacious in changing alcohol consumption and increasingly take advantage of the internet to reach high-risk populations such as students.

AIMS

To evaluate the effectiveness of a brief online intervention, controlling for the possible effects of the research process.

METHOD

A three-arm parallel groups design was used to explore the magnitude of the feedback and assessment component effects. The three groups were: alcohol assessment and feedback (group 1); alcohol assessment only without feedback (group 2); and no contact, and thus neither assessment nor feedback (group 3). Outcomes were evaluated after 3 months via an invitation to participate in a brief cross-sectional lifestyle survey. The study was undertaken in two universities randomising the email addresses of all 14 910 students (the AMADEUS-1 study, trial registration: ISRCTN28328154).

RESULTS

Overall, 52% (n = 7809) of students completed follow-up, with small differences in attrition between the three groups. For each of the two primary outcomes, there was one statistically significant difference between groups, with group 1 having 3.7% fewer risky drinkers at follow-up than group 3 (P = 0.006) and group 2 scoring 0.16 points lower than group 3 on the three alcohol consumption questions from the Alcohol Use Disorders Identification Test (AUDIT-C) (P = 0.039).

CONCLUSIONS

This study provides some evidence of population-level benefit attained through intervening with individual students.

Alcohol assessment & feedback by e-mail for university student hazardous and harmful drinkers: study protocol for the AMADEUS-2 randomised controlled trial

BACKGROUND

Alcohol is responsible for a large and growing proportion of the global burden of disease, as well as being the cause of social problems. Brief interventions are one component of comprehensive policy measures necessary to reduce these harms. Brief interventions increasingly take advantage of the Internet to reach large numbers of high risk groups such as students. The research literature on the efficacy and effectiveness of online interventions is developing rapidly. Although many studies show benefits in the form of reduced consumption, other intervention studies show no effects, for reasons that are unclear. Sweden became the first country in the world to implement a national system in which all university students are offered a brief online intervention via an e-mail.

METHODS/DESIGN

This randomized controlled trial (RCT) aims to evaluate the effectiveness of this national system comprising a brief online intervention among university students who are hazardous and harmful drinkers. This study employs a conventional RCT design in which screening to determine eligibility precedes random allocation to immediate or delayed access to online intervention. The online intervention evaluated comprises three main components; assessment, normative feedback and advice on reducing drinking. Screening is confined to a single question in order to minimise assessment reactivity and to prevent contamination. Outcomes will be evaluated after 2 months, with total weekly alcohol consumption being the primary outcome measure. Invitations to participate are provided by e-mail to approximately 55,000 students in 9 Swedish universities.

DISCUSSION

This RCT evaluates routine service provision in Swedish universities via a delay in offer of intervention to the control group. It evaluates effects in the key population for whom this intervention has been designed. Study findings will inform the further development of the national service provision.

TRIAL REGISTRATION

ISRCTN02335307.

Alcohol email assessment and feedback study dismantling effectiveness for university students (AMADEUS-1): study protocol for a randomized controlled trial

Background

Alcohol causes huge problems for population health and for society, which require interventions with individuals as well as populations to prevent and reduce harms. Brief interventions can be effective and increasingly take advantage of the internet to reach high-risk groups such as students. The research literature on the effectiveness of online interventions is developing rapidly and is confronted by methodological challenges common to other areas of e-health including attrition and assessment reactivity and in the design of control conditions.

Methods/design

The study aim is to evaluate the effectiveness of a brief online intervention, employing a randomized controlled trial (RCT) design that takes account of baseline assessment reactivity, and other possible effects of the research process. Outcomes will be evaluated after 3 months both among student populations as a whole including for a randomized no contact control group and among those who are risky drinkers randomized to brief assessment and feedback (routine practice) or to brief assessment only. A three-arm parallel groups trial will also allow exploration of the magnitude of the feedback and assessment component effects. The trial will be undertaken simultaneously in 2 universities randomizing approximately 15,300 students who will all be blinded to trial participation. All participants will be offered routine practice intervention at the end of the study.

Discussion

This trial informs the development of routine service delivery in Swedish universities and more broadly contributes a new approach to the study of the effectiveness of online interventions in student populations, with relevance to behaviors other than alcohol consumption. The use of blinding and deception in this study raise ethical issues that warrant further attention.

Trial registration

ISRCTN28328154

Effectiveness of E-self-help interventions for curbing adult problem drinking: a meta-analysis

Background

Self-help interventions without professional contact to curb adult problem drinking in the community are increasingly being delivered via the Internet.

Objective

The objective of this meta-analysis was to assess the overall effectiveness of these eHealth interventions.

Methods

In all, 9 randomized controlled trials (RCTs), all from high-income countries, with 9 comparison conditions and a total of 1553 participants, were identified, and their combined effectiveness in reducing alcohol consumption was evaluated by means of a meta-analysis.

Results

An overall medium effect size (g = 0.44, 95% CI 0.17-0.71, random effect model) was found for the 9 studies, all of which compared no-contact interventions to control conditions. The medium effect was maintained (g = 0.39; 95% CI 0.23-0.57, random effect model) after exclusion of two outliers. Type of control group, treatment location, type of analysis, and sample size did not have differential impacts on treatment outcome. A significant difference (P = .04) emerged between single-session personalized normative feedback interventions (g = 0.27, 95% CI 0.11-0.43) and more extended e- self-help (g = 0.61, 95% CI 0.33-0.90).

Conclusion

E-self-help interventions without professional contact are effective in curbing adult problem drinking in high-income countries. In view of the easy scalability and low dissemination costs of such interventions, we recommend exploration of whether these could broaden the scope of effective public health interventions in low- and middle-income countries as well.