itMatters: Optimization of an online intervention to prevent sexually transmitted infections in college students

Decision-making in the multiphase optimization strategy: Applying decision analysis for intervention value efficiency to optimize an information leaflet to promote key antecedents of medication adherence

Advances in the multiphase optimization strategy (MOST) have suggested a new approach, decision analysis for intervention value efficiency (DAIVE), for selecting an optimized intervention based on the results of a factorial optimization trial. The new approach opens possibilities to select optimized interventions based on multiple valued outcomes. We applied DAIVE to identify an optimized information leaflet intended to support eventual adherence to adjuvant endocrine therapy for women with breast cancer. We used empirical performance data for five candidate leaflet components on three hypothesized antecedents of adherence: beliefs about the medication, objective knowledge about AET, and satisfaction with medication information. Using data from a 25 factorial trial (n = 1603), we applied the following steps: (i) We used Bayesian factorial analysis of variance to estimate main and interaction effects for the five factors on the three outcomes. (ii) We used posterior distributions for main and interaction effects to estimate expected outcomes for each leaflet version (32 total). (iii) We scaled and combined outcomes using a linear value function with predetermined weights indicating the relative importance of outcomes. (iv) We identified the leaflet that maximized the value function as the optimized leaflet, and we systematically varied outcome weights to explore robustness. The optimized leaflet included two candidate components, side-effects, and patient input, set to their higher levels. Selection was generally robust to weight variations consistent with the initial preferences for three outcomes. DAIVE enables selection of optimized interventions with the best-expected performance on multiple outcomes.

Innovation in the Delivery of Behavioral Health Services

Several factors motivate the need for innovation to improve the delivery of behavioral health services, including increased rates of mental health and substance use disorders, limited access to services, inconsistent use of evidence-based practices, and persistent racial and ethnic disparities. This narrative review identifies promising innovations that address these challenges, assesses empirical evidence for the effectiveness of these innovations and the extent to which they have been adopted and implemented, and suggests next steps for research. We review five categories of innovations: organizational models, including a range of novel locations for providing services and new ways of organizing services within and across sites; information and communication technologies; workforce; treatment technologies; and policy and regulatory changes. We conclude by discussing the need to strengthen and accelerate the contributions of implementation science to close the gap between the launch of innovative behavioral health services and their widespread use.

Recruiting and retaining first-year college students in online health research: Implementation considerations

Objective: Decreasing participation in intervention research among college students has implications for the external validity of behavioral intervention research. We describe recruitment and retention strategies used to promote participation in intervention research across a series of four randomized experiments. Method: We report the recruitment and retention rates by school for each experiment and qualitative feedback from students about recommendations for improving research participation. Results: There was considerable variation among schools' recruitment (4.9% to 64.7%) and retention (12% to 67.8%) rates. Student feedback suggested study timing (e.g., early in the semester), communication strategies (e.g., social media), and incentive structure (e.g., guaranteed incentives) could improve research participation. The highest survey participation rate was observed at the university which mandated students to complete the intervention (but not the survey). Conclusions: Intervention scientists must consider the population and study context to make informed decisions related to recruitment and retention strategies.

Investigation of Active Ingredients Within Internet-Delivered Cognitive Behavioral Therapy for Depression: A Randomized Optimization Trial

Importance

There is limited understanding of how complex evidence-based psychological interventions such as cognitive behavioral therapy (CBT) for depression work. Identifying active ingredients may help to make therapy more potent, brief, and scalable.

Objective

To test the individual main effects and interactions of 7 treatment components within internet-delivered CBT for depression to investigate its active ingredients.

Design, Setting, and Participants

This randomized optimization trial using a 32-condition, balanced, fractional factorial optimization experiment (IMPROVE-2) recruited adults with depression (Patient Health Questionnaire-9 [PHQ-9] score ≥10) from internet advertising and the UK National Health Service Improving Access to Psychological Therapies service. Participants were randomized from July 7, 2015, to March 29, 2017, with follow-up for 6 months after treatment until December 29, 2017. Data were analyzed from July 2018 to April 2023.

Interventions

Participants were randomized with equal probability to 7 experimental factors within the internet CBT platform, each reflecting the presence vs absence of specific treatment components (activity scheduling, functional analysis, thought challenging, relaxation, concreteness training, absorption, and self-compassion training).

Main Outcomes and Measures

The primary outcome was depression symptoms (PHQ-9 score). Secondary outcomes include anxiety symptoms and work, home, and social functioning.

Results

Among 767 participants (mean age [SD] age, 38.5 [11.62] years; range, 18-76 years; 635 women [82.8%]), 506 (66%) completed the 6-month posttreatment follow-up. On average, participants receiving internet-delivered CBT had reduced depression (pre-to-posttreatment difference in PHQ-9 score, -7.79 [90% CI, -8.21 to -7.37]; 6-month follow-up difference in PHQ-9 score, -8.63 [90% CI, -9.04 to -8.22]). A baseline score-adjusted analysis of covariance model using effect-coded intervention variables (-1 or +1) found no main effect on depression symptoms for the presence vs absence of activity scheduling, functional analysis, thought challenging, relaxation, concreteness training, or self-compassion training (posttreatment: largest difference in PHQ-9 score [functional analysis], -0.09 [90% CI, -0.56 to 0.39]; 6-month follow-up: largest difference in PHQ-9 score [relaxation], -0.18 [90% CI, -0.61 to 0.25]). Only absorption training had a significant main effect on depressive symptoms at 6-month follow-up (posttreatment difference in PHQ-9 score, 0.21 [90% CI, -0.27 to 0.68]; 6-month follow-up difference in PHQ-9 score, -0.54, [90% CI, -0.97 to -0.11]).

Conclusions and Relevance

In this randomized optimization trial, all components of internet-delivered CBT except absorption training did not significantly reduce depression symptoms relative to their absence despite an overall average reduction in symptoms. The findings suggest that treatment benefit from internet-delivered CBT probably accrues from spontaneous remission, factors common to all CBT components (eg, structure, making active plans), and nonspecific therapy factors (eg, positive expectancy), with the possible exception of absorption focused on enhancing direct contact with positive reinforcers.

Trial Registration

isrctn.org Identifier: ISRCTN24117387.

Multiphase optimization strategy: How to build more effective, affordable, scalable and efficient social and behavioural oral health interventions

This commentary introduces the field of social behavioural oral health interventions to the multiphase optimization strategy (MOST). MOST is a principled framework for the development, optimization and evaluation of multicomponent interventions. Drawing from the fields of engineering, behavioural science, economics, decision science and public health, intervention optimization requires a strategic balance of effectiveness with affordability, scalability and efficiency. We argue that interventions developed using MOST are more likely to maximize the public health impact of social behavioural oral health interventions.

Perceived Similarity, Utility, and Social Realism as Potential Mediators of the Link between Pornography Use and Condomless Sex

One of the most consistent findings in content analyses of popular, commonly consumed pornography is the near absence of condoms. A recent meta-analysis found that pornography use is associated with an increased likelihood of condomless sex, but the studies available for analysis rarely included measures of potential cognitive mediators underlying the association. Following the sexual script acquisition, activation, application model (₃AM) of mediated sexual socialization and the differential susceptibility to media effects model (DSMM), the present study examined whether linkages between pornography use and condomless sex are mediated by perceived similarity to actors in pornography and heightened perceptions of pornography's utility and social realism. Social realism and similarity mediated the association between pornography consumption frequency and condomless sex in simple mediation models, but only social realism remained significant in a parallel process model inclusive of all three mediators.

Human-centered design methods to achieve preparation phase goals in the multiphase optimization strategy framework

Background

The public health impact of behavioral and biobehavioral interventions to prevent and treat mental health and substance use problems hinges on developing methods to strategically maximize their effectiveness, affordability, scalability, and efficiency.

Methods

The multiphase optimization strategy (MOST) is an innovative, principled framework that guides the development of multicomponent interventions. Each phase of MOST (Preparation, Optimization, Evaluation) has explicit goals and a range of appropriate research methods to achieve them. Methods for attaining Optimization and Evaluation phase goals are well-developed. However, methods used in the Preparation phase are often highly researcher-specific, and concrete ways to achieve Preparation phase goals are a priority area for further development.

Results

We propose that the discover, design, build, and test (DDBT) framework provides a theory-driven and methods-rich roadmap for achieving the goals of the Preparation phase of MOST, including specifying the conceptual model, identifying and testing candidate intervention components, and defining the optimization objective. The DDBT framework capitalizes on strategies from the field of human-centered design and implementation science to drive its data collection methods.

Conclusions

MOST and DDBT share many conceptual features, including an explicit focus on implementation determinants, being iterative and flexible, and designing interventions for the greatest public health impact. The proposed synthesized DDBT/MOST approach integrates DDBT into the Preparation phase of MOST thereby providing a framework for rigorous and efficient intervention development research to bolster the success of intervention optimization.

Plain Language Summary

1. What is already known about the topic? Optimizing behavioral interventions to balance effectiveness with affordability, scalability, and efficiency requires a significant investment in intervention development.2. What does this paper add? This paper provides a structured approach to integrating human-centered design principles into the Preparation phase of the multiphase optimization strategy (MOST).3. What are the implications for practice, research, or policy? The proposed synthesized model provides a framework for rigorous and efficient intervention development research in the Preparation phase of MOST that will ensure the success of intervention optimization and contribute to improving public health impact of mental health and substance use interventions.

The preparation phase in the multiphase optimization strategy (MOST): a systematic review and introduction of a reporting checklist

Multicomponent behavioral interventions developed using the multiphase optimization strategy (MOST) framework offer important advantages over alternative intervention development models by focusing on outcomes within constraints relevant for effective dissemination. MOST consists of three phases: preparation, optimization, and evaluation. The preparation phase is critical to establishing the foundation for the optimization and evaluation phases; thus, detailed reporting is critical to enhancing rigor and reproducibility. A systematic review of published research using the MOST framework was conducted. A structured framework was used to describe and summarize the use of MOST terminology (i.e., preparation phase and optimization objective) and the presentation of preparation work, the conceptual model, and the optimization. Fifty-eight articles were reviewed and the majority focused on either describing the methodology or presenting results of an optimization trial (n = 38, 66%). Although almost all articles identified intervention components (96%), there was considerable variability in the degree to which authors fully described other elements of MOST. In particular, there was less consistency in use of MOST terminology. Reporting on the MOST preparation phase is varied, and there is a need for increased focus on explicit articulation of key design elements and rationale of the preparation phase. The proposed checklist for reporting MOST studies would significantly advance the use of this emerging methodology and improve implementation and dissemination of MOST. Accurate reporting is essential to reproducibility and rigor of scientific trials as it ensures future research fully understands not only the methodology, but the rationale for intervention and optimization decisions.

Optimization Methods and Implementation Science: An Opportunity for Behavioral and Biobehavioral Interventions

This editorial introduces the multiphase optimization strategy (MOST), a principled framework for the development, optimization and evaluation of multicomponent interventions, to the field of implementation science. We suggest that MOST may be integrated with implementation science to advance the field, moving closer towards the ultimate goal of disseminating effective interventions to those in need. We offer three potential ways MOST may advance implementation science: (1) development of an effective and immediately scalable intervention; (2) adaptation of interventions to local contexts; and (3) optimization of the implementation of an intervention itself. Our goal is to inspire the integration of MOST with implementation science across a number of public health contexts.

Using factorial mediation analysis to better understand the effects of interventions

To improve understanding of how interventions work or why they do not work, there is need for methods of testing hypotheses about the causal mechanisms underlying the individual and combined effects of the components that make up interventions. Factorial mediation analysis, i.e., mediation analysis applied to data from a factorial optimization trial, enables testing such hypotheses. In this commentary, we demonstrate how factorial mediation analysis can contribute detailed information about an intervention's causal mechanisms. We briefly review the multiphase optimization strategy (MOST) and the factorial experiment. We use an empirical example from a 25 factorial optimization trial to demonstrate how factorial mediation analysis opens possibilities for better understanding the individual and combined effects of intervention components. Factorial mediation analysis has important potential to advance theory about interventions and to inform intervention improvements.

The multiphase optimization strategy (MOST) in child maltreatment prevention research

Each year hundreds of thousands of children and families receive behavioral interventions designed to prevent child maltreatment; yet rates of maltreatment have not declined in over a decade. To reduce the prevalence and prevent the life-long negative consequences of child maltreatment, behavioral interventions must not only be effective, but also affordable, scalable, and efficient to meet the demand for these services. An innovative approach to intervention science is needed. The purpose of this article is to introduce the multiphase optimization strategy (MOST) to the field of child maltreatment prevention. MOST is an engineering-inspired framework for developing, optimizing, and evaluating multicomponent behavioral interventions. MOST enables intervention scientists to empirically examine the performance of each intervention component, independently and in combination. Using a hypothetical example of a home visiting intervention and artificial data, this article demonstrates how MOST may be used to optimize the content of a parent-focused in-home intervention and the engagement strategies of an intervention to increase completion rate to identify an intervention that is effective, efficient, economical, and scalable. We suggest that MOST will ultimately improve prevention science and hasten the progress of translational science to prevent child maltreatment.

A Hybrid Evaluation-Optimization Trial to Evaluate an Intervention Targeting the Intersection of Alcohol and Sex in College Students and Simultaneously Test an Additional Component Aimed at Preventing Sexual Violence

BACKGROUND

Using the multiphase optimization strategy (MOST), we previously developed and optimized an online behavioral intervention, itMatters, aimed at reducing the risk of sexually transmitted infections (STI) among first-year college students by targeting the intersection of alcohol use and sexual behaviors.

PURPOSE

We had two goals: (a) to evaluate the optimized itMatters intervention and (b) to determine whether the candidate sexual violence prevention (SVP) component (included at the request of participating universities) had a detectable effect and therefore should be added to create a new version of itMatters. We also describe the hybrid evaluation-optimization trial we conducted to accomplish these two goals in a single experiment.

METHODS

First year college students (N = 3,098) at four universities in the USA were individually randomized in a hybrid evaluation-optimization 2 × 2 factorial trial. Data were analyzed using regression models, with pre-test outcome variables included as covariates in the models. Analyses were conducted separately with (a) immediate post-test scores and (b) 60-day follow-up scores as outcome variables.

RESULTS

Experimental results indicated a significant effect of itMatters on targeted proximal outcomes (norms) and on one distal behavioral outcome (binge drinking). There were no significant effects on other behavioral outcomes, including the intersection of alcohol and sexual behaviors. In addition, there were mixed results (positive short-term effect; no effect at 60-day follow-up) of the SVP component on targeted proximal outcomes (students' self-efficacy to reduce/prevent sexual violence and perceived effectiveness of protective behavioral strategies).

CONCLUSIONS

The hybrid evaluation-optimization trial enabled us to evaluate the individual and combined effectiveness of the optimized itMatters intervention and the SVP component in a single experiment, conserving resources and providing greatly improved efficiency.

TRIAL REGISTRATION

NCT04095065.