Improving the transition from basic efficacy research to effectiveness studies: methodological issues and procedures

Effects of the Family Check-Up 4 Health on Parenting and Child Behavioral Health: A Randomized Clinical Trial in Primary Care

The Family Check-Up 4 Health (FCU4Health) is an adaptation of the Family Check-Up (FCU) for delivery in primary care settings. While maintaining the original FCU's focus on parenting and child behavioral health, we added content targeting health behaviors. This study evaluated whether the adapted FCU maintained positive effects on parenting (positive behavior support, limit setting, parental warmth) and child behavioral health (self-regulation, conduct problems, emotional problems). Pediatric (6-12 years) primary care patients with a BMI ≥ 85^th%ile (n = 240) were recruited from primary care clinics in Phoenix. Children were 75% Latino, 49% female, and 73% Medicaid recipients. This type 2 effectiveness-implementation hybrid trial compared families randomized to FCU4Health (n = 141) or usual care (n = 99). FCU4Health was delivered over a period of 6 months. This study focuses on a priori secondary outcomes included parenting and child behavioral health targets of the original FCU, assessed at baseline and 3, 6, and 12 months. Significant improvements were found for the FCU4Health condition, compared to usual care, in parenting from baseline to the 3-month assessment [β = .17 (.01; .32)]. Parenting predicted improvements in child self-regulation at 6-months [β = .17 (.03; .30)], which in turn predicted reductions in conduct problems [β = - .38 (- .51; - .23)] and emotional problems [β = - .24 (- .38; - .09)] at 12 months. Ethnicity and language of delivery (English or Spanish) did not moderate these effects. The FCU4Health can improve parenting and child behavioral health outcomes when delivered in primary care.Trial Registration Trial registration number: NCT03013309 ClinicalTrials.gov.

Programs for the Prevention of Youth Depression: Evaluation of Efficacy, Effectiveness, and Readiness for Dissemination

The objective of this study was to evaluate the current state of evidence of the effectiveness of depression prevention programs for youth, assess the degree to which current evidence supports broad implementation, and outline additional steps needed to close the gap between effectiveness and dissemination. We used the Society for Prevention Research's Standards of Evidence (Flay et al., 2005 ) to evaluate the degree to which existing depression prevention programs have established intervention efficacy, effectiveness, and readiness for dissemination. We reviewed all depression prevention programs for youth that have been evaluated in at least two published, randomized controlled trials in which the intervention was compared to a no-intervention control group. A total of 37 studies evaluating 11 different programs were reviewed with regard to depressive symptoms and diagnoses at postintervention and follow-up (at least 6 months). Eight programs demonstrated significant main effects on depressive symptoms relative to controls in multiple randomized controlled trials; 5 programs had at least 1 trial with significant main effects present at least 1 year postintervention. Two programs demonstrated efficacy for both depressive symptoms and depressive episodes across multiple independent trials. Regarding effectiveness, 6 programs had at least 1 study showing significant effects when delivered by endogenous service providers; 4 programs had significant effects in studies conducted independently of the program developers. Several programs have demonstrated promise in terms of efficacy, but no depression prevention program for children or adolescents as yet has garnered sufficient evidence of effectiveness under real-world conditions to warrant widespread dissemination at this time.

A Randomized Trial to Evaluate the Efficacy of a Web-Based HIV Behavioral Intervention for High-Risk African American Women

The aim of this study was to develop and test a cost-effective, scalable HIV behavioral intervention for African American women. Eighty-three African American women were recruited from a community health center and randomly assigned to either the web-based Safe Sistah program or to a delayed HIV education control condition. The primary outcome was self-reported condom use. Secondary measures assessed other aspects of the gender-focused training included in Safe Sistah. Participants completed self-report assessments prior to randomization, 1- and 4-months after their program experience. Across the entire study period, women in the experimental condition significantly increased their condom use relative to controls (F = 5.126, p = 0.027). Significant effects were also found for sexual communication, sex refusal, condom use after alcohol consumption, and HIV prevention knowledge. These findings indicate that this web-based program could be an important component in reducing the HIV disparities among African American women.

Psychosocial outcomes of a non-dieting based positive body image community program for overweight adults: a pilot study

BACKGROUND

The limited success of traditional diet focused obesity interventions has led to the development of alternative non-dieting approaches. The current study evaluated the impact of a community based non-dieting positive body image program for overweight/obese people on a range of psychosocial outcomes. The characteristics of this real-world sample presenting for a non-dieting weight management intervention are also described.

METHOD

Overweight and obese participants enrolled in the eight week 'No More Diets' (NMD) group program completed self-report questionnaires assessing disordered eating thoughts and behaviours, body image, motivation for exercise and psychopathology pre- and post-treatment.

RESULTS

Participants (n = 17; 16 female) were aged between 19 and 78 years, with a BMI ranging from 25.2 kg/m(2) (Overweight) to 55.9 kg/m(2) (Severely Obese). They reported elevated levels of eating disorder pathology, body shape preoccupation, depression, anxiety and stress compared to community norms (p < .05). Following treatment there were significant improvements in reported body shape preoccupation, shape concern and eating attitudes (p < .05), and clinically significant changes (small to medium effect sizes; 0.3-0.35) for improvements in reported weight concern, eating competence, stress and health evaluation. There were no changes in reported dietary restraint, emotional eating and uncontrolled eating, or eating concern (p > .05).

CONCLUSION

Individuals presenting for the NMD program demonstrated increased eating disorder pathology and more generalised psychopathology compared to community norms. The NMD program was particularly beneficial for body image and shape concern. Addressing these body image factors may help to address some of the perpetuating factors of obesity and disordered eating, which are often not addressed in the traditional diet-based weight loss interventions.

An effectiveness trial of a new enhanced dissonance eating disorder prevention program among female college students

OBJECTIVE

Efficacy trials indicate that a dissonance-based prevention program in which female high school and college students with body image concerns critique the thin-ideal reduced risk factors, eating disorder symptoms, and future eating disorder onset, but weaker effects emerged from an effectiveness trial wherein high school clinicians recruited students and delivered the program under real-world conditions. The present effectiveness trial tested whether a new enhanced dissonance version of this program produced larger effects when college clinicians recruited students and delivered the intervention using improved procedures to select, train, and supervise clinicians.

METHOD

Young women recruited from seven universities across the US (N = 408, M age = 21.6, SD = 5.64) were randomized to the dissonance intervention or an educational brochure control condition.

RESULTS

Dissonance participants showed significantly greater decreases in risk factors (thin-ideal internalization, body dissatisfaction, dieting, negative affect) and eating disorder symptoms versus controls at posttest and 1-year follow-up, resulting in medium average effect size (d = .60). Dissonance participants also reported significant improvements in psychosocial functioning, but not reduced health care utilization or unhealthy weight gain.

CONCLUSIONS

This novel multisite effectiveness trial with college clinicians found that the enhanced dissonance version of this program and the improved facilitator selection/training procedures produced average effects that were 83% larger than effects observed in the high school effectiveness trial.

Guided self-help treatment for recurrent binge eating: replication and extension

OBJECTIVE

The aim of this study was to replicate and extend results of a previous blended efficacy and effectiveness trial of a low-intensity, manual-based guided self-help form of cognitive-behavioral therapy (CBT-GSH) for the treatment of binge eating disorders in a large health maintenance organization (HMO) and to compare them with usual care.

METHODS

To extend previous findings, the investigators modified earlier recruitment and assessment approaches and conducted a randomized clinical trial to better reflect procedures that may be reasonably carried out in real-world practices. The intervention was delivered by master's-level interventionists to 160 female members of a health maintenance organization who met diagnostic criteria for recurrent binge eating. Data collected at baseline, immediately posttreatment, and at six- and 12-month follow-ups were used in intent-to-treat analyses.

RESULTS

At the 12-month follow-up, CBT-GSH resulted in greater remission from binge eating (35%, N=26) than usual care (14%, N=10) (number needed to treat=5). The CBT-GSH group also demonstrated greater improvements in dietary restraint (d=.71) and eating, shape, and weight concerns (d=1.10, 1.24, and .98, respectively) but not weight change.

CONCLUSIONS

Replication of the pattern of previous findings suggests that CBT-GSH is a robust treatment for patients with recurrent binge eating. The magnitude of changes was significantly smaller than in the original study, however, suggesting that patients recruited and assessed with less intensive procedures may respond differently from their counterparts enrolled in trials requiring more comprehensive procedures.

Promotoras as mental health practitioners in primary care: a multi-method study of an intervention to address contextual sources of depression

We assessed the role of promotoras--briefly trained community health workers--in depression care at community health centers. The intervention focused on four contextual sources of depression in underserved, low-income communities: underemployment, inadequate housing, food insecurity, and violence. A multi-method design included quantitative and ethnographic techniques to study predictors of depression and the intervention's impact. After a structured training program, primary care practitioners (PCPs) and promotoras collaboratively followed a clinical algorithm in which PCPs prescribed medications and/or arranged consultations by mental health professionals and promotoras addressed the contextual sources of depression. Based on an intake interview with 464 randomly recruited patients, 120 patients with depression were randomized to enhanced care plus the promotora contextual intervention, or to enhanced care alone. All four contextual problems emerged as strong predictors of depression (chi square, p < .05); logistic regression revealed housing and food insecurity as the most important predictors (odds ratios both 2.40, p < .05). Unexpected challenges arose in the intervention's implementation, involving infrastructure at the health centers, boundaries of the promotoras' roles, and "turf" issues with medical assistants. In the quantitative assessment, the intervention did not lead to statistically significant improvements in depression (odds ratio 4.33, confidence interval overlapping 1). Ethnographic research demonstrated a predominantly positive response to the intervention among stakeholders, including patients, promotoras, PCPs, non-professional staff workers, administrators, and community advisory board members. Due to continuing unmet mental health needs, we favor further assessment of innovative roles for community health workers.

Fidelity of implementation in Project Towards No Drug Abuse (TND): a comparison of classroom teachers and program specialists

This paper presents the results of an effectiveness trial of Project Towards No Drug Abuse [TND], in which we compared program delivery by regular classroom teachers and program specialists within the same high schools. Within 18 schools that were randomly assigned to the program or control conditions, health classrooms were assigned to program delivery by teachers or (outside) specialists. Classroom sessions were observed by pairs of observers to assess three domains of implementation fidelity: adherence, classroom process, and perceived student acceptance of the program. Pre- and immediate posttest survey data were collected from 2331 students. Of the four composite indexes of implementation fidelity that were examined, only one (quality of delivery) showed a difference between specialists and teachers, with marginally higher ratings of specialists (p < .10). Both teachers and program specialists achieved effects on three of the five immediate outcome measures, including program-specific knowledge, addiction concern, and social self-control. Students' posttest ratings of the program overall and the quality of program delivery failed to reveal differences between the teacher- and specialist-led classrooms. These results suggest that motivated, trained classroom teachers can implement evidence-based prevention programs with fidelity and achieve immediate effects.

Statistical issues in multisite effectiveness trials: the case of brief strategic family therapy for adolescent drug abuse treatment

The statistical development of the multisite Brief Strategic Family Therapy (BSFT) Trial of the National Institute on Drug Abuse's Clinical Trials Network provides a useful, real example of how an effectiveness trial can differ from an efficacy trial. In particular, two design elements distinguish this effectiveness trial from an efficacy trial. First, because the goal of the trial is to show that the use of BSFT would be an improvement on current practice, it was decided to compare BSFT to treatment as usual at each location. This decision ensures that the trial has the most ecological validity to the participating community treatment providers. Second, the desire to generalize the results to general clinical practice dictates that variability (in effect) across community treatment providers be estimated using a random effects model. These two decisions jointly influence the sample size calculations. Allowing variation in treatment as usual, will increase the variability in effect sizes across sites and estimation of this variability as a random effect necessitates a larger sample size (both number of community treatment providers and participants per community treatment provider), than is the case for a fixed site effect estimate. Details of these effects and their implications for the statistical design are presented.

Predicting the outcome of treatment

The clinical question--"Which treatment(s) for which patients with what set of subgrouping characteristics working by what mechanism(s)?"--rests at the heart of differential therapeutics. Experimentally, this question reduces to a test of how well we can predict the outcome of treatment using the treatment conditions plus other moderating and mediating variables. Reflecting the discussions held at a recent National Institute of Mental Health (NIMH) conference on psychosocial treatments, and using pediatric anxiety disorders as a case in point, we discuss the problem of prediction in treatment outcome studies from the standpoint of definition of terms, using the general linear model of prediction. We also outline types of studies that may be useful in testing potential predictors, and put forward a possible matrix of predictor variables as currently implemented in an NIMH-funded treatment outcome study of pediatric obsessive-compulsive disorder (OCD). We conclude by making specific suggestions for implementing a broader approach to the study of predictors.