Implementing treatment and protecting the validity of the independent variable in treatment matching studies

A Brief Negotiation Interview Adherence Scale for Smoking Cessation: A psychometric evaluation

Practitioner adherence to the Brief Negotiation Interview (BNI) for high-risk alcohol consumption and opioid use disorder can be measured using the BNI Adherence Scale, for alcohol (BAS-A) and opioids (BAS-O), respectively. However, no psychometrically validated brief intervention adherence scale for smoking cessation has been reported in the literature. Our objective was to develop and examine the psychometric properties of a BAS for smoking cessation (BASS). In the context of a clinical trial of the BNI in an emergency department (ED)-which incorporates motivational interviewing (MI), feedback, and behavioral contracting-plus nicotine replacement therapy (NRT), and a Smokers' Quitline referral and brochure (BNI), compared with brochure-only (control), we developed and examined the psychometric properties of the BAS-S, a scale that requires raters to answer whether each critical action of the BNI was implemented. Three independent raters rated three hundred and eighty-eight audio-recorded BNI sessions. The results indicated that the BAS-S had excellent internal consistency, and discriminant validity, inter-rater reliability, and construct validity. The following 3-factor (10-item) solution accounted for 43% of the variance: factor 1, "Feedback,", factor 2, "NRT Motivation," and factor 3, "Plan Negotiation." The study found predictive validity for the Feedback factor, suggesting that patients who were provided feedback on the harms of their smoking were significantly less likely to achieve biologically confirmed 7-day tobacco abstinence at their 3-month follow-up than those who were not provided such feedback (p < 0.03). The BAS-S is a psychometrically valid measure of adherence to the BNI for smoking cessation.

Distilling the Core Elements of Family Therapy for Adolescent Substance Use: Conceptual and Empirical Solutions

This article describes several barriers to widespread dissemination of manualized family-based treatments for adolescent substance use (ASU). We then offer a highly promising solution for adopting and sustaining family therapy in usual care: distilling the core practice elements of empirically validated family therapy models for ASU. We present a conceptual distillation of family therapy for ASU grounded in existing observational fidelity measures for three manualized models, a process that yielded four core elements: Family Engagement, Relational Reframing, Family Behavior Change, and Family Restructuring. We then introduce an innovative empirical method for distilling core elements that can serve as a template for rigorous distillation of other treatment approaches. Finally, we discuss how core elements can enhance family therapy services within the diverse workforce of usual care for ASU.

Concurrent Treatment with Prolonged Exposure for Co-Occurring Full or Subthreshold Posttraumatic Stress Disorder and Substance Use Disorders: A Randomized Clinical Trial

BACKGROUND

To test whether an integrated prolonged exposure (PE) approach could address posttraumatic stress disorder (PTSD) symptoms effectively in individuals with co-occurring substance use disorders (SUD), we compared concurrent treatment of PTSD and SUD using PE (COPE) to relapse prevention therapy (RPT) for SUD and an active monitoring control group (AMCG).

METHODS

We conducted a randomized 12-week trial with participants (n = 110; 64% males; 59% African Americans) who met Diagnostic and Statistical Manual of Mental Disorders, fourth edition, text revision criteria for full or subthreshold PTSD and SUD. Participants were randomly assigned to COPE (n = 39), RPT (n = 43), or AMCG (n = 28).

RESULTS

At the end-of-treatment, COPE and RPT demonstrated greater reduction in PTSD symptom severity relative to AMCG (COPE-AMCG = -34.06, p < 0.001; RPT-AMCG = -22.58, p = 0.002). Although the difference between COPE and RPT was not significant in the complete sample, the subset of participants with full (vs. subthreshold) PTSD demonstrated significantly greater reduction of PTSD severity in COPE relative to RPT. Both treatments were superior to AMCG in reducing the days of primary substance use (COPE-AMCG = -0.97, p = 0.01; RPT-AMCG = -2.07, p < 0.001). Relative to COPE, RPT showed significantly more improvement in SUD outcome at end-of-treatment (RPT-COPE = -1.10, p = 0.047). At 3-month follow-up, COPE and RPT maintained their treatment gains and were not significantly different in PTSD severity or days of primary substance use.

CONCLUSION

COPE and RPT reduced PTSD and SUD severity in participants with PTSD + SUD. Findings suggest that among those with full PTSD, COPE improves PTSD symptoms more than a SUD-only treatment. The use of PE for PTSD was associated with significant decreases in PTSD symptoms without worsening of substance use.

Interventions to reduce intimate partner violence perpetration among people with substance use disorders

The social and economic cost of intimate partner violence (IPV) is exorbitant and highlights the need for policy reform as it pertains to IPV interventions at a global level. There are multiple variables associated with the aetiology of IPV and, hence, multiple treatment needs must be considered. Substance use is one of several factors likely to influence the occurrence of IPV, but often goes unaddressed in standard treatment approaches. This review will discuss several treatment models for substance using offenders of IPV, including Psycho-educational Models, Cognitive Behavioural Therapy, Couples' Treatments, Parenting Programmes, Integration of Care Models, and Pharmacotherapies. Clinical recommendations will be discussed. Treatment outcomes among substance using offenders of IPV may be improved by implementing changes in protocol that increase diagnostic evaluations, integrate care with evidence-based models, require limits to the number of offenders in a group, and require qualifications for clinicians who treat offenders (licensed and trained psychologists, social workers, and/or psychiatrists).

Behaviour change techniques: the development and evaluation of a taxonomic method for reporting and describing behaviour change interventions (a suite of five studies involving consensus methods, randomised controlled trials and analysis of qualitative data)

BACKGROUND

Meeting global health challenges requires effective behaviour change interventions (BCIs). This depends on advancing the science of behaviour change which, in turn, depends on accurate intervention reporting. Current reporting often lacks detail, preventing accurate replication and implementation. Recent developments have specified intervention content into behaviour change techniques (BCTs) - the 'active ingredients', for example goal-setting, self-monitoring of behaviour. BCTs are 'the smallest components compatible with retaining the postulated active ingredients, i.e. the proposed mechanisms of change. They can be used alone or in combination with other BCTs' (Michie S, Johnston M. Theories and techniques of behaviour change: developing a cumulative science of behaviour change. Health Psychol Rev 2012;6:1-6). Domain-specific taxonomies of BCTs have been developed, for example healthy eating and physical activity, smoking cessation and alcohol consumption. We need to build on these to develop an internationally shared language for specifying and developing interventions. This technology can be used for synthesising evidence, implementing effective interventions and testing theory. It has enormous potential added value for science and global health.

OBJECTIVE

(1) To develop a method of specifying content of BCIs in terms of component BCTs; (2) to lay a foundation for a comprehensive methodology applicable to different types of complex interventions; (3) to develop resources to support application of the taxonomy; and (4) to achieve multidisciplinary and international acceptance for future development.

DESIGN AND PARTICIPANTS

Four hundred participants (systematic reviewers, researchers, practitioners, policy-makers) from 12 countries engaged in investigating, designing and/or delivering BCIs. Development of the taxonomy involved a Delphi procedure, an iterative process of revisions and consultation with 41 international experts; hierarchical structure of the list was developed using inductive 'bottom-up' and theory-driven 'top-down' open-sort procedures (n = 36); training in use of the taxonomy (1-day workshops and distance group tutorials) (n = 161) was evaluated by changes in intercoder reliability and validity (agreement with expert consensus); evaluating the taxonomy for coding interventions was assessed by reliability (intercoder; test-retest) and validity (n = 40 trained coders); and evaluating the taxonomy for writing descriptions was assessed by reliability (intercoder; test-retest) and by experimentally testing its value (n = 190).

RESULTS

Ninety-three distinct, non-overlapping BCTs with clear labels and definitions formed Behaviour Change Technique Taxonomy version 1 (BCTTv1). BCTs clustered into 16 groupings using a 'bottom-up' open-sort procedure; there was overlap between these and groupings produced by a theory-driven, 'top-down' procedure. Both training methods improved validity (both p < 0.05), doubled the proportion of coders achieving competence and improved confidence in identifying BCTs in workshops (both p < 0.001) but did not improve intercoder reliability. Good intercoder reliability was observed for 80 of the 93 BCTs. Good within-coder agreement was observed after 1 month (p < 0.001). Validity was good for 14 of 15 BCTs in the descriptions. The usefulness of BCTTv1 to report descriptions of observed interventions had mixed results.

CONCLUSIONS

The developed taxonomy (BCTTv1) provides a methodology for identifying content of complex BCIs and a foundation for international cross-disciplinary collaboration for developing more effective interventions to improve health. Further work is needed to examine its usefulness for reporting interventions.

FUNDING

This project was funded by the Medical Research Council Ref: G0901474/1. Funding also came from the Peninsula Collaboration for Leadership in Applied Health Research and Care.

Randomized trial of web-based training to promote counselor use of cognitive behavioral therapy skills in client sessions

BACKGROUND AND METHODS

The authors designed and delivered an innovative Web course on cognitive behavioral therapy (CBT), a specific empirically based treatment, to a diverse group of addiction counselors and supervisors in 54 addiction units across the country, and conducted a randomized controlled trial of its effectiveness with 127 counselors. The primary focus of the trial was to assess "adequate adherence to CBT practice" after training as judged by raters blinded to training condition who listened to audiotapes of actual client sessions. Counselors who passed were judged to satisfy 2 criteria: (a) low pass or greater on at least 1 of 3 "CBT-generic skills" assessing session structure; and (b) low pass or greater on at least 1 of 3 "CBT-specific skills" related to use of functional analysis, cognitive skills practice, or behavioral skills practice.

RESULTS

Although the counselors' use of CBT skills in sessions increased after Web course training, it was not statistically significant and not larger than the gain of control-group counselors trained with a written CBT manual.

Efficacy of disulfiram and Twelve Step Facilitation in cocaine-dependent individuals maintained on methadone: a randomized placebo-controlled trial

BACKGROUND

Cocaine use remains a major problem within methadone maintenance programs. Disulfiram's efficacy in reducing cocaine use has been demonstrated in several trials, but its relative efficacy among individuals who use versus abstain from alcohol remains unclear. Treatment approaches which seek to enhance substance users' involvement in self-help activities (Twelve Step Facilitation, TSF) have been associated with better outcomes among alcohol and cocaine users, but have rarely been evaluated among methadone-maintained cocaine-opioid users.

METHODS

We conducted a randomized, placebo-controlled, double blind (for medication condition), factorial (2×2) trial with 4 treatment conditions: Disulfiram plus TSF, disulfiram plus standard counseling only, placebo plus TSF, and placebo plus standard counseling in the context of a community-based methadone maintenance program. Participants (N=112) received either disulfiram (250 mg/d) or placebo in conjunction with daily methadone maintenance.

RESULTS

Assignment to TSF was associated with less cocaine use throughout treatment and a higher number of cocaine-negative urines. While there were no significant main effects of disulfiram versus placebo, individuals without an alcohol use disorder demonstrated greater reductions in cocaine use over time when assigned to disulfiram.

CONCLUSIONS

TSF appears feasible in this methadone maintenance program and was associated with modest reductions in cocaine use, an often intractable problem in this setting. Support for the efficacy of disulfiram was weaker, as it appeared effective only for those without a current alcohol use disorder for this sample.

Implementation fidelity of Multidimensional Family Therapy in an international trial

Implementation fidelity, a critical aspect of clinical trials research that establishes adequate delivery of the treatment as prescribed in treatment manuals and protocols, is also essential to the successful implementation of effective programs into new practice settings. Although infrequently studied in the drug abuse field, stronger implementation fidelity has been linked to better outcomes in practice but appears to be more difficult to achieve with greater distance from model developers. In the INternational CAnnabis Need for Treatment (INCANT) multi-national randomized clinical trial, investigators tested the effectiveness of Multidimensional Family Therapy (MDFT) in comparison to individual psychotherapy (IP) in Brussels, Berlin, Paris, The Hague, and Geneva with 450 adolescents with a cannabis use disorder and their parents. This study reports on the implementation fidelity of MDFT across these five Western European sites in terms of treatment adherence, dose and program differentiation, and discusses possible implications for international implementation efforts.

Identification of behaviour change techniques to reduce excessive alcohol consumption

BACKGROUND

  Interventions to reduce excessive alcohol consumption have a small but important effect, but a better understanding is needed of their 'active ingredients'.

AIMS

  This study aimed to (i) develop a reliable taxonomy of behaviour change techniques (BCTs) used in interventions to reduce excessive alcohol consumption (not to treat alcohol dependence) and (ii) to assess whether use of specific BCTs in brief interventions might be associated with improved effectiveness.

METHODS

  A selection of guidance documents and treatment manuals, identified via expert consultation, were analysed into BCTs by two coders. The resulting taxonomy of BCTs was applied to the Cochrane Review of brief alcohol interventions, and the associations between the BCTs and effectiveness were investigated using meta-regression.

FINDINGS

  Forty-two BCTs were identified, 34 from guidance documents and an additional eight from treatment manuals, with average inter-rater agreement of 80%. Analyses revealed that brief interventions that included the BCT 'prompt self-recording' (P = 0.002) were associated with larger effect sizes.

CONCLUSIONS

  It is possible to identify specific behaviour change techniques reliably in manuals and guidelines for interventions to reduce excessive alcohol consumption. In brief interventions, promoting self-monitoring is associated with improved outcomes. More research is needed to identify other behaviour change techniques or groupings of behaviour change techniques that can produce optimal results in brief interventions and to extend the method to more intensive interventions and treatment of alcohol dependence.

Training and retaining staff to competently deliver an evidence-based practice: the role of staff attributes and perceptions of organizational functioning

Within the context of an initiative to implement evidence-based practices (EBPs) for adolescents with substance use disorders, this study examined the extent to which staff factors measured at an initial EBP training workshop were predictive of EBP competence and turnover status of staff (N = 121) measured 6, 9, and 12 months posttraining. By the final assessment point, 52.3% of staff transitioned to the employed/EBP-competent category, 26.6% transitioned to the not employed/not EBP-competent category, 4.6% transitioned to the not employed/EBP-competent category, and 16.5% had not transitioned out of the initial category. Multilevel multinomial regression analysis identified several measures that were significant predictors of staff transitions to the not employed/not EBP-competent category (e.g., program needs, job satisfaction, burnout) and transitions to the employed/EBP-competent category (e.g., months in position, pressures for change, influence). Findings have implications for the development and testing of strategies to train and retain staff to deliver EBPs in practice settings.

Intervention fidelity: aspects of complementary and alternative medicine research

BACKGROUND

The Treatment Fidelity Workgroup (TFW) established by the National Institutes of Health provides a 5-point structure for intervention fidelity: dosing, interventionists' consistency, intervention delivery, receipt, and enactment of the intervention. Using our reflexology trial, we apply the first 3 points.

OBJECTIVES

Study objectives were to (1) evaluate key dosage dimensions associated with complementary and alternative medicine (CAM) research, (2) evaluate approaches to interventionists' consistency of delivery of CAM protocols, and (3) evaluate and discuss data that reflect CAM intervention fidelity.

METHODS

Women with late-stage breast cancer (N = 318) were randomly assigned to either 4 weeks of reflexology, placebo, or standard care.

RESULTS

Dosing consists of 3 dimensions: frequency (4 sessions), duration (30 minutes), and interval between sessions (5-9 days). Interventionist consistency revealed more than a 90% accuracy rate in following the protocol; 84% and 89% completion rate of the 4 sessions in the reflexology and placebo groups, respectively; and no differences in attrition after randomization between reflexology and placebo groups (17% and 15%, respectively). Intervention delivery, examined through debriefing data, indicated a significantly higher rate of correct guesses on group assignment in the reflexology group as compared with the placebo (82% vs 46%, P = .0002).

CONCLUSION

This study points out the relevance of dosing, interventionists' consistency, and delivery data within a CAM clinical trial, as well as the challenges of blinding.

IMPLICATIONS FOR PRACTICE

Monitoring intervention fidelity by using the key areas identified by the TFW ensures that findings from a clinical trial are meaningful and have the potential to be translated to clinical practice.

A Large-scale Dissemination and Implementation Model for Evidence-based Treatment and Continuing Care

Multiple evidence-based treatments for adolescents with substance use disorders are available; however, the diffusion of these treatments in practice remains minimal. A dissemination and implementation model incorporating research-based training components for simultaneous implementation across 33 dispersed sites and over 200 clinical staff is described. Key elements for the diffusion of the Adolescent Community Reinforcement Approach and Assertive Continuing Care were: (a) three years of funding to support local implementation; (b) comprehensive training, including a 3.5 day workshop, bi-weekly coaching calls, and ongoing performance feedback facilitated by a web tool; (c) a clinician certification process; (d) a supervisor certification process to promote long-term sustainability; and (e) random fidelity reviews after certification. Process data are summarized for 167 clinicians and 64 supervisors.

The impact of pay-for-performance on therapists' intentions to deliver high-quality treatment

This article examined the extent to which assignment to a pay-for-performance (P4P) experimental condition impacted therapists' intentions to deliver high-quality treatment and the extent to which therapists' intentions could be explained by the theory of planned behavior. Data were collected from 95 therapists who agreed to participate in a P4P experiment related to their implementation of an evidence-based treatment (EBT) for adolescents with substance use problems. Relative to those in the control condition, therapists in the P4P condition reported significantly greater intentions to achieve monthly competence (B = 1.41, p < .001) and deliver a targeted threshold level of treatment to clients (B = 1.31, p < .001). In addition, therapists' intentions could be partially explained by the theory of planned behavior. Meta-analyses have found intentions to be one of the best predictors of behavior; thus, these findings provide initial support for using P4P approaches as a method of increasing the quality of substance use treatment.

Treatment process, alliance and outcome in brief versus extended treatments for marijuana dependence

AIMS

The Marijuana Treatment Project, a large multi-site randomized clinical trial, compared a delayed treatment control condition with a brief (two-session) and extended (nine-session) multi-component treatment among 450 marijuana-dependent participants. In this report we present treatment process data, including the fidelity of treatment delivery in the three community-based treatment settings as well as the relationships between treatment process and outcome.

DESIGN

Independent evaluations of clinician adherence and competence ratings were made based on 633 videotaped sessions from 163 participants. Relationships between clinician adherence and competence, ratings of the working alliance and marijuana treatment outcomes were evaluated.

FINDINGS

Protocol treatments were implemented with strong fidelity to manual specifications and with few significant differences in adherence and competence ratings across sites. In the brief two-session treatment condition, only the working alliance was associated significantly with frequency of marijuana use, but in the extended treatment therapist ratings of working alliance predicted outcomes, as did the interaction of alliance and curvilinear adherence.

CONCLUSIONS

Behavioral treatments for marijuana use were delivered in community settings with good fidelity. Participant and therapist working alliance scores were associated significantly with improved marijuana use outcomes in a brief behavioral treatment for adults with marijuana dependence. In extended treatment the therapist ratings of working alliance were associated with more positive outcome. However, in that treatment there was also a significant interaction between alliance and curvilinear adherence.

Psychometric properties of the contingency management competence scale

Contingency management (CM) is an evidence-based treatment, and clinicians are beginning to implement this intervention in practice. However, little research exists on methods for assuring appropriate implementation of CM. This study describes the development and psychometric properties of the 12-item CM Competence Scale (CMCS). Thirty-five therapists from nine community-based clinics participated; following a training period, a randomized trial evaluated the efficacy of CM in cocaine abusing patients. Analyses of the CMCS are based on ratings from 1613 audiotapes of therapist interactions with 78 patients enrolled in the training phase and 103 patients in the randomized phase. Inter-rater reliability from 11 raters and internal consistency of items on the CMCS was good to excellent. Items loaded onto two factors: one contained items specific to discussions of the outcomes of urine testing and reinforcement, and the other contained general items related to use of praise, communication of confidence, empathy, skillfulness, and maintaining session structure, as well as discussions of self-reports of drug use when they occurred. During the training phase in CM delivery, scores on the CMCS rose significantly between earlier and later training sessions, and during the randomized phase, CM sessions were rated more highly than non-CM sessions. Scores on the subscale assessing general items were significantly correlated with indices of the therapeutic alliance and predictive of durations of cocaine abstinence achieved. These data suggest that the CMCS is reliable and valid in assessing delivery of CM and that competence in CM delivery is associated with improved patient outcomes.

Randomized controlled pilot study of cognitive-behavioral therapy in a sample of incarcerated women with substance use disorder and PTSD

This randomized controlled pilot study compared a cognitive-behavioral therapy (Seeking Safety; SS) plus treatment-as-usual (TAU) to TAU-alone in 49 incarcerated women with substance use disorder (SUD) and posttraumatic stress disorder (PTSD; full or subthreshold). Seeking Safety consisted of a voluntary group treatment during incarceration and individual treatment after prison release. TAU was required in the prison and comprised 180 to 240 hours of individual and group treatment over 6 to 8 weeks. Assessments occurred at intake, 12 weeks after intake, and 3 and 6 months after release from prison. There were no significant differences between conditions on all key domains (PTSD, SUD, psychopathology, and legal problems); but both conditions showed significant improvements from intake to later time points on all of these outcomes across time. Secondary analyses at follow-up found trends for SS participants improving on clinician-rated PTSD symptoms and TAU participants worsening on self-reported PTSD symptoms. Also, SS demonstrated continued improvement on psychopathology at 3 and 6 months, whereas TAU did not. However, alcohol use improved more for TAU during follow-up. Satisfaction with SS was high, and a greater number of SS sessions was associated with greater improvement on PTSD and drug use. Six months after release from prison, 53% of the women in both conditions reported a remission in PTSD. Study limitations include lack of assessment of SS outcomes at end of group treatment; lack of blind assessment; omission of the SS case management component; and possible contamination between the two conditions. The complex needs of this population are discussed.

A treatment for substance abusing pregnant women

We describe the adaptation of a manualized behavioral treatment for substance using pregnant women that includes components of motivational interviewing and cognitive therapy. In a pilot study conducted in 2006-2007, five non-behavioral health clinicians were trained to provide the treatment to 14 women. Therapy was administered concurrent with routine prenatal care at inner-city maternal health clinics in New Haven and Bridgeport, Connecticut, small urban cities in the USA. Substance use was monitored by self report, and urine and breath tests. Treatment fidelity was assessed using the Yale Adherence and Competence System. Behavioral treatment delivery in this setting is feasible and is being evaluated in a randomized, controlled, clinical trial.

Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems

This study examined the impact of treatment adherence and therapist competence on treatment outcome in a controlled trial of individual cognitive-behavioral therapy (CBT) and multidimensional family therapy (MDFT) for adolescent substance use and related behavior problems. Participants included 136 adolescents (62 CBT, 74 MDFT) assessed at intake, discharge, and 6-month follow-up. Observational ratings of adherence and competence were collected on early and later phases of treatment (192 CBT sessions, 245 MDFT sessions) by using a contextual measure of treatment fidelity. Adherence and competence effects were tested after controlling for therapeutic alliance. In CBT only, stronger adherence predicted greater declines in drug use (linear effect). In CBT and MDFT, (a) stronger adherence predicted greater reductions in externalizing behaviors (linear effect) and (b) intermediate levels of adherence predicted the largest declines in internalizing behaviors, with high and low adherence predicting smaller improvements (curvilinear effect). Therapist competence did not predict outcome and did not moderate adherence-outcome relations; however, competence findings are tentative due to relatively low interrater reliability for the competence ratings. Clinical and research implications for attending to both linear and curvilinear adherence effects in manualized treatments for behavior disorders are discussed.

Combining cognitive behavioral therapy with contingency management for smoking cessation in adolescent smokers: a preliminary comparison of two different CBT formats

This pilot study evaluated the optimal format of cognitive behavioral therapy (CBT) to combine with contingency management (CM) in a four-week, high school-based smoking cessation program. Thirty-four adolescent smokers received a standard weekly version of CBT or a frequent brief behavioral intervention. Results indicate a trend toward a higher seven-day point prevalence end-of-treatment abstinence rate and percent days abstinent during treatment in the CBT condition. In addition, significantly more participants in the CBT group completed treatment. These preliminary results suggest that when combined with CM, the standard weekly format of CBT is more acceptable to adolescent smokers.

Site matters: multisite randomized trial of motivational enhancement therapy in community drug abuse clinics

The effectiveness of motivational enhancement therapy (MET) in comparison with counseling as usual (CAU) for increasing retention and reducing substance use was evaluated in a multisite randomized clinical trial. Participants were 461 outpatients treated by 31 therapists within 1 of 5 outpatient substance abuse programs. There were no retention differences between the 2 brief intervention conditions. Although both 3-session interventions resulted in reductions in substance use during the 4-week therapy phase, MET resulted in sustained reductions during the subsequent 12 weeks whereas CAU was associated with significant increases in substance use over this follow-up period. This finding was complicated by program site main effects and higher level interactions. MET resulted in more sustained substance use reductions than CAU among primary alcohol users, but no difference was found for primary drug users. An independent evaluation of session audiotapes indicated that MET and CAU were highly and comparably discriminable across sites.

Enhancing the validity and utility of randomized clinical trials in addictions treatment research: I. Treatment implementation and research design

PURPOSE

This paper is the first in a series that examines methods for improving the validity and utility of randomized clinical trials (RCTs) in addictions treatment research. The specific foci of this article are treatment implementation and research design.

SCOPE

We begin by considering the conditions under which the RCT provides an appropriate design choice. Sections that follow discuss methodological issues with respect to RCT structure and collaborative arrangements; treatment specification, delivery and cost; experimental design; and randomization/blinding procedures. We emphasize the importance of advance planning; treatment integrity and discriminability; treatment standardization; staff training and supervision; client compliance; maintenance of between-group equivalence across study conditions; and inclusion of appropriate comparison groups in study designs.

CONCLUSIONS

Investigators are encouraged to maximize the internal validity of RCTs, but also to consider methods for enhancing external validity. The utility of addictions RCTs for advancing theory and improving clinical practice can be enhanced by investigating underlying mechanisms of action.

A vision of the next generation of behavioral therapies research in the addictions

Whither, or wither, empirically supported therapies? Increasingly rigorous research in behavioral therapies has yielded a large number of effective therapies, but comparatively little work, demonstrating that integrating empirically supported therapies (ESTs) into standard practice results in meaningful improvements in patient outcomes. Methodology and strategies for evaluating ESTs and their effectiveness in clinical practice is a fairly recent innovation, and a host of unanswered questions remain regarding issues such as selection among different ESTs and what type of ESTs should be emphasized in dissemination efforts, what type of clinicians should be trained in what type of ESTs, the most effective training strategies for various types of clinicians, the need for ongoing supervision to maintain minimum levels of treatment fidelity and skill. In this review, we call for broader use of new research strategies and methods relevant to dissemination of ESTs; these may include adaptive designs, identification of mechanisms of action to foster greater emphasis on effective change principles, training and adoption trials, as well as novel implementation strategies including computer-assisted therapy and computer-assisted training.

Training and fidelity monitoring of behavioral interventions in multi-site addictions research

AIMS

Methods for the training and fidelity monitoring of behavioral interventions in multi-site addictions research are reviewed, including five published studies and seven ongoing studies sponsored by the National Institute on Drug Abuse-funded Clinical Trials Network.

METHODS

Methods are categorized and reviewed consistent with a technology model of treatment delivery. Topics include: therapist selection, training, certification, and supervision; selection, training, and certification of supervisors; scales and processes used for monitoring of the quality of treatment; and processes followed to provide new training for replacement staff once trials have begun.

RESULTS

The review reveals both a wide array of procedures and emerging standards for multi-site trials. Methodological weakness was observed with respect to limited empirical support for many adherence scales, little or no evaluation of supervisory processes, and no evaluation of re-training practices.

CONCLUSIONS

Methods used in multi-site trials are important not only to ensure validity of those trials, but also to inform the wider dissemination of empirically based treatment into community agencies. Studies examining noted weaknesses are needed. Training and fidelity models that delegate responsibility to participating sites appear most relevant for establishing best practices for dissemination of behavioral interventions. The effectiveness of these distributed training and supervision models should be subjected to empirical study at a level of rigor comparable to the evaluation of their corresponding treatments.

Intervention research: establishing fidelity of the independent variable in nursing clinical trials

BACKGROUND

Internal validity of a randomized clinical trial of a nursing intervention is dependent on intervention fidelity. Although several methods have been developed, evaluating audio or audiovisual tapes for prescribed and proscribed interventionist behaviors is considered the gold standard test of treatment fidelity. This approach requires development of a psychometrically sound instrument to meaningfully categorize and quantify interventionist behaviors.

OBJECTIVE

To outline critical steps necessary to develop a treatment fidelity instrument.

METHODS

A comprehensive literature review was conducted to determine procedures used by other researchers. The literature review produced five quantitative studies of treatment fidelity, all in the field of psychotherapy, and two replication studies. A synthesis of methodologies across studies combined with researchers' experiences resulted in identification of the steps necessary to develop a treatment fidelity measure.

RESULTS

Seven sequential steps were identified as essential to the development of a valid and reliable measure of treatment fidelity. These steps include (a) identification of the essential elements of the experimental and control treatment modalities; (b) construction of scale items; (c) development of item scaling; (d) identification of the units for coding; (e) item testing and revision; (f) specification of rater qualifications and development of rater training program; and (g) development and completion of pilot testing to test psychometric properties. Development of the Possibilities Project Psychotherapy Coding Questionnaire is described as an illustration of the seven-step process.

DISCUSSION

The results show the essential steps that are unique to the development of treatment fidelity measures and show the feasibility of using these steps to construct a psychometrically sound treatment-specific fidelity measure.

One small step for manuals: Computer-assisted training in twelve-step facilitation

OBJECTIVE

The burgeoning number of empirically validated therapies has not been met with systematic evaluation of practical, inexpensive means of teaching large numbers of clinicians to use these treatments effectively. An interactive, computer-assisted training program that sought to impart skills associated with the Project MATCH (Matching Alcoholism Treatments to Client Heterogeneity) Twelve-Step Facilitation (TSF) manual was developed to address this need.

METHOD

Twenty-five community-based substance use-treatment clinicians were randomized to one of two training conditions: (1) access to the computer- assisted training program plus the TSF manual or (2) access to the manual only. The primary outcome measure was change from preto posttraining in the clinicians' ability to demonstrate key TSF skills.

RESULTS

The data suggested that the clinicians' ability to implement TSF, as assessed by independent ratings of adherence and skill for the key TSF interventions, was significantly higher after training for those who had access to the computerized training condition than those who were assigned to the manual-only condition. Those assigned to the computer-assisted training condition also demonstrated greater gains in a knowledge test assessing familiarity with concepts presented in the TSF manual.

CONCLUSIONS

Computer-based training may be a feasible and effective means of training larger numbers of clinicians in empirically supported, manual-guided therapies.

A randomized controlled pilot study of motivational interviewing for patients with psychotic and drug use disorders

AIMS

This pilot study examined the efficacy of a two-session motivational interview adapted for dually diagnosed psychotic and drug-related disordered patients (DDMI) in comparison to a two-session standard psychiatric interview (SI).

DESIGN

The study used a randomized controlled trial design. Participants received either DDMI or SI and were assessed at baseline, 4-, 8- and 12-week follow-up points. The principal analysis for examination of treatment effects across time was a random effects regression model.

SETTING

Both DDMI and SI interviews served as pre-admission intake interventions to an ambulatory specialty dual diagnosis intensive out-patient and partial hospital program.

PARTICIPANTS

Forty-four treatment-seeking participants (DDMI = 24; SI = 20) who had co-occurring psychotic and drug-related disorders were assigned randomly to the treatment conditions. Measurements Primary outcomes were days of primary drug use, secondary drug use, alcohol use and psychotropic medication adherence, proportion of participants admitted into the program and days of attendance.

FINDINGS AND CONCLUSIONS

DDMI and SI resulted in improved treatment outcomes, but there were no main effects for the sample as a whole. Separate examination of primary cocaine and primary marijuana using subsamples, however, suggested that DDMI resulted in significantly better primary drug treatment outcomes for the cocaine-using group, whereas SI resulted in significantly better primary drug treatment outcomes for the marijuana-using group. These findings indicate that MI may not work equally well for all types of psychotic disordered dually diagnosed patients and that alternative approaches may be as effective in fostering improved substance use treatment outcomes for subgroups of these individuals.

Intensive referral to 12-Step self-help groups and 6-month substance use disorder outcomes

AIMS

This study implemented and evaluated procedures to help clinicians make effective referrals to 12-Step self-help groups.

DESIGN

Randomized controlled trial. Setting Out-patient substance use disorder treatment. Participants Individuals with substance use disorders (SUDs) entering a new treatment episode (n = 345) who were assigned randomly to a standard referral- or an intensive referral-to-self-help condition.

MEASUREMENTS

Self-reports of 12-Step group attendance and involvement and substance use at baseline and a 6-month follow-up.

INTERVENTION

The intensive referral intervention focused on encouraging patients to attend 12-Step meetings by connecting them to 12-Step volunteers.

FINDINGS

Among patients with relatively less previous 12-Step meeting attendance, intensive referral was associated with more meeting attendance during follow-up than was standard referral. Among all patients, compared with those who received standard referral, those who received intensive referral were more likely to be involved with 12-Step groups during the 6-month follow-up (i.e. had provided service, had a spiritual awakening and currently had a sponsor). Intensive referral patients also had better alcohol and drug use outcomes at 6 months. Twelve-Step involvement mediated part of the association between referral condition and alcohol outcomes.

CONCLUSIONS

The brief intensive referral intervention was associated with improved 12-Step group involvement and substance use outcomes even among patients with considerable previous 12-Step group exposure and formal treatment. Future 12-Step intensive referral procedures should focus on encouraging 12-Step group involvement in addition to attendance to benefit patients most effectively.

Effectiveness of workshop training for psychosocial addiction treatments: A systematic review

Workshop training for psychosocial substance abuse treatment has been an important part of the transfer of evidence-based approaches into larger practice. Although they are widely used, training methods such as self-study reading, internet-based courses, and educational workshops have not traditionally been the focus of empirical investigations. Based on electronic and manual searches of the literature, we summarize 17 evaluations of workshop training that describe the training program and the educational outcomes. In general, training tends to improve attendees' knowledge, attitudes, and confidence in working with clients who have substance abuse problems. Some skill improvements, when measured, are usually seen immediately after training but are less often maintained over a longer time. Extended contact, through follow-up consultation, supervision, or feedback, appears to be necessary for the long-term adoption of skills. There are also a number of institutional factors that may influence the extent to which providers adopt new practices. Given the popularity of this training format, the role of workshop training needs to be a focus of future evaluative research.

When treatment meets research: clinical perspectives from the CSAT Methamphetamine Treatment Project

Integrating research-based treatments into clinical settings has become a priority in the substance abuse treatment field. This article examines the introduction of research, via manualized treatment (i.e., the Matrix Model), into community treatment settings that participated in the Center for Substance Abuse Treatment Methamphetamine Treatment Project, a multi-site randomized controlled trial (RCT) that provided free treatment to 1016 methamphetamine-dependent individuals. With both empirical (qualitative) and anecdotal data from those involved clinically in the project, the article utilizes the framework of practitioner concerns set forth by Addis, Wade, and Hatgis (1999) to assess the issues realized during the implementation of this manualized treatment. Despite fairly smooth implementation of the model, the authors conclude that introducing manualized treatment in the context of an RCT may not be the best way to bring research-based treatment into the practice world.

SETTING STANDARDS FOR TRAINING AND COMPETENCE: THE UK ALCOHOL TREATMENT TRIAL

AIMS

To examine factors that influence the recruitment and training of therapists and their achievement of competence to practise two psychological therapies for alcohol dependence, and the resources required to deliver this.

METHODS

The protocol for the UK Alcohol Treatment Trial required trial therapists to be competent in one of the two trial treatments: Social Behaviour and Network Therapy (SBNT) or Motivational Enhancement Therapy (MET). Therapists were randomised to practise one or other type of therapy. To ensure standardisation and consistent delivery of treatment in the trial, the trial training centre trained and supervised all therapists.

RESULTS

Of 76 therapists recruited and randomised, 72 commenced training and 52 achieved competence to practise in the trial. Length of prior experience did not predict completion of training. However, therapists with a university higher qualification, and medical practitioners compared to other professionals, were more likely to complete. The average number of clients needed to be treated before the trainee achieved competence was greater for MET than SBNT, and there was a longer duration of training for MET.

CONCLUSIONS

Training therapists of differing professional backgrounds, randomised to provide a specific therapy type, is feasible. Supervision after initial training is important, and adds to the training costs.

Outcomes with the ARISE approach to engaging reluctant drug- and alcohol-dependent individuals in treatment

Our goal was to explore, through a Stage I NIH clinical study, the effectiveness of a manual-driven, timely response method for helping the "concerned other" get resistant substance abusers into treatment/self-help with minimum professional time/effort. A manual-driven protocol, "A Relational Sequence for Engagement (ARISE)," was applied with 110 consecutive, initial calls/contacts from concerned others; no cases excluded for research, refusal, or other reasons. The research was conducted at two upstate New York outpatient drug/alcohol clinics. Participants were concerned others who called regarding a cocaine, alcohol, or "other drug" abuser (N = 110); participating family/friends: 11 ARISE clinicians; and 110 substance abusers. ARISE is a graduated continuum starting with the least demanding option/stage, increasing effort as needed to engage substance abusers in treatment/self-help. Stage I: Coaching the concerned other to arrange a meeting of significant others, inviting the substance abuser; Stage II: 1 to 5 additional meetings (median = 2); Stage III: A modified Johnson "Intervention." Primary outcome variables were substance abuser engagement (or not) in treatment/self-help; days between first call and engagement; clinician time/effort. Predictors were concerned other, substance abuser, and clinician demographics; number of participants per case; and Collateral Addiction Severity Index. ARISE resulted in an 83% success rate (55% at Stage I). Median days to engagement was 7 (IQR = 2 to 14). Average total time (telephone, sessions) per case was 1.5 hours. Treatment/self-help chosen was 95% treatment and 5% self-help. Number of family/ friends involved correlated 0.69 with a success/efficiency index. Conclusions. A call from a family member or concerned other for help in getting a loved one into treatment is a rich opportunity for treatment professionals and agencies to engage substance abusers in treatment. These initial calls are similar to referral calls from EAPs or probation officers looking to get an individual started in treatment. ARISE provides an effective, swift, and cost-efficient option for engaging substance abusers in treatment or self-help. The more significant others involved, the greater the success of treatment engagement.

A randomized trial of methods to help clinicians learn motivational interviewing

The Evaluating Methods for Motivational Enhancement Education trial evaluated methods for learning motivational interviewing (MI). Licensed substance abuse professionals (N = 140) were randomized to 5 training conditions: (a) clinical workshop only; (b) workshop plus practice feedback; (c) workshop plus individual coaching sessions; (d) workshop, feedback, and coaching; or (e) a waiting list control group of self-guided training. Audiotaped practice samples were analyzed at baseline, posttraining, and 4, 8, and 12 months later. Relative to controls, the 4 trained groups showed larger gains in proficiency. Coaching and/or feedback also increased posttraining proficiency. After delayed training, the waiting list group showed modest gains in proficiency. Posttraining proficiency was generally well maintained throughout follow-up. Clinician self-reports of MI skillfulness were unrelated to proficiency levels in observed practice.

Comparing outcomes of best-practice and research-based outpatient treatment protocols for adolescents

Comparisons of well-developed practice-based interventions with research-based interventions are rare. This quasi-experimental study compares the outcomes of 274 adolescents (75% male; 63% weekly+ users; 54% dependent; 27% prior treatment; 73% with co-occurring problems) who received Chestnut Health System's best-practice Outpatient Treatment (CHS) or Cannabis Youth Treatment (CYT) research-based interventions. Ninety-five percent of participants completed follow-up interviews at three, six, nine, and 12 months after their intake GAIN interview. Initially, the CYT cohort scores indicated greater severity on several substance-related measures, while the CHS cohort scored higher on prior mental health treatment, victimization, and illegal activities measures. Adolescents in the CHS cohort were more likely to have longer lengths of stay and receive over three times as many hours of treatment. Mixed effects models revealed that CHS participants were significantly more likely to report a decrease in recovery environment risk, an increase in self-help attendance after treatment, and greater decreases in emotional problems, while CYT participants were significantly more likely to report decreases in their substance use. The results suggest that neither the best-practice nor the research-based interventions were clearly superior and call for a more rigorous randomized field experiment to better understand the differences in effectiveness between interventions.

Characteristics, beliefs, and practices of community clinicians trained to provide manual-guided therapy for substance abusers

The successful dissemination of empirically supported addiction therapies to community providers requires an appreciation of the characteristics of those practitioners who might be willing participants in this process of technology transfer. Clinicians (N = 66) from 11 community treatment programs associated with six research-clinic partnerships of the National Drug Abuse Clinical Trials Network volunteered to be trained in Motivational Interviewing or Motivational Enhancement Therapy (MET/MI) and were assessed prior to training. The sample of clinicians was heterogeneous in education and credentials, had a high level of counseling experience, reported using a wide range of counseling techniques and orientations, but had limited prior exposure to MET/MI or to the use of treatment manuals of empirically supported therapies. In general, many of the clinicians reported beliefs and techniques that were consistent with their stated theoretical orientation and recovery status. Relatively few participants reported relying on one dominant orientation or set of techniques.

MET meets the real world: design issues and clinical strategies in the Clinical Trials Network

The Clinical Trials Network (CTN) represents a major initiative intended to bridge the gap between research and practice in substance abuse treatment by implementing a range of studies evaluating behavioral, pharmacologic, and combined treatments in community-based drug abuse treatment programs across the country. This article describes the development of CTN protocols evaluating the effectiveness of Motivational Enhancement Therapy and Motivational Interviewing. Design, training, and implementation challenges associated with conducting a clinical trial of brief behavioral treatments in community programs are discussed. Issues requiring attention included the diversity in treatments offered across sites, heterogeneity in the study sample, and training of clinicians drawn from the staff of community programs to deliver the study treatments.

Stages in the implementation of innovative clinical programs in complex organizations

Organizational processes can have an important impact on the introduction of innovative treatments into practice. Conceptual frameworks from organization theory and experiences implementing several hundred specialized mental health programs in the Department of Veterans Affairs (VA) over the past 15 years are used to illustrate stages and processes in the implementation of new treatment models. Four phases in the implementation of new treatments in complex organizational settings are described: a) the decision to implement, b) initial implementation, c) sustained implementation, and d) termination or transformation. Key strategies for moving research into practice include constructing decision-making coalitions, linking new initiatives to legitimate goals and values, quantitative monitoring of implementation and performance, and the development of self-sustaining communities of practice as well as learning organizations. Effective dissemination of new treatment methods requires different organizational strategies at different phases of implementation.

Clinicians' impact on the quality of substance use disorder treatment

Clinicians' impact on substance use disorder treatment has been much less studied than therapy and patient variables. Yet, in this selective review of literature, a growing body of empirical work on clinicians' impact highlights several key issues that have relevance both to clinical practice and future research. These issues include clinicians' effect on treatment retention and outcome, professional characteristics, recovery status, adherence to protocols, counter-transference, alliance, personality, beliefs about treatment, and professional practice issues. Specific recommendations are offered to help improve the quality of care clinicians provide. In particular, it is suggested that greater accountability for clinicians' performance be balanced with increased support for their very difficult role. Methodological issues in studying clinicians are also addressed.

A general system for evaluating therapist adherence and competence in psychotherapy research in the addictions

The Yale Adherence and Competence Scale (YACS) is a general system for rating therapist adherence and competence in delivering behavioral treatments for substance use disorders. The system includes three scales measuring 'general' aspects of drug abuse treatment (assessment, general support, goals of treatment), as well as three scales measuring critical elements of three treatments that are frequently implemented as control or comparison treatments in clinical research in the addictions (clinical management (CM), twelve step facilitation (TSF), and cognitive behavioral therapy (CBT)). Validation of the YACS using data from a randomized clinical trial indicated that the scales have excellent reliability, factor structure, concurrent and discriminant validity. Correlations between adherence and competence scores within scales were in the moderate range, indicating independence (and thus nonredundancy) of these dimensions. Strategies for using the YACS in both psychotherapy and pharmacotherapy research in the addictions are described.

Matching alcoholism treatments to client heterogeneity: Project MATCH three-year drinking outcomes

This study reports 3-year outcomes for clients who had been treated in the five outpatient sites of Project MATCH, a multisite clinical trial designed to test a priori client treatment matching hypotheses. The main purpose of this study was to characterize the status of the matching hypotheses at the 3-year follow-up. This entailed investigating which matching findings were sustained or even strengthened across the 3-year study period, and whether any hypotheses that were not supported earlier eventually emerged at 3 years, or conversely, whether matching findings discerned earlier dissipated at this later time. This research also examines the prognostic effects of the client matching attributes, characterizes the overall outcomes at 37 to 39 months, and explores differential effects of the three treatments at extended follow-up. With regard to the matching effects, client anger demonstrated the most consistent interaction in the trial, with significant matching effects evident at both the 1-year and 3-year follow-ups. As predicted, clients high in anger fared better in Motivational Enhancement Therapy (MET) than in the other two MATCH treatments: Cognitive-Behavioral Therapy (CBT) and Twelve-Step Facilitation (TSF). Among subjects in the highest third of the anger variable, clients treated in MET had on average 76.4% abstinent days, whereas their counterparts in the other two treatments (CBT and TSF) had on average 66% abstinent days. Conversely, clients low in anger performed better after treatment in CBT and TSF than in MET. Significant matching effects for the support for drinking variable emerged in the 3-year outcome analysis, such that clients whose social networks were more supportive of drinking derived greater benefit from TSF treatment than from MET. Among subjects in the highest third of the support for drinking variable, TSF participants were abstinent 16.1% more days than MET participants. At the lower end of this variable, difference in percent days abstinent between MET and TSF was 3%, with MET clients having more abstinent days. A significant matching effect for psychiatric severity that appeared in the first year posttreatment was not observed after 3 years. Of the 21 client attributes used in testing the matching hypotheses, 11 had prognostic value at 3 years. Among these, readiness-to-change and self-efficacy emerged as the strongest predictors of long-term drinking outcome. With regard to the overall outcomes, the reductions in drinking that were observed in the first year after treatment were sustained over the 3-year follow-up period: almost 30% of the subjects were totally abstinent in months 37 to 39, whereas those who did report drinking nevertheless remained abstinent an average of two-thirds of the time. As in the 1-year follow-up, there were few differences among the three treatments, although TSF continued to show a possible slight advantage.

Treatment approaches for pathological gamblers

Outcome literature on the treatment of pathological gambling is reviewed, encompassing psychodynamic, behavioral, cognitive, cognitive-behavioral, multimodal, pharmacotherapeutic, and 12-step approaches. No properly controlled research has been conducted with psychodynamic or 12-step methods, and pharmacotherapies require replications with larger samples to determine their efficacy. Multimodal approaches have been tested most often in inpatient settings, and given the range of methods combined it is difficult to infer specific efficacy for treatment components. The largest volume of outcome research has focused on behavioral, cognitive, and combined cognitive-behavioral treatment methods, and findings from controlled and uncontrolled trials provide support for efficacy of these approaches. As a whole, the literature indicates that pathological gambling can be treated with highly successful outcomes. Needs for further research are considered.

Treatment matching. Theoretic basis and practical implications

An extensive knowledge base supports the development of treatment matching methods for alcohol and other drug abuse. Many matching variables have been identified that relate to specific modalities of treatment and to specific levels of care. Physicians and other providers can use many putative matching variables to address patients' substance use problems on a highly individualized basis. These variables include demographic factors (age, gender, culture), typology and severity (age of onset; severity of intoxication; withdrawal; quantity, recency, frequency of substance use), intrapersonal characteristics (psychiatric diagnosis, cognitive function, self-efficacy, stage of change), and interpersonal function (social stability). The evolution of formal criteria for patient placement such as the ASAM criteria is a beneficial, adaptive process that is underway in numerous states, managed care entities, professional provider societies, and provider groups. Currently, matching approaches rely more heavily on consensus recommendations than on empiric matching data. The technology for conducting psychosocial treatment matching studies is rapidly increasing in sophistication. Although predictive validity has not yet been fully demonstrated on a large, multisite basis, the national research portfolio on treatment matching is expanding in size and complexity. This is an essential public health need, given dramatic cost pressures, if addiction services are to continue to grow in quality and availability.

Alcoholism treatment outcome studies, 1980-1992: methodological characteristics and quality

We examine the methodological characteristics and provision of study information in 339 alcoholism treatment outcome studies reported between 1980 and 1992. We consider factors in four methodological domains: sampling and description of patients, specification of treatments, outcome variable assessment and follow-up, and treatment effect estimates; we also consider the methodological quality of the studies. Although methodological quality has improved over time, there remains room for improvement. Of special concern is the low statistical power of many studies. Multiple treatment group studies had an average .54 probability of detecting a treatment effect of medium size.