Using a NIATx based local learning collaborative for performance improvement

Adapting the stages of implementation completion to an evidence-based implementation strategy: The development of the NIATx stages of implementation completion

Background

Dissemination and implementation frameworks provide the scaffolding to explore the effectiveness of evidence-based practices (EBPs) targeting process of care and organizational outcomes. Few instruments, like the stages of implementation completion (SIC) examine implementation fidelity to EBP adoption and how organizations differ in their approach to implementation. Instruments to measure organizational competency in the utilization of implementation strategies are lacking.

Method

An iterative process was utilized to adapt the SIC to the NIATx implementation strategies. The new instrument, NIATx-SIC, was applied in a randomized controlled trial involving 53 addiction treatment agencies in Washington state to improve agency co-occurring capacity. NIATx-SIC data were reported by state staff and external facilitators and through participating agency documentation. Proportion and duration scores for each stage and phase of the NIATx-SIC were calculated for each agency. Competency was assessed using the NIATx fidelity tool. Comparisons of proportion, duration, and NIATx activities completed were determined using independent sample t-tests by agency competency level.

Results

The NIATx-SIC distinguished between agencies achieving competency (n  =  23) and those not achieving competency (n  =  26). Agencies achieving competency completed a greater proportion of implementation phase activities and had a significantly longer Stage 7 duration. These agencies participated in significantly more individual and group coaching calls, attended more in-person meetings, implemented more change projects, and spent approximately 64 more days, on average, engaging in all NIATx activities.

Conclusions

Organizational participation in dissemination and implementation research requires a significant investment of staff resources. The inability of an organization to achieve competency when utilizing a set of implementation strategies waste an opportunity to institutionalize knowledge of how to apply implementation strategies to future change efforts. The NIATx-SIC provides evidence that competency is not an attribute of the organization but rather a result of the application of the NIATx implementation strategies to improve agency co-occurring capacity.

Trial Registration

ClinicalTrials.gov, NCT03007940. Registered January 2, 2017, https://clinicaltrials.gov/ct2/show/NCT03007940.

The development and initial validation of the Russian version of the BASIS-24

BACKGROUND

Efficient and linguistically appropriate instruments are needed to assess response to addiction treatment, including severity of addiction/mental health status. This is critical for Russian-speaking persons in Eastern Europe and Central Asia (EECA) where Medications for Opioid Use Disorder (MOUD) remain underscaled to address expanding and intertwined opioid, HIV, HCV and tuberculosis epidemics. We developed and conducted a pilot validation of a Russian version of the 24-item Behavior and Symptom Identification Scale (BASIS-24), an addiction/mental health severity instrument with six subscales, previously validated in English.

METHODS

Using the Mapi approach, we reviewed, translated, and back-translated the content to Russian, pilot-tested the Russian-version (BASIS-24-R) among new MOUD patients in Ukraine (N = 283). For a subset of patients (n = 44), test-rest was performed 48 h after admission to reassess reliability of BASIS-24-R. Exploratory principal component analysis (PCA) assessed underlying structure of BASIS-24-R.

RESULTS

Cronbach alpha coefficients for overall BASIS-24-R and 5 subscales exceeded 0.65; coefficient for Relationship subscale was 0.42. The Pearson correlation coefficients for overall score and all subscales on the BASIS-24-R exceeded 0.8. Each item loaded onto factors that corresponded with English BASIS-24 subscales ≥ 0.4 in PCA.

CONCLUSION

Initial version of BASIS-24-R appears statistically valid in Russian. Use of the BASIS-24-R has potential to guide MOUD treatment delivery in the EECA region and help to align addiction treatment with HIV prevention goals in a region where HIV is concentrated in people who inject opioids and where healthcare professionals have not traditionally perceived MOUD as effective treatment, particularly for those with mental health co-morbidities.

Stepped implementation-to-target: a study protocol of an adaptive trial to expand access to addiction medications

BACKGROUND

In response to the US opioid epidemic, significant national campaigns have been launched to expand access to `opioid use disorder (MOUD). While adoption has increased in general medical care settings, specialty addiction programs have lagged in both reach and adoption. Elevating the quality of implementation strategy, research requires more precise methods in tailoring strategies rather than a one-size-fits-all-approach, documenting participant engagement and fidelity to the delivery of the strategy, and conducting an economic analysis to inform decision making and policy. Research has yet to incorporate all three of these recommendations to address the challenges of implementing and sustaining MOUD in specialty addiction programs.

METHODS

This project seeks to recruit 72 specialty addiction programs in partnership with the Washington State Health Care Authority and employs a measurement-based stepped implementation-to-target approach within an adaptive trial design. Programs will be exposed to a sequence of implementation strategies of increasing intensity and cost: (1) enhanced monitoring and feedback (EMF), (2) 2-day workshop, and then, if outcome targets are not achieved, randomization to either internal facilitation or external facilitation. The study has three aims: (1) evaluate the sequential impact of implementation strategies on target outcomes, (2) examine contextual moderators and mediators of outcomes in response to the strategies, and (3) document and model costs per implementation strategy. Target outcomes are organized by the RE-AIM framework and the Addiction Care Cascade.

DISCUSSION

This implementation project includes elements of a sequential multiple assignment randomized trial (SMART) design and a criterion-based design. An innovative and efficient approach, participating programs only receive the implementation strategies they need to achieve target outcomes. Findings have the potential to inform implementation research and provide key decision-makers with evidence on how to address the opioid epidemic at a systems level.

TRIAL REGISTRATION

This trial was registered at ClinicalTrials.gov (NCT05343793) on April 25, 2022.

Effectiveness of value-based purchasing for substance use treatment engagement and retention

INTRODUCTION

Many people drop out of substance use disorder (SUD) treatment within the first few sessions, which suggests the need for innovative strategies to address this. We examined the effectiveness of incentive-based contracting for Maine's publicly funded outpatient (OP) and intensive outpatient (IOP) SUD treatment, to determine its potential for improving treatment engagement and retention.

METHODS

Maine's incentive-based contract with federally block grant-funded OP and IOP treatment agencies created a natural experiment, in which we could compare treatment engagement and retention with a group of state-licensed treatment agencies that were not part of the incentive-based contract. We used administrative data for OP (N = 18,375) and IOP (N = 5986) SUD treatment admissions from FY2005-FY2011 to capture trends prior to and after the FY2008 contract implementation date. We performed multivariable difference-in-difference logistic regression models following propensity score matching of clients.

RESULTS

Two-thirds (66%) of OP admissions engaged in treatment, defined as 4+ treatment sessions, and 85% of IOP admissions satisfied the similar criteria of 4+ treatment days. About 40-45% of OP admissions reached the threshold for retention, defined as 90 days in treatment. IOP treatment completion was attained by 50-58% of admissions. For OP, the incentive and nonincentive groups had no significant differences in percentages with treatment engagement (AOR = 1.28, DID = 5.9%, p = .19), and 90-day retention was significant in the opposite direction of what we hypothesized (AOR = 0.80, DID = -4.6%, p = .0003). For IOP, the incentive group had a significant, but still small, increase in percentage with treatment engagement (AOR = 1.52, DID = 5.5%, p = .003), but the corresponding increase in treatment completion was not similarly significant (AOR = 1.12, DID = 2.7%, p = .53). In all models, individual-level variables were strong predictors of outcomes.

CONCLUSION

We found little to no impact of the incentive-based contract on the treatment engagement, retention, and completion measures, adding to the body of evidence that shows few or null results for value-based purchasing in SUD treatment programs. The limited success of such efforts is likely to reflect the bandwidth that providers and programs have to focus on new endeavors, the importance of the incentive funding to their bottom line, and forces beyond their immediate control.

Influence of participation in a quality improvement collaborative on staff perceptions of organizational sustainability

BACKGROUND

Sustainability capacity (SC), which is an organization's ability to implement and maintain change, is influenced by internal attributes, environmental contextual influencers, and intervention attributes. Temporal changes in staff SC perceptions, as well as the influence of quality improvement collaborative (QIC) participation, has generally not been explored. This project addresses this gap, measuring staff SC perceptions at four time points (baseline and every 9 months) for clinics participating in an intervention - the Network for the Improvement of Addiction Treatment QIC initiative (called NIATx200).

METHODS

A mixed linear model repeated measures analysis was applied to matched staff members (n = 908, representing 2329 total cases) across the evaluation timeframe. Three separate statistical models assessed potential predictors of SC perceptions: Time (Models I-III); NIATx200 intervention, staff job function, and tenure (Models II &III); and NIATx200 participation hours and four organizational variables (Model III).

RESULTS

For Model I, staff perceptions of total SC increased throughout most of the study (t1,4 = - 6.74, p < .0001; t2,4 = - 3.100, p < .036; t3,4 = - 0.23, p = ns). Model II did not change Model I's overall Time effect, but combined NIATx200 services (t = - 2.23, p = .026), staff job function (t = - 3.27, p = .001), and organizational administrators (t = - 3.50, p = .001) were also significantly associated with greater perceptions of total SC. Inclusion of additional variables in Model III demonstrated the importance of a higher participation level (t = - 3.09, p < .002) and being in a free-standing clinic (t = - 2.06, p < .04) on staff perceptions of total SC.

CONCLUSION

Although staff exposure to sustainability principals was minimal in NIATx200, staff perceptions about their organization's SC significantly differed over time. However, an organization's participation level in a QIC became the principal predictor of staff SC perceptions, regardless of other factors' influence. Given these findings, it is possible to develop and introduce specific sustainability content within the structure of a QIC to assess the impact on staff SC perceptions over time and the sustainment of organizational change.

TRIAL REGISTRATION

ClinicalTrials.gov , NCT00934141 . Registered July 6, 2009. Retrospectively registered.

Pay-for-performance as a cost-effective implementation strategy: results from a cluster randomized trial

BACKGROUND

Pay-for-performance (P4P) has been recommended as a promising strategy to improve implementation of high-quality care. This study examined the incremental cost-effectiveness of a P4P strategy found to be highly effective in improving the implementation and effectiveness of the Adolescent Community Reinforcement Approach (A-CRA), an evidence-based treatment (EBT) for adolescent substance use disorders (SUDs).

METHODS

Building on a $30 million national initiative to implement A-CRA in SUD treatment settings, urn randomization was used to assign 29 organizations and their 105 therapists and 1173 patients to one of two conditions (implementation-as-usual (IAU) control condition or IAU+P4P experimental condition). It was not possible to blind organizations, therapists, or all research staff to condition assignment. All treatment organizations and their therapists received a multifaceted implementation strategy. In addition to those IAU strategies, therapists in the IAU+P4P condition received US $50 for each month that they demonstrated competence in treatment delivery (A-CRA competence) and US $200 for each patient who received a specified number of treatment procedures and sessions found to be associated with significantly improved patient outcomes (target A-CRA). Incremental cost-effectiveness ratios (ICERs), which represent the difference between the two conditions in average cost per treatment organization divided by the corresponding average difference in effectiveness per organization, and quality-adjusted life years (QALYs) were the primary outcomes.

RESULTS

At trial completion, 15 organizations were randomized to the IAU condition and 14 organizations were randomized to the IAU+P4P condition. Data from all 29 organizations were analyzed. Cluster-level analyses suggested the P4P strategy led to significantly higher average total costs compared to the IAU control condition, yet this average increase of 5% resulted in a 116% increase in the average number of months therapists demonstrated competence in treatment delivery (ICER = $333), a 325% increase in the average number of patients who received the targeted dosage of treatment (ICER = $453), and a 325% increase in the number of days of abstinence per patient in treatment (ICER = $8.134). Further supporting P4P as a cost-effective implementation strategy, the cost per QALY was only $8681 (95% confidence interval $1191-$16,171).

CONCLUSION

This study provides experimental evidence supporting P4P as a cost-effective implementation strategy.

TRIAL REGISTRATION

NCT01016704 .

Assessing long-term sustainment of clinic participation in NIATx200: Results and a new methodological approach

BACKGROUND

Sustainability frameworks differentiate between sustainability capacity and sustainment of organizational change. Multiple studies have examined sustainability capacity. Methodological approaches to assess long-term sustainment have not been explored. This study addresses this gap by describing the development of a long-term sustainment methodology and evaluating its application utilizing data from substance abuse clinics participating in a quality improvement collaborative.

METHODS

The study involved clinics (n = 121) in three states (MI, NY and WA) participating in the 2007-2009 NIATx200 quality improvement (QI) intervention. It extended the primary analysis to focus on clinics' long-term sustainment of wait time, retention and admission improvements. Long-term sustainment was defined as two years post end of the active implementation period (Calendar Years 2010 and 2011). The analysis defined case exclusion criteria and spline "knot" time intervals; allowed for C_p statistic use to address clinic data volatility; established the structure of sustainment plots and explored differences between NIATx implementation strategies.

RESULTS

Example spline and sustain plots highlight the application of the long-term sustainment methodology for NIATx200 clinics. In clinics with available longitudinal outcome data, 40.8% (n = 31 of 76 clinics) sustained improvements in wait time, 26.7% (n = 20 of 75 clinics) in retention, and 28.1% (n = 32 of 114 clinics) for admissions. Clinic assignment to a NIATx200 implementation strategy did not significantly influence a clinics' long-term sustainment except for lower wait time changes in the interest circle interventions. Thirty clinics (24.8%) sustained improvements for two outcomes and six clinics (5.0%) did so for all three outcomes. The clinics that sustained multiple outcome improvements were assigned to the interest circle (n = 12), learning session (n = 10), combination (n = 8), and coaching (n = 6) implementation strategies. Guidance for applying the long-term sustainment methodology in other quality improvement settings is described.

CONCLUSIONS

Research about sustainability capacity and sustainment of change has become increasingly important in dissemination and implementation research. Assessment of long-term sustainment in a multi-organizational quality improvement collaborative (QIC) is needed to identify when program drift and intervention decay occurs. If "cut-points" indicate when effects diminish, specific sustainability modules could be developed and introduced within the structure of a QIC to improve organizational long-term sustainment. Coaches and change teams could be trained to focus on organizational change sustainment and strengthen the likelihood of institutionalization. ClinicalTrials.gov Identifier: NCT00934141 Registered July 6, 2009. Retrospectively registered.

Using NIATx strategies to implement integrated services in routine care: a study protocol

BACKGROUND

Access to integrated services for individuals with co-occurring substance use and mental health disorders is a long-standing public health issue. Receiving integrated treatment services are both more effective and preferred by patients and families versus parallel or fragmented care. National policy statements and expert consensus guidelines underscore the benefits of integrated treatment. Despite decades of awareness, adequate treatment for individuals with co-occurring substance use and mental health disorders occurs infrequently. The underlying disease burden associated with alcohol, illicit and prescription drug problems, as well as mental health disorders, such as depression, posttraumatic stress disorder and schizophrenia, is substantial.

METHODS

This cluster randomized controlled trial (RCT) is designed to determine if the multi-component Network for the Improvement of Addiction Treatment (NIATx) strategies are effective in implementing integrated services for persons with co-occurring substance use and mental health disorders. In this study, 50 behavioral health programs in Washington State will be recruited and then randomized into one of two intervention arms: 1) NIATx implementation strategies, including coaching and learning sessions over a 12-month intervention period to implement changes targeting integrated treatment services; or 2) wait-list control. Primary outcome measures include: 1) fidelity - a standardized organizational assessment of integrated services (Dual Diagnosis in Addiction Treatment [DDCAT] Index); and 2) penetration - proportion of patients screened and diagnosed with co-occurring disorders, proportion of eligible patients receiving substance use and mental health services, and psychotropic or substance use disorder medications. Barriers and facilitators, as determinants of implementation outcomes, will be assessed using the Consolidated Framework for Implementation Research (CFIR) Index. Fidelity to and participation in NIATx strategies will be assessed utilizing the NIATx Fidelity Scale and Stages of Implementation Completion (SIC).

DISCUSSION

This study addresses an issue of substantial public health significance: the gap in access to an evidence-based practice for integrated treatment for individuals with co-occurring mental health and substance use disorders. The study utilizes rigorous and reproducible quantitative approaches to measuring implementation determinants and strategies, and may address a longstanding gap in the quality of care for persons with co-occurring disorders.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03007940 . Registered 02 January 2017 - Retrospectively Registered.

Testing the implementation and sustainment facilitation (ISF) strategy as an effective adjunct to the Addiction Technology Transfer Center (ATTC) strategy: study protocol for a cluster randomized trial

BACKGROUND

Improving the extent to which evidence-based practices (EBPs)-treatments that have been empirically shown to be efficacious or effective-are integrated within routine practice is a well-documented challenge across numerous areas of health. In 2014, the National Institute on Drug Abuse funded a type 2 effectiveness-implementation hybrid trial titled the substance abuse treatment to HIV Care (SAT2HIV) Project. Aim 1 of the SAT2HIV Project tests the effectiveness of a motivational interviewing-based brief intervention (MIBI) for substance use as an adjunct to usual care within AIDS service organizations (ASOs) as part of its MIBI Experiment. Aim 2 of the SAT2HIV Project tests the effectiveness of implementation and sustainment facilitation (ISF) as an adjunct to the Addiction Technology Transfer Center (ATTC) model for training staff in motivational interviewing as part of its ISF Experiment. The current paper describes the study protocol for the ISF Experiment.

METHODS

Using a cluster randomized design, case management and leadership staff from 39 ASOs across the United States were randomized to receive either the ATTC strategy (control condition) or the ATTC + ISF strategy (experimental condition). The ATTC strategy is staff-focused and includes 10 discrete strategies (e.g., provide centralized technical assistance, conduct educational meetings, provide ongoing consultation). The ISF strategy is organization-focused and includes seven discrete strategies (e.g., use an implementation advisor, organize implementation team meetings, conduct cyclical small tests of change). Building upon the exploration-preparation-implementation-sustainment (EPIS) framework, the effectiveness of the ISF strategy is examined via three staff-level measures: (1) time-to-proficiency (i.e., preparation phase outcome), (2) implementation effectiveness (i.e., implementation phase outcome), and (3) level of sustainment (i.e., sustainment phase outcome).

DISCUSSION

Although not without limitations, the ISF experiment has several strengths: a highly rigorous design (randomized, hypothesis-driven), high-need setting (ASOs), large sample size (39 ASOs), large geographic representation (23 states and the District of Columbia), and testing along multiple phases of the EPIS continuum (preparation, implementation, and sustainment). Thus, study findings will significantly improve generalizable knowledge regarding the best preparation, implementation, and sustainment strategies for advancing EBPs along the EPIS continuum. Moreover, increasing ASO's capacity to address substance use may improve the HIV Care Continuum. Trial registration ClinicalTrials.gov: NCT03120598.

Using nominal group technique among clinical providers to identify barriers and prioritize solutions to scaling up opioid agonist therapies in Ukraine

BACKGROUND

Opioid agonist therapies (OAT) like methadone and buprenorphine maintenance treatment remain markedly under-scaled in Ukraine despite adequate funding. Clinicians and administrators were assembled as part of an implementation science strategy to scale-up OAT using the Network for Improvement of Addiction Treatment (NIATx) approach.

METHODS

Nominal Group Technique (NGT), a key ingredient of the NIATx toolkit, was directed by three trained coaches within a learning collaborative of 18 OAT clinicians and administrators to identify barriers to increase OAT capacity at the regional "oblast" level, develop solutions, and prioritize local change projects. NGT findings were supplemented from detailed notes collected during the NGT discussion.

RESULTS

The top three identified barriers included: (1) Strict regulations and inflexible policies dictating distribution and dispensing of OAT; (2) No systematic approach to assessing OAT needs on regional or local level; and (3) Limited funding and financing mechanisms combined with a lack of local/regional control over funding for OAT treatment services.

CONCLUSIONS

NGT provides a rapid strategy for individuals at multiple levels to work collaboratively to identify and address structural barriers to OAT scale-up. This technique creates a transparent process to address and prioritize complex issues. Targeting these priorities allowed leaders at the regional and national level to advocate collectively for approaches to minimize obstacles and create policies to improve OAT services.

Building capacity for continuous quality improvement (CQI): A pilot study

BACKGROUND AND OBJECTIVE

Little is known about the feasibility, effectiveness, and sustainability of CQI approaches in substance use disorder treatment settings.

METHODS

In the initial phase of this study, eight programs were randomly assigned to receive a CQI intervention or to a waitlist control condition to obtain preliminary information about potential effectiveness. In the second phase, the initially assigned control programs received the CQI intervention to gain additional information about intervention feasibility while sustainability was explored among the initially assigned intervention programs.

RESULTS AND CONCLUSIONS

Although CQI was feasible and sustainable, demonstrating its effectiveness using administrative data was challenging suggesting the need to better align performance measurement systems with CQI efforts. Further, although the majority of staff were enthusiastic about utilizing this approach and reported provider and patient benefits, many noted that dedicated time was needed in order to implement and sustain it.

Continuous quality improvement in substance abuse treatment facilities: How much does it cost?

Continuous quality improvement (CQI) has grown in the U.S. since the 1970s, yet little is known about the costs to implement CQI in substance abuse treatment facilities. This paper is part of a larger group randomized control trial in a large urban county evaluating the impact of Plan-Study-Do-Act (PDSA)-CQI designed for community service organizations (Hunter, Ober, Paddock, Hunt, & Levan, 2014). Operated by one umbrella organization, each of the eight facilities of the study, four residential and four outpatient substance abuse treatment facilities, selected their own CQI Actions, including administrative- and clinical care-related Actions. Using an activity-based costing approach, we collected labor and supplies and equipment costs directly attributable to CQI Actions over a 12-month trial period. Our study finds implementation of CQI and meeting costs of this trial per facility were approximately $2000 to $10,500 per year ($4500 on average), or $10 to $60 per admitted client. We provide a description of the sources of variation in these costs, including differing intensity of the CQI Actions selected, which should help decision makers plan use of PDSA-CQI.

Using a Theory-Guided Learning Collaborative Model to Improve Implementation of EBPs in a State Children's Mental Health System: A Pilot Study

Learning collaboratives (LCs) are used widely to promote implementation of evidence-based practices. However, there has been limited research on the effectiveness of LCs and models vary widely in their structure, focus and components. The goal of the present study was to develop and field test a theory-based LC model to augment a state-led, evidence-based training program for clinicians providing mental health services to children. Analysis of implementation outcomes contrasted LC sites to matched comparison sites that participated in the clinical training program alone. Results suggested that clinicians from sites participating in the LC were more highly engaged in the state-led clinical training program and were more likely to complete program requirements.

Defining Success: Insights From a Random Assignment, Multisite Study of Implementing HIV Prevention, Testing, and Linkage to Care in U.S. Jails and Prisons

In the emerging field of implementation science, measuring the extent to which a new or modified healthcare program or practice is successfully implemented following an intervention is a critical component in understanding how evidence-based treatments become part of regular practice. This paper is intended to expand our understanding of factors that influence the successful adoption of new or modified HIV services in correctional settings. The nine-site project developed and directed an organization-level intervention designed to implement improvements in preventing, detecting, and treating HIV for persons under correctional supervision. Using semi-structured interviews to elicit perceptions from Senior Researchers and Executive Sponsors at each of the nine sites, this paper presents their views and observations regarding the success of the experimental intervention in their criminal justice setting. Within the areas of focus for implementation (either HIV prevention, testing, or linkage to community treatment) the complexity of programmatic needs was very influential with regards to perceptions of success. An organization's pre-existing characteristics, staffing, funding, and interorganizational relationships contributed to either the ease or difficulty of programmatic implementation. Results are discussed pertaining to furthering our understanding of why new or modified healthcare interventions achieve success, including whether the intervention is a modification of existing practice or is a new intervention, and the choice of implementation strategy.

Admissions to detoxification after treatment: Does engagement make a difference?

BACKGROUND

Treatment engagement is a well-established performance measure for the treatment of substance use disorders. This study examined whether outpatient treatment engagement is associated with a reduced likelihood of subsequent detoxification admissions.

METHODS

This study used administrative data on treatment services received by clients in specialty treatment facilities licensed in Massachusetts. The sample consisted of 11,591 adult clients who began an outpatient treatment episode in 2006. Treatment engagement was defined as receipt of at least 1 treatment service within 14 days of beginning a new outpatient treatment episode and receipt of at least 2 additional treatment services in the next 30 days. The outcome was a subsequent detoxification admission. Multilevel survival models examined the relationship between engagement and outcomes, with time to detoxification admission as the dependent variable censored at 365 days.

RESULTS

Only 35% of clients met the outpatient engagement criteria, and 15% of clients had a detoxification admission within a year after beginning their outpatient treatment episode. Controlling for client demographics, insurance type, and substance use severity, clients who met the engagement criteria had a lower hazard of having a detoxification admission during the year following the index outpatient visit than those who did not engage (hazard ratio = 0.87, P < .01).

CONCLUSIONS

Treatment engagement is a useful measure for monitoring quality of care. The findings from this study could help inform providers and policy makers on ways to target care and reduce the likelihood of more intensive services.

Effects of a strategy to improve offender assessment practices: Staff perceptions of implementation outcomes

BACKGROUND

This implementation study examined the impact of an organizational process improvement intervention (OPII) on a continuum of evidence based practices related to assessment and community reentry of drug-involved offenders: Measurement/Instrumentation, Case Plan Integration, Conveyance/Utility, and Service Activation/Delivery.

METHODS

To assess implementation outcomes (staff perceptions of evidence-based assessment practices), a survey was administered to correctional and treatment staff (n=1509) at 21 sites randomly assigned to an Early- or Delayed-Start condition. Hierarchical linear models with repeated measures were used to examine changes in evidence-based assessment practices over time, and organizational characteristics were examined as covariates to control for differences across the 21 research sites.

RESULTS

Results demonstrated significant intervention and sustainability effects for three of the four assessment domains examined, although stronger effects were obtained for intra- than inter-agency outcomes. No significant effects were found for Conveyance/Utility.

CONCLUSIONS

Implementation interventions such as the OPII represent an important tool to enhance the use of evidence-based assessment practices in large and diverse correctional systems. Intra-agency assessment activities that were more directly under the control of correctional agencies were implemented most effectively. Activities in domains that required cross-systems collaboration were not as successfully implemented, although longer follow-up periods might afford detection of stronger effects.

A literature review of learning collaboratives in mental health care: used but untested

OBJECTIVE

Policy makers have increasingly turned to learning collaboratives (LCs) as a strategy for improving usual care through the dissemination of evidence-based practices. The purpose of this review was to characterize the state of the evidence for use of LCs in mental health care.

METHODS

A systematic search of major academic databases for peer-reviewed articles on LCs in mental health care generated 421 unique articles across a range of disciplines; 28 mental health articles were selected for full-text review, and 20 articles representing 16 distinct studies met criteria for final inclusion. Articles were coded to identify the LC components reported, the focus of the research, and key findings.

RESULTS

Most of the articles included assessments of provider- or patient-level variables at baseline and post-LC. Only one study included a comparison condition. LC targets ranged widely, from use of a depression screening tool to implementation of evidence-based treatments. Fourteen crosscutting LC components (for example, in-person learning sessions, phone meetings, data reporting, leadership involvement, and training in quality improvement methods) were identified. The LCs reviewed reported including, on average, seven components, most commonly in-person learning sessions, plan-do-study-act cycles, multidisciplinary quality improvement teams, and data collection for quality improvement.

CONCLUSIONS

LCs are being used widely in mental health care, although there is minimal evidence of their effectiveness and unclear reporting in regard to specific components. Rigorous observational and controlled research studies on the impact of LCs on targeted provider- and patient-level outcomes are greatly needed.

Reducing appointment no-shows: going from theory to practice

Addiction appointment no-shows adversely impact clinical outcomes and healthcare productivity. During 2007-2010, 67 treatment organizations in the Strengthening Treatment Access and Retention program were asked to reduce their no-show rates by using practices taken from no-show research and theory. These treatment organizations reduced outpatient no-show rates from 37.4% to 19.9% (p = .000), demonstrated which practices they preferred to implement, and which practices were most effective in reducing no-show rates. This study provides an applied synthesis of addiction treatment no-show research and suggests future directions for no-show research and practice.

Measuring sustainability within the Veterans Administration Mental Health System Redesign initiative

OBJECTIVE

To examine how attributes affecting sustainability differ across Veterans Health Administration organizational components and by staff characteristics.

SUBJECTS

Surveys of 870 change team members and 50 staff interviews within the Veterans Affairs' Mental Health System Redesign initiative.

METHODS

A 1-way ANOVA with a Tukey post hoc test examined differences in sustainability by Veteran Integrated Service Networks, job classification, and tenure from staff survey data of the Sustainability Index. Qualitative interviews used an iterative process to identify "a priori" and "in vivo" themes. A simple stepwise linear regression explored predictors of sustainability.

RESULTS

Sustainability differed across Veteran Integrated Service Networks and staff tenure. Job classification differences existed for the following: (1) benefits and credibility of the change and (2) staff involvement and attitudes toward change. Sustainability barriers were staff and institutional resistance and nonsupportive leadership. Facilitators were commitment to veterans, strong leadership, and use of quality improvement tools. Sustainability predictors were outcomes tracking, regular reporting, and use of Plan, Do, Study, Adjust cycles.

CONCLUSIONS

Creating homogeneous implementation and sustainability processes across a national health system is difficult. Despite the Veterans Affairs' best evidence-based implementation efforts, there was significant variance. Locally tailored interventions might better support sustainability than "one-size-fits-all" approaches. Further research is needed to understand how participation in a quality improvement collaborative affects sustainability.