The development of patient profiles for Minnesota's treatment outcomes monitoring system

Engaging stakeholders in review and recommendations for models of outcome monitoring for substance abuse treatment

We present an example of a collaborative process designed to review models of outcome monitoring for substance abuse services, with a view to assessing the feasibility of different approaches in Ontario, Canada. A conceptual framework that describes the parameters of an outcome monitoring system and four models of outcome monitoring were identified. Consultations were held with stakeholders (managers, directors, researchers, clinicians, and governmental representatives) about the types of information they would like to obtain from an outcome monitoring system. Our process is useful as a model for collaborative research with respect to performance measurement. The study's implications and limitations are noted.

Implementing an adapted version of the job seekers' workshop in a residential program for patients with substance use disorders

OBJECTIVES

To evaluate the feasibility and effectiveness of adapting the Job Seekers' Workshop (JSW) to a residential setting within a Massachusetts-based substance use disorder treatment agency.

METHODS

Implementation of the adapted JSW consisted of a continual sequence of three weekly sessions that focused on job interview rehearsals, practice completing job applications, and identification of job leads. Data were compiled on the employment rates of the 188 patients discharged from the residential treatment program during July - December 2006 (baseline participants, n = 95) and January - June 2007 (JSW intervention participants, n = 93). The effectiveness of the adapted JSW was evaluated through a comparison of baseline and intervention participants' employment rates at discharge from residential treatment.

RESULTS

Analyses indicated a trend towards a significant increase in employment at discharge for the intervention period (40.9%) compared to baseline (29.5%), χ(2)(1, N = 188) = 2.675, p = .051.

CONCLUSIONS

Further evaluation of the JSW in residential settings is necessary, but this preliminary research suggests that the intervention could begin to address the need for vocational services in residential treatment for substance use disorders.

Baseline Abstinence May Moderate Substance Use Disorder Intrvention Effects

In many evaluations of care for patients with substance use disorders (SUD), a substantial proportion of patients are abstinent from substances prior to their baseline assessment. For researchers evaluating the effect of treatments or patterns of care on follow-up abstinence, especially with non-randomized designs, this raises important conceptual and statistical questions. We discuss several ways researchers might address baseline heterogeneity in abstinence either through design or statistical approaches. For illustration, we estimate the effect of continuing care duration on follow-up abstinence in a sample of 3,181 SUD outpatients treated in 54 Veterans Health Affairs (VHA) facilities. We show that failure to separately examine individuals who are and are not abstinent at baseline overestimates the “treatment effect” on substance use outcomes for those abstinent at baseline and underestimates the treatment effect for those not abstinent at baseline. We also describe comparable findings for the effect of attendance in 12-step self-help groups.

Assessing comorbid substance use in detained psychiatric patients: issues and instruments for evaluating treatment outcome

This review assesses the issues involved in the selection and treatment of patients comorbid for mental illness and substance misuse being treated in secure psychiatric facilities. It includes those individuals who have a history of offending and whose placement is the result of severe behavioral disturbance. The relevant issues in the assessment and treatment of these patients are reviewed and a battery of tests is suggested on the basis of their usefulness with this population in terms of their brevity, ease of administration, and for their value in planning treatment, providing motivational feedback, and monitoring change. The paucity of assessment tools developed specifically for this patient population is highlighted.

A practical system for monitoring the outcomes of substance use disorder patients

Outcomes monitoring and management are receiving mounting attention because of increased emphasis on health care accountability and cost containment. Efficient, practical outcomes monitoring systems (OMSs) are crucial if health care system performance is to be determined and effective/cost-effective treatments are to be identified, but such practical monitoring systems generally are lacking. This article describes the features of such a system for monitoring the care received by, and the substance use and psychosocial outcomes of, patients treated for substance use disorders (SUDs) in the Department of Veterans Affairs (VA). In contrast to a 15-21% follow-up rate achieved by VA SUD program staff under a previously mandated system-wide monitoring system, the monitoring system used in this project achieved a 67% follow-up rate without paying patients for their participation. We provide data on patient characteristics and treatment outcomes, estimate the cost of implementing this type of monitoring system on a broad scale, and provide recommendations for OMSs in other large health care organizations.

Development and validation of a brief instrument for routine outcome monitoring in opioid maintenance pharmacotherapy services: the brief treatment outcome measure (BTOM)

AIM

To develop a brief, multi-dimensional instrument for routine, on-going treatment outcome monitoring in alcohol and other drug (AOD) services in Australia and examine the underlying psychometric properties. This study focuses on opioid maintenance pharmacotherapy (OMP) services.

DESIGN

Researcher-administered test-retest interviews and clinician administered interviews.

SETTING

The test-retest interviews took place in two private and two public OMP clinics in Sydney, Australia. The clinician-administered interviews took place in 37 metropolitan, rural and prison OMP services in New South Wales, Australia.

PARTICIPANTS

One hundred and sixty current OMP clients for the test-retest interviews and 2004 clients commencing OMP treatment for the clinician-administered interviews.

MEASURES

Thirty-two items across the domains of dependence, blood-borne virus exposure risk, drug use, health/psychological functioning and social functioning.

FINDINGS

The internal reliability of the brief treatment outcome measure (BTOM) is satisfactory. Retest reliabilities for the scales and drug use scores are good to excellent indicating their consistency with multiple measurements across time and different interviewers and concurrent validation of BTOM scales with analogous scales from similar instruments yielded acceptable agreement. Average completion times of the BTOM were 14.5 min, when administered in a research context and 21 min in a clinical context.

CONCLUSIONS

A brief, valid and reliable questionnaire has been developed for monitoring treatment outcome over the range of OMP services. The BTOM contains a core set of outcome measures, which can be easily adapted for integration into routine clinical practice across the range of AOD services.

The evaluation of treatment services and systems for substance use disorders

Scientific research and program evaluation have not played a major role in shaping the development of treatment services and systems in most countries. This has led to disparities in the development, management and monitoring of national treatment systems. In the evaluation of treatment for substance use disorders, the evaluation practitioner will usually be working at one of five levels: single case, treatment activity, treatment service, treatment agency or treatment system. One of the major barriers to undertaking internal program evaluation is the belief that it is a complicated research process best left to those with specific research training. Program managers and staff can plan and initiate an evaluation process for their program if they have access to research expertise when needed for certain parts of the process. There are seven main components of an evaluation process that can be planned and implemented: need assessment; evaluation planning, process evaluation, cost analysis, client satisfaction evaluation, outcome evaluation and economic evaluation. However, evaluation is more than the techniques and technology required to implement these types of activities. It also involves the routine questioning of current practice even if the feedback may be less positive than anticipated. A healthy culture for evaluation is one in which feedback loops are woven into the fabric of the treatment service or system. There are many barriers to evaluation in substance abuse services but these barriers can be overcome with careful planning and commitment to the delivery of evidence-based services.

The relationship between problem severity and ancillary treatment services: is substance abuse treatment responsive to client need?

This study examined whether a greater self-reported problem severity at intake is associated with a greater likelihood of receiving related ancillary treatment services. The sample of 3,625 adult substance abuse treatment completers represents 162 outpatient and 41 short-term inpatient/residential programs located throughout Minnesota. Problem severity and receipt of ancillary services were assessed in medical, psychological, family/social, employment, financial, and legal domains. Across treatment setting and client gender, individuals with higher problem severity were significantly more likely to receive related ancillary services in the areas of medical services, psychotropic medication, family/relationship counseling, financial services, and legal services. For other services, the positive association between problem severity and receipt of related ancillary services held for some but not all groups defined by treatment setting or gender. Despite consistent relationships between service need and receipt, large gaps remain with respect to programs' responsiveness to client needs.

Outcomes monitoring in Minnesota: treatment implications, practical limitations

Minnesota developed its treatment outcomes monitoring system to evaluate its service delivery system and to identify specific ways to enhance treatment outcomes. Standardized patient and treatment data were collected for all outpatient and inpatient admissions, and an assessment interview and weekly treatment services records were completed for 4953 adults and 387 adolescents who consented to participate in the outcomes study. Six-month follow-up interviews were completed for 64.4% of these adults and 83.5% of these adolescents. The study identified predictors of treatment completion and outcome for adults and adolescents, and predictors of outcome by setting for adults. The study also examined results for programs designed especially for adolescents, women, and culturally specific populations, as well as the relationships to outcomes of ancillary services and continuing care activities. Anticipated policy and program design changes resulting from the findings are discussed, along with the practical limitations of conducting a state outcomes monitoring system.

Developing an integrated information system for specialized addiction treatment agencies

This article outlines the development of an integrated information system for specialized alcohol and drug treatment agencies in Ontario, Canada. The system is being developed following a strategic planning process involving provincial funding ministries and coalitions of service providers. An overview of the system's development is provided and the implementation of one subcomponent, a client-tracking system, is described. Some challenges to the implementation of this component are identified.