Sensitivity analysis in multiple imputation in effectiveness studies of psychotherapy

The importance of preventing and treating incomplete data in effectiveness studies is nowadays emphasized. However, most of the publications focus on randomized clinical trials (RCT). One flexible technique for statistical inference with missing data is multiple imputation (MI). Since methods such as MI rely on the assumption of missing data being at random (MAR), a sensitivity analysis for testing the robustness against departures from this assumption is required. In this paper we present a sensitivity analysis technique based on posterior predictive checking, which takes into consideration the concept of clinical significance used in the evaluation of intra-individual changes. We demonstrate the possibilities this technique can offer with the example of irregular longitudinal data collected with the Outcome Questionnaire-45 (OQ-45) and the Helping Alliance Questionnaire (HAQ) in a sample of 260 outpatients. The sensitivity analysis can be used to (1) quantify the degree of bias introduced by missing not at random data (MNAR) in a worst reasonable case scenario, (2) compare the performance of different analysis methods for dealing with missing data, or (3) detect the influence of possible violations to the model assumptions (e.g., lack of normality). Moreover, our analysis showed that ratings from the patient's and therapist's version of the HAQ could significantly improve the predictive value of the routine outcome monitoring based on the OQ-45. Since analysis dropouts always occur, repeated measurements with the OQ-45 and the HAQ analyzed with MI are useful to improve the accuracy of outcome estimates in quality assurance assessments and non-randomized effectiveness studies in the field of outpatient psychotherapy.

Lending an ear to feedback systems: evaluation of recovery and non-response in psychotherapy in a German outpatient setting

Systems providing feedback on treatment progress have been implemented in outpatient psychotherapy. They are recognized as a helpful tool to identify possible treatment failures. This report presents the ideas underlying the planning of feedback interventions and the implementation of such programs into practice settings. Strategies to identify patients at risk for treatment failure (rationally- and empirically-derived decision rules) are presented. Additionally, evidence for the usefulness of feedback systems is discussed. The report ends with the description of an ongoing feedback intervention study in private practices in Germany (aimed at gathering information on 400 therapists with 2,000 patients).

Empirically and clinically useful decision making in psychotherapy: Differential predictions with treatment response models

In the delivery of clinical services, outcomes monitoring (i.e., repeated assessments of a patient's response to treatment) can be used to support clinical decision making (i.e., recurrent revisions of outcome expectations on the basis of that response). Outcomes monitoring can be particularly useful in the context of established practice research networks. This article presents a strategy to disaggregate patients into homogeneous subgroups to generate optimal expected treatment response profiles, which can be used to predict and track the progress of patients in different treatment modalities. The study was based on data from 618 diagnostically diverse patients treated with either a cognitive-behavioral treatment protocol (n = 262) or an integrative cognitive-behavioral and interpersonal treatment protocol (n = 356). The validity of expected treatment response models to predict treatment in those 2 protocols for individual patients was evaluated. The ways such a procedure might be used in outpatient centers to learn more about patients, predict treatment response, and improve clinical practice are discussed.

Reliabilität von Kurzformen gängiger psychometrischer Instrumente zur Evaluation des therapeutischen Fortschritts in Psychotherapie und Psychiatrie

Zusammenfassung. Im Rahmen der sich in den letzten Jahren immer stärker etablierenden Wissenschaftler-Praktiker-Netzwerke und Qualitätssicherungsstudien in der Psychotherapie und Psychiatrie wird der Einsatz von ökonomischen, aber multidimensionalen und veränderungssensitiven klinischen Messinstrumenten zur Verlaufs- und Ergebnismessung therapeutischer Behandlungen immer wichtiger. In der vorliegenden Arbeit wird die Entwicklung von Kurzfragebögen vier gängiger Messinstrumente (der Symptom-Checkliste, SCL-90-R; dem Emotionalitätsinventar, EMI; dem Inventar Interpersonaler Probleme, IIP und dem Inkongruenzfragebogen, INK) sowie die Validität, Reliabilität und Veränderungssensitivität dieser Kurzfragebögen anhand einer Stichprobe der Allgemeinbevölkerung (N = 436), einer Stichprobe ambulanter Psychotherapiepatienten (N = 169) sowie einer Stichprobe von Patienten in stationärer Psychotherapie (N = 134) dargestellt. Die psychometrischen Eigenschaften dieser Kurzformen sowie ein daraus generierter auf 43 Items beruhender Gesamtwert für die allgemeine psychische Beeinträchtigung erlauben den kombinierten Einsatz dieser Instrumente für eine kontinuierliche und ökonomische Erhebung des therapeutischen Fortschritts.

Zur Entwicklung von Entscheidungsregeln in der Psychotherapie

Zusammenfassung. Theoretischer Hintergrund und Fragestellung: Das Monitoring von Psychotherapieverläufen, d.h. die wiederholte Messung der durch die psychotherapeutische Behandlung bewirkten Veränderungen beim einzelnen Patienten und der Vergleich mit statistisch vorhergesagten Veränderungen, kann die Entscheidungsprozeduren hinsichtlich des therapeutischen Vorgehens, gerade in etablierten Wissenschaftler-Praktiker-Netzwerken, maßgeblich unterstützen. Die vorliegende Arbeit adaptiert eine bereits in der Lawinenforschung etablierte Strategie zur Vorhersagen auf der Basis ähnlicher Gruppen, um ein enstprechendes differenzielles Vorhersagesystem für psychotherapeutische Behandlungsverläufe zu entwickeln. Methode und Ergebnisse: Die Studie basiert auf insgesamt 618 Patientinnen und Patienten mit breitem Diagnosespektrum, welche entweder mit einem kombiniert bewältigungs- und klärungsorientierten (n = 356) oder einem rein bewältigungsorientierten Vorgehen (n = 262) behandelt wurden. Bei diesem Vorgehen wurden die Patienten in einem ersten Schritt in homogene Subgruppen eingeteilt, um in einem zweiten Schritt mit Hilfe von Wachstumsanalysen ihren Psychotherapieverlauf vorherzusagen und mit dem tatsächlichen Verlauf vergleichen zu können. Die Validität der statistischen Vorhersagemodelle für die einzelnen Patienten und Behandlungsmodalitäten wurde mit einer sequenzanalytischen Verfahren überprüft. Schlussfolgerungen: Mögliche Anwendungen und Implikationen für die klinische Praxis, welche sowohl mehr Informationen über die Patienten zur Verfügung stellen als auch damit klinische Entscheidungsprozesse verbessern können, werden diskutiert. Momentane Grenzen des Ansatzes sowie notwendige weitere Forschungsaspekte werden aufgezeigt.

Reconsidering the evaluation of addiction treatment: from retrospective follow-up to concurrent recovery monitoring

Historically, addiction treatments have been delivered and evaluated under an acute-care format. Fixed amounts or durations of treatment have been provided and their effects evaluated 6-12 months after completion of care. The explicit expectation of treatment has been enduring reductions in substance use, improved personal health and social function, generally referred to as 'recovery'. In contrast, treatments for chronic illnesses such as diabetes, hypertension and asthma have been provided for indeterminate periods and their effects evaluated during the course of those treatments. Here the expectations are for most of the same results, but only during the course of continuing care and monitoring. The many similarities between addiction and mainstream chronic illnesses stand in contrast to the differences in the ways addiction is conceptualized, treated and evaluated. This paper builds upon established methods of during-treatment evaluation developed for the treatment of other chronic illnesses and suggests a parallel evaluation system for out-patient, continuing-care forms of addiction treatment. The suggested system retains traditional patient-level, behavioral outcome measures of recovery, but suggests that these outcomes should be collected and reported immediately and regularly by clinicians at the beginning of addiction treatment sessions, as a way of evaluating recovery progress and making decisions about continuing care. We refer to this paradigm as 'concurrent recovery monitoring' and discuss its potential for producing more timely, efficient, clinically relevant and accountable evaluations.

A conceptual framework for drug treatment process and outcomes

Evidence from specialized treatment evaluations and large-scale natural studies of treatment effectiveness is organized conceptually into a "treatment model" for summarizing how drug treatment works. Sequential relationships between patient and treatment program attributes, early patient engagement, recovery stages, retention, and favorable outcomes are discussed--along with behavioral, cognitive, and skills training interventions that have been shown to be effective for enhancing specific stages of the patient recovery process. Applications of the treatment model for incorporating science-based innovations into clinical practice for improving early engagement and retention, performance measurements of patient progress, program monitoring and management using aggregated patient records, and organizational functioning and systems change also are addressed.

Computer-supported monitoring of patient treatment response

The use of information technology (IT) for the purpose of improving psychotherapy outcomes in the context of routine clinical practice is described. IT efforts from two research programs, one in Germany and the other in the United States, are based on evidence that not all patients who enter treatment have a positive outcome and that continuous monitoring of patient treatment response with immediate feedback to therapists can be used to increase the likelihood of success for the poorly responding client. Such monitoring and feedback can best be accomplished by IT methodologies. Suitable IT systems are described, and their strengths and limitations are highlighted. Evidence is presented that supports the positive value of such efforts for improving patient treatment response.

Clinicians and outcome measurement: what's the use?

The goal of this study was to learn more about clinicians' experiences with, and perceptions of the utility, validity, and feasibility of standardized outcome measures in practice. Fifty randomly selected clinicians from multiple disciplines and multiple service agencies in a large children's public mental health service system were interviewed individually (n = 30) or in focus groups (n = 20) using semistructured interviews. There was great variability across clinicians in attitudes about empirical methods of treatment evaluation. There was consensus regarding feasibility challenges of administering standardized measures, including time burden and literacy barriers. Although all participants had received scored assessment profiles for their clients, the vast majority reported that they did not use the scores in treatment planning or monitoring. Their suggestions for improved clinical utility of outcome measurement are included. With increased attention and resources devoted to performance outcome assessment, it is concerning that most clinicians perceive little clinical utility of outcome measurement.

Measuring coping style and resistance among Spanish and Argentine samples: development of the Systematic Treatment Selection Self-Report in Spanish

This article reports the reliability and validity of the Coping Style and Resistance subscales of the Systematic Treatment Selection Self-Report (STS-SR) scale after its translation to Spanish. The English versions of the STS system identify eight empirically derived patient variables that have been found to moderate the effects of various classes of mental health treatment procedures. Research on these dimensions as treatment indicators, as assessed both by clinicians and through patient self-reports, indicates that they increase the efficiency and effectiveness of treatment among English-speaking samples. This article reports a study of the convergent and discriminant validity of two of the subscales, Coping Style and Trait Resistance. The subscales were translated English into Spanish and administered to large Spanish and Argentine subject pools, along with established measures of the same and related constructs. Results revealed that both scales possessed good internal consistency and adequate discriminant validity. The implications of these findings are discussed.

How practitioners can systematically use empirical evidence in treatment selection

Contemporary concerns with "empirically supported treatments" emphasize the differences in outcomes that are associated with reliably delivered treatments, representing different models and theories. This approach often fails to address the fact that there is no consensus among scientists about whether there are enough differences between and among treatments to make this effort productive. There is a considerable body of data that suggests that all treatments produce very similar effects. This article reviews these viewpoints and presents a third position, suggesting that identifying common and differential principles of change may be more productive than focusing on the relative value of different theoretical models.