Relationships among diverse measures of psychotherapy outcome

Client Perceptions of Helpfulness in Therapy: a Novel Video-Rating Methodology for Examining Process Variables at Brief Intervals During a Single Session

BACKGROUND

The value of clients' reports of their experiences in therapy is widely recognized, yet quantitative methodology has rarely been used to measure clients' self-reported perceptions of what is helpful over a single session.

AIMS

A video-rating method using was developed to gather data at brief intervals using process measures of client perceived experience and standardized measures of working alliance (Session Rating Scale; SRS). Data were collected over the course of a single video-recorded session of cognitive therapy (Method of Levels Therapy; Carey, 2006; Mansell et al., 2012). We examined the acceptability and feasibility of the methodology and tested the concurrent validity of the measure by utilizing theory-led constructs.

METHOD

Eighteen therapy sessions were video-recorded and clients each rated a 20-minute session of therapy at two-minute intervals using repeated measures. A multi-level analysis was used to test for correlations between perceived levels of helpfulness and client process variables.

RESULTS

The design proved to be feasible. Concurrent validity was borne out through high correlations between constructs. A multi-level regression examined the independent contributions of client process variables to client perceived helpfulness. Client perceived control (b = 0.39, 95% CI .05 to 0.73), the ability to talk freely (b = 0.30, SE = 0.11, 95% CI .09 to 0.51) and therapist approach (b = 0.31, SE = 0.14, 95% CI .04 to 0.57) predicted client-rated helpfulness.

CONCLUSIONS

We identify a feasible and acceptable method for studying continuous measures of helpfulness and their psychological correlates during a single therapy session.

Does the Brief Symptom Inventory-18 case rule apply in adult survivors of childhood cancer? Comparison with the Symptom Checklist-90

OBJECTIVE

Screening for psychological distress is an important tool for improving survivors' access to psychosocial care. The Brief Symptom Inventory-18 (BSI-18) has been widely used to identify psychological distress in cancer survivors, but few studies have reported on its validity. This study evaluated validity of the BSI-18 by comparing it to the Symptom Checklist-90 (SCL-90).

METHODS

Concordance of cases identified by the BSI-18 and SCL-90 was examined in a sample of 193 adult survivors of childhood cancer.

RESULTS

Receiver operating characteristics analysis showed strong diagnostic utility of the BSI-18 (area under curve = 0.922). However, the standard BSI-18 case-rule demonstrated low sensitivity (45.2%) against the SCL-90. An alternative case-rule showed better results; sensitivity (87.10%), specificity (83.33%).

CONCLUSIONS

The BSI-18 is a useful measure for evaluating distress in adult survivors of childhood cancers; however, the standard BSI-18 case-rule has not been validated for this population, and an alternative case rule should be considered.

Clients' and therapists' views of the therapeutic alliance: similarities, differences and relationship to therapy outcome

To better understand how clients' and therapists' views of the therapeutic alliance differ and overlap, this study investigated, first, the components of the alliance that are relevant to the therapy participants; second, their relationship to post-therapy outcome; and third, the relationships between participants' alliance constructs. To identify participants' views, exploratory factor analyses were performed on clients' (n = 176) and therapists' (n = 133 observations) ratings of the Working Alliance Inventory (short form), the Helping Alliance Questionnaire and the California Psychotherapy Alliance Scales and conducted both on each measure separately and on the three measures combined. The results of the separate analyses indicated in general poor correspondence between the participant-derived components and each measure's a priori constructs. Results of the joint analyses suggested that clients view the alliance in terms of six basic components (Collaborative Work Relationship, Productive Work, Active Commitment, Bond, Non-disagreement on Goals/Tasks and Confident Progress), five of which were found to predict client-rated and/or therapist-rated post-therapy outcome. Results for therapists suggested four basic components (Collaborative Work Relationship, Therapist Confidence & Dedication, Client Commitment & Confidence, Client Working Ability), of which three predicted post-therapy outcome. Findings of significant, but modest to low moderate, correlations between several client and therapist joint factors suggested that despite similarities, the therapy partners' views of the alliance differ in important ways. Compared with therapists, clients appear to place greater emphasis on helpfulness, joint participation in the work of therapy and negative signs of the alliance. Implications of these findings are discussed.

KEY PRACTITIONER MESSAGE

Therapists should not assume that their views of the therapeutic relationship and therapeutic work are shared by their clients and are encouraged to seek the client's feedback. Therapists may benefit from conveying that the client's perspective on problems and relevant work is valued and that they are working with the client as a team. Therapists may need to explicitly address how the therapeutic work is helpful and conducive to desired changes.

Clinical efficacy and cognitive and neuropsychological effects of levetiracetam in epilepsy: an open-label multicenter study

The aim of this prospective, multicenter, open-label study was to investigate the efficacy of levetiracetam (LEV) and determine its effects on cognitive and neuropsychological function. Sixty-nine patients were evaluated for effects of LEV on seizure control, cognitive (Mini-Mental State Examination [MMSE]) and neuropsychological (Symptom Checklist-90 Revised [SCL-90-R]) functions, and quality of life (Quality of Life in Epilepsy--10 [QOLIE-10]) assessments at 3 and 12 months of follow-up. Thirty-nine percent of patients achieved seizure freedom, and 68% had a > or =50% seizure frequency reduction after 1 year of LEV (1235.5+/-392.7 mg/day). There were also significant improvements in mean MMSE score and in the recall and language items of MMSE. There were modest improvements in interpersonal sensitivity and paranoid ideation scales of the SCL-90-R, and improvements in cognition and medication effect items of the QOLIE-10. The results demonstrate that LEV not only effectively reduces seizure frequency, but also possibly contributes to improvements in neuropsychological functions such as recall, language, interpersonal sensitivity, and paranoid ideation.

Screening childhood cancer survivors with the brief symptom inventory-18: classification agreement with the symptom checklist-90-revised

The Brief Symptom Inventory-18 (BSI-18) is an 18-item symptom checklist used as a brief distress screening in cancer and other medical patients. This study evaluated the validity of the BSI-18 in a sample of 221 adult survivors of childhood cancers ages 18-55 (median = 26). Validity of the BSI-18 was compared to the Symptom Checklist-90-Revised (SCL-90-R). Results indicated the BSI-18 scales had acceptable internal consistency (alpha >0.80) and were highly correlated with the corresponding SCL-90-R subscales (correlations from 0.88 to 0.94). When subjects were classified as case positive (significantly distressed) using the BSI-18 manual case-rule, classification agreement with the SCL-90-R was poor as evidenced by low sensitivity (41.78%). An alternative BSI-18 case-rule previously developed for cancer patients using the General Severity Index (GSI; GSI t-score >or=57) demonstrated better sensitivity (83.54%). ROC analysis indicated the BSI-18 had strong diagnostic utility relative to the SCL-90-R (AUC = 0.98) and several possible GSI cut-off scores were evaluated. The optimal cut-of score was a t-score >or=50 which had a sensitivity of 97.47% and a specificity of 85.21%. Results support use of the BSI-18 with adult survivors of childhood cancer but indicate an alternative case-rule must be used.

Long-term changes after brief dynamic psychotherapy: symptomatic versus dynamic assessments

Dynamic change in psychotherapy, as measured by theory-related or mode-specific instruments, have been criticized for being too intercorrelated with symptomatic change measures. In this study, long-term changes after brief dynamic psychotherapy were studied in 45 moderately disturbed neurotic patients by a reliable outcome battery. The factor structure of all the change variables suggested that they tapped 2 distinct and stable sources of variance: dynamic and symptomatic change. The categories of overall dynamic change were different from categories of change on the Global Assessment Scale. A small systematic difference was found between the categories of overall dynamic change and the categories of target complaints change also, due to false solutions of dynamic conflicts.

Posttreatment adjustment of clients who drop out early vs. late in treatment

Symptom improvement from intake until 4 months later was assessed for 33 adult and 35 child psychotherapy drop-outs. Client ratings, therapist ratings, and symptom checklist changes were used as measures. Early drop-outs (one or two visits) were less well adjusted than late drop-outs (three or more visits) for both adults and children, though the difference was stronger for adults. Few differences were found between late drop-outs and completers. It was concluded that outcome researchers should use routinely the same procedures for assessing drop-outs' and completers' outcomes and that improvement of early and late drop-outs should be distinguished.

The differential effect of psychotherapy and psychoanalysis with anaclitic and introjective patients: the Menninger Psychotherapy Research Project revisited

Analyses of the data from the Menninger Psychotherapy Research Project (MPRP) have consistently indicated little difference in the therapeutic outcome between patients seen in psychoanalysis and those seen in psychotherapy. Reanalysis of the data from the MPRP, utilizing a distinction between two broad configurations of psychopathology (Blatt, 1974, 1990a; Blatt and Shichman, 1983), however, indicates that patients whose pathology focuses primarily on disruptions of interpersonal relatedness and who use primarily avoidant defenses (anaclitic patients), and patients whose pathology focuses primarily on issues of self-definition, autonomy, and selfworth and who use primarily counteractive defenses (introjective patients) differ in their responsiveness to psychotherapy and psycho-analysis. Based on recently developed procedures for systematically evaluating the quality of object representation on the Rorschach, reanalysis of the Menninger data reveals that anaclitic patients have significantly greater positive change in psychotherapy, while introjective patients have significantly greater positive change in psychoanalysis. These statistically significant patient-by-treatment interactions are discussed in terms of their clinical implications as well as the importance of differentiating among types of patients in studies of therapeutic outcome and of therapeutic process.

Psychotherapy with depressed incarcerated felons: a comparative evaluation of treatments

Depressive syndromes are prevalent among prison inmates. By design, penal institutions attempt to punish, deter, and isolate criminals. The present study represents a beginning step in evaluating treatment interventions which may alleviate depression among incarcerated felons. 10 inmates at a large maximum-security prison were randomly assigned to group cognitive therapy or individual supportive treatment plus brief counseling contacts. Analysis indicated statistical and clinical significance on measures of depressive symptomatology for treatment conditions across the periods of assessment. The findings support the utility of depression-treatment programs for prisoners. Moreover, the treatment modalities are presented along a variety of dimensions which may be useful when considering such interventions at facilities where psychological services are limited.

Handicraft or interactional groups: a comparative outcome study of neurotic inpatients

A comparative outcome study of 2 contrasting activity-based groups with 80 hospitalized nonpsychotic patients is described. One group focused on activities designed to evoke emotional or interpersonal reactions followed by a subsequent reflection. The other focused on handicrafts and non-emotionally challenging activities. Ego strength was also measured with an instrument developed in conjunction with the study. The 2 groups were demonstrated to be consistent with the preconditions and significantly different by independent scoring of videotaped sessions. There was a greater rated therapeutic gain in the interactional group at discharge, but patient ratings did not differ between groups. There were no differences at follow-up between the groups. Measured ego strength strongly predicted outcome after correcting for the initial symptom levels. Ego strength did not interact with activity type. Clinical diagnosis did not predict differential outcome. The groups had no differential effects on specific symptom clusters or social functioning.

An examination of Mann's model of time-limited individual psychotherapy

The present study attempted a sensitive test of the dynamic sequence proposed by Mann for time-limited individual psychotherapy (TLP). Session evaluation ratings were collected from both patient and therapist. Indices were constructed that reflected the consistency of patient ratings, the consistency of therapist ratings, and the congruence between patient and therapist ratings, in each third of TLP. Measures of symptoms and social functioning were employed to differentiate cases of "good" versus "poor" treatment outcome. Therapists' adherence to the TLP technique was assessed by categorizing all interventions in each session. Fourteen patients were treated according to the TLP model. Measures of therapist behaviour indicated that treatments had been conducted in general accordance with Mann's technical recommendations. Measures of consistency and congruence failed to support the proposed TLP dynamic sequence. Analyses of mean evaluation ratings indicated a trend towards more positive evaluations as therapy progressed; "good" outcome cases were significantly more likely to evidence this pattern. Results are discussed in terms of interplay between inductive hypothesis building and empirical verification in the development of theories of psychotherapy.

Individualized measures of outcome versus standardized rating scales in evaluation of in-hospital psychiatric treatment. A methodological study

The differentiating power of two individually related measures of outcome, target complaints and psychiatrists' evaluation of individually defined treatment objectives, were studied in two models of treatment planning, one with and one without active patient participation. Target complaints were measured at admission, after 5 days and at discharge. Treatment objectives were initially defined in written treatment contracts and evaluated at discharge. These measures were compared with patients' self-reported symptoms as well as with independently rated clinical symptoms at admission and discharge. Both target complaints and psychiatrists' evaluations measured improvement during treatment. The differentiating power between the two experimental conditions was, however, much weaker than that of the symptom rating scales. The psychiatrists' evaluations were strongly influenced by state at discharge and only to a minor extent by changes during treatment. It is concluded that symptom rating scales are superior to individualized measures of outcome in studies on a general psychiatric ward, possibly because the patients are fairly homogenous concerning anxiety.

Interpersonal behavior and therapeutic progress: therapists and clients rate themselves and each other

To measure the extent to which progress in psychotherapy can be predicted from the interpersonal styles of therapists and clients, 42 outpatients and their 11 therapists rated themselves and each other on Benjamin's Structural Analysis of Social Behavior (SASB) scales. In addition, therapists rated amount of client change and therapy success-failure, based on the outcome section of the Rogers and Dymond (1954) scale, and clients used the Strupp, Fox, and Lessler (1969) questionnaire to evaluate progress-to-date. Clients also completed the Hopkins Symptom Checklist (HSCL) at the start of therapy and after they had completed at least 6 treatment sessions. Canonical analyses indicated that when optimally weighted, SASB scores can predict at least 65% of the reported progress variance, lending credence to claims that "reflexive social behavior" and "the relationship" are important determinants of certain aspects of therapeutic outcome. Consistent with previous literature, clients reported more satisfaction with therapy when they perceived their therapists as "warm and freeing," but such perceptions were not systematically related to reported symptom reduction. Unfavorable self-concept ratings by therapists and therapist ratings of clients as "helping or protecting," which suggest role reversal, were negatively associated with symptom amelioration.

The role of client satisfaction in evaluating university counseling services

Though ubiquitously used for evaluating university counseling services, client satisfaction assessment has been hampered by inadequate instrumentation. Systematic use of a short form of the Client Satisfaction Questionnaire (CSQ) in one such center over the past 5 years is described, together with strategies to ensure maximal accuracy and utilization of results. Several method factors were investigated. Optional respondent identification was not found to reduce response rate or increase reported satisfaction compared to anonymity, while a substudy obtaining very high response suggested bias from nonresponse to the routine survey (response rate = 40%) was not great. The CSQ was found to have excellent psychometric properties and many advantages for use in student service settings. Relationships between satisfaction and a number of other variables such as demographics, precounseling expectancies, problem type and severity, counselor differences, and duration of counseling are reported. Ways such findings were incorporated in service planning are briefly discussed.