Evaluating therapist adherence in motivational interviewing by comparing performance with standardized and real patients

Screening, brief intervention, and referral to treatment for pain management for veterans separating from military service: study protocol of a hybrid type 2 study testing implementation facilitation versus training-as-usual

BACKGROUND

Veterans transitioning to civilian life often have chronic pain from service-related musculoskeletal disorders (MSD) with higher risk for substance misuse. Many seek VA (Department of Veterans Affairs) compensation for MSD. Use of Screening, Brief Intervention, and Referral to Treatment for Pain Management (SBIRT-PM) by VA Post-9/11 Military2VA (M2VA) case managers presents an opportunity to engage these veterans in VA pain care and address substance misuse. Implementation facilitation might help case managers use SBIRT-PM and engage veterans in services to improve outcomes.

DESIGN

This study is a 2-cohort multisite cluster-randomized hybrid type 2 effectiveness-implementation trial. Within 2 separate cohorts of 14 VA sites each, sites will be allocated to receive an implementation strategy through the use of a constrained randomization procedure: virtual implementation facilitation or training-as-usual. Sites and M2VA case managers will receive the assigned implementation strategy to support use of SBIRT-PM. Recently discharged veterans (n = 1848) claiming service-connected MSD will be recruited, with case managers blind to veterans' study enrollment. The proportion of participants who receive any SBIRT-PM will be the primary implementation outcome. Veteran participants will complete baseline, 12-week, and 36-week assessments, irrespective of whether case managers conduct SBIRT-PM with them (intent-to-treat). Pain intensity and interference will be the primary clinical outcomes. The study emphasizes pragmatic over explanatory methodological features.

SUMMARY

This pragmatic trial will examine implementation facilitation versus training-as-usual in implementing SBIRT-PM to promote veteran engagement in nonpharmacological pain services. Using innovative methods to train and support VA case managers in SBIRT-PM, study outcomes could have broad implications for case management systems of care across the VA.

Clinician motivational interviewing skills in 'simulated' and 'real-life' consultations differ and show predictive validity for 'real life' client change talk under differing integrity thresholds

Background

Accurate and reliable assessment of clinician integrity in the delivery of empirically supported treatments is critical to effective research and training interventions. Assessment of clinician integrity can be performed through recording simulated (SI) or real-life (RL) consultations, yet research examining the equivalence of these data is in its infancy. To explore the strength of integrity assessment between SI and RL samples in Motivational Interviewing (MI) consultations, this article examines whether Motivational Interviewing Treatment Integrity (MITI) assessments differ between SI and RL consultations and reviews the predictive validity of SI and RL MI skills categorisations for RL client response language.

Methods

This study first compared MITI coding obtained in SI and RL consultations for 36 veterinary clinicians. Multilevel models of 10 MITI behaviour counts and four MITI global scores were run using MLwiN 3.02 to assess if a significant difference existed between SI and RL MITI data, with consultation within clinician within cohort (A and B) as nested random effects. Second, we investigated the effect of SI and RL MI skills groupings on rate of RL client response talk using three multivariable regression models. Two Poisson regression models, with random intercepts for farm and veterinarian and offset for number of minutes of the recordings, were estimated in the statistical software R using the package glmmTMB for the two response variables Change Talk and Sustain Talk. A logistic regression model, with the same random intercepts, with the response variable Proportion Change Talk was also estimated using the same package.

Results

Veterinary clinicians were less MI consistent in RL consultations, evidenced through significantly lower global MITI Cultivating Change Talk (p < 0.001), Partnership (p < 0.001) and Empathy (p = 0.003) measures. Despite lower objective MI skills groupings in RL consultations, ranking order of veterinary clinicians by MI skills was similar between contexts. The predictive validity of SI and RL MI skills groupings for RL client Change Talk was therefore similar, with significantly more RL client Change Talk associated with veterinarians categorised in the highest grouping ('moderate') in both SI (p = 0.01) and RL (p = 0.02) compared to untrained veterinarians in each respective context.

Conclusions

Findings suggest SI and RL data may not be interchangeable. Whilst both data offer useful insights for specific research and training purposes, differing contextual MI skills thresholds may offer a more equitable assessment of clinician RL client-facing MI integrity. Further research is needed to explore the applicability of these findings across health contexts.

You never know what you are going to get: Large-scale assessment of therapists' supportive counseling skill use

Supportive counseling skills like empathy and active listening are critical ingredients of all psychotherapies, but most research relies on client or therapist reports of the treatment process. This study utilized machine-learning models trained to evaluate counseling skills to evaluate supportive skill use in 3,917 session recordings. We analyzed overall skill use and variation in practice patterns using a series of mixed effects models. On average, therapists scored moderately high on observer-rated empathy (i.e., 3.8 out of 5), 3.3% of the therapists' utterances in a session were open questions, and 12.9% of their utterances were reflections. However, there were substantial differences in skill use across therapists as well as across clients within-therapist caseloads. These findings highlight the substantial variability in the process of counseling that clients may experience when they access psychotherapy. We discuss findings in the context of both the need for therapists to be responsive and flexible with their clients, but also potential costs related to the lack of a more uniform experience of care. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Designing Practical Motivational Interviewing Training for Mental Health Practitioners Implementing Behavioral Lifestyle Interventions: Protocol for 3 Pilot Intervention Studies

BACKGROUND

Motivational interviewing (MI) is an evidence-based, patient-centered communication method shown to be effective in helping persons with serious mental illness (SMI) to improve health behaviors. In clinical trials where study staff conducted lifestyle interventions incorporating an MI approach, cardiovascular disease (CVD) risk profiles of participants with SMI showed improvement. Given the disproportionate burden of CVD in this population, practitioners who provide somatic and mental health care to persons with SMI are ideally positioned to deliver patient-centered CVD risk reduction interventions. However, the time for MI training (traditionally 16-24 hours), follow-up feedback, and the coaching required to develop and maintain patient-centered skills are significant barriers to incorporating MI when scaling up these evidence-based practices.

OBJECTIVE

We describe the design and development of the following 2 scalable MI training approaches for community mental health practitioners: real-time brief workshops and follow-up asynchronous avatar training. These approaches are being used in 3 different pilot implementation research projects that address weight loss, smoking cessation, and CVD risk reduction in people with SMI who are a part of ALACRITY Center, a research-to-practice translation center funded by the National Institute of Mental Health.

METHODS

Clinicians and staff in community mental health clinics across Maryland were trained to deliver 3 distinct evidence-based physical health lifestyle interventions using an MI approach to persons with SMI. The real-time brief MI workshop training for ACHIEVE-D weight loss coaches was 4 hours; IMPACT smoking cessation counselors received 2-hour workshops and prescribers received 1-hour workshops; and RHYTHM CVD risk reduction program staff received 4 hours of MI. All workshop trainings occurred over videoconference. The asynchronous avatar training includes 1 common didactic instructional module for the 3 projects and 1 conversation simulation unique to each study's target behavior. Avatar training is accessible on a commercial website. We plan to assess practitioners' attitudes and beliefs about MI and evaluate the impact of the 2 MI training approaches on their MI skills 3, 6, and 12 months after training using the MI Treatment Integrity 4.2.1 coding tool and the data generated by the avatar-automated scoring system.

RESULTS

The ALACRITY Center was funded in August 2018. We have implemented the MI training for 126 practitioners who are currently delivering the 3 implementation projects. We expect the studies to be complete in May 2023.

CONCLUSIONS

This study will contribute to knowledge about the effect of brief real-time training augmented with avatar skills practice on clinician MI skills. If MI Treatment Integrity scoring shows it to be effective, brief videoconference trainings supplemented with avatar skills practice could be used to train busy community mental health practitioners to use an MI approach when implementing physical health interventions.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/44830.

Promoting Evidence-Based Tobacco Cessation Treatment in Community Mental Health Clinics: Protocol for a Pilot Study (Preprint)

BACKGROUND

Tobacco smoking is highly prevalent among persons with serious mental illness (SMI) and is the largest contributor to premature mortality in this population. Evidence-based smoking cessation therapy with medications and behavioral counseling is effective for persons with SMI, but few receive this treatment. Mental health providers have extensive experience working with clients with SMI and frequent treatment contacts, making them well positioned to deliver smoking cessation treatment. However, few mental health providers feel adequately trained to deliver this treatment, and many providers believe that smokers with SMI are not interested in quitting or have concerns about the safety of smoking cessation pharmacotherapy, despite substantial evidence to the contrary.

OBJECTIVE

We present the protocol for the pilot "IMPACT" (Implementing Action for Tobacco Smoking Cessation Treatment) study, which aims to pilot test a multicomponent implementation intervention to increase the delivery of evidence-based tobacco smoking cessation treatment in community mental health clinics.

METHODS

We are using a prepost observational design to examine the effects of an implementation intervention designed to improve mental health providers' delivery of the following four evidence-based practices related to smoking cessation treatment: (1) assessment of smoking status, (2) assessment of willingness to quit, (3) behavioral counseling, and (4) pharmacotherapy prescribing. To overcome key barriers related to providers' knowledge and self-efficacy of smoking cessation treatment, the study will leverage implementation strategies including (1) real-time and web-based training for mental health providers about evidence-based smoking cessation treatment and motivational interviewing, including an avatar practice module; (2) a tobacco smoking treatment protocol; (3) expert consultation; (4) coaching; and (5) organizational strategy meetings. We will use surveys and in-depth interviews to assess the implementation intervention's effects on providers' knowledge and self-efficacy, the mechanisms of change targeted by the intervention, as well as providers' perceptions of the acceptability, appropriateness, and feasibility of both the evidence-based practices and implementation strategies. We will use data on care delivery to assess providers' implementation of evidence-based smoking cessation practices.

RESULTS

The IMPACT study is being conducted at 5 clinic sites. More than 50 providers have been enrolled, exceeding our recruitment target. The study is ongoing.

CONCLUSIONS

In order for persons with SMI to realize the benefits of smoking cessation treatment, it is important for clinicians to implement evidence-based practices successfully. This pilot study will result in a set of training modules, implementation tools, and resources for clinicians working in community mental health clinics to address tobacco smoking with their clients. Trial Registration: ClinicalTrials.gov NCT04796961; https://clinicaltrials.gov/ct2/show/NCT04796961.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04796961; https://clinicaltrials.gov/ct2/show/NCT04796961.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/44787.

Veterinary communication can influence farmer Change Talk and can be modified following brief Motivational Interviewing training

Current veterinary communication skills training often focuses on the strategies necessary to successfully transfer information and promote shared decision making rather than inspiring client motivation to engage in behaviour change(s). One evidence-based communication methodology with a specific focus on enhancing conversations about change is Motivational Interviewing (MI), which is perceived by veterinarians to be highly relevant to their profession. We examined whether veterinarians who experienced brief (4–5 hours) MI training (BMIT) were able to change their communication behaviours to be more MI consistent. Fourteen veterinarians recorded 31 veterinary herd health consultations before (n = 15) and after (n = 16) BMIT to allow pre-post intervention analysis of veterinarian and farmer verbal behaviour. Additionally, using a sequential linguistic analysis of 3885 veterinarian-farmer communication events within these consultations, the influence of veterinarians’ verbal behaviours on farmers’ response language was explored. Analysis of veterinary consultations undertaken before and after BMIT revealed that veterinarians changed their communication style to be more consistent with the MI methodology, including more use of reflection statements, a more empathic and partnership-oriented consultation style and greater emphasis on clients’ own language in favour of change goals. In response, farmers contributed more to the conversation and discussed more herd health-related changes. Sequential linguistic analysis suggested that following a veterinarian emphasising something positive about the farmer (e.g. efforts, strengths), seeking collaboration or emphasising farmer choice, farmers were subsequently more likely to express arguments in favour of change (‘Change Talk’), especially phrases indicative of commitment. This study offers the first evidence of the potential value of a BMIT experience to enhance veterinary communication skills, although conscious and disciplined use of MI principles, strategies and Spirit–an ethos of compassion, acceptance, partnership and evocation—requires longer and more complex training. Further studies examining the longevity and consistency of these verbal behaviour changes following BMIT are required.

Dairy herd health management activities in relation to training of veterinarians in motivational interviewing

Communication has been identified as one constraint for cattle veterinarians to act as proactive health consultants. Motivational Interviewing (MI), an evidence-based client-centered communication methodology, may therefore be a useful tool in veterinary herd health management (VHHM). The aim of this exploratory study was to investigate associations between training in MI and different measures of performance in VHHM. MI skills of 36 Swedish cattle veterinarians randomly assigned to two groups - MI veterinarians (n = 18) who had received six-months training in MI and control veterinarians (n = 18) who were not trained - were assessed using the MI Treatment Integrity coding system. Skills were categorized as untrained, trained_poor, trained_near moderate and trained_moderate. Veterinarians each visited two to six cattle farms for VHHM consultations and revisited the same farms three to six months later to document implementation of recommendations. A gamma model and logistic or ordered logistic regression models were used to evaluate associations between MI training and performance measured as: a) proportion of consultations resulting in written herd health plans, b) proportion of recommended measures fully or partially implemented, c) Likert scores of different estimates of client satisfaction and d) total time during 6 months allocated to VHHM visits on cattle farms. We found no evidence of any significant effects of MI training on the eight performance variables analyzed. However, a pattern of numerically higher values (better performance) in trained veterinarians with poor or moderate MI skills than in untrained veterinarians was observed in most variables. Clients were highly satisfied with veterinarians' attitude, competence and time efficiency. Farms implemented a median of 66.7% (interquartile range: 50-100) of the measures partially or fully. Veterinarians allocated little time to VHHM visits (median: 2.2; interquartile range: 0.65-4.1 h per week of full-time work within cattle). Veterinarians with moderate skills spent 2.14 times more time on VHHM visits than untrained veterinarians, but the confidence interval was wide (0.88-5.21). The pattern of numerically higher values in trained veterinarians supported findings from other studies in the same project indicating a positive effect of MI training on VHHM. Our observations suggest a considerable potential to increase veterinarians' engagement in VHHM.

The temporal relationships between therapist adherence and patient outcomes in dialectical behavior therapy

OBJECTIVE

Although Dialectical Behavior Therapy (DBT) is a well-established evidence-based psychotherapy, little is known about the role of therapist adherence in promoting positive outcomes. This study evaluated the temporal relationships between therapist adherence to DBT and patient outcomes, as well as potential moderators of these relationships.

METHOD

Data were from six clinical trials conducted in research and community settings with a variety of patient populations. In these trials, trained observers rated 83 therapists for adherence during 1,262 DBT individual therapy sessions with 288 patients. Patient outcomes included suicide attempts, nonsuicidal self-injury (NSSI), treatment dropout, psychiatric hospitalizations, and global functioning. Longitudinal mixed-effects models evaluated the time-ordered, bidirectional relationships between adherence and outcomes.

RESULTS

Higher therapist adherence significantly predicted fewer subsequent suicide attempts (p = .002, η = 0.32) and a lower risk of dropout (p = .002, η = 0.33), and the latter relationship was strongest among patients with comorbid opioid dependence. Higher therapist adherence predicted fewer subsequent hospitalizations among community therapists (p = .001, η = 0.35) and patients that were not exclusively suicidal/self-injuring (p < .001, η = 0.41). Conversely, more frequent NSSI (p = .03, η = 0.22) and worse global functioning (p = .01, η = 0.26) predicted higher subsequent therapist adherence, and the latter relationship was moderated by patient population.

CONCLUSIONS

Therapist adherence improves several key patient outcomes and retention, highlighting the importance of delivering DBT with adherence to the manual. Therapists may find it easier to deliver DBT adherently to more severely impaired patients. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The Resistance Vignette Task: Validating a rapid measure of therapist skill at managing resistance

BACKGROUND

Therapist appropriate responsivity to client ambivalence and resistance is considered an important interpersonal skill to avoid disengagement and ensure a continued collaborative, productive process. The present study examined the predictive validity of the newly developed Resistance Vignette Task (RVT), a 10-item rapidly administered measure of therapist ability to appropriately respond to various presentations of client resistance.

METHODS

Following a resistance management workshop, the concurrent and prospective predictive capacity of RVT scores were examined through test interviews with ambivalent simulators and volunteers.

RESULTS

Prospectively, in test interviews with ambivalent interviewees, higher RVT scores immediately postworkshop were associated with significantly greater responsivity (appropriate responsivity and fewer responsivity errors) at 4-month follow-up. RVT scores at the 4-month follow-up point were also concurrently associated with significantly greater therapist responsivity and lower levels of interviewee resistance.

CONCLUSIONS

These findings provide further validation for the RVT as a measure of therapist responsivity in vivo, in actual interviews by predicting and being concurrently associated with therapist performance in response to client resistance. Thus, the RVT holds promise in advancing therapist training, as well as research on resistance as it represents an efficient measure of this key therapist skill.

Standardized Vaccine-Hesitant Patients in the Assessment of the Effectiveness of Vaccine Communication Training

OBJECTIVES

To determine if training residents in a structured communication method elicits specific behaviors in a laboratory model of interaction with vaccine-hesitant parents.

STUDY DESIGN

Standardized patients portraying vaccine-hesitant parents were used to assess the effectiveness of training in the Announce, Inquire, Mirror, Secure (AIMS) Method for Healthy Conversations. Blinded pediatric residents were pseudorandomized to receive AIMS or control training and underwent pre- and post-training encounters with blinded standardized patients. Encounters were assessed by blinded raters using a novel tool. Participant confidence and standardized patient evaluations of the participants' general communication skills were assessed.

RESULTS

Ratings were available for 27 AIMS and 26 control participants. Statistically significant increases in post-training scores (maximum = 30) were detected in AIMS, but not in control, participants (median, 21.3 [IQR, 19.8-24.8] vs 18.8 [IQR, 16.9-20.9]; P < .001). Elements (maximum score = 6) with significant increases were Inquire (0.67 [IQR, 0-1.76] vs -0.33 [IQR, -0.67 to 0.33]; P < .001); Mirror (1.33 [IQR, 0 to 2] vs -0.33 [IQR, -0.92 to 0]; P < .001) and Secure (0.33 [IQR, 0 to 1.67] vs -0.17 [IQR, -0.67 to 0.33]; P = .017). Self-confidence increased equally in both groups. Standardized patients did not detect a difference in communication skills after training and between groups. Internal consistency and inter-rater reliability of the assessment tool were modest.

CONCLUSIONS

Standardized patients proved useful in studying the effectiveness of structured communication training, but may have been limited in their ability to perceive a difference between groups owing to the predetermined encounter outcome of vaccine refusal. AIMS training should be studied in real-world scenarios to determine if it impacts vaccine acceptance.

Role-Play Assessment of Therapist Adherence and Skill in Implementation of Trauma-Focused Cognitive-Behavioral Therapy

Numerous efforts are underway to train clinicians in evidence-based practices. Unfortunately, the field has few practical measures of therapist adherence and skill with which to judge the success of these training and implementation efforts. One possible assessment method is using behavioral rehearsal, or role-play, as an analogue for therapist in-session behavior. The current study describes aspects of reliability, validity and utility of a behavioral role-play assessment developed to evaluate therapist adherence and skill in implementing Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT). TF-CBT role-play assessments were conducted with a sample of 43 therapists as part of a larger training study. The TF-CBT role-play assessments were independently coded for TF-CBT adherence and skill by a certified TF-CBT trainer and three clinical psychology doctoral students. Findings indicated good interrater reliability for the individual items (ICC: M = .71, SD = .15). Regarding utility, 67.19% (n = 43/64) of contacted therapists completed the role-play assessment, which took an average of 30 min (M = 31.42, SD = 5.65) to complete and 60 min (M = 62.84, SD = 11.31) to code. Therapists with a master's degree were more likely to complete the role-play assessment than those with other degrees but no other differences in demographic variables, practice characteristics, or TF-CBT knowledge or training were found between participants and nonparticipants. Role-play assessments may offer an alternative to observational coding for assessing therapist adherence and skill, particularly in contexts where session recordings are not feasible.

Ongoing supervision as a method to implement Motivational interviewing: Results of a randomized controlled trial

OBJECTIVE

To assess skills in Motivational interviewing (MI) at the Swedish National Board of Institutional Care (SiS), and to evaluate different ways to provide MI supervision.

METHODS

SiS practitioners (n = 134) were randomized to regular group supervision, or individual telephone supervision based on only the behavioral component of a feedback protocol, or the full protocol. Participant's mean age was 43.2 (SD =10.2), and the majority (62.7%) were females.

RESULTS

Many participants showed beginning proficiency already at baseline, indicating a successful implementation. Still, results varied widely. The regular supervision and the supervision based on objective feedback were equally effective, and the group receiving feedback based on fewer variables of the protocol performed better on only one of the seven skill measures. The objective feedback did not provoke supervisee discomfort/distress, or negatively affect the supervisory relationship.

CONCLUSIONS

Extensive MI implementation can increase practitioners' skills in MI, but the question of the best mode of ongoing supervision needs further attention.

PRACTICE IMPLICATIONS

Objective feedback does not seem to negatively affect the supervisee's skill acquisition or the supervisor-supervisee working alliance, but the question of how to most efficiently provide feedback from multifaceted feedback tools remains unanswered.

The Dialectical Behavior Therapy Adherence Coding Scale (DBT ACS): Psychometric properties

The Dialectical Behavior Therapy Adherence Coding Scale (DBT ACS) is an observer-rated measure used to evaluate the extent to which therapists deliver individual and group DBT with adherence to the manual. Despite its frequent use in clinical trials of DBT, relatively little is known about its psychometric properties. The present study utilized data from six clinical trials conducted in research and community settings with a variety of patient populations. Across these studies, the DBT ACS was used to code a total of 1,271 DBT individual therapy sessions and 180 DBT group sessions. Results indicate the DBT ACS computed global score has good internal consistency (α = .81) and excellent interrater reliability (ICC = .93). A confirmatory factor analysis found that a single factor yielded acceptable goodness of fit indices. The DBT ACS discriminated between DBT and another treatment and between research and community therapists. Across studies, variability in adherence scores was attributable more to therapists (33%) than to patients (15%). Both therapist and patient variability were higher in effectiveness than efficacy trials. Generalizability coefficients indicated that 5 sessions are needed to estimate a dependable adherence score at the patient level, whereas 9-15 sessions are needed to achieve adequate generalizability at the therapist level. Fewer sessions were needed to yield dependable scores for community therapists compared to research therapists. The DBT ACS appears to be a reliable, valid, and dependable method of assessing therapist adherence to individual and group DBT across diverse treatment settings, therapist types, and patient populations. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Standardized Patients in Clinical Psychology: From Research to Practice

Background: Within the pending reform of the German law of psychotherapy training, education in clinical psychology and the examination of psychotherapeutic competencies are established as more practice oriented. Students will acquire and demonstrate therapeutic skills through interactions with standardized patients (SPs). The aim of the current paper is to enhance evidence-based implementation of these new methods of education and examination by presenting the current evidence regarding the use of SPs and by pointing out areas for further research. Results: Results of recent studies demonstrate that SPs are able to present mental disorders authentically. Prerequisites are, among others, the selection of suitable SPs, detailed role scripts, specific training, feedback, and corrective training. Although some research questions, including the comparative effectiveness of SPs, remain unanswered, practice implications for using SPs in education, examination, and research can be drawn. These implications are illustrated schematically. Conclusions: The use of SPs has large potential for education in clinical psychology and for research on psychotherapy training. With a view to encouraging the widespread use of SPs, we provide exemplary materials (e.g., role script) within the online supplements (for all online suppl. material, see www.karger.com/doi/10.1159/000510049).

Einsatz standardisierter Patienten im Psychologiestudium: Von der Forschung in die Praxis

<b><i>Hintergrund:</i></b> Im Rahmen des reformierten Psychotherapeutengesetzes wird eine stärkere Praxisorientierung in der klinisch-psychologischen Lehre und in der Prüfung psychotherapeutischer Kompetenzen verankert. Hierbei sollen Studierende durch die Interaktion mit standardisierten Patient*innen (SP) therapeutische Kompetenzen erwerben und demonstrieren. <b><i>Fragestellung:</i></b> Das Ziel des vorliegenden Beitrags ist es, eine evidenzbasierte Umsetzung dieser neuen Lehr- und Prüfungsformate zu unterstützen, indem bisherige Forschungsbefunde zum Einsatz von SP dargestellt und Bereiche, in denen weitere Forschung notwendig ist, aufgezeigt werden. <b><i>Ergebnisse:</i></b> Empirische Befunde zeigen, dass SP psychische Störungen authentisch darstellen können. Voraussetzung dafür sind beispielsweise die Auswahl geeigneter SP, detaillierte Rollenanleitungen, spezifisches Training, Feedback und Nachschulungen. Auch wenn einige Forschungsfragen, wie zur vergleichenden Wirksamkeit des Einsatzes von SP, noch unbeantwortet sind, lassen sich praktische Implikationen für SP-Programme in Lehre, Prüfung und Forschung ableiten, die in einem Ablaufschema dargestellt werden. <b><i>Schlussfolgerungen:</i></b> Der Einsatz von SP bietet großes Potenzial für die klinisch-psychologische Lehre und Ausbildungsforschung. Um den Einsatz von SP an anderen Standorten zu unterstützen, werden Beispielmaterialien (z.B. Rollenanleitung) in den elektronischen Supplementen (siehe www.karger.com/doi/10.1159/000509249 für alle Supplemente) zum Artikel zur Verfügung gestellt.

Assessment of motivational interviewing: Psychometric characteristics of the MITS 2.1 in general practice

OBJECTIVE

Motivational Interviewing (MI) is increasingly used in healthcare. The Motivational Interviewing Target Scheme 2.1 (MITS) can be used to assess MI in short consultations. This quantitative validation study is a sequel to a qualitative study, which showed that the MITS is suitable for low-stakes assessment. We collected validity evidence to determine whether its suitability for high-stakes assessment in the GP-setting.

METHODS

Consultations of GPs and GP-trainees were assessed using the MITS. The internal structure was studied using generalizability theory; intra class correlation (ICC), convergent and divergent validity was determined.

RESULTS

Two coders and seven consultations were found to be necessary for high stakes assessment. We found higher ICCs as coders were more experienced. Convergent validity was found; results for divergent validity were mixed.

CONCLUSION

The MITS is a suitable instrument for high-stakes MI assessments in GP-setting. The number of consultations and coders that are needed for assessment are comparable to other instruments for assessing communication skills.

PRACTICE IMPLICATIONS

The MITS can be used to assess conversations for their MI consistency in GP-setting where most consultations are relatively short and are only partially dedicated to behaviour change. As the MITS assesses complex communication skills, experienced coders are needed.

Dairy veterinarians' skills in motivational interviewing are linked to client verbal behavior

Veterinarians often give advice in a persuasive form, a style that has been shown to evoke resistance to change in clients experiencing psychological ambivalence (i.e. those who see both advantages and disadvantages to changing). With this style of communication, veterinarians run the risk of counteracting their purpose to encourage clients to follow recommendations. Motivational interviewing (MI) is a client-centered communication methodology that aims to facilitate clients’ internal motivation to change. In MI, Change Talk represents clients’ own statements expressing consideration of, motivation for or commitment to behavior change and has been shown to be strongly correlated with behavior change. Sustain Talk is corresponding statements related to maintaining the status quo. The aim of this exploratory study was to evaluate the potential of MI to facilitate behavior change in veterinary herd health management (VHHM) by investigating the effect of dairy cattle veterinarians’ MI skills on client Change and Sustain Talk. We recorded VHHM consultancies on 170 Swedish cattle farms performed by 36 veterinarians, randomly distributed into 2 groups: MI veterinarians (n = 18) had received 6-month training in MI and control veterinarians (n = 18) had not received any training. Veterinarians’ MI skills were assessed using the Motivational Interviewing Treatment Integrity coding system 4.2.1 and categorized as poor_untrained, poor_trained, near moderate and moderate. Client communication was coded using the Client Language Easy Rating coding system. The effect of MI skills on Change Talk, Sustain Talk and Proportion of Change Talk(Change Talk divided by the sum of Sustain Talk plus Change Talk) was investigated using cross-classified regression models with random intercepts for veterinarian and client (farm). The models also included additional explanatory variables (e.g. type of veterinarian and client’s satisfaction with the consultation). The veterinarian’s MI skills were associated with the client’s Change Talk, but results regarding Sustain Talk or Proportion of Change Talk were inconclusive. Clients of veterinarians reaching the highest (i.e. moderate) MI skills expressed 1.5 times more Change Talk than clients of untrained veterinarians. Clients of general large animal practitioners expressed less Sustain Talk than clients of animal health veterinarians and had higher Proportion of Change Talk. Results indicate that learning to practice MI may be one means to improve adherence to veterinary recommendations and to improve efficiency in VHHM services.

Training in motivational interviewing improves cattle veterinarians' communication skills for herd health management

BACKGROUND

Communication skills to promote changes in management routines are especially important in veterinary herd health management (VHHM). Motivational interviewing (MI) is a communication methodology shown to be effective in stimulating client behaviour change. This study aimed to evaluate a 6-month MI-training programme for veterinarians.

METHODS

Thirty-eight cattle veterinarians gathered in groups of four to eight at six workshops separated by 3-4 weeks, during which they read literature and practised their skills. MI skills were evaluated before and after training using audio recordings of role-play conversations with professional actors. Recordings were coded using the MI Treatment Integrity Code (MITI) V.4.2.1. The effect of training was evaluated by 16 regression models. Participants filled in questionnaires about their experiences.

RESULTS

All participants improved their MI skills after training in at least one parameter and significant improvements were found in all but 3 of the 16 statistically evaluated MITI variables. The mean (25th-75th percentiles) ratings of the veterinarians' perceived relevance of MI skills in their work was 4.9 (4.0-6.0) and of their satisfaction with the programme was 5.1 (5.0-6.0) on a 6-point Likert scale.

CONCLUSIONS

Results show that MI training was perceived to be useful and relevant and successfully improved veterinarians' communication skills in VHHM.

The Structure of Competence: Evaluating the Factor Structure of the Cognitive Therapy Rating Scale

The Cognitive Therapy Rating Scale (CTRS) is an observer-rated measure of cognitive behavioral therapy (CBT) treatment fidelity. Although widely used, the factor structure and psychometric properties of the CTRS are not well established. Evaluating the factorial validity of the CTRS may increase its utility for training and fidelity monitoring in clinical practice and research. The current study used multilevel exploratory factor analysis to examine the factor structure of the CTRS in a large sample of therapists (n = 413) and observations (n = 1,264) from community-based CBT training. Examination of model fit and factor loadings suggested that three within-therapist factors and one between-therapist factor provided adequate fit and the most parsimonious and interpretable factor structure. The three within-therapist factors included items related to (a) session structure, (b) CBT-specific skills and techniques, and (c) therapeutic relationship skills, although three items showed some evidence of cross-loading. All items showed moderate to high loadings on the single between-therapist factor. Results support continued use of the CTRS and suggest factors that may be a relevant focus for therapists, trainers, and researchers.

Evaluation of a Level I trauma center provider training in patient-centered alcohol brief interventions using the Behavior Change Counseling Index rated by standardized patients

Background

Traumatic injury requiring hospitalization is common in the USA and frequently related to alcohol consumption. The American College of Surgeons requires that Level I and II verified trauma centers implement universal alcohol screening and brief intervention for injured patients. We examined whether Level I trauma center provider skill in patient-centered alcohol brief interventions improved after training and whether professional role (eg, nursing, social work) and education were associated with these skills.

Methods

We present evaluation data collected as part of training in alcohol brief interventions embedded within a larger clinical trial of a collaborative care intervention targeting posttraumatic stress disorder and related comorbidities. Sixty-five providers from 25 US Level I trauma centers engaged in a 1-day workshop, with 2 hours dedicated to training in patient-centered alcohol brief interventions followed by 6 months of weekly coaching in a collaborative care model. Providers completed standardized patient role-plays prior to and 6 months after the workshop training. The standardized patient actors rated provider quality of alcohol brief interventions immediately after each role-play using the Behavior Change Counseling Index (BECCI), a pragmatic measure designed to assess the quality of behavior change counseling, an adaptation of motivational interviewing suitable for brief healthcare consultations about behavior change.

Results

Seventy-two percent of providers completed both standardized patient role-play assessments. A statistically significant improvement in overall BECCI scores (t(41)=−2.53, p=0.02, Cohen’s d=−0.39) was observed among those providers with available pre–post data. Provider professional role was associated with BECCI scores at pre-training (F(3, 58)=11.25, p<0.01) and post-training (F(3, 41)=8.10, p<0.01).

Discussion

Findings underscore the need for training in patient-centered alcohol brief interventions and suggest that even a modest training helps providers engage in a more patient-centered way during a role-play assessment.

Level of evidence

Level V, therapeutic/care management.

Providing objective feedback in supervision in motivational interviewing: results from a randomized controlled trial

BACKGROUND

The effects of the use of objective feedback in supervision on the supervisory relationship and skill acquisition is unknown.

AIMS

The objective of this study was to evaluate the effects of two different types of objective feedback provided during supervision in motivational interviewing (MI) on: (a) the supervisory relationship, including potential feelings of discomfort/distress, provoked by the supervision sessions, and (b) the supervisees' skill acquisition.

METHOD

Data were obtained from a MI dissemination study conducted in five county councils across five county councils across Sweden. All 98 practitioners recorded sessions with standardized clients and were randomized to either systematic feedback based on only the behavioral component of a feedback protocol, or systematic feedback based on the entire protocol.

RESULTS

The two different ways to provide objective feedback did not negatively affect the supervisory relationship, or provoke discomfort/distress among the supervisees, and the group that received the behavioural component of the feedback protocol performed better on only two of the seven skill measures.

CONCLUSIONS

Objective feedback does not seem to negatively affect either the supervisor-supervisee working alliance or the supervisees' supervision experience. The observed differences in MI skill acquisition were small, and constructive replications are needed to ascertain the mode and complexity of feedback that optimizes practitioners' learning, while minimizing the sense of discomfort and distress.

Communication styles of Swedish veterinarians involved in dairy herd health management: A motivational interviewing perspective

Veterinary communication has evolved in the context of traditional veterinary tasks, such as diagnosing and treating sick animals, and this may encourage a directive communication style, in which the veterinarian acts as an expert, directing the client in what to do and how to do it. This type of communication style has been shown to evoke resistance to change in clients who are experiencing psychological ambivalence, a well-known aspect of farmers' herd health decisions. Veterinary herd health management (VHHM) constitutes an increasing proportion of the work of cattle veterinarians and often focuses on behavior change. The present study aimed to characterize communication styles of Swedish dairy cattle veterinarians involved in VHHM in relation to their skills in facilitating behavior change. A secondary aim was to investigate whether these skills differed among veterinarians depending on their experience in the veterinary profession or in VHHM. Audio-recorded role-play conversations (n = 123; reflecting VHHM telephone consultations) with 42 veterinarians involved in VHHM in Swedish dairy herds and audio-recorded on-farm consultations (n = 86) with 18 of those veterinarians were coded using a system developed to evaluate motivational interviewing (MI) skills. Motivational interviewing is a communication methodology aimed at facilitating clients' internal motivation to change. The MI Treatment Integrity (MITI) code identifies frequency counts of 10 verbal behaviors, and assesses 4 global variables on a Likert scale, based on 20 min of conversation. It also suggests 6 summary measurements of MI competency based on these 14 original variables. Of the 42 veterinarians, 39 also responded to a web questionnaire about their age, continuing education, and experience in the profession, in dairy herds and in VHHM. We analyzed associations between the 6 summary MITI variables from the role-play conversations and characteristics of the 39 veterinarians using logistic and linear multivariable regression models. Veterinarians in the role-play and on-farm conversations relied predominantly on giving information, questions, and persuasion in their consultation approaches. They generally did not explore the client's expectations or wishes regarding the agenda for the consultation, or acknowledge the client's right to make decisions about actions. Veterinarians gave advice without exploring the client's need for the advice or how the information was perceived. We found a significant reduction in so-called relational scores (Empathy plus Partnership) and an increase in MI-nonadherent behaviors (Persuasion plus Confront) as years of veterinary experience increased. Results showed that there is room for improvement in the communication style of veterinarians involved in VHHM. Wider literature suggests that training veterinarians in a client-centered communication methodology such as MI may increase the demand for and success of VHHM.

Increasing substance use disorder counselors' self-efficacy and skills in talking to patients about sex and HIV risk: A randomized training trial

BACKGROUND

People with substance use disorder (SUD) experience increased risk for HIV, Hepatitis C, and sexually transmitted illnesses via risky sex. This high-risk population would benefit from sexual risk reduction interventions integrated into SUD treatment. However, many SUD counselors report lack of skill or confidence in addressing sexual risk with patients.

METHODS

This study was part of a larger nested 2 × 2 factorial repeated measures design, which compared two levels of counselor training (Basic-2 h versus Enhanced-10 h plus ongoing coaching). We determined whether counselors receiving Enhanced training addressing their motivation, confidence and skills (a) increased knowledge about sexual issues; (b) increased self-efficacy to discuss sex with patients; and (c) improved skills in discussing sex as part of SUD treatment, compared with those receiving shorter information-based training. Counselors providing individual therapy at two opioid treatment programs (OTP) and two psychosocial outpatient programs in the United States were eligible. Randomization occurred after Basic training. Measures included self-report (self-efficacy and knowledge) and blinded coding of standardized patient interviews (skill).

RESULTS

Counselors receiving Enhanced training (n = 28) showed significant improvements compared to their Basic training counterparts (n = 32) in self-efficacy, use of reflections, and use of decision-making and communication strategies with standardized patients. These improvements were maintained from post-training to 3-month follow-up. No adverse effects of study participation were reported.

CONCLUSIONS

Results suggest that counselors can improve their knowledge, self-efficacy and skill related to sexual risk conversations with patients based on modest skills-based training.

Design feasibility of an automated, machine-learning based feedback system for motivational interviewing

Direct observation of psychotherapy and providing performance-based feedback is the gold-standard approach for training psychotherapists. At present, this requires experts and training human coding teams, which is slow, expensive, and labor intensive. Machine learning and speech signal processing technologies provide a way to scale up feedback in psychotherapy. We evaluated an initial proof of concept automated feedback system that generates motivational interviewing quality metrics and provides easy access to other session data (e.g., transcripts). The system automatically provides a report of session-level metrics (e.g., therapist empathy) and therapist behavior codes at the talk-turn level (e.g., reflections). We assessed usability, therapist satisfaction, perceived accuracy, and intentions to adopt. A sample of 21 novice (n = 10) or experienced (n = 11) therapists each completed a 10-min session with a standardized patient. The system received the audio from the session as input and then automatically generated feedback that therapists accessed via a web portal. All participants found the system easy to use and were satisfied with their feedback, 83% found the feedback consistent with their own perceptions of their clinical performance, and 90% reported they were likely to use the feedback in their practice. We discuss the implications of applying new technologies to evaluation of psychotherapy. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Modeling empathy as synchrony in clinician and patient vocally encoded emotional arousal: A failure to replicate

Empathy is a well-defined active ingredient in clinical encounters. To measure empathy, the current gold standard is behavioral coding (i.e., trained coders attribute overall ratings of empathy to clinician behaviors within an encounter), which is labor intensive and subject to important reliability challenges. Recently, an alternative measurement has been proposed: capturing empathy as synchrony in vocally encoded arousal, which can be measured as the mean fundamental frequency of the voice (mean F0). This method has received preliminary support by one study (Imel, Barco, et al., 2014). We aimed to replicate this study by using 2 large samples of clinical interactions (alcohol brief motivational interventions with young adults, N = 208; general practice consultations, N = 204). Audio files were segmented to identify respective speakers and mean F0 was measured using speech signal processing software. All sessions were independently rated by behavioral coders using 2 validated empathy scales. Synchrony between clinician and patient F0 was analyzed using multivariate multilevel models and compared with high and low levels of empathy derived from behavioral coding. Findings showed no support for our hypothesis that mean F0 synchrony between clinicians and patients would be higher in high-empathy sessions. This lack of replication was consistent for both clinical samples, both behavioral coding instruments, and using measures of F0 synchrony occurring at both the session-level and minute-level. We considered differences in culture and language, patients' characteristics, and setting as explanations for this failure to replicate. Further replication testing and new developments regarding measurement methods and modeling are needed. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Task-shifting to improve the reach of mental health interventions for trauma patients: findings from a pilot study of trauma nurse training in patient-centered activity scheduling for PTSD and depression

Despite high rates of posttraumatic stress disorder (PTSD) and depression among traumatically injured patients, engagement in session-based psychotherapy early after trauma is limited due to various service utilization and readiness barriers. Task-shifting brief mental health interventions to routine trauma center providers is an understudied but potentially critical part of the continuum of care. This pilot study assessed the feasibility of training trauma nurses to engage patients in patient-centered activity scheduling based on a Behavioral Activation paradigm, which is designed to counteract dysfunctional avoidance/withdrawal behavior common among patients after injury. Nurses (N = 4) and patients (N = 40) were recruited from two level II trauma centers. A portion of a one day in-person workshop included didactics, demonstrations, and experiential activities to teach brief intervention delivery. Nurses completed pre- and posttraining standardized patient role-plays prior to and two months after training, which were coded for adherence to the intervention. Nurses also completed exit interviews to assess their perspectives on the training and addressing patient mental health concerns. Findings support the feasibility of training trauma nurses in a brief mental health intervention. Task-shifting brief interventions holds promise for reaching more of the population in need of posttrauma mental health care.

A systematic review of therapist effects: A critical narrative update and refinement to review

OBJECTIVE

To review the therapist effects literature since Baldwin and Imel's (2013) review.

METHOD

Systematic literature review of three databases (PsycINFO, PubMed and Web of Science) replicating Baldwin and Imel (2013) search terms. Weighted averages of therapist effects (TEs) were calculated, and a critical narrative review of included studies conducted.

RESULTS

Twenty studies met inclusion criteria (3 RCTs; 17 practice-based) with 19 studies using multilevel modeling. TEs were found in 19 studies. The TE range for all studies was 0.2% to 29% (weighted average = 5%). For RCTs, 1%-29% (weighted average = 8.2%). For practice-based studies, 0.2-21% (weighted average = 5%). The university counseling subsample yielded a lower TE (2.4%) than in other groupings (i.e., primary care, mixed clinical settings, and specialist/focused settings). Therapist sample sizes remained lower than recommended, and few studies appeared to be designed specifically as TE studies, with too few examples of maximising the research potential of large routine patient datasets.

CONCLUSIONS

Therapist effects are a robust phenomenon although considerable heterogeneity exists across studies. Patient severity appeared related to TE size. TEs from RCTs were highly variable. Using an overall therapist effects statistic may lack precision, and TEs might be better reported separately for specific clinical settings.

Rating motivational interviewing fidelity from thin slices

Monitoring fidelity to psychosocial treatments is critical to dissemination, process and outcome research, and internal validity in efficacy trials. However, the costs required to behavior code fidelity to treatments like motivational interviewing (MI) over many therapists and sessions quickly become intractable. Coding less of a session accelerates the process, but it is not clear how much of a session must be evaluated to capture the fidelity of the entire session. The present study used a "thin slice" (Ambady & Rosenthal, 1992) paradigm to explore the degree to which variously sized thin slices of MI fidelity related to fidelity ratings for a full session. We randomly selected contiguous and noncontiguous segments of MI sessions at each whole percent of sessions (i.e., a slice consisting of 1% of session utterances, another at 2%, etc.). We then computed MI fidelity scores from these segments and calculated agreement with fidelity ratings obtained from the full session. We compared thin slice agreement with full sessions against interrater agreement and found that approximately a third of a session (9 min, 26 seconds in our sample) had sufficient agreement to approach interrater levels. These results provide a reference for researchers and clinicians to make efficient and informed use of their behavior coding resources. In addition, our results add to the behavior slicing literature, indicating that small therapist behavior samples adequately describe overall session behavior. (PsycINFO Database Record

"It's hard to argue with a computer:" Investigating Psychotherapists' Attitudes towards Automated Evaluation

We present CORE-MI, an automated evaluation and assessment system that provides feedback to mental health counselors on the quality of their care. CORE-MI is the first system of its kind for psychotherapy, and an early example of applied machine-learning in a human service context. In this paper, we describe the CORE-MI system and report on a qualitative evaluation with 21 counselors and trainees. We discuss the applicability of CORE-MI to clinical practice and explore user perceptions of surveillance, workplace misuse, and notions of objectivity, and system reliability that may apply to automated evaluation systems generally.

Counselor competence for telephone Motivation Interviewing addressing lifestyle change among Dutch older adults

Counselor competence in telephone Motivation Interviewing (MI) to change lifestyle behaviors in a primary care population was assessed using the Motivational Interviewing Treatment Integrity (MITI) rating system. Counselor behavior was evaluated by trained raters. Twenty minutes of a random sample of 336 MI sessions were coded representing 232 counselees. Ninety-four sessions (28%) were double coded to assess inter-rater agreement. The MI fidelity was examined by comparing the MI fidelity scores direction, empathy, spirit, % open questions, % complex reflections, reflections-to-questions ratio, % MI-adherent responses with the matching beginner proficiency MITI threshold. The inter-rater agreements for the MI fidelity summary scores were good (spirit, reflections-to-questions ratio), fair (empathy, % open questions, % MI-adherent responses) or poor (direction, % complex reflection). The MI fidelity scores for direction, empathy, spirit and the percentage of complex reflections exceeded the MITI threshold, but lower scores were found for the percentage of open questions, the reflections-to-questions ratio and the percentage of MI-adherent responses. In conclusion, evidence that MI was implemented was revealed. However, the inter-rater agreements scores and some fidelity scores leave room for improvement indicating that raters and counselors may need more ongoing training and feedback to achieve and maintain adequate competence. These findings apply to more complex skills (as rating complex reflections) in particular.

Benchmarking Treatment Adherence and Therapist Competence in Individual Cognitive-Behavioral Treatment for Youth Anxiety Disorders

Evidence-based treatments (EBTs) for youth are typically developed and established through studies in research settings designed to ensure treatment integrity, that is, protocol adherence and competence by therapists. An important question for implementation science is how well integrity is maintained when these EBTs are delivered in community settings. The present study investigated whether the integrity achieved by therapists in community settings achieved a benchmark set by therapists in a research setting when they delivered the same EBT-an individual cognitive-behavioral treatment (ICBT) for youth anxiety. Therapists (N = 29; 68.97% White; 13.79% male) provided ICBT to 68 youths (M age = 10.60 years, SD = 2.03; 82.35% White; 52.94% male) diagnosed with a principal anxiety disorder in research or community settings. Training and supervision protocols were the same across settings. Two independent teams of trained coders rated 744 sessions using observational instruments designed to assess ICBT adherence and competence. Both adherence and competence were higher in the research setting. Group differences in competence were consistent across treatment, but differences in adherence were most pronounced when treatment shifted to exposure, widely viewed as the most critical component of ICBT. When using the benchmarks from the research setting, therapists from the community settings fell short for indices of adherence and competence. However, given differences between therapists and clients, as well as the fact that treatment outcomes were similar across settings, our findings raise questions about whether it is appropriate to use treatment integrity benchmarks from research settings for community.

Effectiveness of training final-year undergraduate nutritionists in motivational interviewing

OBJECTIVES

To assess the efficacy of a motivational interviewing (MI) training programme on trainee nutritionists.

METHODS

A repeated measures design was applied to assess clinician behaviours in a 'helping' conversation. Participants were 32 nutrition students, assessed at baseline and one-month follow-up.

RESULTS

The training significantly reduced the use of closed questions and MI non-adherent behaviours (MINA) (P for both=<0.001). Trainees significantly increased reflections, affirmations, summaries (P for all=<0.001) and the use of open questions (P=<0.013) which are all key indicators of MI beginner-competence. The talk-time ratio of the nutritionists also changed significantly, in favour of the client which serves as an indication of MI being used effectively. There were also significant increases in 'global' scores for empathy, direction, autonomy/support, collaboration and evocation.

CONCLUSIONS

Newly trained nutritionists 1 month post-training have a consultation style which suggested positive outcomes for clients. The trainees' scores at the one month post-training assessment were verifiable as 'beginning proficiency'.

PRACTICE IMPLICATIONS

Behaviour change counselling skills for nutritionists were enhanced, at one month post-training. MI training workshops with video feedback enhances communication skills which are likely to lead to positive consultation-behaviour changes in the trainee nutritionists.

Implementing the teen marijuana check-up in schools-a study protocol

BACKGROUND

Substance misuse is now encountered in settings beyond addiction specialty care, with schools a point-of-contact for student access to behavioral health services. Marijuana is a leading impetus for adolescent treatment admissions despite declining risk perception, for which the Teen Marijuana Check-Up (TMCU)-a tailored adaptation of motivational enhancement therapy-offers an efficacious service option. To bridge the knowledge gap concerning effective and affordable technical assistance strategies for implementing empirically supported services, the described trial will test such a strategy to facilitate school-based TMCU implementation.

METHODS

A type II effectiveness/implementation hybrid trial will test a novel strategy for a TMCU purveyor to provide technical assistance on an 'as-needed' basis when triggered by a fidelity drift alarm bell, compared to resource-intensive 'gold-standard' technical assistance procedures of prior efficacy trials. Trial procedures adhere to the EPIS framework as follows: (1) initial mixed-method exploration of the involved school contexts and identification of TMCU interventionist candidates in elicitation interviews; (2) interventionist preparation via a formally evaluated training process involving a two-day workshop and sequence of three training cases; (3) post-training implementation for 24 months for which trained interventionists are randomized to 'as-needed' or 'gold-standard' technical assistance and self-referring students randomized (in 2:1 ratio) to TMCU or waitlist/control; and (4) examination of TMCU sustainment via interventionist completion of biannual outcome assessments, cost analyses, and exit interviews. Hypothesized effects include non-differential influence of the competing technical assistance methods on both TMCU fidelity and intervention effectiveness, with lesser school costs for the 'as-needed' than 'gold-standard' technical assistance and greater reduction in the frequency of marijuana use expected among TMCU-exposed students relative to those assigned to waitlist/control.

DISCUSSION

This trial-occurring in Washington state as legislative, fiscal, and sociocultural forces converge to heighten exposure of American adolescents to marijuana-related harms-is set to advance understanding of best implementation practices for this and other efficacious, school-based interventions through examination of a data-driven technical assistance method. If shown to be clinically useful and affordable, the concept of a fidelity drift alarm could be readily translated to other empirically supported services and in other health settings.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03111667 registered 7 April 2017.

The dissemination of motivational interviewing in Swedish county councils: Results of a randomized controlled trial

Objective

A significant number of Swedish practitioners are offered workshop trainings in motivational interviewing through community-based implementation programs. The objective of this randomized controlled trial was to evaluate to what extent the practitioners acquire and retain skills from additional supervision consisting of feedback based on monitoring of practice.

Materials and methods

A total of 174 practitioners in five county councils across Sweden were randomized to one of the study's two groups: 1) Regular county council workshop training, 2) Regular county council workshop training followed by six sessions of supervision. The participant’s mean age was 43.3 years, and the majority were females (88.1%).

Results

Recruiting participants proved difficult, which may have led to a biased sample of practitioners highly motivated to learn the method. Although slightly different in form and content, all the workshop trainings increased the participants’ skills to the same level. Also, consistent with previous research, the additional supervision group showed larger gains in proficiency compared to the group who received workshop training only at the six-month follow-up. However, analyses showed generally maintained levels of skills for all the participants at the follow-up assessment, and the majority of participants did not attain beginning proficiency levels at either post-workshop or follow-up.

Conclusions

The results of this study address the real-life implications of dissemination of evidence-based practices. The maintained level of elevated skills for all participants is a promising finding. However, the low interest for obtaining additional supervision among the Swedish practitioners is problematic. In addition, neither the workshop trainings nor the additional supervision, although improving skills, were sufficient for most of the participants to reach beginning proficiency levels. This raises questions regarding the most efficient form of training to attain and sustain adequate practice standards, and how to create incentive and interest among practitioners to participate in such training.

Building an Evidence Base for Effective Supervision Practices: An Analogue Experiment of Supervision to Increase EBT Fidelity

Treatments that are efficacious in research trials perform less well under routine conditions; differences in supervision may be one contributing factor. This study compared the effect of supervision using active learning techniques (e.g. role play, corrective feedback) versus "supervision as usual" on therapist cognitive restructuring fidelity, overall CBT competence, and CBT expertise. Forty therapist trainees attended a training workshop and were randomized to supervision condition. Outcomes were assessed using behavioral rehearsals pre- and immediately post-training, and after three supervision meetings. EBT knowledge, attitudes, and fidelity improved for all participants post-training, but only the SUP+ group demonstrated improvement following supervision.

Content Coding of Psychotherapy Transcripts Using Labeled Topic Models

Psychotherapy represents a broad class of medical interventions received by millions of patients each year. Unlike most medical treatments, its primary mechanisms are linguistic; i.e., the treatment relies directly on a conversation between a patient and provider. However, the evaluation of patient-provider conversation suffers from critical shortcomings, including intensive labor requirements, coder error, nonstandardized coding systems, and inability to scale up to larger data sets. To overcome these shortcomings, psychotherapy analysis needs a reliable and scalable method for summarizing the content of treatment encounters. We used a publicly available psychotherapy corpus from Alexander Street press comprising a large collection of transcripts of patient-provider conversations to compare coding performance for two machine learning methods. We used the labeled latent Dirichlet allocation (L-LDA) model to learn associations between text and codes, to predict codes in psychotherapy sessions, and to localize specific passages of within-session text representative of a session code. We compared the L-LDA model to a baseline lasso regression model using predictive accuracy and model generalizability (measured by calculating the area under the curve (AUC) from the receiver operating characteristic curve). The L-LDA model outperforms the lasso logistic regression model at predicting session-level codes with average AUC scores of 0.79, and 0.70, respectively. For fine-grained level coding, L-LDA and logistic regression are able to identify specific talk-turns representative of symptom codes. However, model performance for talk-turn identification is not yet as reliable as human coders. We conclude that the L-LDA model has the potential to be an objective, scalable method for accurate automated coding of psychotherapy sessions that perform better than comparable discriminative methods at session-level coding and can also predict fine-grained codes.

Within-Provider Variability in Motivational Interviewing Integrity for Three Years after MI Training: Does Time Heal?

This study examined variability in Motivational Interviewing (MI) integrity among 15 providers for three years following training. Data come from an effectiveness trial in which providers were trained to deliver brief single-session MI interventions. Each session was audio-recorded and coded for MI integrity using the Motivational Interviewing Treatment Integrity (MITI) 3.1.1 rating system. Within-provider variation in MI integrity was large, especially for behavior count scores (e.g., open questions, complex reflections) and only slightly smaller for global session scores of MI Spirit and Empathy. Within-provider variability was in most cases larger than between-provider variability and there was no evidence that providers improved appreciably over time. These findings raise concerns about the quality of MI being delivered in large-scale implementation efforts and have implications for the monitoring and training of higher quality MI.

Finding harmony so the music plays on: pragmatic trial design considerations to promote organizational sustainment of an empirically-supported behavior therapy

Background

Pragmatic trials of empirically-supported behavior therapies may inform clinical and policy decisions concerning therapy sustainment. This retrospective trial design paper describes and discusses pragmatic features of a hybrid type III implementation/effectiveness trial of a contingency management (CM) intervention at an opioid treatment program. Prior reporting (Hartzler et al., J Subst Abuse Treat 46:429–438, 2014; Hartzler, Subst Abuse Treat Prev Policy 10:30, 2015) notes success in recruiting program staff for voluntary participation, durable impacts of CM training on staff-level outcomes, provisional setting implementation of the intervention, documentation of clinical effectiveness, and post-trial sustainment of CM.

Methods/design

Six pragmatic design features, and both scientific and practical bases for their inclusion in the trial, are presented: (1) a collaborative intervention design process, (2) voluntary recruitment of program staff for therapy training and implementation, (3) serial training outcome assessments, with quasi-experimental staff randomization to either single or multiple baseline assessment conditions, (4) designation of a 90-day period immediately after training in which the setting implemented the intervention on a provisional basis, (5) inclusive patient eligibility for receipt of the CM intervention, and (6) designation of two staff as local implementation leaders to oversee clinical/administrative issues in provisional implementation.

Discussion

Each pragmatic trial design feature is argued to have contributed to sustainment of CM. Contributions implicate the building of setting proprietorship for the CM intervention, culling of internal staff expertise in its delivery, iterative use of assessment methods that limited setting burden, documentation of setting-specific clinical effectiveness, expanded penetration of CM among staff during provisional implementation, and promotion of setting self-reliance in the oversight of sustainable implementation procedures. It is hoped this discussion offers ideas for how to impact local clinical and policy decisions via effective behavior therapy dissemination.

Predictive Validity of Addiction Treatment Clinicians' Post-Training Contingency Management Skills for Subsequent Clinical Outcomes

In the context of a contingency management (CM) implementation/effectiveness hybrid trial, the post-training implementation domains of direct-care clinicians (N=19) were examined in relation to a targeted clinical outcome of subsequently CM-exposed clients. Clinicians' CM skillfulness, a behavioral measure of their capability to skillfully deliver the intended CM intervention, was found to be a robust and specific predictor of their subsequent client outcomes. Analyses also revealed CM skillfulness to: (1) fully mediate an association between a general therapeutic effectiveness and client outcome, (2) partially mediate an association of in-training exposure to CM and client outcome, and (3) be composed of six component clinical practice behaviors that each contributed meaningfully to this behavior fidelity index. Study findings offer preliminary evidence of the predictive validity of post-training CM skillfulness for subsequent client outcomes, and inform suggestions for the design and delivery of skills-focused CM training curricula for the addiction treatment workforce.

A Randomized Evaluation of Motivational Interviewing Training for Mandated Implementation of Alcohol Screening and Brief Intervention in Trauma Centers

The American College of Surgeons has mandated that level I and level II trauma centers implement universal alcohol screening and brief intervention (SBI) for injured patients. This study was a secondary analysis of a national, 20-hospital, cluster-randomized implementation trial focusing on practical issues of training and supervising alcohol SBI providers in motivational interviewing (MI). The purpose of this study was to examine whether real-world trauma center providers can be trained to provide higher quality counseling using MI as part of brief interventions for alcohol and whether MI skills can be maintained over time. Sites were randomly assigned to receive a 1day workshop training in MI for alcohol SBI or not, and all providers regardless of training completed up to seven standardized patient assessments of MI fidelity over 27months. Six domains on the Motivational Interviewing Treatment Integrity (MITI) coding system were assessed and compared to proficiency criteria. Providers in the intervention training group showed substantially improved MITI scores over the course of the 27-month time period. Domains that had particularly strong improvement were MI spirit and empathy; however, despite the overall improvement in the intervention group scores, expert-derived proficiency criteria were attained only for the global scores. Routine trauma center providers who receive MI training can deliver higher quality counseling in alcohol brief interventions, but may not, however, attain previously derived proficiency standards. Future implementation efforts in real-world acute care medical settings could further elucidate provider characteristics that predict training response and also strive to demonstrate that higher quality alcohol SBI implementation is associated with improved patient-level outcomes.

How to Measure Motivational Interviewing Fidelity in Randomized Controlled Trials: Practical Recommendations

Many randomized controlled trials in which motivational interviewing (MI) is a key intervention make no provision for the assessment of treatment fidelity. This methodological shortcoming makes it impossible to distinguish between high- and low-quality MI interventions, and, consequently, to know whether MI provision has contributed to any intervention effects. This article makes some practical recommendations for the collection, selection, coding and reporting of MI fidelity data, as measured using the Motivational Interviewing Treatment Integrity Code. We hope that researchers will consider these recommendations and include MI fidelity measures in future studies.