Unintended effects of training on clinicians' interest, confidence, and commitment in using motivational interviewing

Relationship between clinician-level attributes and implementation outcomes from the Pathways to Comorbidity Care training program

BACKGROUND

The process of determining the best strategy for increasing the uptake of evidence-based practice might be improved through an understanding of relevant clinician-level factors. The Pathways to Comorbidity Care (PCC) training program (Louie E, et al., J Dual Diagnosis 17:304-12, 2021) aimed to facilitate integrated management of comorbid drug and alcohol and mental disorders amongst drug and alcohol clinicians. We hypothesised that uptake of integrated management of comorbidity following the implementation of the PCC program would be associated with clinician-level: (i) demographics (gender, education, experience), (ii) attitudes (evidence-based practice, therapist manuals, counselling self-efficacy), and (iii) organisational readiness to change.

METHODS

Twenty clinicians participated in the 9-month PCC training program. Attitudes towards evidence-based practices and psychotherapist manuals, self-efficacy, and organisational readiness to change, along with demographics, were measured at baseline. At follow-up, change in Comorbidity Practice (CoP) scores related to integrated comorbidity management were obtained using a file audit checklist and categorised into high (at least 60% increase in CoP), medium or low (a decrease of - 20% or less in CoP). Clinician-level characteristics were examined across the implementation categories.

RESULTS

There were no significant differences found between implementation groups on sociodemographic variables (p's > 0.30), attitudes to evidence-based practices, attitudes to therapist manuals, and self-efficacy (p's > 0.52). The high implementation group demonstrated significantly higher scores on leadership practices aspect of organisational readiness to change relative to the low and medium implementation group ((F(2, 16) = 3.63, p = 0.05; Cohen's d = .31) but not on the other subscales (p's > 0.07).

CONCLUSIONS

Confidence that leadership will play a positive role in the implementation process may improve effectiveness of comorbidity training programs for drug and alcohol clinicians. On the other hand, contrary to our hypothesis, counselling self-efficacy, evidence-based practice attitudes, attitudes towards therapist manuals, gender, education and experience were not distinguishing factors.

A systematic review of evidence-based practice implementation in drug and alcohol settings: applying the consolidated framework for implementation research framework

Background

There is a paucity of translational research programmes to improve implementation of evidence-based care in drug and alcohol settings. This systematic review aimed to provide a synthesis and evaluation of the effectiveness of implementation programmes of treatment for patients with drug and alcohol problems using the Consolidated Framework for Implementation Research (CFIR).

Methods

A comprehensive systematic review was conducted using five online databases (from inception onwards). Eligible studies included clinical trials and observational studies evaluating strategies used to implement evidence-based psychosocial treatments for alcohol and substance use disorders. Extracted data were qualitatively synthesised for common themes according to the CFIR. Primary outcomes included the implementation, service system or clinical practice. Risk of bias of individual studies was appraised using appropriate tools. A protocol was registered with (PROSPERO) (CRD42019123812) and published previously (Louie et al. Systematic 9:2020).

Results

Of the 2965 references identified, twenty studies were included in this review. Implementation research has employed a wide range of strategies to train clinicians in a few key evidence-based approaches to treatment. Implementation strategies were informed by a range of theories, with only two studies using an implementation framework (Baer et al. J Substance Abuse Treatment 37:191-202, 2009) used Context-Tailored Training and Helseth et al. J Substance Abuse Treatment 95:26-34, 2018) used the CFIR). Thirty of the 36 subdomains of the CFIR were evaluated by included studies, but the majority were concerned with the Characteristics of Individuals domain (75%), with less than half measuring Intervention Characteristics (45%) and Inner Setting constructs (25%), and only one study measuring the Outer Setting and Process domains. The most common primary outcome was the effectiveness of implementation strategies on treatment fidelity. Although several studies found clinician characteristics influenced the implementation outcome (40%) and many obtained clinical outcomes (40%), only five studies measured service system outcomes and only four studies evaluated the implementation.

Conclusions

While research has begun to accumulate in domains such as Characteristics of Individuals and Intervention Characteristics (e.g. education, beliefs and attitudes and organisational openness to new techniques), this review has identified significant gaps in the remaining CFIR domains including organisational factors, external forces and factors related to the process of the implementation itself. Findings of the review highlight important areas for future research and the utility of applying comprehensive implementation frameworks.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01090-7.

Therapist-Observer Concordance in Ratings of EBP Strategy Delivery: Challenges and Targeted Directions in Pursuing Pragmatic Measurement in Children's Mental Health Services

Pragmatic measures of therapist delivery of evidence-based practice (EBP) are critical to assessing the impact of large-scale, multiple EBP implementation efforts. As an initial step in the development of pragmatic measurement, the current study examined the concordance between therapist and observer ratings of items assessing delivery of EBP strategies considered essential for common child EBP targets. Possible EBP-, session-, and therapist-levels factors associated with concordance were also explored. Therapists and independent observers rated the extensiveness of therapist (n = 103) EBP strategy delivery in 680 community psychotherapy sessions in which six EBPs were used. Concordance between therapist- and observer-report of the extensiveness of therapist EBP strategy use was at least fair (ICC ≥ .40) for approximately half of the items. Greater therapist-observer concordance was observed in sessions where a structured EBP was delivered and in sessions where therapists reported being able to carry out planned activities. Findings highlighted conditions that may improve or hinder therapists' ability to report on their own EBP strategy delivery in a way that is consistent with independent observers. These results can help inform the development of pragmatic therapist-report measures of EBP strategy delivery and implementation efforts more broadly.

Implementing Motivational Interviewing for Substance Misuse on Medical Inpatient Units: a Randomized Controlled Trial

BACKGROUND

General medical hospitals provide care for a disproportionate share of patients who misuse substances. Hospitalization provides a unique opportunity to identify and motivate patients to address their substance misuse.

OBJECTIVE

To determine the effectiveness of three strategies for implementing motivational interviewing for substance misuse with general medical inpatients.

DESIGN

Type 3 hybrid effectiveness-implementation randomized controlled trial (Clinical Trials.gov: NCT01825057).

PARTICIPANTS

Thirty-eight providers (physicians, physician assistants, nurses) from 13 general medical inpatient services, and 1173 of their patients admitted to an academically affiliated acute care hospital.

INTERVENTIONS

Implementation strategies included (1) a continuing medical education workshop on detection of substance misuse and provision of a motivational interview; (2) workshop plus bedside supervision (apprenticeship condition); and (3) a workshop plus ability to place a medical order for an interview from a consultation-liaison service (consult condition).

MAIN MEASURES

Primary outcomes were the percentage of study-eligible patients who received an interview for substance misuse and the integrity (adherence, competence) of the interviews. The secondary outcome was the percent of patient statements within the interviews that indicated motivation for reducing substance misuse.

KEY RESULTS

20.5% of patients in the consult condition received an interview, compared to 0.8% (Hedge's g = 1.49) and 3.0% (Hedge's g = 1.26) in the respective workshop only and apprenticeship conditions (p < 0.001). Motivational interviews in the consult condition were performed with more fundamental motivational interviewing adherence and competence than the other conditions. Most statements made by patients during the interviews favored reducing substance misuse, with no differences between conditions.

CONCLUSIONS

Providers' ability to place an order to have experts from the consultation-liaison service deliver a motivational interview was a more effective implementation strategy than a workshop or apprenticeship method for ensuring motivational interviewing is available to medical inpatients who misuse substances.

TRIAL REGISTRY

NCT01825057.

Effectiveness of Motivational Interviewing Training for Primary Care Dentists and Dental Health Technicians: Results from a Community Clinical Trial

Motivational interviewing (MI) is an approach that uses dialogue about behavioral change to encourage a constructive relationship between professionals and patients. The aim of this study was to evaluate the maintenance of basic MI skills in the daily practice of dentists and oral health technicians (OHTs) after a training course and for two years of follow-up in the context of primary health care (PHC). A randomized community trial, from September 2012 to September 2014, was conducted at the Community Health Service of Conceição Hospital Group in Porto Alegre, RS, Brazil. The experimental group consisted of all 41 dentists and OHTs who received an intensive eight-hour training course in basic MI principles; the control group consisted of 31 dentists and OHTs who did not receive MI training. The follow-up assessments were at one and two years using three instruments validated for MI. In the one- and two-year follow-ups, improvement was maintained in responses with a statistically significant difference for use of open questioning, reflective listening, and total percentage of correct answers (p<0.001), with a large effect size (ES=1.12). On the Helpful Responses Questionnaire, the participants continued using open questioning and reflective listening (p<0.001), maintaining an increase in the percentage of responses compatible with MI (p<0.001). Likewise, the effect size remained large (ES=1.33) over time. These results suggest that the training course with dentists and OHTs of the PHC oral health staff was effective over the two-year follow-up in enabling them to act in the spirit and techniques of MI.

Entrevista Motivacional para Estudantes da Área da Saúde: Resultados de um Treinamento

Resumo Este estudo procurou avaliar a efetividade de um treinamento em Entrevista Motivacional (EM) para estudantes do último ano de diferentes cursos na área da saúde. Usando um delineamento pré e pós-teste quase-experimental, 60 estudantes receberam um treinamento de oito horas de habilidades básicas da Entrevista Motivacional. As habilidades foram avaliadas por meio de uma vinheta clínica, das Réguas de Confiança e Importância e do Questionário de Resposta Útil, codificado utilizando o Motivational Interviewing Skill Code. O treinamento foi associado com melhora no uso das habilidades básicas da Entrevista Motivacional. Os alunos também aumentaram a percepção da importância em utilizar essas habilidades com os pacientes, mesmo com um treinamento breve.

Motivational interviewing training of substance use treatment professionals: A systematic review

Background: Through evaluations of training programs, systematic reviews, and meta-analyses, advances in identifying best practices for disseminating motivational interviewing (MI) have emerged. To advance this work further, inclusion of thorough descriptions of the following is needed in research publications: study (design, trainee characteristics, setting characteristics), training and coaching methods (if applicable), trainer qualifications, and evaluation of MI skills. Methods: The purpose of this study was to systematically evaluate the research on MI training of substance use treatment professionals for the inclusion of such descriptions. Twenty-five studies were reviewed using a scoring rubric developed by the authors. Results: Just over two thirds of the studies (68%) were randomized controlled trials of MI training. The majority of studies provided information about (a) trainee characteristics (professional background = 76%, education = 60%, experience = 56%); (b) setting characteristics (80%); (c) training methods (format = 96%, length = 92%); (d) coaching (76%); and (e) evaluation of MI skills (92%). Conclusion: Findings suggest advancements in MI training studies since previous reviews, especially in regards to the inclusion of feedback and coaching. However, this review also found that inconsistencies in methods and reporting of training characteristics, as well as limited follow-up assessment of trainees' skill, continue to limit knowledge of effective training methods.

Assessment of the relationship between a written measure of empathy and an independently rated interview of Motivational Interviewing

Motivational Interviewing (MI) is an evidence-based practice shown to be effective when working with people in treatment for substance use disorders. However, MI is a complex treatment modality optimized by training with feedback. Feedback, assessment and monitoring of treatment fidelity require measurement, which is typically done using audiotaped sessions. The gold standard for such measurement of MI skill has been an audiotaped interview, scored by a rater with a detailed structured instrument such as the Motivational Interviewing Treatment Integrity 2.0 (MITI 2.0) Coding System (Moyers, et al., 2005). The Helpful Responses Questionnaire (HRQ) (Miller, Hedrick, & Orlofsky, 1991) is a pen-and-paper test of empathy (a foundational MI skill) that does not require an audiotaped session. A randomized trial of three different regimens for training counselors in MI (live supervision using Teleconferencing, Tape-based supervision and Workshop only) (Smith et al., 2012) offered the opportunity to evaluate the performance of the HRQ as a measure of MI ability, compared to the several MITI 2.0 global scores and subscales. Participants were counselors (N=97) working at community-based substance use treatment programs, whose MI proficiency was measured at four time points: baseline (before an initial 2-day MI workshop), post-workshop, 8weeks post-workshop (i.e., post-supervision), and 20weeks post-workshop with both MITI 2.0 and HRQ. HRQ total scores correlated significantly with the Reflection to Question Ratio from the MITI 2.0 at post-workshop (r=0.33), week 8 (r=0.34), and week 20 (r=0.38), and with the Spirit (r=0.32) and Empathy (r=0.32) global scores at week 20. Correlations of HRQ with other MITI 2.0 subscales and time points after workshop were small and not significant. As predicted, HRQ scores differed between training conditions (X²(2)=7.88, p=0.02), with counselors assigned to live supervision achieving better HRQ scores than those in Workshop only. In summary, HRQ is a modestly accurate measure, mainly of the Reflection to Question Ratio, considered a core marker of MI skill. It is sensitive to training effects and may help identify counselors needing more intensive supervision. Given its ease of administration and scoring, HRQ may be a useful marker of MI skill during training efforts.

Predisposition for Empathy, Intercultural Sensitivity, and Intentions for Using Motivational Interviewing in First Year Pharmacy Students

Objective. To assess first-year pharmacy (P1) students' predispositions (eg, perceptions for empathy, intercultural sensitivity, and motivational interviewing (MI) as a patient-centered communication skillset) and identify potential curricula content/communication skills training needs. Methods. A cross-sectional survey was used to collect students' self-reported perceptions for empathy, intercultural sensitivity, counseling contexts, and projected future MI use. Relationships between variables were explored and logistic regression was used to evaluate intention for using MI in future patient encounters. Results. There were 134 students who participated. Higher predisposition for empathy and for intercultural sensitivity were significantly correlated. Significant predictors for applying MI in future patient encounters were sex, confidence with counseling skills, and current use of MI. Conclusion. Results suggest the need to incorporate innovative training strategies in communication skills curricula. Potential areas include empathy, intercultural sensitivity and significant predictor variables for future MI use. Further investigation in other schools is needed.

Self-Report After Randomly Assigned Supervision Does not Predict Ability to Practice Motivational Interviewing

The objective of this study was to investigate the relation between self-report and objective assessment of motivational interviewing (MI) skills following training and supervision. After an MI workshop, 96 clinicians from 26 community programs (age 21-68, 65% female, 40.8% Black, 29.6% Caucasian, 24.5% Hispanic, 2.0% Asian, 3.1% other) were randomized to supervision (tele-conferencing or tape-based), or workshop only. At four time points, trainees completed a self-report of MI skill, using items from the MI understanding questionnaire (MIU), and were objectively assessed by raters using the Motivational Interviewing Treatment Integrity (MITI) system. Correlations were calculated between MIU and MITI scores. A generalized linear mixed model was tested on MIU scores, with MITI scores, supervision condition and time as independent variables. MIU scores increased from pre-workshop (mean = 4.74, SD = 1.79) to post-workshop (mean = 6.31, SD = 1.03) (t = 8.69, p < .0001). With supervision, scores continued to increase, from post-workshop to week 8 (mean = 7.07, SD = 0.91, t = 5.60, p < .0001) and from week 8 to week 20 (mean = 7.28, SD = 0.94, t = 2.43, p = .02). However, MIU scores did not significantly correlate with MITI scores, with or without supervision. Self-reported ability increased with supervision, but self-report was not an indicator of objectively measured skill. This suggests that training does not increase correspondence between self-report and objective assessment, so community treatment programs should not rely on clinician self-report to assess the need for ongoing training and supervision and it may be necessary to train clinicians to accurately assess their own skill.

Modeling the Innovation-Decision Process: Dissemination and Adoption of a Motivational Interviewing Preparatory Procedure In Addiction Outpatient Clinics

Widespread adoption of empirically-supported treatment innovations has the potential to improve effectiveness of treatment received by individuals with substance use disorders. However, the process of disseminating such innovations has been complex, slow, and difficult. We empirically describe the dissemination and adoption of a treatment innovation--an alcohol-treatment preparatory therapeutic procedure based on motivational interviewing (MI)--in the context of Rogers' (2003) five stages of innovation-decision process (knowledge, persuasion, decision, implementation and confirmation). To this end, 145 randomly-chosen outpatient addiction treatment clinics in New York State received an onsite visit from a project trainer delivering one of three randomly-assigned dissemination intensities: a 15-minute, a half-day or a full-day presentation. Across these clinics, 141 primary administrators and 837 clinicians completed questionnaires assessing aspects of five innovation-decision stages. At each clinic, questionnaire administration occurred immediately pre- and post-dissemination, as well as 1 and 6 months after dissemination. Consistent with Rogers' theory, earlier stages of the innovation-decision process predicted later stages. As hypothesized, dissemination intensity predicted clinicians' post-dissemination knowledge. Clinician baseline characteristics (including gender, pre-dissemination knowledge regarding the MI preparatory technique, education, case load, beliefs regarding the nature of alcohol problems, and beliefs and behavior with regard to therapeutic style) predicted knowledge and persuasion stage variables. One baseline clinic characteristic (i.e., clinic mean beliefs and behavior regarding an MI-consistent therapeutic style) predicted implementation stage variables. Findings suggest that dissemination strategies should accommodate clinician and clinic characteristics.