1
|
Imel ZE, Tanana MJ, Soma CS, Hull TD, Pace BT, Stanco SC, Creed TA, Moyers TB, Atkins DC. Mental Health Counseling From Conversational Content With Transformer-Based Machine Learning. JAMA Netw Open 2024; 7:e2352590. [PMID: 38252437 PMCID: PMC10804269 DOI: 10.1001/jamanetworkopen.2023.52590] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Accepted: 11/28/2023] [Indexed: 01/23/2024] Open
Abstract
Importance Use of asynchronous text-based counseling is rapidly growing as an easy-to-access approach to behavioral health care. Similar to in-person treatment, it is challenging to reliably assess as measures of process and content do not scale. Objective To use machine learning to evaluate clinical content and client-reported outcomes in a large sample of text-based counseling episodes of care. Design, Setting, and Participants In this quality improvement study, participants received text-based counseling between 2014 and 2019; data analysis was conducted from September 22, 2022, to November 28, 2023. The deidentified content of messages was retained as a part of ongoing quality assurance. Treatment was asynchronous text-based counseling via an online and mobile therapy app (Talkspace). Therapists were licensed to provide mental health treatment and were either independent contractors or employees of the product company. Participants were self-referred via online sign-up and received services via their insurance or self-pay and were assigned a diagnosis from their health care professional. Exposure All clients received counseling services from a licensed mental health clinician. Main Outcomes and Measures The primary outcomes were client engagement in counseling (number of weeks), treatment satisfaction, and changes in client symptoms, measured via the 8-item version of Patient Health Questionnaire (PHQ-8). A previously trained, transformer-based, deep learning model automatically categorized messages into types of therapist interventions and summaries of clinical content. Results The total sample included 166 644 clients treated by 4973 therapists (20 600 274 messages). Participating clients were predominantly female (75.23%), aged 26 to 35 years (55.4%), single (37.88%), earned a bachelor's degree (59.13%), and were White (61.8%). There was substantial variability in intervention use and treatment content across therapists. A series of mixed-effects regressions indicated that collectively, interventions and clinical content were associated with key outcomes: engagement (multiple R = 0.43), satisfaction (multiple R = 0.46), and change in PHQ-8 score (multiple R = 0.13). Conclusions and Relevance This quality improvement study found associations between therapist interventions, clinical content, and client-reported outcomes. Consistent with traditional forms of counseling, higher amounts of supportive counseling were associated with improved outcomes. These findings suggest that machine learning-based evaluations of content may increase the scale and specificity of psychotherapy research.
Collapse
Affiliation(s)
| | | | | | | | | | | | - Torrey A. Creed
- Beck Community Initiative, Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | | | | |
Collapse
|
2
|
Eghaneyan BH, Sanchez K, Haeny AM, Montgomery L, Lopez-Castro T, Burlew AK, Rezaeizadeh A, Killian MO. Hispanic participants in the National Institute on Drug Abuse's Clinical Trials Network: A scoping review of two decades of research. Addict Behav Rep 2020; 12:100287. [PMID: 32637563 PMCID: PMC7330873 DOI: 10.1016/j.abrep.2020.100287] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2020] [Revised: 05/10/2020] [Accepted: 05/25/2020] [Indexed: 01/08/2023] Open
Abstract
INTRODUCTION Hispanics significantly underutilize substance abuse treatment and are at greater risk for poor treatment outcomes and dropout. Two decades of research from the National Drug Abuse Treatment Clinical Trials Network (CTN) offers an opportunity to increase our understanding in how to address the disparities experienced by Hispanics in substance abuse treatment. METHODS A scoping review was utilized to determine what has been learned from the CTN about Hispanic populations with substance use disorder. A systematic search was conducted within the CTN Dissemination Library and nine databases. Potentially relevant studies were independently assessed by two reviewers for inclusion. RESULTS Twenty-four studies were included in the review. Results identified issues in measurement, characteristics of Hispanic substance use, effective interventions, and gaps for future research. Characteristics that interfere with treatment participation were also identified including low employment rates, less likelihood of having insurance, lower rates of internet access, and increased travel time to services, as were treatment issues such as high rates of alcohol and tobacco use. Effective interventions were identified; however, the effectiveness of these interventions may be limited to specific factors. CONCLUSIONS Despite efforts to improve inclusion of minority populations, Hispanics remain underrepresented in clinical trials. Future research including Hispanic populations should examine measurement equivalence and consider how cultural and historical experiences, as well as patient characteristics, influence utilization of services. Finally, more studies are needed that examine the impact of structural factors that act as barriers to treatment access and engagement and result in significant disparities in treatment outcomes.
Collapse
Affiliation(s)
- Brittany H. Eghaneyan
- School of Social Work, University of Texas at Arlington, 211 South Cooper Street, Box 19129, Arlington, TX 76019, United States
| | - Katherine Sanchez
- School of Social Work, University of Texas at Arlington, 211 South Cooper Street, Box 19129, Arlington, TX 76019, United States
- Department of Psychiatry, UT Southwestern Medical Center, 5323 Harry Hines Blvd., Dallas, TX 75390, United States
| | - Angela M. Haeny
- Yale School of Medicine, 34 Park St., New Haven, CT 06519, United States
| | - LaTrice Montgomery
- Center for Addiction Research, Department of Psychiatry and Behavioral Neuroscience, University of Cincinnati College of Medicine, 3131 Harvey Avenue, Suite 204, Cincinnati, Ohio 45221, United States
| | - Teresa Lopez-Castro
- The City College of New York, 160 Convent Ave., New York, NY 10031, United States
| | - A. Kathleen Burlew
- University of Cincinnati, 2600 Clifton Ave., Cincinnati, OH 45220, United States
| | - Afsaneh Rezaeizadeh
- Department of Psychiatry, UT Southwestern Medical Center, 5323 Harry Hines Blvd., Dallas, TX 75390, United States
| | - Michael O. Killian
- College of Social Work, Florida State University, 296 Champions Way, UCC 3407, Tallahassee, FL 32309, United States
| |
Collapse
|
3
|
Wendt DC, Gone JP. Complexities with group therapy facilitation in substance use disorder specialty treatment settings. J Subst Abuse Treat 2018; 88:9-17. [PMID: 29606227 DOI: 10.1016/j.jsat.2018.02.002] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2017] [Revised: 02/08/2018] [Accepted: 02/10/2018] [Indexed: 11/17/2022]
Abstract
In spite of increased attention to research-based interventions for substance use disorders (SUDs), a formidable research-practice gap impedes the implementation of evidence-based treatments (EBTs). An underappreciated dimension of this gap is a mismatch in treatment modality: Whereas clinical trial and implementation research has focused primarily on individual therapy, the majority of SUD specialty treatment is in group format, with open-enrolling groups being most common. This study aims to narrow this research-practice gap by exploring clinicians' perspectives on complexities with group therapy facilitation in SUD specialty treatment settings. Semi-structured interviews were conducted with 13 group clinicians from among three outpatient SUD specialty clinics-diverse in their operational structures, treatment philosophies, clientele, and services-located in the same Midwestern U.S. metropolitan area. Interview questions addressed organizational characteristics, services provided, group therapy curricula, and use of EBTs or other structured treatments. Clinicians emphasized the importance of having flexibility in facilitating groups, through built-in group processes and clinicians' own adaptions and accommodations; this flexibility was especially emphasized for the use of EBTs or manualized interventions. Clinicians also had difficulties with group facilitation generally, as evidenced by their reported difficulty in managing complex group dynamics, their limited group therapy experience and training, and their reliance on educational groups. We discuss specific strategies for improved innovation and implementation of EBTs for SUD group therapy.
Collapse
Affiliation(s)
- Dennis C Wendt
- Department of Psychology, University of Michigan, 530 Church St., Ann Arbor, MI 48109, USA.
| | - Joseph P Gone
- Department of Psychology, University of Michigan, 530 Church St., Ann Arbor, MI 48109, USA.
| |
Collapse
|
4
|
Wendt DC, Gone JP. Group Therapy for Substance Use Disorders: A Survey of Clinician Practices. JOURNAL OF GROUPS IN ADDICTION & RECOVERY 2017; 12:243-259. [PMID: 30546274 PMCID: PMC6289265 DOI: 10.1080/1556035x.2017.1348280] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
Substance use disorder (SUD) treatment is typically delivered in group format, but clinical research focuses on individual therapy. This exploratory study narrows this gap through a survey of 566 SUD group clinicians in the United States, concerning most commonly used group practices, attitudes about evidence-based treatments (EBTs), and beliefs about addiction. Clinicians reported high use of open groups, moderately high utilization of EBT practices, and moderate use of questionable practices. Clinicians' attitudes about EBTs and beliefs about addiction were correlated with the use of certain EBTs and questionable practices. Strategies for implementation of EBTs in group settings are discussed.
Collapse
|
5
|
Affiliation(s)
- Kathleen M Carroll
- Yale University School of Medicine, West Haven, Connecticut, CT 06516, USA.
| |
Collapse
|
6
|
Magill M, Walthers J, Mastroleo NR, Gaume J, Longabaugh R, Apodaca T. Therapist and client discussions of drinking and coping: a sequential analysis of therapy dialogues in three evidence-based alcohol use disorder treatments. Addiction 2016; 111:1011-20. [PMID: 26780564 PMCID: PMC4861699 DOI: 10.1111/add.13313] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/27/2015] [Revised: 08/31/2015] [Accepted: 01/14/2016] [Indexed: 11/27/2022]
Abstract
BACKGROUND AND AIMS Research into the active ingredients of behavioral interventions for alcohol use disorders (AUD) has focused upon treatment-specific factors, often yielding disappointing results. The present study examines common factors of change in motivational enhancement therapy, cognitive-behavioral therapy and 12-Step facilitation therapy by (1) estimating transitional probabilities between therapist behaviors and subsequent client Change (CT) and Sustain (ST) Talk and (2) examining therapist skillfulness as a potential predictor of transition probability magnitude. DESIGN Secondary data analysis examined temporal associations in therapy dialogues. SETTING United States: data were from Project MATCH (Matching Alcoholism Treatments to Client Homogeneity) (1997). PARTICIPANTS One hundred and twenty-six participants who received motivational enhancement therapy, cognitive-behavioral therapy or 12-Step facilitation therapy. MEASUREMENTS Therapist behaviors were measured in three categories (exploring, teaching, connecting) and client statements included five categories (CT-distal, ST-distal, CT-proximal, ST-proximal, neutral). Therapist skillfulness was measured using a five-point ordinal scale. FINDINGS Relative to chance, therapist exploratory behaviors predicted subsequent client discussion of distal, drinking behavior [odds ratio (OR) = 1.37-1.78, P < 0.001] while suppressing discussion of proximal coping and neutral content (OR = 0.83-0.90, P < 0.01). Unexpectedly, therapist teaching suppressed distal drinking language (OR = 0.48-0.53, P < 0.001) and predicted neutral content (OR = 1.45, P < 0.001). Connecting behaviors increased both drinking and coping language, particularly language in favor of change (CT OR = 1.15-1.84, P < 0.001). Analyses of exploring and connecting skillfulness revealed that high skillfulness maximized these behaviors effect on client responses, but not teaching skillfulness. CONCLUSIONS In motivational enhancement therapy, cognitive-behavioral therapy, and 12-Step facilitation therapy for alcohol use disorders, the therapists who explore and connect with clients appear to be more successful at eliciting discussion about change than therapists who engage in teaching behavior. Therapists who are more skilled achieve better results than those who are less skilled.
Collapse
Affiliation(s)
- M. Magill
- Center for Alcohol and Addiction Studies, Brown University, Providence, RI, USA
| | - J. Walthers
- Center for Alcohol and Addiction Studies, Brown University, Providence, RI, USA
| | - NR Mastroleo
- Center for Alcohol and Addiction Studies, Brown University, Providence, RI, USA
| | - J. Gaume
- Center for Alcohol and Addiction Studies, Brown University, Providence, RI, USA,Lausanne University Hospital, Lausanne, Switzerland
| | - R. Longabaugh
- Center for Alcohol and Addiction Studies, Brown University, Providence, RI, USA
| | - T.R. Apodaca
- Children’s Mercy Kansas City, MO, USA,University of Missouri - Kansas City School of Medicine, MO, USA
| |
Collapse
|
7
|
Miller WR, Rollnick S. The effectiveness and ineffectiveness of complex behavioral interventions: Impact of treatment fidelity. Contemp Clin Trials 2014; 37:234-41. [DOI: 10.1016/j.cct.2014.01.005] [Citation(s) in RCA: 149] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2013] [Revised: 01/13/2014] [Accepted: 01/18/2014] [Indexed: 12/26/2022]
|
8
|
Measurement equivalence of the Revised Helping Alliance Questionnaire across African American and non-Latino White substance using adult outpatients. J Subst Abuse Treat 2013; 45:173-8. [PMID: 23522849 DOI: 10.1016/j.jsat.2013.02.002] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2012] [Revised: 02/03/2013] [Accepted: 02/13/2013] [Indexed: 11/23/2022]
Abstract
Analyses of the effectiveness of substance abuse treatments across racial/ethnic groups should ensure that outcome measures have the same conceptual meaning (i.e., measurement equivalence) across groups. Because racial groups differ in perceptions and experiences of the therapeutic alliance, this study investigated measurement equivalence properties of the Revised Helping Alliance Questionnaire (HAq-II) across racial groups. The sample included 138 African American and 133 non-Latino White participants, age 18-64 years, who participated in a randomized clinical trial investigating the effectiveness of Motivational Enhancement Therapy in the National Institute on Drug Abuse Clinical Trials Network. Results demonstrated configural invariance and two forms of metric invariance (weak and strong/scalar), suggesting that conceptualizations of therapeutic alliance and overall levels of endorsement of therapeutic alliance are comparable across racial groups. The groups indicated partial, strict metric nonequivalence. No studies to date reported measurement equivalence properties of the HAq-II. Findings support valid measurement and interpretation of HAq-II outcomes.
Collapse
|
9
|
Brocato J. The Impact of Acculturation, Motivation and the Therapeutic Alliance on Treatment Retention and Outcomes for Hispanic Drug Involved Probationers. JOURNAL OF ETHNICITY IN CRIMINAL JUSTICE 2013; 11:150-180. [PMID: 23976877 PMCID: PMC3746999 DOI: 10.1080/15377938.2012.756845] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
Factors associated with retention and outcomes for Hispanic offenders mandated to treatment for substance use disorders have been overlooked in the literature resulting in an impediment to providing evidence-based, culturally relevant treatment services. This project examined the roles of motivational factors, the therapeutic relationship, and acculturation in predicting treatment retention and recidivism among Hispanic male probationers mandated to residential treatment. By following a treatment cohort over one hundred and twenty days, this research identifies factors that may be targeted to improve interventions and policies. The following conclusions are supported: among Hispanic offenders, the number of days in treatment is positively related to motivation to change and level of acculturation.
Collapse
Affiliation(s)
- Jo Brocato
- School of Social Work California State University Long Beach, 1250 Bellflower Boulevard, SPAA 141, Long Beach, CA 92804.
| |
Collapse
|
10
|
|
11
|
Carroll KM, Ball SA, Jackson R, Martino S, Petry NM, Stitzer ML, Wells EA, Weiss RD. Ten take home lessons from the first 10 years of the CTN and 10 recommendations for the future. THE AMERICAN JOURNAL OF DRUG AND ALCOHOL ABUSE 2011; 37:275-82. [PMID: 21854269 PMCID: PMC3232679 DOI: 10.3109/00952990.2011.596978] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
BACKGROUND The first 10 years of the National Institute on Drug Abuse's Clinical Trials Network (CTN) yielded a wealth of data on the effectiveness of a number of behavioral, pharmacological, and combined approaches in community-based settings. METHODS We summarize some of the methodological contributions and lessons learned from the behavioral trials conducted during its first ten years, including the capacity and enormous potential of this national research infrastructure. RESULTS The CTN made contributions to the methodology of effectiveness research; new insights from secondary analyses; the extent to which approaches with strong evidence bases, such as contingency management, extend their effectiveness to real world clinical settings; new data on 'standard treatment' as actually practiced in community programs, the extent to which retention remains a major issue in the field; important data on the safety of specific behavioral therapies for addiction; and heightened the importance of continued sustained attention to bridging the gap between treatment and research. CONCLUSIONS Areas of focus for the CTN's future include defining common outcome measures to be used in treatment outcome studies for illicit drugs; incorporating performance indicators and measures of clinical significance; conducting comparative outcome studies; contributing to the understanding of effective treatments of comorbidity; reaching underserved populations; building implementation science; understanding long-term outcomes of current treatments and sustaining treatment effects; and conducting future trials more efficiently.
Collapse
Affiliation(s)
- Kathleen M Carroll
- Division of Substance Abuse, Department of Psychiatry, Yale University School of Medicine, West Haven, CT, USA.
| | | | | | | | | | | | | | | |
Collapse
|