Using progress feedback to improve outcomes and reduce drop-out, treatment duration, and deterioration: A multilevel meta-analysis

Study protocol for a triple-blind randomised controlled trial evaluating a machine learning-based predictive clinical decision support tool for internet-delivered cognitive behaviour therapy (ICBT) for depression and anxiety

Introduction

Therapist-supported internet-based Cognitive Behavioural Therapy (ICBT) has strong scientific support, but all patients are not helped, and further improvements are needed. Personalized medicine could enhance ICBT. One promising approach uses a Machine learning (ML) based predictive decision support tool (DST) to help therapists identify patients at risk of treatment failure and adjust their treatments accordingly. ICBT is a suitable clinical context for developing and testing such predictive DST's, since its delivery is quite flexible and can quickly be adapted for probable non-responders, for example by increasing the level and nature of therapist support, to avoid treatment failures and improve overall outcomes. This type of strategy has never been tested in a triple-blind randomised controlled trial (RCT) and has rarely been studied in ICBT.The aim of this protocol is to expand on previous registered protocols with more detailed descriptions of methods and analyses before analyses is being conducted.

Methods and analysis

A triple blind RCT comparing ICBT with a DST (DST condition), to ICBT as usual (TAU condition). The primary objective is to evaluate if the DST condition is superior to the TAU condition in decreasing diagnose-specific symptoms among patients identified to be at risk of failure. Secondary objectives are to evaluate if the DST improves functioning, interaction, adherence, patient satisfaction, and therapist time efficiency and decreases the number of failed treatments. Additionally, we will investigate the therapists' experience of using the DST.Patients and therapists have been recruited nationally. They were randomised and given a sham rationale for the trial to ensure allocation blindness. The total number of patients included was 401, and assessments were administered pre-treatment, weekly during treatment, at post-treatment and at 12-month follow-up. Primary outcome is one of the three diagnosis-specific symptom rating scales for respective treatment and primary analysis is difference in change from pre- to post-treatment for at-risk patients on these scales.

Human ethics and consent to participate

Informed consent to participate in the study was obtained from all participants. Both therapists and patients are participants in this trial. For patients, informed consent to participate in the study was obtained when they registered interest for the study via the study's secure web platform and carried out initial screening before the diagnostic and fit for treatment assessment, they first received the research subject information and were asked for consent by digitally signing that they had read and understood the information. For therapists who were part of the study, consent was requested after they had registered their interest. Therapists then received an email with a link to the study's secure web platform with the research person's information and were asked for consent by digitally signing that they had read and understood the information. All documents are stored in secure, locked filing cabinets on the clinic's premises or on a secure digital consent database.

Approval committee

Approved by the Swedish Ethical Review Authority (SERA), record number 2020-05772.

Adapting Measurement-Based Care to VA Home-Based Primary Care Mental Health Treatment: A Quality Improvement Project

OBJECTIVES

This quality improvement project sought to develop guidance for Home-Based Primary Care (HPBC) Mental Health (MH) clinicians on integrating Measurement-Based Care (MBC) into their practice and gain participating psychologists' feedback on their experience using MBC for treating mental health concerns with HBPC Veterans.

METHODS

Based on feedback from the HBPC MH community and in consultation with national leadership, a workgroup of HBPC psychologists developed a guide tailoring MBC to HBPC Veterans. Eight HBPC psychologists piloted the adapted MBC approach with 53 Veterans. Participating psychologists provided feedback on measure administration, Veterans' responses to MBC, and perceived benefits and challenges.

RESULTS

Pilot participants' feedback suggested that MBC can be a highly useful tool for delivering mental health services in HBPC, although feedback varied about specific MBC measures. Qualitative feedback was primarily positive, but participants noted challenges based on the nature of the presenting problem and Veteran-specific characteristics.

CONCLUSIONS

Findings indicate that MBC can be utilized with appropriate HBPC Veterans and has the potential to benefit care. Further research is needed to clarify factors that enhance or reduce MBC's utility within HBPC.

CLINICAL IMPLICATIONS

HBPC MH providers identified MBC as a useful tool particularly when adapted to meet the needs of HBPC Veterans.

Adolescent Client and Clinician Predictors of Measurement-Based Care Fidelity in Community Mental Health Settings

OBJECTIVE

Regularly administering outcome measures, measurement-based care (MBC), informs clinical decision-making and improves youth mental health. Understanding predictors of high-fidelity MBC delivery helps ensure all youth can benefit from this evidence-based practice. Research on client and clinician predictors of MBC fidelity has mixed findings.

METHOD

Participants included 53 clinicians and 115 adolescents from a randomized controlled effectiveness trial with MBC only and MBC + Unified Protocol for the Transdiagnostic Treatment of Emotional Disorders in Adolescents (UPA) conditions. Clients and clinicians reported demographic information. Clinicians completed broad- and narrow-band attitude measures. MBC fidelity was operationalized using the implementation index, combining rates of administering and viewing questionnaires, using objective MBC data.

RESULTS AND DISCUSSION

The average MBC fidelity was 56.75% (SD = 30.36) and was significantly higher in the MBC only condition (M = 67.46%, SD = 27.63%) than in MBC + UPA condition (M = 46.58%, SD = 29.52%; p = .012). Clients identifying as White received significantly higher MBC fidelity than youth not identifying as White. Youth ethnicity, family income, age, severity, and number of sessions did not significantly predict MBC fidelity. Clinicians with less experience at their agency, less confidence, and who found manualized treatments less appealing, had significantly higher MBC fidelity than other clinicians. These results can inform future efforts to increase fidelity and equity in MBC delivery.

Bridging the feedback implementation gap: A comparison of empirical and rational decision rules in naturalistic psychotherapy

Objective: Previous research indicates positive effects of feedback based on rational or empirical decision rules in psychotherapy. The implementation of these usually session-to-session-based feedback systems into clinical practice, however, remains challenging. This study aims to evaluate decision rules based on routine outcome monitoring with reduced assessment frequency. Method: Data routinely collected every 5-20 sessions of N = 3758 patients treated with CBT in an outpatient clinic (Msessions = 42.8, SD = 15.4) were used to develop feedback decision rules based on the expected treatment response and nearest neighbors approach, the reliable change index, and method of percental improvement. The detection of patients at risk of treatment failure served as primary endpoint. Results: Significantly lower reliable improvement, higher reliable deterioration rates, and smaller effect sizes were found for patients identified at risk of treatment failure by all rules. The nearest neighbors-based approach showed the highest sensitivity regarding the detection of reliably deteriorated cases. Conclusion: Consistent with previous research, the empirical models outperformed the rational rules. Still, the first-time used percental improvement-based rule also showed satisfactory results. Overall, the results point to the potential of basic feedback systems that might be easier to implement in practice than session-to-session based systems.

What time-varying network models based on functional analysis tell us about the course of a patient's problem

Background: Relations among psychological variables are assumed to be complex and to vary over time. Personalized networks can model multivariate complex interactions. The development of time-varying networks allows to model the variation of parameters over time. Objectives: We aimed to determine the value of time-varying networks for clinical practice. Methods: We applied time-varying mixed graphical models (TV-MGM) and time-varying vector autoregressive models (TV-VAR) to intensive longitudinal data of nine participants with depressive symptoms (n = 6) or anxiety (n = 3). Results: Most of the participants showed temporal changes in network topology within the assessment period of 30 days. Time-varying networks of participants with small, medium, and large time variability in edge parameters clearly show the different temporal evolvements of dynamic interactions between variables. The case example indicates clinical utility but also limitations to the application of time-varying networks in clinical practice. Conclusion: Time-varying network models provide a data-driven and exploratory approach that could complement current diagnostic standards by reflecting interacting, often mutually reinforcing processes of mental health problems and by accounting for variation over time. They can be used to generate hypotheses for further confirmatory and clinical testing.

Effects of an Organizational Implementation Strategy on Sustainment of Measurement-Based Care in Community Mental Health

OBJECTIVE

Little is known about how to sustain evidence-based interventions with fidelity in community mental health settings. Phase 1 of the Working to Implement and Sustain Digital Outcome Measures (WISDOM) trial showed that an organizational strategy improved the implementation of measurement-based care (MBC) in mental health services for youths 1-12 months after clinician MBC training. The authors report results from phase 2 of the trial, in which the strategy's effects on MBC sustainment 13-26 months after clinician MBC training were examined.

METHODS

Twenty-one outpatient mental health clinics were randomly assigned to MBC training and technical assistance plus the Leadership and Organizational Change for Implementation (LOCI) strategy (11 clinics) or to training and technical assistance only (10 clinics). In phase 2, the primary outcomes of MBC completion rate, youth symptom improvement, and MBC fidelity were examined for 452 youths who entered treatment 13-26 months after clinician MBC training.

RESULTS

No differences were found in MBC completion rate or symptom improvement between the two conditions; however, among the 81 youths who received MBC, fidelity was significantly higher at LOCI sites relative to control sites (24%, SE=11.1 vs. 1%, SE=1.0, respectively; p=0.003).

CONCLUSIONS

During phase 2, LOCI sites (vs. control sites) sustained superior MBC fidelity when MBC was used; however, superior MBC completion rates and clinical outcomes were not sustained. Sustainment of MBC may require strategies that improve its fit with regulatory and reimbursement environments in addition to strategies that develop clinic infrastructure.

An examination of therapists' professional characteristics as moderators of the effect of feedback on psychological treatment outcomes

Feedback-informed treatment (FIT) has been shown to reduce the gap between more and less effective therapists. This study aimed to examine therapists' professional characteristics as potential moderators of the effect of feedback on treatment outcomes.

The IAPT-FIT Trial was a clinical trial where therapists were randomly assigned to a FIT group or a usual care control group. Treatment response was monitored using measures of depression (PHQ-9), anxiety (GAD-7) and functional impairment (WSAS). In a secondary analysis of this trial (n = 1,835 patients; t = 67 therapists), we used multilevel modelling to examine interactions between therapists' professional characteristics (e.g., attitude towards and self-efficacy regarding feedback utilization, decision-making style, job satisfaction, burnout, difficulties in practice, coping styles, caseload size) with random allocation (FIT vs. controls) to identify moderators of the effects of feedback.

Between 9.6% and 10.8% of variability in treatment outcomes was attributable to therapist effects. Therapist-level caseload sizes and external feedback propensity (EFP) moderated the effect of feedback on depression outcomes. No statistically significant main effects were found for any of the included therapist characteristics.

FIT reduced variability in outcomes between therapists and was particularly effective for therapists with high EFP and larger caseloads.

Training Community Therapists in Core Elements of CBT and Family Therapy for Adolescent Externalizing Problems

OBJECTIVE

This pilot study tested pragmatic methods for training therapists in core techniques of two evidence-based interventions (EBIs) for adolescent externalizing problems: cognitive-behavioral therapy (CBT) or family therapy (FT). Training methods were designed to help therapists accurately self-monitor their use of EBIs and increase delivery of EBIs with current clients. The study compared coder training only versus coder training plus fidelity-focused consultation.

METHOD

Therapists (N = 42) from seven behavioral health clinics reported on 65 youth clients; four clinics elected to train in CBT and three in FT. Therapists were randomized to either coder training only, consisting of a 25-week observational coder training course (didactic instruction and mock session coding exercises in core EBI techniques); or coder training plus fidelity-focused consultation, consisting of direct-to-therapist fidelity measurement feedback along with fidelity-focused expert consultation. During the 25 weeks of training, therapists submitted self-report data on EBI use along with companion session audiotapes subsequently coded by observational raters.

RESULTS

Compared to coder training only, coder training plus fidelity-focused consultation produced superior effects in therapist ability to judge the extensiveness of EBI techniques in online coding sessions, as well as therapist ability to self-rate use of EBI techniques with their own cases. In both conditions, therapists who trained in CBT showed a significant, though modest, increase in real-world delivery of core CBT techniques; this did not occur for FT.

CONCLUSIONS

Pragmatic training and consultation methods show promise as viable and effective options for enhancing EBI fidelity monitoring and, for CBT, increasing EBI delivery.

The Relationship Between Therapist Perceptions and Engagement in Measurement-Based Care

Measurement-based care (MBC) is an evidence-based practice that has been supported by the American Psychological Association for almost two decades (APA 2006; Boswell et al. 2023). However, the widespread dissemination of technology-enabled platforms supporting the implementation of MBC in routine clinical settings has failed to significantly increase therapist rates of engagement in MBC. The present study was designed to examine the relationship between therapist perceptions of and their engagement in MBC, an area of inquiry that has yet to be pursued. A sample of 196 therapists and their respective clients (N = 12,166) was extracted from the therapist pool of a technology-enabled, digital mental health platform offering an automated MBC system. The sample of therapists was administered a previously validated measure assessing their perceptions of different aspects of MBC. Additionally, their frequency of viewing client outcome scores was assessed as an indicator of engagement in MBC. Analyses failed to detect a direct relationship between therapist perceptions of each aspect of MBC and their frequency of viewing client scores. Nonetheless, a subsequent cluster analysis identified a three-cluster solution. The cluster displaying the most positive perceptions of all aspects of MBC was associated with higher frequency of viewing client score. These findings suggest that therapist perceptions of MBC's clinical usefulness could have a unique impact on therapist engagement in MBC.

Challenging Cognitive Load Theory: The Role of Educational Neuroscience and Artificial Intelligence in Redefining Learning Efficacy

Background/Objectives: This systematic review integrates Cognitive Load Theory (CLT), Educational Neuroscience (EdNeuro), Artificial Intelligence (AI), and Machine Learning (ML) to examine their combined impact on optimizing learning environments. It explores how AI-driven adaptive learning systems, informed by neurophysiological insights, enhance personalized education for K-12 students and adult learners. This study emphasizes the role of Electroencephalography (EEG), Functional Near-Infrared Spectroscopy (fNIRS), and other neurophysiological tools in assessing cognitive states and guiding AI-powered interventions to refine instructional strategies dynamically. Methods: This study reviews n = 103 papers related to the integration of principles of CLT with AI and ML in educational settings. It evaluates the progress made in neuroadaptive learning technologies, especially the real-time management of cognitive load, personalized feedback systems, and the multimodal applications of AI. Besides that, this research examines key hurdles such as data privacy, ethical concerns, algorithmic bias, and scalability issues while pinpointing best practices for robust and effective implementation. Results: The results show that AI and ML significantly improve Learning Efficacy due to managing cognitive load automatically, providing personalized instruction, and adapting learning pathways dynamically based on real-time neurophysiological data. Deep Learning models such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Support Vector Machines (SVMs) improve classification accuracy, making AI-powered adaptive learning systems more efficient and scalable. Multimodal approaches enhance system robustness by mitigating signal variability and noise-related limitations by combining EEG with fMRI, Electrocardiography (ECG), and Galvanic Skin Response (GSR). Despite these advances, practical implementation challenges remain, including ethical considerations, data security risks, and accessibility disparities across learner demographics. Conclusions: AI and ML are epitomes of redefinition potentials that solid ethical frameworks, inclusive design, and scalable methodologies must inform. Future studies will be necessary for refining pre-processing techniques, expanding the variety of datasets, and advancing multimodal neuroadaptive learning for developing high-accuracy, affordable, and ethically responsible AI-driven educational systems. The future of AI-enhanced education should be inclusive, equitable, and effective across various learning populations that would surmount technological limitations and ethical dilemmas.

Personalization of structured group psychotherapy through add-on interventions: A potential for active engagement

OBJECTIVE

Routine Outcome Monitoring (ROM) in group psychotherapy has shown varied results, as personalizing therapy for multiple patients is challenging. This study explored the impact of ROM and individual Add-On Interventions (AOIs) for patients with depression who were not progressing during Group Cognitive Behavioral Therapy (GCBT). We followed the research question, "How can individual AOIs contribute to patients' therapy courses?".

METHOD

We interviewed patients and therapists involved in GCBT with ROM and individual AOIs prompted by Not-On-Track alerts. Thematic analysis was used to generate themes related to participants' experiences. Three cases were selected to illustrate the structure of the themes.

RESULTS

The main theme, "AOIs: Potential for active engagement," was constituted by seven subthemes concerning the influence of ROM, therapeutic collaboration, shared decision-making, allowing for person-tailored interventions, and the dynamic between group and individual sessions. The potential for active engagement was nurtured through flexible awareness, as patients shifted between a first-person perspective and an external perspective on their therapy process.

CONCLUSION

Individual AOIs, prompted by patient feedback, could create opportunities for reflection and engagement. However, the impact was shaped by the therapeutic relationships, the shared decision-making process, and the extent to which the AOIs offered complementary approaches to group psychotherapy.

The impact of patient choice on uptake, adherence, and outcomes across depression, anxiety, and eating disorders: a systematic review and meta-analysis

Growing evidence highlights the critical role of patient choice of treatment, with significant benefits for outcomes found in some studies. While four meta-analyses have previously examined the association between treatment choice and outcomes in mental health, robust conclusions have been limited by the inclusion of studies with biased preference trial designs. The current systematic review included 30 studies across three common and frequently comorbid mental health disorders (depression N = 23; anxiety, N = 5; eating disorders, N = 2) including 7055 participants (Mage 42.5 years, SD 11.7; 69.5% female). Treatment choice most often occurred between psychotherapy and antidepressant medication (43.3%), followed by choice between two different forms of psychotherapy, or elements within psychotherapy (36.7%). There were insufficient studies with stringent designs to conduct meta-analyses for anxiety or eating disorders as outcomes, or for treatment uptake. Treatment choice significantly improved outcomes for depression (d = 0.17, n = 18) and decreased therapy dropout, both in a combined sample targeting depression (n = 12), anxiety (n = 4) and eating disorders (n = 1; OR = 1.46, 95% CI: 1.17, 1.83), and in a smaller sample of the depression studies alone (OR = 1.65, 95% CI: 1.05, 2.59). All studies evaluated the impact of adults making treatment choices with none examining the effect of choice in adolescents. Clear directions in future research are indicated, in terms of designing studies that can adequately test the treatment choice and outcome association in anxiety and eating disorder treatment, and in youth.

Improved dropout prediction in group cognitive behavior therapy (CBT) using classification trees

OBJECTIVE

Dropout is a major factor undermining the effectiveness of psychotherapy, however, it remains poorly anticipated in clinical practice. Classification trees may offer simple, accessible, and practical solutions to identifying patients at-risk of dropout by synthesizing potentially complex patterns of relationships among intake measures.

METHOD

Intake variables were collected from day-patients who attended a Cognitive Behavior Therapy (CBT) group program at a private psychiatric hospital between 2015 and 2019. Based on these variables, two classification trees were trained and tested to predict dropout in (1) a weekly group, and (2) an intensive daily program.

RESULTS

Dropout was lower in the intensive treatment (Weekly CBT = 21.9%, Daily CBT = 13.2%), however, in both programs, the number of comorbid diagnoses was the most important factor predicting dropout. Overall balanced accuracy was comparable for both tree models, with the Weekly CBT model identifying 63.18% of dropouts successfully, and the Daily CBT model identifying dropouts with 62.06% accuracy.

CONCLUSION

Findings suggest that comorbidity may be the most important factor to consider when assessing dropout risk in CBT, and that dropout can be predicted with moderate accuracy early in therapy via simple models. Furthermore, findings suggest that condensed, intensive treatments may bolster patient retention.

A mixed-methods study exploring the benefits, drawbacks, and utilization of data in care: Findings from the EPI-CAL early psychosis learning health care network

Using data collected in routine care delivery to inform treatment is a key feature of a learning health system (LHS). In this study, we explored the experiences of service users and providers adopting measurement-based care (MBC) in early psychosis (EP) specialty care settings. Qualitative interviews were conducted with 32 providers and 12 service users across 18 programs in the Early Psychosis Intervention Network of California (EPI-CAL). These findings were compared with quantitative data from Beehive, EPI-CAL's data collection and review application. Regarding the clinical benefits of MBC in EP, three broad themes were identified - supporting safety monitoring and response, the assessment process, and delivery of psychotherapy. Outside of direct clinical care, Beehive was considered to support clinical supervision and external reporting, while service users reported data collection facilitated self-reflection. In the quantitative Beehive application data collected from 23 EP programs, high utilization of the safety alert system was evident (349 alerts in total, of which 338 [96.85 %] were resolved at a median of 2.03 days). However, service users' key survey data was only reviewed by assigned providers in 32.22 % (142 of 441) of cases. While providers and service users saw many benefits to Beehive, utilization was highly inconsistent outside of the alert system. Going forward, further consideration of how best to support EP providers to consistently use data in care is necessary to maximize the utility of the LHS approach and positively impact outcomes.

Treatment completion among Australians attending publicly-funded specialist alcohol and other drug treatment services

INTRODUCTION

Completion of alcohol and other drug (AOD) treatment is associated with improved health and social outcomes. Previous research has largely focused on individual (e.g. demographic) or service-level (e.g. treatment modality/setting) factors related to treatment completion in isolation. This study investigates the relationship between treatment completion and service-level and substance related factors, after controlling for demographics.

STUDY DESIGN

Retrospective cohort of 53,430 people engaging in government funded specialist AOD treatment across New South Wales, Australia, between 1 January 2015-31 December 2018.

METHODS

Generalised linear mixed models were used to analyse factors associated with treatment completion, accounting for repeated measures across treatment episodes.

RESULTS

Approximately two-thirds (69.8 %, n = 37,330) of the cohort completed treatment at least once during the study period, 42.3 % (22,605) on a single, and 27.5 % (14,725) on multiple occasions. After controlling for demographics, treatment episodes for amphetamines were least likely to be recorded as complete (52.5 %), while those for MDMA were 1.9 (95 % CI: 1.49, 2.45) times more likely (67 %) than amphetamine episodes to be completed. Treatment episodes mandated through criminal justice/child protection agencies were 1.25 (95 % CI: 1.20, 1.30) times more likely to be completed compared to those originating from self-referral. There were no differences in treatment completion between self or health professional referrals (aOR: 0.98, 95 % CI: 0.95, 1.02). Episodes involving involuntary AOD treatment modalities and residential withdrawal were 6.67 times (95% CI: 4.53, 9.81) and 5.02 times (95 % CI: 4.46, 5.64) more likely respectively to be completed compared to those for community rehabilitation. Case management episodes were also more likely (aOR: 2.43, 95 % CI: 2.16, 2.73) to be completed. Episodes of longer treatment duration (≥90 days) were 1.89 times (95 % CI: 1.82, 1.97) more likely to be completed compared to shorter (≤30 days) treatment episodes.

CONCLUSIONS

Treatment completion was moderated by a range of factors including drug type, treatment modality and duration, and referral source. Low rates of amphetamine treatment completion across treatment modalities confirms the urgent need for further research investigating more effective treatment options for amphetamine use disorders.

Decoding emotions: Exploring the validity of sentiment analysis in psychotherapy

OBJECTIVE

Given the importance of emotions in psychotherapy, valid measures are essential for research and practice. As emotions are expressed at different levels, multimodal measurements are needed for a nuanced assessment. Natural Language Processing (NLP) could augment the measurement of emotions. The study explores the validity of sentiment analysis in psychotherapy transcripts.

METHOD

We used a transformer-based NLP algorithm to analyze sentiments in 85 transcripts from 35 patients. Construct and criterion validity were evaluated using self- and therapist reports and process and outcome measures via correlational, multitrait-multimethod, and multilevel analyses.

RESULTS

The results provide indications in support of the sentiments' validity. For example, sentiments were significantly related to self- and therapist reports of emotions in the same session. Sentiments correlated significantly with in-session processes (e.g., coping experiences), and an increase in positive sentiments throughout therapy predicted better outcomes after treatment termination.

DISCUSSION

Sentiment analysis could serve as a valid approach to assessing the emotional tone of psychotherapy sessions and may contribute to the multimodal measurement of emotions. Future research could combine sentiment analysis with automatic emotion recognition in facial expressions and vocal cues via the Nonverbal Behavior Analyzer (NOVA). Limitations (e.g., exploratory study with numerous tests) and opportunities are discussed.

A multilevel framework for recruitment and retention in implementation trials: An illustrative example

BACKGROUND

Implementation and hybrid effectiveness-implementation trials aspire to speed the translation of science into practice by generating crucial evidence for improving the uptake of effective health interventions. By design, they pose unique recruitment and retention challenges due to their aims, units of analysis, and sampling plans, which typically require many clinical sites (i.e. often 20 or more) and participation by individuals who are related across multiple levels (e.g. linked organizational leaders, clinicians, and patients). In this article, we present a new multilevel, theory-informed, and relationship-centered framework for conceptualizing recruitment and retention in implementation and hybrid effectiveness-implementation trials which integrates and builds on prior work on recruitment and retention strategies in patient-focused trials. We describe the framework's application in the Working to Implement and Sustain Digital Outcome Measures hybrid type III trial, which occurred in part during the COVID-19 pandemic.

METHODS

Recruitment for the Working to Implement and Sustain Digital Outcome Measures trial occurred from October 2019 to February 2022. Development of recruitment and retention strategies was guided by a newly developed multilevel framework, which targeted the capability, opportunity, and motivation of organizational leaders, clinicians, patient-facing administrative staff, and patients to engage in research. A structured assessment guide was developed and applied to refine recruitment and retention approaches throughout the trial. We describe the framework and its application amid the onset of the COVID-19 pandemic which required rapid adjustments to address numerous barriers.

RESULTS

The Working to Implement and Sustain Digital Outcome Measures trial enrolled 21 outpatient clinics in three US states, incorporating 252 clinicians and 686 caregivers of youth (95% of patient recruitment target) across two distinct phases. Data completion rates for organizational leaders and clinicians averaged 90% over five waves spanning 18 months, despite the onset of the COVID pandemic. Caregiver completion rates of monthly follow-up assessments ranged from 80%-88% across 6 months. This article presents the multilevel framework, assessment guide, and strategies used to achieve recruitment and retention targets at each level.

CONCLUSION

We conducted a multi-state hybrid type III effectiveness-implementation trial that maintained high recruitment and retention across all relevant levels amid a global pandemic. The newly developed multilevel recruitment and retention framework and assessment guide presented here, which integrates behavioral theory, a relationship-focused lens, and evidence-based strategies for participant recruitment and retention at multiple levels, can be adapted and used by other researchers for implementation, hybrid, and multilevel pragmatic trials as well as other implementation studies.

To Use or Not to Use: Exploring Therapists' Experiences with Pre-Treatment EMA-Based Personalized Feedback in the TheraNet Project

BACKGROUND

Using idiographic network models in psychotherapy has been a growing area of interest. However, little is known about the perceived clinical utility of network models. The present study aims to explore therapists' experiences with network model-based feedback within the context of the TheraNet Project.

METHODS

In total, 18 therapists who had received network-based feedback for at least 1 patient at least 2 months prior were invited to retrospective focus groups. The focus group questions related to how participation in the study influenced the therapeutic relationship, how the networks were used, and what might improve their clinical utility. The transcribed focus groups were analyzed descriptively using qualitative content analysis.

RESULTS

Most therapists mentioned using the feedback to support their existingtheir case concept, while fewer therapists discussed the feedback directly with the patients. Several barriers to using the feedback were discussed, as well as various suggestions for how to make it more clinically useful. Many therapists reported skepticism with regards to research in the outpatient training center in general, though they were also all pleasantly surprised by being involved, having their opinions heard, and showing a readiness to adapt research to their needs/abilities.

CONCLUSIONS

This study highlights the gap between researchers' and therapists' perceptions about what useful feedback should look like. The TheraNet therapists' interest in adapting the feedback and building more informative feedback systems signals a general openness to the implementation of clinically relevant research. We provide suggestions for future implementations of network-based feedback systems in the outpatient clinical training center setting.

Process and Implementation Elements of Measurement Feedback Systems: A Systematic Review

Measurement feedback systems (MFS) can help guide treatment and improve clinical outcomes. Studies of MFS are heterogeneous both in execution and results, and the effects of MFS seem restricted by limited attention to process and implementation elements and by limited adoption by health professionals. The current systematic review mapped the use of process and implementation elements in MFS studies. An overview of therapists' use of and attitudes toward MFS is provided. Three-level meta-analyses were used to test theoretically informed process and implementation elements as moderators of the effects of MFS. Hypotheses and general propositions from Clinical Performance Feedback Intervention Theory (CP-FIT) were used to organize the elements of the studies and were used as moderator variables. Previous studies on MFS interventions have had a limited focus on implementation efforts and process elements that may increase the effects of MFS and their use among therapists. Efforts have sparsely been made to reduce barriers to MFS use, and several studies have reported limited engagement with MFS among therapists. Therapists' attitudes toward MFS, feedback, or standardized measures were heterogeneously reported, making data synthesis challenging. Identified process and implementation elements were not significantly associated with effect sizes in the studies and the results did not support the propositions of CP-FIT. The lack of statistically significant associations may be due to limited reporting of details about process and implementation aspects. More research designed to test hypotheses regarding process and implementation elements is needed to improve the use and effects of MFS. Future studies should aspire to report findings in a manner that allows for an understanding of the implementation process and therapists' adoption of these systems.

Barriers to Progress Feedback Adoption in Outpatient Geriatric Mental Healthcare: Exploring Age-Related Factors - A Qualitative Study

Monitoring treatment progress through progress feedback is recognized for its efficacy and demonstrated value. However, its integration and utilization within treatments still need to be improved. Insufficient understanding exists regarding the factors within geriatric mental healthcare influencing the adoption of progress feedback. This study aimed to explore the determinants impacting the utilization of progress feedback within outpatient geriatric mental healthcare, specifically focusing on age-related perspectives and patient group characteristics. This qualitative investigation employed semi-structured interviews involving clinicians (N = 14) selected from four outpatient geriatric teams. The findings revealed both inhibiting and facilitating perspectives concerning progress feedback. Clinicians preferred user-friendly, specific, and tailored measures. Challenges included organizational support, integration in work processes, training, and the digital progress feedback system. Age-related perspectives such as older adults' diverse issues, limited digital skills, and cognitive problems hindered implementation, particularly in the oldest generation of older patients. In outpatient geriatric mental healthcare, many factors and attitudes influencing progress feedback align with those observed in adult psychiatry literature. Moreover, this study highlights specific age-related factors that impede the adoption and implementation of progress feedback, shedding light on the specific barriers within this context.

Strategic Treatment and Assessment for Youth (STAY): A Theoretically-Driven, Culturally-Tailored MBC Approach

Racial and ethnic minoritized (REM) youth are at greater risk for depression and suicide than their White peers. Despite this, REM youth are much more likely than their White peers to prematurely dropout of treatment. Culturally tailored and scalable engagement models to improve mental health treatment retention among REM youth with depressive symptoms and suicidal thoughts and behaviors (STB) are urgently needed. Strategic Treatment Assessment for Youth (STAY) is a theoretically-driven, culturally tailored measurement-based care (MBC) approach to treatment engagement for REM youth with depressive symptoms and suicide risk. Specifically, STAY uses MBC feedback processes to reduce perceptual barriers to treatment, thus improving treatment retention and ultimately, client outcomes among REM youth. In addition to standard MBC components, STAY includes a greater emphasis on providing a client-centered rationale for MBC which includes assessing and discussing treatment expectations, the use of individualized progress measures and alliance measures, and cultural competence training. The goal of this manuscript is to describe the STAY model based on initial theoretical development and preliminary clinician-informed refinements. Further, a case example of STAY is presented with a particular focus on the use of feedback processes. Finally, the current and future directions to empirically examine STAY as a treatment retention strategy with REM populations are provided.

Psychotherapists' Experience with In-Session Use of Routine Outcome Monitoring: A Qualitative Meta-analysis

Routine outcome monitoring (ROM) has become an increasingly utilized tool in therapeutic practice that has the potential to improve therapy outcomes. This study aimed to synthesize the findings of existing qualitative studies investigating how clinicians use ROM in their work with clients. A systematic search of qualitative studies on clinicians' experience with the use of ROM in mental health services was conducted via PsycInfo, PsycArticles, Medline, Web of Science, and Scopus databases. Qualitative meta-analysis was used to synthesize the finding of the primary studies. Forty-seven studies met the inclusion criteria. The analysis resulted in 21 meta-categories organized into six clusters, namely (1) obtaining clinically relevant information, (2) adapting treatment, (3) facilitating communication, (4) enhancing the therapeutic relationship, (5) facilitating change in clients, and (6) personalized usage of ROM. The meta-analysis revealed that clinicians utilized ROM in diverse ways, including both informational and communicational functions. From the clinicians' perspective, ROM was an element that, on the one hand, introduced additional structure and standardization in treatment and, on the other hand, allowed for greater flexibility and tailoring of treatment.

Dose-Response Effects of MittEcho, a Measurement Feedback System, in an Indicated Mental Health Intervention for Children in Municipal and School Services in Norway

Including routine client feedback can increase the effectiveness of mental health interventions for children, especially when implemented as intended. Rate of implementation, or dose, of such feedback interventions has been shown to moderate results in some studies. Variation in implementation and use of client feedback may also contribute to the mixed results observed within the feedback literature. This study evaluates dose-response associations of client feedback using a novel Measurement Feedback System (MFS) within an indicated group intervention. The primary aim was to determine whether the rate of MFS implementation predicts symptom reduction in anxiety and depression among school-aged children. The secondary aim was to assess whether the rate of MFS implementation influences children's satisfaction with the group intervention or their dropout rates. Data were collected via a randomized factorial study (clinicaltrials.gov NCT04263558) across 58 primary schools in Norway. Children aged 8 to 12 years (N = 701) participated in a group-based, transdiagnostic intervention targeting elevated symptoms of anxiety or depression. Half of the child groups also received the feedback intervention using the MittEcho MFS. Group leaders (N = 83), recruited locally, facilitated the interventions. The MFS dose was measured using the Implementation Index, which combines the use of MFS by both children and providers (group leaders) into a single dose variable. Results showed no significant additional effect of dose of MFS on change in depression or anxiety scores, on user satisfaction with the intervention or on intervention dropout. The discussion addresses potential reasons for these non-significant findings and implications for MFS implementation in preventive, group-based interventions in school settings.

Effects and experiences of idiographic patient-reported outcome measures for feedback in psychotherapy: A systematic review and secondary analysis of the empirical literature

Objective: There is a growing interest in idiographic patient-reported outcome measures (I-PROMs) for routine outcome monitoring (ROM) and clinical feedback (CF) in psychotherapy, but to our knowledge, no systematic reviews of the empirical literature. Method: We conducted a systematic literature search for empirical papers investigating effects and experiences of I-PROMs for ROM/CF and found a total of 13 papers. There was only one experimental controlled effectiveness study. Results: We formulated a narrative summary of the data set as a whole. We conducted a secondary analysis of nine papers containing qualitative data on stakeholder experiences with I-PROMs and found three superordinate themes and eight subthemes, summarized as "I-PROMs can facilitate self-reflection for patients, assist in identifying and tracking therapeutic topics, and make patients more committed to therapy by giving them a greater sense of responsibility and empowerment. Formulating goals and problems can be difficult, and lack of progress can be demotivating. Time in therapy is precious and must be spent wisely. Effective use of I-PROMs is facilitated by flexibility and therapists' use of clinical skills." Conclusion: We discuss the limitations of the study and provide recommendations for future research and clinical practice.

Using Progress Feedback to Enhance Treatment Outcomes: A Narrative Review

We face increasing demand for greater access to effective routine mental health services, including telehealth. However, treatment outcomes in routine clinical practice are only about half the size of those reported in controlled trials. Progress feedback, defined as the ongoing monitoring of patients' treatment response with standardized measures, is an evidence-based practice that continues to be under-utilized in routine care. The aim of the current review is to provide a summary of the current evidence base for the use of progress feedback, its mechanisms of action and considerations for successful implementation. We reviewed ten available meta-analyses, which report small to medium overall effect sizes. The results suggest that adding feedback to a wide range of psychological and psychiatric interventions (ranging from primary care to hospitalization and crisis care) tends to enhance the effectiveness of these interventions. The strongest evidence is for patients with common mental health problems compared to those with very severe disorders. Effect sizes for not-on-track cases, a subgroup of cases that are not progressing well, are found to be somewhat stronger, especially when clinical support tools are added to the feedback. Systematic reviews and recent studies suggest potential mechanisms of action for progress feedback include focusing the clinician's attention, altering clinician expectations, providing new information, and enhancing patient-centered communication. Promising approaches to strengthen progress feedback interventions include advanced systems with signaling technology, clinical problem-solving tools, and a broader spectrum of outcome and progress measures. An overview of methodological and implementation challenges is provided, as well as suggestions for addressing these issues in future studies. We conclude that while feedback has modest effects, it is a small and affordable intervention that can potentially improve outcomes in psychological interventions. Further research into mechanisms of action and effective implementation strategies is needed.

Gender, Race/Ethnicity, and Patient-Therapist Matching on Gender and Race/Ethnicity: Predictors/Moderators of the Effectiveness of Trust/Respect Feedback

A plethora of studies has exhibited the effectiveness of using measurement-based care feedback systems within mental health services to improve treatment outcomes; however, patient gender/race and patient-therapist matching on gender and race remain relatively unexplored as predictors/moderators in feedback studies. We conducted predictor/moderator analyses focusing on the relation of gender, race/ethnicity, and patient-therapist gender and race/ethnicity matching on two outcomes: patient self-reported levels of (1) functioning and (2) trust/respect within the therapeutic relationship. We used data from a randomized controlled trial studying the effectiveness of a feedback system comparing patient-reported levels of trust and respect towards their provider (together with symptom feedback) in comparison to symptom only feedback. We found that men improved in functioning more than women when their therapists received trust/respect feedback compared to symptom only feedback (F[1, 902] = 9.79, p = .002, d = 0.21). We also found that dyads matched on race/ethnicity but not gender, and those matched on gender but not race/ethnicity, improved in functioning over time more than dyads mismatched on both gender and race/ethnicity and those matched on both gender and race/ethnicity (F[1, 897] = 8.63, p = .0034, d = 0.20). On trust/respect outcomes, we found a gender difference over time (F[1, 759] = 6.61, p = .01, d = 0.19), a gender matching difference by feedback condition interaction (F[1, 757] = 5.25, p = .02, d = 0.17), and a racial/ethnic matching difference on trust/respect scores over time (F[1, 785] = 3.89, p = .049, d = 0.14). Male patients showed an initial decrease followed by a steady increase in trust/respect over time while female patients showed an initial increase followed by a steady decrease. Gender-matched therapeutic dyads showed higher levels of trust/respect compared to mismatched dyads when therapists received symptom only feedback, but this difference was not apparent when trust/respect feedback was provided. Dyads mismatched on race/ethnicity improved steadily in trust/respect over time, but matched dyads decreased in trust/respect after an initial increase. Future research should focus on the use of feedback systems to enhance outcomes for patients with specific gender and racial/ethnic identities.

Renewed Feedback-Informed Group Treatment for Patients with Anxiety and Depressive Disorders

Feedback-Informed Group Treatment (FIGT) shows promise for improving outcomes, but results are mixed. The aim was investigating the feasibility, acceptability and effects of renewed FIGT on clinical outcomes and therapy processes. In a quasi-experimental pilot study, 65 patients with anxiety or depressive disorders and 15 therapists of interpersonal psychotherapy or cognitive behavioural therapy groups using renewed FIGT were included. Renewed FIGT contained three additions compared to the previous tool: (1) personalized goals along with the Outcome Questionnaire-45 (OQ-45), (2) therapists' training, coaching and intervision, and (3) instructions to actively use feedback in the group. Data on feasibility, acceptability, outcomes and process factors were analysed and compared with those of historical cohorts using only OQ-45 feedback or no feedback, using descriptive, multilevel and covariance statistical analyses. Feasibility was mostly improved, with patients experiencing more feedback discussions and better usability compared to only OQ-45 feedback. At least two thirds of the patients and therapists give preference to using feedback in the future. At the end of the study, therapists were less convinced that the OQ-45 and goals were able to detect change. Renewed FIGT did not improve effectiveness on clinical outcomes. Compared to no feedback, patients experienced more cohesion, engagement and less avoidance, but improved less on depressive symptoms. Even when renewed FIGT is more feasible and usable than only OQ-45 feedback and associated with more cohesiveness and engagement, it may not automatically lead to improved effectiveness on clinical outcomes in short-term group therapy. Implications and future directions are described.

Motives of Therapists for Using Routine Outcome Monitoring (ROM) and How it is Used by Them in Clinical Practice: Two Qualitative Studies

Despite its demonstrated value, many mental health institutions struggle to implement progress feedback effectively. There is also insufficient information about how therapists utilize progress feedback. To gain more insight, two qualitative studies were conducted. The first study compared the attitudes and motives of therapists who used and those who did not use progress feedback. The second study examined how psychologists incorporated progress feedback into their practice. In total, 23 therapists were interviewed, and the data were analyzed using thematic analysis. The first study found that almost all the therapists had a positive attitude about progress feedback. Those who did not use it indicated reasons such as a heavy workload and patient-related factors, and they also lacked sufficient information about the potential benefits of progress feedback. The second study revealed four major ways in which the therapists utilized feedback, namely: supporting actions to discuss progress feedback, discussing progress feedback with patients, modifications in the ongoing treatment, and peer consultation. However, discussions during peer consultations about using progress feedback for patients who were not benefitting from treatment and how the treatment might be adjusted accordingly were often lacking. In conclusion, it is crucial that training and education be provided to therapists on how to use progress feedback effectively. Having regular discussions about progress feedback during peer consultations could also facilitate its integration into clinical practice.

Understanding non-response in psychotherapy: A meta-synthesis

INTRODUCTION

There is considerable research focusing on the negative outcomes of psychotherapy, however, there remains an overlooked population: those who appear to show no response after treatment. This qualitative evidence synthesis aimed to review the available literature exploring client and therapist experiences of psychotherapy nonresponse.

METHOD

Seven databases were searched for studies using qualitative approaches to explore nonresponse. Twenty-four studies met inclusion criteria and were analysed using Thematic Synthesis.

RESULTS

Six overarching client perspective themes and 18 subthemes were identified: Hopes and fears; A difficult task; A Disconnected relationship; Staying involved; Therapy was not worth the investment and, On a trajectory for improvement. Four overarching therapist perspective themes and 10 subthemes were identified: High expectations; Experiencing a disconnect; Feeling threatened and Holding onto hope.

DISCUSSION

A model of psychotherapy nonresponse is proposed. The experience of nonresponse appears to involve both clients and therapists finding therapy difficult, experiencing problems connecting and not being open in the therapeutic relationship. Importantly, nonresponse appears not to be an absence of effects, but a range of experiences that are potentially harmful, particularly to clients. However, there were varied experiences of nonresponse, which indicates the importance of qualitative outcome measurement.

Making the most out of timeseries symptom data: A machine learning study on symptom predictions of internet-based CBT

Objective

Predicting who will not benefit enough from Internet-Based Cognitive Behavioral (ICBT) Therapy early on can assist in better allocation of limited mental health care resources. Repeated measures of symptoms during treatment is the strongest predictor of outcome, and we want to investigate if methods that explicitly account for time-dependency are superior to methods that do not, with data from (a) only two pre-treatment timepoints and (b) the pre-treatment timepoints and three timepoints during initial treatment.

Methods

We use 1) commonly used time-independent methods (i.e., Linear Regression and Random Forest models) and 2) time-dependent methods (i.e., multilevel model regression, mixed-effects random forest, and a Long Short-Term Memory model) to predict symptoms during treatment, including the final outcome. This is done with symptom scores from 6436 ICBT patients from regular care, using robust multiple imputation and nested cross-validation methods.

Results

The models had a 14 %-12 % root mean squared error (RMSE) in predicting the post-treatment outcome, corresponding to a balanced accuracy of 67-74 %. Time-dependent models did not have higher accuracies. Using data for the initial treatment period (b) instead of only from before treatment (a) increased prediction results by 1.3 % percentage points (12 % to 10.7 %) RMSE and 6 % percentage points BACC (69 % to 75 %).

Conclusion

Training prediction models on only symptom scores of the first few weeks is a promising avenue for symptom predictions in treatment, regardless of which model is used. Further research is necessary to better understand the interaction between model complexity, dataset length and width, and the prediction tasks at hand.

Talking About Things Important to Me: Mental Health Consumers' Experiences of Consumer-Rated Measures

Since 2002, National Outcomes and Casemix Collection of clinician-rated and consumer-rated outcome measures has become part of routine care within Australian clinical mental health services, aiming to ensure that services understand, improve and are accountable for effectiveness of treatment and care provision. Consumer-rated outcome measures, implemented well, support basic human rights of consumers to be asked, heard and included equally in their own care. However, their use has lagged due to clinician inertia, uncertainty about their value to clinical care, assumptions about consumers' capacity to complete the measures and organisational cultural issues that have hampered more holistic assessment, consumer inclusion and care collaboration. Much is known about negative, largely tokenistic use of such measures, poor uptake and dominance of clinical approaches to measurement that privilege clinical expertise; however, little is known about consumers' positive experiences of using consumer-rated measures, Therefore, our aims were as follows: to seek the views and experiences of mental health consumers of using consumer-rated measures in their encounters with clinicians; to understand better whether there were benefits (and if so what) of consumer-rated measures being used in routine mental health practice; to understand how feedback on the use of consumer-rated measures can inform training for mental health staff; and to promote their wider use within mental health services. In-depth interviews conducted with 10 Australian mental health consumers used interview questions co-designed with lived experience and clinical advocates. Descriptive thematic analyses produced four themes emphasising consumers' preferences for completing the measures, the importance of explaining their purpose, how the process validated their feelings and was an opportunity for self-reflection, sense-making, trust-building, and transparency in the encounter and empowerment. This research offers recommendations about the value of effective implementation of consumer-rated measures.

Effectiveness of a Novel Web-Based Intervention to Enhance Therapeutic Relationships and Treatment Outcomes in Adult Individual Psychotherapy: Randomized Controlled Trial and Analysis of Predictors of Dropouts

BACKGROUND

Routine process and outcome monitoring interventions added to psychotherapy are known to improve treatment outcomes, although they vary in format and effectiveness.

OBJECTIVE

This study aimed to evaluate whether a therapist-independent, internet-based routine process monitoring and feedback system could significantly reduce psychological distress and enhance the quality of the therapeutic relationship compared with a treatment-as-usual control group among individuals already engaged in individual psychotherapy.

METHODS

We randomized 475 participants into either the intervention group, which received access to an internet-based routine process monitoring and feedback system in addition to psychotherapy, or the control group, which received only psychotherapy. The trial lasted for 10 weeks. Follow-up assessments at 5 weeks and 10 weeks used the Clinical Outcomes in Routine Evaluation-Outcome Measure as the primary outcome, with the Working Alliance Inventory-Short Revised and the Real Relationship Inventory-Client form as secondary outcomes.

RESULTS

Per-protocol analyses (n=166) showed that psychological distress decreased in both groups, but there was no significant advantage for the intervention group. The intervention group experienced a decline in the genuineness dimension score of the real relationship, with an effect size of d=-0.27, compared with d=0.01 in the control group. In the intervention group (but not in the control group), dropouts showed significantly lower real relationship levels (P=.002), working alliance quality (P=.051), and emotional disclosure (P=.01) compared with those who completed the study. Additionally, logistic regression revealed distinct predictors of dropout within the control group and intervention group.

CONCLUSIONS

The findings do not provide conclusive evidence for the efficacy of the new internet-based intervention in enhancing self-monitoring and prompting reflection on patients' emotional responses to their therapists. However, the intervention appears to influence patients' perceptions of the genuineness dimension in the therapeutic relationship, warranting further investigation. We hypothesize that this alteration in the genuineness dimension could be attributed to the intervention facilitating a more realistic and accurate perception of the therapeutic relationship among participants.

TRIAL REGISTRATION

ClinicalTrials.gov NCT06038747; https://clinicaltrials.gov/study/NCT06038747.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/55369.

Measurement-Based Care as a Tool to Detect and Prevent Harm in Youth Psychotherapy

Measurement-based care (MBC), the use of routine assessment to guide clinical decision-making, has the potential to significantly enhance the quality of mental health services for youth by improving the detection and prevention of harm. Concerns exist, however, that widespread efforts to implement MBC may have a negative impact on youth mental health care. We explore both perspectives by describing how MBC can be leveraged as a tool to detect and prevent harmful treatment in youth and how misapplication of MBC also has the potential to cause harm. This discussion focuses on how MBC can help therapists identify harm caused by ineffective practices (i.e., deterioration or no symptoms changes), poor client engagement (e.g., early drop-out, poor alliance), and adverse events (e.g., injury caused by treatment). We also discuss multi-level applications of aggregated MBC data to prevent harm through workforce development and organizational and health systems quality and cost of care improvement initiatives. Misapplication of MBC by failing to adhere to MBC best practices, overreliance on data above clinical judgement, providing insufficient organizational support, and using MBC data to justify cost containment are all discussed as possible sources of harm. We conclude with a discussion of future directions for research needed to advance the application of MBC to the prevention of harm in youth psychotherapy.

Building Mutually Beneficial Collaborations Between Digital Navigators, Mental Health Professionals, and Clients: Naturalistic Observational Case Study

Unlabelled

Despite the efficacy of digital mental health technologies (DMHTs) in clinical trials, low uptake and poor engagement are common in real-world settings. Accordingly, digital technology experts or "digital navigators" are increasingly being used to enhance engagement and shared decision-making between health professionals and clients. However, this area is relatively underexplored and there is a lack of data from naturalistic settings. In this paper, we report observational findings from the implementation of a digital navigator in a multidisciplinary mental health clinic in Sydney, Australia. The digital navigator supported clients and health professionals to use a measurement-based DMHT (the Innowell platform) for improved multidimensional outcome assessment and to guide personalized decision-making. Observational data are reported from implementation logs, platform usage statistics, and response rates to digital navigator emails and phone calls. Ultimately, support from the digital navigator led to improved data collection and clearer communications about goals for using the DMHT to track client outcomes; however, this required strong partnerships between health professionals, the digital navigator, and clients. The digital navigator helped to facilitate the integration of DMHT into care, rather than providing a stand-alone service. Thus, collaborations between health professionals and digital navigators are mutually beneficial and empower clients to be more engaged in their own care.

If You Give a Therapist a Network: A Qualitative Analysis of Therapists' Reactions to Their Patients' EMA-Based Network Models

OBJECTIVE

Network models hold much promise for use in the form of personalized feedback, which the TheraNet Project aims to investigate. In the present study, we share therapists' first reactions to patient network models based on ecological momentary assessment (EMA) data.

METHOD

Therapists (N = 24) were taught to interpret the network models based on their patient's data. They were provided with concrete use suggestions at the end of the workshop. The workshops were analysed using descriptive qualitative content analysis. The resulting codes were then grouped thematically.

RESULTS

Therapists spoke about using the networks for "case conceptualization", "therapy planning", and "psychoeducation", but also commented on the "study design". Before the concrete suggestions, therapists most frequently voiced thoughts on its uses for "case conceptualization". After the suggestions, "psychoeducation" was mentioned most frequently.

CONCLUSIONS

Therapists intuitively connected network models to case conceptualizations. Once concrete suggestions were provided, therapists more frequently discussed networks as psychoeducative tools.

Response processes for patients providing quantitative self-report data: a qualitative study

OBJECTIVE

To identify factors that influence response processes for patients providing quantitative self-report data. Secondly, due to the lack of integrative and explanatory models in this area, to develop a model of patients' response processes that can guide what to look for when considering validity evidence and interpreting scores on individual items.

METHODS

Participants (n = 13) were recruited from a specialized substance use disorder treatment clinic and interviewed while responding to items from a clinical feedback system implemented for routine outcome monitoring in that setting. The interview approach was based on cognitive interviewing. Data collection and analysis were inspired by a grounded theory approach.

RESULTS

We identified several variables that influenced the participants' response processes. The variables were organized into five categories: context-related variables; item-related variables; response base variables; reasoning strategies; and response selection strategies. We also found that the participants' responses for many items were affected by different aspects of the response process in ways that are relevant to interpretation but not necessarily discernible from the numerical scores alone, and we developed response categories to capture this.

CONCLUSION

The findings suggest that patients providing quantitative self-report data encounter conditions in the response process that challenge and influence their ability to convey meaning and accuracy. This results in responses that for many of the items reflect messages important for interpretation and follow-up, even if it does not appear from the numerical scores alone. The proposed model may be a useful tool when developing items, assessing validity, and interpreting responses.

State of the Science: The Hierarchical Taxonomy of Psychopathology (HiTOP)

The Hierarchical Taxonomy of Psychopathology (HiTOP) is a dimensional framework for psychopathology advanced by a consortium of nosologists. In the HiTOP system, psychopathology is grouped hierarchically from super-spectra, spectra, and subfactors at the upper levels to homogeneous symptom components and maladaptive traits and their constituent symptoms, and maladaptive behaviors at the lower levels. HiTOP has the potential to improve clinical outcomes by planning treatment based on symptom severity rather than heterogeneous diagnoses, targeting treatment across different levels of the hierarchy, and assessing distress and impairment separately from the observed symptom profile. Assessments can be performed according to this framework with the recently developed HiTOP-Self-Report (HiTOP-SR). Examples of how to use HiTOP in clinical practice are provided for the internalizing spectrum, including the use of the Unified Protocol and other modularized treatments, measurement-based care, psychopharmacology, and in traditionally underserved populations. Future directions are discussed in this State of the Science review including HiTOP's use in further developing transdiagnostic treatments, extending the model to include other information such as environmental factors, establishing the treatment utility of clinical assessment for the HiTOP-SR, developing new treatments, and disseminating the model.

Do Therapists Know When Their Clients Deteriorate? An Investigation of Therapists' Ability to Estimate and Predict Client Change During and After Psychotherapy

In routine outcome monitoring, psychotherapists receive feedback from their clients about their self-reported progress during therapy. This practice is based on research indicating that therapists overestimate their effectiveness and cannot detect and predict negative client change. However, this assumption is based on only a few studies. This study aimed to investigate whether trainee therapists could estimate client deterioration after each session and after therapy and whether they, from session to session, could predict client post-therapy outcomes. Fifty-three postgraduate trainee therapists at a university clinic treated 105 clients, with an average of 13.1 sessions. A questionnaire was developed to measure the therapists' estimation of client change at each session and their session-by-session prediction of client post-therapy outcomes. The 10-item version of the Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) was used to measure clients' self-reported change at each session. The data included 96 (91.5%) therapist estimations of client post-therapy outcomes, 1015 (87.8%) estimations of client change from the second to the penultimate session, and 1140 predictions of post-therapy outcomes. The study found that at post-therapy, the therapists were only able to identify one out of eight clients who showed deterioration on the CORE-10. Additionally, during therapy, they could accurately estimate only six out of 83 sessions in which clients had experienced deterioration on the CORE-10, and they failed to predict any of the eight clients who ended therapy with deterioration. In conclusion, therapists cannot rely on their clinical judgement alone to assess client progress and outcomes and will depend on routine outcome monitoring to detect client deterioration.

Factor Structure and Psychometric Properties of the ED-15 in People With an Eating Disorder

OBJECTIVE

This study examines the factorial structure and psychometric properties of the Eating Disorder-15 questionnaire (ED-15) in a large clinical sample, as well as the instrument's sensitivity to early clinical change in therapy and ability to measure remission.

METHOD

Participants with eating disorders (N = 278) referred to the Flinders University Services for Eating Disorders in South Australia completed the ED-15 as well as other measures of eating disorder symptoms and co-occurring psychopathology, including depression, anxiety, and stress.

RESULTS

Confirmatory factor analysis (CFA) revealed a two-factor model for the ED-15. The ED-15 had good internal consistency. It showed satisfactory concurrent validity with moderate correlations with the EDE-Q global score and contribution of unique variance to that score. Correlations indicated good convergent validity with clinical impairment and good divergent validity from depression, anxiety, and stress. The ED-15 showed a significant medium effect size change within the first four sessions of therapy. Good discriminant validity was indicated by cut-off scores used for remission, with significantly different levels of ED psychopathology and other impairments between the two groups.

DISCUSSION

This study adds to the four previous psychometric studies of the ED-15, confirming robustness of the English version in a clinical sample. The brevity and psychometric robustness of the ED-15 makes it a preferable measure to the Eating Disorder Examination for sessional assessment of progress in treatment.

Does the STIC measurement and feedback system improve multisystemic outcomes in individual, couple and family therapy? A randomized clinical trial

OBJECTIVE

In a randomized clinical trial, we evaluated whether the STIC (Systemic Therapy Inventory of Change) measurement and feedback system (MFS), the first MFS to explicitly integrate the family systems perspective, improved outcomes in individual, couple and family therapy.

METHOD

Nine hundred and seventy clients seeking individual, couple or family therapy, entered therapy with 93 therapists at four sites in the Chicago metropolitan area. All therapists were trained with the STIC and participated in both Treatment as Usual (TAU) and TAU with the STIC (STIC). After agreeing to participate, clients were randomly assigned to TAU or STIC. Therapists did not know the condition to which a case was assigned, until just prior to the first session. Therapy was not time-limited or constrained, except for the use of the STIC in the STIC condition. All clients were evaluated on a non-STIC multi-systemic battery of widely used outcome measures pre-and-post therapy.

RESULTS

STIC clients improved more than TAU clients regardless of treatment modality or outcome measure. Clinically significant change was also greater for STIC than TAU clients across outcome measures.

CONCLUSION

The STIC MFS holds promise for improving outcomes beyond TAU in individual, couple, and family therapy.

Measurement-Based Care in a Remote Intensive Outpatient Program: Pilot Implementation Initiative

BACKGROUND

The ongoing mental health crisis, especially among youth, has led to a greater demand for intensive treatment at the intermediate level, such as intensive outpatient programs (IOPs). Defining best practices in remote IOPs more broadly is critical to understanding the impact of these offerings for individuals with high-acuity mental health service needs in the outpatient setting. Measurement-based care (MBC), or the routine and systematic collection of patient-reported data throughout the course of care to make meaningful changes to treatment, is one such practice that has been shown to improve patient outcomes in mental health treatment. Despite the literature linking MBC to beneficial clinical outcomes, the adoption of MBC in clinical practice has been slow and inconsistent, and more research is needed around MBC in youth-serving settings.

OBJECTIVE

The aim of this paper is to help bridge these gaps, illustrating the implementation of MBC within an organization that provides remote-first, youth-oriented IOP for individuals with high-acuity psychiatric needs.

METHODS

A series of 2 quality improvement pilot studies were conducted with select clinicians and their clients at Charlie Health, a remote IOP program that treats high-acuity teenagers and young adults who present with a range of mental health disorders. Both studies were carefully designed, including thorough preparation and planning, clinician training, feedback collection, and data analysis. Using process evaluation data, MBC deployment was repeatedly refined to enhance the clinical workflow and clinician experience.

RESULTS

The survey completion rate was 80.08% (3216/4016) and 86.01% (4218/4904) for study 1 and study 2, respectively. Quantitative clinician feedback showed marked improvement from study 1 to study 2. Rates of successful treatment completion were 22% and 29% higher for MBC pilot clients in study 1 and study 2, respectively. Depression, anxiety, and psychological well-being symptom reduction were statistically significantly greater for MBC pilot clients (P<.05).

CONCLUSIONS

Our findings support the feasibility and clinician acceptability of a rigorous MBC process in a real-world, youth-serving, remote-first, intermediate care setting. High survey completion data across both studies and improved clinician feedback over time suggest strong clinician buy-in. Client outcomes data suggest MBC is positively correlated with increased treatment completion and symptom reduction. This paper provides practical guidance for MBC implementation in IOPs and can extend to other mental health care settings.

Patient-Reported Outcome Measures as an Intervention: A Comprehensive Overview of Systematic Reviews on the Effects of Feedback

OBJECTIVES

Patient-reported outcome measures (PROMs) have emerged as a promising approach to involve patients in their treatment process. Beyond serving as outcome measures, PROMs can be applied to provide feedback to healthcare providers and patients, thereby offering valuable insights that can improve health outcomes and care processes. This overview offers a comprehensive synthesis of the effects of PROM feedback, contributing to the evidence-based discussion on PROMs' potential to enhance patient care.

METHODS

Following Cochrane Collaboration recommendations, this overview included literature reviews across diverse treatment areas, investigating the impact of PROM feedback on patient health outcomes (including quality of life, symptoms, or survival) and care process outcomes (including communication, symptom identification, or clinical practice). The methodological quality of the evidence was assessed with a modified version of A Measurement Tool to Assess Systematic Reviews 2, and the potential overlap of primary studies was quantified. Results were narratively synthesized.

RESULTS

Forty reviews grouped into 4 categories of treatment areas were included. Overall, their methodological quality was low. The overall overlap of primary studies was 2.2%, reaching up to 15.7% within specific treatment areas. The results indicate that PROM feedback may enhance the quality-of-care processes, whereas its effects on patient health outcomes remained less conclusive.

CONCLUSIONS

PROM feedback positively influences the interaction between physicians and patients across the included treatment areas. Further research is needed to comprehend the trickle-down effects of PROM feedback and how to enhance its potential in yielding health benefits for patients.

A guide for self-help guides: best practice implementation

Guided self-help is an evidence-based intervention used globally. Self-help is a fundamental part of the stepped care model of mental health services that enables the efficient use of limited resources. Despite its importance, there is little information defining the role of the guide and the key competences required. In this context, the guide is defined as the person who facilitates and supports the use self-help materials. This article sets out the role of the guide in guided self-help. It considers practical issues such as the importance of engagement to motivate clients for early change, personalising the intervention, structuring sessions, how best to use routine outcome monitoring and supervision requirements. Key competences are proposed, including generic competences to build the relationship as well as specific competences such as being able to clearly convey the role of the guide to clients. Guides should be prepared for "self-help drift", a concept akin to therapist drift in more traditional therapies. Knowing how to identify mental health problems, use supervision and manage risk and comorbidity are all key requirements for guides. The paper concludes by calling for increased recognition and value of the role of the guide within mental health services.

Precision Mental Health and Data-Informed Decision Support in Psychological Therapy: An Example

Outcome measurement including data-informed decision support for therapists in psychological therapy has developed impressively over the past two decades. New technological developments such as computerized data assessment, and feedback tools have facilitated advanced implementation in several seetings. Recent developments try to improve the clinical decision-making process by connecting clinical practice better with empirical data. For example, psychometric data can be used by clinicians to personalize the selection of therapeutic programs, strategies or modules and to monitor a patient's response to therapy in real time. Furthermore, clinical support tools can be used to improve the treatment for patients at risk for a negative outcome. Therefore, measurement-based care can be seen as an important and integral part of clinical competence, practice, and training. This is comparable to many other areas in the healthcare system, where continuous monitoring of health indicators is common in day-to-day clinical practice (e.g., fever, blood pressure). In this paper, we present the basic concepts of a data-informed decision support system for tailoring individual psychological interventions to specific patient needs, and discuss the implications for implementing this form of precision mental health in clinical practice.

Using the 'Leapfrog' Design as a Simple Form of Adaptive Platform Trial to Develop, Test, and Implement Treatment Personalization Methods in Routine Practice

The route for the development, evaluation and dissemination of personalized psychological therapies is complex and challenging. In particular, the large sample sizes needed to provide adequately powered trials of newly-developed personalization approaches means that the traditional treatment development route is extremely inefficient. This paper outlines the promise of adaptive platform trials (APT) embedded within routine practice as a method to streamline development and testing of personalized psychological therapies, and close the gap to implementation in real-world settings. It focuses in particular on a recently-developed simplified APT design, the 'leapfrog' trial, illustrating via simulation how such a trial may proceed and the advantages it can bring, for example in terms of reduced sample sizes. Finally it discusses models of how such trials could be implemented in routine practice, including potential challenges and caveats, alongside a longer-term perspective on the development of personalized psychological treatments.

Patients' and Therapists' Experiences of Standardized Group Cognitive Behavioral Therapy: Needs for a Personalized Approach

Personalizing psychotherapy can be challenging within standardized group Cognitive Behavioral Therapy (CBT), in which sessions are structured according to a protocol and must accommodate the needs and preferences of multiple patients. In the current study, we aimed to examine patients' and therapists' experiences of standardized group CBT and identify their perceptions of different patient needs. Furthermore, we explored how these needs can inform possible content of add-on interventions for patients who are not improving as expected during group CBT.We conducted 21 individual in-depth interviews with patients with depression and their therapists about their experiences during group CBT with Routine Outcome Monitoring (ROM) and feedback. Interviews were analyzed by using a hermeneutic-phenomenological thematic analysis. Five themes, representing different patient needs, were identified: (1) Individual attention, (2) Psychological exploration, (3) A focus on the patient's life outside of therapy, (4) Extended assessment, and (5) Agreement on therapeutic tasks.The study supports that "one size does not fit all" when it comes to psychotherapy. Patients have varying needs when they are not making progress in therapy, and these needs, when unmet, can negatively impact the overall experience of group CBT. By acknowledging the unique needs of each patient and providing additional individual sessions as necessary, we can move towards a more personalized approach that maximizes the benefits of group psychotherapy.

A systematic review of predictors and moderators of treatment outcomes in internet- and mobile-based interventions for depression

This systematic review aimed to synthesize evidence on predictors and moderators of treatment outcomes in internet- and mobile-based interventions (IMIs) for depression, informing personalized care. A systematic search across PubMed, PsycInfo, and Cochrane yielded 33,002 results. Two reviewers independently performed screening, data extraction, risk of bias assessment, and methodological quality evaluation. Fifty-eight single studies (m = 466 analyses) focusing on baseline-predictors (59.7 %, m = 278), process-predictors (16.5 %, m = 77), and moderators (21.9 %, m = 102), and six individual patient data meta-analyses (m = 93) were included. Only 24.0 % (m = 112/466) of analyses in single studies and 15.1 % (m = 14/93) in individual patient data meta-analyses were significant. Evidence from single studies was rated as insufficient for all variable categories with only 2 out of 40 categories showing >50 % significant results. Baseline depression severity had the strongest predictive value with higher scores linked to better outcomes followed by variables indicative for the course-of-change. Other frequently analyzed and potentially relevant variables with significant results were adherence, age, educational level, ethnicity, relationship status, treatment history, and behavioral variables. More high quality quantitative studies with sufficient power are essential to validate and expand findings, identifying predictors and moderators specifically relevant in IMIs to explain differential treatment effects.

Early Change as a Predictor of Treatment Outcome in Patients with a Personality Disorder

A significant proportion of patients with a personality disorder do not benefit from treatment. Monitoring treatment progress can help adjust ineffective treatments. This study examined whether early changes in symptoms and personality dysfunction during the first phase of therapy could predict treatment outcomes. Data from 841 patients who received specialized treatment for personality disorders were analyzed. The study focused on whether changes in the Outcome Questionnaire-45.2 (OQ-45.2) symptom distress scale (SD), the General Assessment of Personality Disorder (GAPD), and Severity Indices of Personality Problems (SIPP) in the early phase of therapy predicted post-treatment personality dysfunction, as measured by the SIPP and GAPD. Early changes within a specific SIPP domain were the strongest predictors of post-treatment outcomes in that same domain. Early changes in symptoms significantly predicted outcomes in Self-Control, Relational Functioning, and Identity Integration, while the GAPD predicted outcomes in Self-Control and Social Attunement on the SIPP. For the GAPD, early changes on the GAPD itself, followed by early changes on the OQ-45 SD and the SIPP domain Social Attunement, were significant predictors. Thus, when it comes to personality dysfunction, early changes in a specific domain or measure are the best predictors of outcomes in that same domain. While the OQ-45 predicted some aspects of personality dysfunction, it should not replace disorder-specific measures. Additionally, the SIPP domains and the GAPD should not be used interchangeably to predict each other. In sum, considering these factors, monitoring early change can be useful in assessing progress in the treatment of patients with personality disorders.

Evaluation of completely online psychotherapy with app-support versus therapy as usual for clients with depression or anxiety disorder: A retrospective matched cohort study investigating the effectiveness, efficiency, client satisfaction, and costs

INTRODUCTION

Depressive and anxiety disorders are common mental disorders ranking among the leading causes of global disease burden. Not all clients currently benefit from therapy and clients are looking for modern ways of therapy. Online psychotherapy is a promising option for better meeting clients' needs. Recently, a new psychotherapy concept has emerged that combines videoconferencing sessions with support through a mobile application. The latter allows for ecological momentary assessments and interventions, facilitates communication between patients and therapists in between sessions through chat, and allows for incorporating feedback-informed treatment principles.

MATERIAL AND METHODS

The study was a retrospective observational matched cohort study, comparing online psychotherapy with Therapy As Usual (TAU) for clients with depressive or anxiety disorders. Data were obtained via questionnaires, which are part of standard clinical care. Primary outcomes included general mental functioning, and symptoms of depression and anxiety. Secondary outcomes were efficiency, client satisfaction, and therapy costs. Primary endpoints were analyzed using linear mixed models analysis, with an interaction term between time and group. Secondary outcomes were analyzed using linear regression.

RESULTS

Larger improvements were observed in the online compared to the TAU group for general mental functioning and depressive disorder (i.e., General mental functioning: B = -8.50, 95 CI: -15.01 - -1.97, p = 0.011; Depressive disorder: B = - 3.66, 95 % CI: -5.79 - -1.54p < 0.01). No significant differences in change over time between the two groups were observed for anxiety disorder (B = -3.64, 95 % CI: (-13.10 - 5.82) p = 0.447). The total number of sessions was significantly higher in the online psychotherapy group than in TAU (B = 3.71, p < 0.01), although clients were matched on treatment time in weeks. Treatment session duration in minutes was comparable across the groups.

DISCUSSION

Online psychotherapy with app support showed to be a promising alternative to TAU for depressive and anxiety disorders. More research is needed to evaluate the effectiveness, cost-effectiveness and client satisfaction of online psychotherapy compared to TAU, such as randomized controlled trials or studies multiple baseline series designs, and in-depth qualitative research.

Prediction of adolescent psychotherapy outcomes using youth- and caregiver-reported symptoms data

OBJECTIVE

We used longitudinal youth- and caregiver-reports of adolescent psychological symptoms from three samples of youth receiving mental health services in routine treatment settings to derive expected change trajectories and identify cases at risk for treatment failure.

METHOD

Participants were 1906 youth (1053 caregivers) receiving treatment in community mental health settings, merged across three samples. The Symptoms and Functioning Severity Scale (SFSS) was used as an indicator of weekly clinical change. Multilevel modeling methods were used to develop expected change trajectories and identify cases at risk for treatment failure (not on track; NOT). Logistic regression was used to predict client improvement as a function of NOT status.

RESULTS

The SFSS was a reliable indicator of therapeutic change according to youth-reported symptoms. Caregiver reports were not as robust. Whereas predictive accuracy of NOT status yielded moderately high sensitivity in detecting improvement according to youth report, caregiver reports were not as predictive.

CONCLUSIONS

The youth-reported version of the SFSS-based algorithm seems appropriate for implementation in clinical care. Future studies should search for similarly predictive measures for caregivers.