Behind the Scenes of Online Therapeutic Feedback in Blended Therapy for Depression: Mixed-Methods Observational Study

Implementing internet-delivered cognitive behavioral therapy in healthcare services: a qualitative exploration of stakeholder experience

Background

This study consisted of a qualitative exploration of stakeholder experience regarding the implementation of internet-delivered cognitive behavioral therapy (iCBT) as part of routine service provision within the UK's Improving Access to Psychological Therapies program.

Methods

Stakeholder groups included service providers (n = 6), commercial iCBT representatives (n = 6) and patients who received a course of iCBT as part of treatment at the service (n = 7). Participants took part in a semi-structured interview over the telephone, and subsequent data were qualitatively analyzed using the descriptive-interpretive approach.

Results

Service providers highlighted the importance of effective leadership and management, training initiatives, the provision of feedback to commercial iCBT representatives and creating work structures around iCBT to facilitate iCBT supporting staff in their use of it. Commercial iCBT representatives similarly reported the importance of training clinicians in iCBT use, identifying the appropriate individuals at all levels of the service to drive iCBT implementation, and the importance of being responsive to any problems or needs that arise from the service. Patients reported an overall positive experience of receiving iCBT but highlighted a need for more information from their supporter and the intervention to better structure their usage. Contextual factors, in terms of barriers and facilitators, were also highlighted by service provider and commercial participants; citing negative clinician attitudes and technological issues/bugs as barriers to implementation, and the exposure to iCBT created by COVID-19 and perseverance in using iCBT over time as facilitators.

Discussion

The findings contribute to a growing field of literature that seeks to understand the experience of relevant stakeholders who are involved with and contribute to iCBT implementation, including commercial iCBT representatives who, to our knowledge, have not been accounted for as part of published research to date.

Personalization strategies in digital mental health interventions: a systematic review and conceptual framework for depressive symptoms

Introduction

Personalization is a much-discussed approach to improve adherence and outcomes for Digital Mental Health interventions (DMHIs). Yet, major questions remain open, such as (1) what personalization is, (2) how prevalent it is in practice, and (3) what benefits it truly has.

Methods

We address this gap by performing a systematic literature review identifying all empirical studies on DMHIs targeting depressive symptoms in adults from 2015 to September 2022. The search in Pubmed, SCOPUS and Psycinfo led to the inclusion of 138 articles, describing 94 distinct DMHIs provided to an overall sample of approximately 24,300 individuals.

Results

Our investigation results in the conceptualization of personalization as purposefully designed variation between individuals in an intervention's therapeutic elements or its structure. We propose to further differentiate personalization by what is personalized (i.e., intervention content, content order, level of guidance or communication) and the underlying mechanism [i.e., user choice, provider choice, decision rules, and machine-learning (ML) based approaches]. Applying this concept, we identified personalization in 66% of the interventions for depressive symptoms, with personalized intervention content (32% of interventions) and communication with the user (30%) being particularly popular. Personalization via decision rules (48%) and user choice (36%) were the most used mechanisms, while the utilization of ML was rare (3%). Two-thirds of personalized interventions only tailored one dimension of the intervention.

Discussion

We conclude that future interventions could provide even more personalized experiences and especially benefit from using ML models. Finally, empirical evidence for personalization was scarce and inconclusive, making further evidence for the benefits of personalization highly needed.

Systematic Review Registration

Identifier: CRD42022357408.

Blending Internet-based and tele group treatment: Acceptability, effects, and mechanisms of change of cognitive behavioral treatment for depression

The current COV-19 pandemic increases the need for remote treatment. Among several provision strategies, tele groups have been tested as an efficient option. Still, the number of studies is comparably low, with a clear lack of studies investigating supposed treatment mechanisms. Sixty-one mildly to moderately depressed participants from Salzburg, Bavaria, and Upper Austria were randomized to the intervention or a waiting list control group (RCT). The seven-week treatment comprised preparatory online modules, followed by personalized feedback and a subsequent tele group session. Large treatment effects were observed for depression (CES-D: d = 0.99, p < .001; PHQ-9: d = 0.87, p = .002), together with large effects for cognitive behavioral skills (cognitive style, and behavioral activation, d = 0.88-0.97). Changes in skills mediated treatment outcomes for CES-D and PHQ-9, suggesting comparable mechanisms as in face-to-face therapy. Two typical moderators, therapeutic alliance, and group cohesion, however, failed to predict outcome (p = .289), or only exhibited statistical tendencies (p = .049 to .071). Client satisfaction, system usability, and treatment adherence were high. Blending Internet-based and tele group interventions offers additional options for low-threshold care that is less dependent on population density, commuting distances, or constraints due to the current COV-19 crisis. Results indicate that the blended intervention is clinically effective by fostering core CBT skills. While findings suggest the notion that working alliance and group cohesion can be established online, their relevancy for outcomes of blended treatment needs to be further investigated.

Does it blend? Exploring therapist fidelity in blended CBT for anxiety disorders

Blended cognitive-behavioural therapy (bCBT) combines face-to-face CBT (FtFCBT) and Internet-based CBT (iCBT) into one integrated treatment protocol, opening up new ways to deliver therapy, increase cost-effectiveness and resolve scarcity of therapist availability. When traditional therapy is transformed into a new format, there is a need to evaluate whether principles of the new protocol are consistently applied. This study aimed to explore therapist fidelity to bCBT protocols for anxiety disorders in specialised mental health care and to assess whether fidelity is related to patient characteristics. Adult patients (N = 44) received bCBT within a randomised controlled trial. Ratio of FtF to online sessions, session frequency and therapist adherence to instructions were assessed. Overall therapist fidelity with regard to ratio of blending, session frequency and instructions was high. Correlations were found between patients' share of online sessions and both session frequency (r = 0.373, p = .013), as well as patient computer experience (r = 0.314, p = .038). Adherence to instructions in FtF sessions was based on a subset of patients (n = 23) and should therefore be interpreted with caution. The blended approach was generally delivered as intended, indicating that the format is feasible in specialised mental health.

The working alliance in blended versus face-to-face cognitive therapy for depression: A secondary analysis of a randomized controlled trial

INTRODUCTION

We explored the working alliance as measured by both clients and therapists. The working alliance has been known to predict the outcome of psychotherapy and is often considered an important common factor. This study raised the question of how to conceptualize the working alliance in the blended format.

METHODS

This was an exploratory study derived from a randomized controlled trial comparing bCBT and face-to-face cognitive behavioural therapy (ftf CBT) on depression. The change in depressive symptoms was measured with the Patient Health Questionnaire (PHQ-9) and the working alliance was measured using the Working Alliance Inventory.

ANALYSES

Correlation coefficients were calculated for the working alliance as reported by clients and therapists, working alliance was then examined at item and dimension level (task, bond, goal). Linear regression models were applied to investigate the predictive value of the working alliance on treatment outcome. Interaction between the level of working alliance and treatment condition was also considered.

RESULTS

Client and therapist working alliance ratings correlated at r = 0.44 and clients rated the working alliance higher than therapists (x̄ _clients  = 48.6, x̄ _therapists  = 44.6). ftf CBT and bCBT had comparable joint working alliance ratings (x̄ _ftf  = 46.4, x̄ _bCBT  = 46.8). Items had little deviation apart from item 4 with high positive values. The level of working alliance from the total sample did not significantly predict outcome based on the client's perspective. Contrarily, seen from the therapist's perspective, it did (b = 0.00, p = .044). Using the mean from these predictors as a composite variable, was also significant (b = 0.00, p = .039, R² _adj  = 0.07). There was no significant interaction with treatment condition.

DISCUSSION

Clients and therapists may lay emphasis on different aspects of the working alliance. The finding that therapist-rated working alliance was better than client working alliance at predicting outcome went against common findings; this pattern may be specific to bCBT.

CONCLUSION

Clients rated the working alliance slightly higher than therapists on average. Clients and therapists as well as treatment conditions had different profiles on dimension deviations. Therapist ratings of the working alliance appeared to better predict treatment outcome than client ratings. Joint working alliance predicted outcome using client and therapist composite means. The working alliance was equally strong in ftf CBT and bCBT. The absence of interaction with treatment condition indicated that the working alliance was equally predictive of outcome in ftf CBT as in bCBT.

Improving adherence to an online intervention for low mood with a virtual coach: study protocol of a pilot randomized controlled trial

Background

Internet-based cognitive-behavioral therapy (iCBT) is more effective when it is guided by human support than when it is unguided. This may be attributable to higher adherence rates that result from a positive effect of the accompanying support on motivation and on engagement with the intervention. This protocol presents the design of a pilot randomized controlled trial that aims to start bridging the gap between guided and unguided interventions. It will test an intervention that includes automated support delivered by an embodied conversational agent (ECA) in the form of a virtual coach.

Methods/design

The study will employ a pilot two-armed randomized controlled trial design. The primary outcomes of the trial will be (1) the effectiveness of iCBT, as supported by a virtual coach, in terms of improved intervention adherence in comparison with unguided iCBT, and (2) the feasibility of a future, larger-scale trial in terms of recruitment, acceptability, and sample size calculation. Secondary aims will be to assess the virtual coach’s effect on motivation, users’ perceptions of the virtual coach, and general feasibility of the intervention as supported by a virtual coach. We will recruit N = 70 participants from the general population who wish to learn how they can improve their mood by using Moodbuster Lite, a 4-week cognitive-behavioral therapy course. Candidates with symptoms of moderate to severe depression will be excluded from study participation. Included participants will be randomized in a 1:1 ratio to either (1) Moodbuster Lite with automated support delivered by a virtual coach or (2) Moodbuster Lite without automated support. Assessments will be taken at baseline and post-study 4 weeks later.

Discussion

The study will assess the preliminary effectiveness of a virtual coach in improving adherence and will determine the feasibility of a larger-scale RCT. It could represent a significant step in bridging the gap between guided and unguided iCBT interventions.

Trial registration

Netherlands Trial Register (NTR) NL8110. Registered on 23 October 2019.

Role of Moderators on Engagement of Adolescents With Depression or Anxiety in a Social Media Intervention: Content Analysis of Web-Based Interactions

BACKGROUND

The Supporting Our Valued Adolescents (SOVA) intervention aims to use a moderated social media website to encourage peer discussion about negative health beliefs, which may prevent treatment uptake. Web moderators with a background in behavioral health are used to facilitate peer conversation to promote a sense of community, provide social support, and ensure safety.

OBJECTIVE

Although moderation is a core component of this intervention, little is known on best practices for moderators to ensure safety while encouraging engagement. This study sought to describe interactions between moderators and peer users and understand moderator experiences through individual interviews.

METHODS

Adolescents and young adults aged 14 to 26 years with depression or anxiety history were recruited for a usability study of the SOVA intervention. During this study, 14 moderators were trained to regularly review comments to blog posts for safety, facilitate conversation, and correct misinformation. A total of 110 blog posts and their associated comments were extracted and coded using a codebook based on items from the supportive accountability model and a peer social support analysis. Closing interviews with 12 moderators assessing their experience of moderating were conducted, recorded, and transcribed. Blog post text and comments as well as transcripts of moderator interviews were assessed using a thematic analysis approach, and blog posts were examined for trends in content of moderator comments comparing blog posts with differences in comment contributor order.

RESULTS

There were no safety concerns during the study, and moderators only intervened to remove identifiable information. Web moderators exhibited elements of supportive accountability (such as being perceived as experts and using verbal rewards as well as offering informational and emotional support). When the moderators provided the last comment under a blog post, thereby potentially ending contribution by users, they were at times found to be commenting about their own experiences. Moderators interviewed after completing their role expressed challenges in engaging users. A cohort of moderators who received more extensive training on supportive accountability and peer social support felt their ability to engage users improved because of the training.

CONCLUSIONS

Moderators of a Web-based support site for adolescents with depression or anxiety were able to ensure safety while promoting user engagement. Moderators can elicit user engagement by offering gratitude and encouragement to users, asking users follow-up questions, and limiting their own opinions and experiences when responding to comments.

From Research to Practice: Ten Lessons in Delivering Digital Mental Health Services

There is a large body of research showing that psychological treatment can be effectively delivered via the internet, and Digital Mental Health Services (DMHS) are now delivering those interventions in routine care. However, not all attempts to translate these research outcomes into routine care have been successful. This paper draws on the experience of successful DMHS in Australia and Canada to describe ten lessons learned while establishing and delivering internet-delivered cognitive behavioural therapy (ICBT) and other mental health services as part of routine care. These lessons include learnings at four levels of analysis, including lessons learned working with (1) consumers, (2) therapists, (3) when operating DMHS, and (4) working within healthcare systems. Key themes include recognising that DMHS should provide not only treatment but also information and assessment services, that DMHS require robust systems for training and supervising therapists, that specialist skills are required to operate DMHS, and that the outcome data from DMHS can inform future mental health policy. We also confirm that operating such clinics is particularly challenging in the evolving funding, policy, and regulatory context, as well as increasing expectations from consumers about DMHS. Notwithstanding the difficulties of delivering DMHS, we conclude that the benefits of such services for the broader community significantly outweigh the challenges.

Unraveling the Black Box: Exploring Usage Patterns of a Blended Treatment for Depression in a Multicenter Study

BACKGROUND

Blended treatments, combining digital components with face-to-face (FTF) therapy, are starting to find their way into mental health care. Knowledge on how blended treatments should be set up is, however, still limited. To further explore and optimize blended treatment protocols, it is important to obtain a full picture of what actually happens during treatments when applied in routine mental health care.

OBJECTIVE

The aims of this study were to gain insight into the usage of the different components of a blended cognitive behavioral therapy (bCBT) for depression and reflect on actual engagement as compared with intended application, compare bCBT usage between primary and specialized care, and explore different usage patterns.

METHODS

Data used were collected from participants of the European Comparative Effectiveness Research on Internet-Based Depression Treatment project, a European multisite randomized controlled trial comparing bCBT with regular care for depression. Patients were recruited in primary and specialized routine mental health care settings between February 2015 and December 2017. Analyses were performed on the group of participants allocated to the bCBT condition who made use of the Moodbuster platform and for whom data from all blended components were available (n=200). Included patients were from Germany, Poland, the Netherlands, and France; 64.5% (129/200) were female and the average age was 42 years (range 18-74 years).

RESULTS

Overall, there was a large variability in the usage of the blended treatment. A clear distinction between care settings was observed, with longer treatment duration and more FTF sessions in specialized care and a more active and intensive usage of the Web-based component by the patients in primary care. Of the patients who started the bCBT, 89.5% (179/200) also continued with this treatment format. Treatment preference, educational level, and the number of comorbid disorders were associated with bCBT engagement.

CONCLUSIONS

Blended treatments can be applied to a group of patients being treated for depression in routine mental health care. Rather than striving for an optimal blend, a more personalized blended care approach seems to be the most suitable. The next step is to gain more insight into the clinical and cost-effectiveness of blended treatments and to further facilitate uptake in routine mental health care.

Development of the Internet-Delivered Cognitive Behaviour Therapy Undesirable Therapist Behaviours Scale (ICBT-UTBS)

Internet-delivered cognitive behaviour therapy (ICBT) is often provided with therapist assistance via asynchronous secure emails, but there is limited research on undesirable behaviours exhibited by therapists in their correspondence with patients. In this study, an ICBT-Undesirable Therapist Behaviour Scale (ICBT-UTBS) was developed and used to assess the nature, frequency, and correlates of undesirable therapist behaviours in routine practice. Thematic analysis was used to identify undesirable therapist behaviours in 720 emails sent to 91 randomly selected patients in the context of a previous clinical trial of transdiagnostic ICBT for depression and anxiety. The following undesirable behaviours were identified, albeit infrequently, in therapist emails: inadequate detail (6.4%), unaddressed content (4.0%), unsupportive tone (0.6%), missed correspondence (0.6%), inappropriate self-disclosure (0.6%), and unmanaged risk (0.3%). At least one undesirable behaviour was found in 10.7% of all emails coded. Moreover, 37.4% of patients received at least one email containing an undesirable therapist behaviour. Number of undesirable therapist behaviours was not correlated with patient engagement, working alliance, treatment satisfaction, or patient outcome variables. However, undesirable therapist behaviours were negatively correlated with patient gender and therapist characteristics (e.g., clinical setting, therapist profession). The results of the present study provide preliminary psychometric support for the ICBT-UTBS, a measure of ICBT treatment integrity. In the future, the ICBT-UTBS should be used in combination with the ICBT-Therapist Rating Scale (ICBT-TRS), a measure of desirable or recommended therapist behaviours, for training purposes and to monitor ICBT therapists in routine practice.

Validating Automated Sentiment Analysis of Online Cognitive Behavioral Therapy Patient Texts: An Exploratory Study

Introduction

Sentiment analysis may be a useful technique to derive a user’s emotional state from free text input, allowing for more empathic automated feedback in online cognitive behavioral therapy (iCBT) interventions for psychological disorders such as depression. As guided iCBT is considered more effective than unguided iCBT, such automated feedback may help close the gap between the two. The accuracy of automated sentiment analysis is domain dependent, and it is unclear how well the technology is applicable to iCBT. This paper presents an empirical study in which automated sentiment analysis by an algorithm for the Dutch language is validated against human judgment.

Methods

A total of 493 iCBT user texts were evaluated on overall sentiment and the presence of five specific emotions by an algorithm, and by 52 psychology students who evaluated 75 randomly selected texts each, providing about eight human evaluations per text. Inter-rater agreement (IRR) between algorithm and humans, and humans among each other, was analyzed by calculating the intra-class correlation under a numerical interpretation of the data, and Cohen’s kappa, and Krippendorff’s alpha under a categorical interpretation.

Results

All analyses indicated moderate agreement between the algorithm and average human judgment with respect to evaluating overall sentiment, and low agreement for the specific emotions. Somewhat surprisingly, the same was the case for the IRR among human judges, which means that the algorithm performed about as well as a randomly selected human judge. Thus, considering average human judgment as a benchmark for the applicability of automated sentiment analysis, the technique can be considered for practical application.

Discussion/Conclusion

The low human-human agreement on the presence of emotions may be due to the nature of the texts, it may simply be difficult for humans to agree on the presence of the selected emotions, or perhaps trained therapists would have reached more consensus. Future research may focus on validating the algorithm against a more solid benchmark, on applying the algorithm in an application in which empathic feedback is provided, for example, by an embodied conversational agent, or on improving the algorithm for the iCBT domain with a bottom-up machine learning approach.

Toward an Objective Assessment of Implementation Processes for Innovations in Health Care: Psychometric Evaluation of the Normalization Measure Development (NoMAD) Questionnaire Among Mental Health Care Professionals

Background

Successfully implementing eMental health (eMH) interventions in routine mental health care constitutes a major challenge. Reliable instruments to assess implementation progress are essential. The Normalization MeAsure Development (NoMAD) study developed a brief self-report questionnaire that could be helpful in measuring implementation progress. Based on the Normalization Process Theory, this instrument focuses on 4 generative mechanisms involved in implementation processes: coherence, cognitive participation, collective action, and reflexive monitoring.

Objective

The aim of this study was to translate the NoMAD questionnaire to Dutch and to confirm the factor structure in Dutch mental health care settings.

Methods

Dutch mental health care professionals involved in eMH implementation were invited to complete the translated NoMAD questionnaire. Confirmatory factor analysis (CFA) was conducted to verify interpretability of scale scores for 3 models: (1) the theoretical 4-factor structure, (2) a unidimensional model, and (3) a hierarchical model. Potential improvements were explored, and correlated scale scores with 3 control questions were used to assess convergent validity.

Results

A total of 262 professionals from mental health care settings in the Netherlands completed the questionnaire (female: 81.7%; mean age: 45 [SD=11]). The internal consistency of the 20-item questionnaire was acceptable (.62≤alpha≤.85). The theorized 4-factor model fitted the data slightly better in the CFA than the hierarchical model (Comparative Fit Index=0.90, Tucker Lewis Index=0.88, Root Mean Square Error of Approximation=0.10, Standardized Root Mean Square Residual=0.12, χ²₂=22.5, P≤.05). However, the difference is small and possibly not outweighing the practical relevance of a total score and subscale scores combined in one hierarchical model. One item was identified as weak (λ_CA.2=0.10). A moderate-to-strong convergent validity with 3 control questions was found for the Collective Participation scale (.47≤r≤.54, P≤.05).

Conclusions

NoMAD’s theoretical factor structure was confirmed in Dutch mental health settings to acceptable standards but with room for improvement. The hierarchical model might prove useful in increasing the practical utility of the NoMAD questionnaire by combining a total score with information on the 4 generative mechanisms. Future research should assess the predictive value and responsiveness over time and elucidate the conceptual interpretability of NoMAD in eMH implementation practices.

Guided Internet-Based Cognitive Behavioral Therapy for Adult Depression and Anxiety in Routine Secondary Care: Observational Study

BACKGROUND

Internet-based cognitive behavioral therapy (iCBT) is a promising new treatment method for depression and anxiety. However, it is important to determine whether its results can be replicated in routine care before its implementation on a large scale. Although many studies have demonstrated the efficacy of iCBT under controlled conditions, only a few studies have investigated its effectiveness in routine care. Furthermore, several effects of iCBT such as treatment effects in routine care are unclear.

OBJECTIVE

This study aimed to evaluate the clinical effectiveness of iCBT for depression and anxiety in routine secondary care.

METHODS

n a retrospective cohort study, we analysed patients treated for depression or anxiety in a dedicated iCBT clinic in secondary care in Denmark. Patients were examined before treatment and weekly thereafter by using the Patient Health Questionnaire-9 and the Generalized Anxiety Disorder-7 scales for the diagnoses of depression and anxiety, respectively. Primary analyses were conducted using a linear mixed-effects model with random slope and intercept. Secondary analyses were conducted using baseline characteristics as predictors (gender, age, highest level of education, occupational status, marital status, psychotropic medication use, consumption of alcohol, and leisure drugs). Additionally, logistic regression analyses were used to predict noncompletion of treatment.

RESULTS

A total of 203 (depression, N=60; anxiety, N=143) patients were included. Participants were mainly female (78.3% with depression and 65.7% with anxiety), with a mean age of 36.03 (SD 10.97) years (range, 19-67 years) for patients with depression and 36.80 (SD 13.55) years (range, 19-69 years) for patients with anxiety. The completion rates were 62% (37) and 40% (57) for depression and anxiety treatments, respectively. The primary analyses revealed large and significant reductions in the symptom levels of depression (beta=-6.27, SE 0.83, P<.001, d=1.0) and anxiety (beta=-3.78, SE 0.43, P<.001, d=1.1). High baseline severity of the primary disorder was associated with high treatment gains (r=-0.31 for depression; r=-0.41 for anxiety). In patients with anxiety, high baseline severity also predicted a high risk of noncompletion (odds ratio=1.08, CI=1.01-1.16, P=.03). An increase in the baseline severity of the comorbid disorder slightly increased the risk of noncompletion for both disorders (depression: odds ratio=1.03, CI=1.01-1.06, P=.02; anxiety: odds ratio=1.08, CI=1.01-1.16, P=.03).

CONCLUSIONS

iCBT can be clinically effective in routine care. Since depression and anxiety are costly and debilitating disorders that are vastly undertreated, this finding is important. Additionally, iCBT may help bridge the gap between the need for treatment and its provision. Our results are comparable to the within-group results of efficacy and effectiveness studies. Our noncompletion rates are similar to those observed in psychotherapy but are higher than those reported in similar clinics. Multiple factors predicted outcome and noncompletion. However, all predictor effects were statistically weak.