Assessing Therapeutic Alliance in the Context of mHealth Interventions for Mental Health Problems: Development of the Mobile Agnew Relationship Measure (mARM) Questionnaire

Quality assessment of mHealth apps: a scoping review

Introduction

The number of mHealth apps has increased rapidly during recent years. Literature suggests a number of problems and barriers to the adoption of mHealth apps, including issues such as validity, usability, as well as data privacy and security. Continuous quality assessment and assurance systems might help to overcome these barriers. Aim of this scoping review was to collate literature on quality assessment tools and quality assurance systems for mHealth apps, compile the components of the tools, and derive overarching quality dimensions, which are potentially relevant for the continuous quality assessment of mHealth apps.

Methods

Literature searches were performed in Medline, EMBASE and PsycInfo. Articles in English or German language were included if they contained information on development, application, or validation of generic concepts of quality assessment or quality assurance of mHealth apps. Screening and extraction were carried out by two researchers independently. Identified quality criteria and aspects were extracted and clustered into quality dimensions.

Results

A total of 70 publications met inclusion criteria. Included publications contain information on five quality assurance systems and further 24 quality assessment tools for mHealth apps. Of these 29 systems/tools, 8 were developed for the assessment of mHealth apps for specific diseases, 16 for assessing mHealth apps for all fields of health and another five are not restricted to health apps. Identified quality criteria and aspects were extracted and grouped into a total of 14 quality dimensions, namely "information and transparency", "validity and (added) value", "(medical) safety", "interoperability and compatibility", "actuality", "engagement", "data privacy and data security", "usability and design", "technology", "organizational aspects", "social aspects", "legal aspects", "equity and equality", and "cost(-effectiveness)".

Discussion

This scoping review provides a broad overview of existing quality assessment and assurance systems. Many of the tools included cover only a few dimensions and aspects and therefore do not allow for a comprehensive quality assessment or quality assurance. Our findings can contribute to the development of continuous quality assessment and assurance systems for mHealth apps.

Systematic Review Registration

https://www.researchprotocols.org/2022/7/e36974/, International Registered Report Identifier, IRRID (DERR1-10.2196/36974).

Acceptability and experience of a smartphone symptom monitoring app for people with psychosis in China (YouXin): a qualitative study

BACKGROUND

Access to high-quality mental healthcare remains challenging for people with psychosis globally, including China. Smartphone-based symptom monitoring has the potential to support scalable mental healthcare. However, no such tool, until now, has been developed and evaluated for people with psychosis in China. This study investigated the acceptability and the experience of using a symptom self-monitoring smartphone app (YouXin) specifically developed for people with psychosis in China.

METHODS

Semi-structured interviews were conducted with 10 participants with psychosis to explore the acceptability of YouXin. Participants were recruited from the non-randomised feasibility study that tested the validity, feasibility, acceptability and safety of the YouXin app. Data analysis was guided by the theoretical framework of acceptability.

RESULTS

Most participants felt the app was acceptable and easy to use, and no unbearable burdens or opportunity costs were reported. Participants found completing the self-monitoring app rewarding and experienced a sense of achievement. Privacy and data security were not major concerns for participants, largely due to trust in their treating hospital around data protection. Participants found the app easy to use and attributed this to the training provided at the beginning of the study. A few participants said they had built some form of relationship with the app and would miss the app when the study finished.

CONCLUSIONS

The YouXin app is acceptable for symptom self-monitoring in people with experience of psychosis in China. Participants gained greater insights about their symptoms by using the YouXin app. As we only collected retrospective acceptability in this study, future studies are warranted to assess hypothetical acceptability before the commencement of study to provide a more comprehensive understanding of implementation.

Development and Evaluation of a Digital App for Patient Self-Management of Opioid Use Disorder: Usability, Acceptability, and Utility Study

BACKGROUND

Self-management of opioid use disorder (OUD) is an important component of treatment. Many patients receiving opioid agonist treatment in methadone maintenance treatment settings benefit from counseling treatments to help them improve their recovery skills but have insufficient access to these treatments between clinic appointments. In addition, many addiction medicine clinicians treating patients with OUD in a general medical clinic setting do not have consistent access to counseling referrals for their patients. This can lead to decreases in both treatment retention and overall progress in the patient's recovery from substance misuse. Digital apps may help to bridge this gap by coaching, supporting, and reinforcing behavioral change that is initiated and directed by their psychosocial and medical providers.

OBJECTIVE

This study aimed to conduct an acceptability, usability, and utility pilot study of the KIOS app to address these clinical needs.

METHODS

We developed a unique, patient-centered computational software system (KIOS; Biomedical Development Corporation) to assist in managing OUD in an outpatient, methadone maintenance clinic setting. KIOS tracks interacting self-reported symptoms (craving, depressed mood, anxiety, irritability, pain, agitation or restlessness, difficulty sleeping, absenteeism, difficulty with usual activities, and conflicts with others) to determine changes in both the trajectory and severity of symptom patterns over time. KIOS then applies a proprietary algorithm to assess the individual's patterns of symptom interaction in accordance with models previously established by OUD experts. After this analysis, KIOS provides specific behavioral advice addressing the individual's changing trajectory of symptoms to help the person self-manage their symptoms. The KIOS software also provides analytics on the self-reported data that can be used by patients, clinicians, and researchers to track outcomes.

RESULTS

In a 4-week acceptability, usability (mean System Usability Scale-Modified score 89.5, SD 9.2, maximum of 10.0), and utility (mean KIOS utility questionnaire score 6.32, SD 0.25, maximum of 7.0) pilot study of 15 methadone-maintained participants with OUD, user experience, usability, and software-generated advice received high and positive assessment scores. The KIOS clinical variables closely correlated with craving self-report measures. Therefore, managing these variables with advice generated by the KIOS software could have an impact on craving and ultimately substance use.

CONCLUSIONS

KIOS tracks key clinical variables and generates advice specifically relevant to the patient's current and changing clinical state. Patients in this pilot study assigned high positive values to the KIOS user experience, ease of use, and the appropriateness, relevance, and usefulness of the specific behavioral guidance they received to match their evolving experiences. KIOS may therefore be useful to augment in-person treatment of opioid agonist patients and help fill treatment gaps that currently exist in the continuum of care. A National Institute on Drug Abuse-funded randomized controlled trial of KIOS to augment in-person treatment of patients with OUD is currently being conducted.

Experiences of a digital health intervention for young people exposed to technology assisted sexual abuse: a qualitative study

BACKGROUND

There is growing evidence that Technology Assisted Sexual Abuse (TASA) represents a serious problem for large numbers of children. To date, there are very few evidence-based interventions available to young people (YP) after they have been exposed to this form of abuse, and access to support services remains a challenge. Digital tools such as smartphones have the potential to increase access to mental health support and may provide an opportunity for YP to both manage their distress and reduce the possibility of further victimization. The current study explores the acceptability of a digital health intervention (DHI; the i-Minds app) which is a theory-driven, co-produced, mentalization-based DHI designed for YP aged 12-18 who have experienced TASA.

METHODS

Semi-structured interviews were conducted with 15 YP recruited through Child and Adolescent Mental Health Services, a Sexual Assault Referral Centre and an e-therapy provider who had access to the i-Minds app as part of a feasibility clinical trial. Interviews focused on the acceptability and usability of i-Minds and were coded to themes based on the Acceptability of Healthcare Interventions framework.

RESULTS

All participants found the i-Minds app acceptable. Many aspects of the app were seen as enjoyable and useful in helping YP understand their abuse, manage feelings, and change behavior. The app was seen as usable and easy to navigate, but for some participants the level of text was problematic and aspects of the content was, at times, emotionally distressing at times.

CONCLUSIONS

The i-Minds app is useful in the management of TASA and helping change some risk-related vulnerabilities. The app was designed, developed and evaluated with YP who had experienced TASA and this may account for the high levels of acceptability seen.

TRIAL REGISTRATION

The trial was registered on the ISRCTN registry on the 12/04/2022 as i-Minds: a digital intervention for young people exposed to online sexual abuse (ISRCTN43130832).

Longitudinal outcomes of a therapist-supported digital mental health intervention for depression and anxiety symptoms: A retrospective cohort study

PURPOSE

This study examined treatment outcomes (depression and anxiety symptoms) up to 24 months after completion of a therapist-supported digital mental health intervention (DMHI).

METHODS

The sample consisted of 380 participants who participated in an eight-week DMHI from February 6, 2017 to May 20, 2019. Participants reported depression and anxiety symptoms at eight timepoints from baseline to 24 months. Mixed-effects modelling was used to investigate symptom changes over time. The proportion of participants meeting criteria for treatment response, clinically significant change, and remission of depression and anxiety symptoms were calculated, including proportions demonstrating each outcome sustained up to each timepoint.

RESULTS

Multivariate analyses yielded statistically significant reductions in depression (β = -5.40) and anxiety (β = -3.31) symptoms from baseline to end of treatment (8 weeks). Symptom levels remained significantly reduced from baseline through 24 months. The proportion of participants meeting criteria for clinical treatment outcomes remained constant over 24 months, although there were linear decreases in the proportions experiencing sustained clinical outcomes.

CONCLUSIONS

Treatment gains were made for depression and anxiety symptoms at the end of treatment and up to 24 months. Future studies should determine the feasibility of integrating post-treatment programmes into DMHIs to address symptom deterioration.

Mental health service users' experiences of telehealth interventions facilitated during the COVID-19 pandemic and their relevance to nursing: An integrative review

WHAT THIS PAPER ADDS TO EXISTING KNOWLEDGE?: Service users report telehealth interventions to be useful in terms of access and convenience however, a preference for face-to-face interventions remains. Nurses are using telehealth interventions in clinical practice however, further research is necessary in this area as the evidence of their involvement is limited. WHAT ARE THE IMPLICATIONS FOR PRACTICE?: This paper highlights the use of telehealth interventions should augment rather than replace face-to-face care provision.

ABSTRACT

INTRODUCTION: The Covid-19 pandemic saw the swift implementation of physical and social distancing that impacted the way in which mental health services were facilitated. Consequently, telehealth/e-health interventions are increasing in use.

AIM

This integrative review aims to explore existing literature regarding mental health service users' experiences of telehealth interventions facilitated through the COVID-19 pandemic, to determine the visibility of nursing involvement in the facilitation of telehealth interventions and to use these experiences to inform nursing practice.

METHOD

A methodical search of eight (n = 8) academic databases was undertaken using CINAHL, SCOPUS, EMBASE, PsycINFO, Web of Science, Cochrane, MEDLINE and Academic Search Complete between January 2020 and January 2022.

RESULTS

A total of 5133 papers were screened by title and abstract of which (n = 77) progressed for full-text screening. Five (n = 5) papers met the inclusion criteria for this review and results were mapped onto the four meta-paradigms of nursing: person; where the findings discuss the acceptability of telehealth interventions; environment; where the findings highlight barriers and facilitators to the use of telehealth interventions; health; where the findings discuss staff time and logistical issues relating to telehealth interventions and nursing; where the findings centre around the therapeutic relationship.

DISCUSSION

This review highlights there is a paucity of direct evidence relating to nursing involvement in the facilitation of telehealth interventions. However, there are benefits to the use of telehealth interventions that include increased access to services, less perceived stigma and increased engagement which are relevant to nursing practice. A lack of individual contact and concerns relating to infrastructure indicates a fondness for face-to-face interventions remains high.

IMPLICATIONS FOR PRACTICE

There is a need for further research on the role of the nurse in the facilitation of telehealth interventions, specific interventions used and outcomes of such interventions.

German Version of the Mobile Agnew Relationship Measure: Translation and Validation Study

BACKGROUND

The mobile Agnew Relationship Measure (mARM) is a self-report questionnaire for the evaluation of digital mental health interventions and their interactions with users. With the global increase in digital mental health intervention research, translated measures are required to conduct research with local populations.

OBJECTIVE

The aim of this study was to translate and validate the original English version of the mARM into a German version (mARM-G).

METHODS

A total of 2 native German speakers who spoke English as their second language conducted forward translation of the original items. This version was then back translated by 2 native German speakers with a fluent knowledge of English. An independent bilingual reviewer then compared these drafts and created a final German version. The mARM-G was validated by 15 experts in the field of mobile app development and 15 nonexperts for content validity and face validity; 144 participants were recruited to conduct reliability testing as well as confirmatory factor analysis.

RESULTS

The content validity index of the mARM-G was 0.90 (expert ratings) and 0.79 (nonexperts). The face validity index was 0.89 (experts) and 0.86 (nonexperts). Internal consistency for the entire scale was Cronbach α=.91. Confirmatory factor analysis results were as follows: the chi-square statistic to df ratio was 1.66. Comparative Fit Index was 0.87 and the Tucker-Lewis Index was 0.86. The root mean square error of approximation was 0.07.

CONCLUSIONS

The mARM-G is a valid and reliable tool that can be used for future studies in German-speaking countries.

User engagement in a randomised controlled trial for a digital health intervention for early psychosis (Actissist 2.0 trial)

Digital Health Interventions (DHIs) can help support people with mental health problems. Achieving satisfactory levels of patient engagement is a crucial, yet often underexplored, pre-requisite for health improvement. Actissist is a co-produced DHI delivered via a smartphone app for people with early psychosis, based on Cognitive Behaviour Therapy principles. This study describes and compares engagement patterns among participants in the two arms of the Actissist 2.0 randomised controlled trial. Engagement frequency and duration were measured among participants using the Actissist app in the intervention arm (n = 87) and the ClinTouch symptom monitoring only app used as the control condition (n = 81). Overall, 47.1 % of Actissist and 45.7 % of ClinTouch users completed at least a third of scheduled alerts while active in the study. The mean frequency (77.1 versus 60.2 total responses) and the median duration (80 versus 75 days until last response) of engagement were not significantly higher among Actissist users compared to ClinTouch users. Older age, White ethnicity, using their own smartphone device and, among Actissist users, an increased sense of therapeutic alliance were significantly associated with increased engagement. Through exploiting detailed usage data, this study identifies possible participant-level and DHI-level predictors of engagement to inform the practical implementation of future DHIs.

Exploring user experiences of a text message-delivered intervention among individuals on opioid use disorder treatment in Kenya: A qualitative study

Opioid use disorder causes significant burden of disease and treatment comprises pharmacotherapy and psychosocial treatment. Cognitive behavioral therapy is an effective psychosocial intervention used in substance use disorders treatment and can be delivered using digital approach. There is limited use of digital treatment among individuals with opioid use disorder in Kenya. This study aimed to describe the experiences and feedback from participants with opioid use disorder enrolled in a text-message intervention in Kenya. Qualitative data was collected from participants in the intervention arm of a feasibility trial testing a text-message intervention based on cognitive behavioral therapy. Data was collected using open-ended questions in a questionnaire and structured in-depth interviews amongst those who received the intervention. Framework method was applied for analysis. Twenty-four participants (83.3% males) were enrolled with a mean age of 32.5 years (SD9.5). Five themes were identified namely: (1) Gain of cognitive behavioral therapy skills which included: identification and change of substance use patterns; drug refusal skills; coping with craving and self-efficacy; (2) Therapeutic alliance which included: development of a bond and agreement on treatment goals; (3) Feedback on intervention components and delivery such as: frequency, and duration of the text message intervention; (4) Challenges experienced during the intervention such as: technical problems with phones; and barriers related to intervention delivery; (5) Recommendations for improvement of intervention in future implementations. The findings demonstrated participants' satisfaction with intervention, gain of skills to change substance use patterns, highlighted challenges experienced and suggestions on improving the intervention among individuals with opioid use disorder. The feedback and recommendations provided by the participants can guide implementation of such interventions to allow acceptability, effectiveness and sustainability. Trial registration: This study was part of a randomized feasibility trial. Clinical trial registration: Pan African Clinical Trial Registry: Registration number: PACTR202201736072847. Date of registration: 10th January 2022.

Conceptualizing the digital therapeutic alliance in the context of fully automated mental health apps: A thematic analysis

Fully automated mental health apps provide a promising opportunity for increasing access to mental health care and resources. Given this opportunity, continued research into the utility and effectiveness of mental health apps is crucial. Therapeutic alliance (TA) refers to the relationship between a client and a healthcare professional, and has been shown to be an important predictor of clinical outcomes in face-to-face therapy. Given the significance of TA in traditional therapy, it is important to explore whether the notion of a digital therapeutic alliance (DTA) in the context of fully automated mental health apps also plays an important role in clinical outcomes. Current evidence shows that the conceptualization of DTA in the context of fully automated mental health apps can be potentially different to TA in face-to-face therapy. Thus, a new DTA conceptual model is necessary for comprehensively understanding the mechanisms underpinning DTA for fully automated mental health apps. To the best of our knowledge, this is the first study that qualitatively explored the dimensions of a DTA in the context of fully automated mental health apps. We conducted interviews with 20 users of mental health apps to explore the key dimensions comprising DTA in the context of fully automated mental health apps. We found that although conceptualizations of DTA and TA have shared dimensions, flexibility and emotional experiences are unique domains in DTA. On the other hand, although agreement on goals between a therapist and a client is important in face to face therapy, we found that users can have an alliance with an app without a goal. The importance of goal needs further investigations.

Developing a Game (Inner Dragon) Within a Leading Smartphone App for Smoking Cessation: Design and Feasibility Evaluation Study

BACKGROUND

Several stand-alone smartphone apps have used serious games to provide an engaging approach to quitting smoking. So far, the uptake of these games has been modest, and the evidence base for their efficacy in promoting smoking cessation is still evolving. The feasibility of integrating a game into a popular smoking cessation app is unclear.

OBJECTIVE

The aim of this paper was to describe the design and iterative development of the Inner Dragon game within Smoke Free, a smartphone app with proven efficacy, and the results of a single-arm feasibility trial as part of a broad program that seeks to assess the effectiveness of the gamified app for smoking cessation.

METHODS

In phase 1, the study team undertook a multistep process to design and develop the game, including web-based focus group discussions with end users (n=15). In phase 2, a single-arm study of Smoke Free users who were trying to quit (n=30) was conducted to assess the feasibility and acceptability of the integrated game and to establish the feasibility of the planned procedures for a randomized pilot trial.

RESULTS

Phase 1 led to the final design of Inner Dragon, informed by principles from psychology and behavioral economics and incorporating several game mechanics designed to increase user engagement and retention. Inner Dragon users maintain an evolving pet dragon that serves as a virtual avatar for the users' progress in quitting. The phase-2 study established the feasibility of the study methods. The mean number of app sessions completed per user was 13.8 (SD 13.1; median 8; range 1-46), with a mean duration per session of 5.8 (median 1.1; range 0-81.1) minutes. Overall, three-fourths (18/24, 75%) of the participants entered the Inner Dragon game at least once and had a mean of 2.4 (SD 2.4) sessions of game use. The use of Inner Dragon was positively associated with the total number of app sessions (correlation 0.57). The mean satisfaction score of participants who provided ratings (11/24, 46%) was 4.2 (SD 0.6) on a 5-point scale; however, satisfaction ratings for Inner Dragon were only completed by 13% (3/24) of the participants.

CONCLUSIONS

Findings supported further development and evaluation of Inner Dragon as a beneficial feature of Smoke Free. The next step of this study is to conduct a randomized pilot trial to determine whether the gamified version of the app increases user engagement over a standard version of the app.

Waiting for a digital therapist: three challenges on the path to psychotherapy delivered by artificial intelligence

Growing demand for broadly accessible mental health care, together with the rapid development of new technologies, trigger discussions about the feasibility of psychotherapeutic interventions based on interactions with Conversational Artificial Intelligence (CAI). Many authors argue that while currently available CAI can be a useful supplement for human-delivered psychotherapy, it is not yet capable of delivering fully fledged psychotherapy on its own. The goal of this paper is to investigate what are the most important obstacles on our way to developing CAI systems capable of delivering psychotherapy in the future. To this end, we formulate and discuss three challenges central to this quest. Firstly, we might not be able to develop effective AI-based psychotherapy unless we deepen our understanding of what makes human-delivered psychotherapy effective. Secondly, assuming that it requires building a therapeutic relationship, it is not clear whether psychotherapy can be delivered by non-human agents. Thirdly, conducting psychotherapy might be a problem too complicated for narrow AI, i.e., AI proficient in dealing with only relatively simple and well-delineated tasks. If this is the case, we should not expect CAI to be capable of delivering fully-fledged psychotherapy until the so-called "general" or "human-like" AI is developed. While we believe that all these challenges can ultimately be overcome, we think that being mindful of them is crucial to ensure well-balanced and steady progress on our path to AI-based psychotherapy.

Assessing Medical Students' Comfort with Telemedicine

OBJECTIVE

The Association of American Medical Colleges (AAMC) has recommended that the provision of telemedicine services become an entrustable professional activity (EPA). Given its increased scope, medical student comfort with telemedicine was explored.

METHODS

An institutional review board-approved 17-question, anonymous voluntary survey was created based on the AAMC's EPAs and administered to students at Northeast Ohio Medical University across 4 weeks. The primary outcome of this study was to assess medical students' self-reported telemedicine comfort levels.

RESULTS

The response rate was 141 students (22%). At least 80% of students believed that they were able to gather essential and accurate patient information, counsel patients and families, and communicate effectively across a broad range of social, economic, and cultural backgrounds using telemedicine. In total, 57% and 53% of students, respectively, believed that they were able to gather information and diagnose patients using telemedicine as well as they did in person, 38% of respondents believed that their patient's health outcome was the same via telemedicine or in-person visits, and 74% of respondents wished that telemedicine was formally taught in school. Most of the students believed they could effectively gather essential information and counsel patients via telemedicine, but there was a notable decrease in confidence for medical students when comparing telemedicine and in-person care directly.

CONCLUSIONS

Despite the EPAs created by the AAMC, students did not self-report the same comfort level with telemedicine as they had with in-person patient visits. There are opportunities for improvement in the telemedicine medical school curriculum.

A relational agent for treating substance use in adults: Protocol for a randomized controlled trial with a psychoeducational comparator

BACKGROUND

Substance use disorders (SUDs) are prevalent and compromise health and wellbeing. Scalable solutions, such as digital therapeutics, may offer a population-based strategy for addressing SUDs. Two formative studies supported the feasibility and acceptability of the relational agent Woebot, an animated screen-based social robot, for treating SUDs (W-SUDs) in adults. Participants randomized to W-SUDs reduced their substance use occasions from baseline to end-of-treatment (EOT) relative to a waitlist control.

OBJECTIVE

To further develop the evidence base, the current randomized trial extends follow-up to 1-month post-treatment and will test the efficacy of W-SUDs relative to a psychoeducational control.

METHODS

This study will recruit, screen, and consent 400 adults online reporting problematic substance use. Following baseline assessment, participants will be randomized to 8 weeks of W-SUDs or a psychoeducational control. Assessments will be conducted at weeks 4, 8 (EOT), and 12 (1-month post-treatment). Primary outcome is past-month number of substance use occasions, summed across all substances. Secondary outcomes are number of heavy drinking days, the percent of days abstinent from all substances, substance use problems, thoughts about abstinence, cravings, confidence to resist substance use, symptoms of depression and anxiety, and work productivity. If significant group differences are found, we will explore moderators and mediators of treatment effects.

CONCLUSIONS

The current study builds upon emerging evidence of a digital therapeutic for reducing problematic substance use by examining sustained effects and testing against a psychoeducational control condition. If efficacious, the findings have implications for scalable mobile health interventions for reducing problematic substance use.

TRIAL REGISTRATION

NCT04925570.

A qualitative exploration of patients' experience of mobile telephone-delivered contingency management to promote adherence to supervised methadone

INTRODUCTION

Despite an increasing evidence base for mobile telephone-delivered contingency management (mCM), there had been no previous qualitative exploration of patients' experience of receiving mCM and the factors that might influence that experience and outcome in a UK setting. The aim of this study was to understand patients' views and experience of receiving mCM by exploring their beliefs, expectations and perceived benefits within the context of the UK's first mCM intervention.

METHODS

Qualitative interviews (N = 15) were conducted with patients undergoing opioid agonist treatment in a UK drug treatment service and receiving mCM to encourage adherence with supervised methadone as part of an existing study. Interviews were conducted at two time points and analysed using Framework to explore patients' expectations and beliefs during the early stage of the intervention (2 weeks) and their perceived benefits and experience at the end of the intervention (12 weeks).

RESULTS

The mCM was perceived as a motivator, providing validation of achievement, and involving discreet and positive interactions. Perceived benefits included enhanced methadone adherence, reduced drug use and the development of a supportive and non-judgemental connection that resembled a therapeutic alliance.

DISCUSSION AND CONCLUSIONS

The mechanisms underpinning contingency management appeared to operate in the absence of human interaction, and the mCM intervention was deemed to be meaningful, acceptable and well received by patients. These findings not only provide support for the application of mCM in this context but also offer insight into the factors that influence outcomes and should be considered in the development of future mCM interventions.

Evaluating Conversational Agents for Mental Health: Scoping Review of Outcomes and Outcome Measurement Instruments (Preprint)

BACKGROUND

Rapid proliferation of mental health interventions delivered through conversational agents (CAs) calls for high-quality evidence to support their implementation and adoption. Selecting appropriate outcomes, instruments for measuring outcomes, and assessment methods are crucial for ensuring that interventions are evaluated effectively and with a high level of quality.

OBJECTIVE

We aimed to identify the types of outcomes, outcome measurement instruments, and assessment methods used to assess the clinical, user experience, and technical outcomes in studies that evaluated the effectiveness of CA interventions for mental health.

METHODS

We undertook a scoping review of the relevant literature to review the types of outcomes, outcome measurement instruments, and assessment methods in studies that evaluated the effectiveness of CA interventions for mental health. We performed a comprehensive search of electronic databases, including PubMed, Cochrane Central Register of Controlled Trials, Embase (Ovid), PsychINFO, and Web of Science, as well as Google Scholar and Google. We included experimental studies evaluating CA mental health interventions. The screening and data extraction were performed independently by 2 review authors in parallel. Descriptive and thematic analyses of the findings were performed.

RESULTS

We included 32 studies that targeted the promotion of mental well-being (17/32, 53%) and the treatment and monitoring of mental health symptoms (21/32, 66%). The studies reported 203 outcome measurement instruments used to measure clinical outcomes (123/203, 60.6%), user experience outcomes (75/203, 36.9%), technical outcomes (2/203, 1.0%), and other outcomes (3/203, 1.5%). Most of the outcome measurement instruments were used in only 1 study (150/203, 73.9%) and were self-reported questionnaires (170/203, 83.7%), and most were delivered electronically via survey platforms (61/203, 30.0%). No validity evidence was cited for more than half of the outcome measurement instruments (107/203, 52.7%), which were largely created or adapted for the study in which they were used (95/107, 88.8%).

CONCLUSIONS

The diversity of outcomes and the choice of outcome measurement instruments employed in studies on CAs for mental health point to the need for an established minimum core outcome set and greater use of validated instruments. Future studies should also capitalize on the affordances made available by CAs and smartphones to streamline the evaluation and reduce participants' input burden inherent to self-reporting.

Alliance With an Unguided Smartphone App: Validation of the Digital Working Alliance Inventory

The working alliance may be relevant in unguided smartphone-based interventions, but no validated measure exists. We evaluated the psychometric properties of the six-item Digital Working Alliance Inventory (DWAI) using a cross-sectional survey of meditation app users (n = 290) and the intervention arm of a randomized trial testing a smartphone-based meditation app (n = 314). Exploratory factor analysis suggested a single-factor solution which was replicated using longitudinal confirmatory factor analysis. The DWAI showed adequate internal consistency and test-retest reliability. Discriminant validity was supported by a lack of association with social desirability, psychological distress, and preference for a waitlist condition. Convergent validity was supported by positive associations with perceived app effectiveness and preference for an app condition. Supporting predictive validity, DWAI scores positively predicted self-reported and objective app utilization. When assessed at Weeks 3 or 4 of the intervention, but not earlier, DWAI scores predicted pre-post reductions in psychological distress.

I Need to Know: Using the CeHRes Roadmap to Develop a Treatment Feedback Tool for Youngsters with Mental Health Problems

Patient-Reported Outcome Measures (PROMs) are often used to monitor treatment outcomes in youth mental health care. Unfortunately, youngsters are rarely informed about the results of their PROMs or, when they are, it is in an insufficient manner. Therefore, a web application was developed-together with youngsters-aimed at giving them feedback about their PROMs. The aim of this study is to describe the development process of the application. An expert panel consisting of youngsters, web designers and researchers, as well as a representative from a client organisation, developed the e-health application INK (short for 'I Need to Know') in an iterative process based on the Centre for eHealth Research roadmap (CeHRes roadmap). Youngsters prefer, among other aspects, a simple, easy-to-use e-health application with a colourful appearance and want to be able to compare their results across different time points and informants. The INK tool provides youngsters with insight into their PROM results. Based on the youngsters' preferences, INK users can choose which feedback information is visible. INK facilitates youngsters' active participation in their treatment as well as shared decision-making with their professional caregivers.

Connecting over the internet: Establishing the therapeutic alliance in an internet-based treatment for depressed adolescents

Internet-based treatments have been developed for youth mental health difficulties, with promising results. However, little is known about the features of therapeutic alliance, and how it is established and maintained, in text-based interactions between adolescents and therapists in internet-based treatments. This study uses data collected during a pilot evaluation of a psychodynamic internet-based therapy for depressed adolescents. The adolescents had instant-messaging chats with their therapists once a week, over 10 weeks. The adolescents also rated the therapeutic alliance each week, using the Session Alliance Inventory. The present study uses qualitative methods to analyse transcripts of text-based communication between the young people and their therapists. The aim is to identify and describe the key features of therapeutic alliance, and reflect upon the implications for theory and clinical practice. Analysis identified three 'values' that may underpin a strong therapeutic alliance: togetherness, agency and hope. A number of therapist techniques were also found, which seemed to create a sense of these values during text-chat sessions. These findings are discussed, alongside implications for future research.

Conceptual Invariance, Trajectories, and Outcome Associations of Working Alliance in Unguided and Guided Internet-Based Psychological Interventions: Secondary Analysis of a Randomized Controlled Trial

BACKGROUND

The role of working alliance remains unclear for many forms of internet-based interventions (IBIs), a set of effective psychotherapy alternatives that do not require synchronous interactions between patients and therapists.

OBJECTIVE

This study examined the conceptual invariance, trajectories, and outcome associations of working alliance across an unguided IBI and guided IBIs that incorporated clinician support through asynchronous text messaging or video messaging.

METHODS

Adults with high education attainment (n=145) with subclinical levels of anxiety, stress, or depressive symptoms were randomized to 1 of 3 treatment conditions for 7 weeks. All participants received treatments from MyCompass, an unguided IBI using cognitive behavior therapy. Participants in condition 2 and 3 received supplemental, asynchronous clinician support through text and video, respectively. Working alliance with the IBIs was measured weekly using select items from the 12-item version of the Agnew Relationship Measure. Symptom and functional outcomes were assessed at baseline, at the end of treatment, and 1-month follow-up.

RESULTS

Working alliance with the IBIs was conceptually invariant across the 3 conditions. Working alliance followed a quadratic pattern of change over time for all conditions and declined significantly only in the text-support condition. After controlling for baseline symptoms, higher baseline levels of working alliance predicted less depression and less functional impairment at follow-up, whereas faster increases in working alliance predicted less worry at the end of treatment and at follow-up, all of which only occurred in the video-support condition.

CONCLUSIONS

Working alliance with the IBIs was generally established in the initial sessions. Although working alliance is conceptually invariant across IBIs with or without clinician support, the associations between working alliance and treatment outcomes among IBIs may differ depending on clinician involvement and the modalities of support.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05122429; https://clinicaltrials.gov/ct2/show/NCT05122429.

Global Scientific Research Landscape on Medical Informatics From 2011 to 2020: Bibliometric Analysis

Background

With the emerging information and communication technology, the field of medical informatics has dramatically evolved in health care and medicine. Thus, it is crucial to explore the global scientific research landscape on medical informatics.

Objective

This study aims to present a visual form to clarify the overall scientific research trends of medical informatics in the past decade.

Methods

A bibliometric analysis of data retrieved and extracted from the Web of Science Core Collection (WoSCC) database was performed to analyze global scientific research trends on medical informatics, including publication year, journals, authors, institutions, countries/regions, references, and keywords, from January 1, 2011, to December 31, 2020.

Results

The data set recorded 34,742 articles related to medical informatics from WoSCC between 2011 and 2020. The annual global publications increased by 193.86% from 1987 in 2011 to 5839 in 2020. Journal of Medical Internet Research (3600 publications and 63,932 citations) was the most productive and most highly cited journal in the field of medical informatics. David W Bates (99 publications), Harvard University (1161 publications), and the United States (12,927 publications) were the most productive author, institution, and country, respectively. The co-occurrence cluster analysis of high-frequency author keywords formed 4 clusters: (1) artificial intelligence in health care and medicine; (2) mobile health; (3) implementation and evaluation of electronic health records; (4) medical informatics technology application in public health. COVID-19, which ranked third in 2020, was the emerging theme of medical informatics.

Conclusions

We summarize the recent advances in medical informatics in the past decade and shed light on their publication trends, influential journals, global collaboration patterns, basic knowledge, research hotspots, and theme evolution through bibliometric analysis and visualization maps. These findings will accurately and quickly grasp the research trends and provide valuable guidance for future medical informatics research.

Barriers and Considerations in the Design and Implementation of Digital Behavioral Interventions: Qualitative Analysis

Background

Digital behavioral interventions have become increasingly popular for their ability to support patient diagnosis and treatment, chronic disease self-management, behavior change, and adherence to recommended care. However, digital intervention development is impeded by challenges such as limited technical skills, limited access to developers, and cost. The purpose of this study is to elicit in-depth qualitative feedback from intervention developers who have interest in digital behavioral interventions but lack programming skills regarding the barriers they experience and key considerations in the design and implementation of digital interventions.

Objective

This study aims to understand barriers in the design and implementation of digital behavioral interventions, as well as to identify key considerations for researchers who are developing these interventions.

Methods

We conducted semistructured qualitative interviews with 18 researchers who had experience either designing (but not coding) digital behavioral interventions or running research studies with them. Participants were a convenience sample of users of the Computerized Intervention Authoring System platform, an existing no-code development platform for building digital intervention content, and were recruited through either direct email solicitation or snowball sampling. All interviews were conducted and recorded over videoconference between February and April 2020. Recordings from interviews were transcribed and thematically analyzed by multiple coders.

Results

Interviews were completed with 18 participants and lasted between 24 and 65 (mean 46.9, SD 11.3) minutes. Interviewees were predominantly female (17/18, 94%) and represented different job roles, ranging from researcher to project/study staff. Three key barriers in the development of digital behavior interventions were identified during interviews: lack of cross-disciplinary understanding; variability in recipients’ technology access, infrastructure, and literacy; and the idea that evidence-based in-person interactions do not translate directly to digital interactions. Interviewees identified several key considerations that interventionists learned to prioritize, which have the potential to overcome these barriers and lead to successful interventions.

Conclusions

Barriers in the development of digital behavioral interventions are often created by a lack of cross-disciplinary understanding, which can lead to difficulties conceptualizing interventions, unrealistic expectations in terms of cost, and confusion about the development process. Moreover, concerns about research study participant characteristics and access to technology, as well as the translation of in-person interventions to digital, are apparent. Appropriate training in how to work with software development teams may help future digital behavior intervention creators overcome these barriers and may lead to new, exciting innovations in this space.

[Teletherapy after cochlear implantation during the COVID-19 pandemic]

BACKGROUND

Due to the COVID-19 pandemic, digitalization in healthcare grew rapidly. Auditory training after cochlear implantation usually takes place face-to-face but social distancing interferes with this therapeutic approach.

MATERIALS AND METHODS

In follow-up treatment, 42 adult cochlear implant (CI) users aged 53.8 (±15.6) years received video therapy 1 x/week for 5 weeks on a certified platform. After each therapy session, the technical process and therapeutic content were assessed. At the end of the study, usability and the relationship between therapist and patient were evaluated by patients and therapists using the System Usability Scale (SUS), a final questionnaire and by the Skala Therapeutische Allianz - Revised (STA-R). Furthermore, a cost-benefit analysis was done.

RESULTS

Usability for both users was high (87.97 versus 93.0). Despite the lack of personal contact, therapeutic alliance was highly appreciated by patients and therapists (87.8% versus 84.8%). The main advantages for the patients were reductions in time and costs. In contrast, the rehabilitation center faced higher costs initially due to the longer time therapists needed to prepare the lessons. Technical problems had to be solved in > 75% of the first sessions but did not bother training thereafter. In total, 47.6% of the patients believe that teletherapy can completely fulfill their therapeutic needs.

CONCLUSION

Video therapy has been judged as a useful tool by all users and the majority wants to continue. However, it remains questionable whether the therapist-patient relationship can be sufficiently maintained over a longer period and whether online therapy is as effective as face-to-face therapy.

Digital interventions in mental health: evidence syntheses and economic modelling

BACKGROUND

Economic evaluations provide evidence on whether or not digital interventions offer value for money, based on their costs and outcomes relative to the costs and outcomes of alternatives.

OBJECTIVES

(1) Evaluate and summarise published economic studies about digital interventions across different technologies, therapies, comparators and mental health conditions; (2) synthesise clinical evidence about digital interventions for an exemplar mental health condition; (3) construct an economic model for the same exemplar mental health condition using the previously synthesised clinical evidence; and (4) consult with stakeholders about how they understand and assess the value of digital interventions.

METHODS

We completed four work packages: (1) a systematic review and quality assessment of economic studies about digital interventions; (2) a systematic review and network meta-analysis of randomised controlled trials on digital interventions for generalised anxiety disorder; (3) an economic model and value-of-information analysis on digital interventions for generalised anxiety disorder; and (4) a series of knowledge exchange face-to-face and digital seminars with stakeholders.

RESULTS

In work package 1, we reviewed 76 economic evaluations: 11 economic models and 65 within-trial analyses. Although the results of the studies are not directly comparable because they used different methods, the overall picture suggests that digital interventions are likely to be cost-effective, compared with no intervention and non-therapeutic controls, whereas the value of digital interventions compared with face-to-face therapy or printed manuals is unclear. In work package 2, we carried out two network meta-analyses of 20 randomised controlled trials of digital interventions for generalised anxiety disorder with a total of 2350 participants. The results were used to inform our economic model, but when considered on their own they were inconclusive because of the very wide confidence intervals. In work package 3, our decision-analytic model found that digital interventions for generalised anxiety disorder were associated with lower net monetary benefit than medication and face-to-face therapy, but greater net monetary benefit than non-therapeutic controls and no intervention. Value for money was driven by clinical outcomes rather than by intervention costs, and a value-of-information analysis suggested that uncertainty in the treatment effect had the greatest value (£12.9B). In work package 4, stakeholders identified several areas of benefits and costs of digital interventions that are important to them, including safety, sustainability and reducing waiting times. Four factors may influence their decisions to use digital interventions, other than costs and outcomes: increasing patient choice, reaching underserved populations, enabling continuous care and accepting the 'inevitability of going digital'.

LIMITATIONS

There was substantial uncertainty around effect estimates of digital interventions compared with alternatives. This uncertainty was driven by the small number of studies informing most comparisons, the small samples in some of these studies and the studies' high risk of bias.

CONCLUSIONS

Digital interventions may offer good value for money as an alternative to 'doing nothing' or 'doing something non-therapeutic' (e.g. monitoring or having a general discussion), but their added value compared with medication, face-to-face therapy and printed manuals is uncertain. Clinical outcomes rather than intervention costs drive 'value for money'.

FUTURE WORK

There is a need to develop digital interventions that are more effective, rather than just cheaper, than their alternatives.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42018105837.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 26, No. 1. See the NIHR Journals Library website for further project information.

Recommender systems for mental health apps: advantages and ethical challenges

Recommender systems assist users in receiving preferred or relevant services and information. Using such technology could be instrumental in addressing the lack of relevance digital mental health apps have to the user, a leading cause of low engagement. However, the use of recommender systems for digital mental health apps, particularly those driven by personal data and artificial intelligence, presents a range of ethical considerations. This paper focuses on considerations particular to the juncture of recommender systems and digital mental health technologies. While separate bodies of work have focused on these two areas, to our knowledge, the intersection presented in this paper has not yet been examined. This paper identifies and discusses a set of advantages and ethical concerns related to incorporating recommender systems into the digital mental health (DMH) ecosystem. Advantages of incorporating recommender systems into DMH apps are identified as (1) a reduction in choice overload, (2) improvement to the digital therapeutic alliance, and (3) increased access to personal data & self-management. Ethical challenges identified are (1) lack of explainability, (2) complexities pertaining to the privacy/personalization trade-off and recommendation quality, and (3) the control of app usage history data. These novel considerations will provide a greater understanding of how DMH apps can effectively and ethically implement recommender systems.

The Acceptability of a Smartphone App (BlueIce) for University Students Who Self-harm

University students are twice as likely to self-harm than community controls but, unfortunately, help-seeking among this population is particularly low. Given the stigma around self-harm, the face-to-face nature of traditional support for self-harm can be a barrier to help-seeking. Smartphone applications (apps) are a possible alternative source of support, and research has shown that students are receptive to this option. This study sought to assess the acceptability of a smartphone app called BlueIce for university students who self-harm. Semi-structured interviews were conducted with 25 students with a history of self-harm. A qualitative content analysis was undertaken and five categories were identified: the content of BlueIce, the use of BlueIce with university students, the function of BlueIce, comparison with other support, and the implementation and uptake of BlueIce. Responses to BlueIce were very positive with students believing BlueIce to be a helpful resource that was perceived as more accessible than alternative support. Participants believed it could provide help in moments of distress as well as helping individuals learn longer-term coping skills. Others felt that BlueIce would not be adequate for some people and would be better used alongside other face-to-face support. Overall, it was clear that BlueIce was acceptable to the students in this study. Future research should seek to evaluate the feasibility and effectiveness of BlueIce within a university setting with students. HIGHLIGHTSStudents who self-harm found the BlueIce app to be an acceptable and appealing source of support for self-harm as well as other mental health difficulties.Participants felt that the app was more accessible than other forms of support, particularly for individuals who prefer not to discuss self-harm with a professional.Some felt that BlueIce could provide immediate support in moments of distress, while others believed it to be a longer-term solution that could help students learn more adaptive coping strategies.

Digital Therapeutic Alliance With Fully Automated Mental Health Smartphone Apps: A Narrative Review

Fully automated mental health smartphone apps show strong promise in increasing access to psychological support. Therefore, it is crucial to understand how to make these apps effective. The therapeutic alliance (TA), or the relationship between healthcare professionals and clients, is considered fundamental to successful treatment outcomes in face-to-face therapy. Thus, understanding the TA in the context of fully automated apps would bring us insights into building effective smartphone apps which engage users. However, the concept of a digital therapeutic alliance (DTA) in the context of fully automated mental health smartphone apps is nascent and under-researched, and only a handful of studies have been published in this area. In particular, no published review paper examined the DTA in the context of fully automated apps. The objective of this review was to integrate the extant literature to identify research gaps and future directions in the investigation of DTA in relation to fully automated mental health smartphone apps. Our findings suggest that the DTA in relation to fully automated smartphone apps needs to be conceptualized differently to traditional face-to-face TA. First, the role of bond in the context of fully automated apps is unclear. Second, human components of face-to-face TA, such as empathy, are hard to achieve in the digital context. Third, some users may perceive apps as more non-judgmental and flexible, which may further influence DTA formation. Subdisciplines of computer science, such as affective computing and positive computing, and some human-computer interaction (HCI) theories, such as those of persuasive technology and human-app attachment, can potentially help to foster a sense of empathy, build tasks and goals and develop bond or an attachment between users and apps, which may further contribute to DTA formation in fully automated smartphone apps. Whilst the review produced a relatively limited quantity of literature, this reflects the novelty of the topic and the need for further research.

A randomized controlled trial of a therapeutic relational agent for reducing substance misuse during the COVID-19 pandemic

BACKGROUND

The COVID-19 pandemic disrupted access to treatment for substance use disorders (SUDs), while alcohol and cannabis retail sales increased. During the pandemic, we tested a tailored digital health solution, Woebot-SUDs (W-SUDs), for reducing substance misuse.

METHODS

In a randomized controlled trial, we compared W-SUDs for 8 weeks to a waitlist control. U.S. adults (N = 180) who screened positive for substance misuse (CAGE-AID>1) were enrolled June-August 2020. The primary outcome was the change in past-month substance use occasions from baseline to end-of-treatment (EOT). Study retention was 84%. General linear models tested group differences in baseline-to-EOT change scores, adjusting for baseline differences and attrition.

RESULTS

At baseline, the sample (age M = 40, SD = 12, 65% female, 68% non-Hispanic white) averaged 30.2 (SD = 18.6) substance occasions in the past month. Most (77%) reported alcohol problems, 28% cannabis, and 45% multiple substances; 46% reported moderate-to-severe depressive symptoms. Treatment participants averaged 920 in-app text messages (SD = 892, Median = 701); 96% of completed lessons were rated positively; and 88% would recommend W-SUDs. Relative to waitlist, W-SUDs participants significantly reduced past-month substance use occasions (M = -9.1, SE = 2.0 vs. M = -3.3, SE = 1.8; p = .039). Secondary substance use and mood outcomes did not change significantly by group; however, reductions in substance use occasions correlated significantly with increased confidence and fewer substance use problems, cravings, depression and anxiety symptoms, and pandemic-related mental health effects (p-value<.05).

CONCLUSIONS

W-SUDs was associated with significant reductions in substance use occasions. Reduction in substance use occasions was associated with better outcomes, including improved mental health. W-SUDs satisfaction was high.

The Digital Therapeutic Alliance: Prospects and Considerations

The growing prevalence of digital approaches to mental health care raises a range of questions and considerations. A notion that has recently emerged is that of the digital therapeutic alliance, prompting consideration of whether and how the concept of therapeutic alliance, which has proven to be a central ingredient of successful traditional psychotherapy, could translate to mental health care via digital technologies. This special issue editorial article outlines the topic of digital therapeutic alliance and introduces the five articles that comprise the special issue.

Predicting response to transdiagnostic iCBT for emotional disorders from patient and therapist involvement

BACKGROUND

Transdiagnostic iCBT has been shown to be effective for the treatment of emotional disorders. Less is known about the optimal level of therapist and patient involvement in these interventions. Specific characteristics of Internet-delivered interventions include treatment adherence (e.g., amount of review of the materials) and guidance (e.g., amount of therapist support). Exploring the importance of these elements in treatment outcome may help to maximize the efficiency of Internet-delivered psychological interventions.

AIM

In this study, we aimed to analyze the relationship between patient and therapist involvement (i.e., platform usage and amount of therapist guidance) in a sample of patients with emotional disorders who received transdiagnostic iCBT in Spanish public specialized mental healthcare services.

METHOD

This is a secondary analysis of a randomized controlled trial. The sample included 63 patients who completed transdiagnostic iCBT for emotional disorders. Platform usage metrics included number of logins into the platform and number of times the participants reviewed the modules. Therapist guidance was measured as the number of support phone calls with a therapist and their total duration (minutes). Logistic regressions and ROC analyses were performed to explore the predictive value of platform usage and therapist guidance in symptom reduction. Clinical outcomes included depressive and anxiety symptoms assessed at baseline and post-intervention. The bivariate relationship between the platform usage and therapist guidance variables was also explored.

RESULTS

Overall, platform usage and therapist guidance were not associated with symptom improvement. However, the patient and therapist involvement parameters were intercorrelated. Specifically, the number of calls and their duration were associated with a greater number of logins (r = 0.61; p < .001) and more frequent reviews of the modules (0.46 ≤ r ≤ 0.60; p < .001). Higher baseline depression and anxiety were, respectively, associated with greater improvements in depression (r = -0.37, p = .003) and anxiety after treatment completion (r = -0.48, p < .001).

DISCUSSION

The results suggest that there is no reliable cut-off point for platform usage and therapist guidance in predicting optimal symptom reduction. However, significant associations were found between platform usage and guidance variables that warrant additional research. More research on this topic is necessary to further clarify the role of these and other platform usage and guidance variables in Internet-delivered iCBT outcomes.

A Therapeutic Relational Agent for Reducing Problematic Substance Use (Woebot): Development and Usability Study

Background

Misuse of substances is common, can be serious and costly to society, and often goes untreated due to barriers to accessing care. Woebot is a mental health digital solution informed by cognitive behavioral therapy and built upon an artificial intelligence–driven platform to deliver tailored content to users. In a previous 2-week randomized controlled trial, Woebot alleviated depressive symptoms.

Objective

This study aims to adapt Woebot for the treatment of substance use disorders (W-SUDs) and examine its feasibility, acceptability, and preliminary efficacy.

Methods

American adults (aged 18-65 years) who screened positive for substance misuse without major health contraindications were recruited from online sources and flyers and enrolled between March 27 and May 6, 2020. In a single-group pre/postdesign, all participants received W-SUDs for 8 weeks. W-SUDs provided mood, craving, and pain tracking and modules (psychoeducational lessons and psychotherapeutic tools) using elements of dialectical behavior therapy and motivational interviewing. Paired samples t tests and McNemar nonparametric tests were used to examine within-subject changes from pre- to posttreatment on measures of substance use, confidence, cravings, mood, and pain.

Results

The sample (N=101) had a mean age of 36.8 years (SD 10.0), and 75.2% (76/101) of the participants were female, 78.2% (79/101) were non-Hispanic White, and 72.3% (73/101) were employed. Participants’ W-SUDs use averaged 15.7 (SD 14.2) days, 12.1 (SD 8.3) modules, and 600.7 (SD 556.5) sent messages. About 94% (562/598) of all completed psychoeducational lessons were rated positively. From treatment start to end, in-app craving ratings were reduced by half (87/101, 86.1% reporting cravings in the app; odds ratio 0.48, 95% CI 0.32-0.73). Posttreatment assessment completion was 50.5% (51/101), with better retention among those who initially screened higher on substance misuse. From pre- to posttreatment, confidence to resist urges to use substances significantly increased (mean score change +16.9, SD 21.4; P<.001), whereas past month substance use occasions (mean change −9.3, SD 14.1; P<.001) and scores on the Alcohol Use Disorders Identification Test-Concise (mean change −1.3, SD 2.6; P<.001), 10-item Drug Abuse Screening Test (mean change −1.2, SD 2.0; P<.001), Patient Health Questionnaire-8 item (mean change 2.1, SD 5.2; P=.005), Generalized Anxiety Disorder-7 (mean change −2.3, SD 4.7; P=.001), and cravings scale (68.6% vs 47.1% moderate to extreme; P=.01) significantly decreased. Most participants would recommend W-SUDs to a friend (39/51, 76%) and reported receiving the service they desired (41/51, 80%). Fewer felt W-SUDs met most or all of their needs (22/51, 43%).

Conclusions

W-SUDs was feasible to deliver, engaging, and acceptable and was associated with significant improvements in substance use, confidence, cravings, depression, and anxiety. Study attrition was high. Future research will evaluate W-SUDs in a randomized controlled trial with a more diverse sample and with the use of greater study retention strategies.

Trial Registration

ClinicalTrials.gov NCT04096001; http://clinicaltrials.gov/ct2/show/NCT04096001.

Heartrate variability biofeedback for migraine using a smartphone application and sensor: A randomized controlled trial

INTRODUCTION

Although hand temperature and electromyograph biofeedback have evidence for migraine prevention, to date, no study has evaluated heartrate variability (HRV) biofeedback for migraine.

METHODS

2-arm randomized trial comparing an 8-week app-based HRV biofeedback (HeartMath) to waitlist control. Feasibility/acceptability outcomes included number and duration of sessions, satisfaction, barriers and adverse events. Primary clinical outcome was Migraine-Specific Quality of Life Questionnaire (MSQv2).

RESULTS

There were 52 participants (26/arm). On average, participants randomized to the Hearthmath group completed 29 sessions (SD = 29, range: 2-86) with an average length of 6:43 min over 36 days (SD = 27, range: 0, 88) before discontinuing. 9/29 reported technology barriers. 43% said that they were likely to recommend Heartmath to others. Average MSQv2 decreases were not significant between the Heartmath and waitlist control (estimate = 0.3, 95% CI = -3.1 - 3.6). High users of Heartmath reported a reduction in MSQv2 at day 30 (-12.3 points, p = 0.010) while low users did not (p = 0.765).

DISCUSSION

App-based HRV biofeedback was feasible and acceptable on a time-limited basis for people with migraine. Changes in the primary clinical outcome did not differ between biofeedback and control; however, high users of the app reported more benefit than low users.

The Digital Therapeutic Alliance and Human-Computer Interaction

The therapeutic alliance (TA), the relationship that develops between a therapist and a client/patient, is a critical factor in the outcome of psychological therapy. As mental health care is increasingly adopting digital technologies and offering therapeutic interventions that may not involve human therapists, the notion of a TA in digital mental health care requires exploration. To date, there has been some incipient work on developing measures to assess the conceptualization of a digital TA for mental health apps. However, the few measures that have been proposed have more or less been derivatives of measures from psychology used to assess the TA in traditional face-to-face therapy. This conceptual paper explores one such instrument that has been proposed in the literature, the Mobile Agnew Relationship Measure, and examines it through a human-computer interaction (HCI) lens. Through this process, we show how theories from HCI can play a role in shaping or generating a more suitable, purpose-built measure of the digital therapeutic alliance (DTA), and we contribute suggestions on how HCI methods and knowledge can be used to foster the DTA in mental health apps.

Evaluation of a Mobile App to Enhance Relational Awareness and Change During Cognitive Analytic Therapy: Mixed Methods Case Series

BACKGROUND

There has been a lack of technological innovation regarding improving the delivery of integrative psychotherapies. This project sought to evaluate an app designed to replace previous paper-based methods supporting relational awareness and change during cognitive analytic therapy (CAT).

OBJECTIVE

We aimed to assess patients' and therapists' experience of using the technology (ie, the "CAT-App") and to evaluate the relationship between app usage and clinical outcome.

METHODS

The design was a mixed methods case series. Patients completed the Clinical Outcomes in Routine Evaluation-Outcome Measure pre- and post-CAT. Mood data plus the frequency and effectiveness of relational awareness and change were collected via the app. Therapists and patients were interviewed about their experiences using the app.

RESULTS

Ten patients (treated by 3 therapists) were enrolled; seven completed treatment and 4 had a reliable improvement in their mental health. App usage and mood change did not differ according to clinical outcome, but there was a statistically significant difference in app usage between completers and dropouts. The qualitative themes described by the therapists were (1) the challenge of incorporating the technology into their clinical practice and (2) the barriers and benefits of the technology. Clients' themes were (1) data protection, (2) motivation and engagement, and (3) restrictions versus flexibility.

CONCLUSIONS

The CAT-App is capable of supporting relational awareness and change and is an upgrade on older, paper-based formats. Further clinical evaluation is required.

Psychological Screening and Tracking of Athletes and Digital Mental Health Solutions in a Hybrid Model of Care: Mini Review

BACKGROUND

There is a persistent need for mental ill-health prevention and intervention among at-risk and vulnerable subpopulations. Major disruptions to life, such as the COVID-19 pandemic, present an opportunity for a better understanding of the experience of stressors and vulnerability. Faster and better ways of psychological screening and tracking are more generally required in response to the increased demand upon mental health care services. The argument that mental and physical health should be considered together as part of a biopsychosocial approach is garnering acceptance in elite athlete literature. However, the sporting population are unique in that there is an existing stigma of mental health, an underrecognition of mental ill-health, and engagement difficulties that have hindered research, prevention, and intervention efforts.

OBJECTIVE

The aims of this paper are to summarize and evaluate the literature on athletes' increased vulnerability to mental ill-health and digital mental health solutions as a complement to prevention and intervention, and to show relationships between athlete mental health problems and resilience as well as digital mental health screening and tracking, and faster and better treatment algorithms.

METHODS

This mini review shapes literature in the fields of athlete mental health and digital mental health by summarizing and evaluating journal and review articles drawn from PubMed Central and the Directory of Open Access Journals.

RESULTS

Consensus statements and systematic reviews indicated that elite athletes have comparable rates of mental ill-health prevalence to the general population. However, peculiar subgroups require disentangling. Innovative expansion of data collection and analytics is required to respond to engagement issues and advance research and treatment programs in the process. Digital platforms, machine learning, deep learning, and artificial intelligence are useful for mental health screening and tracking in various subpopulations. It is necessary to determine appropriate conditions for algorithms for use in recommendations. Partnered with real-time automation and machine learning models, valid and reliable behavior sensing, digital mental health screening, and tracking tools have the potential to drive a consolidated, measurable, and balanced risk assessment and management strategy for the prevention and intervention of the sequelae of mental ill-health.

CONCLUSIONS

Athletes are an at-risk subpopulation for mental health problems. However, a subgroup of high-level athletes displayed a resilience that helped them to positively adjust after a period of overwhelming stress. Further consideration of stress and adjustments in brief screening tools is recommended to validate this finding. There is an unrealized potential for broadening the scope of mental health, especially symptom and disorder interpretation. Digital platforms for psychological screening and tracking have been widely used among general populations, but there is yet to be an eminent athlete version. Sports in combination with mental health education should address the barriers to help-seeking by increasing awareness, from mental ill-health to positive functioning. A hybrid model of care is recommended, combining traditional face-to-face approaches along with innovative and evaluated digital technologies, that may be used in prevention and early intervention strategies.

Towards a conceptual framework of the working alliance in a blended low-intensity cognitive behavioural therapy intervention for depression in primary mental health care: a qualitative study

OBJECTIVES

To examine and adapt a conceptual framework of the working alliance (WA) in the context of a low-intensity blended (psychological well-being practitioner (PWP) plus computerised program) cognitive behavioural therapy intervention (b-CBT) for depression.

DESIGN

Patient involvement was enlisted to collaboratively shape the design of the project from the onset, before data collection. In-depth semi-structured interviews were carried out with participants who experienced b-CBT as part of the E-compared trial. A thematic analysis was conducted using a constant comparative method informed by grounded theory.

SETTING

Recruitment was carried out in four psychological primary care services across the UK.

PARTICIPANTS

Nineteen trial participants with major depressive disorder who completed at least one computerised program and face-to-face session with a PWP in the b-CBT arm were recruited to the study.

RESULTS

Qualitative interviews that were guided by WA theory and patient involvement, revealed four themes: (1) a healthcare provider (PWP and computerised program) with good interpersonal competencies for building a working relationship with the client ('bond'); (2) collaborative efforts between the client and the provider to appropriately identify what the client hopes to achieve through therapy ('goals'); (3) the selection of acceptable therapeutic activities that address client goals and the availability of responsive support ('task') and (4) the promotion of active engagement and autonomous problem solving ('usability heuristics'). Participants described how the PWP and computerised program uniquely and collectively contributed to different WA needs.

CONCLUSIONS

This study is the first to offer a preliminary conceptual framework of WA in b-CBT for depression, and how such demands can be addressed through blended PWP-computerised program delivery. These findings can be used to promote WA in technological design and clinical practice, thereby promoting engagement to b-CBT interventions and effective deployment of practitioner and program resources.

TRIAL REGISTRATION NUMBER

ISRCTN12388725.

Working Alliance Inventory for Online Interventions-Short Form (WAI-TECH-SF): The Role of the Therapeutic Alliance between Patient and Online Program in Therapeutic Outcomes

Background: Therapeutic alliance (TA) between the patient and therapist has been related to positive therapeutic outcomes. Because Internet-based interventions are increasingly being implemented, a tool is needed to measure the TA with Internet-based self-guided programs. The Working Alliance Inventory for online interventions (WAI-TECH-SF) was adapted based on the WAI Short Form (Hatcher & Gillaspy, 2006). The objectives of this study were: (1) to analyse the psychometric properties of the WAI-TECH-SF; (2) to explore the differences in the WAI-TECH-SF scores according to different categories of the sample; and (3) to analyse whether the WAI-TECH-SF can predict therapeutic outcomes and satisfaction with the treatment. Methods: 193 patients diagnosed with depression were included and received blended Cognitive-Behavioural Therapy. Measures of preferences, satisfaction, and credibility about the treatment, TA with the online program, depressive symptoms, and satisfaction with the treatment were administered. Results: An exploratory factor analysis revealed a one-dimensional structure with adequate internal consistency. Linear regression analyses showed that the WAI-TECH-SF predicted changes in depressive symptoms and satisfaction with the treatment. Conclusions: WAI-TECH-SF is a reliable questionnaire to assess the TA between the patient and the online program, which is associated with positive therapeutic outcomes and satisfaction with the treatment.

Sharing clinical notes, and placebo and nocebo effects: Can documentation affect patient health?

This paper connects findings from the field of placebo studies with research into patients' interactions with their clinician's visit notes, housed in their electronic health records. We propose specific hypotheses about how features of clinicians' written notes might trigger mechanisms of placebo and nocebo effects to elicit positive or adverse health effects among patients. Bridging placebo studies with (a) survey data assaying patient and clinician experiences with portals and (b) randomized controlled trials provides preliminary support for our hypotheses. We conclude with actionable proposals for testing our understanding of the health effects of access to visit notes.

The Therapeutic Alliance in Digital Mental Health Interventions for Serious Mental Illnesses: Narrative Review

BACKGROUND

Digital mental health interventions offer unique advantages, and research indicates that these interventions are effective for a range of mental health concerns. Although these interventions are less established for individuals with serious mental illnesses, they demonstrate significant promise. A central consideration in traditional face-to-face therapies is the therapeutic alliance, whereas the nature of a digital therapeutic alliance and its relationship with outcomes requires further attention, particularly for individuals with serious mental illnesses.

OBJECTIVE

This narrative review aims to encourage further consideration and critical evaluation of the therapeutic alliance in digital mental health, specifically for individuals with serious mental illnesses.

METHODS

A narrative review was conducted by combining 3 main areas of the literature: the first examining the evidence for digital mental health interventions for serious mental illnesses, the second illuminating the nature and role of the therapeutic alliance in digital interventions, and the third surrounding practical considerations to enhance a digital therapeutic alliance.

RESULTS

Results indicated that a therapeutic alliance can be cultivated in digital interventions for those with serious mental illnesses, but that it may have unique, yet-to-be-confirmed characteristics in digital contexts. In addition, a therapeutic alliance appears to be less directly associated with outcomes in digital interventions than with those in face-to-face therapies. One possibility is that the digital therapeutic alliance is associated with increased engagement and adherence to digital interventions, through which it appears to influence outcomes. A number of design and implementation considerations may enhance the digital therapeutic alliance, including human support and technological features.

CONCLUSIONS

More research is required to further understand the nature and specific role of a therapeutic alliance in digital interventions for serious mental illnesses, particularly in informing their design. This review revealed several key research priorities to advance the therapeutic alliance in digital interventions.

AI in mental health

With the advent of digital approaches to mental health, modern artificial intelligence (AI), and machine learning in particular, is being used in the development of prediction, detection and treatment solutions for mental health care. In terms of treatment, AI is being incorporated into digital interventions, particularly web and smartphone apps, to enhance user experience and optimise personalised mental health care. In terms of prediction and detection, modern streams of abundant data mean that data-driven AI methods can be employed to develop prediction/detection models for mental health conditions. In particular, an individual's 'digital exhaust', the data gathered from their numerous personal digital device and social media interactions, can be mined for behavioural or mental health insights. Language, long considered a window into the human mind, can now be quantitatively harnessed as data with powerful computer-based natural language processing to also provide a method of inferring mental health. Furthermore, natural language processing can also be used to develop conversational agents used for therapeutic intervention.

Measuring Alliance Toward Embodied Virtual Therapists in the Era of Automated Treatments With the Virtual Therapist Alliance Scale (VTAS): Development and Psychometric Evaluation

BACKGROUND

Automated virtual reality exposure therapies (VRETs) are self-help treatments conducted by oneself and supported by a virtual therapist embodied visually and/or with audio feedback. This simulates many of the nonspecific relational elements and common factors present in face-to-face therapy and may be a means of improving adherence to and efficacy of self-guided treatments. However, little is known about alliance toward the virtual therapist, despite alliance being an important predictor of treatment outcome.

OBJECTIVE

In this study, we aimed to evaluate the first alliance instrument developed for use with embodied virtual therapists in an automated treatment format-the Virtual Therapist Alliance Scale (VTAS)-by (1) assessing its psychometric properties, (2) verifying the dimensionality of the scale, and (3) determining the predictive ability of the scale with treatment outcome.

METHODS

A psychometric evaluation and exploratory factor analysis of the VTAS was conducted using data from two samples of spider-fearful patients treated with VRET and the help of an embodied, voice-based virtual therapist (n=70). Multiple regression models and bivariate correlations were used to assess the VTAS relationship with treatment outcome, according to self-reported fear and convergence with presence and user-friendliness process measures.

RESULTS

The VTAS showed a sound two-factor solution composed of a primary factor covering task, goal, and copresence; adequate internal consistency; and good convergent validity, including moderate correlation (r=.310, P=.01) with outcomes over follow-up.

CONCLUSIONS

These preliminary results suggest that alliance toward a virtual therapist is a significant predictor of treatment outcome, favors the importance of a task-goal over bond-factor, and should be explored in studies with larger sample sizes and in additional forms of embodiment.

Delivery of Compassionate Mental Health Care in a Digital Technology-Driven Age: Scoping Review

Background

Compassion is a vital component to the achievement of positive health outcomes, particularly in mental health care. The rise of digital technologies may influence the delivery of compassionate care, and thus this relationship between compassion and digital health care needs to be better understood.

Objective

This scoping review aimed to identify existing digital technologies being used by patients and health professionals in the delivery of mental health care, understand how digital technologies are being used in the delivery of compassionate mental health care, and determine the facilitators of and barriers to digital technology use among patients and health professionals in the delivery of compassionate mental health care.

Methods

We conducted this scoping review through a search of Cumulative Index to Nursing and Allied Health Literature, Medical Literature Analysis and Retrieval System Online (MEDLINE), MEDLINE In-Process and EPub Ahead of Print, PsycINFO, and Web of Science for articles published from 1990 to 2019.

Results

Of the 4472 articles screened, 37 articles were included for data extraction. Telemedicine was the most widely used technology by mental health professionals. Digital technologies were described as facilitating compassionate care and were classified using a conceptual model to identify each digital intersection with compassionate care. Facilitators of and barriers to providing compassionate care through digital technology were identified, including increased safety for providers, health care professional perceptions and abilities, and the use of picture-in-picture feedback to evaluate social cues.

Conclusions

Implementing digital technology into mental health care can improve the current delivery of compassionate care and create novel ways to provide compassion. However, as this is a new area of study, mental health professionals and organizations alike should be mindful that compassionate human-centered care is maintained in the delivery of digital health care. Future research could develop tools to facilitate and evaluate the enactment of compassion within digital health care.

Examining Opportunities for Goal-Directed Self-Tracking to Support Chronic Condition Management

Although self-tracking offers potential for a more complete, accurate, and longer-term understanding of personal health, many people struggle with or fail to achieve their goals for health-related self-tracking. This paper investigates how to address challenges that result from current self-tracking tools leaving a person's goals for their data unstated and lacking explicit support. We examine supporting people and health providers in expressing and pursuing their tracking-related goals via goal-directed self-tracking, a novel method to represent relationships between tracking goals and underlying data. Informed by a reanalysis of data from a prior study of migraine tracking goals, we created a paper prototype to explore whether and how goal-directed self-tracking could address current disconnects between the goals people have for data in their chronic condition management and the tools they use to support such goals. We examined this prototype in interviews with 14 people with migraine and 5 health providers. Our findings indicate the potential for scaffolding goal-directed self-tracking to: 1) elicit different types and hierarchies of management and tracking goals; 2) help people prepare for all stages of self-tracking towards a specific goal; and 3) contribute additional expertise in patient-provider collaboration. Based on our findings, we present implications for the design of tools that explicitly represent and support an individual's specific self-tracking goals.

Design and Development of the "POD Adventures" Smartphone Game: A Blended Problem-Solving Intervention for Adolescent Mental Health in India

Introduction: Digital technology platforms offer unparalleled opportunities to reach vulnerable adolescents at scale and overcome many barriers that exist around conventional service provision. This paper describes the design and development of POD Adventures, a blended problem-solving game-based intervention for adolescents with or at risk of anxiety, depression and conduct difficulties in India. This intervention was developed as part of the PRemIum for ADolEscents (PRIDE) research programme, which aims to establish a suite of transdiagnostic psychological interventions organized around a stepped care system in Indian secondary schools. Methods and Materials: Intervention development followed a person-centered approach consisting of four iterative activities: (i) review of recent context-specific evidence on mental health needs and preferences for the target population of school-going Indian adolescents, including a multiple stakeholder analysis of school counseling priorities and pilot studies of a brief problem-solving intervention; (ii) new focus group discussions with N = 46 student participants and N = 8 service providers; (iii) co-design workshops with N = 22 student participants and N = 8 service providers; and (iv) user-testing with N = 50 student participants. Participants were aged 12-17 years and recruited from local schools in New Delhi and Goa, including a subgroup with self-identified mental health needs (N = 6). Results: Formative data from existing primary sources, new focus groups and co-design workshops supported a blended format for delivering a brief problem-solving intervention, with counselors supporting use of a game-based app on "offline" smartphones. User-testing with prototypes identified a need for simplification of language, use of concrete examples of concepts and practice elements to enhance engagement. There were also indications that participants most valued relatability and interactivity within real-world stories with judicious support from an in-app guide. The final prototype comprised a set of interactive and gamified vignettes and a structured set of problem-solving questions to consolidate and generalize learning while encouraging real-world application. Discussion: Findings shaped the design of POD Adventures and its delivery as an open-access blended intervention for secondary school students with a felt need for psychological support, consistent with an early intervention paradigm. A randomized controlled trial is planned to evaluate processes and impacts of POD Adventures when delivered for help-seeking students in low-resource school settings.

Technology-Based Mental Health Treatment and the Impact on the Therapeutic Alliance

PURPOSE OF REVIEW

Telemental health, which is treatment mediated by technology, is an increasingly common method of delivering mental health care. However, its impact on the therapeutic alliance is unclear. This review examines studies of telemental health and its impact on therapeutic alliance.

RECENT FINDINGS

Findings indicate that a therapeutic alliance can be maintained through a variety of communication technologies, with some caveats. Considerations on maintaining a successful therapeutic alliance include using technology as an adjunct to treatment and ensuring patients and providers have back-up plans for continuing communication in the event of technical difficulties. Overall, the studies found that clinicians often have more concerns about alliance than patients do, suggesting that clinicians could make some changes to feel more comfortable. Recommendations are offered for implementing techniques into practice that will help clinicians increase their awareness of ways to support the therapeutic alliance when using telemental health.

The digital revolution and its impact on mental health care

The digital revolution is evolving at an unstoppable pace. Alongside the unprecedented explosion of digital technology facilities and systems, mental health care is under greater pressure than ever before. With its emphasis on big data, computing power, mobile technology, and network information, digital technology is set to transform health care delivery. This article reviews the field of digital health technology assessment and intervention primarily in secondary service mental health care, including the barriers and facilitators to adopting and implementing digitally mediated interventions in service delivery. We consider the impact of digitally mediated communication on human interaction and its potential impact on various mental states such as those linked to mood, anxiety but also well-being. These developments point to a need for both theory- and data-driven approaches to digital health care. We argue that, as developments in digital technology are outpacing the evaluation of rigorous digital health interventions, more advanced methodologies are needed to keep up with the pace of digital technology development. The need for co-production of digital tools with and for people with chronic and mental health difficulties, and implications of digital technology for psychotherapy practice, will be central to this development. PRACTITIONER POINTS: Mental health problems are one of the main causes of global and societal burden and are a growing public health. People with mental health problems around the world have limited, if any, chance of accessing psychological help at all. Technological innovations and solutions are being considered in an attempt to address the size and scale of the mental health crisis worldwide. Digital platforms allow people to self-monitor and self-manage in a way that face-to-face/paper-based methods of assessment have up until now not allowed. We provide examples of digital tools that are being developed and used in the secondary setting and identify a number of challenges in the digital health field that require careful consideration.

"They Are Not Hard-to-Reach Clients. We Have Just Got Hard-to-Reach Services." Staff Views of Digital Health Tools in Specialist Mental Health Services

Background: Digital health products designed to help people with severe mental health problems appear to be feasible, acceptable, and efficacious. The challenge facing the digital mental health field is implementing digital tools in routine service delivery. To date, there has been a paucity of qualitative research exploring staff views of digital health solutions in the context of mental healthcare. Engaging and involving frontline staff in the design and rollout of new technology to improve utilization is imperative for successful uptake and adoption of digital tools. The aim of the current study is to explore frontline staff views regarding the utility and appropriateness of using digital tools in the healthcare pathway for people accessing specialist secondary care mental health services. Method: Qualitative study using framework analysis was used with 48 mental health staff working in early intervention for psychosis services. Six groups comprising 5-10 early intervention service staff members in each group were conducted across the Northwest of England. Robust measures were used to develop a stable framework, including member checking, triangulation, and consensus meetings. Results: Three themes were identified a priori: i) perceived barriers to adopting smartphone apps for early psychosis; ii) acceptability of digital health tools for early psychosis patients; and iii) data security, safety, and risk. Alongside exploring the a priori topics, one theme was generated a posteriori: iv) relationships. Conclusions: Staff working in specialist early intervention for psychosis services found digital tools on the whole acceptable in mental health service provision, but raised a number of concerns that will likely affect implementation of such systems into routine service delivery and practice. Thirteen recommendations are made in this paper as a result of the themes generated in these data. Implementing of digital systems needs to be simple and uncomplicated and improve clinical workflows for staff rather than hinder and increase clinical workflows. Furthermore, organizational support with a clear plan for implementing technological innovations is required for successful adoption of digital systems. Consideration of staff views around digital systems is important if successful adoption and implementation of such systems are to occur. Clinical Trial Registration: http://www.isrctn.com, identifier ISRCTN34966555.