Development and usability testing of a chatbot to promote mental health services use among individuals with eating disorders following screening

OBJECTIVE

A significant gap exists between those who need and those who receive care for eating disorders (EDs). Novel solutions are needed to encourage service use and address treatment barriers. This study developed and evaluated the usability of a chatbot designed for pairing with online ED screening. The tool aimed to promote mental health service utilization by improving motivation for treatment and self-efficacy among individuals with EDs.

METHODS

A chatbot prototype, Alex, was designed using decision trees and theoretically-informed components: psychoeducation, motivational interviewing, personalized recommendations, and repeated administration. Usability testing was conducted over four iterative cycles, with user feedback informing refinements to the next iteration. Post-testing, participants (N= 21) completed the System Usability Scale (SUS), the Usefulness, Satisfaction, and Ease of Use Questionnaire (USE), and a semi-structured interview.

RESULTS

Interview feedback detailed chatbot aspects participants enjoyed and aspects necessitating improvement. Feedback converged on four themes: user experience, chatbot qualities, chatbot content, and ease of use. Following refinements, users described Alex as humanlike, supportive, and encouraging. Content was perceived as novel and personally relevant. USE scores across domains were generally above average (~5 out of 7), and SUS scores indicated "good" to "excellent" usability across cycles, with the final iteration receiving the highest average score.

DISCUSSION

Overall, participants generally reflected positively on interactions with Alex, including the initial version. Refinements between cycles further improved user experiences. This study provides preliminary evidence of the feasibility and acceptance of a chatbot designed to promote motivation for and use of services among individuals with EDs.

PUBLIC SIGNIFICANCE

Low rates of service utilization and treatment have been observed among individuals following online eating disorder screening. Tools are needed, including scalable, digital options, that can be easily paired with screening, to improve motivation for addressing eating disorders and promote service utilization.

Digital interventions to address mental health needs in colleges: Perspectives of student stakeholders

Objective

The need for clinical services in U.S. colleges exceeds the supply. Digital Mental health Interventions (DMHIs) are a potential solution, but successful implementation depends on stakeholder acceptance. This study investigated the relevance of DMHIs from students' perspectives.

Methods

In 2020-2021, an online cross-sectional survey using mixed methods was conducted with 479 students at 23 colleges and universities. Respondents reported views and use of standard mental health services and DMHIs and rated the priority of various DMHIs to be offered through campus services. Qualitative data included open-ended responses.

Findings

Among respondents, 91% reported having experienced mental health problems, of which 91% reported barriers to receiving mental health services. Students highlighted therapy and counseling as desired and saw flexible access to services as important. With respect to DMHIs, respondents had the most experience with physical health apps (46%), mental health questionnaires (41%), and mental well-being apps (39%). Most were unaware of or had not used apps or self-help programs for mental health problems. Students were most likely to report the following DMHIs as high priorities: a crisis text line (76%), telehealth (66%), websites for connecting to services (62%), and text/messaging with counselors (62%). They considered a self-help program with coach support to be convenient but some also perceived such services to be possibly less effective than in-person therapy.

Conclusions

Students welcome DMHIs on campus and indicate preference for mental health services that include human support. The findings, with particular focus on characteristics of the DMHIs prioritized, and students' awareness and perceptions of scalable DMHIs emphasized by policymakers, should inform schools looking to implement DMHIs.

Effectiveness of a chatbot for eating disorders prevention: A randomized clinical trial

OBJECTIVE

Prevention of eating disorders (EDs) is of high importance. However, digital programs with human moderation are unlikely to be disseminated widely. The aim of this study was to test whether a chatbot (i.e., computer program simulating human conversation) would significantly reduce ED risk factors (i.e., weight/shape concerns, thin-ideal internalization) in women at high risk for an ED, compared to waitlist control, as well as whether it would significantly reduce overall ED psychopathology, depression, and anxiety and prevent ED onset.

METHOD

Women who screened as high risk for an ED were randomized (N = 700) to (1) chatbot based on the StudentBodies© program; or (2) waitlist control. Participants were followed for 6 months.

RESULTS

For weight/shape concerns, there was a significantly greater reduction in intervention versus control at 3- (d = -0.20; p = .03) and 6-m-follow-up (d = -0.19; p = .04). There were no differences in change in thin-ideal internalization. The intervention was associated with significantly greater reductions than control in overall ED psychopathology at 3- (d = -0.29; p = .003) but not 6-month follow-up. There were no differences in change in depression or anxiety. The odds of remaining nonclinical for EDs were significantly higher in intervention versus control at both 3- (OR = 2.37, 95% CI [1.37, 4.11]) and 6-month follow-ups (OR = 2.13, 95% CI [1.26, 3.59]).

DISCUSSION

Findings provide support for the use of a chatbot-based EDs prevention program in reducing weight/shape concerns through 6-month follow-up, as well as in reducing overall ED psychopathology, at least in the shorter-term. Results also suggest the intervention may reduce ED onset.

PUBLIC SIGNIFICANCE

We found that a chatbot, or a computer program simulating human conversation, based on an established, cognitive-behavioral therapy-based eating disorders prevention program, was successful in reducing women's concerns about weight and shape through 6-month follow-up and that it may actually reduce eating disorder onset. These findings are important because this intervention, which uses a rather simple text-based approach, can easily be disseminated in order to prevent these deadly illnesses.

TRIAL REGISTRATION

OSF Registries; https://osf.io/7zmbv.

The Challenges in Designing a Prevention Chatbot for Eating Disorders: Observational Study

BACKGROUND

Chatbots have the potential to provide cost-effective mental health prevention programs at scale and increase interactivity, ease of use, and accessibility of intervention programs.

OBJECTIVE

The development of chatbot prevention for eating disorders (EDs) is still in its infancy. Our aim is to present examples of and solutions to challenges in designing and refining a rule-based prevention chatbot program for EDs, targeted at adult women at risk for developing an ED.

METHODS

Participants were 2409 individuals who at least began to use an EDs prevention chatbot in response to social media advertising. Over 6 months, the research team reviewed up to 52,129 comments from these users to identify inappropriate responses that negatively impacted users' experience and technical glitches. Problems identified by reviewers were then presented to the entire research team, who then generated possible solutions and implemented new responses.

RESULTS

The most common problem with the chatbot was a general limitation in understanding and responding appropriately to unanticipated user responses. We developed several workarounds to limit these problems while retaining some interactivity.

CONCLUSIONS

Rule-based chatbots have the potential to reach large populations at low cost but are limited in understanding and responding appropriately to unanticipated user responses. They can be most effective in providing information and simple conversations. Workarounds can reduce conversation errors.

Preliminary data on help-seeking intentions and behaviors of individuals completing a widely available online screen for eating disorders in the United States

OBJECTIVE

Scaling an online screen that provides referrals may be key in closing the treatment gap for eating disorders (EDs), but we need to understand respondents' help-seeking intentions and behaviors after receiving screen results. This study reported on these constructs among respondents to the National Eating Disorders Association online screen who screened positive or at high risk for an ED.

METHOD

Respondents completed the screen over 18 months (February 9, 2018-August 28, 2019). Those screening positive or at high risk for an ED (n = 343,072) had the option to provide data on help-seeking intentions (after screen completion) and behaviors (2-month follow-up).

RESULTS

Of eligible respondents, 4.8% (n = 16,396) provided data on help-seeking intentions, with only 33.7% of those reporting they would seek help. Only 7.6% of eligible respondents opted in to the 2-month follow-up, with 10.6% of those completing it (n = 2,765). Overall, 8.9% of respondents to the follow-up reported being in treatment when they took the screen, 15.5% subsequently initiated treatment, and 75.5% did not initiate/were not already in treatment.

DISCUSSION

Preliminary results suggest that among the small minority who provided data, only one-third expressed help-seeking intentions and 16% initiated treatment. Online screening should consider ways to increase respondents' motivation for and follow-through with care.