The Effects of Health Care Chatbot Personas With Different Social Roles on the Client-Chatbot Bond and Usage Intentions: Development of a Design Codebook and Web-Based Study

Determinants of Dropout From a Virtual Agent-Based App for Insomnia Management in a Self-Selected Sample of Users With Insomnia Symptoms: Longitudinal Study

Background

Fully automated digital interventions delivered via smartphone apps have proven efficacious for a wide variety of mental health outcomes. An important aspect is that they are accessible at a low cost, thereby increasing their potential public impact and reducing disparities. However, a major challenge to their successful implementation is the phenomenon of users dropping out early.

Objective

The purpose of this study was to pinpoint the factors influencing early dropout in a sample of self-selected users of a virtual agent (VA)-based behavioral intervention for managing insomnia, named KANOPEE, which is freely available in France.

Methods

From January 2021 to December 2022, of the 9657 individuals, aged 18 years or older, who downloaded and completed the KANOPEE screening interview and had either subclinical or clinical insomnia symptoms, 4295 (44.5%) dropped out (ie, did not return to the app to continue filling in subsequent assessments). The primary outcome was a binary variable: having dropped out after completing the screening assessment (early dropout) or having completed all the treatment phases (n=551). Multivariable logistic regression analysis was used to identify predictors of dropout among a set of sociodemographic, clinical, and sleep diary variables, and users' perceptions of the treatment program, collected during the screening interview.

Results

The users' mean age was 47.95 (SD 15.21) years. Of those who dropped out early and those who completed the treatment, 65.1% (3153/4846) were women and 34.9% (1693/4846) were men. Younger age (adjusted odds ratio [AOR] 0.98, 95% CI 0.97-0.99), lower education level (compared to middle school; high school: AOR 0.56, 95% CI 0.35-0.90; bachelor's degree: AOR 0.35, 95% CI 0.23-0.52; master's degree or higher: AOR 0.35, 95% CI 0.22-0.55), poorer nocturnal sleep (sleep efficiency: AOR 0.64, 95% CI 0.42-0.96; number of nocturnal awakenings: AOR 1.13, 95% CI 1.04-1.23), and more severe depression symptoms (AOR 1.12, 95% CI 1.04-1.21) were significant predictors of dropping out. When measures of perceptions of the app were included in the model, perceived benevolence and credibility of the VA decreased the odds of dropout (AOR 0.91, 95% CI 0.85-0.97).

Conclusions

As in traditional face-to-face cognitive behavioral therapy for insomnia, the presence of significant depression symptoms plays an important role in treatment dropout. This variable represents an important target to address to increase early engagement with fully automated insomnia management programs. Furthermore, our results support the contention that a VA can provide relevant user stimulation that will eventually pay out in terms of user engagement.

Ethical Considerations in Human-Centered AI: Advancing Oncology Chatbots Through Large Language Models

The integration of chatbots in oncology underscores the pressing need for human-centered artificial intelligence (AI) that addresses patient and family concerns with empathy and precision. Human-centered AI emphasizes ethical principles, empathy, and user-centric approaches, ensuring technology aligns with human values and needs. This review critically examines the ethical implications of using large language models (LLMs) like GPT-3 and GPT-4 (OpenAI) in oncology chatbots. It examines how these models replicate human-like language patterns, impacting the design of ethical AI systems. The paper identifies key strategies for ethically developing oncology chatbots, focusing on potential biases arising from extensive datasets and neural networks. Specific datasets, such as those sourced from predominantly Western medical literature and patient interactions, may introduce biases by overrepresenting certain demographic groups. Moreover, the training methodologies of LLMs, including fine-tuning processes, can exacerbate these biases, leading to outputs that may disproportionately favor affluent or Western populations while neglecting marginalized communities. By providing examples of biased outputs in oncology chatbots, the review highlights the ethical challenges LLMs present and the need for mitigation strategies. The study emphasizes integrating human-centric values into AI to mitigate these biases, ultimately advocating for the development of oncology chatbots that are aligned with ethical principles and capable of serving diverse patient populations equitably.

Perceptions and expectations of an artificially intelligent physical activity digital assistant - A focus group study

Artificially intelligent physical activity digital assistants that use the full spectrum of machine learning capabilities have not yet been developed and examined. This study aimed to explore potential users' perceptions and expectations of using such a digital assistant. Six 90-min online focus group meetings (n = 45 adults) were conducted. Meetings were recorded, transcribed and thematically analysed. Participants embraced the idea of a 'digital assistant' providing physical activity support. Participants indicated they would like to receive notifications from the digital assistant, but did not agree on the number, timing, tone and content of notifications. Likewise, they indicated that the digital assistant's personality and appearance should be customisable. Participants understood the need to provide information to the digital assistant to allow for personalisation, but varied greatly in the extent of information that they were willing to provide. Privacy issues aside, participants embraced the idea of using artificial intelligence or machine learning in return for a more functional and personal digital assistant. In sum, participants were ready for an artificially intelligent physical activity digital assistant but emphasised a need to personalise or customise nearly every feature of the application. This poses challenges in terms of cost and complexity of developing the application.

Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review

BACKGROUND

Chatbots, or conversational agents, have emerged as significant tools in health care, driven by advancements in artificial intelligence and digital technology. These programs are designed to simulate human conversations, addressing various health care needs. However, no comprehensive synthesis of health care chatbots' roles, users, benefits, and limitations is available to inform future research and application in the field.

OBJECTIVE

This review aims to describe health care chatbots' characteristics, focusing on their diverse roles in the health care pathway, user groups, benefits, and limitations.

METHODS

A rapid review of published literature from 2017 to 2023 was performed with a search strategy developed in collaboration with a health sciences librarian and implemented in the MEDLINE and Embase databases. Primary research studies reporting on chatbot roles or benefits in health care were included. Two reviewers dual-screened the search results. Extracted data on chatbot roles, users, benefits, and limitations were subjected to content analysis.

RESULTS

The review categorized chatbot roles into 2 themes: delivery of remote health services, including patient support, care management, education, skills building, and health behavior promotion, and provision of administrative assistance to health care providers. User groups spanned across patients with chronic conditions as well as patients with cancer; individuals focused on lifestyle improvements; and various demographic groups such as women, families, and older adults. Professionals and students in health care also emerged as significant users, alongside groups seeking mental health support, behavioral change, and educational enhancement. The benefits of health care chatbots were also classified into 2 themes: improvement of health care quality and efficiency and cost-effectiveness in health care delivery. The identified limitations encompassed ethical challenges, medicolegal and safety concerns, technical difficulties, user experience issues, and societal and economic impacts.

CONCLUSIONS

Health care chatbots offer a wide spectrum of applications, potentially impacting various aspects of health care. While they are promising tools for improving health care efficiency and quality, their integration into the health care system must be approached with consideration of their limitations to ensure optimal, safe, and equitable use.

The Role of Humanization and Robustness of Large Language Models in Conversational Artificial Intelligence for Individuals With Depression: A Critical Analysis

Unlabelled

Large language model (LLM)-powered services are gaining popularity in various applications due to their exceptional performance in many tasks, such as sentiment analysis and answering questions. Recently, research has been exploring their potential use in digital health contexts, particularly in the mental health domain. However, implementing LLM-enhanced conversational artificial intelligence (CAI) presents significant ethical, technical, and clinical challenges. In this viewpoint paper, we discuss 2 challenges that affect the use of LLM-enhanced CAI for individuals with mental health issues, focusing on the use case of patients with depression: the tendency to humanize LLM-enhanced CAI and their lack of contextualized robustness. Our approach is interdisciplinary, relying on considerations from philosophy, psychology, and computer science. We argue that the humanization of LLM-enhanced CAI hinges on the reflection of what it means to simulate "human-like" features with LLMs and what role these systems should play in interactions with humans. Further, ensuring the contextualization of the robustness of LLMs requires considering the specificities of language production in individuals with depression, as well as its evolution over time. Finally, we provide a series of recommendations to foster the responsible design and deployment of LLM-enhanced CAI for the therapeutic support of individuals with depression.

Attrition in Conversational Agent-Delivered Mental Health Interventions: Systematic Review and Meta-Analysis

BACKGROUND

Conversational agents (CAs) or chatbots are computer programs that mimic human conversation. They have the potential to improve access to mental health interventions through automated, scalable, and personalized delivery of psychotherapeutic content. However, digital health interventions, including those delivered by CAs, often have high attrition rates. Identifying the factors associated with attrition is critical to improving future clinical trials.

OBJECTIVE

This review aims to estimate the overall and differential rates of attrition in CA-delivered mental health interventions (CA interventions), evaluate the impact of study design and intervention-related aspects on attrition, and describe study design features aimed at reducing or mitigating study attrition.

METHODS

We searched PubMed, Embase (Ovid), PsycINFO (Ovid), Cochrane Central Register of Controlled Trials, and Web of Science, and conducted a gray literature search on Google Scholar in June 2022. We included randomized controlled trials that compared CA interventions against control groups and excluded studies that lasted for 1 session only and used Wizard of Oz interventions. We also assessed the risk of bias in the included studies using the Cochrane Risk of Bias Tool 2.0. Random-effects proportional meta-analysis was applied to calculate the pooled dropout rates in the intervention groups. Random-effects meta-analysis was used to compare the attrition rate in the intervention groups with that in the control groups. We used a narrative review to summarize the findings.

RESULTS

The systematic search retrieved 4566 records from peer-reviewed databases and citation searches, of which 41 (0.90%) randomized controlled trials met the inclusion criteria. The meta-analytic overall attrition rate in the intervention group was 21.84% (95% CI 16.74%-27.36%; I2=94%). Short-term studies that lasted ≤8 weeks showed a lower attrition rate (18.05%, 95% CI 9.91%- 27.76%; I2=94.6%) than long-term studies that lasted >8 weeks (26.59%, 95% CI 20.09%-33.63%; I2=93.89%). Intervention group participants were more likely to attrit than control group participants for short-term (log odds ratio 1.22, 95% CI 0.99-1.50; I2=21.89%) and long-term studies (log odds ratio 1.33, 95% CI 1.08-1.65; I2=49.43%). Intervention-related characteristics associated with higher attrition include stand-alone CA interventions without human support, not having a symptom tracker feature, no visual representation of the CA, and comparing CA interventions with waitlist controls. No participant-level factor reliably predicted attrition.

CONCLUSIONS

Our results indicated that approximately one-fifth of the participants will drop out from CA interventions in short-term studies. High heterogeneities made it difficult to generalize the findings. Our results suggested that future CA interventions should adopt a blended design with human support, use symptom tracking, compare CA intervention groups against active controls rather than waitlist controls, and include a visual representation of the CA to reduce the attrition rate.

TRIAL REGISTRATION

PROSPERO International Prospective Register of Systematic Reviews CRD42022341415; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022341415.

Public opinion on types of voice systems for older adults

Public opinion may influence the adoption of technologies for older adults, yet studies on different contexts of technology for older adults is limited. In an online YouGov survey (N = 500) with text-and-image vignettes, participants gave more positive ratings of social acceptability, trust, and perceived impact on eldercare when the voice assistant ("VA" system) shown in the vignette performed a functional task (medication adherence) versus when it performed a social task (companionship). The VA received more positive sentiment comments when it appeared to use a machine learning (ML)-based dialogue system compared to when it appeared to be using a rule-based dialogue system. These results may assist designers and stakeholders select what type of voice system to develop or use with older adults.

Waiting for a digital therapist: three challenges on the path to psychotherapy delivered by artificial intelligence

Growing demand for broadly accessible mental health care, together with the rapid development of new technologies, trigger discussions about the feasibility of psychotherapeutic interventions based on interactions with Conversational Artificial Intelligence (CAI). Many authors argue that while currently available CAI can be a useful supplement for human-delivered psychotherapy, it is not yet capable of delivering fully fledged psychotherapy on its own. The goal of this paper is to investigate what are the most important obstacles on our way to developing CAI systems capable of delivering psychotherapy in the future. To this end, we formulate and discuss three challenges central to this quest. Firstly, we might not be able to develop effective AI-based psychotherapy unless we deepen our understanding of what makes human-delivered psychotherapy effective. Secondly, assuming that it requires building a therapeutic relationship, it is not clear whether psychotherapy can be delivered by non-human agents. Thirdly, conducting psychotherapy might be a problem too complicated for narrow AI, i.e., AI proficient in dealing with only relatively simple and well-delineated tasks. If this is the case, we should not expect CAI to be capable of delivering fully-fledged psychotherapy until the so-called "general" or "human-like" AI is developed. While we believe that all these challenges can ultimately be overcome, we think that being mindful of them is crucial to ensure well-balanced and steady progress on our path to AI-based psychotherapy.

Development of "LvL UP 1.0": a smartphone-based, conversational agent-delivered holistic lifestyle intervention for the prevention of non-communicable diseases and common mental disorders

Background

Non-communicable diseases (NCDs) and common mental disorders (CMDs) are the leading causes of death and disability worldwide. Lifestyle interventions via mobile apps and conversational agents present themselves as low-cost, scalable solutions to prevent these conditions. This paper describes the rationale for, and development of, "LvL UP 1.0″, a smartphone-based lifestyle intervention aimed at preventing NCDs and CMDs.

Materials and Methods

A multidisciplinary team led the intervention design process of LvL UP 1.0, involving four phases: (i) preliminary research (stakeholder consultations, systematic market reviews), (ii) selecting intervention components and developing the conceptual model, (iii) whiteboarding and prototype design, and (iv) testing and refinement. The Multiphase Optimization Strategy and the UK Medical Research Council framework for developing and evaluating complex interventions were used to guide the intervention development.

Results

Preliminary research highlighted the importance of targeting holistic wellbeing (i.e., both physical and mental health). Accordingly, the first version of LvL UP features a scalable, smartphone-based, and conversational agent-delivered holistic lifestyle intervention built around three pillars: Move More (physical activity), Eat Well (nutrition and healthy eating), and Stress Less (emotional regulation and wellbeing). Intervention components include health literacy and psychoeducational coaching sessions, daily "Life Hacks" (healthy activity suggestions), breathing exercises, and journaling. In addition to the intervention components, formative research also stressed the need to introduce engagement-specific components to maximise uptake and long-term use. LvL UP includes a motivational interviewing and storytelling approach to deliver the coaching sessions, as well as progress feedback and gamification. Offline materials are also offered to allow users access to essential intervention content without needing a mobile device.

Conclusions

The development process of LvL UP 1.0 led to an evidence-based and user-informed smartphone-based intervention aimed at preventing NCDs and CMDs. LvL UP is designed to be a scalable, engaging, prevention-oriented, holistic intervention for adults at risk of NCDs and CMDs. A feasibility study, and subsequent optimisation and randomised-controlled trials are planned to further refine the intervention and establish effectiveness. The development process described here may prove helpful to other intervention developers.

Exploring the potential of mobile health interventions to address behavioural risk factors for the prevention of non-communicable diseases in Asian populations: a qualitative study

BACKGROUND

Changing lifestyle patterns over the last decades have seen growing numbers of people in Asia affected by non-communicable diseases and common mental health disorders, including diabetes, cancer, and/or depression. Interventions targeting healthy lifestyle behaviours through mobile technologies, including new approaches such as chatbots, may be an effective, low-cost approach to prevent these conditions. To ensure uptake and engagement with mobile health interventions, however, it is essential to understand the end-users' perspectives on using such interventions. The aim of this study was to explore perceptions, barriers, and facilitators to the use of mobile health interventions for lifestyle behaviour change in Singapore.

METHODS

Six virtual focus group discussions were conducted with a total of 34 participants (mean ± SD; aged 45 ± 3.6 years; 64.7% females). Focus group recordings were transcribed verbatim and analysed using an inductive thematic analysis approach, followed by deductive mapping according to perceptions, barriers, facilitators, mixed factors, or strategies.

RESULTS

Five themes were identified: (i) holistic wellbeing is central to healthy living (i.e., the importance of both physical and mental health); (ii) encouraging uptake of a mobile health intervention is influenced by factors such as incentives and government backing; (iii) trying out a mobile health intervention is one thing, sticking to it long term is another and there are key factors, such as personalisation and ease of use that influence sustained engagement with mobile health interventions; (iv) perceptions of chatbots as a tool to support healthy lifestyle behaviour are influenced by previous negative experiences with chatbots, which might hamper uptake; and (v) sharing health-related data is OK, but with conditions such as clarity on who will have access to the data, how it will be stored, and for what purpose it will be used.

CONCLUSIONS

Findings highlight several factors that are relevant for the development and implementation of mobile health interventions in Singapore and other Asian countries. Recommendations include: (i) targeting holistic wellbeing, (ii) tailoring content to address environment-specific barriers, (iii) partnering with government and/or local (non-profit) institutions in the development and/or promotion of mobile health interventions, (iv) managing expectations regarding the use of incentives, and (iv) identifying potential alternatives or complementary approaches to the use of chatbots, particularly for mental health.

Evaluating Conversational Agents for Mental Health: Scoping Review of Outcomes and Outcome Measurement Instruments

BACKGROUND

Rapid proliferation of mental health interventions delivered through conversational agents (CAs) calls for high-quality evidence to support their implementation and adoption. Selecting appropriate outcomes, instruments for measuring outcomes, and assessment methods are crucial for ensuring that interventions are evaluated effectively and with a high level of quality.

OBJECTIVE

We aimed to identify the types of outcomes, outcome measurement instruments, and assessment methods used to assess the clinical, user experience, and technical outcomes in studies that evaluated the effectiveness of CA interventions for mental health.

METHODS

We undertook a scoping review of the relevant literature to review the types of outcomes, outcome measurement instruments, and assessment methods in studies that evaluated the effectiveness of CA interventions for mental health. We performed a comprehensive search of electronic databases, including PubMed, Cochrane Central Register of Controlled Trials, Embase (Ovid), PsychINFO, and Web of Science, as well as Google Scholar and Google. We included experimental studies evaluating CA mental health interventions. The screening and data extraction were performed independently by 2 review authors in parallel. Descriptive and thematic analyses of the findings were performed.

RESULTS

We included 32 studies that targeted the promotion of mental well-being (17/32, 53%) and the treatment and monitoring of mental health symptoms (21/32, 66%). The studies reported 203 outcome measurement instruments used to measure clinical outcomes (123/203, 60.6%), user experience outcomes (75/203, 36.9%), technical outcomes (2/203, 1.0%), and other outcomes (3/203, 1.5%). Most of the outcome measurement instruments were used in only 1 study (150/203, 73.9%) and were self-reported questionnaires (170/203, 83.7%), and most were delivered electronically via survey platforms (61/203, 30.0%). No validity evidence was cited for more than half of the outcome measurement instruments (107/203, 52.7%), which were largely created or adapted for the study in which they were used (95/107, 88.8%).

CONCLUSIONS

The diversity of outcomes and the choice of outcome measurement instruments employed in studies on CAs for mental health point to the need for an established minimum core outcome set and greater use of validated instruments. Future studies should also capitalize on the affordances made available by CAs and smartphones to streamline the evaluation and reduce participants' input burden inherent to self-reporting.