Engaging women with an embodied conversational agent to deliver mindfulness and lifestyle recommendations: A feasibility randomized control trial

User Experiences of Well-Being Chatbots

OBJECTIVE

The current paper conducted two parallel studies to explore user experiences of well-being conversational agents (CAs) and identify important features for engagement.

BACKGROUND

Students transitioning into university life take on greater responsibility, yet tend to sacrifice healthy behaviors to strive for academic and financial gain. Additionally, students faced an unprecedented pandemic, leading to remote courses and reduced access to healthcare services. One tool designed to improve healthcare accessibility is well-being CAs. CAs have addressed mental health support in the general population but have yet to address physical well-being support and accessibility to those in disadvantaged socio-economic backgrounds where healthcare access is further limited.

METHOD

Study One comprised a thematic analysis of mental health applications featuring CAs from the public forum, Reddit. Study Two explored emerging usability themes of an SMS-based CA designed to improve accessibility to well-being services alongside a commercially available CA, Woebot.

RESULTS

Study One identified several themes, including accessibility and availability, communication style, and anthropomorphism as important features. Study Two identified themes such as user response modality, perceived CA role, question specificity, and conversation flow control as critical for user engagement.

CONCLUSION

Various themes emerged from individuals' experiences regarding CA features, functionality, and responses. The mixed experiences relevant to the communication and conversational styles between the CA and the user suggest varied motivations for using CAs for mental and physical well-being.

APPLICATION

Practical recommendations to encourage continued use include providing dynamic response modalities, anthropomorphizing the chatbot, and calibrating expectations early.

A Factorial Randomized Controlled Trial to Optimize User Engagement With a Chatbot-Led Parenting Intervention: Protocol for the ParentText Optimisation Trial

BACKGROUND

Violence against children (VAC) is a serious public health concern with long-lasting adverse effects. Evidence-based parenting programs are one effective means to prevent VAC; however, these interventions are not scalable in their typical in-person group format, especially in low- and middle-income countries where the need is greatest. While digital delivery, including via chatbots, offers a scalable and cost-effective means to scale up parenting programs within these settings, it is crucial to understand the key pillars of user engagement to ensure their effective implementation.

OBJECTIVE

This study aims to investigate the most effective and cost-effective combination of external components to optimize user engagement with ParentText, an open-source chatbot-led parenting intervention to prevent VAC in Mpumalanga, South Africa.

METHODS

This study will use a mixed methods design incorporating a 2 × 2 factorial cluster-randomized controlled trial and qualitative interviews. Parents of adolescent girls (32 clusters, 120 participants [60 parents and 60 girls aged 10 to 17 years] per cluster; N=3840 total participants) will be recruited from the Ehlanzeni and Nkangala districts of Mpumalanga. Clusters will be randomly assigned to receive 1 of the 4 engagement packages that include ParentText alone or combined with in-person sessions and a facilitated WhatsApp support group. Quantitative data collected will include pretest-posttest parent- and adolescent-reported surveys, facilitator-reported implementation data, and digitally tracked engagement data. Qualitative data will be collected from parents and facilitators through in-person or over-the-phone individual semistructured interviews and used to expand the interpretation and understanding of the quantitative findings.

RESULTS

Recruitment and data collection started in August 2023 and were finalized in November 2023. The total number of participants enrolled in the study is 1009, with 744 caregivers having completed onboarding to the chatbot-led intervention. Female participants represent 92.96% (938/1009) of the sample population, whereas male participants represent 7.03% (71/1009). The average participant age is 43 (SD 9) years.

CONCLUSIONS

The ParentText Optimisation Trial is the first study to rigorously test engagement with a chatbot-led parenting intervention in a low- or middle-income country. The results of this study will inform the final selection of external delivery components to support engagement with ParentText in preparation for further evaluation in a randomized controlled trial in 2024.

TRIAL REGISTRATION

Open Science Framework (OSF); https://doi.org/10.17605/OSF.IO/WFXNE.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/52145.

Digital human ergonomics training for remote office workers: Comparing a novel method to a traditional online format

OBJECTIVE

This randomized controlled trial investigated the effectiveness of an online ergonomics training program with a digital human compared to an online ergonomics training program without a digital human.

METHOD

Remote office workers (n = 138) were randomly assigned to either a digital human training, a traditional webpage training without a digital human, or a control group. Musculoskeletal discomfort, knowledge retention, and behavior change were measured.

RESULTS

The overall group differences for increased behavior change and knowledge retention were statistically significant (p < 0.05). For knowledge retention, the digital human training group showed comparable improvement in knowledge scores compared to the traditional training group. For behavior scores, the traditional training showed improvement compared to the control group. Decreases in musculoskeletal discomfort for all groups were not statistically significant (p > 0.05).

CONCLUSION

Digital humans have the potential to meet large-scale remote worker training needs.

Development and Evaluation of a Digital App for Patient Self-Management of Opioid Use Disorder: Usability, Acceptability, and Utility Study

BACKGROUND

Self-management of opioid use disorder (OUD) is an important component of treatment. Many patients receiving opioid agonist treatment in methadone maintenance treatment settings benefit from counseling treatments to help them improve their recovery skills but have insufficient access to these treatments between clinic appointments. In addition, many addiction medicine clinicians treating patients with OUD in a general medical clinic setting do not have consistent access to counseling referrals for their patients. This can lead to decreases in both treatment retention and overall progress in the patient's recovery from substance misuse. Digital apps may help to bridge this gap by coaching, supporting, and reinforcing behavioral change that is initiated and directed by their psychosocial and medical providers.

OBJECTIVE

This study aimed to conduct an acceptability, usability, and utility pilot study of the KIOS app to address these clinical needs.

METHODS

We developed a unique, patient-centered computational software system (KIOS; Biomedical Development Corporation) to assist in managing OUD in an outpatient, methadone maintenance clinic setting. KIOS tracks interacting self-reported symptoms (craving, depressed mood, anxiety, irritability, pain, agitation or restlessness, difficulty sleeping, absenteeism, difficulty with usual activities, and conflicts with others) to determine changes in both the trajectory and severity of symptom patterns over time. KIOS then applies a proprietary algorithm to assess the individual's patterns of symptom interaction in accordance with models previously established by OUD experts. After this analysis, KIOS provides specific behavioral advice addressing the individual's changing trajectory of symptoms to help the person self-manage their symptoms. The KIOS software also provides analytics on the self-reported data that can be used by patients, clinicians, and researchers to track outcomes.

RESULTS

In a 4-week acceptability, usability (mean System Usability Scale-Modified score 89.5, SD 9.2, maximum of 10.0), and utility (mean KIOS utility questionnaire score 6.32, SD 0.25, maximum of 7.0) pilot study of 15 methadone-maintained participants with OUD, user experience, usability, and software-generated advice received high and positive assessment scores. The KIOS clinical variables closely correlated with craving self-report measures. Therefore, managing these variables with advice generated by the KIOS software could have an impact on craving and ultimately substance use.

CONCLUSIONS

KIOS tracks key clinical variables and generates advice specifically relevant to the patient's current and changing clinical state. Patients in this pilot study assigned high positive values to the KIOS user experience, ease of use, and the appropriateness, relevance, and usefulness of the specific behavioral guidance they received to match their evolving experiences. KIOS may therefore be useful to augment in-person treatment of opioid agonist patients and help fill treatment gaps that currently exist in the continuum of care. A National Institute on Drug Abuse-funded randomized controlled trial of KIOS to augment in-person treatment of patients with OUD is currently being conducted.

The potential and limitations of large language models in identification of the states of motivations for facilitating health behavior change

IMPORTANCE

The study highlights the potential and limitations of the Large Language Models (LLMs) in recognizing different states of motivation to provide appropriate information for behavior change. Following the Transtheoretical Model (TTM), we identified the major gap of LLMs in responding to certain states of motivation through validated scenario studies, suggesting future directions of LLMs research for health promotion.

OBJECTIVES

The LLMs-based generative conversational agents (GAs) have shown success in identifying user intents semantically. Little is known about its capabilities to identify motivation states and provide appropriate information to facilitate behavior change progression.

MATERIALS AND METHODS

We evaluated 3 GAs, ChatGPT, Google Bard, and Llama 2 in identifying motivation states following the TTM stages of change. GAs were evaluated using 25 validated scenarios with 5 health topics across 5 TTM stages. The relevance and completeness of the responses to cover the TTM processes to proceed to the next stage of change were assessed.

RESULTS

3 GAs identified the motivation states in the preparation stage providing sufficient information to proceed to the action stage. The responses to the motivation states in the action and maintenance stages were good enough covering partial processes for individuals to initiate and maintain their changes in behavior. However, the GAs were not able to identify users' motivation states in the precontemplation and contemplation stages providing irrelevant information, covering about 20%-30% of the processes.

DISCUSSION

GAs are able to identify users' motivation states and provide relevant information when individuals have established goals and commitments to take and maintain an action. However, individuals who are hesitant or ambivalent about behavior change are unlikely to receive sufficient and relevant guidance to proceed to the next stage of change.

CONCLUSION

The current GAs effectively identify motivation states of individuals with established goals but may lack support for those ambivalent towards behavior change.

CuentosIE: can a chatbot about "tales with a message" help to teach emotional intelligence?

In this article, we present CuentosIE (TalesEI: chatbot of tales with a message to develop Emotional Intelligence), an educational chatbot on emotions that also provides teachers and psychologists with a tool to monitor their students/patients through indicators and data compiled by CuentosIE. The use of "tales with a message" is justified by their simplicity and easy understanding, thanks to their moral or associated metaphors. The main contributions of CuentosIE are the selection, collection, and classification of a set of highly specialized tales, as well as the provision of tools (searching, reading comprehension, chatting, recommending, and classifying) that are useful for both educating users about emotions and monitoring their emotional development. The preliminary evaluation of the tool has obtained encouraging results, which provides an affirmative answer to the question posed in the title of the article.

Development and Evaluation of a Smartphone-Based Chatbot Coach to Facilitate a Balanced Lifestyle in Individuals With Headaches (BalanceUP App): Randomized Controlled Trial

BACKGROUND

Primary headaches, including migraine and tension-type headaches, are widespread and have a social, physical, mental, and economic impact. Among the key components of treatment are behavior interventions such as lifestyle modification. Scalable conversational agents (CAs) have the potential to deliver behavior interventions at a low threshold. To our knowledge, there is no evidence of behavioral interventions delivered by CAs for the treatment of headaches.

OBJECTIVE

This study has 2 aims. The first aim was to develop and test a smartphone-based coaching intervention (BalanceUP) for people experiencing frequent headaches, delivered by a CA and designed to improve mental well-being using various behavior change techniques. The second aim was to evaluate the effectiveness of BalanceUP by comparing the intervention and waitlist control groups and assess the engagement and acceptance of participants using BalanceUP.

METHODS

In an unblinded randomized controlled trial, adults with frequent headaches were recruited on the web and in collaboration with experts and allocated to either a CA intervention (BalanceUP) or a control condition. The effects of the treatment on changes in the primary outcome of the study, that is, mental well-being (as measured by the Patient Health Questionnaire Anxiety and Depression Scale), and secondary outcomes (eg, psychosomatic symptoms, stress, headache-related self-efficacy, intention to change behavior, presenteeism and absenteeism, and pain coping) were analyzed using linear mixed models and Cohen d. Primary and secondary outcomes were self-assessed before and after the intervention, and acceptance was assessed after the intervention. Engagement was measured during the intervention using self-reports and usage data.

RESULTS

A total of 198 participants (mean age 38.7, SD 12.14 y; n=172, 86.9% women) participated in the study (intervention group: n=110; waitlist control group: n=88). After the intervention, the intention-to-treat analysis revealed evidence for improved well-being (treatment: β estimate=-3.28, 95% CI -5.07 to -1.48) with moderate between-group effects (Cohen d=-0.66, 95% CI -0.99 to -0.33) in favor of the intervention group. We also found evidence of reduced somatic symptoms, perceived stress, and absenteeism and presenteeism, as well as improved headache management self-efficacy, application of behavior change techniques, and pain coping skills, with effects ranging from medium to large (Cohen d=0.43-1.05). Overall, 64.8% (118/182) of the participants used coaching as intended by engaging throughout the coaching and completing the outro.

CONCLUSIONS

BalanceUP was well accepted, and the results suggest that coaching delivered by a CA can be effective in reducing the burden of people who experience headaches by improving their well-being.

TRIAL REGISTRATION

German Clinical Trials Register DRKS00017422; https://trialsearch.who.int/Trial2.aspx?TrialID=DRKS00017422.

Digitally Assisted Mindfulness in Training Self-Regulation Skills for Sustainable Mental Health: A Systematic Review

The onset of the COVID-19 pandemic has led to an increased demand for mental health interventions, with a special focus on digitally assisted ones. Self-regulation describes a set of meta-skills that enable one to take control over his/her mental health and it is recognized as a vital indicator of well-being. Mindfulness training is a promising training strategy for promoting self-regulation, behavioral change, and mental well-being. A growing body of research outlines that smart technologies are ready to revolutionize the way mental health training programs take place. Artificial intelligence (AI); extended reality (XR) including virtual reality (VR), augmented reality (AR), and mixed reality (MR); as well as the advancements in brain computer interfaces (BCIs) are ready to transform these mental health training programs. Mindfulness-based interventions assisted by smart technologies for mental, emotional, and behavioral regulation seem to be a crucial yet under-investigated issue. The current systematic review paper aims to explore whether and how smart technologies can assist mindfulness training for the development of self-regulation skills among people at risk of mental health issues as well as populations with various clinical characteristics. The PRISMA 2020 methodology was utilized to respond to the objectives and research questions using a total of sixty-six experimental studies that met the inclusion criteria. The results showed that digitally assisted mindfulness interventions supported by smart technologies, including AI-based applications, chatbots, virtual coaches, immersive technologies, and brain-sensing headbands, can effectively assist trainees in developing a wide range of cognitive, emotional, and behavioral self-regulation skills, leading to a greater satisfaction of their psychological needs, and thus mental wellness. These results may provide positive feedback for developing smarter and more inclusive training environments, with a special focus on people with special training needs or disabilities.

The Use of Voice Assistant for Psychological Assessment Elicits Empathy and Engagement While Maintaining Good Psychometric Properties

This study aimed to use the Alexa vocal assistant as an administerer of psychometric tests, assessing the efficiency and validity of this measurement. A total of 300 participants were administered the Interpersonal Reactivity Index (IRI). After a week, the administration was repeated, but the participants were randomly divided into groups of 100 participants each. In the first, the test was administered by means of a paper version; in the second, the questionnaire was read to the participants in person, and the operator contemporaneously recorded the answers declared by the participants; in the third group, the questionnaire was directly administered by the Alexa voice device, after specific reprogramming. The third group was also administered, as a post-session survey, the Engagement and Perceptions of the Bot Scale (EPVS), a short version of the Communication Styles Inventory (CSI), the Marlowe-Crowne Social Desirability Scale (MCSDS), and an additional six items to measure degrees of concentration, ease, and perceived pressure at the beginning and at the end of the administration. The results confirmed that the IRI did keep measurement invariance within the three conditions. The administration through vocal assistant showed an empathic activation effect significantly superior to the conditions of pencil-paper and operator-in-presence. The results indicated an engagement and positive evaluation of the interactive experience, with reported perceptions of closeness, warmth, competence, and human-likeness associated with higher values of empathetic activation and lower values of personal discomfort.

Systematic review and meta-analysis of the effectiveness of chatbots on lifestyle behaviours

Chatbots (also known as conversational agents and virtual assistants) offer the potential to deliver healthcare in an efficient, appealing and personalised manner. The purpose of this systematic review and meta-analysis was to evaluate the efficacy of chatbot interventions designed to improve physical activity, diet and sleep. Electronic databases were searched for randomised and non-randomised controlled trials, and pre-post trials that evaluated chatbot interventions targeting physical activity, diet and/or sleep, published before 1 September 2022. Outcomes were total physical activity, steps, moderate-to-vigorous physical activity (MVPA), fruit and vegetable consumption, sleep quality and sleep duration. Standardised mean differences (SMD) were calculated to compare intervention effects. Subgroup analyses were conducted to assess chatbot type, intervention type, duration, output and use of artificial intelligence. Risk of bias was assessed using the Effective Public Health Practice Project Quality Assessment tool. Nineteen trials were included. Sample sizes ranged between 25-958, and mean participant age ranged between 9-71 years. Most interventions (n = 15, 79%) targeted physical activity, and most trials had a low-quality rating (n = 14, 74%). Meta-analysis results showed significant effects (all p < 0.05) of chatbots for increasing total physical activity (SMD = 0.28 [95% CI = 0.16, 0.40]), daily steps (SMD = 0.28 [95% CI = 0.17, 0.39]), MVPA (SMD = 0.53 [95% CI = 0.24, 0.83]), fruit and vegetable consumption (SMD = 0.59 [95% CI = 0.25, 0.93]), sleep duration (SMD = 0.44 [95% CI = 0.32, 0.55]) and sleep quality (SMD = 0.50 [95% CI = 0.09, 0.90]). Subgroup analyses showed that text-based, and artificial intelligence chatbots were more efficacious than speech/voice chatbots for fruit and vegetable consumption, and multicomponent interventions were more efficacious than chatbot-only interventions for sleep duration and sleep quality (all p < 0.05). Findings from this systematic review and meta-analysis indicate that chatbot interventions are efficacious for increasing physical activity, fruit and vegetable consumption, sleep duration and sleep quality. Chatbot interventions were efficacious across a range of populations and age groups, with both short- and longer-term interventions, and chatbot only and multicomponent interventions being efficacious.

A Survey of Conversational Agents and Their Applications for Self-Management of Chronic Conditions

Conversational agents have gained their ground in our daily life and various domains including healthcare. Chronic condition self-management is one of the promising healthcare areas in which conversational agents demonstrate significant potential to contribute to alleviating healthcare burdens from chronic conditions. This survey paper introduces and outlines types of conversational agents, their generic architecture and workflow, the implemented technologies, and their application to chronic condition self-management.

Feasibility and effectiveness of artificial intelligence-driven conversational agents in healthcare interventions: A systematic review of randomized controlled trials

BACKGROUND

A virtual conversational agent is a program that typically utilizes artificial intelligence technology to mimic human interactions. Many robust and high-quality clinical trials have been conducted to test the effectiveness of conversational agent-based interventions. However, there is a lack of systematic reviews of randomized controlled trials that evaluate the effects of artificial intelligence-driven conversational agents in healthcare interventions.

OBJECTIVE

To examine the feasibility and effectiveness of conversational agent-based interventions evaluated by randomized controlled trials in the healthcare context, as well as to evaluate the information quality of artificial intelligence-driven conversational agents.

DESIGN

A systematic review.

DATA SOURCE

A systematic search of relevant literature published in English in Scopus, Pubmed, Embase, PsycINFO, Cochrane Library, Information Science & Technology, and Web of Science, was performed. Only randomized controlled trials from the inception of the databases until May 2022 were included.

REVIEW METHODS

Two reviewers independently selected the articles according to the inclusion and exclusion criteria. Study findings were narratively synthesized and summarized. The studies' risk of bias was evaluated using the Risk of Bias 2.0 tool. The Silberg Scale was used to evaluate the quality of the conversational agent system utilized in each reviewed study.

RESULTS

Twenty-one studies were included in the data synthesis. The recruitment rates ranged from 34% to 100% (mean = 84%), and completion rates ranged from 40% to 100% (mean = 83%). A moderate to high level of intervention acceptability was reported. The intervention approaches included health counseling and education (n = 8), cognitive-behavioral interventions (n = 7), storytelling (n = 1), acceptance and commitment therapy (n = 1), and coping skills training (n = 1). Findings indicated inconsistent effects on improving participants' physical activity and function, healthy lifestyle modifications, knowledge of the diseases, and mental health and psychosocial outcomes. The overall risk of bias varied from low risk (n = 6) to high risk (n = 7) across the studies. The mean Silberg score of included studies was 5.4/9, with a standard deviation of 1.6.

CONCLUSION

Our review findings indicated that conversational agent-based interventions were feasible, acceptable, and had positive effects on physical functioning, healthy lifestyle, mental health and psychosocial outcomes. Conversational agents can provide low-threshold access to healthcare services. They can serve as remote medical assistants to support patients' recovery or health promotion needs before or after medical treatments. The conversational agent-based interventions can also play adjunctive roles and be integrated into current healthcare systems, which could improve the comprehensiveness of services and make more efficient use of physicians' and nurses' time.

Deploying a robotic positive psychology coach to improve college students' psychological well-being

Despite the increase in awareness and support for mental health, college students' mental health is reported to decline every year in many countries. Several interactive technologies for mental health have been proposed and are aiming to make therapeutic service more accessible, but most of them only provide one-way passive contents for their users, such as psycho-education, health monitoring, and clinical assessment. We present a robotic coach that not only delivers interactive positive psychology interventions but also provides other useful skills to build rapport with college students. Results from our on-campus housing deployment feasibility study showed that the robotic intervention showed significant association with increases in students' psychological well-being, mood, and motivation to change. We further found that students' personality traits were associated with the intervention outcomes as well as their working alliance with the robot and their satisfaction with the interventions. Also, students' working alliance with the robot was shown to be associated with their pre-to-post change in motivation for better well-being. Analyses on students' behavioral cues showed that several verbal and nonverbal behaviors were associated with the change in self-reported intervention outcomes. The qualitative analyses on the post-study interview suggest that the robotic coach's companionship made a positive impression on students, but also revealed areas for improvement in the design of the robotic coach. Results from our feasibility study give insight into how learning users' traits and recognizing behavioral cues can help an AI agent provide personalized intervention experiences for better mental health outcomes.

Do you like me? Behavioral and physical features for socially and emotionally engaging interactive systems

With the aim to give an overview of the most recent discoveries in the field of socially engaging interactive systems, the present paper discusses features affecting users' acceptance of virtual agents, robots, and chatbots. In addition, questionnaires exploited in several investigations to assess the acceptance of virtual agents, robots, and chatbots (voice only) are discussed and reported in the Supplementary material to make them available to the scientific community. These questionnaires were developed by the authors as a scientific contribution to the H2020 project EMPATHIC (http://www.empathic-project.eu/), Menhir (https://menhir-project.eu/), and the Italian-funded projects SIROBOTICS (https://www.exprivia.it/it-tile-6009-si-robotics/) and ANDROIDS (https://www.psicologia.unicampania.it/android-project) to guide the design and implementation of the promised assistive interactive dialog systems. They aimed to quantitatively evaluate Virtual Agents Acceptance (VAAQ), Robot Acceptance (RAQ), and Synthetic Virtual Agent Voice Acceptance (VAVAQ).

An Online Acceptance and Mindfulness Intervention for Chronic Pain in Veterans: Development and Protocol for a Pilot Feasibility Randomized Controlled Trial

BACKGROUND

In the veteran community, chronic pain is particularly prevalent and often debilitating. Until recently, veterans with chronic pain were offered primarily pharmacological intervention options, which rarely suffice and can also have negative health consequences. To better address chronic pain in veterans, the Veterans Health Administration has invested in novel, nonpharmacological behavior interventions that target both pain management and chronic pain-related functional issues. One approach, acceptance and commitment therapy (ACT) for chronic pain, is supported by decades of efficacy evidence for improving pain outcomes; however, ACT can be difficult to obtain owing to issues such as a lack of trained therapists or veterans having difficulty committing to the time and resources needed for the full clinician-led ACT protocol. Given the strong ACT evidence base combined with access limitations, we set out to develop and evaluate Veteran ACT for Chronic Pain (VACT-CP), an online program guided by an embodied conversational agent to improve pain management and functioning.

OBJECTIVE

The aims of this study are to develop, iteratively refine, and then conduct a pilot feasibility randomized controlled trial (RCT) of a VACT-CP group (n=20) versus a waitlist and treatment-as-usual control group (n=20).

METHODS

This research project includes 3 phases. In phase 1, our research team consulted with pain and virtual care experts, developed the preliminary VACT-CP online program, and conducted interviews with providers to obtain their feedback on the intervention. In phase 2, we incorporated feedback from phase 1 into the VACT-CP program and completed initial usability testing with veterans with chronic pain. In phase 3, we are conducting a small pilot feasibility RCT, with the primary outcome being assessment of usability of the VACT-CP system.

RESULTS

This study is currently in phase 3; recruitment for the RCT began in April 2022 and is expected to continue through April 2023. Data collection is expected to be completed by October 2023, with full data analysis completed by late 2023.

CONCLUSIONS

The findings from this research project will provide information on the usability of the VACT-CP intervention, as well as secondary outcomes related to treatment satisfaction, pain outcomes (pain-related daily functioning and pain severity), ACT processes (pain acceptance, behavioral avoidance, and valued living), and mental and physical functioning.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03655132; https://clinicaltrials.gov/ct2/show/NCT03655132.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/45887.

Comparing the Feasibility and Acceptability of a Virtual Human, Teletherapy, and an e-Manual in Delivering a Stress Management Intervention to Distressed Adult Women: Pilot Study

BACKGROUND

Virtual humans (VHs), teletherapy, and self-guided e-manuals may increase the accessibility of psychological interventions. However, there is limited research on how these technologies compare in terms of their feasibility and acceptability in delivering stress management interventions.

OBJECTIVE

We conducted a preliminary comparison of the feasibility and acceptability of a VH, teletherapy, and an e-manual at delivering 1 module of cognitive behavioral stress management (CBSM) to evaluate the feasibility of the trial methodology in preparation for a future randomized controlled trial (RCT).

METHODS

A pilot RCT was conducted with a parallel, mixed design. A community sample of distressed adult women were randomly allocated to receive 1 session of CBSM involving training in cognitive and behavioral techniques by a VH, teletherapy, or an e-manual plus homework over 2 weeks. Data were collected on the feasibility of the intervention technologies (technical support and homework access), trial methods (recruitment methods, questionnaire completion, and methodological difficulty observations), intervention acceptability (intervention completion, self-report ratings, therapist rapport, and trust), and acceptability of the trial methods (self-report ratings and observations). Qualitative data in the form of written responses to open-ended questions were collected to enrich and clarify the findings on intervention acceptability.

RESULTS

Overall, 38 participants' data were analyzed. A VH (n=12), teletherapy (n=12), and an e-manual (n=14) were found to be feasible and acceptable for delivering 1 session of CBSM to distressed adult women based on the overall quantitative and qualitative findings. Technical difficulties were minimal and did not affect intervention completion, and no significant differences were found between the conditions (P=.31). The methodology was feasible, although improvements were identified for a future trial. All conditions achieved good satisfaction and perceived engagement ratings, and no significant group differences were found (P>.40). Participants had similar willingness to recommend each technology (P=.64). There was a nonsignificant trend toward participants feeling more open to using the VH and e-manual from home than teletherapy (P=.10). Rapport (P<.001) and trust (P=.048) were greater with the human teletherapist than with the VH. The qualitative findings enriched the quantitative results by revealing the unique strengths and limitations of each technology that may have influenced acceptability.

CONCLUSIONS

A VH, teletherapy, and a self-guided e-manual were found to be feasible and acceptable methods of delivering 1 session of a stress management intervention to a community sample of adult women. The technologies were found to have unique strengths and limitations that may affect which works best for whom and in what circumstances. Future research should test additional CBSM modules for delivery by these technologies and conduct a larger RCT to compare their feasibility, acceptability, and effectiveness when delivering a longer home-based stress management program.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ACTRN12620000859987; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=380114&isReview=true.

Early detection of depression using a conversational AI bot: A non-clinical trial

BACKGROUND

Artificial intelligence (AI) has gained momentum in behavioural health interventions in recent years. However, a limited number of studies use or apply such methodologies in the early detection of depression. A large population needing psychological-intervention is left unidentified due to barriers such as cost, location, stigma and a global shortage of health workers. Therefore, it is essential to develop a mass screening integrative approach that can identify people with depression at its early stage to avoid a potential crisis.

OBJECTIVES

This study aims to understand the feasibility and efficacy of using AI-enabled chatbots in the early detection of depression.

METHODS

We use Dialogflow as a conversation interface to build a Depression Analysisn (DEPRA) chatbot. A structured and authoritative early detection depression interview guide, which contains 27 questions combining the structured interview guide for the Hamilton Depression Scale (SIGH-D) and the inventory of depressive symptomatology (IDS-C), underpins the design of the conversation flow. To attain better accuracy and a wide variety of responses, we train Dialogflow with the utterances collected from a focus group of 10 people. The occupation of the focus group members included academics and HDR candidates who are conscious, vigilant and have a clear understanding of the questions. In addition, DEPRA is integrated with a social media platform to provide practical access to all the participants. For the non-clinical trial, we recruited 50 participants aged between 18 and 80 from across Australia. To evaluate the practicability and performance of DEPRA, we also asked participants to submit a user satisfaction survey at the end of the conversation.

RESULTS

A sample of 50 participants, with an average age of 34.7 years, completed this non-clinical trial. More than half of the participants (54%) are male and the major ethnicities are Asian (63%), Middle Eastern (25%), and others 12%. The first group comprises professional academic staff and HDR candidates, the second and third groups comprise relatives, friends, and volunteers who were recruited via social media promotions. DEPRA uses two scientific scoring systems, QIDS-SR and IDS-SR to verify the results of early depression detection. As the results indicate, both scoring systems return a similar outcome with slight variations for different depression levels. According to IDS-SR, 30% of participants were healthy, 14% mild, 22% moderate, 14% severe, and 20% very severe. QIDS-SR suggests 32% were healthy, 18% mild, 10% moderate, 18% severe, and 22% very severe. Furthermore, the overall satisfaction rate of using DEPRA was 79% indicating that the participants had a high rate of user satisfaction and engagement.

CONCLUSION

DEPRA shows promises as a feasible option for developing a mass screening integrated approach for early detection of depression. Although the chatbot is not intended to replace the functionality of mental health professionals, it does show promise as a means of assisting with automation and concealed communication with verified scoring systems.

A chatbot-based intervention with ELME to improve stress and health-related parameters in a stressed sample: Study protocol of a randomised controlled trial

Background

Stress levels in the general population had already been increasing in recent years, and have subsequently been exacerbated by the global pandemic. One approach for innovative online-based interventions are "chatbots" - computer programs that can simulate a text-based interaction with human users via a conversational interface. Research on the efficacy of chatbot-based interventions in the context of mental health is sparse. The present study is designed to investigate the effects of a three-week chatbot-based intervention with the chatbot ELME, aiming to reduce stress and to improve various health-related parameters in a stressed sample.

Methods

In this multicenter, two-armed randomised controlled trial with a parallel design, a three-week chatbot-based intervention group including two daily interactive intervention sessions via smartphone (á 10-20 min.) is compared to a treatment-as-usual control group. A total of 130 adult participants with a medium to high stress levels will be recruited in Germany. Assessments will take place pre-intervention, post-intervention (after three weeks), and follow-up (after six weeks). The primary outcome is perceived stress. Secondary outcomes include self-reported interoceptive accuracy, mindfulness, anxiety, depression, personality, emotion regulation, psychological well-being, stress mindset, intervention credibility and expectancies, affinity for technology, and attitudes towards artificial intelligence. During the intervention, participants undergo ecological momentary assessments. Furthermore, satisfaction with the intervention, the usability of the chatbot, potential negative effects of the intervention, adherence, potential dropout reasons, and open feedback questions regarding the chatbot are assessed post-intervention.

Discussion

To the best of our knowledge, this is the first chatbot-based intervention addressing interoception, as well as in the context with the target variables stress and mindfulness. The design of the present study and the usability of the chatbot were successfully tested in a previous feasibility study. To counteract a low adherence of the chatbot-based intervention, a high guidance by the chatbot, short sessions, individual and flexible time points of the intervention units and the ecological momentary assessments, reminder messages, and the opportunity to postpone single units were implemented.

Trial registration

The trial is registered at the WHO International Clinical Trials Registry Platform via the German Clinical Trials Register (DRKS00027560; date of registration: 06 January 2022). This is protocol version No. 1. In case of important protocol modifications, trial registration will be updated.

Effects of Cognitive Behavioral Stress Management Delivered by a Virtual Human, Teletherapy, and an E-Manual on Psychological and Physiological Outcomes in Adult Women: An Experimental Test

Technology may expand the reach of stress management to broader populations. However, issues with engagement can reduce intervention effectiveness. Technologies with highly social interfaces, such as virtual humans (VH), may offer advantages in this space. However, it is unclear how VH compare to telehealth and e-manuals at delivering psychological interventions. This experiment compared the effects of single laboratory session of Cognitive Behavioral Stress Management (CBSM) delivered by a VH (VH-CBSM), human telehealth (T-CBSM), and an e-manual (E-CBSM) on psychological and physiological outcomes in a community sample of stressed adult women. A pilot randomized controlled trial (RCT) with a parallel, mixed design was conducted. Adult women (M age =43.21, SD = 10.70) who self-identified as stressed were randomly allocated to VH-CBSM, T-CBSM, or E-CBSM involving one 90 min session and homework. Perceived stress, stress management skills, negative affect, optimism, relaxation, and physiological stress were measured. Mixed factorial ANOVAs and pairwise comparisons with Bonferroni correction investigated main and interaction effects of time and condition. Participants’ data (N = 38) were analysed (12 = VH-CBSM; 12 = T-CBSM; 14 = E-CBSM). Each condition significantly improved stress, negative affect, optimism, relaxation, and physiological stress over time with large effect sizes. No significant differences were found between conditions on outcomes. Overall, all three technologies showed promise for remotely delivering CBSM in a controlled setting. The findings suggest feasibility of the VH-CBSM delivery approach and support conducting a fully powered RCT to examine its effectiveness when delivering a full 10-week CBSM intervention.

Social robots as eating companions

Previous research shows that eating together (i.e., commensality) impacts food choice, time spent eating, and enjoyment. Conversely, eating alone is considered a possible cause of unhappiness. In this paper, we conceptually explore how interactive technology might allow for the creation of artificial commensal companions: embodied agents providing company to humans during meals (e.g., a person living in isolation due to health reasons). We operationalize this with the design of our commensal companion: a system based on the MyKeepon robot, paired with a Kinect sensor, able to track the human commensal's activity (i.e., food picking and intake) and able to perform predefined nonverbal behavior in response. In this preliminary study with 10 participants, we investigate whether this autonomous social robot-based system can positively establish an interaction that humans perceive and whether it can influence their food choices. In this study, the participants are asked to taste some chocolates with and without the presence of an artificial commensal companion. The participants are made to believe that the study targets the food experience, whilst the presence of a robot is accidental. Next, we analyze their food choices and feedback regarding the role and social presence of the artificial commensal during the task performance. We conclude the paper by discussing the lessons we learned about the first interactions we observed between a human and a social robot in a commensality setting and by proposing future steps and more complex applications for this novel kind of technology.

Chatbots to Support Young Adults’ Mental Health: an Exploratory Study of Acceptability

Despite the prevalence of mental health conditions, stigma, lack of awareness and limited resources impede access to care, creating a need to improve mental health support. The recent surge in scientific and commercial interest in conversational agents and their potential to improve diagnosis and treatment seems a potentially fruitful area in this respect, particularly for young adults who widely use such systems in other contexts. Yet, there is little research that considers the acceptability of conversational agents in mental health. This study, therefore, presents three research activities that explore whether conversational agents and, in particular, chatbots can be an acceptable solution in mental healthcare for young adults. First, a survey of young adults (in a university setting) provides an understanding of the landscape of mental health in this age group and of their views around mental health technology, including chatbots. Second, a literature review synthesises current evidence relating to the acceptability of mental health conversational agents and points to future research priorities. Third, interviews with counsellors who work with young adults, supported by a chatbot prototype and user-centred design techniques, reveal the perceived benefits and potential roles of mental health chatbots from the perspective of mental health professionals, while suggesting preconditions for the acceptability of the technology. Taken together, these research activities: provide evidence that chatbots are an acceptable solution to offering mental health support for young adults; identify specific challenges relating to both the technology and environment; and argue for the application of user-centred approaches during development of mental health chatbots and more systematic and rigorous evaluations of the resulting solutions.

Conversational agents in healthcare: a scoping review of their behavior change techniques and underpinning theory (Preprint)

Background

Conversational agents (CAs) are increasingly used in health care to deliver behavior change interventions. Their evaluation often includes categorizing the behavior change techniques (BCTs) using a classification system of which the BCT Taxonomy v1 (BCTTv1) is one of the most common. Previous studies have presented descriptive summaries of behavior change interventions delivered by CAs, but no in-depth study reporting the use of BCTs in these interventions has been published to date.

Objective

This review aims to describe behavior change interventions delivered by CAs and to identify the BCTs and theories guiding their design.

Methods

We searched PubMed, Embase, Cochrane’s Central Register of Controlled Trials, and the first 10 pages of Google and Google Scholar in April 2021. We included primary, experimental studies evaluating a behavior change intervention delivered by a CA. BCTs coding followed the BCTTv1. Two independent reviewers selected the studies and extracted the data. Descriptive analysis and frequent itemset mining to identify BCT clusters were performed.

Results

We included 47 studies reporting on mental health (n=19, 40%), chronic disorders (n=14, 30%), and lifestyle change (n=14, 30%) interventions. There were 20/47 embodied CAs (43%) and 27/47 CAs (57%) represented a female character. Most CAs were rule based (34/47, 72%). Experimental interventions included 63 BCTs, (mean 9 BCTs; range 2-21 BCTs), while comparisons included 32 BCTs (mean 2 BCTs; range 2-17 BCTs). Most interventions included BCTs 4.1 “Instruction on how to perform a behavior” (34/47, 72%), 3.3 “Social support” (emotional; 27/47, 57%), and 1.2 “Problem solving” (24/47, 51%). A total of 12/47 studies (26%) were informed by a behavior change theory, mainly the Transtheoretical Model and the Social Cognitive Theory. Studies using the same behavior change theory included different BCTs.

Conclusions

There is a need for the more explicit use of behavior change theories and improved reporting of BCTs in CA interventions to enhance the analysis of intervention effectiveness and improve the reproducibility of research.

The Use of Artificial Intelligence-Based Conversational Agents (Chatbots) for Weight Loss: Scoping Review and Practical Recommendations

Background

Overweight and obesity have now reached a state of a pandemic despite the clinical and commercial programs available. Artificial intelligence (AI) chatbots have a strong potential in optimizing such programs for weight loss.

Objective

This study aimed to review AI chatbot use cases for weight loss and to identify the essential components for prolonging user engagement.

Methods

A scoping review was conducted using the 5-stage framework by Arksey and O’Malley. Articles were searched across nine electronic databases (ACM Digital Library, CINAHL, Cochrane Central, Embase, IEEE Xplore, PsycINFO, PubMed, Scopus, and Web of Science) until July 9, 2021. Gray literature, reference lists, and Google Scholar were also searched.

Results

A total of 23 studies with 2231 participants were included and evaluated in this review. Most studies (8/23, 35%) focused on using AI chatbots to promote both a healthy diet and exercise, 13% (3/23) of the studies used AI chatbots solely for lifestyle data collection and obesity risk assessment whereas only 4% (1/23) of the studies focused on promoting a combination of a healthy diet, exercise, and stress management. In total, 48% (11/23) of the studies used only text-based AI chatbots, 52% (12/23) operationalized AI chatbots through smartphones, and 39% (9/23) integrated data collected through fitness wearables or Internet of Things appliances. The core functions of AI chatbots were to provide personalized recommendations (20/23, 87%), motivational messages (18/23, 78%), gamification (6/23, 26%), and emotional support (6/23, 26%). Study participants who experienced speech- and augmented reality–based chatbot interactions in addition to text-based chatbot interactions reported higher user engagement because of the convenience of hands-free interactions. Enabling conversations through multiple platforms (eg, SMS text messaging, Slack, Telegram, Signal, WhatsApp, or Facebook Messenger) and devices (eg, laptops, Google Home, and Amazon Alexa) was reported to increase user engagement. The human semblance of chatbots through verbal and nonverbal cues improved user engagement through interactivity and empathy. Other techniques used in text-based chatbots included personally and culturally appropriate colloquial tones and content; emojis that emulate human emotional expressions; positively framed words; citations of credible information sources; personification; validation; and the provision of real-time, fast, and reliable recommendations. Prevailing issues included privacy; accountability; user burden; and interoperability with other databases, third-party applications, social media platforms, devices, and appliances.

Conclusions

AI chatbots should be designed to be human-like, personalized, contextualized, immersive, and enjoyable to enhance user experience, engagement, behavior change, and weight loss. These require the integration of health metrics (eg, based on self-reports and wearable trackers), personality and preferences (eg, based on goal achievements), circumstantial behaviors (eg, trigger-based overconsumption), and emotional states (eg, chatbot conversations and wearable stress detectors) to deliver personalized and effective recommendations for weight loss.

Evaluating the Therapeutic Alliance With a Free-Text CBT Conversational Agent (Wysa): A Mixed-Methods Study

The present study aims to examine whether users perceive a therapeutic alliance with an AI conversational agent (Wysa) and observe changes in the t‘herapeutic alliance over a brief time period. A sample of users who screened positively on the PHQ-4 for anxiety or depression symptoms (N = 1,205) of the digital mental health application (app) Wysa were administered the WAI-SR within 5 days of installing the app and gave a second assessment on the same measure after 3 days (N = 226). The anonymised transcripts of user's conversations with Wysa were also examined through content analysis for unprompted elements of bonding between the user and Wysa (N = 950). Within 5 days of initial app use, the mean WAI-SR score was 3.64 (SD 0.81) and the mean bond subscale score was 3.98 (SD 0.94). Three days later, the mean WAI-SR score increased to 3.75 (SD 0.80) and the mean bond subscale score increased to 4.05 (SD 0.91). There was no significant difference in the alliance scores between Assessment 1 and Assessment 2.These mean bond subscale scores were found to be comparable to the scores obtained in recent literature on traditional, outpatient-individual CBT, internet CBT and group CBT. Content analysis of the transcripts of user conversations with the CA (Wysa) also revealed elements of bonding such as gratitude, self-disclosed impact, and personification. The user's therapeutic alliance scores improved over time and were comparable to ratings from previous studies on alliance in human-delivered face-to-face psychotherapy with clinical populations. This study provides critical support for the utilization of digital mental health services, based on the evidence of the establishment of an alliance.

A Study on the Impacts of Slot Types and Training Data on Joint Natural Language Understanding in a Spanish Medication Management Assistant Scenario

This study evaluates the impacts of slot tagging and training data length on joint natural language understanding (NLU) models for medication management scenarios using chatbots in Spanish. In this study, we define the intents (purposes of the sentences) for medication management scenarios and two types of slot tags. For training the model, we generated four datasets, combining long/short sentences with long/short slots, while for testing, we collect the data from real interactions of users with a chatbot. For the comparative analysis, we chose six joint NLU models (SlotRefine, stack-propagation framework, SF-ID network, capsule-NLU, slot-gated modeling, and a joint SLU-LM model) from the literature. The results show that the best performance (with a sentence-level semantic accuracy of 68.6%, an F1-score of 76.4% for slot filling, and an accuracy of 79.3% for intent detection) is achieved using short sentences and short slots. Our results suggest that joint NLU models trained with short slots yield better results than those trained with long slots for the slot filling task. The results also indicate that short slots could be a better choice for the dialog system because of their simplicity. Importantly, the work demonstrates that the performance of the joint NLU models can be improved by selecting the correct slot configuration according to the usage scenario.

Automatic topic selection for long-term interaction with embodied conversational agents in health coaching: A micro-randomized trial

INTRODUCTION

Embodied Conversational Agents (ECAs) can be included in health coaching applications as virtual coaches. The engagement with these virtual coaches could be improved by presenting users with tailored coaching dialogues. In this article, we investigate if the suggestion of an automatically tailored topic by an ECA leads to higher engagement by the user and thus longer sessions of interaction.

METHODS

A Micro-Randomized Trial (MRT) was conducted in which two types of interaction with an ECA were compared: (a) the coach suggests a relevant topic to discuss, and (b) the coach asks the user to select a topic from a set of options. Every time the user would interact with the ECA, one of those conditions would be randomly selected. Participants interacted in their daily life with the ECA that was part of a multi-agent health coaching application for 4-8 weeks.

RESULTS

In two rounds, 82 participants interacted with the micro-randomized coach a total of 1011 times. Interactions in which the coach took the initiative were found to be of equal length as interactions in which the user was allowed to choose the topic, and the acceptance of topic suggestions was high (71.1% overall, 75.8% for coaching topics).

CONCLUSION

Tailoring coaching conversations with ECAs by letting the coach automatically suggest a topic that is tailored to the user is perceived as a natural variation in the flow of interaction. Future research could focus on improving the novel coaching engine component that supports the topic selection process for these suggestions or on investigating how the amount of initiative and coaching approach by the ECA could be tailored.

21-Day Stress Detox: Open Trial of a Universal Well-Being Chatbot for Young Adults

There has been a lot of interest in digital mental health interventions but adherence to online programmes has been less than optimal. Chatbots that mimic brief conversations may be a more engaging and acceptable mode of delivery. We developed a chatbot, called 21-Day Stress Detox, to deliver stress management techniques for young adults. The purpose of the study was to explore the feasibility, acceptability, and potential efficacy of this low-intensity digital mental health intervention in a non-clinical population of young adults. The content was derived from cognitive behavioural therapy (CBT) and included evidence-informed elements such as mindfulness and gratitude journaling. It was delivered over 21 daily sessions using the Facebook Messenger platform. Each session was intended to last about 5–7 min and included text, animated GIFs, relaxation tracks and reflective exercises. We conducted an open single-arm trial collecting app usage through passive data collection as well as self-rated satisfaction and qualitative (open-ended) feedback. Efficacy was assessed via outcome measures of well-being (World Health Organisation (Five) Well-being Index; WHO-5; and Personal Well-being Measure; ONS4); stress (Perceived Stress Scale–10 item version; PSS-10); and anxiety (Generalized Anxiety Disorder 7-item scale; GAD-7). One hundred and ten of the 124 participants who completed baseline commenced the chatbot and 64 returned the post-intervention assessment. Eighty-one percent were female and 51% were first year students. Forty-five percent were NZ European and 41% were Asian. Mean engagement was 11 days out 21 days (SD = 7.8). Most (81%) found the chatbot easy to use. Sixty-three percent rated their satisfaction as 7 out of 10 or higher. Qualitative feedback revealed that convenience and relatable content were the most valued features. There was a statistically significant improvement on the WHO-5 of 7.38 (SD = 15.07; p < 0.001) and a mean reduction on the PSS-10 of 1.77 (SD = 4.69; p = 0.004) equating to effect sizes of 0.49 and 0.38, respectively. Those who were clinically anxious at baseline (n = 25) experienced a greater reduction of GAD-7 symptoms than those (n = 39) who started the study without clinical anxiety (−1.56, SD = 3.31 vs. 0.67, SD = 3.30; p = 0.011). Using a chatbot to deliver universal psychological support appears to be feasible, acceptable, have good levels of engagement, and lead to significant improvements in well-being and stress. Future iterations of the chatbot should involve a more personalised content.

Use and effect of embodied conversational agents for improving eating behavior and decreasing loneliness among community-dwelling older adults: A randomized controlled trial (Preprint)

Background

Embodied conversational agents (ECAs) have been proposed as a promising interaction modality for the delivery of programs focused on promoting lifestyle changes. However, it is not understood what factors influence the health effects of ECAs or their use.

Objective

We aimed to (1) identify whether ECAs could persuade community-dwelling older adults to change their dietary behavior and whether ECA use could decrease loneliness, (2) test the pathways to these effects, and (3) understand factors influencing the use of ECAs.

Methods

A randomized controlled trial was conducted. The intervention group received access to the PACO service for 8 weeks. The waitlist group started PACO use after waiting for 4 weeks. Two primary outcomes (eating behavior and loneliness) were assessed via online questionnaires at intake, upon joining the waitlist, after 4 weeks, and after 8 weeks. The third primary outcome (use) was assessed via data logs. Secondary outcomes were measured at the same time points, via questionnaires or an optional interview.

Results

In total, 32 participants completed the intervention. We found a significant correlation between use in minutes on the one hand, and perceived usefulness (r=0.39, P=.03) and enjoyment on the other (r=0.38, P=.03). However, these did not predict use in the full regression model (F_2,29=1.98, P=.16, R²=0.12). Additionally, PACO use did not lead to improvement in eating behavior (χ²₂=0.34, P=.85) or a decrease in loneliness (χ²₂=0.02, P=.99).

Conclusions

Our study did not provide any concluding evidence about factors that are linked to the use or health effects of ECAs. Future service design could benefit from either creating a functional design catering to the predominant stage in the precaution adoption process model of the targeted population, or by personalizing the service based on an intake in which the end user’s stage is determined.

Trial Registration

ClinicalTrials.gov NCT04510883; https://clinicaltrials.gov/ct2/show/NCT04510883

International Registered Report Identifier (IRRID)

RR2-10.2196/22186

Promoting Physical Activity Through Conversational Agents: Mixed Methods Systematic Review

BACKGROUND

Regular physical activity (PA) is crucial for well-being; however, healthy habits are difficult to create and maintain. Interventions delivered via conversational agents (eg, chatbots or virtual agents) are a novel and potentially accessible way to promote PA. Thus, it is important to understand the evolving landscape of research that uses conversational agents.

OBJECTIVE

This mixed methods systematic review aims to summarize the usability and effectiveness of conversational agents in promoting PA, describe common theories and intervention components used, and identify areas for further development.

METHODS

We conducted a mixed methods systematic review. We searched seven electronic databases (PsycINFO, PubMed, Embase, CINAHL, ACM Digital Library, Scopus, and Web of Science) for quantitative, qualitative, and mixed methods studies that conveyed primary research on automated conversational agents designed to increase PA. The studies were independently screened, and their methodological quality was assessed using the Mixed Methods Appraisal Tool by 2 reviewers. Data on intervention impact and effectiveness, treatment characteristics, and challenges were extracted and analyzed using parallel-results convergent synthesis and narrative summary.

RESULTS

In total, 255 studies were identified, 7.8% (20) of which met our inclusion criteria. The methodological quality of the studies was varied. Overall, conversational agents had moderate usability and feasibility. Those that were evaluated through randomized controlled trials were found to be effective in promoting PA. Common challenges facing interventions were repetitive program content, high attrition, technical issues, and safety and privacy concerns.

CONCLUSIONS

Conversational agents hold promise for PA interventions. However, there is a lack of rigorous research on long-term intervention effectiveness and patient safety. Future interventions should be based on evidence-informed theories and treatment approaches and should address users' desires for program variety, natural language processing, delivery via mobile devices, and safety and privacy concerns.

Multilingual Conversational Systems to Drive the Collection of Patient-Reported Outcomes and Integration into Clinical Workflows

Patient-reported outcomes (PROs) and their use in the clinical workflow can improve cancer survivors’ outcomes and quality of life. However, there are several challenges regarding efficient collection of the patient-reported outcomes and their integration into the clinical workflow. Patient adherence and interoperability are recognized as main barriers. This work implements a cancer-related study which interconnects artificial intelligence (spoken language algorithms, conversational intelligence) and natural sciences (embodied conversational agents) to create an omni-comprehensive system enabling symmetric computer-mediated interaction. Its goal is to collect patient information and integrate it into clinical routine as digital patient resources (the Fast Healthcare Interoperability Resources). To further increase convenience and simplicity of the data collection, a multimodal sensing network is delivered. In this paper, we introduce the main components of the system, including the mHealth application, the Open Health Connect platform, and algorithms to deliver speech enabled 3D embodied conversational agent to interact with the cancer survivors in five different languages. The system integrates cancer patients’ reported information as patient gathered health data into their digital clinical record. The value and impact of the integration will be further evaluated in the clinical study.

Efficacy of Zemedy, a Mobile Digital Therapeutic for the Self-management of Irritable Bowel Syndrome: Crossover Randomized Controlled Trial

Background

Patients with irritable bowel syndrome (IBS) experience abdominal pain, altered bowel habits, and defecation-related anxiety, which can result in reduced productivity and impaired health-related quality of life (HRQL). Cognitive behavioral therapy (CBT) has been shown to reduce symptoms of IBS and to improve HRQL, but access to qualified therapists is limited. Smartphone-based digital therapeutic interventions have potential to increase access to guided CBT at scale, but require careful study to assess their benefits and risks.

Objective

The aim of this study was to test the efficacy of a novel app, Zemedy, as a mobile digital therapeutic that delivers a comprehensive CBT program to individuals with IBS.

Methods

This was a crossover randomized controlled trial. Participants were recruited online and randomly allocated to either immediate treatment (n=62) or waitlist control (n=59) groups. The Zemedy app consists of 8 modules focusing on psychoeducation, relaxation training, exercise, the cognitive model of stress management, applying CBT to IBS symptoms, reducing avoidance through exposure therapy, behavioral experiments, and information about diet. Users interact with a chatbot that presents the information and encourages specific plans, homework, and exercises. The treatment was fully automated, with no therapist involvement or communication. At baseline and after 8 weeks, participants were asked to complete the battery of primary (Irritable Bowel Syndrome Quality of Life [IBS-QOL], Gastrointestinal Symptom Rating Scale [GSRS]) and secondary (Fear of Food Questionnaire [FFQ], Visceral Sensitivity Index [VSI], Gastrointestinal Cognition Questionnaire [GI-COG], Depression Anxiety Stress Scale [DASS], and Patient Health Questionnaire-9 [PHQ-9]) outcome measures. Waitlist controls were then offered the opportunity to crossover to treatment. All participants were assessed once more at 3 months posttreatment.

Results

Both intention-to-treat and completer analyses at posttreatment revealed significant improvement for the immediate treatment group compared to the waitlist control group on both primary and secondary outcome measures. Gains were generally maintained at 3 months posttreatment. Scores on the GSRS, IBS-QoL, GI-COG, VSI, and FFQ all improved significantly more in the treatment group (F_1,79=20.49, P<.001, Cohen d=1.01; F_1,79=20.12, P<.001, d=1.25; F_1,79=34.71, P<.001, d=1.47; F_1,79=18.7, P<.001, d=1.07; and F_1,79=12.13, P=.001, d=0.62, respectively). Depression improved significantly as measured by the PHQ-9 (F_1,79=10.5, P=.002, d=1.07), and the DASS Depression (F_1,79=6.03, P=.02, d=.83) and Stress (F_1,79=4.47, P=.04, d=0.65) subscales in the completer analysis but not in the intention-to-treat analysis. The impact of treatment on HRQL was mediated by reductions in catastrophizing and visceral sensitivity.

Conclusions

Despite its relatively benign physical profile, IBS can be an extraordinarily debilitating condition. Zemedy is an effective modality to deliver CBT for individuals with IBS, and could increase accessibility of this evidence-based treatment.

Trial Registration

ClinicalTrials.gov NCT04170686; https://www.clinicaltrials.gov/ct2/show/NCT04170686

Can intelligent agents improve data quality in online questiosnnaires? A pilot study

We explored the utility of chatbots for improving data quality arising from collection via sonline surveys. Three-hundred Australian adults sampled via Prolific Academic were randomized across chatbot-supported or unassisted online questionnaire conditions. The questionnaire comprised validated measures, along with challenge items formulated to be confusing yet aligned with the validated targets. The chatbot condition provided optional assistance with item clarity via a virtual support agent. Chatbot use and user satisfaction were measured through session logs and user feedback. Data quality was operationalized as between-group differences in relationships among validated and challenge measures. Findings broadly supported chatbot utility for online surveys, showing that most participants with chatbot access utilized it, found it helpful, and demonstrated modestly improved data quality (vs. controls). Absence of confusion for one challenge item is believed to have contributed to an underestimated effect. Findings show that assistive chatbots can enhance data quality, will be utilized by many participants if available, and are perceived as beneficial by most users. Scope constraints for this pilot study are believed to have led to underestimated effects. Future testing with longer-form questionnaires incorporating expanded item difficulty may further understanding of chatbot utility for survey completion and data quality.

Changes to the Psychiatric Chatbot Landscape: A Systematic Review of Conversational Agents in Serious Mental Illness: Changements du paysage psychiatrique des chatbots: une revue systématique des agents conversationnels dans la maladie mentale sérieuse

OBJECTIVE

The need for digital tools in mental health is clear, with insufficient access to mental health services. Conversational agents, also known as chatbots or voice assistants, are digital tools capable of holding natural language conversations. Since our last review in 2018, many new conversational agents and research have emerged, and we aimed to reassess the conversational agent landscape in this updated systematic review.

METHODS

A systematic literature search was conducted in January 2020 using the PubMed, Embase, PsychINFO, and Cochrane databases. Studies included were those that involved a conversational agent assessing serious mental illness: major depressive disorder, schizophrenia spectrum disorders, bipolar disorder, or anxiety disorder.

RESULTS

Of the 247 references identified from selected databases, 7 studies met inclusion criteria. Overall, there were generally positive experiences with conversational agents in regard to diagnostic quality, therapeutic efficacy, or acceptability. There continues to be, however, a lack of standard measures that allow ease of comparison of studies in this space. There were several populations that lacked representation such as the pediatric population and those with schizophrenia or bipolar disorder. While comparing 2018 to 2020 research offers useful insight into changes and growth, the high degree of heterogeneity between all studies in this space makes direct comparison challenging.

CONCLUSIONS

This review revealed few but generally positive outcomes regarding conversational agents' diagnostic quality, therapeutic efficacy, and acceptability, which may augment mental health care. Despite this increase in research activity, there continues to be a lack of standard measures for evaluating conversational agents as well as several neglected populations. We recommend that the standardization of conversational agent studies should include patient adherence and engagement, therapeutic efficacy, and clinician perspectives.

A Therapeutic Relational Agent for Reducing Problematic Substance Use (Woebot): Development and Usability Study

Background

Misuse of substances is common, can be serious and costly to society, and often goes untreated due to barriers to accessing care. Woebot is a mental health digital solution informed by cognitive behavioral therapy and built upon an artificial intelligence–driven platform to deliver tailored content to users. In a previous 2-week randomized controlled trial, Woebot alleviated depressive symptoms.

Objective

This study aims to adapt Woebot for the treatment of substance use disorders (W-SUDs) and examine its feasibility, acceptability, and preliminary efficacy.

Methods

American adults (aged 18-65 years) who screened positive for substance misuse without major health contraindications were recruited from online sources and flyers and enrolled between March 27 and May 6, 2020. In a single-group pre/postdesign, all participants received W-SUDs for 8 weeks. W-SUDs provided mood, craving, and pain tracking and modules (psychoeducational lessons and psychotherapeutic tools) using elements of dialectical behavior therapy and motivational interviewing. Paired samples t tests and McNemar nonparametric tests were used to examine within-subject changes from pre- to posttreatment on measures of substance use, confidence, cravings, mood, and pain.

Results

The sample (N=101) had a mean age of 36.8 years (SD 10.0), and 75.2% (76/101) of the participants were female, 78.2% (79/101) were non-Hispanic White, and 72.3% (73/101) were employed. Participants’ W-SUDs use averaged 15.7 (SD 14.2) days, 12.1 (SD 8.3) modules, and 600.7 (SD 556.5) sent messages. About 94% (562/598) of all completed psychoeducational lessons were rated positively. From treatment start to end, in-app craving ratings were reduced by half (87/101, 86.1% reporting cravings in the app; odds ratio 0.48, 95% CI 0.32-0.73). Posttreatment assessment completion was 50.5% (51/101), with better retention among those who initially screened higher on substance misuse. From pre- to posttreatment, confidence to resist urges to use substances significantly increased (mean score change +16.9, SD 21.4; P<.001), whereas past month substance use occasions (mean change −9.3, SD 14.1; P<.001) and scores on the Alcohol Use Disorders Identification Test-Concise (mean change −1.3, SD 2.6; P<.001), 10-item Drug Abuse Screening Test (mean change −1.2, SD 2.0; P<.001), Patient Health Questionnaire-8 item (mean change 2.1, SD 5.2; P=.005), Generalized Anxiety Disorder-7 (mean change −2.3, SD 4.7; P=.001), and cravings scale (68.6% vs 47.1% moderate to extreme; P=.01) significantly decreased. Most participants would recommend W-SUDs to a friend (39/51, 76%) and reported receiving the service they desired (41/51, 80%). Fewer felt W-SUDs met most or all of their needs (22/51, 43%).

Conclusions

W-SUDs was feasible to deliver, engaging, and acceptable and was associated with significant improvements in substance use, confidence, cravings, depression, and anxiety. Study attrition was high. Future research will evaluate W-SUDs in a randomized controlled trial with a more diverse sample and with the use of greater study retention strategies.

Trial Registration

ClinicalTrials.gov NCT04096001; http://clinicaltrials.gov/ct2/show/NCT04096001.

Delivering Perinatal Health Information via a Voice Interactive App (SMILE): Mixed Methods Feasibility Study

BACKGROUND

Perinatal health care is critically important for maternal health outcomes in infants. The United States fares considerably worse than comparable countries for maternal and infant mortality rates. As such, alternative models of care or engagement are warranted. Ubiquitous digital devices and increased use of digital health tools have the potential to extend the reach to women and infants in their everyday lives and make a positive impact on their health outcomes. As voice technology becomes more mainstream, research is prudent to establish evidence-based practice on how to best leverage voice technology to promote maternal-infant health.

OBJECTIVE

The aim of this study is to assess the feasibility of using voice technology to support perinatal health and infant care practices.

METHODS

Perinatal women were recruited from a large Midwest Children's Hospital via hospital email announcements and word of mouth. Owing to the technical aspects of the intervention, participants were required to speak English and use an iPhone. Demographics, patterns of technology use, and technology use specific to perinatal health or self-care practices were assessed at baseline. Next, participants were onboarded and asked to use the intervention, Self-Management Intervention-Life Essentials (SMILE), over the course of 2 weeks. SMILE provided users with perinatal health content delivered through mini podcasts (ranging from 3 to 8 minutes in duration). After each podcast, SMILE prompted users to provide immediate verbal feedback to the content. An exit interview was conducted with participants to gather feedback on the intervention and further explore participants' perceptions of voice technology as a means to support perinatal health in the future.

RESULTS

In total, 19 pregnant women (17 to 36 weeks pregnant) were consented. Themes identified as important for perinatal health information include establishing routines, expected norms, and realistic expectations and providing key takeaways. Themes identified as important for voice interaction include customization and user preferences, privacy, family and friends, and context and convenience. Qualitative analysis suggested that perinatal health promotion content delivered by voice should be accurate and succinctly delivered and highlight key takeaways. Perinatal health interventions that use voice should provide users with the ability to customize the intervention but also provide opportunities to engage family members, particularly spouses. As a number of women multitasked while the intervention was being deployed, future interventions should leverage the convenience of voice technology while also balancing the influence of user context (eg, timing or ability to listen or talk versus nonvoice interaction with the system).

CONCLUSIONS

Our findings demonstrate the short-term feasibility of disseminating evidence-based perinatal support via podcasts and curate voice-captured data from perinatal women. However, key areas of improvement have been identified specifically for perinatal interventions leveraging voice technology. Findings contribute to future content, design, and delivery considerations of perinatal digital health interventions.

Modelling User Preference for Embodied Artificial Intelligence and Appearance in Realistic Humanoid Robots

Realistic humanoid robots (RHRs) with embodied artificial intelligence (EAI) have numerous applications in society as the human face is the most natural interface for communication and the human body the most effective form for traversing the manmade areas of the planet. Thus, developing RHRs with high degrees of human-likeness provides a life-like vessel for humans to physically and naturally interact with technology in a manner insurmountable to any other form of non-biological human emulation. This study outlines a human–robot interaction (HRI) experiment employing two automated RHRs with a contrasting appearance and personality. The selective sample group employed in this study is composed of 20 individuals, categorised by age and gender for a diverse statistical analysis. Galvanic skin response, facial expression analysis, and AI analytics permitted cross-analysis of biometric and AI data with participant testimonies to reify the results. This study concludes that younger test subjects preferred HRI with a younger-looking RHR and the more senior age group with an older looking RHR. Moreover, the female test group preferred HRI with an RHR with a younger appearance and male subjects with an older looking RHR. This research is useful for modelling the appearance and personality of RHRs with EAI for specific jobs such as care for the elderly and social companions for the young, isolated, and vulnerable.

Computer-Controlled Virtual Humans in Patient-Facing Systems: Systematic Review and Meta-Analysis

BACKGROUND

Virtual humans (VH) are computer-generated characters that appear humanlike and simulate face-to-face conversations using verbal and nonverbal cues. Unlike formless conversational agents, like smart speakers or chatbots, VH bring together the capabilities of both a conversational agent and an interactive avatar (computer-represented digital characters). Although their use in patient-facing systems has garnered substantial interest, it is unknown to what extent VH are effective in health applications.

OBJECTIVE

The purpose of this review was to examine the effectiveness of VH in patient-facing systems. The design and implementation characteristics of these systems were also examined.

METHODS

Electronic bibliographic databases were searched for peer-reviewed articles with relevant key terms. Studies were included in the systematic review if they designed or evaluated VH in patient-facing systems. Of the included studies, studies that used a randomized controlled trial to evaluate VH were included in the meta-analysis; they were then summarized using the PICOTS framework (population, intervention, comparison group, outcomes, time frame, setting). Summary effect sizes, using random-effects models, were calculated, and the risk of bias was assessed.

RESULTS

Among the 8,125 unique records identified, 53 articles describing 33 unique systems, were qualitatively, systematically reviewed. Two distinct design categories emerged - simple VH and VH augmented with health sensors and trackers. Of the 53 articles, 16 (26 studies) with 44 primary and 22 secondary outcomes were included in the meta-analysis. Meta-analysis of the 44 primary outcome measures revealed a significant difference between intervention and control conditions, favoring the VH intervention (SMD = .166, 95% CI .039-.292, P=.012), but with evidence of some heterogeneity, I2=49.3%. There were more cross-sectional (k=15) than longitudinal studies (k=11). The intervention was delivered using a personal computer in most studies (k=18), followed by a tablet (k=4), mobile kiosk (k=2), head-mounted display (k=1), and a desktop computer in a community center (k=1).

CONCLUSIONS

We offer evidence for the efficacy of VH in patient-facing systems. Considering that studies included different population and outcome types, more focused analysis is needed in the future. Future studies also need to identify what features of virtual human interventions contribute toward their effectiveness.

Technical Metrics Used to Evaluate Health Care Chatbots: Scoping Review

BACKGROUND

Dialog agents (chatbots) have a long history of application in health care, where they have been used for tasks such as supporting patient self-management and providing counseling. Their use is expected to grow with increasing demands on health systems and improving artificial intelligence (AI) capability. Approaches to the evaluation of health care chatbots, however, appear to be diverse and haphazard, resulting in a potential barrier to the advancement of the field.

OBJECTIVE

This study aims to identify the technical (nonclinical) metrics used by previous studies to evaluate health care chatbots.

METHODS

Studies were identified by searching 7 bibliographic databases (eg, MEDLINE and PsycINFO) in addition to conducting backward and forward reference list checking of the included studies and relevant reviews. The studies were independently selected by two reviewers who then extracted data from the included studies. Extracted data were synthesized narratively by grouping the identified metrics into categories based on the aspect of chatbots that the metrics evaluated.

RESULTS

Of the 1498 citations retrieved, 65 studies were included in this review. Chatbots were evaluated using 27 technical metrics, which were related to chatbots as a whole (eg, usability, classifier performance, speed), response generation (eg, comprehensibility, realism, repetitiveness), response understanding (eg, chatbot understanding as assessed by users, word error rate, concept error rate), and esthetics (eg, appearance of the virtual agent, background color, and content).

CONCLUSIONS

The technical metrics of health chatbot studies were diverse, with survey designs and global usability metrics dominating. The lack of standardization and paucity of objective measures make it difficult to compare the performance of health chatbots and could inhibit advancement of the field. We suggest that researchers more frequently include metrics computed from conversation logs. In addition, we recommend the development of a framework of technical metrics with recommendations for specific circumstances for their inclusion in chatbot studies.

A Smartphone-Based Health Care Chatbot to Promote Self-Management of Chronic Pain (SELMA): Pilot Randomized Controlled Trial

BACKGROUND

Ongoing pain is one of the most common diseases and has major physical, psychological, social, and economic impacts. A mobile health intervention utilizing a fully automated text-based health care chatbot (TBHC) may offer an innovative way not only to deliver coping strategies and psychoeducation for pain management but also to build a working alliance between a participant and the TBHC.

OBJECTIVE

The objectives of this study are twofold: (1) to describe the design and implementation to promote the chatbot painSELfMAnagement (SELMA), a 2-month smartphone-based cognitive behavior therapy (CBT) TBHC intervention for pain self-management in patients with ongoing or cyclic pain, and (2) to present findings from a pilot randomized controlled trial, in which effectiveness, influence of intention to change behavior, pain duration, working alliance, acceptance, and adherence were evaluated.

METHODS

Participants were recruited online and in collaboration with pain experts, and were randomized to interact with SELMA for 8 weeks either every day or every other day concerning CBT-based pain management (n=59), or weekly concerning content not related to pain management (n=43). Pain-related impairment (primary outcome), general well-being, pain intensity, and the bond scale of working alliance were measured at baseline and postintervention. Intention to change behavior and pain duration were measured at baseline only, and acceptance postintervention was assessed via self-reporting instruments. Adherence was assessed via usage data.

RESULTS

From May 2018 to August 2018, 311 adults downloaded the SELMA app, 102 of whom consented to participate and met the inclusion criteria. The average age of the women (88/102, 86.4%) and men (14/102, 13.6%) participating was 43.7 (SD 12.7) years. Baseline group comparison did not differ with respect to any demographic or clinical variable. The intervention group reported no significant change in pain-related impairment (P=.68) compared to the control group postintervention. The intention to change behavior was positively related to pain-related impairment (P=.01) and pain intensity (P=.01). Working alliance with the TBHC SELMA was comparable to that obtained in guided internet therapies with human coaches. Participants enjoyed using the app, perceiving it as useful and easy to use. Participants of the intervention group replied with an average answer ratio of 0.71 (SD 0.20) to 200 (SD 58.45) conversations initiated by SELMA. Participants' comments revealed an appreciation of the empathic and responsible interaction with the TBHC SELMA. A main criticism was that there was no option to enter free text for the patients' own comments.

CONCLUSIONS

SELMA is feasible, as revealed mainly by positive feedback and valuable suggestions for future revisions. For example, the participants' intention to change behavior or a more homogenous sample (eg, with a specific type of chronic pain) should be considered in further tailoring of SELMA.

TRIAL REGISTRATION

German Clinical Trials Register DRKS00017147; https://tinyurl.com/vx6n6sx, Swiss National Clinical Trial Portal: SNCTP000002712; https://www.kofam.ch/de/studienportal/suche/70582/studie/46326.

Family health history collected by virtual conversational agents: An empirical study to investigate the efficacy of this approach

Family health history (FHx) is one of the simplest and most cost-effective and efficient ways to collect health information that could help diagnose and treat genetic diseases at an early stage. This study evaluated the efficacy of collecting such family health histories through a virtual conversational agent (VCA) interface, a new method for collecting this information. Standard and VCA interfaces for FHx collection were investigated with 50 participants, recruited via email and word of mouth, using a within-subject experimental design with the order of the interfaces randomized and counterbalanced. Interface workload, usability, preference, and satisfaction were assessed using the NASA Task Load Index workload instrument, the IBM Computer System Usability Questionnaire, and a brief questionnaire derived from the Technology Acceptance Model. The researchers also recorded the number of errors and the total task completion time. It was found that the completion times for 2 of the 5 tasks were shorter for the VCA interface than for the standard one, but the overall completion time was longer (17 min 44 s vs. 16 min 51 s, p = .019). We also found the overall workload to be significantly lower (34.32 vs. 42.64, p = .003) for the VCA interface, and usability metrics including overall satisfaction (5.62 vs. 4.72, p < .001), system usefulness (5.76 vs. 4.84, p = .001), information quality (5.43 vs. 4.62, p < .001), and interface quality (5.66 vs. 4.64, p < .001) to be significantly higher for this interface as well. Approximately 3 out of 4 participants preferred the VCA interface to the standard one. Although the overall time taken was slightly longer than with standard interface, the VCA interface was rated significantly better across all other measures and was preferred by the participants. These findings demonstrate the advantages of an innovative VCA interface for collecting FHx, validating the efficacy of using VCAs to collect complex patient-specific data in health care.

Technical Metrics Used to Evaluate Health Care Chatbots: Scoping Review (Preprint).. [DOI: 10.2196/preprints.18301] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

BACKGROUND

Dialog agents (chatbots) have a long history of application in health care, where they have been used for tasks such as supporting patient self-management and providing counseling. Their use is expected to grow with increasing demands on health systems and improving artificial intelligence (AI) capability. Approaches to the evaluation of health care chatbots, however, appear to be diverse and haphazard, resulting in a potential barrier to the advancement of the field.

OBJECTIVE

This study aims to identify the technical (nonclinical) metrics used by previous studies to evaluate health care chatbots.

METHODS

Studies were identified by searching 7 bibliographic databases (eg, MEDLINE and PsycINFO) in addition to conducting backward and forward reference list checking of the included studies and relevant reviews. The studies were independently selected by two reviewers who then extracted data from the included studies. Extracted data were synthesized narratively by grouping the identified metrics into categories based on the aspect of chatbots that the metrics evaluated.

RESULTS

Of the 1498 citations retrieved, 65 studies were included in this review. Chatbots were evaluated using 27 technical metrics, which were related to chatbots as a whole (eg, usability, classifier performance, speed), response generation (eg, comprehensibility, realism, repetitiveness), response understanding (eg, chatbot understanding as assessed by users, word error rate, concept error rate), and esthetics (eg, appearance of the virtual agent, background color, and content).

CONCLUSIONS

The technical metrics of health chatbot studies were diverse, with survey designs and global usability metrics dominating. The lack of standardization and paucity of objective measures make it difficult to compare the performance of health chatbots and could inhibit advancement of the field. We suggest that researchers more frequently include metrics computed from conversation logs. In addition, we recommend the development of a framework of technical metrics with recommendations for specific circumstances for their inclusion in chatbot studies.

Developing Embodied Conversational Agents for Coaching People in a Healthy Lifestyle: Scoping Review

BACKGROUND

Embodied conversational agents (ECAs) are animated computer characters that simulate face-to-face counseling. Owing to their capacity to establish and maintain an empathic relationship, they are deemed to be a promising tool for starting and maintaining a healthy lifestyle.

OBJECTIVE

This review aimed to identify the current practices in designing and evaluating ECAs for coaching people in a healthy lifestyle and provide an overview of their efficacy (on behavioral, knowledge, and motivational parameters) and use (on usability, usage, and user satisfaction parameters).

METHODS

We used the Arksey and O'Malley framework to conduct a scoping review. PsycINFO, Medical Literature Analysis and Retrieval System Online, and Scopus were searched with a combination of terms related to ECA and lifestyle. Initially, 1789 unique studies were identified; 20 studies were included.

RESULTS

Most often, ECAs targeted physical activity (n=16) and had the appearance of a middle-aged African American woman (n=13). Multiple behavior change techniques (median=3) and theories or principles (median=3) were applied, but their interpretation and application were usually not reported. ECAs seemed to be designed for the end user rather than with the end user. Stakeholders were usually not involved. A total of 7 out of 15 studies reported better efficacy outcomes for the intervention group, and 5 out of 8 studies reported better use-related outcomes, as compared with the control group.

CONCLUSIONS

ECAs are a promising tool for persuasive communication in the health domain. This review provided valuable insights into the current developmental processes, and it recommends the use of human-centered, stakeholder-inclusive design approaches, along with reporting on the design activities in a systematic and comprehensive manner. The gaps in knowledge were identified on the working mechanisms of intervention components and the right timing and frequency of coaching.

Using Health Information Technology to Engage African American Women on Nutrition and Supplement Use During the Preconception Period

IMPORTANCE

Healthy nutrition and appropriate supplementation during preconception have important implications for the health of the mother and newborn. The best way to deliver preconception care to address health risks related to nutrition is unknown.

METHODS

We conducted a secondary analysis of data from a randomized controlled trial designed to study the impact of conversational agent technology in 13 domains of preconception care among 528 non-pregnant African American and Black women. This analysis is restricted to those 480 women who reported at least one of the ten risks related to nutrition and dietary supplement use.

INTERVENTIONS

An online conversational agent, called "Gabby", assesses health risks and delivers 12 months of tailored dialogue for over 100 preconception health risks, including ten nutrition and supplement risks, using behavioral change techniques like shared decision making and motivational interviewing. The control group received a letter listing their preconception risks and encouraging them to talk to a health care provider.

RESULTS

After 6 months, women using Gabby (a) reported progressing forward on the stage of change scale for, on average, 52.9% (SD, 35.1%) of nutrition and supplement risks compared to 42.9% (SD, 35.4) in the control group (IRR 1.22, 95% CI 1.03-1.45, P = 0.019); and (b) reported achieving the action and maintenance stage of change for, on average, 52.8% (SD 37.1) of the nutrition and supplement risks compared to 42.8% (SD, 37.9) in the control group (IRR 1.26, 96% CI 1.08-1.48, P = 0.004). For subjects beginning the study at the contemplation stage of change, intervention subjects reported progressing forward on the stage of change scale for 75.0% (SD, 36.3%) of their health risks compared to 52.1% (SD, 47.1%) in the control group (P = 0.006).

CONCLUSION

The scalability of Gabby has the potential to improve women's nutritional health as an adjunct to clinical care or at the population health level. Further studies are needed to determine if improving nutrition and supplement risks can impact clinical outcomes including optimization of weight.

CLINICAL TRIAL REGISTRATION

ClinicalTrials.gov, identifier NCT01827215.

Effectiveness of integrative medicine group visits in chronic pain and depressive symptoms: A randomized controlled trial

BACKGROUND

Current treatment options for chronic pain and depression are largely medication-based, which may cause adverse side effects. Integrative Medical Group Visits (IMGV) combines mindfulness techniques, evidence based integrative medicine, and medical group visits, and is a promising adjunct to medications, especially for diverse underserved patients who have limited access to non-pharmacological therapies.

OBJECTIVE

Determine the effectiveness of IMGV compared to a Primary Care Provider (PCP) visit in patients with chronic pain and depression.

DESIGN

9-week single-blind randomized control trial with a 12-week maintenance phase (intervention-medical groups; control-primary care provider visit).

SETTING

Academic tertiary safety-net hospital and 2 affiliated federally-qualified community health centers.

PARTICIPANTS

159 predominantly low income racially diverse adults with nonspecific chronic pain and depressive symptoms.

INTERVENTIONS

IMGV intervention- 9 weekly 2.5 hour in person IMGV sessions, 12 weeks on-line platform access followed by a final IMGV at 21 weeks.

MEASUREMENTS

Data collected at baseline, 9, and 21 weeks included primary outcomes depressive symptoms (Patient Health Questionnaire 9), pain (Brief Pain Inventory). Secondary outcomes included pain medication use and utilization.

RESULTS

There were no differences in pain or depression at any time point. At 9 weeks, the IMGV group had fewer emergency department visits (RR 0.32, 95% CI: 0.12, 0.83) compared to controls. At 21 weeks, the IMGV group reported reduction in pain medication use (Odds Ratio: 0.42, CI: 0.18-0.98) compared to controls.

LIMITATIONS

Absence of treatment assignment concealment for patients and disproportionate group attendance in IMGV.

CONCLUSION

Results demonstrate that low-income racially diverse patients will attend medical group visits that focus on non-pharmacological techniques, however, in the attention to treat analysis there was no difference in average pain levels between the intervention and the control group.

TRIAL REGISTRATION

clinicaltrials.gov NCT02262377.

Computational Commensality: From Theories to Computational Models for Social Food Preparation and Consumption in HCI

Food and eating are inherently social activities taking place, for example, around the dining table at home, in restaurants, or in public spaces. Enjoying eating with others, often referred to as “commensality,” positively affects mealtime in terms of, among other factors, food intake, food choice, and food satisfaction. In this paper we discuss the concept of “Computational Commensality,” that is, technology which computationally addresses various social aspects of food and eating. In the past few years, Human-Computer Interaction started to address how interactive technologies can improve mealtimes. However, the main focus has been made so far on improving the individual's experience, rather than considering the inherently social nature of food consumption. In this survey, we first present research from the field of social psychology on the social relevance of Food- and Eating-related Activities (F&EA). Then, we review existing computational models and technologies that can contribute, in the near future, to achieving Computational Commensality. We also discuss the related research challenges and indicate future applications of such new technology that can potentially improve F&EA from the commensality perspective.

Conversational Agents in the Treatment of Mental Health Problems: Mixed-Method Systematic Review

BACKGROUND

The use of conversational agent interventions (including chatbots and robots) in mental health is growing at a fast pace. Recent existing reviews have focused exclusively on a subset of embodied conversational agent interventions despite other modalities aiming to achieve the common goal of improved mental health.

OBJECTIVE

This study aimed to review the use of conversational agent interventions in the treatment of mental health problems.

METHODS

We performed a systematic search using relevant databases (MEDLINE, EMBASE, PsycINFO, Web of Science, and Cochrane library). Studies that reported on an autonomous conversational agent that simulated conversation and reported on a mental health outcome were included.

RESULTS

A total of 13 studies were included in the review. Among them, 4 full-scale randomized controlled trials (RCTs) were included. The rest were feasibility, pilot RCTs and quasi-experimental studies. Interventions were diverse in design and targeted a range of mental health problems using a wide variety of therapeutic orientations. All included studies reported reductions in psychological distress postintervention. Furthermore, 5 controlled studies demonstrated significant reductions in psychological distress compared with inactive control groups. In addition, 3 controlled studies comparing interventions with active control groups failed to demonstrate superior effects. Broader utility in promoting well-being in nonclinical populations was unclear.

CONCLUSIONS

The efficacy and acceptability of conversational agent interventions for mental health problems are promising. However, a more robust experimental design is required to demonstrate efficacy and efficiency. A focus on streamlining interventions, demonstrating equivalence to other treatment modalities, and elucidating mechanisms of action has the potential to increase acceptance by users and clinicians and maximize reach.