1
|
Torabgar M, Figeys M, Esmail S, Stroulia E, Ríos Rincón AM. Machine Learning Analysis of Engagement Behaviors in Older Adults With Dementia Playing Mobile Games: Exploratory Study. JMIR Serious Games 2025; 13:e54797. [PMID: 40029608 DOI: 10.2196/54797] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Revised: 12/08/2024] [Accepted: 12/11/2024] [Indexed: 03/05/2025] Open
Abstract
Background The prevalence of dementia is expected to rise with an aging population, necessitating accessible early detection methods. Serious games have emerged as potential cognitive screening tools. They provide not only an engaging platform for assessing cognitive function but also serve as valuable indicators of cognitive health through engagement levels observed during play. Objective This study aims to examine the differences in engagement-related behaviors between older adults with and without dementia during serious gaming sessions. Further, it seeks to identify the key contributors that enhance the effectiveness of machine learning for dementia classification based on engagement-related behaviors. Methods This was an exploratory proof-of-concept study. Over 8 weeks, 20 older adults, 6 of whom were living with dementia, were enrolled in a single-case design study. Participants played 1 of 4 "Vibrant Minds" serious games (Bejeweled, Whack-A-Mole, Mah-jong, and Word-Search) over 8 weeks (16 30-min sessions). Throughout the study, sessions were recorded to analyze engagement-related behaviors. This paper reports on the analysis of the engagement-related behaviors of 15 participants. The videos of these 15 participants (10 cognitively intact, 5 with dementia) were analyzed by 2 independent raters, individually annotating engagement-related behaviors at 15-second intervals using a coding system. This analysis resulted in 1774 data points categorized into 47 behavior codes, augmented by 54 additional features including personal characteristics, technical issues, and environmental factors. Each engagement-related behavior was compared between older adults living with dementia and older adults without dementia using the χ² test with a 2×2 contingency table with a significance level of .05. Codes underwent one-hot encoding and were processed using random forest classifiers to distinguish between participant groups. Results Significant differences in 64% of engagement-related behaviors were found between groups, notably in torso movements, voice modulation, facial expressions, and concentration. Including engagement-related behaviors, environmental disturbances, technical issues, and personal characteristics resulted in the best model for classifying cases of dementia correctly, achieving an F1-score of 0.91 (95% CI 0.851-0.963) and an area under the receiver operating curve of 0.99 (95% CI 0.984-1.000). Conclusions Key features distinguishing between older adults with and without dementia during serious gameplay included torso, voice, facial, and concentration behaviors, as well as age. The best performing machine learning model identified included features of engagement-related behavios, environmental disturbances, technical challenges, and personal attributes. Engagement-related behaviors observed during serious gaming offer crucial markers for identifying dementia. Machine learning models that incorporate these unique behavioral markers present a promising, noninvasive approach for early dementia screening in a variety of settings.
Collapse
Affiliation(s)
- Melika Torabgar
- Department of Occupational Therapy, University of Alberta, Edmonton, AB, Canada
| | - Mathieu Figeys
- Department of Occupational Therapy, University of Alberta, Edmonton, AB, Canada
| | - Shaniff Esmail
- Department of Occupational Therapy, University of Alberta, Edmonton, AB, Canada
| | - Eleni Stroulia
- Department of Computing Science, University of Alberta, Edmonton, AB, Canada
| | | |
Collapse
|
2
|
Gilmore-Bykovskyi A, Dillon K, Coulson A, Fehland J, Benson C, Sanson-Miles L, Mueller K. An observational study protocol to capture, validate and characterise lucid episodes in people living with advanced dementia receiving hospice care. BMJ Open 2024; 14:e085897. [PMID: 38760031 PMCID: PMC11103230 DOI: 10.1136/bmjopen-2024-085897] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/28/2024] [Accepted: 04/25/2024] [Indexed: 05/19/2024] Open
Abstract
INTRODUCTION Lucid episodes (LEs) in advanced neurodegenerative disease, characterised by a transient recovery of abilities, have been reported across neurological conditions, including Alzheimer's disease and related dementias. Evidence on LEs in dementia is extremely limited and draws predominantly from retrospective case reports. Lucidity in dementia has received growing attention given the clinical, caregiving and potential epidemiological implications of even a temporary return of abilities in advanced disease. Following a funding initiative by the National Institute on Aging, several new investigations are focused on establishing foundational evidence on lucidity in dementia. The objectives of this study are to capture, characterise and validate potential LEs via audiovisual observation, computational linguistic and timed-event coding of audiovisual data, and informant case review for face validation of LEs. METHODS AND ANALYSIS This prospective multifaceted observational study will investigate LEs in advanced dementia through longitudinal audiovisual observation within an inpatient hospice unit. Audiovisual data will be coded to generate variables of participant verbal output, verbal expressions, non-verbal communicative actions and functional behaviours to enable measurement of features that can be used to characterise LEs. Multiple methods will be used to identify potential LEs including field interviews with caregivers/clinicians who witness significant events during data collection, reports from research staff who witness significant events during data collection and detection by researchers during video data processing procedures. Potential LEs will undergo a structured case review with informants familiar with the participant to facilitate validation and enable triangulation across measures generated through coding. ETHICS AND DISSEMINATION This study will be conducted in accordance with all Federal Policies for the Protection of Human Subjects and the protocol (ID 2021-1243) has been approved by the University of Wisconsin-Madison Institutional Review Board. Findings will be disseminated via scientific conferences, journal publications and newsletters shared with participants and through dementia-focused and caregiver-focused networks.
Collapse
Affiliation(s)
- Andrea Gilmore-Bykovskyi
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison School of Medicine and Public Health, Madison, Wisconsin, USA
| | - Kayla Dillon
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison School of Medicine and Public Health, Madison, Wisconsin, USA
| | - Alison Coulson
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison School of Medicine and Public Health, Madison, Wisconsin, USA
| | - Jess Fehland
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison School of Medicine and Public Health, Madison, Wisconsin, USA
| | - Clark Benson
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison School of Medicine and Public Health, Madison, Wisconsin, USA
| | - Leah Sanson-Miles
- Department of Communication Sciences and Disorders, University of Wisconsin-Madison, Madison, Wisconsin, USA
| | - Kimberly Mueller
- Department of Communication Sciences and Disorders, University of Wisconsin-Madison, Madison, Wisconsin, USA
| |
Collapse
|
3
|
Tanner A, Urech A, Schulze H, Manser T. Older Adults' Engagement and Mood During Robot-Assisted Group Activities in Nursing Homes: Development and Observational Pilot Study. JMIR Rehabil Assist Technol 2023; 10:e48031. [PMID: 38145484 PMCID: PMC10775040 DOI: 10.2196/48031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Revised: 10/23/2023] [Accepted: 11/15/2023] [Indexed: 12/26/2023] Open
Abstract
BACKGROUND Promoting the well-being of older adults in an aging society requires new solutions. One resource might be the use of social robots for group activities that promote physical and cognitive stimulation. Engaging in a robot-assisted group activity may help in the slowdown of physical and cognitive decline in older adults. Currently, our knowledge is limited on whether older adults engage in group activities with humanlike social robots and whether they experience a positive affect while doing so. Both are necessary preconditions to achieve the intended effects of a group activity. OBJECTIVE Our pilot study has 2 aims. First, we aimed to develop and pilot an observational coding scheme for robot-assisted group activities because self-report data on engagement and mood of nursing home residents are often difficult to obtain, and the existing observation instruments do have limitations. Second, we aimed to investigate older adults' engagement and mood during robot-assisted group activities in 4 different nursing care homes in the German-speaking part of Switzerland. METHODS We developed an observation system, inspired by existing tools, for a structured observation of engagement and mood of older adults during a robot-assisted group activity. In this study, 85 older adult residents from 4 different care homes in Switzerland participated in 5 robot-assisted group activity sessions, and they were observed using our developed system. The data were collected in the form of video clips that were assessed by 2 raters regarding engagement (direction of gaze, posture as well as body expression, and activity) and mood (positive and negative affects). Both variables were rated on a 5-point rating scale. RESULTS Our pilot study findings show that the engagement and mood of older adults can be assessed reliably by using the proposed observational coding scheme. Most participants actively engaged in robot-assisted group activities (mean 4.19, SD 0.47; median 4.0). The variables used to measure engagement were direction of gaze (mean 4.65, SD 0.49; median 5.0), posture and body expression (mean 4.03, SD 0.71; median 4.0), and activity (mean 3.90, SD 0.65; median 4.0). Further, we observed mainly positive affects in this group. Almost no negative affect was observed (mean 1.13, SD 0.20; median 1.0), while the positive affect (mean 3.22, SD 0.55; median 3.2) was high. CONCLUSIONS The developed observational coding system can be used and further developed in future studies on robot-assisted group activities in the nursing home context and potentially in other settings. Additionally, our pilot study indicates that cognitive and physical stimulation of older adults can be promoted by social robots in a group setting. This finding encourages future technological development and improvement of social robots and points to the potential of observational research to systematically evaluate such developments.
Collapse
Affiliation(s)
- Alexandra Tanner
- School of Applied Psychology, University of Applied Sciences and Arts Northwestern Switzerland, Olten, Switzerland
- City of Bern (Digital Stadt Bern), Bern, Switzerland
| | - Andreas Urech
- School of Applied Psychology, University of Applied Sciences and Arts Northwestern Switzerland, Olten, Switzerland
| | - Hartmut Schulze
- School of Applied Psychology, University of Applied Sciences and Arts Northwestern Switzerland, Olten, Switzerland
| | - Tanja Manser
- School of Applied Psychology, University of Applied Sciences and Arts Northwestern Switzerland, Olten, Switzerland
| |
Collapse
|
4
|
Staehler M, Benson C, Block L, Roberts T, Gilmore-Bykovskyi A. Verbal and Nonverbal Expressions of Persons Living With Dementia as Indicators of Person-Centered Caregiving. THE GERONTOLOGIST 2022; 62:1299-1310. [PMID: 35092676 PMCID: PMC9579465 DOI: 10.1093/geront/gnac012] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2021] [Indexed: 01/31/2023] Open
Abstract
BACKGROUND AND OBJECTIVES Improved measures capable of capturing the influence of person-centered caregiving by staff in formal care settings on people living with dementia beyond deficit-oriented outcomes such as absence or reduction of symptoms are important for measuring progress toward improvements in well-being. This exploratory ethnographic study aimed to identify verbal and nonverbal expressions evidenced by people living with dementia surrounding person-centered caregiving approaches and to consider their use in informing temporally specific observational measures. RESEARCH DESIGN AND METHODS This study adopted a microethnographic approach through secondary analysis of 5.3 h of audiovisual observations of people living with dementia (N = 9) in nursing home settings at mealtimes. We observed expressions surrounding person-centered caregiving approaches. A systematic review of audiovisual observations generated codes (observable indicators) of expressions that were characterized at their most discrete and unambiguous level. RESULTS Drawing from 82 observable verbal and nonverbal expressions by people living with dementia, 14 discrete observable indicators were identified, broadly evidencing shifts in engagement and communication. We found that people living with dementia's expressions served both responsive and initiatory communicative purposes. DISCUSSION AND IMPLICATIONS Efforts to expand positive outcome measurement for people living with dementia should extend beyond characterizing them as passive respondents toward active participants in their lived experiences. Identified observable indicators can inform efforts to refine and validate measures of expressions among people living with dementia. Further research can extend this inquiry into different contexts and engage input from people living with dementia and caregivers.
Collapse
Affiliation(s)
- Maya Staehler
- University of Wisconsin–Madison School of Nursing, Madison, Wisconsin, USA
| | - Clark Benson
- University of Wisconsin–Madison School of Nursing, Madison, Wisconsin, USA
| | - Laura Block
- University of Wisconsin–Madison School of Nursing, Madison, Wisconsin, USA
| | - Tonya Roberts
- University of Wisconsin–Madison School of Nursing, Madison, Wisconsin, USA
| | - Andrea Gilmore-Bykovskyi
- University of Wisconsin–Madison School of Nursing, Madison, Wisconsin, USA
- Division of Geriatrics, Department of Medicine, University of Wisconsin–Madison School of Medicine and Public Health, Madison, Wisconsin, USA
| |
Collapse
|
5
|
Madsø KG, Flo-Groeneboom E, Pachana NA, Nordhus IH. Assessing Momentary Well-Being in People Living With Dementia: A Systematic Review of Observational Instruments. Front Psychol 2021; 12:742510. [PMID: 34887803 PMCID: PMC8649635 DOI: 10.3389/fpsyg.2021.742510] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2021] [Accepted: 10/26/2021] [Indexed: 11/17/2022] Open
Abstract
Optimizing the possibility to lead good lives is at the core of treatment and care for people with dementia. This may be monitored by assessing well-being and quality of life. However, cognitive impairment following dementia may complicate recall-based assessment with questionnaires, and proxy-ratings from family-caregivers do not correspond well to self-reports. Thus, using observational measures represents a potentially advanced option. Systematic reviews evaluating measurement properties, interpretability and feasibility of observational instruments assessing well-being in people living with dementia are lacking. Thus, this review performed systematic searches to find peer reviewed validated instruments of relevance in the databases MEDLINE, EMBASE, PsycINFO, Web of Science, CINAHL and ProQuest. Twenty-two instruments assessing well-being were included for evaluation of measurement properties based on the systematic approach of the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN). The evaluation included risk of bias on study level, and assessment of measurement properties on instrument level including content validity, construct validity, structural validity, internal consistency, measurement invariance, cross-cultural validity, measurement error and inter-rater/intra-rater/test-retest reliability and responsiveness. Additionally, the feasibility and interpretability of the measures were evaluated. No single instrument could be recommended based on existing publications. Thus, we provide general recommendations about further assessment and development of these instruments. Finally, we describe the most promising instruments and offer guidance with respect to their implementation and use in clinical and research contexts.
Collapse
Affiliation(s)
- Kristine Gustavsen Madsø
- Department of Clinical Psychology, Faculty of Psychology, University of Bergen, Bergen, Norway
- NKS Olaviken Gerontopsychiatric Hospital, Bergen, Norway
| | | | - Nancy A. Pachana
- School of Psychology, The University of Queensland, Brisbane, QLD, Australia
| | - Inger Hilde Nordhus
- Department of Clinical Psychology, Faculty of Psychology, University of Bergen, Bergen, Norway
- Department of Behavioral Medicine, Faculty of Medicine, University of Oslo, Oslo, Norway
| |
Collapse
|
6
|
Context-Enhanced Human-Robot Interaction: Exploring the Role of System Interactivity and Multimodal Stimuli on the Engagement of People with Dementia. Int J Soc Robot 2021. [DOI: 10.1007/s12369-021-00823-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
AbstractEngaging people with dementia (PWD) in meaningful activities is the key to promote their quality of life. Design towards a higher level of user engagement has been extensively studied within the human-computer interaction community, however, few extend to PWD. It is generally considered that increased richness of experiences can lead to enhanced engagement. Therefore, this paper explores the effects of rich interaction in terms of the role of system interactivity and multimodal stimuli by engaging participants in context-enhanced human-robot interaction activities. The interaction with a social robot was considered context-enhanced due to the additional responsive sensory feedback from an augmented reality display. A field study was conducted in a Dutch nursing home with 16 residents. The study followed a two by two mixed factorial design with one within-subject variable - multimodal stimuli - and one between-subject variable - system interactivity. A mixed method of video coding analysis and observational rating scales was adopted to assess user engagement comprehensively. Results disclose that when additional auditory modality was included besides the visual-tactile stimuli, participants had significantly higher scores on attitude, more positive behavioral engagement during activity, and a higher percentage of communications displayed. The multimodal stimuli also promoted social interaction between participants and the facilitator. The findings provide sufficient evidence regarding the significant role of multimodal stimuli in promoting PWD’s engagement, which could be potentially used as a motivation strategy in future research to improve emotional aspects of activity-related engagement and social interaction with the human partner.
Collapse
|
7
|
Song H, Barakova EI, Markopoulos P, Ham J. Personalizing HRI in Musical Instrument Practicing: The Influence of Robot Roles (Evaluative Versus Nonevaluative) on the Child's Motivation for Children in Different Learning Stages. Front Robot AI 2021; 8:699524. [PMID: 34660701 PMCID: PMC8517074 DOI: 10.3389/frobt.2021.699524] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2021] [Accepted: 08/19/2021] [Indexed: 11/13/2022] Open
Abstract
Learning to play a musical instrument involves skill learning and requires long-term practicing to reach expert levels. Research has already proven that the assistance of a robot can improve children's motivation and performance during practice. In an earlier study, we showed that the specific role (evaluative role versus nonevaluative role) the robot plays can determine children's motivation and performance. In the current study, we argue that the role of the robot has to be different for children in different learning stages (musical instrument expertise levels). Therefore, this study investigated whether children in different learning stages would have higher motivation when assisted by a robot in different supporting roles (i.e., evaluative role versus nonevaluative role). We conducted an empirical study in a real practice room of a music school with 31 children who were at different learning stages (i.e., beginners, developing players, and advanced players). In this study, every child practiced for three sessions: practicing alone, assisted by the evaluative robot, or assisted by the nonevaluative robot (in a random order). We measured motivation by using a questionnaire and analyzing video data. Results showed a significant interaction between condition (i.e., alone, evaluative robot, and nonevaluative robot) and learning stage groups indicating that children in different learning stage groups had different levels of motivation when practicing alone or with an evaluative or nonevaluative robot. More specifically, beginners had higher persistence when practicing with the nonevaluative robot, while advanced players expressed higher motivation after practicing with a robot than alone, but no difference was found between the two robot roles. Exploratory results also indicated that gender might have an interaction effect with the robot roles on child's motivation in music practice with social robots. This study offers more insight into the child-robot interaction and robot role design in musical instrument learning. Specifically, our findings shed light on personalization in HRI, that is, from adapting the role of the robot to the characteristics and the development level of the user.
Collapse
Affiliation(s)
- Heqiu Song
- Department of Industrial Design, Eindhoven University of Technology, Eindhoven, Netherlands
| | - Emilia I. Barakova
- Department of Industrial Design, Eindhoven University of Technology, Eindhoven, Netherlands
| | - Panos Markopoulos
- Department of Industrial Design, Eindhoven University of Technology, Eindhoven, Netherlands
| | - Jaap Ham
- Human-Technology Interaction, Department of Industrial Engineering and Innovation Sciences, Eindhoven University of Technology, Eindhoven, Netherlands
| |
Collapse
|
8
|
Lu LC, Lan SH, Hsieh YP, Lan SJ. Effectiveness of intergenerational participation on residents with dementia: A systematic review and meta-analysis. Nurs Open 2021; 9:920-931. [PMID: 34021969 PMCID: PMC8859040 DOI: 10.1002/nop2.919] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2020] [Revised: 02/18/2021] [Accepted: 04/14/2021] [Indexed: 11/20/2022] Open
Abstract
Aim This meta‐analysis evaluated the effectiveness of intergenerational program participation for long‐term care institution residents with dementia. Design A systematic review and meta‐analysis. Methods Ten electronic databases were systematically searched until August 2020: CINAHL, the Cochrane Library, EBSCO, EMBASE, Ovid Medline, ProQuest, Psychology and Behavioral Sciences Collection, PubMed, Scopus, and Web of Science. The Joanna Briggs Institute tool (JBI tool) was used for the quality appraisal of the included publications, and Review Manager 5.3 was used for the meta‐analysis. Results Thirteen articles were identified (1993–2015). Intergenerational program participation could improve the pleasure level and significantly reduce disengagement behaviours of residents with dementia. The intergenerational program intervention caused no apparent improvement in their quality of life, depression levels, and engagement levels.
Collapse
Affiliation(s)
- Li-Chin Lu
- School of Management, Putian University, Putian, China
| | - Shao-Huan Lan
- School of Pharmaceutical Sciences and Medical Technology, Putian University, Putian, China
| | - Yen-Ping Hsieh
- Department of Long-term Care, National Quemoy University, Jinning Township, Taiwan
| | - Shou-Jen Lan
- School of Basic Medical Science, Putian University, Putian, China
| |
Collapse
|
9
|
Perugia G, Paetzel-Prüsmann M, Alanenpää M, Castellano G. I Can See It in Your Eyes: Gaze as an Implicit Cue of Uncanniness and Task Performance in Repeated Interactions With Robots. Front Robot AI 2021; 8:645956. [PMID: 33898532 PMCID: PMC8058462 DOI: 10.3389/frobt.2021.645956] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2020] [Accepted: 03/04/2021] [Indexed: 11/16/2022] Open
Abstract
Over the past years, extensive research has been dedicated to developing robust platforms and data-driven dialog models to support long-term human-robot interactions. However, little is known about how people's perception of robots and engagement with them develop over time and how these can be accurately assessed through implicit and continuous measurement techniques. In this paper, we explore this by involving participants in three interaction sessions with multiple days of zero exposure in between. Each session consists of a joint task with a robot as well as two short social chats with it before and after the task. We measure participants' gaze patterns with a wearable eye-tracker and gauge their perception of the robot and engagement with it and the joint task using questionnaires. Results disclose that aversion of gaze in a social chat is an indicator of a robot's uncanniness and that the more people gaze at the robot in a joint task, the worse they perform. In contrast with most HRI literature, our results show that gaze toward an object of shared attention, rather than gaze toward a robotic partner, is the most meaningful predictor of engagement in a joint task. Furthermore, the analyses of gaze patterns in repeated interactions disclose that people's mutual gaze in a social chat develops congruently with their perceptions of the robot over time. These are key findings for the HRI community as they entail that gaze behavior can be used as an implicit measure of people's perception of robots in a social chat and of their engagement and task performance in a joint task.
Collapse
Affiliation(s)
- Giulia Perugia
- Uppsala Social Robotics Lab, Department of Information Technology, Uppsala University, Uppsala, Sweden
| | - Maike Paetzel-Prüsmann
- Computational Linguistics, Department of Linguistics, University of Potsdam, Potsdam, Germany
| | - Madelene Alanenpää
- Uppsala Social Robotics Lab, Department of Information Technology, Uppsala University, Uppsala, Sweden
| | - Ginevra Castellano
- Uppsala Social Robotics Lab, Department of Information Technology, Uppsala University, Uppsala, Sweden
| |
Collapse
|
10
|
Fan J, Ullal A, Beuscher L, Mion LC, Newhouse P, Sarkar N. Field Testing of Ro-Tri, a Robot-Mediated Triadic Interaction for Older Adults. Int J Soc Robot 2021; 13:1711-1727. [PMID: 33643494 PMCID: PMC7897418 DOI: 10.1007/s12369-021-00760-2] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/03/2021] [Indexed: 01/10/2023]
Abstract
Older adults residing in long term care (LTC) settings commonly experience apathy, a neuropsychiatric condition with adverse consequences of increased morbidity and mortality. Activities that combine social, physical and cognitive stimuli are most effective in engaging older adults with apathy but are time consuming and require significant staff resources. We present the results from an initial pilot field study of our socially assistive robotic (SAR) system, Ro-Tri, capable of multi-modal interventions to foster social interaction between pairs of older adults. Seven paired participants attended two sessions a week for three weeks. Sessions consisted of robot-mediated triadic interactions with three types of activities repeated once over the 3 weeks. Ro-Tri gathered quantitative interaction data, head pose, vocal sound, and physiological signals to automatically evaluate older adults' activity and social engagement. Ro-Tri functioned smoothly without any technical issues. Older adults had > 90% attendance and 100% completion rate and remained engaged with the system throughout the study duration. Participants' visual attention toward the SAR system and their partners increased 7.2% and 4.7%, respectively, with their interaction effort showing an increase of 2.9%. Older adults and LTC staff had positive perceptions with the system. These initial results demonstrate Ro-Tri's ability to engage older adults, encourage social human-to-human interaction, and assess the changes using quantitative metrics. Future studies will determine SAR's impact on apathy in LTC older adults.
Collapse
Affiliation(s)
- Jing Fan
- Department of EECS, Vanderbilt University, Nashville, TN USA
| | - Akshith Ullal
- Department of EECS, Vanderbilt University, Nashville, TN USA
| | - Linda Beuscher
- School of Nursing, Vanderbilt University, Nashville, TN USA
| | | | - Paul Newhouse
- Center for Cognitive Medicine, Vanderbilt University Medical Center, Nashville, TN USA
- Geriatric Research, Education and Clinical Center, Tennessee Valley Veterans Affairs Medical Center, Nashville, TN USA
| | - Nilanjan Sarkar
- Department of Mechanical Engineering, Vanderbilt University, Nashville, TN USA
| |
Collapse
|
11
|
Berenbaum R, Tziraki C, Baum R, Rosen A, Reback T, Abikhzer J, Naparstek D, Ben-David BM. Focusing on Emotional and Social Intelligence Stimulation of People With Dementia by Playing a Serious Game—Proof of Concept Study. FRONTIERS IN COMPUTER SCIENCE 2020. [DOI: 10.3389/fcomp.2020.536880] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023] Open
Abstract
Context: Dementia is one of the top five chronic diseases, which has an overwhelming impact on patients' life quality, family, and caregivers. Currently, research relating to people with dementia (PwD) focuses on the deterioration of cognitive abilities. A more innovative approach, and the one taken by this paper, is focusing on methods to maintain and improve functionality, communication and quality of life of PwD by building on remaining capacities in the yet unexplored domain of emotional and social intelligence (ESI). The use of serious games for PwD (SG4D) aimed at building social and emotional capacity is a budding field of research.Objectives: Proof of concept that the, low cost, easy to deploy SG4D, called “My Brain Works” (MBW), co-designed with PwD, enhances ESI, based on the Bar-On ESI model.Methods: 27 PwD, clients at MELABEV dementia day center, participated in a mixed methods 12 weeks pilot, proof of concept study using a tablet SG4D co-designed with PwD. Quantitative performance data was collected automatically by the tablet during game sessions. In this paper we focus on the analysis of the qualitative and quantitative data related to ESI, observed by 10 different researchers, during each game session.Results: Quantitative data revealed: both the PwD with high and low MoCA scores had similar average ESI scores. Qualitative analysis revealed that the PwD demonstrated 9 sub-components of the Bar-On ESI Model.Conclusion: While there is no drug to stop cognitive decline associated with dementia, interventions related to ESI, on the other hand, may improve functioning and quality of life. Despite declines in cognitive abilities, our study shows that a tablet based SG4D can stimulate their ESI and evoke responses in self-awareness, empathy, social and communication capacities. Using SG4D to exercise and maintain social skills is an area that may be promising in the future and may help counter the negative effects of social isolation and loneliness. Such games, while not focusing on cognitive improvement, may also impact on cognitive functioning and help bridge the gap between caregiver and PwD. More research is needed with larger sample sizes.
Collapse
|
12
|
The Effects of Long-Term Child–Robot Interaction on the Attention and the Engagement of Children with Autism. ROBOTICS 2020. [DOI: 10.3390/robotics9040079] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
Using a social robot has been proven to have multiple benefits for the training of children with Autism Spectrum Disorder (ASD). However, there is no clarity on the impact of the interaction quality between a child with ASD and a robot on the effectiveness of the therapy. Previous research showed that the use of a robot in Pivotal Response Treatment (PRT) could be an effective treatment component in diminishing ASD-related symptoms. Further analyzing the data from a randomized controlled trial of PRT treatment, we looked at the long-term effects of child–robot game interactions to see whether the interaction quality changes over time. The attention and the engagement of six children were measured through the observation of non-verbal behavior at three different stages in the treatment that took 20 sessions per child. The gaze and arm/hand behavior of the participants towards the robot, the game, and other present humans were observed. The analysis showed no significant decrease in the attention and the engagement of the children towards the robot and the game. However, the attention and engagement toward the parents of the children increased. We conclude that the main result of sustained attention and engagement with the robot is due to the personalization of the games to meet the specific needs of this user group. These specific needs are met through inclusion of variability to the level of development and personal choice of each participating child. We see the additional finding of increased attention towards the parents as especially positive since the children are expected to improve in human–human interaction as a result of this treatment.
Collapse
|
13
|
Ke C, Lou VWQ, Tan KCK, Wai MY, Chan LL. Changes in technology acceptance among older people with dementia: the role of social robot engagement. Int J Med Inform 2020; 141:104241. [PMID: 32739611 DOI: 10.1016/j.ijmedinf.2020.104241] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2020] [Revised: 07/18/2020] [Accepted: 07/25/2020] [Indexed: 02/09/2023]
Abstract
OBJECTIVE Emerging technologies such as social robots have shown to be effective in reducing loneliness and agitation for older people with dementia. However, the acceptance of technology (specifically social robots) was found to be low for older people with dementia. The current understanding of changes in technology acceptance following direct exposure is limited and lacks rigorous study design. This study examined the change in technology acceptance after a direct interaction with a humanoid social robot (Kabochan) that was deployed for long-term care facilities' residents with dementia. METHODS The technology acceptance was assessed using randomised control trials for a time frame of 32 weeks. A total of 103 residents clinically diagnosed with dementia, with a mean age of 87.2 years (SD = 7.4), were recruited from seven long-term care facilities in Hong Kong and were randomly allocated to either Kabochan-engagement group or control group. Participants in the engagement group interacted with Kabochan in an individual, non-facilitated approach. The behavioural engagement with Kabochan was observed by care workers and recorded into constructive engagement and non-engagement. Questionnaire surveys were taken placed at pre- and post-exposure with Kabochan to measure attitudes and beliefs towards technology. The questionnaire was based on previous studies on technology acceptance and included attitudes towards technology, perceived usefulness, perceived ease of use, technology self-efficacy, technology anxiety, and facilitating conditions. RESULTS Perceived ease of use was improved at week 32 for participants who interacted with Kabochan (F = 4.239, p = 0.042) with a small effect (ηp2 = 0.043) in comparison to usual care group. Results further demonstrated that the magnitude of belief changes was related to the intensity of constructive behavioural engagement. Specifically, resident-robot behavioural engagement moderately improved attitudes towards technology (F = 11.62, p < 0.001, ηp2 = 0.11) and perceived usefulness (F = 5.75, p = 0.02, ηp2 = 0.06). CONCLUSIONS The study tentatively supports that exposure to Kabochan has potential for changing perceived ease of use but not for other beliefs and attitudes towards technology among long-term care residents with dementia. Direct engagement with a humanoid social robot might be promising in improving the perceived ease of use towards technology.
Collapse
Affiliation(s)
- Chen Ke
- Sau Po Centre on Ageing, the University of Hong Kong, Hong Kong.
| | - Vivian Wei-Qun Lou
- Sau Po Centre on Ageing, the University of Hong Kong, Hong Kong; Social Work and Social Administration, the University of Hong Kong, Pok Fu Lam, Hong Kong.
| | - Kelvin Cheng-Kian Tan
- Social Work and Social Administration, the University of Hong Kong, Pok Fu Lam, Hong Kong.
| | - Man Yi Wai
- Hong Kong Sheng Kung Hui Welfare Council Ltd, Hong Kong.
| | - Lai Lok Chan
- Hong Kong Sheng Kung Hui Welfare Council Ltd, Hong Kong.
| |
Collapse
|
14
|
Bernardet U, Fdili Alaoui S, Studd K, Bradley K, Pasquier P, Schiphorst T. Assessing the reliability of the Laban Movement Analysis system. PLoS One 2019; 14:e0218179. [PMID: 31194794 PMCID: PMC6564005 DOI: 10.1371/journal.pone.0218179] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2018] [Accepted: 05/28/2019] [Indexed: 11/18/2022] Open
Abstract
The Laban Movement Analysis system (LMA) is a widely used system for the description of human movement. Here we present results of an empirical analysis of the reliability of the LMA system. Firstly, we developed a directed graph-based representation for the formalization of LMA. Secondly, we implemented a custom video annotation tool for stimulus presentation and annotation of the formalized LMA. Using these two elements, we conducted an experimental assessment of LMA reliability. In the experimental assessment of the reliability, experts-Certified Movement Analysts (CMA)-were tasked with identifying the differences between a "neutral" movement and the same movement executed with a specific variation in one of the dimensions of the LMA parameter space. The videos represented variations on the pantomimed movement of knocking at a door or giving directions. To be as close as possible to the annotation practice of CMAs, participants were given full control over the number of times and order in which they viewed the videos. The LMA annotation was captured by means of the video annotation tool that guided the participants through the LMA graph by asking them multiple-choice questions at each node. Participants were asked to first annotate the most salient difference (round 1), and then the second most salient one (round 2) between a neutral and gesture and the variation. To quantify the overall reliability of LMA, we computed Krippendorff's α. The quantitative data shows that the reliability, depending on how the two rounds are integrated, ranges between a weak and an acceptable reliability of LMA. The analysis of viewing behavior showed that, despite relatively large differences at the inter-individual level, there is no simple relationship between viewing behavior and individual performance (quantified as the level of agreement of the individual with the dominant rating). This research advances the state of the art in formalizing and implementing a reliability measure for the Laban Movement Analysis system. The experimental study we conducted allows identifying some of the strengths and weaknesses of the widely used movement coding system. Additionally, we have gained useful insights into the assessment procedure itself.
Collapse
Affiliation(s)
- Ulysses Bernardet
- School of Engineering and Applied Science, Aston University, Birmingham, United Kingdom
- * E-mail:
| | - Sarah Fdili Alaoui
- Université Paris-Sud, CNRS, Inria, Université Paris-Saclay, Orsay, France
| | - Karen Studd
- Laban/Bartenieff Institute of Movement Studies, New York, New York, United States of America
| | - Karen Bradley
- University of Maryland, College Park, Maryland, United States of America
| | - Philippe Pasquier
- School of Interactive Arts and Technology, Simon Fraser University, Vancouver, Canada
| | - Thecla Schiphorst
- School of Interactive Arts and Technology, Simon Fraser University, Vancouver, Canada
| |
Collapse
|
15
|
Tsachor RP, Shafir T. How Shall I Count the Ways? A Method for Quantifying the Qualitative Aspects of Unscripted Movement With Laban Movement Analysis. Front Psychol 2019; 10:572. [PMID: 31001158 PMCID: PMC6455080 DOI: 10.3389/fpsyg.2019.00572] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2018] [Accepted: 02/28/2019] [Indexed: 12/30/2022] Open
Abstract
There is significant clinical evidence showing that creative and expressive movement processes involved in dance/movement therapy (DMT) enhance psycho-social well-being. Yet, because movement is a complex phenomenon, statistically validating which aspects of movement change during interventions or lead to significant positive therapeutic outcomes is challenging because movement has multiple, overlapping variables appearing in unique patterns in different individuals and situations. One factor contributing to the therapeutic effects of DMT is movement's effect on clients' emotional states. Our previous study identified sets of movement variables which, when executed, enhanced specific emotions. In this paper, we describe how we selected movement variables for statistical analysis in that study, using a multi-stage methodology to identify, reduce, code, and quantify the multitude of variables present in unscripted movement. We suggest a set of procedures for using Laban Movement Analysis (LMA)-described movement variables as research data. Our study used LMA, an internationally accepted comprehensive system for movement analysis, and a primary DMT clinical assessment tool for describing movement. We began with Davis's (1970) three-stepped protocol for analyzing movement patterns and identifying the most important variables: (1) We repeatedly observed video samples of validated (Atkinson et al., 2004) emotional expressions to identify prevalent movement variables, eliminating variables appearing minimally or absent. (2) We use the criteria repetition, frequency, duration and emphasis to eliminate additional variables. (3) For each emotion, we analyzed motor expression variations to discover how variables cluster: first, by observing ten movement samples of each emotion to identify variables common to all samples; second, by qualitative analysis of the two best-recognized samples to determine if phrasing, duration or relationship among variables was significant. We added three new steps to this protocol: (4) we created Motifs (LMA symbols) combining movement variables extracted in steps 1-3; (5) we asked participants in the pilot study to move these combinations and quantify their emotional experience. Based on the results of the pilot study, we eliminated more variables; (6) we quantified the remaining variables' prevalence in each Motif for statistical analysis that examined which variables enhanced each emotion. We posit that our method successfully quantified unscripted movement data for statistical analysis.
Collapse
Affiliation(s)
| | - Tal Shafir
- The Emili Sagol Creative Arts Therapies Research Center, University of Haifa, Haifa, Israel
- Department of Psychiatry, University of Michigan, Ann Arbor, MI, United States
| |
Collapse
|
16
|
Pino O, Palestra G, Trevino R, De Carolis B. The Humanoid Robot NAO as Trainer in a Memory Program for Elderly People with Mild Cognitive Impairment. Int J Soc Robot 2019. [DOI: 10.1007/s12369-019-00533-y] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/01/2023]
|
17
|
Development and Preliminary Investigation of a Semiautonomous Socially Assistive Robot (SAR) Designed to Elicit Communication, Motor Skills, Emotion, and Visual Regard (Engagement) from Young Children with Complex Cerebral Palsy: A Pilot Comparative Trial. ADVANCES IN HUMAN-COMPUTER INTERACTION 2019. [DOI: 10.1155/2019/2614060] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
Through play, typically developing children manipulate objects and interact with peers to establish and develop physical, cognitive, language, and social skills. However, children with complex disabilities and/or developmental delays have limited play experiences, thus compromising the quality of play and acquisition of skills. Assistive technologies have been developed to increase opportunities and level of interaction for children with disabilities to facilitate learning and development. One type of technology, Socially Assistive Robotics, is designed to assist the human user through social interaction while creating measurable growth in learning and rehabilitation. The investigators in this study designed, developed, and validated a semiautonomous Socially Assistive Robot to compare with a switch-adapted toy to determine robot effectiveness in quantity of, changes in, and differences in engagement. After interacting with both systems for three sessions each, five of the eight subjects showed a greater level of positive engagement with the robot than the switch-adapted toy, while the remaining three subjects showed slightly higher positive engagement with the toy. The preliminary results of the study suggest that Socially Assistive Robots specifically designed for children with complex cerebral palsy should be further researched and utilized to enrich play interactions and skill development for this population.
Collapse
|