Project INSIDE: towards autonomous semi-unstructured human-robot social interaction in autism therapy

The Need for Technology-Aided Instruction and Intervention Policy for Autistic Youth

This article examines current technology-aided instruction and intervention (TAII) available for autistic transition-age youth (TAY) and existing policies that may support or hinder the delivery of these interventions. Specifically, we focus on policies that might influence the delivery of TAII to autistic TAY. After a careful review of the literature, we observed that postsecondary policy guiding the delivery of TAII designed to support autistic TAY is lacking. TAII has demonstrated effectiveness, usability, sustainability, and cost-effectiveness, particularly with this population. We suggest possibilities for future policies to support the development, implementation, and evaluation of TAII for autistic TAY.

Fully robotic social environment for teaching and practicing affective interaction: Case of teaching emotion recognition skills to children with autism spectrum disorder, a pilot study

21st century brought along a considerable decrease in social interactions, due to the newly emerged lifestyle around the world, which became more noticeable recently of the COVID-19 pandemic. On the other hand, children with autism spectrum disorder have further complications regarding their social interactions with other humans. In this paper, a fully Robotic Social Environment (RSE), designed to simulate the needed social environment for children, especially those with autism is described. An RSE can be used to simulate many social situations, such as affective interpersonal interactions, in which observational learning can take place. In order to investigate the effectiveness of the proposed RSE, it has been tested on a group of children with autism, who had difficulties in emotion recognition, which in turn, can influence social interaction. An A-B-A single case study was designed to show how RSE can help children with autism recognize four basic facial expressions, i.e., happiness, sadness, anger, and fear, through observing the social interactions of two robots speaking about these facial expressions. The results showed that the emotion recognition skills of the participating children were improved. Furthermore, the results showed that the children could maintain and generalize their emotion recognition skills after the intervention period. In conclusion, the study shows that the proposed RSE, along with other rehabilitation methods, can be effective in improving the emotion recognition skills of children with autism and preparing them to enter human social environments.

Design of an Incremental Music Teaching and Assisted Therapy System Based on Artificial Intelligence Attention Mechanism

With the continuous updating and advancement of artificial intelligence technology, it gradually begins to shine in various industries, especially playing an increasingly important role in incremental music teaching and assisted therapy systems. This study designs artificial intelligence models from the perspectives of attention mechanism, contextual information guidance, and distant dependencies combined with incremental music teaching for the segmentation of MS (multiple sclerosis) lesions and achieves the automatic and accurate segmentation of MS lesions through the multidimensional analysis of multimodal magnetic resonance imaging data, which provides a basis for physicians to quantitatively analyze MS lesions, thus assisting them in the diagnosis and treatment of MS. To address the highly variable characteristics of MS lesion location, size, number, and shape, this paper firstly designs a 3D context-guided module based on Kronecker convolution to integrate lesion information from different fields of view, starting from lesion contextual information capture. Then, a 3D spatial attention module is introduced to enhance the representation of lesion features in MRI images. The experiments in this paper confirm that the context-guided module, cross-dimensional cross-attention module, and multidimensional feature similarity module designed for the characteristics of MS lesions are effective, and the proposed attentional context U-Net and multidimensional cross-attention U-Net have greater advantages in the objective evaluation index of lesion segmentation, while being combined with the incremental music teaching approach to assist treatment, which provides a new idea for the intelligent assisted treatment approach. In this paper, from algorithm design to experimental validation, both in terms of accuracy, the operational difficulty of the experiment, consumption of arithmetic power, and time cost, the unique superiority of the artificial intelligence attention-based combined with incremental music teaching adjunctive therapy system proposed in this paper can be seen in the MS lesion segmentation task.

The Quantitative Case-by-Case Analyses of the Socio-Emotional Outcomes of Children with ASD in Robot-Assisted Autism Therapy

With its focus on robot-assisted autism therapy, this paper presents case-by-case analyses of socio-emotional outcomes of 34 children aged 3–12 years old, with different cases of Autism Spectrum Disorder (ASD) and Attention Deficit Hyperactivity Disorder (ADHD). We grouped children by the following characteristics: ASD alone (n = 22), ASD+ADHD (n = 12), verbal (n = 11), non-verbal (n = 23), low-functioning autism (n = 24), and high-functioning autism (n = 10). This paper provides a series of separate quantitative analyses across the first and last sessions, adaptive and non-adaptive sessions, and parent and no-parent sessions, to present child experiences with the NAO robot, during play-based activities. The results suggest that robots are able to interact with children in social ways and influence their social behaviors over time. Each child with ASD is a unique case and needs an individualized approach to practice and learn social skills with the robot. We, finally, present specific child–robot intricacies that affect how children engage and learn over time as well as across different sessions.

“Sequencing Matters”: Investigating Suitable Action Sequences in Robot-Assisted Autism Therapy

Social robots have been shown to be promising tools for delivering therapeutic tasks for children with Autism Spectrum Disorder (ASD). However, their efficacy is currently limited by a lack of flexibility of the robot’s social behavior to successfully meet therapeutic and interaction goals. Robot-assisted interventions are often based on structured tasks where the robot sequentially guides the child towards the task goal. Motivated by a need for personalization to accommodate a diverse set of children profiles, this paper investigates the effect of different robot action sequences in structured socially interactive tasks targeting attention skills in children with different ASD profiles. Based on an autism diagnostic tool, we devised a robotic prompting scheme on a NAO humanoid robot, aimed at eliciting goal behaviors from the child, and integrated it in a novel interactive storytelling scenario involving screens. We programmed the robot to operate in three different modes: diagnostic-inspired (Assess), personalized therapy-inspired (Therapy), and random (Explore). Our exploratory study with 11 young children with ASD highlights the usefulness and limitations of each mode according to different possible interaction goals, and paves the way towards more complex methods for balancing short-term and long-term goals in personalized robot-assisted therapy.

A Long-Term Engagement with a Social Robot for Autism Therapy

Social robots are increasingly being used as a mediator between a therapist and a child in autism therapy studies. In this context, most behavioural interventions are typically short-term in nature. This paper describes a long-term study that was conducted with 11 children diagnosed with either Autism Spectrum Disorder (ASD) or ASD in co-occurrence with Attention Deficit Hyperactivity Disorder (ADHD). It uses a quantitative analysis based on behavioural measures, including engagement, valence, and eye gaze duration. Each child interacted with a robot on several occasions in which each therapy session was customized to a child's reaction to robot behaviours. This paper presents a set of robot behaviours that were implemented with the goal to offer a variety of activities to be suitable for diverse forms of autism. Therefore, each child experienced an individualized robot-assisted therapy that was tailored according to the therapist's knowledge and judgement. The statistical analyses showed that the proposed therapy managed to sustain children's engagement. In addition, sessions containing familiar activities kept children more engaged compared to those sessions containing unfamiliar activities. The results of the interviews with parents and therapists are discussed in terms of therapy recommendations. The paper concludes with some reflections on the current study as well as suggestions for future studies.

Social Robots in Special Education: A Systematic Review

In recent years, social robots have become part of a variety of human activities, especially in applications involving children, e.g., entertainment, education, companionship. The interest of this work lies in the interaction of social robots with children in the field of special education. This paper seeks to present a systematic review of the use of robots in special education, with the ultimate goal of highlighting the degree of integration of robots in this field worldwide. This work aims to explore the technologies of robots that are applied according to the impairment type of children. The study showed a large number of attempts to apply social robots to the special education of children with various impairments, especially in recent years, as well as a wide variety of social robots from the market involved in such activities. The main conclusion of this work is the finding that the specific field of application of social robots is at the first development step; however, it is expected to be of great concern to the research community in the coming years.

A Systematic Review of Robotic Rehabilitation for Cognitive Training

A large and increasing number of people around the world experience cognitive disability. Rehabilitation robotics has provided promising training and assistance approaches to mitigate cognitive deficits. In this article, we carried out a systematic review on recent developments in robot-assisted cognitive training. We included 99 articles in this work and described their applications, enabling technologies, experiments, and products. We also conducted a meta analysis on the articles that evaluated robot-assisted cognitive training protocol with primary end users (i.e., people with cognitive disability). We identified major limitations in current robotics rehabilitation for cognitive training, including the small sample size, non-standard measurement of training and uncontrollable factors. There are still multifaceted challenges in this field, including ethical issues, user-centered (or stakeholder-centered) design, the reliability, trust, and cost-effectiveness, personalization of the robot-assisted cognitive training system. Future research shall also take into consideration human-robot collaboration and social cognition to facilitate a natural human-robot interaction.

How Technology Applied to Music-Therapy and Sound-Based Activities Addresses Motor and Social Skills in Autistic Children

Autism affects how people perceive and make sense of the world around them. Autism is a spectrum condition which impacts people in different ways. Also referred to as Autism Spectrum Disorder (ASD), it is characterized by challenges in the domains of social, cognitive and motor functioning, which differ in severity. Previous research suggests that music can have cognitive, psychosocial, behavioural, and motor benefits in this population. We systematically review the use of technology in Music-therapy and related sound-based activities to improve the motor and social skills of children. In May 2020 we conducted a systematic search on Music-therapy and musical activities for autistic children in research databases including Science Direct, APA PsycNet, Cochrane, IEE and Web of Science, to collect relevant studies. We initially collected 5179 papers of which only 27 studies were identified as suitable for the scope of this review. In the paper, we analyse and describe key characteristics of each project. We then highlight the commonalities, strengths and limitations of existing work, and identify implications for future interaction design.