Using Inverse Reinforcement Learning with Real Trajectories to Get More Trustworthy Pedestrian Simulations

Introducing CARESSER: A framework for in situ learning robot social assistance from expert knowledge and demonstrations

Socially assistive robots have the potential to augment and enhance therapist's effectiveness in repetitive tasks such as cognitive therapies. However, their contribution has generally been limited as domain experts have not been fully involved in the entire pipeline of the design process as well as in the automatisation of the robots' behaviour. In this article, we present aCtive leARning agEnt aSsiStive bEhaviouR (CARESSER), a novel framework that actively learns robotic assistive behaviour by leveraging the therapist's expertise (knowledge-driven approach) and their demonstrations (data-driven approach). By exploiting that hybrid approach, the presented method enables in situ fast learning, in a fully autonomous fashion, of personalised patient-specific policies. With the purpose of evaluating our framework, we conducted two user studies in a daily care centre in which older adults affected by mild dementia and mild cognitive impairment (N = 22) were requested to solve cognitive exercises with the support of a therapist and later on of a robot endowed with CARESSER. Results showed that: (i) the robot managed to keep the patients' performance stable during the sessions even more so than the therapist; (ii) the assistance offered by the robot during the sessions eventually matched the therapist's preferences. We conclude that CARESSER, with its stakeholder-centric design, can pave the way to new AI approaches that learn by leveraging human-human interactions along with human expertise, which has the benefits of speeding up the learning process, eliminating the need for the design of complex reward functions, and finally avoiding undesired states.

Modeling pedestrian behavior in pedestrian-vehicle near misses: A continuous Gaussian Process Inverse Reinforcement Learning (GP-IRL) approach

Using simulation models to conduct safety assessments can have several advantages as it enables the evaluation of the safety of various design and traffic management options before actually making changes. However, limited studies have developed microsimulation models for the safety evaluation of active road users such as pedestrians. This can be attributed to the limited ability of simulation models to capture the heterogeneity in pedestrian behavior and their complex collision avoidance mechanisms. Therefore, the objective of this study is to develop an agent-based framework to realistically model pedestrian behavior in near misses and to improve the understanding of pedestrian evasive action mechanisms in interactions with vehicles. Pedestrian-vehicle conflicts are modeled using the Markov Decision Process (MDP) framework. A continuous Gaussian Process Inverse Reinforcement Learning (GP-IRL) approach is implemented to retrieve pedestrians' reward functions and infer their collision avoidance mechanisms in conflict situations. Video data from a congested intersection in Shanghai, China is used as a case study. Trajectories of pedestrians and vehicles involved in traffic conflicts were extracted with computer vision algorithms. A Deep Reinforcement Learning (DRL) model is used to estimate optimal pedestrian policies in traffic conflicts. Results show that the developed model predicted pedestrian trajectories and their evasive action mechanisms (i.e., swerving maneuver and speed changing) in conflict situations with high accuracy. As well, the model provided predictions of the post encroachment time (PET) conflict indicator that strongly correlated with the corresponding values of the field-measured conflicts. This study is a crucial step in developing a safety-oriented microsimulation tool for pedestrians in mixed traffic conditions.