Optimization of criterion for objective evaluation of HRI performance that approximates subjective evaluation: a case study in robot competition

A Methodology to Design and Evaluate HRI Teaming Tasks in Robotic Competitions

As social robots become more prominent in our lives, their interaction with humans takes an increasing role, and new collaborative scenarios emerge. This development brings the need to realize robust test methods enabling the design and evaluation of HRI teaming tasks to prove functionality and promote adoption. In this paper, we present a general purpose and repeatable methodology for conducting studies in collaborative HRI in the range of robotic competitions. The methodology includes a step-by-step approach to design HRI teaming tasks tailored to be enacted in a robotic competition and to evaluate the performance of social robots to execute the designed tasks, exploring the relationship between robots’ performance and user perceptions based on the feedback of the users participating to such tasks. We assess the feasibility of the methodology to design and evaluate a HRI teaming task in the context of SciRoc (“Smart CIties RObotics Challenges”) competition, which targets at investigating the impact of social of robots in smart cities.

SIGVerse: A Cloud-Based VR Platform for Research on Multimodal Human-Robot Interaction

Research on Human-Robot Interaction (HRI) requires the substantial consideration of an experimental design, as well as a significant amount of time to practice the subject experiment. Recent technology in virtual reality (VR) can potentially address these time and effort challenges. The significant advantages of VR systems for HRI are: 1) cost reduction, as experimental facilities are not required in a real environment; 2) provision of the same environmental and embodied interaction conditions to test subjects; 3) visualization of arbitrary information and situations that cannot occur in reality, such as playback of past experiences, and 4) ease of access to an immersive and natural interface for robot/avatar teleoperations. Although VR tools with their features have been applied and developed in previous HRI research, all-encompassing tools or frameworks remain unavailable. In particular, the benefits of integration with cloud computing have not been comprehensively considered. Hence, the purpose of this study is to propose a research platform that can comprehensively provide the elements required for HRI research by integrating VR and cloud technologies. To realize a flexible and reusable system, we developed a real-time bridging mechanism between the robot operating system (ROS) and Unity. To confirm the feasibility of the system in a practical HRI scenario, we applied the proposed system to three case studies, including a robot competition named RoboCup@Home. via these case studies, we validated the system’s usefulness and its potential for the development and evaluation of social intelligence via multimodal HRI.