End-to-end decentralized formation control using a graph neural network-based learning method

Multi-robot cooperative control has been extensively studied using model-based distributed control methods. However, such control methods rely on sensing and perception modules in a sequential pipeline design, and the separation of perception and controls may cause processing latencies and compounding errors that affect control performance. End-to-end learning overcomes this limitation by implementing direct learning from onboard sensing data, with control commands output to the robots. Challenges exist in end-to-end learning for multi-robot cooperative control, and previous results are not scalable. We propose in this article a novel decentralized cooperative control method for multi-robot formations using deep neural networks, in which inter-robot communication is modeled by a graph neural network (GNN). Our method takes LiDAR sensor data as input, and the control policy is learned from demonstrations that are provided by an expert controller for decentralized formation control. Although it is trained with a fixed number of robots, the learned control policy is scalable. Evaluation in a robot simulator demonstrates the triangular formation behavior of multi-robot teams of different sizes under the learned control policy.

Learning Skill Training Schedules From Domain Experts for a Multi-Patient Multi-Robot Rehabilitation Gym

A robotic gym with multiple rehabilitation robots allows multiple patients to exercise simultaneously under the supervision of a single therapist. The multi-patient training outcome can potentially be improved by dynamically assigning patients to robots based on monitored patient data. In this paper, we present an approach to learn dynamic patient-robot assignment from a domain expert via supervised learning. The dynamic assignment algorithm uses a neural network model to predict assignment priorities between patients. This neural network was trained using a synthetic dataset created in a simulated rehabilitation gym to imitate a domain expert's assignment behavior. The approach is evaluated in three simulated scenarios with different complexities and different expert behaviors meant to achieve different training objectives. Evaluation results show that our assignment algorithm imitates the expert's behavior with mean accuracies ranging from 75.4% to 84.5% across scenarios and significantly outperforms three baseline assignment methods with respect to mean skill gain. Our approach solves simplified patient training scheduling problems without complete knowledge about the patient skill acquisition dynamics and leverages human knowledge to learn automated assignment policies.

Multi-Agent Deep Reinforcement Learning for Multi-Robot Applications: A Survey

Deep reinforcement learning has produced many success stories in recent years. Some example fields in which these successes have taken place include mathematics, games, health care, and robotics. In this paper, we are especially interested in multi-agent deep reinforcement learning, where multiple agents present in the environment not only learn from their own experiences but also from each other and its applications in multi-robot systems. In many real-world scenarios, one robot might not be enough to complete the given task on its own, and, therefore, we might need to deploy multiple robots who work together towards a common global objective of finishing the task. Although multi-agent deep reinforcement learning and its applications in multi-robot systems are of tremendous significance from theoretical and applied standpoints, the latest survey in this domain dates to 2004 albeit for traditional learning applications as deep reinforcement learning was not invented. We classify the reviewed papers in our survey primarily based on their multi-robot applications. Our survey also discusses a few challenges that the current research in this domain faces and provides a potential list of future applications involving multi-robot systems that can benefit from advances in multi-agent deep reinforcement learning.