Coordinated crawling via reinforcement learning

A 3D-Printed Self-Learning Three-Linked-Sphere Robot for Autonomous Confined-Space Navigation

Reinforcement learning control methods can impart robots with the ability to discover effective behavior, reducing their modeling and sensing requirements, and enabling their ability to adapt to environmental changes. However, it remains challenging for a robot to achieve navigation in confined and dynamic environments, which are characteristic of a broad range of biomedical applications, such as endoscopy with ingestible electronics. Herein, a compact, 3D-printed three-linked-sphere robot synergistically integrated with a reinforcement learning algorithm that can perform adaptable, autonomous crawling in a confined channel is demonstrated. The scalable robot consists of three equally sized spheres that are linearly coupled, in which the extension and contraction in specific sequences dictate its navigation. The ability to achieve bidirectional locomotion across frictional surfaces in open and confined spaces without prior knowledge of the environment is also demonstrated. The synergistic integration of a highly scalable robotic apparatus and the model-free reinforcement learning control strategy can enable autonomous navigation in a broad range of dynamic and confined environments. This capability can enable sensing, imaging, and surgical processes in previously inaccessible confined environments in the human body.