Bilateral teleoperation of a group of mobile robots for cooperative tasks

Extended Control With Hybrid Gaze-BCI for Multi-Robot System Under Hands-Occupied Dual-Tasking

Currently there still remains a critical need of human involvements for multi-robot system (MRS) to successfully perform their missions in real-world applications, and the hand-controller has been commonly used for the operator to input MRS control commands. However, in more challenging scenarios involving concurrent MRS control and system monitoring tasks, where the operator's both hands are busy, the hand-controller alone is inadequate for effective human-MRS interaction. To this end, our study takes a first step toward a multimodal interface by extending the hand-controller with a hands-free input based on gaze and brain-computer interface (BCI), i.e., a hybrid gaze-BCI. Specifically, the velocity control function is still designated to the hand-controller that excels at inputting continuous velocity commands for MRS, while the formation control function is realized with a more intuitive hybrid gaze-BCI, rather than with the hand-controller via a less natural mapping. In a dual-task experimental paradigm that simulated the hands-occupied manipulation condition in real-world applications, operators achieved improved performance for controlling simulated MRS (average formation inputting accuracy increases 3%, average finishing time decreases 5 s), reduced cognitive load (average reaction time for secondary task decreases 0.32 s) and perceived workload (average rating score decreases 15.84) with the hand-controller extended by the hybrid gaze-BCI, over those with the hand-controller alone. These findings reveal the potential of the hands-free hybrid gaze-BCI to extend the traditional manual MRS input devices for creating a more operator-friendly interface, in challenging hands-occupied dual-tasking scenarios.

A Novel Cooperative Teleoperation Framework for Nonlinear Time-Delayed Single-Master/Multi-Slave System

This paper proposes a novel control framework for a single-master/multi-slave teleoperation system to grasp and handle an object, considering nonlinearity and uncertainty in the dynamics of the slaves and time-varying delay in the communication channel. Based on passive decomposition approach, the dynamics of the slaves are decomposed into two decoupled systems, and then two higher order sliding mode controllers are designed to control them. Also, a synchronization control methodology for the master is developed. Stability is fully studied using the passivity property and Lyapunov theorem. Finally, simulation and practical results confirm that the control system works well against the conditions.