Hybrid coordination for the fast formation building of multi-small-AUV systems with the on-board cameras and limited communication

Formation building for multi-small-AUV systems with on-board cameras is crucial under the limited communication underwater environment. A hybrid coordination strategy is proposed for the rapid convergence to a leader-follower pattern. The strategy consists of two parts: a time-optimal local-position-based controller (TOLC) and a distributed asynchronous discrete weighted consensus controller (ADWCC). The TOLC controller is designed to optimize the assignation of AUVs' destinations in the given pattern and guide each AUV to its destination by the shortest feasible distance. The ADWCC controller is developed to direct the AUVs blocked by obstacles to reach their destinations with the information from the perceived neighbors by on-board cameras. The rapidity of the proposed strategy is theoretically discussed. The effectiveness of the proposed algorithm has been verified in the simulation environments in both MATLAB and Blender.

The Deep Convolutional Neural Network Role in the Autonomous Navigation of Mobile Robots (SROBO)

The ability to navigate unstructured environments is an essential task for intelligent systems. Autonomous navigation by ground vehicles requires developing an internal representation of space, trained by recognizable landmarks, robust visual processing, computer vision and image processing. A mobile robot needs a platform enabling it to operate in an environment autonomously, recognize the objects, and avoid obstacles in its path. In this study, an open-source ground robot called SROBO was designed to accurately identify its position and navigate certain areas using a deep convolutional neural network and transfer learning. The framework uses an RGB-D MYNTEYE camera, a 2D laser scanner and inertial measurement units (IMU) operating through an embedded system capable of deep learning. The real-time decision-making process and experiments were conducted while the onboard signal processing and image capturing system enabled continuous information analysis. State-of-the-art Real-Time Graph-Based SLAM (RTAB-Map) was adopted to create a map of indoor environments while benefiting from deep convolutional neural network (Deep-CNN) capability. Enforcing Deep-CNN improved the performance quality of the RTAB-Map SLAM. The proposed setting equipped the robot with more insight into its surroundings. The robustness of the SROBO increased by 35% using the proposed system compared to the conventional RTAB-Map SLAM.

Bearing-Only Formation Control With Prespecified Convergence Time

This article considers the bearing-only formation control problem, where the control of each agent only relies on relative bearings of their neighbors. A new control law is proposed to achieve target formations in finite time. Different from the existing results, the control law is based on a time-varying scaling gain. Hence, the convergence time can be arbitrarily chosen by users, and the derivative of the control input is continuous. Furthermore, sufficient conditions are given to guarantee almost global convergence and interagent collision avoidance. Then, a leader-follower control structure is proposed to achieve global convergence. By exploring the properties of the bearing Laplacian matrix, the collision avoidance and smooth control input are preserved. A multirobot hardware platform is designed to validate the theoretical results. Both simulation and experimental results demonstrate the effectiveness of our design.

Estimation and Control of Cooperative Aerial Manipulators for a Payload with an Arbitrary Center-of-Mass

This paper presents an integrated framework that integrates the kinematic and dynamic parameter estimation of an irregular object with non-uniform mass distribution for cooperative aerial manipulators. Unlike existing approaches, including impedance-based control which requires expensive force/torque sensors or the first-order-momentum-based estimator which is weak to noise, this paper suggests a method without such sensor and strong to noise by exploiting the decentralized dynamics and sliding-mode-momentum observer. First, the kinematic estimator estimates the relative distances of multiple aerial manipulators by using translational and angular velocities between aerial robots. By exploiting the distance estimation, the desired trajectories for each aerial manipulator are set. Second, the dynamic parameter estimation is performed for the mass of the common object and the vector between the end-effector frame and the center of mass of the object. Finally, the proposed framework is validated with simulations using aerial manipulators combined with two degrees-of-freedom robotic arms using a noisy measurement. Throughout the simulation, we can decrease the mass estimation error by 60% compared to the existing first-order momentum-based method. In addition, a comparison study shows that the proposed method satisfactorily estimates an arbitrary center-of-mass of an unknown payload in noisy environments.

Multiple View Relations Using the Teaching and Learning-Based Optimization Algorithm

In computer vision, estimating geometric relations between two different views of the same scene has great importance due to its applications in 3D reconstruction, object recognition and digitization, image registration, pose retrieval, visual tracking and more. The Random Sample Consensus (RANSAC) is the most popular heuristic technique to tackle this problem. However, RANSAC-like algorithms present a drawback regarding either the tuning of the number of samples and the threshold error or the computational burden. To relief this problem, we propose an estimator based on a metaheuristic, the Teaching–Learning-Based Optimization algorithm (TLBO) that is motivated by the teaching–learning process. We use the TLBO algorithm in the problem of computing multiple view relations given by the homography and the fundamental matrix. To improve the method, candidate models are better evaluated with a more precise objective function. To validate the efficacy of the proposed approach, several tests, and comparisons with two RANSAC-based algorithms and other metaheuristic-based estimators were executed.

Global Vision-Based Formation Control of Soft Robotic Fish Swarm

Possessing the attributes of high adaptability and low cost, soft robotic individuals can further coordinate and form into a swarming system, enhancing the performances as well as functions in practical applications. However, the formation control of soft robotic swarm remains challenging mainly due to the limitation in relatively low precision and slow response of the soft actuators. In this work, a soft robotic fish swarm system with global vision positioning was studied. The soft robotic fish used in the project is driven by a hybrid power-control system, in which the soft dielectric elastomers and the rigid electrical servo provide forward propulsion and controllable steering function, respectively. Results show that soft robotic fish swarm can quickly shift their formations, mimicking three typical swarming behaviors of natural creatures: highly parallel group, encircling, and torus. The system design and controlling principles of the soft robotic fish swarm may guide the future research of soft robots and robotic swarms, specifically for underwater applications.

Distributed and consistent multi-image feature matching via QuickMatch

In this work, we consider the multi-image object matching problem in distributed networks of robots. Multi-image feature matching is a keystone of many applications, including Simultaneous Localization and Mapping, homography, object detection, and Structure from Motion. We first review the QuickMatch algorithm for multi-image feature matching. We then present NetMatch, an algorithm for distributing sets of features across computational units (agents) that largely preserves feature match quality and minimizes communication between agents (avoiding, in particular, the need to flood all data to all agents). Finally, we present an experimental application of both QuickMatch and NetMatch on an object matching test with low-quality images. The QuickMatch and NetMatch algorithms are compared with other standard matching algorithms in terms of preservation of match consistency. Our experiments show that QuickMatch and Netmatch can scale to larger numbers of images and features, and match more accurately than standard techniques.

A Survey on Swarming With Micro Air Vehicles: Fundamental Challenges and Constraints

This work presents a review and discussion of the challenges that must be solved in order to successfully develop swarms of Micro Air Vehicles (MAVs) for real world operations. From the discussion, we extract constraints and links that relate the local level MAV capabilities to the global operations of the swarm. These should be taken into account when designing swarm behaviors in order to maximize the utility of the group. At the lowest level, each MAV should operate safely. Robustness is often hailed as a pillar of swarm robotics, and a minimum level of local reliability is needed for it to propagate to the global level. An MAV must be capable of autonomous navigation within an environment with sufficient trustworthiness before the system can be scaled up. Once the operations of the single MAV are sufficiently secured for a task, the subsequent challenge is to allow the MAVs to sense one another within a neighborhood of interest. Relative localization of neighbors is a fundamental part of self-organizing robotic systems, enabling behaviors ranging from basic relative collision avoidance to higher level coordination. This ability, at times taken for granted, also must be sufficiently reliable. Moreover, herein lies a constraint: the design choice of the relative localization sensor has a direct link to the behaviors that the swarm can (and should) perform. Vision-based systems, for instance, force MAVs to fly within the field of view of their camera. Range or communication-based solutions, alternatively, provide omni-directional relative localization, yet can be victim to unobservable conditions under certain flight behaviors, such as parallel flight, and require constant relative excitation. At the swarm level, the final outcome is thus intrinsically influenced by the on-board abilities and sensors of the individual. The real-world behavior and operations of an MAV swarm intrinsically follow in a bottom-up fashion as a result of the local level limitations in cognition, relative knowledge, communication, power, and safety. Taking these local limitations into account when designing a global swarm behavior is key in order to take full advantage of the system, enabling local limitations to become true strengths of the swarm.

Distributed nonlinear H∞ control algorithm for multi-agent quadrotor formation flying

A novel distributed formation control algorithm for nonlinear uncertain multi-agent systems is proposed in this paper. The algorithm is developed based on a nonlinear H_∞ framework for a team of aerial robots with nonlinear dynamics and bounded parametric uncertainties. The multi-agent control algorithm reduces to local control laws dependent on agents information and their neighbors data. The proof of stability is presented analytically. Simulation results for two predefined scenarios verify the robust performance of the algorithm in the presence of external disturbances and bounded uncertainties, numerically. Additionally, the effects of communication network configuration are studied using numerical simulations. Calculated ISE and IADU indexes show the effectiveness of the distributed algorithm compared to independent performance of the agents.

Enhanced Positioning Algorithm Using a Single Image in an LCD-Camera System by Mesh Elements' Recalculation and Angle Error Orientation

In this article, we present a method to position the tool in a micromachine system based on a camera-LCD screen positioning system that also provides information about angular deviations of the tool axis during its running. Both position and angular deviations are obtained by reducing a matrix of LEDs in the image to a single rectangle in the conical perspective that is treated by a photogrammetry method. This method computes the coordinates and orientation of the camera with respect to the fixed screen coordinate system. The used image consists of 5 × 5 lit LEDs, which are analyzed by the algorithm to determine a rectangle with known dimensions. The coordinates of the vertices of the rectangle in space are obtained by an inverse perspective computation from the image. The method presents a good approximation of the central point of the rectangle and provides the inclination of the workpiece with respect to the LCD screen reference system of coordinates. A test of the method is designed with the assistance of a Coordinate Measurement Machine (CMM) to check the accuracy of the positioning method. The performed test delivers a good accuracy in the position measurement of the designed method. A high dispersion in the angular deviation is detected, although the orientation of the inclination is appropriate in almost every case. This is due to the small values of the angles that makes the trigonometric function approximations very erratic. This method is a good starting point for the compensation of angular deviation in vision based micromachine tools, which is the principal source of errors in these operations and represents the main volume in the cost of machine elements' parts.

Leader Selection via Supermodular Game for Formation Control in Multiagent Systems

Multiagent systems (MASs) are usually applied with agents classified into leaders and followers, where selecting appropriate leaders is an important issue for formation control applications. In this paper, we investigate two leader selection problems in second-order MAS, namely, the problem of choosing up to a given number of leaders to minimize the formation error and the problem of choosing the minimum number of leaders to achieve a tolerated level of error. We propose a game theoretical method to address them. Specifically, we design a supermodular game for the leader selection problems and theoretically prove its supermodularity. In order to reach Nash equilibrium of the game, we propose strategies for the agents to learn to select leaders based on stochastic fictitious play. Extensive simulation results demonstrate that our method outperforms existing ones.

Modeling and Flight Experiments for Swarms of High Dynamic UAVs: A Stochastic Configuration Control System with Multiplicative Noises

UAV Swarm with high dynamic configuration at a large scale requires a high-precision mathematical model to fully exploit its boundary performance. In order to instruct the engineering application with high confidence, uncertainties induced from either systematic measurement or the environment cannot be ignored. This paper investigates the I t o ^ stochastic model of the UAV Swarm system with multiplicative noises. By combining the cooperative kinematic model with a simplified individual dynamic model of fixed-wing-aircraft for the first time, the configuration control model is derived. Considering the uncertainties in actual flight, multiplicative noises are introduced to complete the I t o ^ stochastic model. Following that, the estimator and controller are designed to control the formation. The mean-square uniform boundedness condition of the proposed stochastic system is presented for the closed-loop system. In the simulation, the stochastic robustness analysis and design (SRAD) method is used to optimize the properties of the formation. More importantly, the effectiveness of the proposed model is also verified using real data of five unmanned aircrafts collected in outfield formation flight experiments.

Control in belief space with temporal logic specifications using vision-based localization

We present a solution for operating a vehicle without global positioning infrastructure while satisfying constraints on its temporal behavior, and on the uncertainty of its position estimate. The proposed solution is an end-to-end framework for mapping an unknown environment using aerial vehicles, synthesizing a control policy for a ground vehicle in that environment, and using a quadrotor to localize the ground vehicle within the map while it executes its control policy. This vision-based localization is noisy, necessitating planning in the belief space of the ground robot. The ground robot’s mission is given using a language called Gaussian Distribution Temporal Logic (GDTL), an extension of Boolean logic that incorporates temporal evolution and noise mitigation directly into the task specifications. We use a sampling-based algorithm to generate a transition system in the belief space and use local feedback controllers to break the curse of history associated with belief space planning. To localize the vehicle, we build a high-resolution map of the environment by flying a team of aerial vehicles in formation with sensor information provided by their onboard cameras. The control policy for the ground robot is synthesized under temporal and uncertainty constraints given the semantically labeled map. Then the ground robot can execute the control policy given pose estimates from a dedicated aerial robot that tracks and localizes the ground robot. The proposed method is validated using two quadrotors to build a map, followed by a two-wheeled ground robot and a quadrotor with a camera for ten successful experimental trials.