Visual Servoing of a Moving Target by an Unmanned Aerial Vehicle

To track moving targets undergoing unknown translational and rotational motions, a tracking controller is developed for unmanned aerial vehicles (UAVs). The main challenges are to control both the relative position and orientation between the target and the UAV to within desired values, and to guarantee that the generated control input to the UAV is feasible (i.e., below its motion capability). Moreover, the UAV is controlled to ensure that the target always remains within the field of view of the onboard camera. These control objectives were achieved by developing a nonlinear-model predictive controller, in which the future motion of the target is predicted by quadratic programming (QP). Since constraints of the feature vector and the control input are considered when solving the optimal control problem, the control inputs can be bounded and the target can remain inside the image. Three simulations were performed to compare the efficacy and performance of the developed controller with a traditional image-based visual servoing controller.

Visual Tracking and Depth Estimation of Mobile Robots Without Desired Velocity Information

In this paper, a visual servoing approach is developed for the trajectory tracking control and depth estimation problem of a mobile robot without a priori knowledge about desired velocities. By exploiting the multiple images captured by the on-board camera, the current and desired poses (i.e., scaled translation and orientation) of the mobile robot are reconstructed to define system errors. Then, an adaptive time-varying controller is proposed to achieve the trajectory tracking task in the presence of nonholonomic constraint and unknown depth parameters. Most of previous works require the measurement of the desired velocity information to facilitate the controller design, leading to tedious offline computation. In this paper, to eliminate this requirement, the desired velocities are estimated in real-time based on a reduced order observer. Moreover, an augmented update law is designed to compensate for the unknown depth parameters and identify the inverse depth constant. The Lyapunov-based method is employed to prove that the proposed controller achieves asymptotic tracking, and the inverse depth estimate converges to its actual value provided that a persistent excitation condition is satisfied. Subsequently, a robust data-driven algorithm is introduced to ensure the convergence of the inverse depth estimate under a relaxed finite excitation condition. Simulation and experimental results are provided to demonstrate the effectiveness of the proposed approach.

Globally stable velocity estimation using normalized velocity measurement

The problem of estimating velocity from a monocular camera and calibrated inertial measurement unit (IMU) measurements is revisited. For the presented setup, it is assumed that normalized velocity measurements are available from the camera. By applying results from nonlinear observer theory, we present velocity estimators with proven global stability under defined conditions, and without the need to observe features from several camera frames. Several nonlinear methods are compared with each other, also against an extended Kalman filter (EKF), where the robustness of the nonlinear methods compared with the EKF are demonstrated in simulations and experiments.

Unfalsified Visual Servoing for Simultaneous Object Recognition and Pose Tracking

In a complex environment, simultaneous object recognition and tracking has been one of the challenging topics in computer vision and robotics. Current approaches are usually fragile due to spurious feature matching and local convergence for pose determination. Once a failure happens, these approaches lack a mechanism to recover automatically. In this paper, data-driven unfalsified control is proposed for solving this problem in visual servoing. It recognizes a target through matching image features with a 3-D model and then tracks them through dynamic visual servoing. The features can be falsified or unfalsified by a supervisory mechanism according to their tracking performance. Supervisory visual servoing is repeated until a consensus between the model and the selected features is reached, so that model recognition and object tracking are accomplished. Experiments show the effectiveness and robustness of the proposed algorithm to deal with matching and tracking failures caused by various disturbances, such as fast motion, occlusions, and illumination variation.

Nonlinear ego-motion estimation from optical flow for online control of a quadrotor UAV

For the control of unmanned aerial vehicles (UAVs) in GPS-denied environments, cameras have been widely exploited as the main sensory modality for addressing the UAV state estimation problem. However, the use of visual information for ego-motion estimation presents several theoretical and practical difficulties, such as data association, occlusions, and lack of direct metric information when exploiting monocular cameras. In this paper, we address these issues by considering a quadrotor UAV equipped with an onboard monocular camera and an inertial measurement unit (IMU). First, we propose a robust ego-motion estimation algorithm for recovering the UAV scaled linear velocity and angular velocity from optical flow by exploiting the so-called continuous homography constraint in the presence of planar scenes. Then, we address the problem of retrieving the (unknown) metric scale by fusing the visual information with measurements from the onboard IMU. To this end, two different estimation strategies are proposed and critically compared: a first exploiting the classical extended Kalman filter (EKF) formulation, and a second one based on a novel nonlinear estimation framework. The main advantage of the latter scheme lies in the possibility of imposing a desired transient response to the estimation error when the camera moves with a constant acceleration norm with respect to the observed plane. We indeed show that, when compared against the EKF on the same trajectory and sensory data, the nonlinear scheme yields considerably superior performance in terms of convergence rate and predictability of the estimation. The paper is then concluded by an extensive experimental validation, including an onboard closed-loop control of a real quadrotor UAV meant to demonstrate the robustness of our approach in real-world conditions.