Unlocking aerobatic potential of quadcopters: Autonomous freestyle flight generation and execution

Quadcopter drones are capable of executing complex aerobatic maneuvers when controlled manually by skilled pilots but are limited to simple aerobatic actions when flying autonomously in open spaces. As such, this study introduces a comprehensive system that enables drones to generate and execute sophisticated aerobatic maneuvers in complex environments with dense obstacle distributions. A universal representation is proposed, succinctly capturing flight as a series of discrete aerobatic intentions. These intentions consist of topology and attitude changes, which can be combined in various ways to describe intricate flight maneuvers. A spatial-temporal joint optimization trajectory planner is also introduced to generate dynamically feasible trajectories that are as smooth as possible and devoid of collisions. In addition, we investigate unique yaw sensitivity issues in aerobatic flight and identify the inherent influence of differential flatness singularities on yaw rotations while avoiding associated dynamics issues. A series of ablation studies confirmed the necessity of these spatial-temporal joint optimization and yaw compensation strategies. Additional simulations and physical experiments validated the stability and feasibility of our proposed system for improving uncrewed aerial flight. The proposed system enables drones to autonomously achieve flight performance usually reserved for professional pilots, unlocking boundless potential for aerobatic flight evolution in uncrewed aerial vehicles.

MineSim: A scenario-based simulation test system and benchmark for autonomous trucks in open-pit mines

Simulation environments are essential for validating algorithms, evaluating system performance, and ensuring safety in autonomous driving systems before real-world deployment. Existing autonomous driving simulators are designed for urban scenarios but lack coverage of unstructured road environments in open-pit mining. This paper introduces MineSim, an open-source, scenario-based simulation test system specifically developed for planning tasks in autonomous trucks operating in open-pit mines. MineSim includes several components: automated scenario parsing, state update models for the ego vehicle, state update policies for other agents, metric evaluation, and scenario visualization tools. It incorporates numerous real-world traffic scenarios from two open-pit mines that capture the unique challenges of unstructured road environments, including irregular intersections, roads without clear lane markings, and the response lags of heavy autonomous mining trucks. Furthermore, MineSim provides scenario libraries and benchmarks for static and dynamic obstacle avoidance problems, facilitating research into planning algorithms in these complex settings. By offering reproducible testing methods and scenario data, MineSim serves as a critical resource for advancing autonomous driving technologies in non-urban and unstructured road environments (see https://buaa-trans-mine-group.github.io/minesim).

Safety-assured high-speed navigation for MAVs

Micro air vehicles (MAVs) capable of high-speed autonomous navigation in unknown environments have the potential to improve applications like search and rescue and disaster relief, where timely and safe navigation is critical. However, achieving autonomous, safe, and high-speed MAV navigation faces systematic challenges, necessitating reduced vehicle weight and size for high-speed maneuvering, strong sensing capability for detecting obstacles at a distance, and advanced planning and control algorithms maximizing flight speed while ensuring obstacle avoidance. Here, we present the safety-assured high-speed aerial robot (SUPER), a compact MAV with a 280-millimeter wheelbase and a thrust-to-weight ratio greater than 5.0, enabling agile flight in cluttered environments. SUPER uses a lightweight three-dimensional light detection and ranging (LIDAR) sensor for accurate, long-range obstacle detection. To ensure high-speed flight while maintaining safety, we introduced an efficient planning framework that directly plans trajectories using LIDAR point clouds. In each replanning cycle, two trajectories were generated: one in known free spaces to ensure safety and another in both known and unknown spaces to maximize speed. Compared with baseline methods, this framework reduced failure rates by 35.9 times while flying faster and with half the planning time. In real-world tests, SUPER achieved autonomous flights at speeds exceeding 20 meters per second, successfully avoiding thin obstacles and navigating narrow spaces. SUPER represents a milestone in autonomous MAV systems, bridging the gap from laboratory research to real-world applications.

Path Planning and Motion Control of Robot Dog Through Rough Terrain Based on Vision Navigation

This article delineates the enhancement of an autonomous navigation and obstacle avoidance system for a quadruped robot dog. Part one of this paper presents the integration of a sophisticated multi-level dynamic control framework, utilizing Model Predictive Control (MPC) and Whole-Body Control (WBC) from MIT Cheetah. The system employs an Intel RealSense D435i depth camera for depth vision-based navigation, which enables high-fidelity 3D environmental mapping and real-time path planning. A significant innovation is the customization of the EGO-Planner to optimize trajectory planning in dynamically changing terrains, coupled with the implementation of a multi-body dynamics model that significantly improves the robot's stability and maneuverability across various surfaces. The experimental results show that the RGB-D system exhibits superior velocity stability and trajectory accuracy to the SLAM system, with a 20% reduction in the cumulative velocity error and a 10% improvement in path tracking precision. The experimental results also show that the RGB-D system achieves smoother navigation, requiring 15% fewer iterations for path planning, and a 30% faster success rate recovery in challenging environments. The successful application of these technologies in simulated urban disaster scenarios suggests promising future applications in emergency response and complex urban environments. Part two of this paper presents the development of a robust path planning algorithm for a robot dog on a rough terrain based on attached binocular vision navigation. We use a commercial-of-the-shelf (COTS) robot dog. An optical CCD binocular vision dynamic tracking system is used to provide environment information. Likewise, the pose and posture of the robot dog are obtained from the robot's own sensors, and a kinematics model is established. Then, a binocular vision tracking method is developed to determine the optimal path, provide a proposal (commands to actuators) of the position and posture of the bionic robot, and achieve stable motion on tough terrains. The terrain is assumed to be a gentle uneven terrain to begin with and subsequently proceeds to a more rough surface. This work consists of four steps: (1) pose and position data are acquired from the robot dog's own inertial sensors, (2) terrain and environment information is input from onboard cameras, (3) information is fused (integrated), and (4) path planning and motion control proposals are made. Ultimately, this work provides a robust framework for future developments in the vision-based navigation and control of quadruped robots, offering potential solutions for navigating complex and dynamic terrains.

Enhanced Visual SLAM for Collision-Free Driving with Lightweight Autonomous Cars

The paper presents a vision-based obstacle avoidance strategy for lightweight self-driving cars that can be run on a CPU-only device using a single RGB-D camera. The method consists of two steps: visual perception and path planning. The visual perception part uses ORBSLAM3 enhanced with optical flow to estimate the car's poses and extract rich texture information from the scene. In the path planning phase, the proposed method employs a method combining a control Lyapunov function and control barrier function in the form of a quadratic program (CLF-CBF-QP) together with an obstacle shape reconstruction process (SRP) to plan safe and stable trajectories. To validate the performance and robustness of the proposed method, simulation experiments were conducted with a car in various complex indoor environments using the Gazebo simulation environment. The proposed method can effectively avoid obstacles in the scenes. The proposed algorithm outperforms benchmark algorithms in achieving more stable and shorter trajectories across multiple simulated scenes.

Gradient-based autonomous obstacle avoidance trajectory planning for B-spline UAVs

Unmanned aerial vehicles (UAVs) have become the focus of current research because of their practicability in various scenarios. However, current local path planning methods often result in trajectories with numerous sharp or inflection points, which are not ideal for smooth UAV flight. This paper introduces a UAV path planning approach based on distance gradients. The key improvements include generating collision-free paths using collision information from initial trajectories and obstacles. Then, collision-free paths are subsequently optimized using distance gradient information. Additionally, a trajectory time adjustment method is proposed to ensure the feasibility and safety of the trajectory while prioritizing smoothness. The Limited-memory BFGS algorithm is employed to efficiently solve optimal local paths, with the ability to quickly restart the trajectory optimization program. The effectiveness of the proposed method is validated in the Robot Operating System simulation environment, demonstrating its ability to meet trajectory planning requirements for UAVs in complex unknown environments with high dynamics. Moreover, it surpasses traditional UAV trajectory planning methods in terms of solution speed, trajectory length, and data volume.

Design and Experimental Validation of Deep Reinforcement Learning-Based Fast Trajectory Planning and Control for Mobile Robot in Unknown Environment

This article is concerned with the problem of planning optimal maneuver trajectories and guiding the mobile robot toward target positions in uncertain environments for exploration purposes. A hierarchical deep learning-based control framework is proposed which consists of an upper level motion planning layer and a lower level waypoint tracking layer. In the motion planning phase, a recurrent deep neural network (RDNN)-based algorithm is adopted to predict the optimal maneuver profiles for the mobile robot. This approach is built upon a recently proposed idea of using deep neural networks (DNNs) to approximate the optimal motion trajectories, which has been validated that a fast approximation performance can be achieved. To further enhance the network prediction performance, a recurrent network model capable of fully exploiting the inherent relationship between preoptimized system state and control pairs is advocated. In the lower level, a deep reinforcement learning (DRL)-based collision-free control algorithm is established to achieve the waypoint tracking task in an uncertain environment (e.g., the existence of unexpected obstacles). Since this approach allows the control policy to directly learn from human demonstration data, the time required by the training process can be significantly reduced. Moreover, a noisy prioritized experience replay (PER) algorithm is proposed to improve the exploring rate of control policy. The effectiveness of applying the proposed deep learning-based control is validated by executing a number of simulation and experimental case studies. The simulation result shows that the proposed DRL method outperforms the vanilla PER algorithm in terms of training speed. Experimental videos are also uploaded, and the corresponding results confirm that the proposed strategy is able to fulfill the autonomous exploration mission with improved motion planning performance, enhanced collision avoidance ability, and less training time.

Position Checking-Based Sampling Approach Combined with Attraction Point Local Optimization for Safe Flight of UAVs

Trading off the allocation of limited computational resources between front-end path generation and back-end trajectory optimization plays a key role in improving the efficiency of unmanned aerial vehicle (UAV) motion planning. In this paper, a sampling-based kinodynamic planning method that can reduce the computational cost as well as the risks of UAV flight is proposed. Firstly, an initial trajectory connecting the start and end points without considering obstacles is generated. Then, a spherical space is constructed around the topological vertices of the environment, based on the intersections of the trajectory with the obstacles. Next, some unnecessary sampling points, as well as node rewiring, are discarded by the designed position-checking strategy to minimize the computational cost and reduce the risks of UAV flight. Finally, in order to make the planning framework adaptable to complex scenarios, the strategies for selecting different attraction points according to the environment are designed, which further ensures the safe flight of the UAV while improving the success rate of the front-end trajectory. Simulations and real-world experiment comparisons are conducted on a vision-based platform to verify the performance of the proposed method.

Developing a Flying Explorer for Autonomous Digital Modelling in Wild Unknowns

Digital modelling stands as a pivotal step in the realm of Digital Twinning. The future trend of Digital Twinning involves automated exploration and environmental modelling in complex scenes. In our study, we propose an innovative solution for robot odometry, path planning, and exploration in unknown outdoor environments, with a focus on Digital modelling. The approach uses a minimum cost formulation with pseudo-randomly generated objectives, integrating multi-path planning and evaluation, with emphasis on full coverage of unknown maps based on feasible boundaries of interest. The approach allows for dynamic changes to expected targets and behaviours. The evaluation is conducted on a robotic platform with a lightweight 3D LiDAR sensor model. The robustness of different types of odometry is compared, and the impact of parameters on motion planning is explored. The consistency and efficiency of exploring completely unknown areas are assessed in both indoor and outdoor scenarios. The experiment shows that the method proposed in this article can complete autonomous exploration and environmental modelling tasks in complex indoor and outdoor scenes. Finally, the study concludes by summarizing the reasons for exploration failures and outlining future focuses in this domain.

Autonomous localized path planning algorithm for UAVs based on TD3 strategy

Unmanned Aerial Vehicles are useful tools for many applications. However, autonomous path planning for Unmanned Aerial Vehicles in unfamiliar environments is a challenging problem when facing a series of problems such as poor consistency, high influence by the native controller of the Unmanned Aerial Vehicles. In this paper, we investigate reinforcement learning-based autonomous local path planning methods for Unmanned Aerial Vehicles with high autonomous decision-making capability and locally high portability. We propose an autonomous local path planning algorithm based on the TD3 strategy to solve the problem of local obstacle avoidance and path planning in unfamiliar environments using autonomous decision-making of Unmanned Aerial Vehicles. The simulation results on Gazebo show that our method can effectively realize the autonomous local path planning task for Unmanned Aerial Vehicles, the success rate of path planning with our method can reach 93% under the interference of no obstacles, and 92% in the environment with obstacles. Finally, our method can be used for autonomous path planning of Unmanned Aerial Vehicles in unfamiliar environments.

An Adaptive Two-Dimensional Voxel Terrain Mapping Method for Structured Environment

Accurate terrain mapping information is very important for foot landing planning and motion control in foot robots. Therefore, a terrain mapping method suitable for an indoor structured environment is proposed in this paper. Firstly, by constructing a terrain mapping framework and adding the estimation of the robot's pose, the algorithm converts the distance sensor measurement results into terrain height information and maps them into the voxel grid, and effectively reducing the influence of pose uncertainty in a robot system. Secondly, the height information mapped into the voxel grid is downsampled to reduce information redundancy. Finally, a preemptive random sample consistency (preemptive RANSAC) algorithm is used to divide the plane from the height information of the environment and merge the voxel grid in the extracted plane to realize the adaptive resolution 2D voxel terrain mapping (ARVTM) in the structured environment. Experiments show that the proposed mapping algorithm reduces the error of terrain mapping by 62.7% and increases the speed of terrain mapping by 25.1%. The algorithm can effectively identify and extract plane features in a structured environment, reducing the complexity of terrain mapping information, and improving the speed of terrain mapping.

Trajectory Planner for UAVs Based on Potential Field Obtained by a Kinodynamic Gene Regulation Network

Quadrotor unmanned aerial vehicles (UAVs) often encounter intricate environmental and dynamic limitations in real-world applications, underscoring the significance of proficient trajectory planning for ensuring both safety and efficiency during flights. To tackle this challenge, we introduce an innovative approach that harmonizes sophisticated environmental insights with the dynamic state of a UAV within a potential field framework. Our proposition entails a quadrotor trajectory planner grounded in a kinodynamic gene regulation network potential field. The pivotal contribution of this study lies in the amalgamation of environmental perceptions and kinodynamic constraints within a newly devised gene regulation network (GRN) potential field. By enhancing the gene regulation network model, the potential field becomes adaptable to the UAV's dynamic conditions and its surroundings, thereby extending the GRN into a kinodynamic GRN (K-GRN). The trajectory planner excels at charting courses that guide the quadrotor UAV through intricate environments while taking dynamic constraints into account. The amalgamation of environmental insights and kinodynamic constraints within the potential field framework bolsters the adaptability and stability of the generated trajectories. Empirical results substantiate the efficacy of our proposed methodology.

Autonomous aerial obstacle avoidance using LiDAR sensor fusion

The obstacle avoidance problem of unmanned aerial vehicle (UAV) mainly refers to the design of a method that can safely reach the target point from the starting point in an unknown flight environment. In this paper, we mainly propose an obstacle avoidance method composed of three modules: environment perception, algorithm obstacle avoidance and motion control. Our method realizes the function of reasonable and safe obstacle avoidance of UAV in low-altitude complex environments. Firstly, we use the light detection and ranging (LiDAR) sensor to perceive obstacles around the environment. Next, the sensor data is processed by the vector field histogram (VFH) algorithm to output the desired speed of drone flight. Finally, the expected speed value is sent to the quadrotor flight control and realizes autonomous obstacle avoidance flight of the drone. We verify the effectiveness and feasibility of the proposed method in 3D simulation environment.

Unmanned Aerial Vehicle Mediated Drug Delivery for First Aid

Timely administration of key medications toward patients with sudden diseases is critical to saving lives. However, slow transport of first-aid therapeutics and the potential absence of trained people for drug usage can lead to severe injuries or even death. Herein, an unmanned aerial vehicle (UAV)-mediated first-aid system for targeted delivery (uFAST) is developed. It allows unattended administration of emergency therapeutics-loaded transdermal microneedle (MN) patches toward patients to relieve symptoms by a contact-triggered microneedle applicator (CTMA). The implementability and safety of the uFAST for first aid is demonstrated in a severe hypoglycemic pig model by automatically delivering a glucagon patch with immediate and bioresponsive dual release modes. This platform technique may facilitate the development of UAV-mediated first-aid treatments for other sudden diseases.

Trust-region filtered sequential convex programming for multi-UAV trajectory planning and collision avoidance

This paper presents an trust-region filtered sequential convex programming (TRF-SCP) to reduce computational burdens of multi-UAV trajectory planning. In TRF-SCP, the trust-region based filter is proposed to remove the inactive collision-avoidance constraints of the convex programming subproblems for decreasing the complexity. The inactive constraints are detected based on the intersection relations between trust regions and collision-avoidance constraints. The trust-region based filter for different types of obstacles are tailored to address complex scenarios. An adaptive trust-region updating mechanism is also developed to mitigate infeasible iteration in TRF-SCP. The sizes of the trust regions are automatically adjusted according to the constraint violation of the optimized trajectory during the SCP iterations. TRF-SCP is then tested on several numerical multi-UAV formation scenarios involving cylindrical, spherical, conical, and polygon obstacles, respectively. Comparative studies demonstrate that TRF-SCP eliminates a large number of collision-avoidance constraints in the entire iterative process and outperforms SCP and Guaranteed Sequential Trajectory Optimization in terms of computational efficiency. The indoor flight experiments are presented to further evaluate the practicability of TRF-SCP.

Upgraded trajectory planning method deployed in autonomous exploration for unmanned aerial vehicle

Autonomous exploration is grounded on target decision and trajectory planning, which is widely deployed on unmanned aerial vehicles. However, existing methods generally only focus on the exploration effect of target decision but neglect the environment information gained with trajectory planning during flight, resulting in redundant exploration trajectories and low exploration efficiency. This article proposes an upgraded method of trajectory planning for autonomous exploration work. We design a fresh cost term considering the frontier information in the part of trajectory optimization. Besides, yaw angles are planned independently to catch more environment information during flight. We present extensive simulations and real-world tests. The results show that our proposed method reduces the exploration cost time by 10–15% compared with the previous one.

An Efficient Online Trajectory Generation Method Based on Kinodynamic Path Search and Trajectory Optimization for Human-Robot Interaction Safety

With the rapid development of robot perception and planning technology, robots are gradually getting rid of fixed fences and working closely with humans in shared workspaces. The safety of human-robot coexistence has become critical. Traditional motion planning methods perform poorly in dynamic environments where obstacles motion is highly uncertain. In this paper, we propose an efficient online trajectory generation method to help manipulator autonomous planning in dynamic environments. Our approach starts with an efficient kinodynamic path search algorithm that considers the links constraints and finds a safe and feasible initial trajectory with minimal control effort and time. To increase the clearance between the trajectory and obstacles and improve the smoothness, a trajectory optimization method using the B-spline convex hull property is adopted to minimize the penalty of collision cost, smoothness, and dynamical feasibility. To avoid the collisions between the links and obstacles and the collisions of the links themselves, a constraint-relaxed links collision avoidance method is developed by solving a quadratic programming problem. Compared with the existing state-of-the-art planning method for dynamic environments and advanced trajectory optimization method, our method can generate a smoother, collision-free trajectory in less time with a higher success rate. Detailed simulation comparison experiments, as well as real-world experiments, are reported to verify the effectiveness of our method.

Swarm of micro flying robots in the wild

Aerial robots are widely deployed, but highly cluttered environments such as dense forests remain inaccessible to drones and even more so to swarms of drones. In these scenarios, previously unknown surroundings and narrow corridors combined with requirements of swarm coordination can create challenges. To enable swarm navigation in the wild, we develop miniature but fully autonomous drones with a trajectory planner that can function in a timely and accurate manner based on limited information from onboard sensors. The planning problem satisfies various task requirements including flight efficiency, obstacle avoidance, and inter-robot collision avoidance, dynamical feasibility, swarm coordination, and so on, thus realizing an extensible planner. Furthermore, the proposed planner deforms trajectory shapes and adjusts time allocation synchronously based on spatial-temporal joint optimization. A high-quality trajectory thus can be obtained after exhaustively exploiting the solution space within only a few milliseconds, even in the most constrained environment. The planner is finally integrated into the developed palm-sized swarm platform with onboard perception, localization, and control. Benchmark comparisons validate the superior performance of the planner in trajectory quality and computing time. Various real-world field experiments demonstrate the extensibility of our system. Our approach evolves aerial robotics in three aspects: capability of cluttered environment navigation, extensibility to diverse task requirements, and coordination as a swarm without external facilities.

Robust and Efficient Trajectory Replanning Based on Guiding Path for Quadrotor Fast Autonomous Flight

Path planning is one of the key parts of unmanned aerial vehicle (UAV) fast autonomous flight in an unknown cluttered environment. However, real-time and stability remain a significant challenge in the field of path planning. To improve the robustness and efficiency of the path planning method in complex environments, this paper presents RETRBG, a robust and efficient trajectory replanning method based on the guiding path. Firstly, a safe guiding path is generated by using an improved A* and path pruning method, which is used to perceive the narrow space in its surrounding environment. Secondly, under the guidance of the path, a guided kinodynamic path searching method (GKPS) is devised to generate a safe, kinodynamically feasible and minimum-time initial path. Finally, an adaptive optimization function with two modes is proposed to improve the optimization quality in complex environments, which selects the optimization mode to optimize the smoothness and safety of the path according to the perception results of the guiding path. The experimental results demonstrate that the proposed method achieved good performance both in different obstacle densities and different resolutions. Compared with the other state-of-the-art methods, the quality and success rate of the planning result are significantly improved.