Adaptive Optimal Tracking Control of an Underactuated Surface Vessel Using Actor-Critic Reinforcement Learning

Reinforcement learning-based trajectory tracking optimal control of unmanned surface vehicles in narrow water areas

For unmanned surface vehicles (USVs) navigating in narrow water areas in the presence of unknown dynamics and ocean environmental disturbances, this paper develops a reinforcement learning (RL)-based optimal control scheme for the trajectory tracking of USVs under motion state constraints. A nonlinear map is introduced to transform constrained motion state errors into bounded transformed errors, and then the motion state-constrained trajectory tracking problem of USVs is equivalently transformed into a boundedness problem of the transformed errors. Furthermore, an actor-critic framework is developed by utilizing adaptive neural networks (NNs). Within the actor-critic framework, a novel weight update law is designed for the critic NN by combining the gradient descent approach and the concurrent learning technology, thereby relaxing the persistent excitation condition required for adaptive critic NN weight updates. Subsequently, a disturbance compensator is designed and combined with the actor-critic framework to learn the trajectory tracking optimal control law for USVs in the presence of unknown dynamics and disturbances. Finally, theoretical analyses prove that the developed control scheme guarantees the boundedness of all signals in the USV closed-loop trajectory tracking control system, and simulation results show that the developed control scheme can make USVs track the desired trajectory in narrow water areas while reducing the energy consumption by approximately 14.6 % compared with an existing controller.

Reinforcement learning-based dynamic field exploration and reconstruction using multi-robot systems for environmental monitoring

In the realm of real-time environmental monitoring and hazard detection, multi-robot systems present a promising solution for exploring and mapping dynamic fields, particularly in scenarios where human intervention poses safety risks. This research introduces a strategy for path planning and control of a group of mobile sensing robots to efficiently explore and reconstruct a dynamic field consisting of multiple non-overlapping diffusion sources. Our approach integrates a reinforcement learning-based path planning algorithm to guide the multi-robot formation in identifying diffusion sources, with a clustering-based method for destination selection once a new source is detected, to enhance coverage and accelerate exploration in unknown environments. Simulation results and real-world laboratory experiments demonstrate the effectiveness of our approach in exploring and reconstructing dynamic fields. This study advances the field of multi-robot systems in environmental monitoring and has practical implications for rescue missions and field explorations.

Laser Rangefinder Methods: Autonomous-Vehicle Trajectory Control in Horticultural Plantings

This article presents a developed motion control system for a robotic platform based on laser-ranging methods, a graph traversal algorithm and the search for the optimal path. The algorithm was implemented in an agricultural building and in the field. As a result, the most efficient algorithm for finding the optimal path (A*) for the robotic platform was chosen when performing various technological operations. In the Rviz visualization environment, a program code was developed for planning the movement path and setting the points of the movement trajectory in real time. To find the optimal navigation graph in an artificial garden, an application was developed using the C# programming language and Visual Studio 2019. The results of the experiments showed that field conditions can differ significantly from laboratory conditions, while the positioning accuracy is significantly lower. The statistical processing of the experimental data showed that, for the movement of a robotic platform along a given trajectory in the field, the most effective conditions are as follows: speed: 2.5 km/h; illumination: 109,600 lux; distance to the tree: 0.5 m. An analysis of the operating parameters of the LiDAR sensor showed that it provides a high degree of positioning accuracy under various lighting conditions at various speeds in the aisles of a garden 3 m wide with an inter-stem distance of 1.5 m and a tree crown width of 0.5 m. The use of sensors-rangefinders of the optical range-allows for the performance of positional movements of the robotic platform and ensures the autonomous performance of the basic technological operations of the units in intensive gardens with a deviation from the specified trajectory of no more than 8.4 cm, which meets the agrotechnical requirements.

An Integrated Autonomous Dynamic Navigation Approach toward a Composite Air-Ground Risk Construction Scenario

Unmanned transportation in construction scenarios presents a significant challenge due to the presence of complex dynamic on-ground obstacles and potential airborne falling objects. Consequently, the typical methodology for composite air-ground risk avoidance in construction scenarios holds enormous importance. In this paper, an integrated potential-field-based risk assessment approach is proposed to evaluate the threat severity of the environmental obstacles. Meanwhile, the self-adaptive dynamic window approach is suggested to manage the real-time motion planning solution for air-ground risks. By designing the multi-objective velocity sample window, we constrain the vehicle's speed planning instructions within reasonable limits. Combined with a hierarchical decision-making mechanism, this approach achieves effective obstacle avoidance with multiple drive modes. Simulation results demonstrate that, in comparison with the traditional dynamic window approach, the proposed method offers enhanced stability and efficiency in risk avoidance, underlining its notable safety and effectiveness.

Bilinear Interpolation of Three-Dimensional Gain-Scheduled Autopilots

Gain-scheduled autopilots have emerged as a dominant strategy to achieve adaptive control of coupled, non-linear engineering complexities, owing to an ability to adapt to changing operational conditions and uncertainties. This study focuses on utilizing bilinear interpolation of gain-scheduled autopilots, emphasizing enhanced system performance and robustness. Through a comprehensive investigation and comparative analysis using three disparate cases, advantages over conventional methods are revealed. Strengths and weaknesses of both simple and specialized variants (such as linear, and real-time gain-scheduling) are introduced. Three missile guidance case-studies utilize simulation time and miss distance figures of merit. Comparing the performance of bilinear interpolation and automatic instantiations to index-search, over comparable traveled distances, missile miss distances were improved 179% and 196% respectively with slightly improved computational burden.