Boundary Control of Linear Uncertain 1-D Parabolic PDE Using Approximate Dynamic Programming

Learning-based optimal boundary control for parabolic distributed parameter system with actuator dynamics

Considering actuator dynamics, we investigate a coupled system of parabolic partial differential equations (PDEs) and ordinary differential equations (ODEs), developing a data-driven boundary optimal controller based on iterative learning. Notably, the boundary input appears in the ODE-style actuator dynamics, making the boundary condition reduction quite difficult. Optimizing infinite-dimensional performance indexes in coupled Hilbert spaces is not a trivial task. This work is the first to solve the optimal control problem for a coupled PDE-ODE system with actuator dynamics under Neumann boundary conditions. We equivalently reformat the coupled PDE-ODE system into a system with homogeneous boundary conditions and then derive its singular perturbation form in an infinite-dimensional space. Subsequently, by constructing critic and actor networks, we design a novel model-free iterative learning optimal control algorithm where weighted residual techniques are used. The algorithm uses a rich set of arbitrary control policies rather than limiting to evaluation policies, enhancing the exploration capability of the learning algorithm and relaxing the requirement for persistent excitation conditions. Furthermore, the uniformly asymptotic stability of the closed-loop coupled system is demonstrated in the infinite-dimensional Hilbert space for each learning iteration, not only for the final one. Finally, the effectiveness of the proposed approach is verified by simulations on the diffusion-reaction process.

Observer-Based Boundary Stabilization of Coupled Semilinear Reaction-Diffusion Neural Networks With Spatially Varying Coefficients via Event-Triggered Controller

The aim of this article is to propose an observer-based event-triggered Robin boundary control strategy for the exponential stabilization of the coupled semilinear reaction-diffusion neural networks with spatially varying coefficients. Toward this aim, we design an observer to estimate the value of system states by using some of these system values as the available measurement. An observer-based event-triggered boundary stabilizer is then presented to exponentially stabilize the considered systems with the Zeno behavior being excluded. Throughout this article, the main used method is backstepping, which yields an explicit expression of the control formulae. Moreover, we see that the proposed event-triggered boundary control scheme can ensure the desired level of control performance with fewer control law updates. A numerical example is finally given to illustrate the effectiveness of our proposed method.

Concurrent Learning Robust Adaptive Fault Tolerant Boundary Regulation of Hyperbolic Distributed Parameter Systems

This article develops a robust adaptive boundary output regulation approach for a class of complex anticollocated hyperbolic partial differential equations subjected to multiplicative unknown faults in both the boundary sensor and actuator. The regulator design is based on the internal model principle, which amounts to stabilize a coupled cascade system, which consists of a finite-dimensional internal model driven by a hyperbolic distributed parameter system (DPS). To this end, a systematic sliding mode equipped with a backstepping approach is developed such that the robust state feedback control can be realized. Moreover, since the available information is a faulty boundary measurement at the right side point, state estimation is required. However, due to the presence of boundary unknown faults, we need to solve an issue of joint fault-state estimation. Restrictive persistent excitation conditions are usually required to guarantee the exact estimation of faults but are unrealistic in practice. To this end, a novel concurrent learning (CL) adaptive observer is proposed so that exponential convergence is obtained. It is the first time that the spirit of CL is introduced to the field of DPSs. Consequently, the observer-based adaptive boundary fault tolerant control scheme is developed, and rigorous theoretical analysis is given such that the exponential output regulation can be achieved. Finally, the effectiveness of the proposed methodology is demonstrated via comparative simulations.

Policy-Iteration-Based Finite-Horizon Approximate Dynamic Programming for Continuous-Time Nonlinear Optimal Control

The Hamilton-Jacobi-Bellman (HJB) equation serves as the necessary and sufficient condition for the optimal solution to the continuous-time (CT) optimal control problem (OCP). Compared with the infinite-horizon HJB equation, the solving of the finite-horizon (FH) HJB equation has been a long-standing challenge, because the partial time derivative of the value function is involved as an additional unknown term. To address this problem, this study first-time bridges the link between the partial time derivative and the terminal-time utility function, and thus it facilitates the use of the policy iteration (PI) technique to solve the CT FH OCPs. Based on this key finding, the FH approximate dynamic programming (ADP) algorithm is proposed leveraging an actor-critic framework. It is shown that the algorithm exhibits important properties in terms of convergence and optimality. Rather importantly, with the use of multilayer neural networks (NNs) in the actor-critic architecture, the algorithm is suitable for CT FH OCPs toward more general nonlinear and complex systems. Finally, the effectiveness of the proposed algorithm is demonstrated by conducting a series of simulations on both a linear quadratic regulator (LQR) problem and a nonlinear vehicle tracking problem.

Event-Triggered Optimal Control for Temperature Field of Roller Kiln Based on Adaptive Dynamic Programming

Temperature field control is crucial for the comprehensive performance of Ni-Co-Mn layered cathode material that is the most important part of lithium-ion batteries. Starting from the aspect of a class of distributed parameter systems described by highly dissipative partial differential equations (PDEs), an event-triggered optimal control (ETOC) method based on adaptive dynamic programming (ADP) for the roller kiln temperature field is proposed. First, we formulate the event-triggered control problem of the temperature field under the general framework of PDE systems. Then, an event-triggered condition is designed based on the stability of the closed-loop PDE system, which also guarantees the upper bound of the performance index. Subsequently, ADP technology is adopted to realize the ETOC, where the critic network is employed to approximate the optimal value function. Since the studied system can be regarded as an impulsive dynamic system with flow dynamics and jump dynamics simultaneously, the stability of the impulsive dynamic system combined with the ADP-based closed-loop PDE system is proved. Finally, simulation results on the temperature field verify the effectiveness of the proposed method.

Sliding-Mode Surface-Based Approximate Optimal Control for Uncertain Nonlinear Systems With Asymptotically Stable Critic Structure

This article develops a novel sliding-mode surface (SMS)-based approximate optimal control scheme for a large class of nonlinear systems affected by unknown mismatched perturbations. The observer-based perturbation estimation procedure is employed to establish the online updated value function. The solution to the Hamilton-Jacobi-Bellman equation is approximated by an SMS-based critic neural network whose weights error dynamics is designed to be asymptotically stable by nested update laws. The sliding-mode control strategy is combined with the approximate optimal control design procedure to obtain a faster control action. The stability is proved based on the Lyapunov's direct method. The simulation results show the effectiveness of the developed control scheme.

Boundary Control of 2-D Burgers' PDE: An Adaptive Dynamic Programming Approach

In this paper, an adaptive dynamic programming-based near optimal boundary controller is developed for partial differential equations (PDEs) modeled by the uncertain Burgers' equation under Neumann boundary condition in 2-D. Initially, Hamilton-Jacobi-Bellman equation is derived in infinite-dimensional space. Subsequently, a novel neural network (NN) identifier is introduced to approximate the nonlinear dynamics in the 2-D PDE. The optimal control input is derived by online estimation of the value function through an additional NN-based forward-in-time estimation and approximated dynamic model. Novel update laws are developed for estimation of the identifier and value function online. The designed control policy can be applied using a finite number of actuators at the boundaries. Local ultimate boundedness of the closed-loop system is studied in detail using Lyapunov theory. Simulation results confirm the optimizing performance of the proposed controller on an unstable 2-D Burgers' equation.