Iterative Learning Model Predictive Control Based on Iterative Data-Driven Modeling

Collaborative twin actors framework using deep deterministic policy gradient for flexible batch processes

Due to its inherent efficiency in the process industry for achieving desired products, batch processing is widely acknowledged for its repetitive nature. Batch-to-batch learning control has traditionally been esteemed as a robust strategy for batch process control. However, the presence of flexible operating conditions in practical batch systems often leads to a lack of prior learning information, hindering learning control from optimizing performance. This article presents a novel approach to flexible batch process control using deep reinforcement learning (DRL) with twin actors. Specifically, a collaborative twin-actor-based deep deterministic policy gradient (CTA-DDPG) method is proposed to generate control policies and ensure safe operation across varying trial lengths and initial conditions. This approach involves the sequential construction of two sets of actor-critic networks with a shared critic. The first set explores meta-policy during an offline stage, while the second set enhances control performance using a supplementary agent during an online stage. To ensure robust policy transfer and efficient learning, a policy integration mechanism and a spatial-temporal experience replay strategy are incorporated, facilitating transfer stability and learning efficiency. The performance of CTA-DDPG is evaluated using both numerical examples and nonlinear injection molding process for tracking control. The results demonstrate the effectiveness and superiority of the proposed method in achieving desired control outcomes.

Dynamic Neural Network Predictive Compensation-Based Point-to-Point Iterative Learning Control With Nonuniform Batch Length

This article discusses the problem of nonuniform running length in incomplete tracking control, which often occurs in industrial processes due to artificial or environmental changes, such as chemical engineering. It affects the design and application of iterative learning control (ILC) that relies on the strictly repetitive property. Therefore, a dynamic neural network (NN) predictive compensation strategy is proposed under the point-to-point ILC framework. To handle the difficulty of establishing an accurate mechanism model for real process control, the data-driven approach is also introduced. First, applying the iterative dynamic linearization (IDL) technique and radial basis function NN (RBFNN) to construct the iterative dynamic predictive data model (IDPDM) relies on input-output (I/O) signal, and the extended variable is defined by a predictive model to compensate for the incomplete operation length. Then, a learning algorithm based on multiple iteration errors is proposed using an objective function. This learning gain is constantly updated through the NN to adapt to changes in the system. In addition, the composite energy function (CEF) and compression mapping prove that the system is convergent. Finally, two numerical simulation examples are given.

A New Grey Wolf Optimizer Tuned Extended Generalized Predictive Control for Distillation Process

The distillation process plays an essential role in the petrochemical industry. However, the high-purity distillation column has complicated dynamic characteristics such as strong coupling and large time delay. To control the distillation column accurately, we proposed an extended generalized predictive control (EGPC) method inspired by the principles of extended state observer and proportional-integral-type generalized predictive control method; the proposed EGPC can adaptively compensate the system for the effects of coupling and model mismatch online and performs well in controlling time-delay systems. The strong coupling of the distillation column needs fast control, and the large time delay requires soft control. To balance the requirement for fast and soft control at the same time, a grey wolf optimizer with reverse learning and adaptive leaders number strategies (RAGWO) was proposed to tune the parameters of EGPC, and these strategies enable RAGWO to have a better initial population and improve its exploitation and exploration ability. The benchmark test results indicate that the RAGWO outperforms the existing optimizers for most of the selected benchmark functions. Extensive simulations show that the proposed method in terms of fluctuation and response time is superior to other methods for controlling the distillation process.

Event-Based Switching Iterative Learning Model Predictive Control for Batch Processes With Randomly Varying Trial Lengths

Iterative learning model predictive control (ILMPC) has been recognized as an excellent batch process control strategy for progressively improving tracking performance along trials. However, as a typical learning-based control method, ILMPC generally requires the strict identity of trial lengths to implement 2-D receding horizon optimization. The randomly varying trial lengths extensively existing in practice can result in the insufficiency of learning prior information, and even the suspension of control update. Regarding this issue, this article embeds a novel prediction-based modification mechanism into ILMPC, to adjust the process data of each trial into the same length by compensating the data of absent running periods with the predictive sequences at the end point. Under this modification scheme, it is proved that the convergence of the classical ILMPC is guaranteed by an inequality condition relative with the probability distribution of trial lengths. Considering the practical batch process with complex nonlinearity, a 2-D neural-network predictive model with parameter adaptability along trials is established to generate highly matched compensation data for the prediction-based modification. To best utilize the real process information of multiple past trials while guaranteeing the learning priority of the latest trials, an event-based switching learning structure is proposed in ILMPC to determine different learning orders according to the probability event with respect to the trial length variation direction. The convergence of the nonlinear event-based switching ILMPC system is analyzed theoretically under two situations divided by the switching condition. The simulations on a numerical example and the injection molding process verify the superiority of the proposed control methods.

Bridging Reinforcement Learning and Iterative Learning Control: Autonomous Motion Learning for Unknown, Nonlinear Dynamics

This work addresses the problem of reference tracking in autonomously learning robots with unknown, nonlinear dynamics. Existing solutions require model information or extensive parameter tuning, and have rarely been validated in real-world experiments. We propose a learning control scheme that learns to approximate the unknown dynamics by a Gaussian Process (GP), which is used to optimize and apply a feedforward control input on each trial. Unlike existing approaches, the proposed method neither requires knowledge of the system states and their dynamics nor knowledge of an effective feedback control structure. All algorithm parameters are chosen automatically, i.e. the learning method works plug and play. The proposed method is validated in extensive simulations and real-world experiments. In contrast to most existing work, we study learning dynamics for more than one motion task as well as the robustness of performance across a large range of learning parameters. The method’s plug and play applicability is demonstrated by experiments with a balancing robot, in which the proposed method rapidly learns to track the desired output. Due to its model-agnostic and plug and play properties, the proposed method is expected to have high potential for application to a large class of reference tracking problems in systems with unknown, nonlinear dynamics.