Q-learning improved golden jackal optimization algorithm and its application to reliability optimization of hydraulic system

To endow the prey with intelligent movement behavior and improve the performance of Golden Jackal Optimization (GJO), a Q-learning Improved Gold Jackal Optimization (QIGJO) algorithm is proposed. This paper introduces five update mechanisms and proposes double-population Q-learning collaborative mechanism to select appropriate update mechanisms to improve GJO performance. Additionally, a new convergence factor is incorporated to enhance convergence capability of GJO. QIGJO demonstrates excellent performance across 23 benchmark functions, CEC2022, and three classical engineering design problems, indicating high convergence accuracy and significantly enhanced global exploration capability. The reliability optimization model of the hydraulic system for concrete pump trucks was established based on a Continuous-time Multi-dimensional T-S dynamic Fault Tree (CM-TSdFT), considering the two-dimensional factors of operating time and number of impacts. Utilizing QIGJO to optimize this model yielded excellent results, providing valuable methodological support for reliability optimization of hydraulic systems.

Action Mapping: A Reinforcement Learning Method for Constrained-Input Systems

Existing approaches to constrained-input optimal control problems mainly focus on systems with input saturation, whereas other constraints, such as combined inequality constraints and state-dependent constraints, are seldom discussed. In this article, a reinforcement learning (RL)-based algorithm is developed for constrained-input optimal control of discrete-time (DT) systems. The deterministic policy gradient (DPG) is introduced to iteratively search the optimal solution to the Hamilton-Jacobi-Bellman (HJB) equation. To deal with input constraints, an action mapping (AM) mechanism is proposed. The objective of this mechanism is to transform the exploration space from the subspace generated by the given inequality constraints to the standard Cartesian product space, which can be searched effectively by existing algorithms. By using the proposed architecture, the learned policy can output control signals satisfying the given constraints, and the original reward function can be kept unchanged. In our study, the convergence analysis is given. It is shown that the iterative algorithm is convergent to the optimal solution of the HJB equation. In addition, the continuity of the iterative estimated Q -function is investigated. Two numerical examples are provided to demonstrate the effectiveness of our approach.

On a Finitely Activated Terminal RNN Approach to Time-Variant Problem Solving

This article concerns with terminal recurrent neural network (RNN) models for time-variant computing, featuring finite-valued activation functions (AFs), and finite-time convergence of error variables. Terminal RNNs stand for specific models that admit terminal attractors, and the dynamics of each neuron retains finite-time convergence. The might-existing imperfection in solving time-variant problems, through theoretically examining the asymptotically convergent RNNs, is pointed out for which the finite-time-convergent models are most desirable. The existing AFs are summarized, and it is found that there is a lack of the AFs that take only finite values. A finitely valued terminal RNN, among others, is taken into account, which involves only basic algebraic operations and taking roots. The proposed terminal RNN model is used to solve the time-variant problems undertaken, including the time-variant quadratic programming and motion planning of redundant manipulators. The numerical results are presented to demonstrate effectiveness of the proposed neural network, by which the convergence rate is comparable with that of the existing power-rate RNN.

Solver-Critic: A Reinforcement Learning Method for Discrete-Time-Constrained-Input Systems

In this article, a solver-critic (SC) architecture is developed for optimal control problems of discrete-time (DT)-constrained-input systems. The proposed design consists of three parts: 1) a critic network; 2) an action solver; and 3) a target network. The critic network first approximates the action-value function using the sum-of-squares (SOS) polynomial. Then, the action solver adopts the SOS programming to obtain control inputs within the constraint set. The target network introduces the soft update mechanism into policy evaluation to stabilize the learning process. By using the proposed architecture, the constrained-input control problem can be solved without adding the nonquadratic functionals into the reward function. In this article, the theoretical analysis of the convergence property is presented. Besides, the effects of both different initial Q -functions and different discount factors are investigated. It is proven that the learned policy converges to the optimal solution of the Hamilton-Jacobi-Bellman equation. Four numerical examples are provided to validate the theoretical analysis and also demonstrate the effectiveness of our approach.

Analog neural nonderivative optimizers

Continuous-time neural networks for solving convex nonlinear unconstrained programming problems without using gradient information of the objective function are proposed and analyzed. Thus, the proposed networks are nonderivative optimizers. First, networks for optimizing objective functions of one variable are discussed. Then, an existing one-dimensional optimizer is analyzed, and a new line search optimizer is proposed. It is shown that the proposed optimizer network is robust in the sense that it has disturbance rejection property. The network can be implemented easily in hardware using standard circuit elements. The one-dimensional net is used as a building block in multidimensional networks for optimizing objective functions of several variables. The multidimensional nets implement a continuous version of the coordinate descent method.

An Extended Projection Neural Network for Constrained Optimization

Recently, a projection neural network has been shown to be a promising computational model for solving variational inequality problems with box constraints. This letter presents an extended projection neural network for solving monotone variational inequality problems with linear and nonlinear constraints. In particular, the proposed neural network can include the projection neural network as a special case. Compared with the modified projection-type methods for solving constrained monotone variational inequality problems, the proposed neural network has a lower complexity and is suitable for parallel implementation. Furthermore, the proposed neural network is theoretically proven to be exponentially convergent to an exact solution without a Lipschitz condition. Illustrative examples show that the extended projection neural network can be used to solve constrained monotone variational inequality problems.

A General Projection Neural Network for Solving Monotone Variational Inequalities and Related Optimization Problems

Recently, a projection neural network for solving monotone variational inequalities and constrained optimization problems was developed. In this paper, we propose a general projection neural network for solving a wider class of variational inequalities and related optimization problems. In addition to its simple structure and low complexity, the proposed neural network includes existing neural networks for optimization, such as the projection neural network, the primal-dual neural network, and the dual neural network, as special cases. Under various mild conditions, the proposed general projection neural network is shown to be globally convergent, globally asymptotically stable, and globally exponentially stable. Furthermore, several improved stability criteria on two special cases of the general projection neural network are obtained under weaker conditions. Simulation results demonstrate the effectiveness and characteristics of the proposed neural network.

Real-time synthesis of sparsely interconnected neural associative memories

The problem of implementing associative memories using sparsely interconnected generalized Brain-State-in-a-Box (gBSB) network is addressed in this paper. In particular, a "designer" neural network that synthesizes the associative memories is proposed. An upper bound on the time required for the designer network to reach a solution is determined. A neighborhood criterion with toroidal geometry for the cellular gBSB network is analyzed, in which the number of adjacent cells is independent of the generic cell location. A design method of neural associative memories with prespecified interconnecting weights is presented. The effectiveness of the proposed synthesis method is demonstrated with numerical examples.

On neural networks that design neural associative memories

The design problem of generalized brain-state-in-a-box (GBSB) type associative memories is formulated as a constrained optimization program, and "designer" neural networks for solving the program in real time are proposed. The stability of the designer networks is analyzed using Barbalat's lemma. The analyzed and synthesized neural associative memories do not require symmetric weight matrices. Two types of the GBSB-based associative memories are analyzed, one when the network trajectories are constrained to reside in the hypercube [-1, 1](n) and the other type when the network trajectories are confined to stay in the hypercube [0, 1](n). Numerical examples and simulations are presented to illustrate the results obtained.

Existence and stability of a unique equilibrium in continuous-valued discrete-time asynchronous Hopfield neural networks

It is shown that the assumption of D-stability of the interconnection matrix, together with the standard assumptions on the activation functions, guarantee the existence of a unique equilibrium under a synchronous mode of operation as well as a class of asynchronous modes. For the synchronous mode, these assumptions are also shown to imply local asymptotic stability of the equilibrium. For the asynchronous mode of operation, two results are derived. First, it is shown that symmetry and stability of the interconnection matrix guarantee local asymptotic stability of the equilibrium under a class of asynchronous modes-this is referred to as local absolute asymptotic stability. Second, it is shown that, under the standard assumptions, if the nonnegative matrix whose elements are the absolute values of the corresponding elements of the interconnection matrix is stable, then the equilibrium is globally absolutely asymptotically stable under a class of asynchronous modes. The results obtained are discussed from the points of view of their applications, robustness, and their relationship to earlier results.

Optimization neural network for solving flow problems

This paper describes a neural network for solving flow problems, which are of interest in many areas of application as in fuel, hydro, and electric power scheduling. The neural network consist of two layers: a hidden layer and an output layer. The hidden units correspond to the nodes of the flow graph. The output units represent the branch variables. The network has a linear order of complexity, it is easily programmable, and it is suited for analog very large scale integration (VLSI) realization. The functionality of the proposed network is illustrated by a simulation example concerning the maximal flow problem.