Nonlinear model predictive control based on collective neurodynamic optimization

Distributed nonconvex optimization subject to globally coupled constraints via collaborative neurodynamic optimization

In this paper, a recurrent neural network is proposed for distributed nonconvex optimization subject to globally coupled (in)equality constraints and local bound constraints. Two distributed optimization models, including a resource allocation problem and a consensus-constrained optimization problem, are established, where the objective functions are not necessarily convex, or the constraints do not guarantee a convex feasible set. To handle the nonconvexity, an augmented Lagrangian function is designed, based on which a recurrent neural network is developed for solving the optimization models in a distributed manner, and the convergence to a local optimal solution is proven. For the search of global optimal solutions, a collaborative neurodynamic optimization method is established by utilizing multiple proposed recurrent neural networks and a meta-heuristic rule. A numerical example, a simulation involving an electricity market, and a distributed cooperative control problem are provided to verify and demonstrate the characteristics of the main results.

A collaborative neurodynamic approach with two-timescale projection neural networks designed via majorization-minimization for global optimization and distributed global optimization

In this paper, two two-timescale projection neural networks are proposed based on the majorization-minimization principle for nonconvex optimization and distributed nonconvex optimization. They are proved to be globally convergent to Karush-Kuhn-Tucker points. A collaborative neurodynamic approach leverages multiple two-timescale projection neural networks repeatedly re-initialized using a meta-heuristic rule for global optimization and distributed global optimization. Two numerical examples are elaborated to demonstrate the efficacy of the proposed approaches.

Hash Bit Selection via Collaborative Neurodynamic Optimization With Discrete Hopfield Networks

Hash bit selection (HBS) aims to find the most discriminative and informative hash bits from a hash pool generated by using different hashing algorithms. It is usually formulated as a binary quadratic programming problem with an information-theoretic objective function and a string-length constraint. In this article, it is equivalently reformulated in the form of a quadratic unconstrained binary optimization problem by augmenting the objective function with a penalty function. The reformulated problem is solved via collaborative neurodynamic optimization (CNO) with a population of classic discrete Hopfield networks. The two most important hyperparameters of the CNO approach are determined based on Monte Carlo test results. Experimental results on three benchmark data sets are elaborated to substantiate the superiority of the collaborative neurodynamic approach to several existing methods for HBS.

Boolean matrix factorization based on collaborative neurodynamic optimization with Boltzmann machines

This paper presents a collaborative neurodynamic approach to Boolean matrix factorization. Based on a binary optimization formulation to minimize the Hamming distance between a given data matrix and its low-rank reconstruction, the proposed approach employs a population of Boltzmann machines operating concurrently for scatter search of factorization solutions. In addition, a particle swarm optimization rule is used to re-initialize the neuronal states of Boltzmann machines upon their local convergence to escape from local minima toward global solutions. Experimental results demonstrate the superior convergence and performance of the proposed approach against six baseline methods on ten benchmark datasets.

Two-Timescale Multilayer Recurrent Neural Networks for Nonlinear Programming

This article presents a neurodynamic approach to nonlinear programming. Motivated by the idea of sequential quadratic programming, a class of two-timescale multilayer recurrent neural networks is presented with neuronal dynamics in their output layer operating at a bigger timescale than in their hidden layers. In the two-timescale multilayer recurrent neural networks, the transient states in the hidden layer(s) undergo faster dynamics than those in the output layer. Sufficient conditions are derived on the convergence of the two-timescale multilayer recurrent neural networks to local optima of nonlinear programming problems. Simulation results of collaborative neurodynamic optimization based on the two-timescale neurodynamic approach on global optimization problems with nonconvex objective functions or constraints are discussed to substantiate the efficacy of the two-timescale neurodynamic approach.

Cardinality-constrained portfolio selection based on collaborative neurodynamic optimization

Portfolio optimization is one of the most important investment strategies in financial markets. It is practically desirable for investors, especially high-frequency traders, to consider cardinality constraints in portfolio selection, to avoid odd lots and excessive costs such as transaction fees. In this paper, a collaborative neurodynamic optimization approach is presented for cardinality-constrained portfolio selection. The expected return and investment risk in the Markowitz framework are scalarized as a weighted Chebyshev function and the cardinality constraints are equivalently represented using introduced binary variables as an upper bound. Then cardinality-constrained portfolio selection is formulated as a mixed-integer optimization problem and solved by means of collaborative neurodynamic optimization with multiple recurrent neural networks repeatedly repositioned using a particle swarm optimization rule. The distribution of resulting Pareto-optimal solutions is also iteratively perfected by optimizing the weights in the scalarized objective functions based on particle swarm optimization. Experimental results with stock data from four major world markets are discussed to substantiate the superior performance of the collaborative neurodynamic approach to several exact and metaheuristic methods.

Power-Type Varying-Parameter RNN for Solving TVQP Problems: Design, Analysis, and Applications

Many practical problems can be solved by being formulated as time-varying quadratic programing (TVQP) problems. In this paper, a novel power-type varying-parameter recurrent neural network (VPNN) is proposed and analyzed to effectively solve the resulting TVQP problems, as well as the original practical problems. For a clear understanding, we introduce this model from three aspects: design, analysis, and applications. Specifically, the reason why and the method we use to design this neural network model for solving online TVQP problems subject to time-varying linear equality/inequality are described in detail. The theoretical analysis confirms that when activated by six commonly used activation functions, VPNN achieves a superexponential convergence rate. In contrast to the traditional zeroing neural network with fixed design parameters, the proposed VPNN has better convergence performance. Comparative simulations with state-of-the-art methods confirm the advantages of VPNN. Furthermore, the application of VPNN to a robot motion planning problem verifies the feasibility, applicability, and efficiency of the proposed method.

A Two-Timescale Duplex Neurodynamic Approach to Biconvex Optimization

This paper presents a two-timescale duplex neurodynamic system for constrained biconvex optimization. The two-timescale duplex neurodynamic system consists of two recurrent neural networks (RNNs) operating collaboratively at two timescales. By operating on two timescales, RNNs are able to avoid instability. In addition, based on the convergent states of the two RNNs, particle swarm optimization is used to optimize initial states of the RNNs to avoid local minima. It is proven that the proposed system is globally convergent to the global optimum with probability one. The performance of the two-timescale duplex neurodynamic system is substantiated based on the benchmark problems. Furthermore, the proposed system is applied for L₁ -constrained nonnegative matrix factorization.

Bounded Neural Network Control for Target Tracking of Underactuated Autonomous Surface Vehicles in the Presence of Uncertain Target Dynamics

This paper is concerned with the target tracking of underactuated autonomous surface vehicles with unknown dynamics and limited control torques. The velocity of the target is unknown, and only the measurements of line-of-sight range and angle are obtained. First, a kinematic control law is designed based on an extended state observer, which is utilized to estimate the uncertain target dynamics due to the unknown velocities. Next, an estimation model based on a single-hidden-layer neural network is developed to approximate the unknown follower dynamics induced by uncertain model parameters, unmodeled dynamics, and environmental disturbances. A bounded control law is designed based on the neural estimation model and a saturated function. The salient feature of the proposed controller is twofold. First, only the measured line-of-sight range and angle are used, and the velocity information of the target is not required. Second, the control torques are bounded with the bounds known as a priori. The input-to-state stability of the closed-loop system is analyzed via cascade theory. Simulations illustrate the effectiveness of the proposed bounded controller for tracking a moving target.

Mobile Robot Networks for Environmental Monitoring: A Cooperative Receding Horizon Temporal Logic Control Approach

This paper deals with the problem of environmental monitoring by designing and analyzing a cooperative receding horizon temporal logic (CRH-TL) control approach for mobile robot networks. First, a radial basis function network is used to model the distribution of environmental attributes in the monitored environment. On the basis of the established environment model, the problem of environmental monitoring can be formulated as a dynamical optimization problem. Second, an acceptable node set is obtained by enforcing appropriate constraints from linear temporal logic (LTL) specifications on the task of environmental monitoring. Third, by designing a cooperative energy function and using the acceptable node set, the CRH-TL control approach is proposed to generate the movement trajectory of each robot, which satisfies the given LTL specifications while guiding mobile robot networks to trace the peaks of environmental attributes. Finally, the effectiveness of the proposed CRH-TL control approach is illustrated for the problem of environmental monitoring.

A Collective Neurodynamic Optimization Approach to Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is an advanced method for nonnegative feature extraction, with widespread applications. However, the NMF solution often entails to solve a global optimization problem with a nonconvex objective function and nonnegativity constraints. This paper presents a collective neurodynamic optimization (CNO) approach to this challenging problem. The proposed collective neurodynamic system consists of a population of recurrent neural networks (RNNs) at the lower level and a particle swarm optimization (PSO) algorithm with wavelet mutation at the upper level. The RNNs act as search agents carrying out precise local searches according to their neurodynamics and initial conditions. The PSO algorithm coordinates and guides the RNNs with updated initial states toward global optimal solution(s). A wavelet mutation operator is added to enhance PSO exploration diversity. Through iterative interaction and improvement of the locally best solutions of RNNs and global best positions of the whole population, the population-based neurodynamic systems are almost sure able to achieve the global optimality for the NMF problem. It is proved that the convergence of the group-best state to the global optimal solution with probability one. The experimental results substantiate the efficacy and superiority of the CNO approach to bound-constrained global optimization with several benchmark nonconvex functions and NMF-based clustering with benchmark data sets in comparison with the state-of-the-art algorithms.

Adaptive Fuzzy Tracking Control of Nonlinear Systems With Asymmetric Actuator Backlash Based on a New Smooth Inverse

This paper is concentrated on the problem of adaptive fuzzy tracking control for an uncertain nonlinear system whose actuator is encountered by the asymmetric backlash behavior. First, we propose a new smooth inverse model which can approximate the asymmetric actuator backlash arbitrarily. By applying it, two adaptive fuzzy control scenarios, namely, the compensation-based control scheme and nonlinear decomposition-based control scheme, are then developed successively. It is worth noticing that the first fuzzy controller exhibits a better tracking control performance, although it recourses to a known slope ratio of backlash nonlinearity. The second one further removes the restriction, and also gets a desirable control performance. By the strict Lyapunov argument, both adaptive fuzzy controllers guarantee that the output tracking error is convergent to an adjustable region of zero asymptotically, while all the signals remain semiglobally uniformly ultimately bounded. Lastly, two comparative simulations are conducted to verify the effectiveness of the proposed fuzzy controllers.