A recurrent neural network for solving bilevel linear programming problem

A continuous-time neurodynamic approach in matrix form for rank minimization

This article proposes a continuous-time neurodynamic approach for solving the rank minimization under affine constraints. As opposed to the traditional neurodynamic approach, the proposed neurodynamic approach extends the form of the variables from the vector form to the matrix form. First, a continuous-time neurodynamic approach with variables in matrix form is developed by combining the optimal rank r projection and the gradient. Then, the optimality of the proposed neurodynamic approach is rigorously analyzed by demonstrating that the objective function satisfies the functional property which is called as (2r,4r)-restricted strong convexity and smoothness ((2r,4r)-RSCS). Furthermore, the convergence and stability analysis of the proposed neurodynamic approach is rigorously conducted by establishing appropriate Lyapunov functions and considering the relevant restricted isometry property (RIP) condition associated with the affine transformation. Finally, through experiments involving low-rank matrix recovery under affine transformations and the completion of low-rank real image, the effectiveness of this approach has been demonstrated, along with its superiority compared to the vector-based approach.

A Segmented Variable-Parameter ZNN for Dynamic Quadratic Minimization With Improved Convergence and Robustness

As a category of the recurrent neural network (RNN), zeroing neural network (ZNN) can effectively handle time-variant optimization issues. Compared with the fixed-parameter ZNN that needs to be adjusted frequently to achieve good performance, the conventional variable-parameter ZNN (VPZNN) does not require frequent adjustment, but its variable parameter will tend to infinity as time grows. Besides, the existing noise-tolerant ZNN model is not good enough to deal with time-varying noise. Therefore, a new-type segmented VPZNN (SVPZNN) for handling the dynamic quadratic minimization issue (DQMI) is presented in this work. Unlike the previous ZNNs, the SVPZNN includes an integral term and a nonlinear activation function, in addition to two specially constructed time-varying piecewise parameters. This structure keeps the time-varying parameters stable and makes the model have strong noise tolerance capability. Besides, theoretical analysis on SVPZNN is proposed to determine the upper bound of convergence time in the absence or presence of noise interference. Numerical simulations verify that SVPZNN has shorter convergence time and better robustness than existing ZNN models when handling DQMI.

A Fixed-Time Projection Neural Network for Solving L₁-Minimization Problem

In this article, a new projection neural network (PNN) for solving L₁ -minimization problem is proposed, which is based on classic PNN and sliding mode control technique. Furthermore, the proposed network can be used to make sparse signal reconstruction and image reconstruction. First, a sign function is introduced into the PNN model to design fixed-time PNN (FPNN). Then, under the condition that the projection matrix satisfies the restricted isometry property (RIP), the stability and fixed-time convergence of the proposed FPNN are proved by the Lyapunov method. Finally, based on the experimental results of signal simulation and image reconstruction, the proposed FPNN shows the effectiveness and superiority compared with that of the existing PNNs.

Hash Bit Selection Based on Collaborative Neurodynamic Optimization

Hash bit selection determines an optimal subset of hash bits from a candidate bit pool. It is formulated as a zero-one quadratic programming problem subject to binary and cardinality constraints. In this article, the problem is equivalently reformulated as a global optimization problem. A collaborative neurodynamic optimization (CNO) approach is applied to solve the problem by using a group of neurodynamic models initialized with particle swarm optimization iteratively in the CNO. Lévy mutation is used in the CNO to avoid premature convergence by ensuring initial state diversity. A theoretical proof is given to show that the CNO with the Lévy mutation operator is almost surely convergent to global optima. Experimental results are discussed to substantiate the efficacy and superiority of the CNO-based hash bit selection method to the existing methods on three benchmarks.

A New Settling-time Estimation Protocol to Finite-time Synchronization of Impulsive Memristor-Based Neural Networks

In this article, the issues of finite-time synchronization and finite-time adaptive synchronization for the impulsive memristive neural networks (IMNNs) with discontinuous activation functions (DAFs) and hybrid impulsive effects are probed into and elaborated on, where the stabilizing impulses (SIs), inactive impulses (IIs), and destabilizing impulses (DIs) are taken into account, respectively. Not resembling several earlier works, a more extensive range of impulses in the context of impulsive effects has been analyzed without using the known average impulsive interval strategy (AIIS). In light of the theories of differential inclusions and set-valued map, as well as impulsive control, new sufficient criteria with respect to the estimated settling time for synchronization of the related IMNNs are established using two types of switching control approaches, which sufficiently utilize information from not only the SIs, DIs, and DAFs but also the impulse sequences. Two simulation experiments are presented to the efficiency of the proposed results.

A Deterministic Annealing Neural Network Algorithm for the Minimum Concave Cost Transportation Problem

In this article, a deterministic annealing neural network algorithm is proposed to solve the minimum concave cost transportation problem. Specifically, the algorithm is derived from two neural network models and Lagrange-barrier functions. The Lagrange function is used to handle linear equality constraints, and the barrier function is used to force the solution to move to the global or near-global optimal solution. In both neural network models, two descent directions are constructed, and an iterative procedure for the optimization of the neural network is proposed. As a result, two corresponding Lyapunov functions are naturally obtained from these two descent directions. Furthermore, the proposed neural network models are proved to be completely stable and converge to the stable equilibrium state, therefore, the proposed algorithm converges. At last, the computer simulations on several test problems are made, and the results indicate that the proposed algorithm always generates global or near-global optimal solutions.

A Discrete-Time Neurodynamic Approach to Sparsity-Constrained Nonnegative Matrix Factorization

Sparsity is a desirable property in many nonnegative matrix factorization (NMF) applications. Although some level of sparseness of NMF solutions can be achieved by using regularization, the resulting sparsity depends highly on the regularization parameter to be valued in an ad hoc way. In this letter we formulate sparse NMF as a mixed-integer optimization problem with sparsity as binary constraints. A discrete-time projection neural network is developed for solving the formulated problem. Sufficient conditions for its stability and convergence are analytically characterized by using Lyapunov's method. Experimental results on sparse feature extraction are discussed to substantiate the superiority of this approach to extracting highly sparse features.

Adaptive Neural Output-Feedback Decentralized Control for Large-Scale Nonlinear Systems With Stochastic Disturbances

This paper addresses the problem of adaptive neural output-feedback decentralized control for a class of strongly interconnected nonlinear systems suffering stochastic disturbances. An state observer is designed to approximate the unmeasurable state signals. Using the approximation capability of radial basis function neural networks (NNs) and employing classic adaptive control strategy, an observer-based adaptive backstepping decentralized controller is developed. In the control design process, NNs are applied to model the uncertain nonlinear functions, and adaptive control and backstepping are combined to construct the controller. The developed control scheme can guarantee that all signals in the closed-loop systems are semiglobally uniformly ultimately bounded in fourth-moment. The simulation results demonstrate the effectiveness of the presented control scheme.

An approximation algorithm for graph partitioning via deterministic annealing neural network

Graph partitioning, a classical NP-hard combinatorial optimization problem, is widely applied to industrial or management problems. In this study, an approximated solution of the graph partitioning problem is obtained by using a deterministic annealing neural network algorithm. The algorithm is a continuation method that attempts to obtain a high-quality solution by following a path of minimum points of a barrier problem as the barrier parameter is reduced from a sufficiently large positive number to 0. With the barrier parameter assumed to be any positive number, one minimum solution of the barrier problem can be found by the algorithm in a feasible descent direction. With a globally convergent iterative procedure, the feasible descent direction could be obtained by renewing Lagrange multipliers red. A distinctive feature of it is that the upper and lower bounds on the variables will be automatically satisfied on the condition that the step length is a value from 0 to 1. Four well-known algorithms are compared with the proposed one on 100 test samples. Simulation results show effectiveness of the proposed algorithm.

A Discrete-Time Projection Neural Network for Sparse Signal Reconstruction With Application to Face Recognition

This paper deals with sparse signal reconstruction by designing a discrete-time projection neural network. Sparse signal reconstruction can be converted into an L₁ -minimization problem, which can also be changed into the unconstrained basis pursuit denoising problem. To solve the L₁ -minimization problem, an iterative algorithm is proposed based on the discrete-time projection neural network, and the global convergence of the algorithm is analyzed by using Lyapunov method. Experiments on sparse signal reconstruction and several popular face data sets are organized to illustrate the effectiveness and performance of the proposed algorithm. The experimental results show that the proposed algorithm is not only robust to different levels of sparsity and amplitude of signals and the noise pixels but also insensitive to the diverse values of scalar weight. Moreover, the value of the step size of the proposed algorithm is close to 1/2, thus a fast convergence rate is potentially possible. Furthermore, the proposed algorithm achieves better classification performance compared with some other algorithms for face recognition.

A Collaborative Neurodynamic Approach to Multiobjective Optimization

There are two ultimate goals in multiobjective optimization. The primary goal is to obtain a set of Pareto-optimal solutions while the secondary goal is to obtain evenly distributed solutions to characterize the efficient frontier. In this paper, a collaborative neurodynamic approach to multiobjective optimization is presented to attain both goals of Pareto optimality and solution diversity. The multiple objectives are first scalarized using a weighted Chebyshev function. Multiple projection neural networks are employed to search for Pareto-optimal solutions with the help of a particle swarm optimization (PSO) algorithm in reintialization. To diversify the Pareto-optimal solutions, a holistic approach is proposed by maximizing the hypervolume (HV) using again a PSO algorithm. The experimental results show that the proposed approach outperforms three other state-of-the-art multiobjective algorithms (i.e., HMOEA/D, MOEA/DD, and NSGAIII) most of times on 37 benchmark datasets in terms of HV and inverted generational distance.

A New Noise-Tolerant Obstacle Avoidance Scheme for Motion Planning of Redundant Robot Manipulators

Avoiding obstacle(s) is a challenging issue in the research of redundant robot manipulators. In addition, noise from truncation, rounding, and model uncertainty is an important factor that affects greatly the obstacle avoidance scheme. In this paper, based on the neural dynamics design formula, a new scheme with the pseudoinverse-type formulation is proposed for obstacle avoidance of redundant robot manipulators in a noisy environment. Such a scheme has the capability of suppressing constant and bounded time-varying noises, and it is thus termed as the noise-tolerant obstacle avoidance (NTOA) scheme in this paper. Theoretical results are also given to show the excellent property of the proposed NTOA scheme (particularly in noise situation). Based on a PA10 robot manipulator with point and window-shaped obstacles, computer simulation results are presented to further substantiate the efficacy and superiority of the proposed NTOA scheme for motion planning of redundant robot manipulators.

A nonnegative matrix factorization algorithm based on a discrete-time projection neural network

This paper presents an algorithm for nonnegative matrix factorization based on a biconvex optimization formulation. First, a discrete-time projection neural network is introduced. An upper bound of its step size is derived to guarantee the stability of the neural network. Then, an algorithm is proposed based on the discrete-time projection neural network and a backtracking step-size adaptation. The proposed algorithm is proven to be able to reduce the objective function value iteratively until attaining a partial optimum of the formulated biconvex optimization problem. Experimental results based on various data sets are presented to substantiate the efficacy of the algorithm.

A Neurodynamic Optimization Approach to Bilevel Quadratic Programming

This paper presents a neurodynamic optimization approach to bilevel quadratic programming (BQP). Based on the Karush-Kuhn-Tucker (KKT) theorem, the BQP problem is reduced to a one-level mathematical program subject to complementarity constraints (MPCC). It is proved that the global solution of the MPCC is the minimal one of the optimal solutions to multiple convex optimization subproblems. A recurrent neural network is developed for solving these convex optimization subproblems. From any initial state, the state of the proposed neural network is convergent to an equilibrium point of the neural network, which is just the optimal solution of the convex optimization subproblem. Compared with existing recurrent neural networks for BQP, the proposed neural network is guaranteed for delivering the exact optimal solutions to any convex BQP problems. Moreover, it is proved that the proposed neural network for bilevel linear programming is convergent to an equilibrium point in finite time. Finally, three numerical examples are elaborated to substantiate the efficacy of the proposed approach.

Synchronization of Switched Neural Networks With Communication Delays via the Event-Triggered Control

This paper addresses the issue of synchronization of switched delayed neural networks with communication delays via event-triggered control. For synchronizing coupled switched neural networks, we propose a novel event-triggered control law which could greatly reduce the number of control updates for synchronization tasks of coupled switched neural networks involving embedded microprocessors with limited on-board resources. The control signals are driven by properly defined events, which depend on the measurement errors and current-sampled states. By using a delay system method, a novel model of synchronization error system with delays is proposed with the communication delays and event-triggered control in the unified framework for coupled switched neural networks. The criteria are derived for the event-triggered synchronization analysis and control synthesis of switched neural networks via the Lyapunov-Krasovskii functional method and free weighting matrix approach. A numerical example is elaborated on to illustrate the effectiveness of the derived results.

An Inertial Projection Neural Network for Solving Variational Inequalities

Recently, projection neural network (PNN) was proposed for solving monotone variational inequalities (VIs) and related convex optimization problems. In this paper, considering the inertial term into first order PNNs, an inertial PNN (IPNN) is also proposed for solving VIs. Under certain conditions, the IPNN is proved to be stable, and can be applied to solve a broader class of constrained optimization problems related to VIs. Compared with existing neural networks (NNs), the presence of the inertial term allows us to overcome some drawbacks of many NNs, which are constructed based on the steepest descent method, and this model is more convenient for exploring different Karush-Kuhn-Tucker optimal solution for nonconvex optimization problems. Finally, simulation results on three numerical examples show the effectiveness and performance of the proposed NN.