Discrete-Time Zhang Neural Network for Online Time-Varying Nonlinear Optimization With Application to Manipulator Motion Generation

An Accelerated Approach on Adaptive Gradient Neural Network for Solving Time-Dependent Linear Equations: A State-Triggered Perspective

To improve the acceleration performance, a hybrid state-triggered discretization (HSTD) is proposed for the adaptive gradient neural network (AGNN) for solving time-dependent linear equations (TDLEs). Unlike the existing approaches that use an activation function or a time-varying coefficient for acceleration, the proposed HSTD is uniquely designed from a control theory perspective. It comprises two essential components: adaptive sampling interval state-triggered discretization (ASISTD) and adaptive coefficient state-triggered discretization (ACSTD). The former addresses the gap in acceleration methods related to the variable sampling period, while the latter considers the underlying evolutionary dynamics of the Lyapunov function to determine coefficients greedily. Finally, compared with commonly used discretization methods, the acceleration performance and computational advantages of the proposed HSTD are substantiated by the numerical simulations and applications to robotics.

A novel discrete zeroing neural network for online solving time-varying nonlinear optimization problems

To reduce transportation time, a discrete zeroing neural network (DZNN) method is proposed to solve the shortest path planning problem with a single starting point and a single target point. The shortest path planning problem is reformulated as an optimization problem, and a discrete nonlinear function related to the energy function is established so that the lowest-energy state corresponds to the optimal path solution. Theoretical analyzes demonstrate that the discrete ZNN model (DZNNM) exhibits zero stability, effectiveness, and real-time performance in handling time-varying nonlinear optimization problems (TVNOPs). Simulations with various parameters confirm the efficiency and real-time performance of the developed DZNNM for TVNOPs, indicating its suitability and superiority for solving the shortest path planning problem in real time.

Explicit Linear Left-and-Right 5-Step Formulas With Zeroing Neural Network for Time-Varying Applications

In this article, being different from conventional time-discretization (simply called discretization) formulas, explicit linear left-and-right 5-step (ELLR5S) formulas with sixth-order precision are proposed. The general sixth-order ELLR5S formula with four variable parameters is developed first, and constraints of these four parameters are displayed to guarantee the zero stability, consistence, and convergence of the formula. Then, by choosing specific parameter values within constraints, eight specific sixth-order ELLR5S formulas are developed. The general sixth-order ELLR5S formula is further utilized to generate discrete zeroing neural network (DZNN) models for solving time-varying linear and nonlinear systems. For comparison, three conventional discretization formulas are also utilized. Theoretical analyses are presented to show the performance of ELLR5S formulas and DZNN models. Furthermore, abundant experiments, including three practical applications, that is, angle-of-arrival (AoA) localization and two redundant manipulators (PUMA560 manipulator and Kinova manipulator) control, are conducted. The synthesized results substantiate the efficacy and superiority of sixth-order ELLR5S formulas as well as the corresponding DZNN models.

Achieving Online Regression Performance of LSTMs With Simple RNNs

Recurrent neural networks (RNNs) are widely used for online regression due to their ability to generalize nonlinear temporal dependencies. As an RNN model, long short-term memory networks (LSTMs) are commonly preferred in practice, as these networks are capable of learning long-term dependencies while avoiding the vanishing gradient problem. However, due to their large number of parameters, training LSTMs requires considerably longer training time compared to simple RNNs (SRNNs). In this article, we achieve the online regression performance of LSTMs with SRNNs efficiently. To this end, we introduce a first-order training algorithm with a linear time complexity in the number of parameters. We show that when SRNNs are trained with our algorithm, they provide very similar regression performance with the LSTMs in two to three times shorter training time. We provide strong theoretical analysis to support our experimental results by providing regret bounds on the convergence rate of our algorithm. Through an extensive set of experiments, we verify our theoretical work and demonstrate significant performance improvements of our algorithm with respect to LSTMs and the other state-of-the-art learning models.

Inverse-Free Discrete ZNN Models Solving for Future Matrix Pseudoinverse via Combination of Extrapolation and ZeaD Formulas

Time-varying matrix pseudoinverse (TVMP) problem has been investigated by many researchers in recent years, but a new class of matrix termed Zhang matrix has been found and not been handled by some conventional models, e.g., Getz-Marsden dynamic model. On the other way, future matrix pseudoinverse (FMP), as a more challenging and intractable discrete-time problem, deserves more attention due to its significant role-playing on some engineering applications, such as redundant manipulator. Based on the zeroing neural network (ZNN), this article concentrates on designing new discrete ZNN models appropriately for computing the FMPs of all matrices of full rank, including the Zhang matrix. First, an inverse-free continuous ZNN model for computing TVMP is derived. Subsequently, Zhang et al. discretization (ZeaD) formulas and equidistant extrapolation formulas are used to discretize the continuous ZNN model to two discrete ZNN models for computing FMPs with different truncation errors. The numerical experiments are conducted for the five conventional discrete models and two new discrete ZNN models. Distinct numerical results substantiate the effectiveness and choiceness of newly proposed models. Finally, one of the newly proposed models is implemented on simulating and physical instances of robot manipulators, respectively, to show its practicability.

Unified Model Solving Nine Types of Time-Varying Problems in the Frame of Zeroing Neural Network

Many time-varying problems have been solved using the zeroing neural network proposed by Zhang et al. In this article, nine types of time-varying problems, namely time-varying nonlinear equation system, time-varying linear equation system, time-varying convex nonlinear optimization under linear equalities, unconstrained time-varying convex nonlinear optimization, time-varying convex quadratic programming under linear equalities, unconstrained time-varying convex quadratic programming, time-varying nonlinear inequality system, time-varying linear inequality system, and time-varying division, are investigated to better understand the essence of zeroing neutral network. Discrete-form time-varying problems are studied by considering the nature of unknown future and the requirement of real-time computation for time-varying problems. A unified model is proposed in the frame of zeroing neural network to uniformly solve these time-varying problems on the basis of their connections and a newly developed discretization formula. Theoretical analyses and numerical experiments, including the tracking control of PUMA560 robot manipulator, verify the effectiveness and precision of the proposed unified model.

Design and Comprehensive Analysis of a Noise-Tolerant ZNN Model With Limited-Time Convergence for Time-Dependent Nonlinear Minimization

Zeroing neural network (ZNN) is a powerful tool to address the mathematical and optimization problems broadly arisen in the science and engineering areas. The convergence and robustness are always co-pursued in ZNN. However, there exists no related work on the ZNN for time-dependent nonlinear minimization that achieves simultaneously limited-time convergence and inherently noise suppression. In this article, for the purpose of satisfying such two requirements, a limited-time robust neural network (LTRNN) is devised and presented to solve time-dependent nonlinear minimization under various external disturbances. Different from the previous ZNN model for this problem either with limited-time convergence or with noise suppression, the proposed LTRNN model simultaneously possesses such two characteristics. Besides, rigorous theoretical analyses are given to prove the superior performance of the LTRNN model when adopted to solve time-dependent nonlinear minimization under external disturbances. Comparative results also substantiate the effectiveness and advantages of LTRNN via solving a time-dependent nonlinear minimization problem.

RNN for Perturbed Manipulability Optimization of Manipulators Based on a Distributed Scheme: A Game-Theoretic Perspective

In order to leverage the unique advantages of redundant manipulators, avoiding the singularity during motion planning and control should be considered as a fundamental issue to handle. In this article, a distributed scheme is proposed to improve the manipulability of redundant manipulators in a group. To this end, the manipulability index is incorporated into the cooperative control of multiple manipulators in a distributed network, which is used to guide manipulators to adjust to the optimal spatial position. Moreover, from the perspective of game theory, this article formulates the problem into a Nash equilibrium. Then, a neural network with anti-noise ability is constructed to seek and approximate the optimal strategy profile of the Nash equilibrium problem with time-varying parameters. Theoretical analyses show that the neural network model has the superior global convergence and noise immunity. Finally, simulation results demonstrate that the neural network is effective in real-time cooperative motion generation of multiple redundant manipulators under perturbations in distributed networks.

New Varying-Parameter ZNN Models With Finite-Time Convergence and Noise Suppression for Time-Varying Matrix Moore-Penrose Inversion

This article aims to solve the Moore-Penrose inverse of time-varying full-rank matrices in the presence of various noises in real time. For this purpose, two varying-parameter zeroing neural networks (VPZNNs) are proposed. Specifically, VPZNN-R and VPZNN-L models, which are based on a new design formula, are designed to solve the right and left Moore-Penrose inversion problems of time-varying full-rank matrices, respectively. The two VPZNN models are activated by two novel varying-parameter nonlinear activation functions. Detailed theoretical derivations are presented to show the desired finite-time convergence and outstanding robustness of the proposed VPZNN models under various kinds of noises. In addition, existing neural models, such as the original ZNN (OZNN) and the integration-enhanced ZNN (IEZNN), are compared with the VPZNN models. Simulation observations verify the advantages of the VPZNN models over the OZNN and IEZNN models in terms of convergence and robustness. The potential of the VPZNN models for robotic applications is then illustrated by an example of robot path tracking.

New Discrete-Time ZNN Models for Least-Squares Solution of Dynamic Linear Equation System With Time-Varying Rank-Deficient Coefficient

In this brief, a new one-step-ahead numerical differentiation rule called six-instant -cube finite difference (6I CFD) formula is proposed for the first-order derivative approximation with higher precision than existing finite difference formulas (i.e., Euler and Taylor types). Subsequently, by exploiting the proposed 6I CFD formula to discretize the continuous-time Zhang neural network model, two new-type discrete-time ZNN (DTZNN) models, namely, new-type DTZNNK and DTZNNU models, are designed and generalized to compute the least-squares solution of dynamic linear equation system with time-varying rank-deficient coefficient in real time, which is quite different from the existing ZNN-related studies on solving continuous-time and discrete-time (dynamic or static) linear equation systems in the context of full-rank coefficients. Specifically, the corresponding dynamic normal equation system, of which the solution exactly corresponds to the least-squares solution of dynamic linear equation system, is elegantly introduced to solve such a rank-deficient least-squares problem efficiently and accurately. Theoretical analyses show that the maximal steady-state residual errors of the two new-type DTZNN models have an pattern, where denotes the sampling gap. Comparative numerical experimental results further substantiate the superior computational performance of the new-type DTZNN models to solve the rank-deficient least-squares problem of dynamic linear equation systems.

Robust Zeroing Neural-Dynamics and Its Time-Varying Disturbances Suppression Model Applied to Mobile Robot Manipulators

This paper proposes a novel robust zeroing neural-dynamics (RZND) approach as well as its associated model for solving the inverse kinematics problem of mobile robot manipulators. Unlike existing works based on the assumption that neural network models are free of external disturbances, four common forms of time-varying disturbances suppressed by the proposed RZND model are investigated in this paper. In addition, theoretical analyses on the antidisturbance performance are presented in detail to prove the effectiveness and robustness of the proposed RZND model with time-varying disturbances suppressed for solving the inverse kinematics problem of mobile robot manipulators. That is, the RZND model converges toward the exact solution of the inverse kinematics problem of mobile robot manipulators with bounded or zero-oriented steady-state position error. Moreover, simulation studies and comprehensive comparisons with existing neural network models, e.g., the conventional Zhang neural network model and the gradient-based recurrent neural network model, together with extensive tests with four common forms of time-varying disturbances substantiate the efficacy, robustness, and superiority of the proposed RZND approach as well as its time-varying disturbances suppression model for solving the inverse kinematics problem of mobile robot manipulators.

Design, Analysis, and Representation of Novel Five-Step DTZD Algorithm for Time-Varying Nonlinear Optimization

Continuous-time and discrete-time forms of Zhang dynamics (ZD) for time-varying nonlinear optimization have been developed recently. In this paper, a novel discrete-time ZD (DTZD) algorithm is proposed and investigated based on the previous research. Specifically, the DTZD algorithm for time-varying nonlinear optimization is developed by adopting a new Taylor-type difference rule. This algorithm is a five-step iteration process, and thus, is referred to as the five-step DTZD algorithm in this paper. Theoretical analysis and results of the proposed five-step DTZD algorithm are presented to highlight its excellent computational performance. The geometric representation of the proposed algorithm for time-varying nonlinear optimization is also provided. Comparative numerical results are illustrated with four examples to substantiate the efficacy and superiority of the proposed five-step DTZD algorithm for time-varying nonlinear optimization compared with the previous DTZD algorithms.

Robustness Analyses and Optimal Sampling Gap of Recurrent Neural Network for Dynamic Matrix Pseudoinversion

This study analyses the robustness and convergence characteristics of a neural network. First, a special class of recurrent neural network (RNN), termed a continuous-time Zhang neural network (CTZNN) model, is presented and investigated for dynamic matrix pseudoinversion. Theoretical analysis of the CTZNN model demonstrates that it has good robustness against various types of noise. In addition, considering the requirements of digital implementation and online computation, the optimal sampling gap for a discrete-time Zhang neural network (DTZNN) model under noisy environments is proposed. Finally, experimental results are presented, which further substantiate the theoretical analyses and demonstrate the effectiveness of the proposed ZNN models for computing a dynamic matrix pseudoinverse under noisy environments.

Integration-Enhanced Zhang Neural Network for Real-Time-Varying Matrix Inversion in the Presence of Various Kinds of Noises

Matrix inversion often arises in the fields of science and engineering. Many models for matrix inversion usually assume that the solving process is free of noises or that the denoising has been conducted before the computation. However, time is precious for the real-time-varying matrix inversion in practice, and any preprocessing for noise reduction may consume extra time, possibly violating the requirement of real-time computation. Therefore, a new model for time-varying matrix inversion that is able to handle simultaneously the noises is urgently needed. In this paper, an integration-enhanced Zhang neural network (IEZNN) model is first proposed and investigated for real-time-varying matrix inversion. Then, the conventional ZNN model and the gradient neural network model are presented and employed for comparison. In addition, theoretical analyses show that the proposed IEZNN model has the global exponential convergence property. Moreover, in the presence of various kinds of noises, the proposed IEZNN model is proven to have an improved performance. That is, the proposed IEZNN model converges to the theoretical solution of the time-varying matrix inversion problem no matter how large the matrix-form constant noise is, and the residual errors of the proposed IEZNN model can be arbitrarily small for time-varying noises and random noises. Finally, three illustrative simulation examples, including an application to the inverse kinematic motion planning of a robot manipulator, are provided and analyzed to substantiate the efficacy and superiority of the proposed IEZNN model for real-time-varying matrix inversion.