Alternative continuous- and discrete-time neural networks for image restoration

This paper presents alternative continuous- and discrete-time neural networks for image restoration in real time by introducing new vectors and transforming its optimization conditions into a system of double projection equations. The proposed neural networks are shown to be stable in the sense of Lyapunov and convergent for any starting point. Compared with the existing neural networks for image restoration, the proposed models have the least neurons, a one-layer structure and the faster convergence, and is suitable to parallel implementation. The validity and transient behaviour of the proposed neural network is demonstrated by numerical examples.

A generalized least absolute deviation method for parameter estimation of autoregressive signals

This paper proposes a generalized least absolute deviation (GLAD) method for parameter estimation of autoregressive (AR) signals under non-Gaussian noise environments. The proposed GLAD method can improve the accuracy of the estimation of the conventional least absolute deviation (LAD) method by minimizing a new cost function with parameter variables and noise error variables. Compared with second- and high-order statistical methods, the proposed GLAD method can obtain robustly an optimal AR parameter estimation without requiring the measurement noise to be Gaussian. Moreover, the proposed GLAD method can be implemented by a cooperative neural network (NN) which is shown to converge globally to the optimal AR parameter estimation within a finite time. Simulation results show that the proposed GLAD method can obtain more accurate estimates than several well-known estimation methods in the presence of different noise distributions.

Constrained least absolute deviation neural networks

It is well known that least absolute deviation (LAD) criterion or L(1)-norm used for estimation of parameters is characterized by robustness, i.e., the estimated parameters are totally resistant (insensitive) to large changes in the sampled data. This is an extremely useful feature, especially, when the sampled data are known to be contaminated by occasionally occurring outliers or by spiky noise. In our previous works, we have proposed the least absolute deviation neural network (LADNN) to solve unconstrained LAD problems. The theoretical proofs and numerical simulations have shown that the LADNN is Lyapunov-stable and it can globally converge to the exact solution to a given unconstrained LAD problem. We have also demonstrated its excellent application value in time-delay estimation. More generally, a practical LAD application problem may contain some linear constraints, such as a set of equalities and/or inequalities, which is called constrained LAD problem, whereas the unconstrained LAD can be considered as a special form of the constrained LAD. In this paper, we present a new neural network called constrained least absolute deviation neural network (CLADNN) to solve general constrained LAD problems. Theoretical proofs and numerical simulations demonstrate that the proposed CLADNN is Lyapunov stable and globally converges to the exact solution to a given constrained LAD problem, independent of initial values. The numerical simulations have also illustrated that the proposed CLADNN can be used to robustly estimate parameters for nonlinear curve fitting, which is extensively used in signal and image processing.

Cooperative recurrent modular neural networks for constrained optimization: a survey of models and applications

Constrained optimization problems arise in a wide variety of scientific and engineering applications. Since several single recurrent neural networks when applied to solve constrained optimization problems for real-time engineering applications have shown some limitations, cooperative recurrent neural network approaches have been developed to overcome drawbacks of these single recurrent neural networks. This paper surveys in details work on cooperative recurrent neural networks for solving constrained optimization problems and their engineering applications, and points out their standing models from viewpoint of both convergence to the optimal solution and model complexity. We provide examples and comparisons to shown advantages of these models in the given applications.

Robust time delay estimation of bioelectric signals using least absolute deviation neural network

The time delay estimation (TDE) is an important issue in modern signal processing and it has found extensive applications in the spatial propagation feature extraction of biomedical signals as well. Due to the extreme complexity and variability of the underlying systems, biomedical signals are usually nonstationary, unstable and even chaotic. Furthermore, due to the limitations of the measurement environments, biomedical signals are often noise-contaminated. Therefore, the TDE of biomedical signals is a challenging issue. A new TDE algorithm based on the least absolute deviation neural network (LADNN) and its application experiments are presented in this paper. The LADNN is the neural implementation of the least absolute deviation (LAD) optimization model, also called unconstrained minimum L1-norm model, with a theoretically proven global convergence. In the proposed LADNN-based TDE algorithm, a given signal is modeled using the moving average (MA) model. The MA parameters are estimated by using the LADNN and the time delay corresponds to the time index at which the MA coefficients have a peak. Due to the excellent features of L1-norm model superior to Lp-norm (p > 1) models in non-Gaussian noise environments or even in chaos, especially for signals that contain sharp transitions (such as biomedical signals with spiky series or motion artifacts) or chaotic dynamic processes, the LADNN-based TDE is more robust than the existing TDE algorithms based on wavelet-domain correlation and those based on higher-order spectra (HOS). Unlike these conventional methods, especially the current state-of-the-art HOS-based TDE, the LADNN-based method is free of the assumption that the signal is non-Gaussian and the noises are Gaussian and, thus, it is more applicable in real situations. Simulation experiments under three different noise environments, Gaussian, non-Gaussian and chaotic, are conducted to compare the proposed TDE method with the existing HOS-based method. Real application experiment is conducted to extract time delay information between every two adjacent channels of gastric myoelectrical activity (GMA) to assess the spatial propagation characteristics of GMA during different phases of the migrating myoelectrical complex (MMC).