Constrained least absolute deviation neural networks

Novel Continuous- and Discrete-Time Neural Networks for Solving Quadratic Minimax Problems With Linear Equality Constraints

This article presents two novel continuous- and discrete-time neural networks (NNs) for solving quadratic minimax problems with linear equality constraints. These two NNs are established based on the conditions of the saddle point of the underlying function. For the two NNs, a proper Lyapunov function is constructed so that they are stable in the sense of Lyapunov, and will converge to some saddle point(s) for any starting point under some mild conditions. Compared with the existing NNs for solving quadratic minimax problems, the proposed NNs require weaker stability conditions. The validity and transient behavior of the proposed models are illustrated by some simulation results.

Two Recurrent Neural Networks With Reduced Model Complexity for Constrained l₁-Norm Optimization

Because of the robustness and sparsity performance of least absolute deviation (LAD or l1 ) optimization, developing effective solution methods becomes an important topic. Recurrent neural networks (RNNs) are reported to be capable of effectively solving constrained l1 -norm optimization problems, but their convergence speed is limited. To accelerate the convergence, this article introduces two RNNs, in form of continuous- and discrete-time systems, for solving l1 -norm optimization problems with linear equality and inequality constraints. The RNNs are theoretically proven to be globally convergent to optimal solutions without any condition. With reduced model complexity, the two RNNs can significantly expedite constrained l1 -norm optimization. Numerical simulation results show that the two RNNs spend much less computational time than related RNNs and numerical optimization algorithms for linearly constrained l1 -norm optimization.

Improved Image Quality for Static BLADE Magnetic Resonance Imaging Using the Total-Variation Regularized Least Absolute Deviation Solver

In order to improve the image quality of BLADE magnetic resonance imaging (MRI) using the index tensor solvers and to evaluate MRI image quality in a clinical setting, we implemented BLADE MRI reconstructions using two tensor solvers (the least-squares solver and the L1 total-variation regularized least absolute deviation (L1TV-LAD) solver) on a graphics processing unit (GPU). The BLADE raw data were prospectively acquired and presented in random order before being assessed by two independent radiologists. Evaluation scores were examined for consistency and then by repeated measures analysis of variance (ANOVA) to identify the superior algorithm. The simulation showed the structural similarity index (SSIM) of various tensor solvers ranged between 0.995 and 0.999. Inter-reader reliability was high (Intraclass correlation coefficient (ICC) = 0.845, 95% confidence interval: 0.817, 0.87). The image score of L1TV-LAD was significantly higher than that of vendor-provided image and the least-squares method. The image score of the least-squares method was significantly lower than that of the vendor-provided image. No significance was identified in L1TV-LAD with a regularization strength of λ= 0.4–1.0. The L1TV-LAD with a regularization strength of λ= 0.4–0.7 was found consistently better than least-squares and vendor-provided reconstruction in BLADE MRI with a SENSitivity Encoding (SENSE) factor of 2. This warrants further development of the integrated computing system with the scanner.

Alternative continuous- and discrete-time neural networks for image restoration

This paper presents alternative continuous- and discrete-time neural networks for image restoration in real time by introducing new vectors and transforming its optimization conditions into a system of double projection equations. The proposed neural networks are shown to be stable in the sense of Lyapunov and convergent for any starting point. Compared with the existing neural networks for image restoration, the proposed models have the least neurons, a one-layer structure and the faster convergence, and is suitable to parallel implementation. The validity and transient behaviour of the proposed neural network is demonstrated by numerical examples.

Discontinuous Neural Networks for Finite-Time Solution of Time-Dependent Linear Equations

This paper considers a class of nonsmooth neural networks with discontinuous hard-limiter (signum) neuron activations for solving time-dependent (TD) systems of algebraic linear equations (ALEs). The networks are defined by the subdifferential with respect to the state variables of an energy function given by the L ₁ norm of the error between the state and the TD-ALE solution. It is shown that when the penalty parameter exceeds a quantitatively estimated threshold the networks are able to reach in finite time, and exactly track thereafter, the target solution of the TD-ALE. Furthermore, this paper discusses the tightness of the estimated threshold and also points out key differences in the role played by this threshold with respect to networks for solving time-invariant ALEs. It is also shown that these convergence results are robust with respect to small perturbations of the neuron interconnection matrices. The dynamics of the proposed networks are rigorously studied by using tools from nonsmooth analysis, the concept of subdifferential of convex functions, and that of solutions in the sense of Filippov of dynamical systems with discontinuous nonlinearities.

Nonlinear identification with local model networks using GTLS techniques and equality constraints

Local model networks approximate a nonlinear system through multiple local models fitted within a partition space. The main advantage of this approach is that the identification of complex nonlinear processes is alleviated by the integration of structured knowledge about the process. This paper extends these concepts by the integration of quantitative process knowledge into the identification procedure. Quantitative knowledge describes explicit dependences between inputs and outputs and is integrated in the parameter estimation process by means of equality constraints. For this purpose, a constrained generalized total least squares algorithm for local parameter estimation is presented. Furthermore, the problem of proper integration of constraints in the partitioning process is treated where an expectation-maximization procedure is combined with constrained parameter estimation. The benefits and the applicability of the proposed concepts are demonstrated by means of two illustrative examples and a practical application using real measurement data.