A discrete-time learning algorithm for image restoration using a novel L2-norm noise constrained estimation

Submission to Special Issue to Explainable Representation Learning-Based Intelligent Inspection and Maintenance of Complex Systems: Synchronization of Inertial Neural Networks With Unbounded Delays via Sampled-Data Control

This article addresses the synchronization issue for inertial neural networks (INNs) with heterogeneous time-varying delays and unbounded distributed delays, in which the state quantization is considered. First, by fully considering the delay and sampling time point information, a modified looped-functional is proposed for the synchronization error system. Compared with the existing Lyapunov-Krasovskii functional (LKF), the proposed functional contains the sawtooth structure term V8(t) and the time-varying terms ex(t-βħ (t)) and ey(t-βħ (t)) . Then, the obtained constraints may be further relaxed. Based on the functional and integral inequality, less conservative synchronization criteria are derived as the basis of controller design. In addition, the required quantized sampled-data controller is proposed by solving a set of linear matrix inequalities. Finally, two numerical examples are given to show the effectiveness and superiority of the proposed scheme in this article.

Insights Into Algorithms for Separable Nonlinear Least Squares Problems

Separable nonlinear least squares (SNLLS) problems have attracted interest in a wide range of research fields such as machine learning, computer vision, and signal processing. During the past few decades, several algorithms, including the joint optimization algorithm, alternated least squares (ALS) algorithm, embedded point iterations (EPI) algorithm, and variable projection (VP) algorithms, have been employed for solving SNLLS problems in the literature. The VP approach has been proven to be quite valuable for SNLLS problems and the EPI method has been successful in solving many computer vision tasks. However, no clear explanations about the intrinsic relationships of these algorithms have been provided in the literature. In this paper, we give some insights into these algorithms for SNLLS problems. We derive the relationships among different forms of the VP algorithms, EPI algorithm and ALS algorithm. In addition, the convergence and robustness of some algorithms are investigated. Moreover, the analysis of the VP algorithm generates a negative answer to Kaufman's conjecture. Numerical experiments on the image restoration task, fitting the time series data using the radial basis function network based autoregressive (RBF-AR) model, and bundle adjustment are given to compare the performance of different algorithms.

Deep Learning Applications with Practical Measured Results in Electronics Industries

This editorial introduces the Special Issue, entitled “Deep Learning Applications with Practical Measured Results in Electronics Industries”, of Electronics. Topics covered in this issue include four main parts: (I) environmental information analyses and predictions, (II) unmanned aerial vehicle (UAV) and object tracking applications, (III) measurement and denoising techniques, and (IV) recommendation systems and education systems. Four papers on environmental information analyses and predictions are as follows: (1) “A Data-Driven Short-Term Forecasting Model for Offshore Wind Speed Prediction Based on Computational Intelligence” by Panapakidis et al.; (2) “Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting” by Wan et al.; (3) “Modeling and Analysis of Adaptive Temperature Compensation for Humidity Sensors” by Xu et al.; (4) “An Image Compression Method for Video Surveillance System in Underground Mines Based on Residual Networks and Discrete Wavelet Transform” by Zhang et al. Three papers on UAV and object tracking applications are as follows: (1) “Trajectory Planning Algorithm of UAV Based on System Positioning Accuracy Constraints” by Zhou et al.; (2) “OTL-Classifier: Towards Imaging Processing for Future Unmanned Overhead Transmission Line Maintenance” by Zhang et al.; (3) “Model Update Strategies about Object Tracking: A State of the Art Review” by Wang et al. Five papers on measurement and denoising techniques are as follows: (1) “Characterization and Correction of the Geometric Errors in Using Confocal Microscope for Extended Topography Measurement. Part I: Models, Algorithms Development and Validation” by Wang et al.; (2) “Characterization and Correction of the Geometric Errors Using a Confocal Microscope for Extended Topography Measurement, Part II: Experimental Study and Uncertainty Evaluation” by Wang et al.; (3) “Deep Transfer HSI Classification Method Based on Information Measure and Optimal Neighborhood Noise Reduction” by Lin et al.; (4) “Quality Assessment of Tire Shearography Images via Ensemble Hybrid Faster Region-Based ConvNets” by Chang et al.; (5) “High-Resolution Image Inpainting Based on Multi-Scale Neural Network” by Sun et al. Two papers on recommendation systems and education systems are as follows: (1) “Deep Learning-Enhanced Framework for Performance Evaluation of a Recommending Interface with Varied Recommendation Position and Intensity Based on Eye-Tracking Equipment Data Processing” by Sulikowski et al. and (2) “Generative Adversarial Network Based Neural Audio Caption Model for Oral Evaluation” by Zhang et al.

Robust Regression Estimation Based on Low-Dimensional Recurrent Neural Networks

The robust Huber's M-estimator is widely used in signal and image processing, classification, and regression. From an optimization point of view, Huber's M-estimation problem is often formulated as a large-sized quadratic programming (QP) problem in view of its nonsmooth cost function. This paper presents a generalized regression estimator which minimizes a reduced-sized QP problem. The generalized regression estimator may be viewed as a significant generalization of several robust regression estimators including Huber's M-estimator. The performance of the generalized regression estimator is analyzed in terms of robustness and approximation accuracy. Furthermore, two low-dimensional recurrent neural networks (RNNs) are introduced for robust estimation. The two RNNs have low model complexity and enhanced computational efficiency. Finally, the experimental results of two examples and an application to image restoration are presented to substantiate superior performance of the proposed method over conventional algorithms for robust regression estimation in terms of approximation accuracy and convergence rate.

Discriminative Structured Dictionary Learning on Grassmann Manifolds and Its Application on Image Restoration

Image restoration is a difficult and challenging problem in various imaging applications. However, despite of the benefits of a single overcomplete dictionary, there are still several challenges for capturing the geometric structure of image of interest. To more accurately represent the local structures of the underlying signals, we propose a new problem formulation for sparse representation with block-orthogonal constraint. There are three contributions. First, a framework for discriminative structured dictionary learning is proposed, which leads to a smooth manifold structure and quotient search spaces. Second, an alternating minimization scheme is proposed after taking both the cost function and the constraints into account. This is achieved by iteratively alternating between updating the block structure of the dictionary defined on Grassmann manifold and sparsifying the dictionary atoms automatically. Third, Riemannian conjugate gradient is considered to track local subspaces efficiently with a convergence guarantee. Extensive experiments on various datasets demonstrate that the proposed method outperforms the state-of-the-art methods on the removal of mixed Gaussian-impulse noise.

Energy-to-Peak State Estimation for Static Neural Networks With Interval Time-Varying Delays

This paper is concerned with energy-to-peak state estimation on static neural networks (SNNs) with interval time-varying delays. The objective is to design suitable delay-dependent state estimators such that the peak value of the estimation error state can be minimized for all disturbances with bounded energy. Note that the Lyapunov-Krasovskii functional (LKF) method plus proper integral inequalities provides a powerful tool in stability analysis and state estimation of delayed NNs. The main contribution of this paper lies in three points: 1) the relationship between two integral inequalities based on orthogonal and nonorthogonal polynomial sequences is disclosed. It is proven that the second-order Bessel-Legendre inequality (BLI), which is based on an orthogonal polynomial sequence, outperforms the second-order integral inequality recently established based on a nonorthogonal polynomial sequence; 2) the LKF method together with the second-order BLI is employed to derive some novel sufficient conditions such that the resulting estimation error system is globally asymptotically stable with desirable energy-to-peak performance, in which two types of time-varying delays are considered, allowing its derivative information is partly known or totally unknown; and 3) a linear-matrix-inequality-based approach is presented to design energy-to-peak state estimators for SNNs with two types of time-varying delays, whose efficiency is demonstrated via two widely studied numerical examples.

Hierarchical Type Stability Criteria for Delayed Neural Networks via Canonical Bessel-Legendre Inequalities

This paper is concerned with global asymptotic stability of delayed neural networks. Notice that a Bessel-Legendre inequality plays a key role in deriving less conservative stability criteria for delayed neural networks. However, this inequality is in the form of Legendre polynomials and the integral interval is fixed on . As a result, the application scope of the Bessel-Legendre inequality is limited. This paper aims to develop the Bessel-Legendre inequality method so that less conservative stability criteria are expected. First, by introducing a canonical orthogonal polynomial sequel, a canonical Bessel-Legendre inequality and its affine version are established, which are not explicitly in the form of Legendre polynomials. Moreover, the integral interval is shifted to a general one . Second, by introducing a proper augmented Lyapunov-Krasovskii functional, which is tailored for the canonical Bessel-Legendre inequality, some sufficient conditions on global asymptotic stability are formulated for neural networks with constant delays and neural networks with time-varying delays, respectively. These conditions are proven to have a hierarchical feature: the higher level of hierarchy, the less conservatism of the stability criterion. Finally, three numerical examples are given to illustrate the efficiency of the proposed stability criteria.