Hybrid deep learning diagonal recurrent neural network controller for nonlinear systems

In the present paper, a hybrid deep learning diagonal recurrent neural network controller (HDL-DRNNC) is proposed for nonlinear systems. The proposed HDL-DRNNC structure consists of a diagonal recurrent neural network (DRNN), whose initial values can be obtained through deep learning (DL). The DL algorithm, which is used in this study, is a hybrid algorithm that is based on a self-organizing map of the Kohonen procedure and restricted Boltzmann machine. The updating weights of the DRNN of the proposed algorithm are developed using the Lyapunov stability criterion. In this concern, simulation tasks such as disturbance signals and parameter variations are performed on mathematical and physical systems to improve the performance and the robustness of the proposed controller. It is clear from the results that the performance of the proposed controller is better than other existent controllers.

TMCS-ENN: A Temporal Feature-Correlation Cuckoo Search-Elman Neural Network for Sugar Raw Materials Demands Prediction

The prediction of the demand for raw materials is of vital importance to modern industries. Most studies are based on traditional regression, linear programming, and other methods. Previous studies have often overlooked the characteristics of the sugar raw materials business and the influence of time factors on raw material demand, resulting in limited prediction accuracy. How to accurately predict the demand for sugar raw materials is one of the key issues for intelligent management. In view of the above problems, combined with the characteristics of the supply and demand cycle of sugar raw materials, this paper aims to predict the demand for raw materials based on their supply and demand in a real sugar company by optimizing the Elman neural network through the modified cuckoo search (MCS) algorithm with temporal features. This study proposes a temporal feature-correlation cuckoo search–Elman neural network (TMCS-ENN) for predicting the demand for sugar raw materials. The experimental results show that the accuracy of the TMCS-ENN model reaches 93.89%, a better performance than that achieved by existing models. Therefore, the study model effectively improves the accuracy of the demand forecast of sugar raw materials for companies. This output will be helpful for improving the production efficiency and automation level, as well as reducing costs.

Robust Recurrent Kernel Online Learning

We propose a robust recurrent kernel online learning (RRKOL) algorithm based on the celebrated real-time recurrent learning approach that exploits the kernel trick in a recurrent online training manner. The novel RRKOL algorithm guarantees weight convergence with regularized risk management through the use of adaptive recurrent hyperparameters for superior generalization performance. Based on a new concept of the structure update error with a variable parameter length, we are the first one to propose the detailed structure update error, such that the weight convergence and robust stability proof can be integrated with a kernel sparsification scheme based on a solid theoretical ground. The RRKOL algorithm automatically weighs the regularized term in the recurrent loss function, such that we not only minimize the estimation error but also improve the generalization performance through sparsification with simulation support.

Numerical analysis of modeling based on improved Elman neural network

A modeling based on the improved Elman neural network (IENN) is proposed to analyze the nonlinear circuits with the memory effect. The hidden layer neurons are activated by a group of Chebyshev orthogonal basis functions instead of sigmoid functions in this model. The error curves of the sum of squared error (SSE) varying with the number of hidden neurons and the iteration step are studied to determine the number of the hidden layer neurons. Simulation results of the half-bridge class-D power amplifier (CDPA) with two-tone signal and broadband signals as input have shown that the proposed behavioral modeling can reconstruct the system of CDPAs accurately and depict the memory effect of CDPAs well. Compared with Volterra-Laguerre (VL) model, Chebyshev neural network (CNN) model, and basic Elman neural network (BENN) model, the proposed model has better performance.

Computer-aided diagnosis system for classifying benign and malignant thyroid nodules in multi-stained FNAB cytological images

An automated computer-aided diagnosis system is developed to classify benign and malignant thyroid nodules using multi-stained fine needle aspiration biopsy (FNAB) cytological images. In the first phase, the image segmentation is performed to remove the background staining information and retain the appropriate foreground cell objects in cytological images using mathematical morphology and watershed transform segmentation methods. Subsequently, statistical features are extracted using two-level discrete wavelet transform (DWT) decomposition, gray level co-occurrence matrix (GLCM) and Gabor filter based methods. The classifiers k-nearest neighbor (k-NN), Elman neural network (ENN) and support vector machine (SVM) are tested for classifying benign and malignant thyroid nodules. The combination of watershed segmentation, GLCM features and k-NN classifier results a lowest diagnostic accuracy of 60 %. The highest diagnostic accuracy of 93.33 % is achieved by ENN classifier trained with the statistical features extracted by Gabor filter bank from the images segmented by morphology and watershed transform segmentation methods. It is also observed that SVM classifier results its highest diagnostic accuracy of 90 % for DWT and Gabor filter based features along with morphology and watershed transform segmentation methods. The experimental results suggest that the developed system with multi-stained thyroid FNAB images would be useful for identifying thyroid cancer irrespective of staining protocol used.

Enhancing weak signal transmission through a feedforward network

The ability to transmit and amplify weak signals is fundamental to signal processing of artificial devices in engineering. Using a multilayer feedforward network of coupled double-well oscillators as well as Fitzhugh-Nagumo oscillators, we here investigate the conditions under which a weak signal received by the first layer can be transmitted through the network with or without amplitude attenuation. We find that the coupling strength and the nodes' states of the first layer act as two-state switches, which determine whether the transmission is significantly enhanced or exponentially decreased. We hope this finding is useful for designing artificial signal amplifiers.

Robust initialization of a Jordan network with recurrent constrained learning

In this paper, we propose a robust initialization of a Jordan network with a recurrent constrained learning (RIJNRCL) algorithm for multilayered recurrent neural networks (RNNs). This novel algorithm is based on the constrained learning concept of the Jordan network with a recurrent sensitivity and weight convergence analysis, which is used to obtain a tradeoff between the training and testing errors. In addition to using classical techniques for the adaptive learning rate and the adaptive dead zone, RIJNRCL employs a recurrent constrained parameter matrix to switch off excessive contributions from the hidden layer neurons based on weight convergence and stability conditions of the multilayered RNNs. It is well known that a good response from the hidden layer neurons and proper initialization play a dominant role in avoiding local minima in multilayered RNNs. The new RIJNRCL algorithm solves the twin problems of weight initialization and selection of the hidden layer neurons via a novel recurrent sensitivity ratio analysis. We provide the detailed steps for using RIJNRCL in a few benchmark time-series prediction problems and show that the proposed algorithm achieves superior generalization performance.

Nonlinear model identification and adaptive model predictive control using neural networks

This paper presents two new adaptive model predictive control algorithms, both consisting of an on-line process identification part and a predictive control part. Both parts are executed at each sampling instant. The predictive control part of the first algorithm is the Nonlinear Model Predictive Control strategy and the control part of the second algorithm is the Generalized Predictive Control strategy. In the identification parts of both algorithms the process model is approximated by a series-parallel neural network structure which is trained by a recursive least squares (ARLS) method. The two control algorithms have been applied to: 1) the temperature control of a fluidized bed furnace reactor (FBFR) of a pilot plant and 2) the auto-pilot control of an F-16 aircraft. The training and validation data of the neural network are obtained from the open-loop simulation of the FBFR and the nonlinear F-16 aircraft models. The identification and control simulation results show that the first algorithm outperforms the second one at the expense of extra computation time.

Convergence study in extended Kalman filter-based training of recurrent neural networks

Recurrent neural network (RNN) has emerged as a promising tool in modeling nonlinear dynamical systems, but the training convergence is still of concern. This paper aims to develop an effective extended Kalman filter-based RNN training approach with a controllable training convergence. The training convergence problem during extended Kalman filter-based RNN training has been proposed and studied by adapting two artificial training noise parameters: the covariance of measurement noise (R) and the covariance of process noise (Q) of Kalman filter. The R and Q adaption laws have been developed using the Lyapunov method and the maximum likelihood method, respectively. The effectiveness of the proposed adaption laws has been tested using a nonlinear dynamical benchmark system and further applied in cutting tool wear modeling. The results show that the R adaption law can effectively avoid the divergence problem and ensure the training convergence, whereas the Q adaption law helps improve the training convergence speed.