Recurrent neuro-fuzzy networks for nonlinear process modeling

Iterative Learning Model Predictive Control Based on Iterative Data-Driven Modeling

Iterative learning model predictive control (ILMPC) has been recognized as an effective approach to realize high-precision tracking for batch processes with repetitive nature because of its excellent learning ability and closed-loop stability property. However, as a model-based strategy, ILMPC suffers from the unavailability of accurate first principal model in many complex nonlinear batch systems. On account of the abundant process data, nonlinear dynamics of batch systems can be identified precisely along the trials by neural network (NN), making it enforceable to design a data-driven ILMPC. In this article, by using a control-affine feedforward neural network (CAFNN), the features in the process data of the former batch are extracted to form a nonlinear affine model for the controller design in the current batch. Based on the CAFNN model, the ILMPC is formulated in a tube framework to attenuate the influence of modeling errors and track the reference trajectory with sustained accuracy. Due to the control-affine structure, the gradients of the objective function can be analytically computed offline, so as to improve the online computational efficiency and optimization feasibility of the tube ILMPC. The robust stability and the convergence of the data-driven ILMPC system are analyzed theoretically. The simulation on a typical batch reactor verifies the effectiveness of the proposed control method.

Temporally local recurrent radial basis function network for modeling and adaptive control of nonlinear systems

In this paper, a novel temporally local recurrent radial basis function network for modeling and adaptive control of nonlinear systems is proposed. The proposed structure consists of recurrent hidden neurons having weighted self-feedback loops and a weighted linear feed-through from the input layer directly to the output layer neuron(s). The dynamic back-propagation algorithm is developed and used for updating the parameters of the proposed structure. To improve the performance of learning algorithm, discrete Lyapunov stability method is used to develop an adaptive learning rate scheme. This scheme ensures the faster convergence of the parameters and maintains the stability of the system. A total of 5 complex nonlinear systems are used to test and compare the performance of the proposed network with other neural network structures. The disturbance rejection tests are also carried out to check whether the proposed scheme is able to handle the external disturbance/noise signals effects or not. The obtained results show the efficacy of the proposed method.

Searching for predictors of sense of quality of health: A study using neural networks on a sample of perimenopausal women

BACKGROUND

We assumed that perimenopausal women's sense of quality of health (SQH) is a subjective evaluation of their psycho-physical health, and comprises three dimensions: sense of quality of life, menopausal symptoms, and the level of positive and negative affect.

PURPOSE

The aim of the study was to: 1) test a model about SQH, and 2) explore the role of personality traits, self-esteem, body self, and self-stereotype as predictors of SQH.

METHODS

The sample included 201 women aged between 45 and 55 (50.11±3.07). Participants filled out the Rosenberg Self-Esteem Scale, the Personality Inventory based on the Big Five Factor Model, the Body Self Questionnaire, and a survey querying perimenopausal women's self-stereotype. To determine the individual SQH dimensions we used the Sense of Quality of Life Questionnaire, the Menopause Symptom List, and the Positive and Negative Affect Schedule. To verify the assumptions of the SQH model and look for SQH predictors we conducted a neural networks analysis with structure optimization via genetic algorithms (a multivariate analysis).

RESULTS

The SQH model was verified in the course of several neural networks analyses with structure optimization via genetic algorithms (R = 0.849, R2 = 0.723, F = 133,232, p < 0.01). Moreover, we confirmed that SQH comprised three dimensions: quality of life, menopausal symptoms, and affect. SQH and menopausal symptoms were correlated. Similarly, positive and negative affect modified the women's global sense of quality of life. SQH predictors included: personality traits, self-esteem, the body-self, and menopausal woman's self-stereotype.

CONCLUSION

In practical terms, our findings may help raise awareness among women and medical practitioners, calling for a holistic approach to the health of menopausal women. Our findings may also facilitate the creation of both prevention and therapeutic programs for women transitioning through menopause, for example, cognitive-behavioral therapy.

Analysis of the Effects of Neuro-Fuzzy Control Configuration Parameters on PH Neutralization Process

The pH neutralization process is typical in chemical, biological and petrochemical industries. One of the major challenges to control it is to understand its nonlinearities and that requires several fine adjustments from conventional controls. Artificial Intelligence has been used to study these nonlinearities; one of them is Neuro-Fuzzy Logic, which was investigated in this work to develop controls dedicated to this process. These controls are formed by logical structures and may be adjusted to different configurations. In practical applications, it is highly important to adapt control parameters based on artificial intelligence to obtain better performance. The present work studied the effect of different configurations of a neuro-fuzzy control on the performance of a regulatory control to pH neutralization process by means of a virtual plant developed in both Indusoft© and Matlab© environments. For both variables, pH and reactor level control, membership function (MF) = [Gaussian], method “OR” = [probabilistic], method “E” = [product], type of MF output = [linear] and the optimization method = [hybrid], have improved control performance, which confirms the importance of configuration choices in neuro-fuzzy control adjustments. Moreover, the most determining factor in NFC performance is the types of membership functions.

Assessing the Health of LiFePO₄ Traction Batteries through Monotonic Echo State Networks

A soft sensor is presented that approximates certain health parameters of automotive rechargeable batteries from on-vehicle measurements of current and voltage. The sensor is based on a model of the open circuit voltage curve. This last model is implemented through monotonic neural networks and estimate over-potentials arising from the evolution in time of the Lithium concentration in the electrodes of the battery. The proposed soft sensor is able to exploit the information contained in operational records of the vehicle better than the alternatives, this being particularly true when the charge or discharge currents are between moderate and high. The accuracy of the neural model has been compared to different alternatives, including data-driven statistical models, first principle-based models, fuzzy observers and other recurrent neural networks with different topologies. It is concluded that monotonic echo state networks can outperform well established first-principle models. The algorithms have been validated with automotive Li-FePO₄ cells.

A Novel Interactively Recurrent Self-Evolving Fuzzy CMAC and Its Classification Applications

In this paper, an Interactively Recurrent Self-evolving Fuzzy Cerebellar Model Articulation Controller (IRSFCMAC) model is developed for solving classification problems. The proposed IRSFCMAC classifier consists of internal feedback and external loops, which are generated by the hypercube cell firing strength to itself and other hypercube cells. The learning process of the IRSFCMAC gets started with an empty hypercube base, and then all of hypercube cells are generated and learned online via structure and parameter learning, respectively. The structure learning algorithm is based on the degree measure to determine the number of hypercube cells. The parameter learning algorithm, based on the gradient descent method, adjusts the shapes of the membership functions and the corresponding fuzzy weights of the IRSFCMAC. Finally, the proposed IRSFCMAC model is tested by four benchmark classification problems. Experimental results show that the proposed IRSFCMAC model has superior performance than traditional FCMAC and other models.

Synchronization of chaotic systems and identification of nonlinear systems by using recurrent hierarchical type-2 fuzzy neural networks

This paper proposes a novel approach for training of proposed recurrent hierarchical interval type-2 fuzzy neural networks (RHT2FNN) based on the square-root cubature Kalman filters (SCKF). The SCKF algorithm is used to adjust the premise part of the type-2 FNN and the weights of defuzzification and the feedback weights. The recurrence property in the proposed network is the output feeding of each membership function to itself. The proposed RHT2FNN is employed in the sliding mode control scheme for the synchronization of chaotic systems. Unknown functions in the sliding mode control approach are estimated by RHT2FNN. Another application of the proposed RHT2FNN is the identification of dynamic nonlinear systems. The effectiveness of the proposed network and its learning algorithm is verified by several simulation examples. Furthermore, the universal approximation of RHT2FNNs is also shown.

Identification and prediction of dynamic systems using an interactively recurrent self-evolving fuzzy neural network

This paper presents a novel recurrent fuzzy neural network, called an interactively recurrent self-evolving fuzzy neural network (IRSFNN), for prediction and identification of dynamic systems. The recurrent structure in an IRSFNN is formed as an external loops and internal feedback by feeding the rule firing strength of each rule to others rules and itself. The consequent part in the IRSFNN is composed of a Takagi-Sugeno-Kang (TSK) or functional-link-based type. The proposed IRSFNN employs a functional link neural network (FLNN) to the consequent part of fuzzy rules for promoting the mapping ability. Unlike a TSK-type fuzzy neural network, the FLNN in the consequent part is a nonlinear function of input variables. An IRSFNNs learning starts with an empty rule base and all of the rules are generated and learned online through a simultaneous structure and parameter learning. An on-line clustering algorithm is effective in generating fuzzy rules. The consequent update parameters are derived by a variable-dimensional Kalman filter algorithm. The premise and recurrent parameters are learned through a gradient descent algorithm. We test the IRSFNN for the prediction and identification of dynamic plants and compare it to other well-known recurrent FNNs. The proposed model obtains enhanced performance results.

ANNEALED CHAOTIC LEARNING FOR TIME SERIES PREDICTION IN IMPROVED NEURO-FUZZY NETWORK WITH FEEDBACKS

A new version of neuro-fuzzy system of feedbacks with chaotic dynamics is proposed in this work. Unlike the conventional neuro-fuzzy, improved neuro-fuzzy system with feedbacks is better able to handle temporal data series. By introducing chaotic dynamics into the feedback neuro-fuzzy system, the system has richer and more flexible dynamics to search for near-optimal solutions. In the experimental results, performance and effectiveness of the presented approach are evaluated by using benchmark data series. Comparison with other existing methods shows the proposed method for the neuro-fuzzy feedback is able to predict the time series accurately.

Fundamentals of a fuzzy-logic-based generalized theory of stability

Stability is one of the fundamental concepts of complex dynamical systems including physical, economical, socioeconomical, and technical systems. In classical terms, the notion of stability inherently associates with any dynamical system and determines whether a system under consideration reaches equilibrium after being exposed to disturbances. Predominantly, this concept comes with a binary (Boolean) quantification (viz., we either quantify that systems are stable or not stable). While in some cases, this definition is well justifiable, with the growing complexity and diversity of systems one could seriously question the Boolean nature of the definition and its underlying semantics. This becomes predominantly visible in human-oriented quantification of stability in which we commonly encounter statements quantifying stability through some linguistic terms such as, e.g., absolutely unstable, highly unstable, ..., absolutely stable, and alike. To formulate human-oriented definitions of stability, we may resort ourselves to the use of a so-called Precisiated Natural Language, which comes as a subset of natural language and one of whose functions is redefining existing concepts, such as stability, optimality, and alike. Being prompted by the discrepancy of the definition of stability and the Boolean character of the concept itself, in this paper, we introduce and develop a Generalized Theory of Stability (GTS) for analysis of complex dynamical systems described by fuzzy differential equations. Different human-centric definitions of stability of dynamical systems are introduced. We also discuss and contrast several fundamental concepts of fuzzy stability, namely, fuzzy stability of systems, binary stability of fuzzy system, and binary stability of systems by showing that all of them arise as special cases of the proposed GTS. The introduced definitions offer an important ability to quantify the concept of stability using some continuous quantification (that is through the use of degrees of stability). In this manner, we radically depart from the previous binary character of the definition. We establish some criteria concerning generalized stability for a wide class of continuous dynamical systems. Next, we present a series of illustrative examples which demonstrate the essence of the concept, and at the same time, stress that the existing Boolean techniques are not capable of capturing the essence of linguistic stability. We also apply the obtained results to investigate the stability of an economical system and show its usefulness in the design of nonlinear fuzzy control systems given some predefined degree of stability.

Hierarchical singleton-type recurrent neural fuzzy networks for noisy speech recognition

This paper proposes noisy speech recognition using hierarchical singleton-type recurrent neural fuzzy networks (HSRNFNs). The proposed HSRNFN is a hierarchical connection of two singleton-type recurrent neural fuzzy networks (SRNFNs), where one is used for noise filtering and the other for recognition. The SRNFN is constructed by recurrent fuzzy if-then rules with fuzzy singletons in the consequences, and their recurrent properties make them suitable for processing speech patterns with temporal characteristics. In n words recognition, n SRNFNs are created for modeling n words, where each SRNFN receives the current frame feature and predicts the next one of its modeling word. The prediction error of each SRNFN is used as recognition criterion. In filtering, one SRNFN is created, and each SRNFN recognizer is connected to the same SRNFN filter, which filters noisy speech patterns in the feature domain before feeding them to the SRNFN recognizer. Experiments with Mandarin word recognition under different types of noise are performed. Other recognizers, including multilayer perceptron (MLP), time-delay neural networks (TDNNs), and hidden Markov models (HMMs), are also tested and compared. These experiments and comparisons demonstrate good results with HSRNFN for noisy speech recognition tasks.

Multifeedback-layer neural network

The architecture and training procedure of a novel recurrent neural network (RNN), referred to as the multifeedback-layer neural network (MFLNN), is described in this paper. The main difference of the proposed network compared to the available RNNs is that the temporal relations are provided by means of neurons arranged in three feedback layers, not by simple feedback elements, in order to enrich the representation capabilities of the recurrent networks. The feedback layers provide local and global recurrences via nonlinear processing elements. In these feedback layers, weighted sums of the delayed outputs of the hidden and of the output layers are passed through certain activation functions and applied to the feedforward neurons via adjustable weights. Both online and offline training procedures based on the backpropagation through time (BPTT) algorithm are developed. The adjoint model of the MFLNN is built to compute the derivatives with respect to the MFLNN weights which are then used in the training procedures. The Levenberg-Marquardt (LM) method with a trust region approach is used to update the MFLNN weights. The performance of the MFLNN is demonstrated by applying to several illustrative temporal problems including chaotic time series prediction and nonlinear dynamic system identification, and it performed better than several networks available in the literature.

Adaptive control with hysteresis estimation and compensation using RFNN for piezo-actuator

Because the control performance of a piezoactuator is always severely deteriorated due to hysteresis effect, an adaptive control with hysteresis estimation and compensation using recurrent fuzzy neural network (RFNN) is proposed in this study to improve the control performance of the piezo-actuator. A new hysteresis model by modifying and parameterizing the hysteresis friction model is proposed. Then, the overall dynamics of the piezo-actuator is completed by integrating the parameterized hysteresis model into a mechanical motion dynamics. Based on this developed dynamics, an adaptive control with hysteresis estimation and compensation is proposed. However, in the designed adaptive controller, the lumped uncertainty E is difficult to obtain in practical application. Therefore, a RFNN is adopted as an uncertainty observer in order to adapt the value of the lumped uncertainty E on line. And, some experimental results show that the proposed controller provides high-performance dynamic characteristics and is robust to the variations of system parameters and external load.

A Recurrent Fuzzy Network for Fuzzy Temporal Sequence Processing and Gesture Recognition

A fuzzified Takagi-Sugeno-Kang (TSK)-type recurrent fuzzy network (FTRFN) for handling fuzzy temporal information is proposed in this paper. The FTRFN extends our previously proposed network, TRFN, to deal with fuzzy temporal signals represented by Gaussian or triangular fuzzy numbers. In the precondition part of FTRFN, matching degrees between input fuzzy variables and fuzzy antecedent sets is performed by similarity measure. In the TSK-type consequence, a linear combination of fuzzy variables is computed, where two sets of combination coefficients, one for the center and the other for the width of each fuzzy number, are used. Derivation of the linear combination results and final network output is based on left-right fuzzy number operation. There are no rules in FTRFN initially; they are constructed online by concurrent structure and parameter learning, where all free parameters in the precondition/consequence of FTRFN are all tunable. FTRFN can be applied on a variety of domains related to fuzzy temporal information processing. In this paper, it has been applied on one-dimensional and two-dimensional fuzzy temporal sequence prediction and CCD-based temporal gesture recognition. The performance of FTRFN is verified from these examples.

A TSK-Type Neurofuzzy Network Approach to System Modeling Problems

We develop a neurofuzzy network technique to extract TSK-type fuzzy rules from a given set of input-output data for system modeling problems. Fuzzy clusters are generated incrementally from the training dataset, and similar clusters are merged dynamically together through input-similarity, output-similarity, and output-variance tests. The associated membership functions are defined with statistical means and deviations. Each cluster corresponds to a fuzzy IF-THEN rule, and the obtained rules can be further refined by a fuzzy neural network with a hybrid learning algorithm which combines a recursive singular value decomposition-based least squares estimator and the gradient descent method. The proposed technique has several advantages. The information about input and output data subspaces is considered simultaneously for cluster generation and merging. Membership functions match closely with and describe properly the real distribution of the training data points. Redundant clusters are combined, and the sensitivity to the input order of training data is reduced. Besides, generation of the whole set of clusters from the scratch can be avoided when new training data are considered.

Prediction and Identification Using Wavelet-Based Recurrent Fuzzy Neural Networks

This paper presents a wavelet-based recurrent fuzzy neural network (WRFNN) for prediction and identification of nonlinear dynamic systems. The proposed WRFNN model combines the traditional Takagi-Sugeno-Kang (TSK) fuzzy model and the wavelet neural networks (WNN). This paper adopts the nonorthogonal and compactly supported functions as wavelet neural network bases. Temporal relations embedded in the network are caused by adding some feedback connections representing the memory units into the second layer of the feedforward wavelet-based fuzzy neural networks (WFNN). An online learning algorithm, which consists of structure learning and parameter learning, is also presented. The structure learning depends on the degree measure to obtain the number of fuzzy rules and wavelet functions. Meanwhile, the parameter learning is based on the gradient descent method for adjusting the shape of the membership function and the connection weights of WNN. Finally, computer simulations have demonstrated that the proposed WRFNN model requires fewer adjustable parameters and obtains a smaller rms error than other methods.

A Hybrid of Genetic Algorithm and Particle Swarm Optimization for Recurrent Network Design

An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production problem. For recurrent fuzzy network design, a Takagi-Sugeno-Kang-type recurrent fuzzy network is designed and applied to dynamic plant control. The performance of HGAPSO is compared to both GA and PSO in these recurrent networks design problems, demonstrating its superiority.