Extracting rules from a (fuzzy/crisp) recurrent neural network using a self-organizing map

Establish a trend fuzzy information granule based short-term forecasting with long-association and k-medoids clustering

In the existing short-term forecasting methods of time series, two challenges are faced: capture the associations of data and avoid cumulative errors. For tackling these challenges, the fuzzy information granule based model catches our attention. The rule used in this model is fuzzy association rule (FAR), in which the FAR is constructed from a premise granule to a consequent granule at consecutive time periods, and then it describes the short-association in data. However, in real time series, another association, the association between a premise granule and a consequent granule at non-consecutive time periods, frequently exists, especially in periodical and seasonal time series. While the existing FAR can’t express such association. To describe it, the fuzzy long-association rule (FLAR) is proposed in this study. This kind of rule reflects the influence of an antecedent trend on a consequent trend, where these trends are described by fuzzy information granules at non-consecutive time periods. Thus, the FLAR can describe the long-association in data. Correspondingly, the existing FAR is called as fuzzy short-association rule (FSAR). Combining the existing FSAR with FLAR, a novel short-term forecasting model is presented. This model makes forecasting at granular level, and then it reduces the cumulative errors in short-term prediction. Note that the prediction results of this model are calculated from the available FARs selected by the k-medoids clustering based rule selection algorithm, therefore they are logical and accurate. The better forecasting performance of this model has been verified by comparing it with existing models in experiments.

Generalizing self-organizing map for categorical data

The self-organizing map (SOM) is an unsupervised neural network which projects high-dimensional data onto a low-dimensional grid and visually reveals the topological order of the original data. Self-organizing maps have been successfully applied to many fields, including engineering and business domains. However, the conventional SOM training algorithm handles only numeric data. Categorical data are usually converted to a set of binary data before training of an SOM takes place. If a simple transformation scheme is adopted, the similarity information embedded between categorical values may be lost. Consequently, the trained SOM is unable to reflect the correct topological order. This paper proposes a generalized self-organizing map model that offers an intuitive method of specifying the similarity between categorical values via distance hierarchies and, hence, enables the direct process of categorical values during training. In fact, distance hierarchy unifies the distance computation of both numeric and categorical values. The unification is done by mapping the values to distance hierarchies and then measuring the distance in the hierarchies. Experiments on synthetic and real datasets were conducted, and the results demonstrated the effectiveness of the generalized SOM model.

Rule Extraction from Recurrent Neural Networks: ATaxonomy and Review

Rule extraction (RE) from recurrent neural networks (RNNs) refers to finding models of the underlying RNN, typically in the form of finite state machines, that mimic the network to a satisfactory degree while having the advantage of being more transparent. RE from RNNs can be argued to allow a deeper and more profound form of analysis of RNNs than other, more or less ad hoc methods. RE may give us understanding of RNNs in the intermediate levels between quite abstract theoretical knowledge of RNNs as a class of computing devices and quantitative performance evaluations of RNN instantiations. The development of techniques for extraction of rules from RNNs has been an active field since the early 1990s. This article reviews the progress of this development and analyzes it in detail. In order to structure the survey and evaluate the techniques, a taxonomy specifically designed for this purpose has been developed. Moreover, important open research issues are identified that, if addressed properly, possibly can give the field a significant push forward.

A real-coded genetic algorithm for training recurrent neural networks

The use of Recurrent Neural Networks is not as extensive as Feedforward Neural Networks. Training algorithms for Recurrent Neural Networks, based on the error gradient, are very unstable in their search for a minimum and require much computational time when the number of neurons is high. The problems surrounding the application of these methods have driven us to develop new training tools. In this paper, we present a Real-Coded Genetic Algorithm that uses the appropriate operators for this encoding type to train Recurrent Neural Networks. We describe the algorithm and we also experimentally compare our Genetic Algorithm with the Real-Time Recurrent Learning algorithm to perform the fuzzy grammatical inference.