Study of distributed learning as a solution to category proliferation in Fuzzy ARTMAP based neural systems

CHECKING ORTHOGONAL TRANSFORMATIONS AND GENETIC ALGORITHMS FOR SELECTION OF FUZZY RULES BASED ON INTERPRETABILITY-ACCURACY CONCEPTS

Fuzzy modeling is one of the most known and used techniques in different areas to model the behavior of systems and processes. In most cases, as in data-driven fuzzy modeling, these fuzzy models reach a high performance from the point of view of accuracy, but from other points of view, such as complexity or interpretability, they can present a poor performance.

Several approaches are found in the bibliography to reduce the complexity and improve the interpretability of the fuzzy models. In this paper, a post-processing approach is carried out via rule selection, whose aim is to choose the most relevant rules for working together on the well-known accuracy-interpretability trade-off. The rule relevancy is based on Orthogonal Transformations, such as the SVD-QR rank revealing approach, the P-QR and OLS transformations. Rule selection is carried out using a genetic algorithm that takes into account the information obtained by the Orthogonal Transformations. The main objective is to check the true significance, drawbacks and advantages of the rule selection based on the orthogonal transformations via the rule firing strength matrix.

In order to carry out this aim, a neuro-fuzzy system, FasArt (Fuzzy Adaptive System ART based), and several case studies, data sets from the KEEL Project Repository, are used to tune and check this selection of rules based on orthogonal transformations, genetic selection and accuracy-interpretability trade-off. This neuro-fuzzy system generates Mamdani fuzzy rule based systems (FRBSs), in an approximative way. NSGA-II is the MOEA tool used to tune the proposed rule selection.

An adaptive multiobjective approach to evolving ART architectures

In this paper, we present the evolution of adaptive resonance theory (ART) neural network architectures (classifiers) using a multiobjective optimization approach. In particular, we propose the use of a multiobjective evolutionary approach to simultaneously evolve the weights and the topology of three well-known ART architectures; fuzzy ARTMAP (FAM), ellipsoidal ARTMAP (EAM), and Gaussian ARTMAP (GAM). We refer to the resulting architectures as MO-GFAM, MO-GEAM, and MO-GGAM, and collectively as MO-GART. The major advantage of MO-GART is that it produces a number of solutions for the classification problem at hand that have different levels of merit [accuracy on unseen data (generalization) and size (number of categories created)]. MO-GART is shown to be more elegant (does not require user intervention to define the network parameters), more effective (of better accuracy and smaller size), and more efficient (faster to produce the solution networks) than other ART neural network architectures that have appeared in the literature. Furthermore, MO-GART is shown to be competitive with other popular classifiers, such as classification and regression tree (CART) and support vector machines (SVMs).

Managing category proliferation in fuzzy ARTMAP caused by overlapping classes

This paper addresses the difficulties brought about by overlapping classes in fuzzy ARTMAP (FAM). Training with such data leads to category proliferation, and classification is made difficult not only by the large number of categories but also the fact that such data can belong to either class. In this paper, changes were proposed to allow more than one class to be predicted during classification, and a number of modifications were explored to reduce the number of categories. The excessive creation of small categories was suppressed with the implementation of the modifications, and the predictive accuracy improved despite the significant reduction in number of categories. No major changes needed to be made to the FAM architecture.

GFAM: Evolving Fuzzy ARTMAP neural networks

This paper focuses on the evolution of Fuzzy ARTMAP neural network classifiers, using genetic algorithms, with the objective of improving generalization performance (classification accuracy of the ART network on unseen test data) and alleviating the ART category proliferation problem (the problem of creating more than necessary ART network categories to solve a classification problem). We refer to the resulting architecture as GFAM. We demonstrate through extensive experimentation that GFAM exhibits good generalization and is of small size (creates few ART categories), while consuming reasonable computational effort. In a number of classification problems, GFAM produces the optimal classifier. Furthermore, we compare the performance of GFAM with other competitive ARTMAP classifiers that have appeared in the literature and addressed the category proliferation problem in ART. We illustrate that GFAM produces improved results over these architectures, as well as other competitive classifiers.

Experiments with Safe muARTMAP : effect of the network parameters on the network performance

Fuzzy ARTMAP (FAM) is currently considered to be one of the premier neural network architectures in solving classification problems. One of the limitations of Fuzzy ARTMAP that has been extensively reported in the literature is the category proliferation problem. That is, Fuzzy ARTMAP has the tendency of increasing its network size, as it is confronted with more and more data, especially if the data are of the noisy and/or overlapping nature. To remedy this problem a number of researchers have designed modifications to the training phase of Fuzzy ARTMAP that had the beneficial effect of reducing this category proliferation. One of these modified Fuzzy ARTMAP architectures was the one proposed by Gomez-Sanchez, and his colleagues, referred to as Safe muARTMAP. In this paper we present reasonable analytical arguments that demonstrate of how we should choose the range of some of the Safe muARTMAP network parameters. Through a combination of these analytical arguments and experimentation we were able to identify good default parameter values for some of the Safe muARTMAP network parameters. This feat would allow one to save computations when a good performing Safe muARTMAP network is needed to be identified for a new classification problem. Furthermore, we performed an exhaustive experimentation to find the best Safe muARTMAP network for a variety of problems (simulated and real problems), and we compared it with other best performing ART networks, including other ART networks that claim to resolve the category proliferation problem in Fuzzy ARTMAP. These experimental results allow one to make appropriate statements regarding the pair-wise comparison of a number of ART networks (including Safe muARTMAP).

Data-partitioning using the Hilbert space filling curves: effect on the speed of convergence of Fuzzy ARTMAP for large database problems

The Fuzzy ARTMAP algorithm has been proven to be one of the premier neural network architectures for classification problems. One of the properties of Fuzzy ARTMAP, which can be both an asset and a liability, is its capacity to produce new nodes (templates) on demand to represent classification categories. This property allows Fuzzy ARTMAP to automatically adapt to the database without having to a priori specify its network size. On the other hand, it has the undesirable side effect that large databases might produce a large network size (node proliferation) that can dramatically slow down the training speed of the algorithm. To address the slow convergence speed of Fuzzy ARTMAP for large database problems, we propose the use of space-filling curves, specifically the Hilbert space-filling curves (HSFC). Hilbert space-filling curves allow us to divide the problem into smaller sub-problems, each focusing on a smaller than the original dataset. For learning each partition of data, a different Fuzzy ARTMAP network is used. Through this divide-and-conquer approach we are avoiding the node proliferation problem, and consequently we speedup Fuzzy ARTMAP's training. Results have been produced for a two-class, 16-dimensional Gaussian data, and on the Forest database, available at the UCI repository. Our results indicate that the Hilbert space-filling curve approach reduces the time that it takes to train Fuzzy ARTMAP without affecting the generalization performance attained by Fuzzy ARTMAP trained on the original large dataset. Given that the resulting smaller datasets that the HSFC approach produces can independently be learned by different Fuzzy ARTMAP networks, we have also implemented and tested a parallel implementation of this approach on a Beowulf cluster of workstations that further speeds up Fuzzy ARTMAP's convergence to a solution for large database problems.