Fuzzy min-max neural networks. I. Classification

A granular reflex fuzzy min-max neural network for classification

Granular data classification and clustering is an upcoming and important issue in the field of pattern recognition. Conventionally, computing is thought to be manipulation of numbers or symbols. However, human recognition capabilities are based on ability to process nonnumeric clumps of information (information granules) in addition to individual numeric values. This paper proposes a granular neural network (GNN) called granular reflex fuzzy min-max neural network (GrRFMN) which can learn and classify granular data. GrRFMN uses hyperbox fuzzy set to represent granular data. Its architecture consists of a reflex mechanism inspired from human brain to handle class overlaps. The network can be trained online using granular or point data. The neuron activation functions in GrRFMN are designed to tackle data of different granularity (size). This paper also addresses an issue to granulate the training data and learn from it. It is observed that such a preprocessing of data can improve performance of a classifier. Experimental results on real data sets show that the proposed GrRFMN can classify granules of different granularity more correctly. Results are compared with general fuzzy min-max neural network (GFMN) proposed by Gabrys and Bargiela and with some classical methods.

HACO2 Method for Evolving Hyperbox Classifiers with Ant Colony Optimization

A method, called HACO2 (Hyperbox classifier with Ant Colony Optimization - type 2), is proposed for evolving a hyperbox classifier using the ant colony meta-heuristic. It reshapes the hyperboxes in a near-optimal way to better fit the data, improving the accuracy and possibly indicating its most discriminative features. HACO2 is validated using artificial 2D data showing over 90% accuracy. It is also applied to the benchmark iris data set (4 features), providing results with over 93% accuracy, and to the MIS data set (11 features), with almost 85% accuracy. For these sets, the two most discriminative features obtained from the method are used in simplified classifiers which result in accuracies of 100% for the iris and 83% for the MIS data sets. Further modifications (automatic parameter setting), extensions (initialization short comings) and applications are discussed.

Fuzzy neural networks for classification and detection of anomalies

In this paper, a new learning algorithm for the Simpson's fuzzy min-max neural network is presented. It overcomes some undesired properties of the Simpson's model: specifically, in it there are neither thresholds that bound the dimension of the hyperboxes nor sensitivity parameters. Our new algorithm improves the network performance: in fact, the classification result does not depend on the presentation order of the patterns in the training set, and at each step, the classification error in the training set cannot increase. The new neural model is particularly useful in classification problems as it is shown by comparison with some fuzzy neural nets cited in literature (Simpson's min-max model, fuzzy ARTMAP proposed by Carpenter, Grossberg et al. in 1992, adaptive fuzzy systems as introduced by Wang in his book) and the classical multilayer perceptron neural network with backpropagation learning algorithm. The tests were executed on three different classification problems: the first one with two-dimensional synthetic data, the second one with realistic data generated by a simulator to find anomalies in the cooling system of a blast furnace, and the third one with real data for industrial diagnosis. The experiments were made following some recent evaluation criteria known in literature and by using Microsoft Visual C++ development environment on personal computers.

Pattern classification using fuzzy relational calculus

Our aim is to design a pattern classifier using fuzzy relational calculus (FRC) which was initially proposed by Pedrycz (1990). In the course of doing so, we first consider a particular interpretation of the multidimensional fuzzy implication (MFI) to represent our knowledge about the training data set. Subsequently, we introduce the notion of a fuzzy pattern vector to represent a population of training patterns in the pattern space and to denote the antecedent part of the said particular interpretation of the MFI. We introduce a new approach to the computation of the derivative of the fuzzy max-function and min-function using the concept of a generalized function. During the construction of the classifier based on FRC, we use fuzzy linguistic statements (or fuzzy membership function to represent the linguistic statement) to represent the values of features (e.g., feature F/sub 1/ is small and F/sub 2/ is big) for a population of patterns. Note that the construction of the classifier essentially depends on the estimate of a fuzzy relation /spl Rfr/ between the input (fuzzy set) and output (fuzzy set) of the classifier. Once the classifier is constructed, the nonfuzzy features of a pattern can be classified. At the time of classification of the nonfuzzy features of the testpatterns, we use the concept of fuzzy masking to fuzzify the nonfuzzy feature values of the testpatterns. The performance of the proposed scheme is tested on synthetic data. Finally, we use the proposed scheme for the vowel classification problem of an Indian language.

Similarity classifier in diagnosis of bladder cancer

In this article a similarity classifier's performance is studied in the diagnosis of bladder cancer. It is demonstrated that good classification results in diagnosis of bladder cancer are already achieved with a very small amount of data in the training set with the use of similarity classifier. When a new disease is initially discovered, the amount of samples are always quite limited (due to a fact that amount of patients is few), and this situation makes clinical work very difficult. A similarity classifier is a fast and accurate tool for medical diagnosis and it is capable of accurate performance already with a limited amount of data. This is quite important because there is a very limited amount of techniques available even to deal with such small sample sizes and especially in the diagnosis of cancer, high diagnosis accuracy is most important. In this study similarity classifier is used in diagnosis of bladder cancer. A good accuracy (of 100%) is already achieved with very small amount of samples in training the classifier. Here only four samples (two persons with bladder cancer and two persons without bladder cancer) were needed to train classifier managing the diagnosis of bladder cancer with 100% accuracy.

PERFORMANCE EVALUATION OF FUZZY SINGLE LAYER WEIGHTLESS NEURAL NETWORK

The paper evaluates the performance of a neuro-fuzzy pattern classification system based on the weightless neural network architecture. The system utilizes a Single Layer Weightless Neural Network (SLWNN) to extract the features vector that measures the similarity of the input pattern to the different classification groups. In contrast to the traditional crisp Winner-Takes-All (WTA) classification scheme used by SLWNN, our system uses a Fuzzy Inference System (FIS) for classification. The network is trained by a hybrid learning scheme that combines a single pass learning phase for training the SLWNN followed by a supervised learning phase for extracting a set of fuzzy rules suitable to classify the training set. The FIS learns fuzzy rules from the feature vectors generated by the SLWNN for the set of training patterns. The recognition of handwritten numerals is employed as a test-bed to demonstrate the effectiveness of the proposed neuro-fuzzy system. Experimental results show that the performance of the proposed system surpasses the performance of the traditional SLWNN.

A Novel Neuro-Fuzzy Inference System with Multi-Level Membership Function for Classification Applications

In this paper, a novel neuro-fuzzy inference system with multi-level membership function (NFIS_MMF) for classification applications is proposed. The NFIS_MMF model is a five-layer structure, which combines the traditional Takagi-Sugeno-Kang (TSK). Layer 2 of the NFIS_MMF model contains multi-level membership functions, which are multilevel activation functions. A self-constructing learning algorithm, which consists of the self-clustering algorithm (SCA), fuzzy entropy, and the backpropagation algorithm, is also proposed to construct the NFIS_MMF model and perform parameter learning. Simulations were conducted to show the performance and applicability of the proposed model.

Pipelining of Fuzzy ARTMAP without matchtracking: Correctness, performance bound, and Beowulf evaluation

Fuzzy ARTMAP neural networks have been proven to be good classifiers on a variety of classification problems. However, the time that Fuzzy ARTMAP takes to converge to a solution increases rapidly as the number of patterns used for training is increased. In this paper we examine the time Fuzzy ARTMAP takes to converge to a solution and we propose a coarse grain parallelization technique, based on a pipeline approach, to speed-up the training process. In particular, we have parallelized Fuzzy ARTMAP without the match-tracking mechanism. We provide a series of theorems and associated proofs that show the characteristics of Fuzzy ARTMAP's, without matchtracking, parallel implementation. Results run on a BEOWULF cluster with three large databases show linear speedup as a function of the number of processors used in the pipeline. The databases used for our experiments are the Forrest CoverType database from the UCI Machine Learning repository and two artificial databases, where the data generated were 16-dimensional Gaussian distributed data belonging to two distinct classes, with different amounts of overlap (5% and 15%).

A Fuzzy Min-Max Neural Network Classifier With Compensatory Neuron Architecture

This paper proposes a fuzzy min-max neural network classifier with compensatory neurons (FMCNs). FMCN uses hyperbox fuzzy sets to represent the pattern classes. It is a supervised classification technique with new compensatory neuron architecture. The concept of compensatory neuron is inspired from the reflex system of human brain which takes over the control in hazardous conditions. Compensatory neurons (CNs) imitate this behavior by getting activated whenever a test sample falls in the overlapped regions amongst different classes. These neurons are capable to handle the hyperbox overlap and containment more efficiently. Simpson used contraction process based on the principle of minimal disturbance, to solve the problem of hyperbox overlaps. FMCN eliminates use of this process since it is found to be erroneous. FMCN is capable to learn the data online in a single pass through with reduced classification and gradation errors. One of the good features of FMCN is that its performance is less dependent on the initialization of expansion coefficient, i.e., maximum hyperbox size. The paper demonstrates the performance of FMCN by comparing it with fuzzy min-max neural network (FMNN) classifier and general fuzzy min-max neural network (GFMN) classifier, using several examples.

Rough set feature selection and rule induction for prediction of malignancy degree in brain glioma

The degree of malignancy in brain glioma is assessed based on magnetic resonance imaging (MRI) findings and clinical data before operation. These data contain irrelevant features, while uncertainties and missing values also exist. Rough set theory can deal with vagueness and uncertainty in data analysis, and can efficiently remove redundant information. In this paper, a rough set method is applied to predict the degree of malignancy. As feature selection can improve the classification accuracy effectively, rough set feature selection algorithms are employed to select features. The selected feature subsets are used to generate decision rules for the classification task. A rough set attribute reduction algorithm that employs a search method based on particle swarm optimization (PSO) is proposed in this paper and compared with other rough set reduction algorithms. Experimental results show that reducts found by the proposed algorithm are more efficient and can generate decision rules with better classification performance. The rough set rule-based method can achieve higher classification accuracy than other intelligent analysis methods such as neural networks, decision trees and a fuzzy rule extraction algorithm based on Fuzzy Min-Max Neural Networks (FRE-FMMNN). Moreover, the decision rules induced by rough set rule induction algorithm can reveal regular and interpretable patterns of the relations between glioma MRI features and the degree of malignancy, which are helpful for medical experts.

Speckle reduction in medical ultrasound: a novel scatterer density weighted nonlinear diffusion algorithm implemented as a neural-network filter

This paper proposes a novel algorithm for speckle reduction in medical ultrasound imaging while preserving the edges with the added advantages of adaptive noise filtering and speed. We propose a nonlinear image diffusion algorithm that incorporates two local parameters of image quality, namely, scatterer density and texture-based contrast in addition to gradient, to weight the nonlinear diffusion process. The scatterer density is proposed to replace the existing traditional measures of quality of the ultrasound diffusion process such as MSE, RMSE, SNR, and PSNR. This novel diffusion filter was then implemented using back propagation neural network for fast parallel processing of volumetric images. The experimental results show that weighting the image diffusion with these parameters produces better noise reduction and produces a better edge detection quality with reasonable computational cost. The proposed filter can be used as a preprocessing phase before applying any ultrasound segmentation or active contour model processes.

Data-partitioning using the Hilbert space filling curves: effect on the speed of convergence of Fuzzy ARTMAP for large database problems

The Fuzzy ARTMAP algorithm has been proven to be one of the premier neural network architectures for classification problems. One of the properties of Fuzzy ARTMAP, which can be both an asset and a liability, is its capacity to produce new nodes (templates) on demand to represent classification categories. This property allows Fuzzy ARTMAP to automatically adapt to the database without having to a priori specify its network size. On the other hand, it has the undesirable side effect that large databases might produce a large network size (node proliferation) that can dramatically slow down the training speed of the algorithm. To address the slow convergence speed of Fuzzy ARTMAP for large database problems, we propose the use of space-filling curves, specifically the Hilbert space-filling curves (HSFC). Hilbert space-filling curves allow us to divide the problem into smaller sub-problems, each focusing on a smaller than the original dataset. For learning each partition of data, a different Fuzzy ARTMAP network is used. Through this divide-and-conquer approach we are avoiding the node proliferation problem, and consequently we speedup Fuzzy ARTMAP's training. Results have been produced for a two-class, 16-dimensional Gaussian data, and on the Forest database, available at the UCI repository. Our results indicate that the Hilbert space-filling curve approach reduces the time that it takes to train Fuzzy ARTMAP without affecting the generalization performance attained by Fuzzy ARTMAP trained on the original large dataset. Given that the resulting smaller datasets that the HSFC approach produces can independently be learned by different Fuzzy ARTMAP networks, we have also implemented and tested a parallel implementation of this approach on a Beowulf cluster of workstations that further speeds up Fuzzy ARTMAP's convergence to a solution for large database problems.

Automatic sign language analysis: a survey and the future beyond lexical meaning

Research in automatic analysis of sign language has largely focused on recognizing the lexical (or citation) form of sign gestures as they appear in continuous signing, and developing algorithms that scale well to large vocabularies. However, successful recognition of lexical signs is not sufficient for a full understanding of sign language communication. Nonmanual signals and grammatical processes which result in systematic variations in sign appearance are integral aspects of this communication but have received comparatively little attention in the literature. In this survey, we examine data acquisition, feature extraction and classification methods employed for the analysis of sign language gestures. These are discussed with respect to issues such as modeling transitions between signs in continuous signing, modeling inflectional processes, signer independence, and adaptation. We further examine works that attempt to analyze nonmanual signals and discuss issues related to integrating these with (hand) sign gestures. We also discuss the overall progress toward a true test of sign recognition systems--dealing with natural signing by native signers. We suggest some future directions for this research and also point to contributions it can make to other fields of research. Web-based supplemental materials (appendicies) which contain several illustrative examples and videos of signing can be found at www.computer.org/publications/dlib.

An on-line algorithm for creating self-organizing fuzzy neural networks

This paper presents a new on-line algorithm for creating a self-organizing fuzzy neural network (SOFNN) from sample patterns to implement a singleton or Takagi-Sugeno (TS) type fuzzy model. The SOFNN is based on ellipsoidal basis function (EBF) neurons consisting of a center vector and a width vector. New methods of the structure learning and the parameter learning, based on new adding and pruning techniques and a recursive on-line learning algorithm, are proposed and developed. A proof of the convergence of both the estimation error and the linear network parameters is also given in the paper. The proposed methods are very simple and effective and generate a fuzzy neural model with a high accuracy and compact structure. Simulation work shows that the SOFNN has the capability of self-organization to determine the structure and parameters of the network automatically.

Study of distributed learning as a solution to category proliferation in Fuzzy ARTMAP based neural systems

An evaluation of distributed learning as a means to attenuate the category proliferation problem in Fuzzy ARTMAP based neural systems is carried out, from both qualitative and quantitative points of view. The study involves two original winner-take-all (WTA) architectures, Fuzzy ARTMAP and FasArt, and their distributed versions, dARTMAP and dFasArt. A qualitative analysis of the distributed learning properties of dARTMAP is made, focusing on the new elements introduced to endow Fuzzy ARTMAP with distributed learning. In addition, a quantitative study on a selected set of classification problems points out that problems have to present certain features in their output classes in order to noticeably reduce the number of recruited categories and achieve an acceptable classification accuracy. As part of this analysis, distributed learning was successfully adapted to a member of the Fuzzy ARTMAP family, FasArt, and similar procedures can be used to extend distributed learning capabilities to other Fuzzy ARTMAP based systems.