Fuzzy min-max neural networks. I. Classification

Category regions as new geometrical concepts in Fuzzy-ART and Fuzzy-ARTMAP

In this paper we introduce novel geometric concepts, namely category regions, in the original framework of Fuzzy-ART (FA) and Fuzzy-ARTMAP (FAM). The definitions of these regions are based on geometric interpretations of the vigilance test and the F2 layer competition of committed nodes with uncommitted ones, that we call commitment test. It turns out that not only these regions have the same geometrical shape (polytope structure), but they also share a lot of common and interesting properties that are demonstrated in this paper. One of these properties is the shrinking of the volume that each one of these polytope structures occupies, as training progresses, which alludes to the stability of learning in FA and FAM, a well-known result. Furthermore, properties of learning of FA and FAM are also proven utilizing the geometrical structure and properties that these regions possess; some of these properties were proven before using counterintuitive, algebraic manipulations and are now demonstrated again via intuitive geometrical arguments. One of the results that is worth mentioning as having practical ramifications is the one which states that for certain areas of the vigilance-choice parameter space (rho,a), the training and performance (testing) phases of FA and FAM do not depend on the particular choices of the vigilance parameter. Finally, it is worth noting that, although the idea of the category regions has been developed under the premises of FA and FAM, category regions are also meaningful for later developed ART neural network structures, such as ARTEMAP, ARTMAP-IC, Boosted ARTMAP, Micro-ARTMAP, Ellipsoid-ART/ARTMAP, among others.

Fuzzy rules to predict degree of malignancy in brain glioma

The current pre-operative assessment of the degree of malignancy in brain glioma is based on magnetic resonance imaging (MRI) findings and clinical data. 280 cases were studied, of which 111 were high-grade malignancies and 169 were low-grade, so that regular and interpretable patterns of the relationships between glioma MRI features and the degree of malignancy could be acquired. However, as uncertainties in the data and missing values existed, a fuzzy rule extraction algorithm based on a fuzzy min-max neural network (FMMNN) was used. The performance of a multi-layer perceptron network (MLP) trained with the error back-propagation algorithm (BP), the decision tree algorithm ID3, nearest neighbour and the original fuzzy min-max neural network were also evaluated. The results showed that two fuzzy decision rules on only six features achieved an accuracy of 84.6% (89.9% for low-grade and 76.6% for high-grade cases). Investigations with the proposed algorithm revealed that age, mass effect, oedema, postcontrast enhancement, blood supply, calcification, haemorrhage and the signal intensity of the T1-weighted image were important diagnostic factors.

Dynamical properties of min-max networks

In this paper we study the dynamical behavior of a class of neural networks where the local transition rules are max or min functions. We prove that sequential updates define dynamics which reach the equilibrium in O(n2) steps, where n is the size of the network. For synchronous updates the equilibrium is reached in O(n) steps. It is shown that the number of fixed points of the sequential update is at most n. Moreover, given a set of p < or = n vectors, we show how to build a network of size n such that all these vectors are fixed points.

Pattern classification by a neurofuzzy network: application to vibration monitoring

An innovative neurofuzzy network is proposed herein for pattern classification applications, specifically for vibration monitoring. A fuzzy set interpretation is incorporated into the network design to handle imprecise information. A neural network architecture is used to automatically deduce fuzzy if-then rules based on a hybrid supervised learning scheme. The neurofuzzy classifier proposed is equipped with a one-pass, on-line, and incremental learning algorithm. This network can be considered a self-organized classifier with the ability to adaptively learn new information without forgetting old knowledge. The classification performance of the proposed neurofuzzy network is validated on the Fisher's Iris data, which is a well-known benchmark data set. For the generalization capability, the neurofuzzy network can achieve 97.33% correct classification. In addition, to demonstrate the efficiency and effectiveness of the proposed neurofuzzy paradigm, numerical simulations have been performed using the Westland data set. The Westland data set consists of vibration data collected from a US Navy CH-46E helicopter test stand. Using a simple fast Fourier transform technique for feature extraction, the proposed neurofuzzy network has shown promising results. Using various torque levels for training and testing, the network achieved 100% correct classification.

Combinatorial evolution of regression nodes in feedforward neural networks

A number of techniques exist with which neural network architectures such as multilayer perceptrons and radial basis function networks can be trained. These include backpropagation, k-means clustering and evolutionary algorithms. The latter method is particularly useful as it is able to avoid local optima in the search space and can optimise parameters for which no gradient information exists. Unfortunately, only moderately sized networks can be trained by this method, owing to the fact that evolutionary optimisation is very computationally intensive. In this paper a novel algorithm (CERN) is therefore proposed which uses a special form of combinatorial search to optimise groups of neural nodes. Oriented, ellipsoidal basis nodes optimised with CERN achieved significantly better accuracy with fewer nodes than spherical basis nodes optimised by k-means clustering. Multilayer perceptrons optimised by CERN were found to be as accurate as those trained by advanced gradient descent techniques. CERN was also found to be significantly more efficient than a conventional evolutionary algorithm that does not use a combinatorial search.

A neural network classifier with disjunctive fuzzy information

This paper presents a neural network classifier that learns disjunctive fuzzy information in the feature space. This neural network consists of two types of nodes in the hidden layer. The prototype nodes and exemplar nodes represent cluster centroids and exceptions in the feature space, respectively. This classifier automatically generates and refines prototypes for distinct clusters in the feature space. The number and sizes of these prototypes are not restricted, so the prototypes will form near-optimal decision regions to meet the distribution of input patterns and classify as many input patterns as possible. Next, exemplars will be created and expanded to learn the patterns that cannot be classified by the prototypes. Such a training strategy can reduce the memory requirement and speed up the process of non-linear classification. In addition, on-line learning is supplied in this classifier and the computational load is lightened. The experimental results manifest that this model can reduce the number of hidden nodes by determining the appropriate number of prototype nodes.

Effective learning in recurrent max-min neural networks

Max and min operations have interesting properties that facilitate the exchange of information between the symbolic and real-valued domains. As such, neural networks that employ max-min activation functions have been a subject of interest in recent years. Since max-min functions are not strictly differentiable, we propose a mathematically sound learning method based on using Fourier convergence analysis of side-derivatives to derive a gradient descent technique for max-min error functions. We then propose a novel recurrent max-min neural network model that is trained to perform grammatical inference as an application example. Comparisons made between this model and recurrent sigmoidal neural networks show that our model not only performs better in terms of learning speed and generalization, but that its final weight configuration allows a deterministic finite automation (DFA) to be extracted in a straightforward manner. In essence, we are able to demonstrate that our proposed gradient descent technique does allow max-min neural networks to learn effectively.

Lightweight fuzzy processes in clinical computing

In spite of advances in computing hardware, many hospitals still have a hard time finding extra capacity in their production clinical information system to run artificial intelligence (AI) modules, for example: to support real-time drug-drug or drug-lab interactions; to track infection trends; to monitor compliance with case specific clinical guidelines; or to monitor/ control biomedical devices like an intelligent ventilator. Historically, adding AI functionality was not a major design concern when a typical clinical system is originally specified. AI technology is usually retrofitted 'on top of the old system' or 'run off line' in tandem with the old system to ensure that the routine work load would still get done (with as little impact from the AI side as possible). To compound the burden on system performance, most institutions have witnessed a long and increasing trend for intramural and extramural reporting, (e.g. the collection of data for a quality-control report in microbiology, or a meta-analysis of a suite of coronary artery bypass grafts techniques, etc.) and these place an ever-growing burden on typical the computer system's performance. We discuss a promising approach to adding extra AI processing power to a heavily-used system based on the notion 'lightweight fuzzy processing (LFP)', that is, fuzzy modules designed from the outset to impose a small computational load. A formal model for a useful subclass of fuzzy systems is defined below and is used as a framework for the automated generation of LFPs. By seeking to reduce the arithmetic complexity of the model (a hand-crafted process) and the data complexity of the model (an automated process), we show how LFPs can be generated for three sample datasets of clinical relevance.

Function approximation using fuzzy neural networks with robust learning algorithm

The paper describes a novel application of the B-spline membership functions (BMF's) and the fuzzy neural network to the function approximation with outliers in training data. According to the robust objective function, we use gradient descent method to derive the new learning rules of the weighting values and BMF's of the fuzzy neural network for robust function approximation. In this paper, the robust learning algorithm is derived. During the learning process, the robust objective function comes into effect and the approximated function will gradually be unaffected by the erroneous training data. As a result, the robust function approximation can rapidly converge to the desired tolerable error scope. In other words, the learning iterations will decrease greatly. We realize the function approximation not only in one dimension (curves), but also in two dimension (surfaces). Several examples are simulated in order to confirm the efficiency and feasibility of the proposed approach in this paper.

Design and implementation of the tree-based fuzzy logic controller

In this paper, a tree-based approach is proposed to design the fuzzy logic controller. Based on the proposed methodology, the fuzzy logic controller has the following merits: the fuzzy control rule can be extracted automatically from the input-output data of the system and the extraction process can be done in one-pass; owing to the fuzzy tree inference structure, the search spaces of the fuzzy inference process are largely reduced; the operation of the inference process can be simplified as a one-dimensional matrix operation because of the fuzzy tree approach; and the controller has regular and modular properties, so it is easy to be implemented by hardware. Furthermore, the proposed fuzzy tree approach has been applied to design the color reproduction system for verifying the proposed methodology. The color reproduction system is mainly used to obtain a color image through the printer that is identical to the original one. In addition to the software simulation, an FPGA is used to implement the prototype hardware system for real-time application. Experimental results show that the effect of color correction is quite good and that the prototype hardware system can operate correctly under the condition of 30 MHz clock rate.

Fuzzy inference neural network for fuzzy model tuning

In fuzzy modeling, it is relatively easy to manually define rough fuzzy rules for a target system by intuition. It is, however, time-consuming and difficult to fine-tune them to improve their behavior. This paper describes a tuning method for fuzzy models which is applicable regardless of the form of fuzzy rules and the used defuzzification method. For this purpose, this paper proposes a fuzzy neural network model which can embody fuzzy models. The proposed model provides the functions to perform fuzzy inference and to tune the parameters for the shape of antecedent linguistic terms, the relative importance degrees of rules, and the relative importance degrees of antecedent linguistic terms in rules. In addition, to show its applicability, we perform some experiments and present the results.

Optoelectronic winner-take-all VLSI shunting neural network

In this paper we present an analog winner-take-all MOS VLSI (metal-oxide semiconductor/very large scale integration) optoelectronic network. By varying either the input current or circuit parameters, the circuit can evidence several different behaviors such as contrast enhancement, strict winner-take-all, or winner-take-all with hysteresis. Simulation and experimental results from the prototype circuit are also discussed.