An enhanced fuzzy min-max neural network for pattern classification

Deep Fuzzy Min-Max Neural Network: Analysis and Design

Fuzzy min-max neural network (FMNN) is one kind of three-layer models based on hyperboxes that are constructed in a sequential way. Such a sequential mechanism inevitably leads to the input order and overlap region problem. In this study, we propose a deep FMNN (DFMNN) based on initialization and optimization operation to overcome these limitations. Initialization operation that can solve the input order problem is to design hyperboxes in a simultaneous way, and side parameters have been proposed to control the size of hyperboxes. Optimization operation that can eliminate overlap region problem is realized by means of deep layers, where the number of layers is immediately determined when the overlap among hyperboxes is eliminated. In the optimization process, each layer consists of three sections, namely, the partition section, combination section, and union section. The partition section aims to divide the hyperboxes into a nonoverlapping hyperbox set and an overlapping hyperbox set. The combination section eliminates the overlap problem of overlapping hyperbox set. The union section obtains the optimized hyperbox set in the current layer. DFMNN is evaluated based on a series of benchmark datasets. A comparative analysis illustrates that the proposed DFMNN model outperforms several models previously reported in the literature.

Self-Organizing Interval Type-2 Fuzzy Neural Network Using Information Aggregation Method

Interval type-2 fuzzy neural networks (IT2FNNs) usually stack adequate fuzzy rules to identify nonlinear systems with high-dimensional inputs, which may result in an explosion of fuzzy rules. To cope with this problem, a self-organizing IT2FNN, based on the information aggregation method (IA-SOIT2FNN), is developed to avoid the explosion of fuzzy rules in this article. First, a relation-aware strategy is proposed to construct rotatable type-2 fuzzy rules (RT2FRs). This strategy uses the individual RT2FR, instead of multiple standard fuzzy rules, to interpret interactive features of high-dimensional inputs. Second, a comprehensive information evaluation mechanism, associated with the interval information and rotation information of RT2FR, is developed to direct the structural adjustment of IA-SOIT2FNN. This mechanism can achieve a compact structure of IA-SOIT2FNN by growing and pruning RT2FRs. Third, a multicriteria-based optimization algorithm is designed to optimize the parameters of IA-SOIT2FNN. The algorithm can simultaneously update the rotatable parameters and the conventional parameters of RT2FR, and further maintain the accuracy of IA-SOIT2FNN. Finally, the experiments showcase that the proposed IA-SOIT2FNN can compete with the state-of-the-art approaches in terms of identification performance.

Random Hyperboxes

This article proposes a simple yet powerful ensemble classifier, called Random Hyperboxes, constructed from individual hyperbox-based classifiers trained on the random subsets of sample and feature spaces of the training set. We also show a generalization error bound of the proposed classifier based on the strength of the individual hyperbox-based classifiers as well as the correlation among them. The effectiveness of the proposed classifier is analyzed using a carefully selected illustrative example and compared empirically with other popular single and ensemble classifiers via 20 datasets using statistical testing methods. The experimental results confirmed that our proposed method outperformed other fuzzy min-max neural networks (FMNNs), popular learning algorithms, and is competitive with other ensemble methods. Finally, we identify the existing issues related to the generalization error bounds of the real datasets and inform the potential research directions.

Generative and Discriminative Fuzzy Restricted Boltzmann Machine Learning for Text and Image Classification

The restricted Boltzmann machine (RBM) is an excellent generative learning model for feature extraction. By extending its parameters from real numbers to fuzzy ones, we have developed the fuzzy RBM (FRBM) which is demonstrated to possess better generative capability than RBM. In this paper, we first propose a generative model named Gaussian FRBM (GFRBM) to deal with real-valued inputs. Then, motivated by the fact that the discriminative variant of RBM can provide a self-contained framework for classification with competitive performance compared with some traditional classifiers, we establish the discriminative FRBM (DFRBM) and discriminative GFRBM (DGFRBM) that combine both the generative and discriminative facility by adding extra neurons next to the input units. Specifically, they can be trained into excellent stand-alone classifiers and retain outstanding generative capability simultaneously. The experimental results including text and image (both clean and noisy) classification indicate that DFRBM and DGFRBM outperform discriminative RBM models in terms of reconstruction and classification accuracy, and they behave more stable when encountering noisy data. Moreover, the proposed learning models show some promising advantages over other standard classifiers.

An Efficient Second-Order Algorithm for Self-Organizing Fuzzy Neural Networks

Intelligent computing technologies are useful and important for online data modeling, where system dynamics may be nonstationary with some uncertainties. In this paper, an efficient learning mechanism is developed for building self-organizing fuzzy neural networks (SOFNNs), where a second-order algorithm (SOA) with adaptive learning rate is employed, the network size and the parameters can be determined simultaneously in the learning process. First, all parameters of SOFNN are adjusted by using the SOA strategy to achieve fast convergence through a powerful search scheme. Second, the structure of SOFNN can be self-organized using the relative importance index of each rule. The fuzzy rules used in SOFNN with SOA (SOA-SOFNN) are generated or pruned automatically to reduce the computational complexity and potentially improve the generalization power. Finally, a theoretical analysis on the learning convergence of the proposed SOA-SOFNN is given to show the computational efficiency. To demonstrate the merits of our proposed approach for data modeling, several benchmark datasets, and a real world application associated with nonlinear systems modeling problems are examined with comparisons against other existing methods. The results indicate that our proposed SOA-SOFNN performs favorably in terms of both learning speed and prediction accuracy for online data modeling.