Deep Fuzzy Min-Max Neural Network: Analysis and Design

Fuzzy min-max neural network (FMNN) is one kind of three-layer models based on hyperboxes that are constructed in a sequential way. Such a sequential mechanism inevitably leads to the input order and overlap region problem. In this study, we propose a deep FMNN (DFMNN) based on initialization and optimization operation to overcome these limitations. Initialization operation that can solve the input order problem is to design hyperboxes in a simultaneous way, and side parameters have been proposed to control the size of hyperboxes. Optimization operation that can eliminate overlap region problem is realized by means of deep layers, where the number of layers is immediately determined when the overlap among hyperboxes is eliminated. In the optimization process, each layer consists of three sections, namely, the partition section, combination section, and union section. The partition section aims to divide the hyperboxes into a nonoverlapping hyperbox set and an overlapping hyperbox set. The combination section eliminates the overlap problem of overlapping hyperbox set. The union section obtains the optimized hyperbox set in the current layer. DFMNN is evaluated based on a series of benchmark datasets. A comparative analysis illustrates that the proposed DFMNN model outperforms several models previously reported in the literature.

Design of Granular Model: A Method Driven by Hyper-Box Iteration Granulation

Recently, granular models have been highlighted in system modeling and applied to many fields since their outcomes are information granules supporting human-centric comprehension and reasoning. In this study, a design method of granular model driven by hyper-box iteration granulation is proposed. The method is composed mainly of partition of input space, formation of input hyper-box information granules with confidence levels, and granulation of output data corresponding to input hyper-box information granules. Among them, the formation of input hyper-box information granules is realized through performing the hyper-box iteration granulation algorithm governed by information granularity on input space, and the granulation of out data corresponding to input hyper-box information granules is completed by the improved principle of justifiable granularity to produce triangular fuzzy information granules. Compared with the existing granular models, the resulting one can yield the more accurate numeric and preferable granular outcomes simultaneously. Experiments completed on the synthetic and publicly available datasets demonstrate the superiority of the granular model designed by the proposed method at granular and numeric levels. Also, the impact of parameters involved in the proposed design method on the performance of ensuing granular model is explored.

Random Hyperboxes

This article proposes a simple yet powerful ensemble classifier, called Random Hyperboxes, constructed from individual hyperbox-based classifiers trained on the random subsets of sample and feature spaces of the training set. We also show a generalization error bound of the proposed classifier based on the strength of the individual hyperbox-based classifiers as well as the correlation among them. The effectiveness of the proposed classifier is analyzed using a carefully selected illustrative example and compared empirically with other popular single and ensemble classifiers via 20 datasets using statistical testing methods. The experimental results confirmed that our proposed method outperformed other fuzzy min-max neural networks (FMNNs), popular learning algorithms, and is competitive with other ensemble methods. Finally, we identify the existing issues related to the generalization error bounds of the real datasets and inform the potential research directions.

Deep-Precognitive Diagnosis: Preventing Future Pandemics by Novel Disease Detection With Biologically-Inspired Conv-Fuzzy Network

Deep learning-based Computer-Aided Diagnosis has gained immense attention in recent years due to its capability to enhance diagnostic performance and elucidate complex clinical tasks. However, conventional supervised deep learning models are incapable of recognizing novel diseases that do not exist in the training dataset. Automated early-stage detection of novel infectious diseases can be vital in controlling their rapid spread. Moreover, the development of a conventional CAD model is only possible after disease outbreaks and datasets become available for training (viz. COVID-19 outbreak). Since novel diseases are unknown and cannot be included in training data, it is challenging to recognize them through existing supervised deep learning models. Even after data becomes available, recognizing new classes with conventional models requires a complete extensive re-training. The present study is the first to report this problem and propose a novel solution to it. In this study, we propose a new class of CAD models, i.e., Deep-Precognitive Diagnosis, wherein artificial agents are enabled to identify unknown diseases that have the potential to cause a pandemic in the future. A de novo biologically-inspired Conv-Fuzzy network is developed. Experimental results show that the model trained to classify Chest X-Ray (CXR) scans into normal and bacterial pneumonia detected a novel disease during testing, unseen by it in the training sample and confirmed to be COVID-19 later. The model is also tested on SARS-CoV-1 and MERS-CoV samples as unseen diseases and achieved state-of-the-art accuracy. The proposed model eliminates the need for model re-training by creating a new class in real-time for the detected novel disease, thus classifying it on all subsequent occurrences. Second, the model addresses the challenge of limited labeled data availability, which renders most supervised learning techniques ineffective and establishes that modified fuzzy classifiers can achieve high accuracy on image classification tasks.

Online Topology Learning by a Gaussian Membership-Based Self-Organizing Incremental Neural Network

In order to extract useful information from data streams, incremental learning has been introduced in more and more data mining algorithms. For instance, a self-organizing incremental neural network (SOINN) has been proposed to extract a topological structure that consists of one or more neural networks to closely reflect the data distribution of data streams. However, SOINN has the tradeoff between deleting previously learned nodes and inserting new nodes, i.e., the stability-plasticity dilemma. Therefore, it is not guaranteed that the topological structure obtained by the SOINN will closely represent data distribution. For solving the stability-plasticity dilemma, we propose a Gaussian membership-based SOINN (Gm-SOINN). Unlike other SOINN-based methods that allow only one node to be identified as a "winner" (the nearest node), the Gm-SOINN uses a Gaussian membership to indicate to which degree the node is a winner. Hence, the Gm-SOINN avoids the topological structure that cannot represent the data distribution because previously learned nodes overly deleted or noisy nodes inserted. In addition, an evolving Gaussian mixture model is integrated into the Gm-SOINN to estimate the density distribution of nodes, thereby avoiding the wrong connection between two nodes. Experiments involving both artificial and real-world data sets indicate that our proposed Gm-SOINN achieves better performance than other topology learning methods.

Can People Really Do Nothing? Handling Annotation Gaps in ADL Sensor Data

In supervised Activities of Daily Living (ADL) recognition systems, annotating collected sensor readings is an essential, yet exhaustive, task. Readings are collected from activity-monitoring sensors in a 24/7 manner. The size of the produced dataset is so huge that it is almost impossible for a human annotator to give a certain label to every single instance in the dataset. This results in annotation gaps in the input data to the adopting learning system. The performance of the recognition system is negatively affected by these gaps. In this work, we propose and investigate three different paradigms to handle these gaps. In the first paradigm, the gaps are taken out by dropping all unlabeled readings. A single “Unknown” or “Do-Nothing” label is given to the unlabeled readings within the operation of the second paradigm. The last paradigm handles these gaps by giving every set of them a unique label identifying the encapsulating certain labels. Also, we propose a semantic preprocessing method of annotation gaps by constructing a hybrid combination of some of these paradigms for further performance improvement. The performance of the proposed three paradigms and their hybrid combination is evaluated using an ADL benchmark dataset containing more than 2.5 × 10 6 sensor readings that had been collected over more than nine months. The evaluation results emphasize the performance contrast under the operation of each paradigm and support a specific gap handling approach for better performance.

Distributed dual vigilance fuzzy adaptive resonance theory learns online, retrieves arbitrarily-shaped clusters, and mitigates order dependence

This paper presents a novel adaptive resonance theory (ART)-based modular architecture for unsupervised learning, namely the distributed dual vigilance fuzzy ART (DDVFA). DDVFA consists of a global ART system whose nodes are local fuzzy ART modules. It is equipped with distributed higher-order activation and match functions and a dual vigilance mechanism. Together, these allow DDVFA to perform unsupervised modularization, create multi-prototype cluster representations, retrieve arbitrarily-shaped clusters, and reduce category proliferation. Another important contribution is the reduction of order-dependence, an issue that affects any agglomerative clustering method. This paper demonstrates two approaches for mitigating order-dependence: pre-processing using visual assessment of cluster tendency (VAT) or post-processing using a novel Merge ART module. The former is suitable for batch processing, whereas the latter also works for online learning. Experimental results in online mode carried out on 30 benchmark data sets show that DDVFA cascaded with Merge ART statistically outperformed the best other ART-based systems when samples were randomly presented. Conversely, they were found to be statistically equivalent in offline mode when samples were pre-processed using VAT. Remarkably, performance comparisons to non-ART-based clustering algorithms show that DDVFA (which learns incrementally) was also statistically equivalent to the non-incremental (offline) methods of density-based spatial clustering of applications with noise (DBSCAN), single linkage hierarchical agglomerative clustering (SL-HAC), and k-means, while retaining the appealing properties of ART. Links to the source code and data are provided. Considering the algorithm's simplicity, online learning capability, and performance, it is an ideal choice for many agglomerative clustering applications.

CATS: A Tool for Clustering the Ensemble of Intrinsically Disordered Peptides on a Flat Energy Landscape

We introduce the combinatorial averaged transient structure (CATS) clustering method as a means to cluster protein structure ensembles based on the distributions of protein backbone descriptor coordinates. In our study, we use phi and psi dihedral angle coordinates of the protein backbone as descriptors due to their translational and rotational invariance. The CATS method was developed to produce unique structure ensembles that are typically difficult to obtain from flat energy landscapes using a one-dimensional separation value (e.g., RMSD cutoff). Through the use of higher-dimensional descriptor coordinates, we remedy structure resolution shortcomings of standard clustering algorithms due to large RMSD fluctuations between structures. We compare the performance of CATS to an RMSD-based clustering method GROMOS, which may not be the best choice for IDP clustering since separation quality heavily relies on cutoff values instead of energy landscape minima. We demonstrate the performance of CATS and GROMOS by analyzing the all-atom molecular dynamics trajectories of the Tau/R2(273-284) fragment in solution with TMAO and urea osmolytes from prior studies. Our study reveals that the CATS method produces more unique clusters than the GROMOS method as a result of higher-dimensional distributions of the descriptor coordinates. The cluster centers produced by CATS correspond to local minima in the multidimensional potential mean force, which generates a structure ensemble that adequately samples the energy landscape.

New Approach of Features Extraction for Numeral Recognition

This paper proposes a new approach of features extraction based on structural and statistical techniques for handwritten, printed and isolated numeral recognition. The structural technique is inspired from the Freeman code, it consists first of contour detection and closing it by morphological operators. After that, the Freeman code was applied by extending its directions to 24-connectivity instead of 8-connectivity. Then, this technique is combined with the statistical method profile projection to determine the attribute vector of the particular numeral. Numeral recognition is carried out in this work through k-nearest neighbors and fuzzy min-max classification. The recognition rate obtained by the proposed system is improved indicating that the numeral extracted features contain more details.

A Temporal Mining Framework for Classifying Un-Evenly Spaced Clinical Data: An Approach for Building Effective Clinical Decision-Making System

BACKGROUND

Clinical time-series data acquired from electronic health records (EHR) are liable to temporal complexities such as irregular observations, missing values and time constrained attributes that make the knowledge discovery process challenging.

OBJECTIVE

This paper presents a temporal rough set induced neuro-fuzzy (TRiNF) mining framework that handles these complexities and builds an effective clinical decision-making system. TRiNF provides two functionalities namely temporal data acquisition (TDA) and temporal classification.

METHOD

In TDA, a time-series forecasting model is constructed by adopting an improved double exponential smoothing method. The forecasting model is used in missing value imputation and temporal pattern extraction. The relevant attributes are selected using a temporal pattern based rough set approach. In temporal classification, a classification model is built with the selected attributes using a temporal pattern induced neuro-fuzzy classifier.

RESULT

For experimentation, this work uses two clinical time series dataset of hepatitis and thrombosis patients. The experimental result shows that with the proposed TRiNF framework, there is a significant reduction in the error rate, thereby obtaining the classification accuracy on an average of 92.59% for hepatitis and 91.69% for thrombosis dataset.

CONCLUSION

The obtained classification results prove the efficiency of the proposed framework in terms of its improved classification accuracy.

Semi-Supervised Fuzzy Clustering with Feature Discrimination

Semi-supervised clustering algorithms are increasingly employed for discovering hidden structure in data with partially labelled patterns. In order to make the clustering approach useful and acceptable to users, the information provided must be simple, natural and limited in number. To improve recognition capability, we apply an effective feature enhancement procedure to the entire data-set to obtain a single set of features or weights by weighting and discriminating the information provided by the user. By taking pairwise constraints into account, we propose a semi-supervised fuzzy clustering algorithm with feature discrimination (SFFD) incorporating a fully adaptive distance function. Experiments on several standard benchmark data sets demonstrate the effectiveness of the proposed method.

An enhanced fuzzy min-max neural network for pattern classification

An enhanced fuzzy min-max (EFMM) network is proposed for pattern classification in this paper. The aim is to overcome a number of limitations of the original fuzzy min-max (FMM) network and improve its classification performance. The key contributions are three heuristic rules to enhance the learning algorithm of FMM. First, a new hyperbox expansion rule to eliminate the overlapping problem during the hyperbox expansion process is suggested. Second, the existing hyperbox overlap test rule is extended to discover other possible overlapping cases. Third, a new hyperbox contraction rule to resolve possible overlapping cases is provided. Efficacy of EFMM is evaluated using benchmark data sets and a real medical diagnosis task. The results are better than those from various FMM-based models, support vector machine-based, Bayesian-based, decision tree-based, fuzzy-based, and neural-based classifiers. The empirical findings show that the newly introduced rules are able to realize EFMM as a useful model for undertaking pattern classification problems.

Simplified interval type-2 fuzzy neural networks

This paper describes a self-evolving interval type-2 fuzzy neural network (FNN) for various applications. As type-1 fuzzy systems cannot effectively handle uncertainties in information within the knowledge base, we propose a simple interval type-2 FNN, which uses interval type-2 fuzzy sets in the premise and the Takagi-Sugeno-Kang (TSK) type in the consequent of the fuzzy rule. The TSK-type consequent of fuzzy rule is a linear combination of exogenous input variables. Given an initially empty the rule-base, all rules are generated with on-line type-2 fuzzy clustering. Instead of the time-consuming K-M iterative procedure, the design factors ql and qr are learned to adaptively adjust the upper and lower positions on the left and right limit outputs, using the parameter update rule based on a gradient descent algorithm. Simulation results demonstrate that our approach yields fewer test errors and less computational complexity than other type-2 FNNs.

Online motor fault detection and diagnosis using a hybrid FMM-CART model

In this brief, a hybrid model combining the fuzzy min-max (FMM) neural network and the classification and regression tree (CART) for online motor detection and diagnosis tasks is described. The hybrid model, known as FMM-CART, exploits the advantages of both FMM and CART for undertaking data classification and rule extraction problems. To evaluate the applicability of the proposed FMM-CART model, an evaluation with a benchmark data set pertaining to electrical motor bearing faults is first conducted. The results obtained are equivalent to those reported in the literature. Then, a laboratory experiment for detecting and diagnosing eccentricity faults in an induction motor is performed. In addition to producing accurate results, useful rules in the form of a decision tree are extracted to provide explanation and justification for the predictions from FMM-CART. The experimental outcome positively shows the potential of FMM-CART in undertaking online motor fault detection and diagnosis tasks.

Multi-level fuzzy min-max neural network classifier

In this paper a multi-level fuzzy min-max neural network classifier (MLF), which is a supervised learning method, is described. MLF uses basic concepts of the fuzzy min-max (FMM) method in a multi-level structure to classify patterns. This method uses separate classifiers with smaller hyperboxes in different levels to classify the samples that are located in overlapping regions. The final output of the network is formed by combining the outputs of these classifiers. MLF is capable of learning nonlinear boundaries with a single pass through the data. According to the obtained results, the MLF method, compared to the other FMM networks, has the highest performance and the lowest sensitivity to maximum size of the hyperbox parameter (θ), with a training accuracy of 100% in most cases.

Evolving granular neural networks from fuzzy data streams

This paper introduces a granular neural network framework for evolving fuzzy system modeling from fuzzy data streams. The evolving granular neural network (eGNN) is able to handle gradual and abrupt parameter changes typical of nonstationary (online) environments. eGNN builds interpretable multi-sized local models using fuzzy neurons for information fusion. An online incremental learning algorithm develops the neural network structure from the information contained in data streams. We focus on trapezoidal fuzzy intervals and objects with trapezoidal membership function representation. More precisely, the framework considers triangular, interval, and numeric types of data to construct granular fuzzy models as particular arrangements of trapezoids. Application examples in classification and function approximation in material and biomedical engineering are used to evaluate and illustrate the neural network usefulness. Simulation results suggest that the eGNN fuzzy modeling approach can handle fuzzy data successfully and outperforms alternative state-of-the-art approaches in terms of accuracy, transparency and compactness.