Fuzzy min-max neural networks. I. Classification

Interpretable optimisation-based approach for hyper-box classification

Data classification is considered a fundamental research subject within the machine learning community. Researchers seek the improvement of machine learning algorithms in not only accuracy, but also interpretability. Interpretable algorithms allow humans to easily understand the decisions that a machine learning model makes, which is challenging for black box models. Mathematical programming-based classification algorithms have attracted considerable attention due to their ability to effectively compete with leading-edge algorithms in terms of both accuracy and interpretability. Meanwhile, the training of a hyper-box classifier can be mathematically formulated as a Mixed Integer Linear Programming (MILP) model and the predictions combine accuracy and interpretability. In this work, an optimisation-based approach is proposed for multi-class data classification using a hyper-box representation, thus facilitating the extraction of compact IF-THEN rules. The key novelty of our approach lies in the minimisation of the number and length of the generated rules for enhanced interpretability. Through a number of real-world datasets, it is demonstrated that the algorithm exhibits favorable performance when compared to well-known alternatives in terms of prediction accuracy and rule set simplicity.

Deep Fuzzy Min-Max Neural Network: Analysis and Design

Fuzzy min-max neural network (FMNN) is one kind of three-layer models based on hyperboxes that are constructed in a sequential way. Such a sequential mechanism inevitably leads to the input order and overlap region problem. In this study, we propose a deep FMNN (DFMNN) based on initialization and optimization operation to overcome these limitations. Initialization operation that can solve the input order problem is to design hyperboxes in a simultaneous way, and side parameters have been proposed to control the size of hyperboxes. Optimization operation that can eliminate overlap region problem is realized by means of deep layers, where the number of layers is immediately determined when the overlap among hyperboxes is eliminated. In the optimization process, each layer consists of three sections, namely, the partition section, combination section, and union section. The partition section aims to divide the hyperboxes into a nonoverlapping hyperbox set and an overlapping hyperbox set. The combination section eliminates the overlap problem of overlapping hyperbox set. The union section obtains the optimized hyperbox set in the current layer. DFMNN is evaluated based on a series of benchmark datasets. A comparative analysis illustrates that the proposed DFMNN model outperforms several models previously reported in the literature.

An evolutionary model for sleep quality analytics using fuzzy system

Electroencephalography (EEG) is a neuro signal reflecting brain activity. These signals provide information about brain activity, eye movements, and muscle tone, which can be used to determine the sleep stage. Categorizing sleep stages can be done manually by visually. Alternatively, automated algorithms can be developed using machine learning techniques to classify sleep stages based on signal features and patterns. This paper aims to automatically classify sleep stages based on extracted patterns from EEG signals. A fuzzy min-max neural network is proposed and implemented for sleep stage classification and clustering. The paper concludes that the fuzzy min-max neural network outperforms other tested methods in sleep stage classification. The models implemented in the study include K-Nearest Neighbor (KNN), Random Forest, Decision Tree, XGBoost, AdaBoost, Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA), Convolutional Neural Network (CNN), and the fuzzy min-max classifier. The results indicate that the fuzzy classifier achieves the highest accuracy of 86%, followed by the CNN model with 81%. Among the machine learning algorithms, Random Forest with an accuracy of 55.46%, followed by XGBoost with 53.18%, surpassing the other algorithms used in the experiment. AdaBoost and Gaussian Naive Bayes both achieve an accuracy of 45.10%. Decision Tree, KNN, LDA, and QDA yield accuracies of 37.66%, 16.46%, 28.57%, and 29.5% respectively. These findings demonstrate the efficiency of the fuzzy min-max neural network and the superiority of the fuzzy classifier and CNN models in sleep stage classification, indicating their potential for accurate automated sleep stage analysis.

Digital Filtering Techniques Using Fuzzy-Rules Based Logic Control

This paper discusses current formulations based on fuzzy-logic control concepts as applied to the removal of impulsive noise from digital images. We also discuss the various principles related to fuzzy-ruled based logic control techniques, aiming at preserving edges and digital image details efficiently. Detailed descriptions of a number of formulations for recently developed fuzzy-rule logic controlled filters are provided, highlighting the merit of each filter. Fuzzy-rule based filtering algorithms may be designed assuming the tailoring of specific functional sub-modules: (a) logical controlled variable selection, (b) the consideration of different methods for the generation of fuzzy rules and membership functions, (c) the integration of the logical rules for detecting and filtering impulse noise from digital images. More specifically, we discuss impulse noise models and window-based filtering using fuzzy inference based on vector directional filters as associated with the filtering of RGB color images and then explain how fuzzy vector fields can be generated using standard operations on fuzzy sets taking into consideration fixed or random valued impulse noise and fuzzy vector partitioning. We also discuss how fuzzy cellular automata may be used for noise removal by adopting a Moore neighbourhood architecture. We also explain the potential merits of adopting a fuzzy rule based deep learning ensemble classifier which is composed of a convolutional neural network (CNN), a recurrent neural networks (RNN), a long short term memory neural network (LSTM) and a gated recurrent unit (GRU) approaches, all within a fuzzy min-max (FMM) ensemble. Fuzzy non-local mean filter approaches are also considered. A comparison of various performance metrics for conventional and fuzzy logic based filters as well as deep learning filters is provided. The algorhitms discussed have the following advantageous properties: high quality of edge preservation, high quality of spatial noise suppression capability especially for complex images, sound properties of noise removal (in cases when both mixed additive and impulse noise are present), and very fast computational implementation.

Design of Granular Model: A Method Driven by Hyper-Box Iteration Granulation

Recently, granular models have been highlighted in system modeling and applied to many fields since their outcomes are information granules supporting human-centric comprehension and reasoning. In this study, a design method of granular model driven by hyper-box iteration granulation is proposed. The method is composed mainly of partition of input space, formation of input hyper-box information granules with confidence levels, and granulation of output data corresponding to input hyper-box information granules. Among them, the formation of input hyper-box information granules is realized through performing the hyper-box iteration granulation algorithm governed by information granularity on input space, and the granulation of out data corresponding to input hyper-box information granules is completed by the improved principle of justifiable granularity to produce triangular fuzzy information granules. Compared with the existing granular models, the resulting one can yield the more accurate numeric and preferable granular outcomes simultaneously. Experiments completed on the synthetic and publicly available datasets demonstrate the superiority of the granular model designed by the proposed method at granular and numeric levels. Also, the impact of parameters involved in the proposed design method on the performance of ensuing granular model is explored.

Hybridization of Deep Learning Pre-Trained Models with Machine Learning Classifiers and Fuzzy Min-Max Neural Network for Cervical Cancer Diagnosis

Medical image analysis and classification is an important application of computer vision wherein disease prediction based on an input image is provided to assist healthcare professionals. There are many deep learning architectures that accept the different medical image modalities and provide the decisions about the diagnosis of various cancers, including breast cancer, cervical cancer, etc. The Pap-smear test is the commonly used diagnostic procedure for early identification of cervical cancer, but it has a high rate of false-positive results due to human error. Therefore, computer-aided diagnostic systems based on deep learning need to be further researched to classify the pap-smear images accurately. A fuzzy min-max neural network is a neuro fuzzy architecture that has many advantages, such as training with a minimum number of passes, handling overlapping class classification, supporting online training and adaptation, etc. This paper has proposed a novel hybrid technique that combines the deep learning architectures with machine learning classifiers and fuzzy min-max neural network for feature extraction and Pap-smear image classification, respectively. The deep learning pretrained models used are Alexnet, ResNet-18, ResNet-50, and GoogleNet. Benchmark datasets used for the experimentation are Herlev and Sipakmed. The highest classification accuracy of 95.33% is obtained using Resnet-50 fine-tuned architecture followed by Alexnet on Sipakmed dataset. In addition to the improved accuracies, the proposed model has utilized the advantages of fuzzy min-max neural network classifiers mentioned in the literature.

Random Hyperboxes

This article proposes a simple yet powerful ensemble classifier, called Random Hyperboxes, constructed from individual hyperbox-based classifiers trained on the random subsets of sample and feature spaces of the training set. We also show a generalization error bound of the proposed classifier based on the strength of the individual hyperbox-based classifiers as well as the correlation among them. The effectiveness of the proposed classifier is analyzed using a carefully selected illustrative example and compared empirically with other popular single and ensemble classifiers via 20 datasets using statistical testing methods. The experimental results confirmed that our proposed method outperformed other fuzzy min-max neural networks (FMNNs), popular learning algorithms, and is competitive with other ensemble methods. Finally, we identify the existing issues related to the generalization error bounds of the real datasets and inform the potential research directions.

Deep-Precognitive Diagnosis: Preventing Future Pandemics by Novel Disease Detection With Biologically-Inspired Conv-Fuzzy Network

Deep learning-based Computer-Aided Diagnosis has gained immense attention in recent years due to its capability to enhance diagnostic performance and elucidate complex clinical tasks. However, conventional supervised deep learning models are incapable of recognizing novel diseases that do not exist in the training dataset. Automated early-stage detection of novel infectious diseases can be vital in controlling their rapid spread. Moreover, the development of a conventional CAD model is only possible after disease outbreaks and datasets become available for training (viz. COVID-19 outbreak). Since novel diseases are unknown and cannot be included in training data, it is challenging to recognize them through existing supervised deep learning models. Even after data becomes available, recognizing new classes with conventional models requires a complete extensive re-training. The present study is the first to report this problem and propose a novel solution to it. In this study, we propose a new class of CAD models, i.e., Deep-Precognitive Diagnosis, wherein artificial agents are enabled to identify unknown diseases that have the potential to cause a pandemic in the future. A de novo biologically-inspired Conv-Fuzzy network is developed. Experimental results show that the model trained to classify Chest X-Ray (CXR) scans into normal and bacterial pneumonia detected a novel disease during testing, unseen by it in the training sample and confirmed to be COVID-19 later. The model is also tested on SARS-CoV-1 and MERS-CoV samples as unseen diseases and achieved state-of-the-art accuracy. The proposed model eliminates the need for model re-training by creating a new class in real-time for the detected novel disease, thus classifying it on all subsequent occurrences. Second, the model addresses the challenge of limited labeled data availability, which renders most supervised learning techniques ineffective and establishes that modified fuzzy classifiers can achieve high accuracy on image classification tasks.

Improvement of grey wolf optimizer with adaptive middle filter to adjust support vector machine parameters to predict diabetes complications

In medical science, collecting and classifying data from various diseases is a vital task. The confused and large amounts of data are problems that prevent us from achieving acceptable results. One of the major problems for diabetic patients is a failure to properly diagnose the disease. As a result of this mistake in diagnosis or failure in early diagnosis, the patient may suffer from complications such as blindness, kidney failure, and cutting off the toes. Nowadays, doctors diagnose the disease by relying on their experience and knowledge and performing complex and time-consuming tests. One of the problems with current diabetic, diagnostic methods is the lack of appropriate features to diagnose the disease and consequently the weakness in its diagnosis, especially in its early stages. Since diabetes diagnosis relies on large amounts of data with many parameters, it is necessary to use machine learning methods such as support vector machine (SVM) to predict the complications of diabetes. One of the disadvantages of SVM is its parameter adjustment, which can be accomplished using metaheuristic algorithms such as particle swarm optimization algorithm (PSO), genetic algorithm, or grey wolf optimizer (GWO). In this paper, after preprocessing and preparing the dataset for data mining, we use SVM to predict complications of diabetes based on selected parameters of a patient acquired by laboratory test using improved GWO. We improve the selection process of GWO by employing dynamic adaptive middle filter, a nonlinear filter that assigns appropriate weight to each value based on the data value. Comparison of the final results of the proposed algorithm with classification methods such as a multilayer perceptron neural network, decision tree, simple Bayes, and temporal fuzzy min–max neural network (TFMM-PSO) shows the superiority of the proposed method over the comparable ones.

A Novel Fuzzy Multilayer Perceptron (F-MLP) for the Detection of Irregularity in Skin Lesion Border Using Dermoscopic Images

Skin lesion border irregularity, which represents the B feature in the ABCD rule, is considered one of the most significant factors in melanoma diagnosis. Since signs that clinicians rely on in melanoma diagnosis involve subjective judgment including visual signs such as border irregularity, this deems it necessary to develop an objective approach to finding border irregularity. Increased research in neural networks has been carried out in recent years mainly driven by the advances of deep learning. Artificial neural networks (ANNs) or multilayer perceptrons have been shown to perform well in supervised learning tasks. However, such networks usually don't incorporate information pertaining the ambiguity of the inputs when training the network, which in turn could affect how the weights are being updated in the learning process and eventually degrading the performance of the network when applied on test data. In this paper, we propose a fuzzy multilayer perceptron (F-MLP) that takes the ambiguity of the inputs into consideration and subsequently reduces the effects of ambiguous inputs on the learning process. A new optimization function, the fuzzy gradient descent, has been proposed to reflect those changes. Moreover, a type-II fuzzy sigmoid activation function has also been proposed which enables finding the range of performance the fuzzy neural network is able to attain. The fuzzy neural network was used to predict the skin lesion border irregularity, where the lesion was firstly segmented from the skin, the lesion border extracted, border irregularity measured using a proposed measure vector, and using the extracted border irregularity measures to train the neural network. The proposed approach outperformed most of the state-of-the-art classification methods in general and its standard neural network counterpart in particular. However, the proposed fuzzy neural network was more time-consuming when training the network.

Distributed dual vigilance fuzzy adaptive resonance theory learns online, retrieves arbitrarily-shaped clusters, and mitigates order dependence

This paper presents a novel adaptive resonance theory (ART)-based modular architecture for unsupervised learning, namely the distributed dual vigilance fuzzy ART (DDVFA). DDVFA consists of a global ART system whose nodes are local fuzzy ART modules. It is equipped with distributed higher-order activation and match functions and a dual vigilance mechanism. Together, these allow DDVFA to perform unsupervised modularization, create multi-prototype cluster representations, retrieve arbitrarily-shaped clusters, and reduce category proliferation. Another important contribution is the reduction of order-dependence, an issue that affects any agglomerative clustering method. This paper demonstrates two approaches for mitigating order-dependence: pre-processing using visual assessment of cluster tendency (VAT) or post-processing using a novel Merge ART module. The former is suitable for batch processing, whereas the latter also works for online learning. Experimental results in online mode carried out on 30 benchmark data sets show that DDVFA cascaded with Merge ART statistically outperformed the best other ART-based systems when samples were randomly presented. Conversely, they were found to be statistically equivalent in offline mode when samples were pre-processed using VAT. Remarkably, performance comparisons to non-ART-based clustering algorithms show that DDVFA (which learns incrementally) was also statistically equivalent to the non-incremental (offline) methods of density-based spatial clustering of applications with noise (DBSCAN), single linkage hierarchical agglomerative clustering (SL-HAC), and k-means, while retaining the appealing properties of ART. Links to the source code and data are provided. Considering the algorithm's simplicity, online learning capability, and performance, it is an ideal choice for many agglomerative clustering applications.

Fuzzy Credit Risk Scoring Rules using FRvarPSO

There is consensus that the best way for reducing insolvency situations in financial institutions is through good risk management, which involves a good client selection process. In the market, there are methodologies for credit scoring, each analyzing a large number of microeconomic and/or macroeconomic variables selected mostly depending on the type of credit to be granted. Since these variables are heterogeneous, the review process carried out by credit analysts takes time. The objective of this article is to propose a solution for this problem by applying fuzzy logic to the creation of classification rules for credit granting. To achieve this, linguistic variables were used to help the analyst interpret the information available from the credit officer. The method proposed here combines the use of fuzzy logic with a neural network and a variable population optimization technique to obtain fuzzy classification rules. It was tested with three databases from financial entities in Ecuador — one credit and savings cooperative and two banks that grant various types of credits. To measure its performance, three benchmarks were used: accuracy, number of classification rules generated, and antecedent length. The results obtained indicate that the hybrid model that is proposed performs better than its previous versions due to the addition of fuzzy logic. At the end of the article, our conclusions are discussed and future research lines are suggested.

New Approach of Features Extraction for Numeral Recognition

This paper proposes a new approach of features extraction based on structural and statistical techniques for handwritten, printed and isolated numeral recognition. The structural technique is inspired from the Freeman code, it consists first of contour detection and closing it by morphological operators. After that, the Freeman code was applied by extending its directions to 24-connectivity instead of 8-connectivity. Then, this technique is combined with the statistical method profile projection to determine the attribute vector of the particular numeral. Numeral recognition is carried out in this work through k-nearest neighbors and fuzzy min-max classification. The recognition rate obtained by the proposed system is improved indicating that the numeral extracted features contain more details.