Combinations of weak classifiers

Classification of Motor-Imagery Tasks Using a Large EEG Dataset by Fusing Classifiers Learning on Wavelet-Scattering Features

Brain-computer or brain-machine interface technology allows humans to control machines using their thoughts via brain signals. In particular, these interfaces can assist people with neurological diseases for speech understanding or physical disabilities for operating devices such as wheelchairs. Motor-imagery tasks play a basic role in brain-computer interfaces. This study introduces an approach for classifying motor-imagery tasks in a brain-computer interface environment, which remains a challenge for rehabilitation technology using electroencephalogram sensors. Methods used and developed for addressing the classification include wavelet time and image scattering networks, fuzzy recurrence plots, support vector machines, and classifier fusion. The rationale for combining outputs from two classifiers learning on wavelet-time and wavelet-image scattering features of brain signals, respectively, is that they are complementary and can be effectively fused using a novel fuzzy rule-based system. A large-scale challenging electroencephalogram dataset of motor imagery-based brain-computer interface was used to test the efficacy of the proposed approach. Experimental results obtained from within-session classification show the potential application of the new model that achieves an improvement of 7% in classification accuracy over the best existing classifier using state-of-the-art artificial intelligence (76% versus 69%, respectively). For the cross-session experiment, which imposes a more challenging and practical classification task, the proposed fusion model improves the accuracy by 11% (54% versus 65%). The technical novelty presented herein and its further exploration are promising for developing a reliable sensor-based intervention for assisting people with neurodisability to improve their quality of life.

Linear B-Cell Epitope Prediction for In Silico Vaccine Design: A Performance Review of Methods Available via Command-Line Interface

Linear B-cell epitope prediction research has received a steadily growing interest ever since the first method was developed in 1981. B-cell epitope identification with the help of an accurate prediction method can lead to an overall faster and cheaper vaccine design process, a crucial necessity in the COVID-19 era. Consequently, several B-cell epitope prediction methods have been developed over the past few decades, but without significant success. In this study, we review the current performance and methodology of some of the most widely used linear B-cell epitope predictors which are available via a command-line interface, namely, BcePred, BepiPred, ABCpred, COBEpro, SVMTriP, LBtope, and LBEEP. Additionally, we attempted to remedy performance issues of the individual methods by developing a consensus classifier, which combines the separate predictions of these methods into a single output, accelerating the epitope-based vaccine design. While the method comparison was performed with some necessary caveats and individual methods might perform much better for specialized datasets, we hope that this update in performance can aid researchers towards the choice of a predictor, for the development of biomedical applications such as designed vaccines, diagnostic kits, immunotherapeutics, immunodiagnostic tests, antibody production, and disease diagnosis and therapy.

A Neural Network Ensemble Classifier for Effective Intrusion Detection Using Fuzzy Clustering and Radial Basis Function Networks

Intrusion Detection Systems have considerable importance in preventing security threats and protecting computer networks against attackers. So far, various classification approaches using data mining and machine learning techniques have been proposed to the problem of intrusion detection. However, using single classifier systems for intrusion detection suffers from some limitations including lower detection rate for low-frequent attacks, detection instability, and complexity in training process. Ensemble classifier systems combine several individual classifiers and obtain a classifier with higher performance. In this paper, we propose a new ensemble classifier using Radial Basis Function (RBF) neural networks and fuzzy clustering in order to increase detection accuracy and stability, reduce false positives, and provide higher detection rate for low-frequent attacks. We also use a hybrid combination method to aggregate the individual predictions of the base classifiers, which helps to increase detection accuracy. The experimental results on NSL-KDD data set demonstrate that our proposed system has a higher detection accuracy compared to other wellknown classification systems. It also performs more effectively for detection of low-frequent attacks. Furthermore, the proposed ensemble method offers better performance compared to popular ensemble methods.

Feature and score fusion based multiple classifier selection for iris recognition

The aim of this work is to propose a new feature and score fusion based iris recognition approach where voting method on Multiple Classifier Selection technique has been applied. Four Discrete Hidden Markov Model classifiers output, that is, left iris based unimodal system, right iris based unimodal system, left-right iris feature fusion based multimodal system, and left-right iris likelihood ratio score fusion based multimodal system, is combined using voting method to achieve the final recognition result. CASIA-IrisV4 database has been used to measure the performance of the proposed system with various dimensions. Experimental results show the versatility of the proposed system of four different classifiers with various dimensions. Finally, recognition accuracy of the proposed system has been compared with existing N hamming distance score fusion approach proposed by Ma et al., log-likelihood ratio score fusion approach proposed by Schmid et al., and single level feature fusion approach proposed by Hollingsworth et al.

SSC: a classifier combination method based on signal strength

We propose a new classifier combination method, the signal strength-based combining (SSC) approach, to combine the outputs of multiple classifiers to support the decision-making process in classification tasks. As ensemble learning methods have attracted growing attention from both academia and industry recently, it is critical to understand the fundamental issues of the combining rule. Motivated by the signal strength concept, our proposed SSC algorithm can effectively integrate the individual vote from different classifiers in an ensemble learning system. Comparative studies of our method with nine major existing combining rules, namely, geometric average rule, arithmetic average rule, median value rule, majority voting rule, Borda count, max and min rule, weighted average, and weighted majority voting rules, is presented. Furthermore, we also discuss the relationship of the proposed method with respect to margin-based classifiers, including the boosting method (AdaBoost.M1 and AdaBoost.M2) and support vector machines by margin analysis. Detailed analyses of margin distribution graphs are presented to discuss the characteristics of the proposed method. Simulation results for various real-world datasets illustrate the effectiveness of the proposed method.

Data Mining Techniques in Agricultural and Environmental Sciences

Data mining techniques are largely used in different sectors of the economy and they increasingly are playing an important role in agriculture and environment-related areas. This paper aims to show our vision on the importance of knowing and efficiently using data mining and machine learning-related techniques for knowledge discovery in the field of agriculture and environment. Efforts for searching hidden patterns in data are not a recent phenomenon. History shows that extensive observations on data have helped discover empirical laws in different fields of research. Therefore, it is important to provide researchers in agriculture and environmental-related areas with the most advanced knowledge discovery techniques. Data mining is the process of extracting important and useful information from large sets of data. This information can be converted into useful knowledge that could help to better understand the problem in study and to better predict future developments. The paper presents the state of the art in data mining and knowledge discovery techniques and provides discussions for future directions.

Learning from labeled and unlabeled data using a minimal number of queries

The considerable time and expense required for labeling data has prompted the development of algorithms which maximize the classification accuracy for a given amount of labeling effort. On the one hand, the effort has been to develop the so-called "active learning" algorithms which sequentially choose the patterns to be explicitly labeled so as to realize the maximum information gain from each labeling. On the other hand, the effort has been to develop algorithms that can learn from labeled as well as the more abundant unlabeled data. Proposed in this paper is an algorithm that integrates the benefits of active learning with the benefits of learning from labeled and unlabeled data. Our approach is based on reversing the roles of the labeled and unlabeled data. Specifically, we use a Genetic Algorithm (GA) to iteratively refine the class membership of the unlabeled patterns so that the maximum a posteriori (MAP) based predicted labels of the patterns in the labeled dataset are in agreement with the known labels. This reversal of the role of labeled and unlabeled patterns leads to an implicit class assignment of the unlabeled patterns. For active learning, we use a subset of the GA population to construct multiple MAP classifiers. Points in the input space where there is maximal disagreement amongst these classifiers are then selected for explicit labeling. The learning from labeled and unlabeled data and active learning phases are interlaced and together provide accurate classification while minimizing the labeling effort.

A Kernel-Based Two-Class Classifier for Imbalanced Data Sets

Many kernel classifier construction algorithms adopt classification accuracy as performance metrics in model evaluation. Moreover, equal weighting is often applied to each data sample in parameter estimation. These modeling practices often become problematic if the data sets are imbalanced. We present a kernel classifier construction algorithm using orthogonal forward selection (OFS) in order to optimize the model generalization for imbalanced two-class data sets. This kernel classifier identification algorithm is based on a new regularized orthogonal weighted least squares (ROWLS) estimator and the model selection criterion of maximal leave-one-out area under curve (LOO-AUC) of the receiver operating characteristics (ROCs). It is shown that, owing to the orthogonalization procedure, the LOO-AUC can be calculated via an analytic formula based on the new regularized orthogonal weighted least squares parameter estimator, without actually splitting the estimation data set. The proposed algorithm can achieve minimal computational expense via a set of forward recursive updating formula in searching model terms with maximal incremental LOO-AUC value. Numerical examples are used to demonstrate the efficacy of the algorithm.

Applications of neural network methods to the processing of earth observation satellite data

The new generation of earth observation satellites carries advanced sensors that will gather very precise data for studying the Earth system and global climate. This paper shows that neural network methods can be successfully used for solving forward and inverse remote sensing problems, providing both accurate and fast solutions. Two examples of multi-neural network systems for the determination of cloud properties and for the retrieval of total columns of ozone using satellite data are presented. The developed algorithms based on multi-neural network are currently being used for the operational processing of European atmospheric satellite sensors and will play a key role in related satellite missions planed for the near future.

Feature extraction and supervised classification of MR images to support proton radiation therapy of eye tumors

Proton therapy has the potential for high-precision radiotherapy of retinal tumors. However, the standardized eye models currently used do not fully account for the patient's individual anatomy. To better exploit the data provided by MR images, a model-based approach was used based on a database of eye models. A face recognition algorithm was advanced to define similarity criteria between the reference image and the actual image. After building a high-dimensional feature vector and using a training data set, the reference model was selected by using the minimum Mahalanobis distance between the image to be classified and the reference images.

Prediction games and arcing algorithms

The theory behind the success of adaptive reweighting and combining algorithms (arcing) such as Adaboost (Freund & Schapire, 1996a, 1997) and others in reducing generalization error has not been well understood. By formulating prediction as a game where one player makes a selection from instances in the training set and the other a convex linear combination of predictors from a finite set, existing arcing algorithms are shown to be algorithms for finding good game strategies. The minimax theorem is an essential ingredient of the convergence proofs. An arcing algorithm is described that converges to the optimal strategy. A bound on the generalization error for the combined predictors in terms of their maximum error is proven that is sharper than bounds to date. Schapire, Freund, Bartlett, and Lee (1997) offered an explanation of why Adaboost works in terms of its ability to produce generally high margins. The empirical comparison of Adaboost to the optimal arching algorithm shows that their explanation is not complete.

Inferential estimation of polymer quality using bootstrap aggregated neural networks

Inferential estimation of polymer quality in a batch polymerisation reactor using bootstrap aggregated neural networks is studied in this paper. Number average molecular weight and weight average molecular weight are estimated from the on-line measurements of reactor temperature, jacket inlet and outlet temperatures, coolant flow rate through the jacket, monomer conversion, and the initial batch conditions. Bootstrap aggregated neural networks are used to enhance the accuracy and robustness of neural network models built from a limited amount of training data. The training data set is re-sampled using bootstrap re-sampling with replacement to form several sets of training data. For each set of training data, a neural network model is developed. The individual neural networks are then combined together to form a bootstrap aggregated neural network. Determination of appropriate weights for combining individual networks using principal component regression is proposed in this paper. Confidence bounds for neural network predictions can also be obtained using the bootstrapping technique. The techniques have been successfully applied to the simulation of a batch methyl methacrylate polymerisation reactor.