Online support vector machine based on convex hull vertices selection

Dual feature-based and example-based explanation methods

A new approach to the local and global explanation based on selecting a convex hull constructed for the finite number of points around an explained instance is proposed. The convex hull allows us to consider a dual representation of instances in the form of convex combinations of extreme points of a produced polytope. Instead of perturbing new instances in the Euclidean feature space, vectors of convex combination coefficients are uniformly generated from the unit simplex, and they form a new dual dataset. A dual linear surrogate model is trained on the dual dataset. The explanation feature importance values are computed by means of simple matrix calculations. The approach can be regarded as a modification of the well-known model LIME. The dual representation inherently allows us to get the example-based explanation. The neural additive model is also considered as a tool for implementing the example-based explanation approach. Many numerical experiments with real datasets are performed for studying the approach. A code of proposed algorithms is available. The proposed results are fundamental and can be used in various application areas. They do not involve specific human subjects and human data.

Online Active Continual Learning for Robotic Lifelong Object Recognition

In real-world applications, robotic systems collect vast amounts of new data from ever-changing environments over time. They need to continually interact and learn new knowledge from the external world to adapt to the environment. Particularly, lifelong object recognition in an online and interactive manner is a crucial and fundamental capability for robotic systems. To meet this realistic demand, in this article, we propose an online active continual learning (OACL) framework for robotic lifelong object recognition, in the scenario of both classes and domains changing with dynamic environments. First, to reduce the labeling cost as much as possible while maximizing the performance, a new online active learning (OAL) strategy is designed by taking both the uncertainty and diversity of samples into account to protect the information volume and distribution of data. In addition, to prevent catastrophic forgetting and reduce memory costs, a novel online continual learning (OCL) algorithm is proposed based on the deep feature semantic augmentation and a new loss-based deep model and replay buffer update, which can mitigate the class imbalance between the old and new classes and alleviate confusion between two similar classes. Moreover, the mistake bound of the proposed method is analyzed in theory. OACL allows robots to select the most representative new samples to query labels and continually learn new objects and new variants of previously learned objects from a nonindependent and identically distributed (i.i.d.) data stream without catastrophic forgetting. Extensive experiments conducted on real lifelong robotic vision datasets demonstrate that our algorithm, even trained with fewer labeled samples and replay exemplars, can achieve state-of-the-art performance on OCL tasks.

Online Semisupervised Active Classification for Multiview PolSAR Data

Polarimetric synthetic aperture radar (PolSAR) data are sequentially acquired and have multiple views obtained from different feature extractors or multiple frequency bands. The fast and accurate classification of PolSAR data in dynamically changing environments is a critical and challenging task. Online learning can handle this task by learning a classifier incrementally from a stream of samples. In this article, we propose an online semisupervised active learning framework for multiview PolSAR data classification, called OSAM. First, a novel online active learning strategy is designed based on the relationships among multiple views and a randomized rule, which allows to only query the labels of some informative incoming samples. Then, in order to utilize both the incoming labeled and unlabeled samples to update the classifiers, a novel online semisupervised learning model is proposed based on co-regularized multiview learning and graph regularization. In addition, the proposed method can deal with the dynamic large-scale multifeature or multifrequency PolSAR data where not only the amount of data but also the number of classes gradually increases in the learning process. Moreover, the mistake bound of the proposed method is derived rigorously. Extensive experiments are conducted on real PolSAR data to evaluate the performance of our algorithm, and the results demonstrate the effectiveness of the proposed method.

Intelligent Diagnosis Method for New Diseases Based on Fuzzy SVM Incremental Learning

The diagnosis of new diseases is a challenging problem. In the early stage of the emergence of new diseases, there are few case samples; this may lead to the low accuracy of intelligent diagnosis. Because of the advantages of support vector machine (SVM) in dealing with small sample problems, it is selected for the intelligent diagnosis method. The standard SVM diagnosis model updating needs to retrain all samples. It costs huge storage and calculation costs and is difficult to adapt to the changing reality. In order to solve this problem, this paper proposes a new disease diagnosis method based on Fuzzy SVM incremental learning. According to SVM theory, the support vector set and boundary sample set related to the SVM diagnosis model are extracted. Only these sample sets are considered in incremental learning to ensure the accuracy and reduce the cost of calculation and storage. To reduce the impact of noise points caused by the reduction of training samples, FSVM is used to update the diagnosis model, and the generalization is improved. The simulation results on the banana dataset show that the proposed method can improve the classification accuracy from 86.4% to 90.4%. Finally, the method is applied in COVID-19's diagnostic. The diagnostic accuracy reaches 98.2% as the traditional SVM only gets 84%. With the increase of the number of case samples, the model is updated. When the training samples increase to 400, the number of samples participating in training is only 77; the amount of calculation of the updated model is small.

Dynamic sampling of images from various categories for classification based incremental deep learning in fog computing

Incremental learning evolves deep neural network knowledge over time by learning continuously from new data instead of training a model just once with all data present before the training starts. However, in incremental learning, new samples are always streaming in whereby the model to be trained needs to continuously adapt to new samples. Images are considered to be high dimensional data and thus training deep neural networks on such data is very time-consuming. Fog computing is a paradigm that uses fog devices to carry out computation near data sources to reduce the computational load on the server. Fog computing allows democracy in deep learning by enabling intelligence at the fog devices, however, one of the main challenges is the high communication costs between fog devices and the centralized servers especially in incremental learning where data samples are continuously arriving and need to be transmitted to the server for training. While working with Convolutional Neural Networks (CNN), we demonstrate a novel data sampling algorithm that discards certain training images per class before training even starts which reduces the transmission cost from the fog device to the server and the model training time while maintaining model learning performance both for static and incremental learning. Results show that our proposed method can effectively perform data sampling regardless of the model architecture, dataset, and learning settings.

An Effective Machine Learning Approach for Identifying Non-Severe and Severe Coronavirus Disease 2019 Patients in a Rural Chinese Population: The Wenzhou Retrospective Study

This paper has proposed an effective intelligent prediction model that can well discriminate and specify the severity of Coronavirus Disease 2019 (COVID-19) infection in clinical diagnosis and provide a criterion for clinicians to weigh scientific and rational medical decision-making. With indicators as the age and gender of the patients and 26 blood routine indexes, a severity prediction framework for COVID-19 is proposed based on machine learning techniques. The framework consists mainly of a random forest and a support vector machine (SVM) model optimized by a slime mould algorithm (SMA). When the random forest was used to identify the key factors, SMA was employed to train an optimal SVM model. Based on the COVID-19 data, comparative experiments were conducted between RF-SMA-SVM and several well-known machine learning algorithms performed. The results indicate that the proposed RF-SMA-SVM not only achieves better classification performance and higher stability on four metrics, but also screens out the main factors that distinguish severe COVID-19 patients from non-severe ones. Therefore, there is a conclusion that the RF-SMA-SVM model can provide an effective auxiliary diagnosis scheme for the clinical diagnosis of COVID-19 infection.

Large Margin Gaussian Mixture Classifier With a Gabriel Graph Geometric Representation of Data Set Structure

This brief presents a geometrical approach for obtaining large margin classifiers. The method aims at exploring the geometrical properties of the data set from the structure of a Gabriel graph, which represents pattern relations according to a given distance metric, such as the Euclidean distance. Once the graph is generated, geometrical support vectors (SVs) (analogous to support vector machines (SVMs) SVs) are obtained in order to yield the final large margin solution from a Gaussian mixture model. Experiments with 20 data sets have shown that the solutions obtained with the proposed method are statistically equivalent to those obtained with SVMs. However, the present method does not require optimization and can also be extended to large data sets using the cascade SVM concept.

Diagnosing Coronavirus Disease 2019 (COVID-19): Efficient Harris Hawks-Inspired Fuzzy K-Nearest Neighbor Prediction Methods

This study is devoted to proposing a useful intelligent prediction model to distinguish the severity of COVID-19, to provide a more fair and reasonable reference for assisting clinical diagnostic decision-making. Based on patients' necessary information, pre-existing diseases, symptoms, immune indexes, and complications, this article proposes a prediction model using the Harris hawks optimization (HHO) to optimize the Fuzzy K-nearest neighbor (FKNN), which is called HHO-FKNN. This model is utilized to distinguish the severity of COVID-19. In HHO-FKNN, the purpose of introducing HHO is to optimize the FKNN's optimal parameters and feature subsets simultaneously. Also, based on actual COVID-19 data, we conducted a comparative experiment between HHO-FKNN and several well-known machine learning algorithms, which result shows that not only the proposed HHO-FKNN can obtain better classification performance and higher stability on the four indexes but also screen out the key features that distinguish severe COVID-19 from mild COVID-19. Therefore, we can conclude that the proposed HHO-FKNN model is expected to become a useful tool for COVID-19 prediction.

A Fast Algorithm of Convex Hull Vertices Selection for Online Classification

Reducing samples through convex hull vertices selection (CHVS) within each class is an important and effective method for online classification problems, since the classifier can be trained rapidly with the selected samples. However, the process of CHVS is NP-hard. In this paper, we propose a fast algorithm to select the convex hull vertices, based on the convex hull decomposition and the property of projection. In the proposed algorithm, the quadratic minimization problem of computing the distance between a point and a convex hull is converted into a linear equation problem with a low computational complexity. When the data dimension is high, an approximate, instead of exact, convex hull is allowed to be selected by setting an appropriate termination condition in order to delete more nonimportant samples. In addition, the impact of outliers is also considered, and the proposed algorithm is improved by deleting the outliers in the initial procedure. Furthermore, a dimension convention technique via the kernel trick is used to deal with nonlinearly separable problems. An upper bound is theoretically proved for the difference between the support vector machines based on the approximate convex hull vertices selected and all the training samples. Experimental results on both synthetic and real data sets show the effectiveness and validity of the proposed algorithm.

Online SVM-based personalizing method for the drowsiness detection of drivers

Inter-driver variation is one of major problems of the drowsiness detecting system-based on physiological signals. This paper proposes an online support vector machine (OSVM)-based method to solve the problem by the inter-driver variation. The method personalizes the drowsiness detecting system for a certain real user using feedback data from the user. The OSVM selects important data in previous training data and retrains itself with new feedback data for the personalization. Two OSVMs having different initial training data are personalized by the feedback data, and a switching method of the two OSVMs is used in the proposed method for low initial error and fast adaptation. Simulation was conducted using the data obtained by a wearable device and an indoor driving simulator, and the usefulness of the proposed method was validated. The detecting accuracy was increased from 72.05 % to 95.66 % on average for 28 subjects. By feedback data and the proposed method, more accurate drowsiness detection will be possible and it will increase the safety of drivers.

Improved Fault Classification in Series Compensated Transmission Line: Comparative Evaluation of Chebyshev Neural Network Training Algorithms

This paper presents the Chebyshev neural network (ChNN) as an improved artificial intelligence technique for power system protection studies and examines the performances of two ChNN learning algorithms for fault classification of series compensated transmission line. The training algorithms are least-square Levenberg-Marquardt (LSLM) and recursive least-square algorithm with forgetting factor (RLSFF). The performances of these algorithms are assessed based on their generalization capability in relating the fault current parameters with an event of fault in the transmission line. The proposed algorithm is fast in response as it utilizes postfault samples of three phase currents measured at the relaying end corresponding to half-cycle duration only. After being trained with only a small part of the generated fault data, the algorithms have been tested over a large number of fault cases with wide variation of system and fault parameters. Based on the studies carried out in this paper, it has been found that although the RLSFF algorithm is faster for training the ChNN in the fault classification application for series compensated transmission lines, the LSLM algorithm has the best accuracy in testing. The results prove that the proposed ChNN-based method is accurate, fast, easy to design, and immune to the level of compensations. Thus, it is suitable for digital relaying applications.

Online State-Based Structured SVM Combined With Incremental PCA for Robust Visual Tracking

In this paper, we propose a robust state-based structured support vector machine (SVM) tracking algorithm combined with incremental principal component analysis (PCA). Different from the current structured SVM for tracking, our method directly learns and predicts the object's states and not the 2-D translation transformation during tracking. We define the object's virtual state to combine the state-based structured SVM and incremental PCA. The virtual state is considered as the most confident state of the object in every frame. The incremental PCA is used to update the virtual feature vector corresponding to the virtual state and the principal subspace of the object's feature vectors. In order to improve the accuracy of the prediction, all the feature vectors are projected onto the principal subspace in the learning and prediction process of the state-based structured SVM. Experimental results on several challenging video sequences validate the effectiveness and robustness of our approach.