Radius margin bounds for support vector machines with the RBF kernel

Exploring recurrence quantification analysis and fractal dimension algorithms for diagnosis of encephalopathy

Electroencephalography (EEG) is a crucial non-invasive medical tool for diagnosing neurological disorder called encephalopathy. There is a requirement for powerful signal processing algorithms as EEG patterns in encephalopathies are not specific to a particular etiology. As visual examination and linear methods of EEG analysis are not sufficient to get the subtle information regarding various neuro pathologies, non-linear analysis methods can be employed for exploring the dynamic, complex and chaotic nature of EEG signals. This work aims identifying and differentiating the patterns specific to cerebral dysfunctions associated with Encephalopathy using Recurrence Quantification Analysis and Fractal Dimension algorithms. This study analysed six RQA features, namely, recurrence rate, determinism, laminarity, diagonal length, diagonal entropy and trapping time and comparing them with fractal dimensions, namely, Higuchi's and Katz's fractal dimension. Fractal dimensions were found to be lower for encephalopathy cases showing decreased complexity when compared to that of normal healthy subjects. On the other hand, RQA features were found to be higher for encephalopathy cases indicating higher recurrence and more periodic patterns in EEGs of encephalopathy compared to that of normal healthy controls. The feature reduction was then performed using Principal Component Analysis and fed to three promising classifiers: SVM, Random Forest and Multi-layer Perceptron. The resultant system provides a practically realizable pipeline for the diagnosis of encephalopathy.

Real-time monitoring of carbon concentration using laser-induced breakdown spectroscopy and machine learning

The spectral analysis based on laser-induced breakdown spectroscopy (LIBS) is an effective approach to carbon concentration monitoring. In this work, a novel LIBS-based method, together with a system designed independently, was developed for carbon monitoring. The experiments were conducted in two modes: static and dynamic. In static monitoring, gases in three scenarios were selected to represent different carbon concentrations, based on which measurements of carbon concentrations were performed through a mathematical model. Then, K-nearest Neighbors (KNN) was adopted for classification, and its accuracy could reach 99.17%, which can be applied for the identification of gas composition and pollution traceability. In dynamic monitoring, respiration and fossil fuel combustion were selected because of their important roles in increasing carbon concentration. In addition, the simulation of combustion degree was performed by the radial basis function (RBF) based on the spectral information, where the accuracy reached 96.41%, which is the first time that LIBS is proposed to be used for combustion prediction. The innovative approach derived from LIBS and machine learning algorithms is fast, online, and in-situ, showing far-reaching application prospects in real-time monitoring of carbon concentrations.

Predicting the Critical Number of Layers for Hierarchical Support Vector Regression

Hierarchical support vector regression (HSVR) models a function from data as a linear combination of SVR models at a range of scales, starting at a coarse scale and moving to finer scales as the hierarchy continues. In the original formulation of HSVR, there were no rules for choosing the depth of the model. In this paper, we observe in a number of models a phase transition in the training error-the error remains relatively constant as layers are added, until a critical scale is passed, at which point the training error drops close to zero and remains nearly constant for added layers. We introduce a method to predict this critical scale a priori with the prediction based on the support of either a Fourier transform of the data or the Dynamic Mode Decomposition (DMD) spectrum. This allows us to determine the required number of layers prior to training any models.

Parameter Selection for Linear Support Vector Regression

In linear support vector regression (SVR), the regularization and error sensitivity parameters are used to avoid overfitting the training data. A proper selection of parameters is very essential for obtaining a good model, but the search process may be complicated and time-consuming. In an earlier work by Chu et al. (2015), an effective parameter-selection procedure by using warm-start techniques to solve a sequence of optimization problems has been proposed for linear classification. We extend their techniques to linear SVR, but address some new and challenging issues. In particular, linear classification involves only the regularization parameter, but linear SVR has an extra error sensitivity parameter. We investigate the effective range of each parameter and the sequence in checking the two parameters. Based on this work, an effective tool for the selection of parameters for linear SVR has been available for public use.

Intelligent Clustering and Dynamic Incremental Learning to Generate Multi-Codebook Fuzzy Neural Network for Multi-Modal Data Classification

Classification in multi-modal data is one of the challenges in the machine learning field. The multi-modal data need special treatment as its features are distributed in several areas. This study proposes multi-codebook fuzzy neural networks by using intelligent clustering and dynamic incremental learning for multi-modal data classification. In this study, we utilized intelligent K-means clustering based on anomalous patterns and intelligent K-means clustering based on histogram information. In this study, clustering is used to generate codebook candidates before the training process, while incremental learning is utilized when the condition to generate a new codebook is sufficient. The condition to generate a new codebook in incremental learning is based on the similarity of the winner class and other classes. The proposed method was evaluated in synthetic and benchmark datasets. The experiment results showed that the proposed multi-codebook fuzzy neural networks that use dynamic incremental learning have significant improvements compared to the original fuzzy neural networks. The improvements were 15.65%, 5.31% and 11.42% on the synthetic dataset, the benchmark dataset, and the average of all datasets, respectively, for incremental version 1. The incremental learning version 2 improved by 21.08% 4.63%, and 14.35% on the synthetic dataset, the benchmark dataset, and the average of all datasets, respectively. The multi-codebook fuzzy neural networks that use intelligent clustering also had significant improvements compared to the original fuzzy neural networks, achieving 23.90%, 2.10%, and 15.02% improvements on the synthetic dataset, the benchmark dataset, and the average of all datasets, respectively.

A Quantum-Inspired Similarity Measure for the Analysis of Complete Weighted Graphs

We develop a novel method for measuring the similarity between complete weighted graphs, which are probed by means of the discrete-time quantum walks. Directly probing complete graphs using discrete-time quantum walks is intractable due to the cost of simulating the quantum walk. We overcome this problem by extracting a commute time minimum spanning tree from the complete weighted graph. The spanning tree is probed by a discrete-time quantum walk which is initialized using a weighted version of the Perron-Frobenius operator. This naturally encapsulates the edge weight information for the spanning tree extracted from the original graph. For each pair of complete weighted graphs to be compared, we simulate a discrete-time quantum walk on each of the corresponding commute time minimum spanning trees and, then, compute the associated density matrices for the quantum walks. The probability of the walk visiting each edge of the spanning tree is given by the diagonal elements of the density matrices. The similarity between each pair of graphs is then computed using either: 1) the inner product or 2) the negative exponential of the Jensen-Shannon divergence between the probability distributions. We show that in both cases the resulting similarity measure is positive definite and, therefore, corresponds to a kernel on the graphs. We perform a series of experiments on publicly available graph datasets from a variety of different domains, together with time-varying financial networks extracted from data for the New York Stock Exchange. Our experiments demonstrate the effectiveness of the proposed similarity measures.

Shared Gaussian Process Latent Variable Model for Incomplete Multiview Clustering

These days, many multiview learning methods have been proposed by integrating the complementary information of multiple views and can significantly improve the performance of machine learning tasks comparing with single-view learning methods. However, most of these methods fail to learn better models when the multiview data are unpaired (or partially paired) or incomplete (or partially complete). Although some previous attempts have been made to address these problems, these methods often lead to poor results when dealing with incomplete multiview data that contain a relatively large number of missing instances. In fact, this incomplete problem is more challenging than the unpaired problem since less shared information can be caught by the model in the former case. In this paper, we propose a shared Gaussian process (GP) latent variable model for incomplete multiview clustering to gain the merits of two worlds (i.e., GP and multiview learning). Specifically, it learns a set of intentionally aligned representative auxiliary points in individual views jointly to not only compensate for missing instances but also implement the group-level constraint. Thus, the shared information among these views can be explicitly built into the model. All of the hyper-parameters and auxiliary points are simultaneously learned by variational inference. Compared with the existing methods, our method naturally inherits the advantages of GP. Furthermore, it is also straightforwardly extended to cases with more than two views without adding any complexity in formulation. In the experiments, we compare it with the state-of-the-art methods for incomplete multiview data clustering to demonstrate its superiorities.

Combustible Gas Classification Modeling using Support Vector Machine and Pairing Plot Scheme

Combustible gases, such as CH₄ and CO, directly or indirectly affect the human body. Thus, leakage detection of combustible gases is essential for various industrial sites and daily life. Many types of gas sensors are used to identify these combustible gases, but since gas sensors generally have low selectivity among gases, coupling issues often arise which adversely affect gas detection accuracy. To solve this problem, we built a decoupling algorithm with different gas sensors using a machine learning algorithm. Commercially available semiconductor sensors were employed to detect CH₄ and CO, and then support vector machine (SVM) applied as a supervised learning algorithm for gas classification. We also introduced a pairing plot scheme to more effectively classify gas type. The proposed model classified CH₄ and CO gases 100% correctly at all levels above the minimum concentration the gas sensors could detect. Consequently, SVM with pairing plot is a memory efficient and promising method for more accurate gas classification.

SVM-Based Classification of sEMG Signals for Upper-Limb Self-Rehabilitation Training

Robot-assisted rehabilitation is a growing field that can provide an intensity, quality, and quantity of treatment that exceed therapist-mediated rehabilitation. Several control algorithms have been implemented in rehabilitation robots to develop a patient-cooperative strategy with the capacity to understand the intention of the user and provide suitable rehabilitation training. In this paper, we present an upper-limb motion pattern recognition method using surface electromyography (sEMG) signals with a support vector machine (SVM) to control a rehabilitation robot, ReRobot, which was built to conduct upper-limb rehabilitation training for post-stroke patients. For poststroke rehabilitation training using the ReRobot, the upper-limb motion of the patient's healthy side is first recognized by detecting and processing the sEMG signals; then, the ReRobot assists the impaired arm in conducting mirror rehabilitation therapy. To train and test the SVM model, five healthy subjects participated in the experiments and performed five standard upper-limb motions, including shoulder flexion, abduction, internal rotation, external rotation, and elbow joint flexion. Good accuracy was demonstrated in experimental results from the five healthy subjects. By recognizing the model motion of the healthy side, the rehabilitation robot can provide mirror therapy to the affected side. This method can be used as a control strategy of upper-limb rehabilitation robots for self-rehabilitation training with stroke patients.

Improved Prediction of Blood-Brain Barrier Permeability Through Machine Learning with Combined Use of Molecular Property-Based Descriptors and Fingerprints

Blood-brain barrier (BBB) permeability of a compound determines whether the compound can effectively enter the brain. It is an essential property which must be accounted for in drug discovery with a target in the brain. Several computational methods have been used to predict the BBB permeability. In particular, support vector machine (SVM), which is a kernel-based machine learning method, has been used popularly in this field. For SVM training and prediction, the compounds are characterized by molecular descriptors. Some SVM models were based on the use of molecular property-based descriptors (including 1D, 2D, and 3D descriptors) or fragment-based descriptors (known as the fingerprints of a molecule). The selection of descriptors is critical for the performance of a SVM model. In this study, we aimed to develop a generally applicable new SVM model by combining all of the features of the molecular property-based descriptors and fingerprints to improve the accuracy for the BBB permeability prediction. The results indicate that our SVM model has improved accuracy compared to the currently available models of the BBB permeability prediction.

A Genetic Algorithm Based Support Vector Machine Model for Blood-Brain Barrier Penetration Prediction

Blood-brain barrier (BBB) is a highly complex physical barrier determining what substances are allowed to enter the brain. Support vector machine (SVM) is a kernel-based machine learning method that is widely used in QSAR study. For a successful SVM model, the kernel parameters for SVM and feature subset selection are the most important factors affecting prediction accuracy. In most studies, they are treated as two independent problems, but it has been proven that they could affect each other. We designed and implemented genetic algorithm (GA) to optimize kernel parameters and feature subset selection for SVM regression and applied it to the BBB penetration prediction. The results show that our GA/SVM model is more accurate than other currently available log BB models. Therefore, to optimize both SVM parameters and feature subset simultaneously with genetic algorithm is a better approach than other methods that treat the two problems separately. Analysis of our log BB model suggests that carboxylic acid group, polar surface area (PSA)/hydrogen-bonding ability, lipophilicity, and molecular charge play important role in BBB penetration. Among those properties relevant to BBB penetration, lipophilicity could enhance the BBB penetration while all the others are negatively correlated with BBB penetration.

Support vector machine firefly algorithm based optimization of lens system

Lens system design is an important factor in image quality. The main aspect of the lens system design methodology is the optimization procedure. Since optimization is a complex, nonlinear task, soft computing optimization algorithms can be used. There are many tools that can be employed to measure optical performance, but the spot diagram is the most useful. The spot diagram gives an indication of the image of a point object. In this paper, the spot size radius is considered an optimization criterion. Intelligent soft computing scheme support vector machines (SVMs) coupled with the firefly algorithm (FFA) are implemented. The performance of the proposed estimators is confirmed with the simulation results. The result of the proposed SVM-FFA model has been compared with support vector regression (SVR), artificial neural networks, and generic programming methods. The results show that the SVM-FFA model performs more accurately than the other methodologies. Therefore, SVM-FFA can be used as an efficient soft computing technique in the optimization of lens system designs.

Machine learning applications in proteomics research: how the past can boost the future

Machine learning is a subdiscipline within artificial intelligence that focuses on algorithms that allow computers to learn solving a (complex) problem from existing data. This ability can be used to generate a solution to a particularly intractable problem, given that enough data are available to train and subsequently evaluate an algorithm on. Since MS-based proteomics has no shortage of complex problems, and since publicly available data are becoming available in ever growing amounts, machine learning is fast becoming a very popular tool in the field. We here therefore present an overview of the different applications of machine learning in proteomics that together cover nearly the entire wet- and dry-lab workflow, and that address key bottlenecks in experiment planning and design, as well as in data processing and analysis.

Improved accuracy of myocardial perfusion SPECT for the detection of coronary artery disease using a support vector machine algorithm

We aimed to improve the diagnostic accuracy of automatic myocardial perfusion SPECT (MPS) interpretation analysis for the prediction of coronary artery disease (CAD) by integrating several quantitative perfusion and functional variables for noncorrected (NC) data by Support Vector Machine (SVM) algorithm, a computer method for machine learning.

METHODS

Rest-stress gated (99m)Tc MPS NC studies (n = 957) from 623 consecutive patients with correlating invasive coronary angiography and 334 with a low likelihood of CAD (<5%) were assessed. Stenosis ≥50% in left main or ≥70% in all other vessels was considered abnormal. Total perfusion deficit (TPD) was computed automatically. In addition, ischemic changes (ISCHs) and ejection fraction changes (EFCs) between stress and rest were derived by quantitative software. The SVM was trained using a group of 125 patients (25 with low-likelihood, 25 with 0-vessel, 25 with 1-vessel, 25 with 2-vessel, and 25 with 3-vessel CAD) with the above quantitative variables and second-order polynomial fitting. The remaining patients (n = 832) were categorized using probability estimates, with CAD defined as a probability estimate ≥ 0.50. The diagnostic accuracy of SVM was also compared with visual segmental scoring by 2 experienced readers.

RESULTS

The sensitivity of SVM (84%) was significantly better than ISCH (75%, P < 0.05) and EFC (31%, P < 0.05). The specificity of SVM (88%) was significantly better than TPD (78%, P < 0.05) and EFC (77%, P < 0.05). The diagnostic accuracy of SVM (86%) was significantly better than TPD (81%), ISCH (81%), or EFC (46%) (P < 0.05 for all). The receiver-operating-characteristic (ROC) area under the curve for SVM (0.92) was significantly better than TPD (0.90), ISCH (0.87), and EFC (0.64) (P < 0.001 for all). The diagnostic accuracy of SVM was comparable to the overall accuracy of both visual readers (86% vs. 84%, P = NS). The ROC area under the curve for SVM (0.92) was significantly better than that of both visual readers (0.87 and 0.88, P < 0.03).

CONCLUSION

Computational integration of quantitative perfusion and functional variables using the SVM approach significantly improves the diagnostic accuracy of MPS and can significantly outperform visual assessment based on ROC analysis.

Determination of global minima of some common validation functions in support vector machine

Tuning of the regularization parameter C is a well-known process in the implementation of a support vector machine (SVM) classifier. Such a tuning process uses an appropriate validation function whose value, evaluated over a validation set, has to be optimized for the determination of the optimal C. Unfortunately, most common validation functions are not smooth functions of C. This brief presents a method for obtaining the global optimal solution of these non-smooth validation functions. The method is guaranteed to find the global optimum and relies on the regularization solution path of SVM over a range of C values. When the solution path is available, the computation needed is minimal.

Maximum likelihood model selection for 1-norm soft margin SVMs with multiple parameters

Adapting the hyperparameters of support vector machines (SVMs) is a challenging model selection problem, especially when flexible kernels are to be adapted and data are scarce. We present a coherent framework for regularized model selection of 1-norm soft margin SVMs for binary classification. It is proposed to use gradient-ascent on a likelihood function of the hyperparameters. The likelihood function is based on logistic regression for robustly estimating the class conditional probabilities and can be computed efficiently. Overfitting is an important issue in SVM model selection and can be addressed in our framework by incorporating suitable prior distributions over the hyperparameters. We show empirically that gradient-based optimization of the likelihood function is able to adapt multiple kernel parameters and leads to better models than four concurrent state-of-the-art methods.

An evolutionary approach for gene selection and classification of microarray data based on SVM error-bound theories

Microarrays have thousands to tens-of-thousands of gene features, but only a few hundred patient samples are available. The fundamental problem in microarray data analysis is identifying genes whose disruption causes congenital or acquired disease in humans. In this paper, we propose a new evolutionary method that can efficiently select a subset of potentially informative genes for support vector machine (SVM) classifiers. The proposed evolutionary method uses SVM with a given subset of gene features to evaluate the fitness function, and new subsets of features are selected based on the estimates of generalization error of SVMs and frequency of occurrence of the features in the evolutionary approach. Thus, in theory, selected genes reflect to some extent the generalization performance of SVM classifiers. We compare our proposed method with several existing methods and find that the proposed method can obtain better classification accuracy with a smaller number of selected genes than the existing methods.

Fast and efficient strategies for model selection of Gaussian support vector machine

Two strategies for selecting the kernel parameter (sigma) and the penalty coefficient (C) of Gaussian support vector machines (SVMs) are suggested in this paper. Based on viewing the model parameter selection problem as a recognition problem in visual systems, a direct parameter setting formula for the kernel parameter is derived through finding a visual scale at which the global and local structures of the given data set can be preserved in the feature space, and the difference between the two structures can be maximized. In addition, we propose a heuristic algorithm for the selection of the penalty coefficient through identifying the classification extent of a training datum in the implementation process of the sequential minimal optimization (SMO) procedure, which is a well-developed and commonly used algorithm in SVM training. We then evaluate the suggested strategies with a series of experiments on 13 benchmark problems and three real-world data sets, as compared with the traditional 5-cross validation (5-CV) method and the recently developed radius-margin bound (RM) method. The evaluation shows that in terms of efficiency and generalization capabilities, the new strategies outperform the current methods, and the performance is uniform and stable.

Evolutionary optimization of sequence kernels for detection of bacterial gene starts

Oligo kernels for biological sequence classification have a high discriminative power. A new parameterization for the K-mer oligo kernel is presented, where all oligomers of length K are weighted individually. The task specific choice of these parameters increases the classification performance and reveals information about discriminative features. For adapting the multiple kernel parameters based on cross-validation the covariance matrix adaptation evolution strategy is proposed. It is applied to optimize the trimer oligo kernels for the detection of bacterial gene starts. The resulting kernels lead to higher classification rates, and the adapted parameters reveal the importance of particular triplets for classification, for example of those occurring in the Shine-Dalgarno Sequence.

Predicting protein N-glycosylation by combining functional domain and secretion information

Protein N-glycosylation plays an important role in protein function. Yet, at present, few computational methods are available for the prediction of this protein modification. This prompted our development of a support vector machine (SVM)-based method for this task, as well as a partial least squares (PLS) regression based prediction method for comparison. A functional domain feature space was used to create SVM and PLS models, which achieved accuracies of 83.91% and 79.89%, respectively, as evaluated by a leave-one-out cross-validation. Subsequently, SVM and PLS models were developed based on functional domain and protein secretion information, which yielded accuracies of 89.13% and 86%, respectively. This analysis demonstrates that the protein functional domain and secretion information are both efficient predictors of N-glycosylation.

Predicting O-glycosylation sites in mammalian proteins by using SVMs

O-glycosylation is one of the most important, frequent and complex post-translational modifications. This modification can activate and affect protein functions. Here, we present three support vector machines models based on physical properties, 0/1 system, and the system combining the above two features. The prediction accuracies of the three models have reached 0.82, 0.85 and 0.85, respectively. The accuracies of the three SVMs methods were evaluated by 'leave-one-out' cross validation. This approach provides a useful tool to help identify the O-glycosylation sites in mammalian proteins. An online prediction web server is available at http://www.biosino.org/Oglyc.