Model selection for support vector machines via uniform design

Comparison of Selection Criteria for Model Selection of Support Vector Machine on Physiological Data with Inter-Subject Variance

Support vector machines (SVMs) utilize hyper-parameters for classification. Model selection (MS) is an essential step in the construction of the SVM classifier as it involves the identification of the appropriate parameters. Several selection criteria have been proposed for MS, but their usefulness is limited for physiological data exhibiting inter-subject variance (ISV) that makes different characteristics between training and test data. To identify an effective solution for the constraint, this study considered a leave-one-subject-out cross validation-based selection criterion (LSSC) with six well-known selection criteria and compared their effectiveness. Nine classification problems were examined for the comparison, and the MS results of each selection criterion were obtained and analyzed. The results showed that the SVM model selected by the LSSC yielded the highest average classification accuracy among all selection criteria in the nine problems. The average accuracy was 2.96% higher than that obtained with the conventional K-fold cross validation-based selection criterion. In addition, the advantage of the LSSC was more evident for data with larger ISV. Thus, the results of this study can help optimize SVM classifiers for physiological data and are expected to be useful for the analysis of physiological data to develop various medical decision systems.

Parameter Selection for Linear Support Vector Regression

In linear support vector regression (SVR), the regularization and error sensitivity parameters are used to avoid overfitting the training data. A proper selection of parameters is very essential for obtaining a good model, but the search process may be complicated and time-consuming. In an earlier work by Chu et al. (2015), an effective parameter-selection procedure by using warm-start techniques to solve a sequence of optimization problems has been proposed for linear classification. We extend their techniques to linear SVR, but address some new and challenging issues. In particular, linear classification involves only the regularization parameter, but linear SVR has an extra error sensitivity parameter. We investigate the effective range of each parameter and the sequence in checking the two parameters. Based on this work, an effective tool for the selection of parameters for linear SVR has been available for public use.

Rapid, non-invasive screening of keratitis based on Raman spectroscopy combined with multivariate statistical analysis

This study proposes a multivariate statistical analysis method based on Raman spectroscopy and different dimensionality reduction methods combined with the support vector machine (SVM) algorithm for rapid, non-invasive, high-accuracy classification of keratitis screenings. In this experiment, tear samples from 19 subjects with keratitis and 27 healthy subjects were detected, Raman spectra of the two groups of subjects were compared and analysed, and we found that their spectral intensities were different at 1005 cm^-1 and 1155 cm^-1 Principal component analysis (PCA) and partial least squares (PLS) were used for feature extraction, which greatly reduced the dimensionality of the high-dimensional spectral data. Then, the above two feature extraction methods were used as input to an SVM to build the discriminant diagnosis model. The average accuracy obtained from the PCA-SVM and PLS-SVM models was 77.86 % and 100 %, respectively. Our results suggest that tear Raman spectroscopy combined with multivariate statistical analysis has great potential in screening for keratitis. We expect this technology to could lead to the development of a portable, non-invasive and highly accurate keratitis screening device.

Macro-classification of meteorites by portable energy dispersive X-ray fluorescence spectroscopy (pED-XRF), principal component analysis (PCA) and machine learning algorithms

The research on meteorites from hot and cold deserts is gaining advantages from the recent improvements of portable technologies such as X-ray fluorescence spectroscopy (XRF). The main advantages of portable instruments include the fast recognition of meteorites through their classification in macro-groups and discrimination from materials such as industrial slags, desert varnish covered rocks and iron oxides, named "meteor-wrongs". In this study, 18 meteorite samples of different nature and origin were discriminated and preliminarily classified into characteristic macro-groups: iron meteorites, stony meteorites and meteor-wrongs, combining a portable energy dispersive XRF instrument (pED-XRF), principal component analysis (PCA) and some machine learning algorithms applied to the XRF spectra. The results showed that 100% accuracy in sample classification was obtained by applying the cubic support vector machine (CSVM), fine kernel nearest neighbor (FKNN), subspace discriminant-ensemble classifiers (SD-EC) and subspace discriminant KNN-EC (SKNN-EC) algorithms on standardized spectra.

A guide to automated apoptosis detection: How to make sense of imaging flow cytometry data

Imaging flow cytometry is a powerful experimental technique combining the strength of microscopy and flow cytometry to enable high-throughput characterization of cell populations on a detailed microscopic scale. This approach has an increasing importance for distinguishing between different cellular phenotypes such as proliferation, cell division and cell death. In the course of undergoing these different pathways, each cell is characterized by a high amount of properties. This makes it hard to filter the most relevant information for cell state discrimination. The traditional methods for cell state discrimination rely on dye based two-dimensional gating strategies ignoring information that is hidden in the high-dimensional property space. In order to make use of the information ignored by the traditional methods, we present a simple and efficient approach to distinguish biological states within a cell population based on machine learning techniques. We demonstrate the advantages and drawbacks of filter techniques combined with different classification schemes. These techniques are illustrated with two case studies of apoptosis detection in HeLa cells. Thereby we highlight (i) the aptitude of imaging flow cytometry regarding automated, label-free cell state discrimination and (ii) pitfalls that are frequently encountered. Additionally a MATLAB script is provided, which gives further insight regarding the computational work presented in this study.

Parzen neural networks: Fundamentals, properties, and an application to forensic anthropology

A novel, unsupervised nonparametric model of multivariate probability density functions (pdf) is introduced, namely the Parzen neural network (PNN). The PNN is intended to overcome the major limitations of traditional (either statistical or neural) pdf estimation techniques. Besides being profitably simple, the PNN turns out to have nice properties in terms of unbiased modeling capability, asymptotic convergence, and efficiency at test time. Several matters pertaining the practical application of the PNN are faced in the paper, too. Experiments are reported, involving (i) synthetic datasets, and (ii) a challenging sex determination task from 1400 scout-view CT-scan images of human crania. Incidentally, the empirical evidence entails also some conclusions of high anthropological relevance.

Automated detection of pathologic white matter alterations in Alzheimer's disease using combined diffusivity and kurtosis method

Diffusion tensor imaging (DTI) and diffusion kurtosis imaging (DKI) are important diffusion MRI techniques for detecting microstructure abnormities in diseases such as Alzheimer's. The advantages of DKI over DTI have been reported generally; however, the indistinct relationship between diffusivity and kurtosis has not been clearly revealed in clinical settings. In this study, we hypothesize that the combination of diffusivity and kurtosis in DKI improves the capacity of DKI to detect Alzheimer's disease compared with diffusivity or kurtosis alone. Specifically, a support vector machine-based approach was applied to combine diffusivity and kurtosis and to compare different indices datasets. Strict assessments were conducted to ensure the reliability of all classifiers. Then, data from the optimized classifiers were used to detect abnormalities. With the combination, high accuracy performances of 96.23% were obtained in 53 subjects, including 27 Alzheimer's patients. More highly scored abnormal regions were selected by the combination than alone. The results revealed that more precise diffusivity and complementary kurtosis mainly contributed to the high performance of the combination in DKI. This study provides further understanding of DKI and the relationship between diffusivity and kurtosis in pathologic white matter alterations in Alzheimer's disease.

Multilevel Weighted Support Vector Machine for Classification on Healthcare Data with Missing Values

This work is motivated by the needs of predictive analytics on healthcare data as represented by Electronic Medical Records. Such data is invariably problematic: noisy, with missing entries, with imbalance in classes of interests, leading to serious bias in predictive modeling. Since standard data mining methods often produce poor performance measures, we argue for development of specialized techniques of data-preprocessing and classification. In this paper, we propose a new method to simultaneously classify large datasets and reduce the effects of missing values. It is based on a multilevel framework of the cost-sensitive SVM and the expected maximization imputation method for missing values, which relies on iterated regression analyses. We compare classification results of multilevel SVM-based algorithms on public benchmark datasets with imbalanced classes and missing values as well as real data in health applications, and show that our multilevel SVM-based method produces fast, and more accurate and robust classification results.

Performance evaluation of the machine learning algorithms used in inference mechanism of a medical decision support system

The importance of the decision support systems is increasingly supporting the decision making process in cases of uncertainty and the lack of information and they are widely used in various fields like engineering, finance, medicine, and so forth, Medical decision support systems help the healthcare personnel to select optimal method during the treatment of the patients. Decision support systems are intelligent software systems that support decision makers on their decisions. The design of decision support systems consists of four main subjects called inference mechanism, knowledge-base, explanation module, and active memory. Inference mechanism constitutes the basis of decision support systems. There are various methods that can be used in these mechanisms approaches. Some of these methods are decision trees, artificial neural networks, statistical methods, rule-based methods, and so forth. In decision support systems, those methods can be used separately or a hybrid system, and also combination of those methods. In this study, synthetic data with 10, 100, 1000, and 2000 records have been produced to reflect the probabilities on the ALARM network. The accuracy of 11 machine learning methods for the inference mechanism of medical decision support system is compared on various data sets.

Data mining in psychiatric research

Mathematical sciences and computational methods have found new applications in fields like medicine over the last few decades. Modern data acquisition and data analysis protocols have been of great assistance to medical researchers and clinical scientists. Especially in psychiatry, technology and science have made new computational methods available to assist the development of predictive modeling and to identify diseases more accurately. Data mining (or knowledge discovery) aims to extract information from large datasets and solve challenging tasks, like patient assessment, early mental disease diagnosis, and drug efficacy assessment. Accurate and fast data analysis methods are very important, especially when dealing with severe psychiatric diseases like schizophrenia. In this paper, we focus on computational methods related to data analysis and more specifically to data mining. Then, we discuss some related research in the field of psychiatry.

A new regularized least squares support vector regression for gene selection

Background

Selection of influential genes with microarray data often faces the difficulties of a large number of genes and a relatively small group of subjects. In addition to the curse of dimensionality, many gene selection methods weight the contribution from each individual subject equally. This equal-contribution assumption cannot account for the possible dependence among subjects who associate similarly to the disease, and may restrict the selection of influential genes.

Results

A novel approach to gene selection is proposed based on kernel similarities and kernel weights. We do not assume uniformity for subject contribution. Weights are calculated via regularized least squares support vector regression (RLS-SVR) of class levels on kernel similarities and are used to weight subject contribution. The cumulative sum of weighted expression levels are next ranked to select responsible genes. These procedures also work for multiclass classification. We demonstrate this algorithm on acute leukemia, colon cancer, small, round blue cell tumors of childhood, breast cancer, and lung cancer studies, using kernel Fisher discriminant analysis and support vector machines as classifiers. Other procedures are compared as well.

Conclusion

This approach is easy to implement and fast in computation for both binary and multiclass problems. The gene set provided by the RLS-SVR weight-based approach contains a less number of genes, and achieves a higher accuracy than other procedures.