Two-Stage Regularized Linear Discriminant Analysis for 2-D Data

Ratio Sum Versus Sum Ratio for Linear Discriminant Analysis

Dimension reduction is a critical technology for high-dimensional data processing, where Linear Discriminant Analysis (LDA) and its variants are effective supervised methods. However, LDA prefers to feature with smaller variance, which causes feature with weak discriminative ability retained. In this paper, we propose a novel Ratio Sum for Linear Discriminant Analysis (RSLDA), which aims at maximizing discriminative ability of each feature in subspace. To be specific, it maximizes the sum of ratio of the between-class distance to the within-class distance in each dimension of subspace. Since the original RSLDA problem is difficult to obtain the closed solution, an equivalent problem is developed which can be solved by an alternative optimization algorithm. For solving the equivalent problem, it is transformed into two sub-problems, one of which can be solved directly, the other is changed into a convex optimization problem, where singular value decomposition is employed instead of matrix inversion. Consequently, performance of algorithm cannot be affected by the non-singularity of covariance matrix. Furthermore, Kernel RSLDA (KRSLDA) is presented to improve the robustness of RSLDA. Additionally, time complexity of RSLDA and KRSLDA are analyzed. Extensive experiments show that RSLDA and KRSLDA outperforms other comparison methods on toy datasets and multiple public datasets.

Robust and Sparse Linear Discriminant Analysis via an Alternating Direction Method of Multipliers

In this paper, we propose a robust linear discriminant analysis (RLDA) through Bhattacharyya error bound optimization. RLDA considers a nonconvex problem with the L₁ -norm operation that makes it less sensitive to outliers and noise than the L₂ -norm linear discriminant analysis (LDA). In addition, we extend our RLDA to a sparse model (RSLDA). Both RLDA and RSLDA can extract unbounded numbers of features and avoid the small sample size (SSS) problem, and an alternating direction method of multipliers (ADMM) is used to cope with the nonconvexity in the proposed formulations. Compared with the traditional LDA, our RLDA and RSLDA are more robust to outliers and noise, and RSLDA can obtain sparse discriminant directions. These findings are supported by experiments on artificial data sets as well as human face databases.

DBSDA : Lowering the Bound of Misclassification Rate for Sparse Linear Discriminant Analysis via Model Debiasing

Linear discriminant analysis (LDA) is a well-known technique for linear classification, feature extraction, and dimension reduction. To improve the accuracy of LDA under the high dimension low sample size (HDLSS) settings, shrunken estimators, such as Graphical Lasso, can be used to strike a balance between biases and variances. Although the estimator with induced sparsity obtains a faster convergence rate, however, the introduced bias may also degrade the performance. In this paper, we theoretically analyze how the sparsity and the convergence rate of the precision matrix (also known as inverse covariance matrix) estimator would affect the classification accuracy by proposing an analytic model on the upper bound of an LDA misclassification rate. Guided by the model, we propose a novel classifier, DBSDA , which improves classification accuracy through debiasing. Theoretical analysis shows that DBSDA possesses a reduced upper bound of misclassification rate and better asymptotic properties than sparse LDA (SDA). We conduct experiments on both synthetic datasets and real application datasets to confirm the correctness of our theoretical analysis and demonstrate the superiority of DBSDA over LDA, SDA, and other downstream competitors under HDLSS settings.

Computed tomography-based radiomics for prediction of neoadjuvant chemotherapy outcomes in locally advanced gastric cancer: A pilot study

Objective

The standard treatment for patients with locally advanced gastric cancer has relied on perioperative radio-chemotherapy or chemotherapy and surgery. The aim of this study was to investigate the wealth of radiomics for pre-treatment computed tomography (CT) in the prediction of the pathological response of locally advanced gastric cancer with preoperative chemotherapy.

Methods

Thirty consecutive patients with CT-staged II/III gastric cancer receiving neoadjuvant chemotherapy were enrolled in this study between December 2014 and March 2017. All patients underwent upper abdominal CT during the unenhanced, late arterial phase (AP) and portal venous phase (PP) before the administration of neoadjuvant chemotherapy. In total, 19,985 radiomics features were extracted in the AP and PP for each patient. Four methods were adopted during feature selection and eight methods were used in the process of building the classifier model. Thirty-two combinations of feature selection and classification methods were examined. Receiver operating characteristic (ROC) curves were used to evaluate the capability of each combination of feature selection and classification method to predict a non-good response (non-GR) based on tumor regression grade (TRG).

Results

The mean area under the curve (AUC) ranged from 0.194 to 0.621 in the AP, and from 0.455 to 0.722 in the PP, according to different combinations of feature selection and the classification methods. There was only one cross-combination machine-learning method indicating a relatively higher AUC (>0.600) in the AP, while 12 cross-combination machine-learning methods presented relatively higher AUCs (all >0.600) in the PP. The feature selection method adopted by a filter based on linear discriminant analysis + classifier of random forest achieved a significantly prognostic performance in the PP (AUC, 0.722±0.108; accuracy, 0.793; sensitivity, 0.636; specificity, 0.889; Z=2.039; P=0.041).

Conclusions

It is possible to predict non-GR after neoadjuvant chemotherapy in locally advanced gastric cancers based on the radiomics of CT.

A Complete Subspace Analysis of Linear Discriminant Analysis and Its Robust Implementation

Linear discriminant analysis has been widely studied in data mining and pattern recognition. However, when performing the eigen-decomposition on the matrix pair (within-class scatter matrix and between-class scatter matrix) in some cases, one can find that there exist some degenerated eigenvalues, thereby resulting in indistinguishability of information from the eigen-subspace corresponding to some degenerated eigenvalue. In order to address this problem, we revisit linear discriminant analysis in this paper and propose a stable and effective algorithm for linear discriminant analysis in terms of an optimization criterion. By discussing the properties of the optimization criterion, we find that the eigenvectors in some eigen-subspaces may be indistinguishable if the degenerated eigenvalue occurs. Inspired from the idea of the maximum margin criterion (MMC), we embed MMC into the eigen-subspace corresponding to the degenerated eigenvalue to exploit discriminability of the eigenvectors in the eigen-subspace. Since the proposed algorithm can deal with the degenerated case of eigenvalues, it not only handles the small-sample-size problem but also enables us to select projection vectors from the null space of the between-class scatter matrix. Extensive experiments on several face images and microarray data sets are conducted to evaluate the proposed algorithm in terms of the classification performance, and experimental results show that our method has smaller standard deviations than other methods in most cases.