A two-stage linear discriminant analysis via QR-decomposition

A novel technology for rapid identification of hemp fibers by terahertz spectroscopy

A novel method for the rapid identification of hemp fibers is proposed in this paper, utilizing terahertz time-domain spectroscopy (THz-TDS) combined with the LargeVis (LV) dimensionality reduction technique. This approach takes advantage of the strengths of THz-TDS while enhancing classification accuracy through LV. To verify the efficacy of this method, terahertz absorption spectral data from three types of hemp fibers were processed. The THz absorption spectra were initially preprocessed using Hanning filtering. Following this, the filtered data underwent dimensionality reduction through three distinct methods: linear Principal Component Analysis (PCA), nonlinear t-Distributed Stochastic Neighbor Embedding (t-SNE), and the LV method. This sequence of steps resulted in a two-dimensional feature data matrix derived from the THz source spectral data. The resultant feature data matrices were then input into both K-Nearest Neighbors (KNN) and Decision Tree (DT) classifiers for analysis. The classification accuracies of six models were evaluated, revealing that the LV-KNN model achieved a 86.67% accuracy rate for the three hemp fiber types. Impressively, the LV-DT model achieved a perfect 100.00% accuracy rate for the same fibers. The LV-DT model, when integrated with THz spectroscopy technology, offers a quick and precise method for identifying various types of hemp fibers. This development introduces an innovative optical measurement scheme for the characterization of textile materials.

Bayesian Hierarchical Graph Neural Networks With Uncertainty Feedback for Trustworthy Fault Diagnosis of Industrial Processes

Deep learning (DL) methods have been widely applied to intelligent fault diagnosis of industrial processes and achieved state-of-the-art performance. However, fault diagnosis with point estimate may provide untrustworthy decisions. Recently, Bayesian inference shows to be a promising approach to trustworthy fault diagnosis by quantifying the uncertainty of the decisions with a DL model. The uncertainty information is not involved in the training process, which does not help the learning of highly uncertain samples and has little effect on improving the fault diagnosis performance. To address this challenge, we propose a Bayesian hierarchical graph neural network (BHGNN) with an uncertainty feedback mechanism, which formulates a trustworthy fault diagnosis on the Bayesian DL (BDL) framework. Specifically, BHGNN captures the epistemic uncertainty and aleatoric uncertainty via a variational dropout approach and utilizes the uncertainty information of each sample to adjust the strength of the temporal consistency (TC) constraint for robust feature learning. Meanwhile, the BHGNN method models the process data as a hierarchical graph (HG) by leveraging the interaction-aware module and physical topology knowledge of the industrial process, which integrates data with domain knowledge to learn fault representation. Moreover, the experiments on a three-phase flow facility (TFF) and secure water treatment (SWaT) show superior and competitive performance in fault diagnosis and verify the trustworthiness of the proposed method.

Motor Imagery Classification Using Effective Channel Selection of Multichannel EEG

Electroencephalography (EEG) is effectively employed to describe cognitive patterns corresponding to different tasks of motor functions for brain-computer interface (BCI) implementation. Explicit information processing is necessary to reduce the computational complexity of practical BCI systems. This paper presents an entropy-based approach to select effective EEG channels for motor imagery (MI) classification in brain-computer interface (BCI) systems. The method identifies channels with higher entropy scores, which is an indication of greater information content. It discards redundant or noisy channels leading to reduced computational complexity and improved classification accuracy. High entropy means a more disordered pattern, whereas low entropy means a less disordered pattern with less information. The entropy of each channel for individual trials is calculated. The weight of each channel is represented by the mean entropy of the channel over all the trials. A set of channels with higher mean entropy are selected as effective channels for MI classification. A limited number of sub-band signals are created by decomposing the selected channels. To extract the spatial features, the common spatial pattern (CSP) is applied to each sub-band space of EEG signals. The CSP-based features are used to classify the right-hand and right-foot MI tasks using a support vector machine (SVM). The effectiveness of the proposed approach is validated using two publicly available EEG datasets, known as BCI competition III-IV(A) and BCI competition IV-I. The experimental results demonstrate that the proposed approach surpasses cutting-edge techniques.

On the Equivalence of Linear Discriminant Analysis and Least Squares Regression

Studying the relationship between linear discriminant analysis (LDA) and least squares regression (LSR) is of great theoretical and practical significance. It is well-known that the two-class LDA is equivalent to an LSR problem, and directly casting multiclass LDA as an LSR problem, however, becomes more challenging. Recent study reveals that the equivalence between multiclass LDA and LSR can be established based on a special class indicator matrix, but under a mild condition which may not hold under the scenarios with low-dimensional or oversampled data. In this article, we show that the equivalence between multiclass LDA and LSR can be established based on arbitrary linearly independent class indicator vectors and without any condition. In addition, we show that LDA is also equivalent to a constrained LSR based on the data-dependent indicator vectors. It can be concluded that under exactly the same mild condition, such two regressions are both equivalent to the null space LDA method. Illuminated by the equivalence of LDA and LSR, we propose a direct LDA classifier to replace the conventional framework of LDA plus extra classifier. Extensive experiments well validate the above theoretic analysis.

Interaction-Aware Graph Neural Networks for Fault Diagnosis of Complex Industrial Processes

Fault diagnosis of complex industrial processes becomes a challenging task due to various fault patterns in sensor signals and complex interactions between different units. However, how to explore the interactions and integrate with sensor signals remains an open question. Considering that the sensor signals and their interactions in an industrial process with the form of nodes and edges can be represented as a graph, this article proposes a novel interaction-aware and data fusion method for fault diagnosis of complex industrial processes, named interaction-aware graph neural networks (IAGNNs). First, to describe the complex interactions in an industrial process, the sensor signals are transformed into a heterogeneous graph with multiple edge types, and the edge weights are learned by the attention mechanism, adaptively. Then, multiple independent graph neural network (GNN) blocks are employed to extract the fault feature for each subgraph with one edge type. Finally, each subgraph feature is concatenated or fused by a weighted summation function to generate the final graph embedding. Therefore, the proposed method can learn multiple interactions between sensor signals and extract the fault feature from each subgraph by message passing operation of GNNs. The final fault feature contains the information from raw data and implicit interactions between sensor signals. The experimental results on the three-phase flow facility and power system (PS) demonstrate the reliable and superior performance of the proposed method for fault diagnosis of complex industrial processes.

Instance Selection-Based Surrogate-Assisted Genetic Programming for Feature Learning in Image Classification

Genetic programming (GP) has been applied to feature learning for image classification and achieved promising results. However, many GP-based feature learning algorithms are computationally expensive due to a large number of expensive fitness evaluations, especially when using a large number of training instances/images. Instance selection aims to select a small subset of training instances, which can reduce the computational cost. Surrogate-assisted evolutionary algorithms often replace expensive fitness evaluations by building surrogate models. This article proposes an instance selection-based surrogate-assisted GP for fast feature learning in image classification. The instance selection method selects multiple small subsets of images from the original training set to form surrogate training sets of different sizes. The proposed approach gradually uses these surrogate training sets to reduce the overall computational cost using a static or dynamic strategy. At each generation, the proposed approach evaluates the entire population on the small surrogate training sets and only evaluates ten current best individuals on the entire training set. The features learned by the proposed approach are fed into linear support vector machines for classification. Extensive experiments show that the proposed approach can not only significantly reduce the computational cost but also improve the generalisation performance over the baseline method, which uses the entire training set for fitness evaluations, on 11 different image datasets. The comparisons with other state-of-the-art GP and non-GP methods further demonstrate the effectiveness of the proposed approach. Further analysis shows that using multiple surrogate training sets in the proposed approach achieves better performance than using a single surrogate training set and using a random instance selection method.

Accurate Classification of Chunmee Tea Grade Using NIR Spectroscopy and Fuzzy Maximum Uncertainty Linear Discriminant Analysis

The grade of tea is closely related to tea quality, so the identification of tea grade is an important task. In order to improve the identification capability of the tea grade system, a fuzzy maximum uncertainty linear discriminant analysis (FMLDA) methodology was proposed based on maximum uncertainty linear discriminant analysis (MLDA). Based on FMLDA, a tea grade recognition system was established for the grade recognition of Chunmee tea. The process of this system is as follows: firstly, the near-infrared (NIR) spectra of Chunmee tea were collected using a Fourier transform NIR spectrometer. Next, the spectra were preprocessed using standard normal variables (SNV). Then, direct linear discriminant analysis (DLDA), maximum uncertainty linear discriminant analysis (MLDA), and FMLDA were used for feature extraction of the spectra, respectively. Finally, the k-nearest neighbor (KNN) classifier was applied to classify the spectra. The k in KNN and the fuzzy coefficient, m, were discussed in the experiment. The experimental results showed that when k = 1 and m = 2.7 or 2.8, the accuracy of the FMLDA could reach 98.15%, which was better than the other two feature extraction methods. Therefore, FMLDA combined with NIR technology is an effective method in the identification of tea grade.

Multi-Classification of Motor Imagery EEG Signals Using Bayesian Optimization-Based Average Ensemble Approach

Motor Imagery (MI) classification using electroencephalography (EEG) has been extensively applied in healthcare scenarios for rehabilitation aims. EEG signal decoding is a difficult process due to its complexity and poor signal-to-noise ratio. Convolutional neural networks (CNN) have demonstrated their ability to extract time–space characteristics from EEG signals for better classification results. However, to discover dynamic correlations in these signals, CNN models must be improved. Hyperparameter choice strongly affects the robustness of CNNs. It is still challenging since the manual tuning performed by domain experts lacks the high performance needed for real-life applications. To overcome these limitations, we presented a fusion of three optimum CNN models using the Average Ensemble strategy, a method that is utilized for the first time for MI movement classification. Moreover, we adopted the Bayesian Optimization (BO) algorithm to reach the optimal hyperparameters’ values. The experimental results demonstrate that without data augmentation, our approach reached 92% accuracy, whereas Linear Discriminate Analysis, Support Vector Machine, Random Forest, Multi-Layer Perceptron, and Gaussian Naive Bayes achieved 68%, 70%, 58%, 64%, and 40% accuracy, respectively. Further, we surpassed state-of-the-art strategies on the BCI competition IV-2a multiclass MI database by a wide margin, proving the benefit of combining the output of CNN models with automated hyperparameter tuning.

Covariance Attention for Semantic Segmentation

The dependency between global and local information can provide important contextual cues for semantic segmentation. Existing attention methods capture this dependency by calculating the pixel wise correlation between the learnt feature maps, which is of high space and time complexity. In this article, a new attention module, covariance attention, is presented, and which is interesting in the following aspects: 1) covariance matrix is used as a new attention module to model the global and local dependency for the feature maps and the local-global dependency is formulated as a simple matrix projection process; 2) since covariance matrix can encode the joint distribution information for the heterogeneous yet complementary statistics, the hand-engineered features are combined with the learnt features effectively using covariance matrix to boost the segmentation performance; 3) a covariance attention mechanism based semantic segmentation framework, CANet, is proposed and very competitive performance has been obtained. Comparisons with the state-of-the-art methods reveal the superiority of the proposed method.

Continuous Monitoring of Mental Load During Virtual Simulator Training for Laparoscopic Surgery Reflects Laparoscopic Dexterity: A Comparative Study Using a Novel Wireless Device

Introduction

Cognitive Load Theory (CLT) relates to the efficiency with which individuals manipulate the limited capacity of working memory load. Repeated training generally results in individual performance increase and cognitive load decrease, as measured by both behavioral and neuroimaging methods. One of the known biomarkers for cognitive load is frontal theta band, measured by an EEG. Simulation-based training is an effective tool for acquiring practical skills, specifically to train new surgeons in a controlled and hazard-free environment. Measuring the cognitive load of young surgeons undergoing such training can help to determine whether they are ready to take part in a real surgery. In this study, we measured the performance of medical students and interns in a surgery simulator, while their brain activity was monitored by a single-channel EEG.

Methods

A total of 38 medical students and interns were divided into three groups and underwent three experiments examining their behavioral performances. The participants were performing a task while being monitored by the Simbionix LAP MENTOR™. Their brain activity was simultaneously measured using a single-channel EEG with novel signal processing (Aurora by Neurosteer^®). Each experiment included three trials of a simulator task performed with laparoscopic hands. The time retention between the tasks was different in each experiment, in order to examine changes in performance and cognitive load biomarkers that occurred during the task or as a result of nighttime sleep consolidation.

Results

The participants’ behavioral performance improved with trial repetition in all three experiments. In Experiments 1 and 2, delta band and the novel VC9 biomarker (previously shown to correlate with cognitive load) exhibited a significant decrease in activity with trial repetition. Additionally, delta, VC9, and, to some extent, theta activity decreased with better individual performance.

Discussion

In correspondence with previous research, EEG markers delta, VC9, and theta (partially) decreased with lower cognitive load and higher performance; the novel biomarker, VC9, showed higher sensitivity to lower cognitive load levels. Together, these measurements may be used for the neuroimaging assessment of cognitive load while performing simulator laparoscopic tasks. This can potentially be expanded to evaluate the efficacy of different medical simulations to provide more efficient training to medical staff and measure cognitive and mental loads in real laparoscopic surgeries.

Noise-insensitive discriminative subspace fuzzy clustering

Discriminative subspace clustering (DSC) can make full use of linear discriminant analysis (LDA) to reduce the dimension of data and achieve effective clustering high-dimension data by clustering low-dimension data in discriminant subspace. However, most existing DSC algorithms do not consider the noise and outliers that may be contained in data sets, and when they are applied to the data sets with noise or outliers, and they often obtain poor performance due to the influence of noise and outliers. In this paper, we address the problem of the sensitivity of DSC to noise and outlier. Replacing the Euclidean distance in the objective function of LDA by an exponential non-Euclidean distance, we first develop a noise-insensitive LDA (NILDA) algorithm. Then, combining the proposed NILDA and a noise-insensitive fuzzy clustering algorithm: AFKM, we propose a noise-insensitive discriminative subspace fuzzy clustering (NIDSFC) algorithm. Experiments on some benchmark data sets show the effectiveness of the proposed NIDSFC algorithm.

Enhancement for P300-speller classification using multi-window discriminative canonical pattern matching

Objective.P300s are one of the most studied event-related potentials (ERPs), which have been widely used for brain-computer interfaces (BCIs). Thus, fast and accurate recognition of P300s is an important issue for BCI study. Recently, there emerges a lot of novel classification algorithms for P300-speller. Among them, discriminative canonical pattern matching (DCPM) has been proven to work effectively, in which discriminative spatial pattern (DSP) filter can significantly enhance the spatial features of P300s. However, the pattern of ERPs in space varies with time, which was not taken into consideration in the traditional DCPM algorithm.Approach.In this study, we developed an advanced version of DCPM, i.e. multi-window DCPM, which contained a series of time-dependent DSP filters to fine-tune the extraction of spatial ERP features. To verify its effectiveness, 25 subjects were recruited and they were asked to conduct the typical P300-speller experiment.Main results.As a result, multi-window DCPM achieved the character recognition accuracy of 91.84% with only five training characters, which was significantly better than the traditional DCPM algorithm. Furthermore, it was also compared with eight other popular methods, including SWLDA, SKLDA, STDA, BLDA, xDAWN, HDCA, sHDCA and EEGNet. The results showed multi-window DCPM preformed the best, especially using a small calibration dataset. The proposed algorithm was applied to the BCI Controlled Robot Contest of P300 paradigm in 2019 World Robot Conference, and won the first place.Significance.These results demonstrate that multi-window DCPM is a promising method for improving the performance and enhancing the practicability of P300-speller.

Deep learning encodes robust discriminative neuroimaging representations to outperform standard machine learning

Recent critical commentaries unfavorably compare deep learning (DL) with standard machine learning (SML) approaches for brain imaging data analysis. However, their conclusions are often based on pre-engineered features depriving DL of its main advantage — representation learning. We conduct a large-scale systematic comparison profiled in multiple classification and regression tasks on structural MRI images and show the importance of representation learning for DL. Results show that if trained following prevalent DL practices, DL methods have the potential to scale particularly well and substantially improve compared to SML methods, while also presenting a lower asymptotic complexity in relative computational time, despite being more complex. We also demonstrate that DL embeddings span comprehensible task-specific projection spectra and that DL consistently localizes task-discriminative brain biomarkers. Our findings highlight the presence of nonlinearities in neuroimaging data that DL can exploit to generate superior task-discriminative representations for characterizing the human brain.

Recent critical commentaries unfavorably compare deep learning (DL) with standard machine learning (SML) for brain imaging data analysis. Here, the authors show that if trained following prevalent DL practices, DL methods substantially improve compared to SML methods by encoding robust discriminative brain representations.

Evaluating the severity of aortic coarctation in infants using anatomic features measured on CTA

OBJECTIVES

A machine learning model was developed to evaluate the severity of aortic coarctation (CoA) in infants based on anatomical features measured on CTA.

METHODS

In total, 239 infant patients undergoing both thorax CTA and echocardiography were retrospectively reviewed. The patients were assigned to either mild or severe CoA group based on their pressure gradient on echocardiography. They were further divided into patent ductus arteriosus (PDA) and non-PDA groups. The anatomical features were measured on double-oblique multiplanar reconstructed CTA images. Then, the optimal features were identified by using the Boruta algorithm. Subsequently, the coarctation severity was classified using linear discriminant analysis (LDA). We further investigated the relationship between the anatomical features and re-coarctation using Cox regression.

RESULTS

Four anatomical features showed significant differences between the mild and severe CoA groups, including the smallest aortic cross-sectional area indexed to body surface area (p < 0.001), the narrowest aortic diameter (CoA diameter) indexed to height (p < 0.001), the diameter of the descending aorta at the diaphragmatic level (p < 0.001) and weight (p = 0.005). With these features, accuracy of 88.6% and 90.2%, sensitivity of 65.0% and 72.1%, and specificity of 92.9% and 100% were obtained for classifying the CoA severity in the non-PDA and PDA groups, respectively. Moreover, CoA diameter indexed to weight was associated with the risk of re-coarctation.

CONCLUSIONS

CoA severity can be evaluated by using LDA with anatomical features. When quantifying the severity of CoA and risk of re-coarctation, both anatomical alternations at the CoA site and the growth of the patients need to be considered.

KEY POINTS

• CTA is routinely ordered for infants with coarctation of the aorta; however, whether anatomical variations observed with CTA could be used to assess the severity of CoA remains unknown. • Using the diameter and area of the coarctation site adjusted to body growth as features, the LDA model achieved an accuracy of 88.6% and 90.2% in differentiating between the mild and severe CoA patients in the non-PDA group and PDA group, respectively. • The narrowest aortic diameter (CoA diameter) indexed to weight has a hazard ratio of 10.29 for re-coarctation.

Automatic Tuning of the RBF Kernel Parameter for Batch-Mode Active Learning Algorithms: A Scalable Framework

Batch-mode active learning algorithms can select a batch of valuable unlabeled samples to manually annotate for reducing the total cost of labeling every unlabeled sample. To facilitate selection of valuable unlabeled samples, many batch-mode active learning algorithms map samples to the reproducing kernel Hilbert space induced by a radial-basis function (RBF) kernel. Setting a proper value to the parameter for the RBF kernel is crucial for such batch-mode active learning algorithms. In this paper, for automatic tuning of the kernel parameter, a hypothesis-margin-based criterion function is proposed. Three frameworks are also developed to incorporate the function of automatic tuning of the kernel parameter with existing batch-model active learning algorithms. In the proposed frameworks, the kernel parameter can be tuned in a single stage or in multiple stages. Tuning the kernel parameter in a single stage aims for the kernel parameter to be suitable for selecting the specified number of unlabeled samples. When the kernel parameter is tuned in multiple stages, the incorporated active learning algorithm can be enforced to make coarse-to-fine evaluations of the importance of unlabeled samples. The proposed framework can also improve the scalability of existing batch-mode active learning algorithms satisfying a decomposition property. Experimental results on data sets comprising hundreds to hundreds of thousands of samples have shown the feasibility of the proposed framework.

Cytosolic 10-formyltetrahydrofolate dehydrogenase regulates glycine metabolism in mouse liver

ALDH1L1 (10-formyltetrahydrofolate dehydrogenase), an enzyme of folate metabolism highly expressed in liver, metabolizes 10-formyltetrahydrofolate to produce tetrahydrofolate (THF). This reaction might have a regulatory function towards reduced folate pools, de novo purine biosynthesis, and the flux of folate-bound methyl groups. To understand the role of the enzyme in cellular metabolism, Aldh1l1^−/− mice were generated using an ES cell clone (C57BL/6N background) from KOMP repository. Though Aldh1l1^−/− mice were viable and did not have an apparent phenotype, metabolomic analysis indicated that they had metabolic signs of folate deficiency. Specifically, the intermediate of the histidine degradation pathway and a marker of folate deficiency, formiminoglutamate, was increased more than 15-fold in livers of Aldh1l1^−/− mice. At the same time, blood folate levels were not changed and the total folate pool in the liver was decreased by only 20%. A two-fold decrease in glycine and a strong drop in glycine conjugates, a likely result of glycine shortage, were also observed in Aldh1l1^−/− mice. Our study indicates that in the absence of ALDH1L1 enzyme, 10-formyl-THF cannot be efficiently metabolized in the liver. This leads to the decrease in THF causing reduced generation of glycine from serine and impaired histidine degradation, two pathways strictly dependent on THF.

Evaluating green tea quality based on multisensor data fusion combining hyperspectral imaging and olfactory visualization systems

BACKGROUND

The instrumental evaluation of tea quality using digital sensors instead of human panel tests has attracted much attention globally. However, individual sensors do not meet the requirements of discriminant accuracy as a result of incomprehensive sensor information. Considering the major factors in the sensory evaluation of tea, the study integrated multisensor information, including spectral, image and olfaction feature information.

RESULTS

To investigate spectral and image information obtained from hyperspectral spectrometers of different bands, principal components analysis was used for dimension reduction and different types of supervised learning algorithms (linear discriminant analysis, K-nearest neighbour and support vector machine) were selected for comparison. Spectral feature information in the near infrared region and image feature information in the visible-near infrared/near infrared region achieved greater accuracy for classification. The results indicated that a support vector machine outperformed other methods with respect to multisensor data fusion, which improved the accuracy of evaluating green tea quality compared to using individual sensor data. The overall accuracy of the calibration set increased from 75% using optimal single sensor information to 92% using multisensor information, and the overall accuracy of the prediction set increased from 78% to 92%.

CONCLUSION

Overall, it can be concluded that multisensory data accurately identify six grades of tea. © 2018 Society of Chemical Industry.

A chemometric approach for the prediction of the aging levels of automatic transmission fluids by mid-infrared spectroscopy

Automatic transmission fluids (ATF) are highly complex multi-component systems with a variety of different additive packages which suffer from manifold aging processes due to interfering factors. This work describes the development of a straightforward approach to model the aging effects by means of Fourier Transform Infrared (FTIR) spectroscopy combined with multivariate data analysis. Therefore, ATF samples were artificially aged under defined conditions by considering effects of product type, temperature, storage time and exposure to metallic materials, yielding 144 samples. For multivariate data analysis, three different approaches have been applied and compared: supervised Fisher's Linear Discriminant Analysis of principal components (PCFDA), regularized FDA (RFDA) of variables, and unsupervised PCA after orthogonalization using Error Removal by Orthogonal Subtraction (EROS + PCA). All methods worked well in reducing unwanted effects and transforming the relevant information to the first components. Combined with k-Nearest-Neighbor (kNN) prediction, RFDA leads to the best model, improving the accuracy ratios by 13%, 41%, and 12% in comparison with direct kNN classification for the target classes storage temperature, additional material and aging level, respectively. These results suggest that RFDA is highly suitable for the reduction of unwanted effects in a dataset with manifold perturbation influences. The model also predicted a correct aging level ranking when applied to unknown field samples.

Classification of micro-calcification in mammograms using scalable linear Fisher discriminant analysis

Breast cancer is one of the major causes of death in women. Computer Aided Diagnosis (CAD) systems are being developed to assist radiologists in early diagnosis. Micro-calcifications can be an early symptom of breast cancer. Besides detection, classification of micro-calcification as benign or malignant is essential in a complete CAD system. We have developed a novel method for the classification of benign and malignant micro-calcification using an improved Fisher Linear Discriminant Analysis (LDA) approach for the linear transformation of segmented micro-calcification data in combination with a Support Vector Machine (SVM) variant to classify between the two classes. The results indicate an average accuracy equal to 96% which is comparable to state-of-the art methods in the literature.

Graphical Abstract

Manifold Partition Discriminant Analysis

We propose a novel algorithm for supervised dimensionality reduction named manifold partition discriminant analysis (MPDA). It aims to find a linear embedding space where the within-class similarity is achieved along the direction that is consistent with the local variation of the data manifold, while nearby data belonging to different classes are well separated. By partitioning the data manifold into a number of linear subspaces and utilizing the first-order Taylor expansion, MPDA explicitly parameterizes the connections of tangent spaces and represents the data manifold in a piecewise manner. While graph Laplacian methods capture only the pairwise interaction between data points, our method captures both pairwise and higher order interactions (using regional consistency) between data points. This manifold representation can help to improve the measure of within-class similarity, which further leads to improved performance of dimensionality reduction. Experimental results on multiple real-world data sets demonstrate the effectiveness of the proposed method.

Supervised Filter Learning for Representation Based Face Recognition

Representation based classification methods, such as Sparse Representation Classification (SRC) and Linear Regression Classification (LRC) have been developed for face recognition problem successfully. However, most of these methods use the original face images without any preprocessing for recognition. Thus, their performances may be affected by some problematic factors (such as illumination and expression variances) in the face images. In order to overcome this limitation, a novel supervised filter learning algorithm is proposed for representation based face recognition in this paper. The underlying idea of our algorithm is to learn a filter so that the within-class representation residuals of the faces' Local Binary Pattern (LBP) features are minimized and the between-class representation residuals of the faces' LBP features are maximized. Therefore, the LBP features of filtered face images are more discriminative for representation based classifiers. Furthermore, we also extend our algorithm for heterogeneous face recognition problem. Extensive experiments are carried out on five databases and the experimental results verify the efficacy of the proposed algorithm.

A Complete Subspace Analysis of Linear Discriminant Analysis and Its Robust Implementation

Linear discriminant analysis has been widely studied in data mining and pattern recognition. However, when performing the eigen-decomposition on the matrix pair (within-class scatter matrix and between-class scatter matrix) in some cases, one can find that there exist some degenerated eigenvalues, thereby resulting in indistinguishability of information from the eigen-subspace corresponding to some degenerated eigenvalue. In order to address this problem, we revisit linear discriminant analysis in this paper and propose a stable and effective algorithm for linear discriminant analysis in terms of an optimization criterion. By discussing the properties of the optimization criterion, we find that the eigenvectors in some eigen-subspaces may be indistinguishable if the degenerated eigenvalue occurs. Inspired from the idea of the maximum margin criterion (MMC), we embed MMC into the eigen-subspace corresponding to the degenerated eigenvalue to exploit discriminability of the eigenvectors in the eigen-subspace. Since the proposed algorithm can deal with the degenerated case of eigenvalues, it not only handles the small-sample-size problem but also enables us to select projection vectors from the null space of the between-class scatter matrix. Extensive experiments on several face images and microarray data sets are conducted to evaluate the proposed algorithm in terms of the classification performance, and experimental results show that our method has smaller standard deviations than other methods in most cases.

Dimensionality Reduction by Integrating Sparse Representation and Fisher Criterion and its Applications

Sparse representation shows impressive results for image classification, however, it cannot well characterize the discriminant structure of data, which is important for classification. This paper aims to seek a projection matrix such that the low-dimensional representations well characterize the discriminant structure embedded in high-dimensional data and simultaneously well fit sparse representation-based classifier (SRC). To be specific, Fisher discriminant criterion (FDC) is used to extract the discriminant structure, and sparse representation is simultaneously considered to guarantee that the projected data well satisfy the SRC. Thus, our method, called SRC-FDC, characterizes both the spatial Euclidean distribution and local reconstruction relationship, which enable SRC to achieve better performance. Extensive experiments are done on the AR, CMU-PIE, Extended Yale B face image databases, the USPS digit database, and COIL20 database, and results illustrate that the proposed method is more efficient than other feature extraction methods based on SRC.

Asymptotic Generalization Bound of Fisher's Linear Discriminant Analysis

Fisher's linear discriminant analysis (FLDA) is an important dimension reduction method in statistical pattern recognition. It has been shown that FLDA is asymptotically Bayes optimal under the homoscedastic Gaussian assumption. However, this classical result has the following two major limitations: 1) it holds only for a fixed dimensionality D, and thus does not apply when D and the training sample size N are proportionally large; 2) it does not provide a quantitative description on how the generalization ability of FLDA is affected by D and N. In this paper, we present an asymptotic generalization analysis of FLDA based on random matrix theory, in a setting where both D and N increase and D/N → γ ∈ [0,1). The obtained lower bound of the generalization discrimination power overcomes both limitations of the classical result, i.e., it is applicable when D and N are proportionally large and provides a quantitative description of the generalization ability of FLDA in terms of the ratio γ = D/N and the population discrimination power. Besides, the discrimination power bound also leads to an upper bound on the generalization error of binary-classification with FLDA.