Tree-structured feature extraction using mutual information

Discriminative Label Relaxed Regression with Adaptive Graph Learning

The traditional label relaxation regression (LRR) algorithm directly fits the original data without considering the local structure information of the data. While the label relaxation regression algorithm of graph regularization takes into account the local geometric information, the performance of the algorithm depends largely on the construction of graph. However, the traditional graph structures have two defects. First of all, it is largely influenced by the parameter values. Second, it relies on the original data when constructing the weight matrix, which usually contains a lot of noise. This makes the constructed graph to be often not optimal, which affects the subsequent work. Therefore, a discriminative label relaxation regression algorithm based on adaptive graph (DLRR_AG) is proposed for feature extraction. DLRR_AG combines manifold learning with label relaxation regression by constructing adaptive weight graph, which can well overcome the problem of label overfitting. Based on a large number of experiments, it can be proved that the proposed method is effective and feasible.

Molecular Classification of Thyroid Nodules with Indeterminate Cytology: Development and Validation of a Highly Sensitive and Specific New miRNA-Based Classifier Test Using Fine-Needle Aspiration Smear Slides

Background: Thyroid nodules can be identified in up to 68% of the population. Fine-needle aspiration (FNA) cytopathology classifies 20%-30% of nodules as indeterminate, and these are often referred for surgery due to the risk of malignancy. However, histological postsurgical reports indicate that up to 84% of cases are benign, highlighting a high rate of unnecessary surgeries. We sought to develop and validate a microRNA (miRNA)-based thyroid molecular classifier for precision endocrinology (mir-THYpe) with both high sensitivity and high specificity, to be performed on the FNA cytology smear slide with no additional FNA. Methods: The expression of 96 miRNA candidates from 39 benign/39 malignant thyroid samples, (indeterminate on FNA) was analyzed to develop and train the mir-THYpe algorithm. For validation, an independent set of 58 benign/37 malignant FNA smear slides (also classified as indeterminate) was used. Results: In the training set, with a 10-fold cross-validation using only 11 miRNAs, the mir-THYpe test reached 89.7% sensitivity, 92.3% specificity, 90.0% negative predictive value and 92.1% positive predictive value. In the FNA smear slide validation set, the mir-THYpe test reached 94.6% sensitivity, 81.0% specificity, 95.9% negative predictive value, and 76.1% positive predictive value. Bayes' theorem shows that the mir-THYpe test performs satisfactorily in a wide range of cancer prevalences. Conclusions: The presented data and comparison with other commercially available tests suggest that the mir-THYpe test can be considered for use in clinical practice to support a more informed clinical decision for patients with indeterminate thyroid nodules and potentially reduce the rates of unnecessary thyroid surgeries.

Logistic Localized Modeling of the Sample Space for Feature Selection and Classification

Conventional feature selection algorithms assign a single common feature set to all regions of the sample space. In contrast, this paper proposes a novel algorithm for localized feature selection for which each region of the sample space is characterized by its individual distinct feature subset that may vary in size and membership. This approach can therefore select an optimal feature subset that adapts to local variations of the sample space, and hence offer the potential for improved performance. Feature subsets are computed by choosing an optimal coordinate space so that, within a localized region, within-class distances and between-class distances are, respectively, minimized and maximized. Distances are measured using a logistic function metric within the corresponding region. This enables the optimization process to focus on a localized region within the sample space. A local classification approach is utilized for measuring the similarity of a new input data point to each class. The proposed logistic localized feature selection (lLFS) algorithm is invariant to the underlying probability distribution of the data; hence, it is appropriate when the data are distributed on a nonlinear or disjoint manifold. lLFS is efficiently formulated as a joint convex/increasing quasi-convex optimization problem with a unique global optimum point. The method is most applicable when the number of available training samples is small. The performance of the proposed localized method is successfully demonstrated on a large variety of data sets. We demonstrate that the number of features selected by the lLFS method saturates at the number of available discriminative features. In addition, we have shown that the Vapnik-Chervonenkis dimension of the localized classifier is finite. Both these factors suggest that the lLFS method is insensitive to the overfitting issue, relative to other methods.

Research on Digital Forensics Based on Uyghur Web Text Classification

This paper mainly discusses the use of mutual information (MI) and Support Vector Machines (SVMs) for Uyghur Web text classification and digital forensics process of web text categorization: automatic classification and identification, conversion and pretreatment of plain text based on encoding features of various existing Uyghur Web documents etc., introduces the pre-paratory work for Uyghur Web text encoding. Focusing on the non-Uyghur characters and stop words in the web texts filtering, we put forward a Multi-feature Space Normalized Mutual Information (M-FNMI) algorithm and replace MI between single feature and category with mutual information (MI) between input feature combination and category so as to extract more accurate feature words; finally, we classify features with support vector machine (SVM) algorithm. The experimental result shows that this scheme has a high precision of classification and can provide criterion for digital forensics with specific purpose.

Detection of small bowel tumor in wireless capsule endoscopy images using an adaptive neuro-fuzzy inference system

Automatic diagnosis tool helps physicians to evaluate capsule endoscopic examinations faster and more accurate. The purpose of this study was to evaluate the validity and reliability of an automatic post-processing method for identifying and classifying wireless capsule endoscopic images, and investigate statistical measures to differentiate normal and abnormal images. The proposed technique consists of two main stages, namely, feature extraction and classification. Primarily, 32 features incorporating four statistical measures (contrast, correlation, homogeneity and energy) calculated from co-occurrence metrics were computed. Then, mutual information was used to select features with maximal dependence on the target class and with minimal redundancy between features. Finally, a trained classifier, adaptive neuro-fuzzy interface system was implemented to classify endoscopic images into tumor, healthy and unhealthy classes. Classification accuracy of 94.2% was obtained using the proposed pipeline. Such techniques are valuable for accurate detection characterization and interpretation of endoscopic images.

Compositional generative mapping for tree-structured data--part I: bottom-up probabilistic modeling of trees

We introduce a novel compositional (recursive) probabilistic model for trees that defines an approximated bottom-up generative process from the leaves to the root of a tree. The proposed model defines contextual state transitions from the joint configuration of the children to the parent nodes. We argue that the bottom-up context postulates different probabilistic assumptions with respect to a top-down approach, leading to different representational capabilities. We discuss classes of applications that are best suited to a bottom-up approach. In particular, the bottom-up context is shown to better correlate and model the co-occurrence of substructures among the child subtrees of internal nodes. A mixed memory approximation is introduced to factorize the joint children-to-parent state transition matrix as a mixture of pairwise transitions. The proposed approach is the first practical bottom-up generative model for tree-structured data that maintains the same computational class of its top-down counterpart. Comparative experimental analyses exploiting synthetic and real-world datasets show that the proposed model can deal with deep structures better than a top-down generative model. The model is also shown to better capture structural information from real-world data comprising trees with a large out-degree. The proposed bottom-up model can be used as a fundamental building block for the development of other new powerful models.

Discriminative least squares regression for multiclass classification and feature selection

This paper presents a framework of discriminative least squares regression (LSR) for multiclass classification and feature selection. The core idea is to enlarge the distance between different classes under the conceptual framework of LSR. First, a technique called ε-dragging is introduced to force the regression targets of different classes moving along opposite directions such that the distances between classes can be enlarged. Then, the ε-draggings are integrated into the LSR model for multiclass classification. Our learning framework, referred to as discriminative LSR, has a compact model form, where there is no need to train two-class machines that are independent of each other. With its compact form, this model can be naturally extended for feature selection. This goal is achieved in terms of L2,1 norm of matrix, generating a sparse learning model for feature selection. The model for multiclass classification and its extension for feature selection are finally solved elegantly and efficiently. Experimental evaluation over a range of benchmark datasets indicates the validity of our method.