Estimating Optimal Feature Subsets Using Efficient Estimation of High-Dimensional Mutual Information

Majority voting based on different feature ranking techniques from gene expression

In bioinformatics studies, many modeling tasks are characterized by high dimensionality, leading to the widespread use of feature selection techniques to reduce dimensionality. There are a multitude of feature selection techniques that have been proposed in the literature, each relying on a single measurement method to select candidate features. This has an impact on the classification performance. To address this issue, we propose a majority voting method that uses five different feature ranking techniques: entropy score, Pearson’s correlation coefficient, Spearman correlation coefficient, Kendall correlation coefficient, and t-test. By using a majority voting approach, only the features that appear in all five ranking methods are selected. This selection process has three key advantages over traditional techniques. Firstly, it is independent of any particular feature ranking method. Secondly, the feature space dimension is significantly reduced compared to other ranking methods. Finally, the performance is improved as the most discriminatory and informative features are selected via the majority voting process. The performance of the proposed method was evaluated using an SVM, and the results were assessed using accuracy, sensitivity, specificity, and AUC on various biomedical datasets. The results demonstrate the superior effectiveness of the proposed method compared to state-of-the-art methods in the literature.

A novel hybrid approach for feature selection enhancement: COVID-19 case study

Feature selection is a promising Artificial Intelligence technique for screening, analysing, predicting, and tracking current COVID-19 patients and likely future patients. Significant applications are developed to track data of confirmed, recovered, and death cases. In this work, we propose a new feature selection method based on a new way of hybridization between filter and wrapper methods. The proposed approach is expected to achieve high classification accuracy with a small feature subset. Specifically, the main contribution of this work is a four steps-based approach organized as follows: First, we remove consecutively duplicate and constant features. Then, we select the highest-ranked feature with Mutual Information. In the last step, we run the 'Backward Feature Elimination' algorithm to delete features from the active subset until a stopping criterion based on the degradation of classification performance is met. We applied the proposed approach to a COVID-19 dataset to test its ability to find the relevant feature for characterizing the disease, such as new cases infected with the virus, people vaccinated, and the number of deaths, to better assess the situation. For evaluation purposes, experiments are conducted at the first stage on the COVID-19 dataset, then on six benchmark datasets that have a high dimensional and large size. The method performance is tracked and measured on these datasets and a comparison with many approaches is provided.

TCMI: a non-parametric mutual-dependence estimator for multivariate continuous distributions

The identification of relevant features, i.e., the driving variables that determine a process or the properties of a system, is an essential part of the analysis of data sets with a large number of variables. A mathematical rigorous approach to quantifying the relevance of these features is mutual information. Mutual information determines the relevance of features in terms of their joint mutual dependence to the property of interest. However, mutual information requires as input probability distributions, which cannot be reliably estimated from continuous distributions such as physical quantities like lengths or energies. Here, we introduce total cumulative mutual information (TCMI), a measure of the relevance of mutual dependences that extends mutual information to random variables of continuous distribution based on cumulative probability distributions. TCMI is a non-parametric, robust, and deterministic measure that facilitates comparisons and rankings between feature sets with different cardinality. The ranking induced by TCMI allows for feature selection, i.e., the identification of variable sets that are nonlinear statistically related to a property of interest, taking into account the number of data samples as well as the cardinality of the set of variables. We evaluate the performance of our measure with simulated data, compare its performance with similar multivariate-dependence measures, and demonstrate the effectiveness of our feature-selection method on a set of standard data sets and a typical scenario in materials science.

A Hybrid Approach to Improve Flood Forecasting by Combining a Hydrodynamic Flow Model and Artificial Neural Networks

Climate change is driving worsening flood events worldwide. In this study, a hybrid approach based on a combination of the optimization of a hydrodynamic model and an error correction modeling that exploit different aspects of the physical system is proposed to improve the forecasting accuracy of flood water levels. In the parameter optimization procedure for the hydrodynamic model, Manning’s roughness coefficients were estimated by considering their spatial distribution and temporal variation in unsteady flow conditions. In the following error correction procedure, the systematic errors of the optimized hydrodynamic model were captured by combining the input variable selection method using partial mutual information (PMI) and artificial neural networks (ANNs), and therefore, complementary information provided by the data was achieved. The developed ANNs were used to analyze the potential non-linear relationships between the considered state variables and simulation errors to predict systematic errors. To assess the hybrid forecasting approach (hydrodynamic model with an ANN-based error correction model), performances of the hydrodynamic model, two ANN-based water-level forecasting models (ANN1 and ANN2), and the hybrid model were compared. Regarding input candidates, ANN1 considers the historical observations only, and ANN2 considers not only the historical observations that used in ANN1 but also the prescribed boundary conditions required for the hydrodynamic forecast model. As a result, the hybrid model significantly improved the forecasting accuracy of flood water levels compared to individual models, which indicates that the hybrid model is able to take advantage of complementary strengths of both the hydrodynamic model and the ANN model. The optimization of the hydrodynamic model allowing spatially and temporally variable parameters estimated water levels with acceptable accuracy. Furthermore, the use of PMI-based input variable selection and optimized ANNs as error correction models for different sites were proven to successfully predict simulation errors in the hydrodynamic model. Hence, the parameter optimization of the hydrodynamic model coupled with error correction modeling for water level forecasting can be used to provide accurate information for flood management.

Breast Cancer Detection using Multimodal Time Series Features from Ultrasound Shear Wave Absolute Vibro-Elastography

In shear wave absolute vibro-elastography (S-WAVE), a steady-state multi-frequency external mechanical excitation is applied to tissue, while a time-series of ultrasound radio-frequency (RF) data are acquired. Our objective is to determine the potential of S-WAVE to classify breast tissue lesions as malignant or benign. We present a new processing pipeline for feature-based classification of breast cancer using S-WAVE data, and we evaluate it on a new data set collected from 40 patients. Novel bi-spectral and Wigner spectrum features are computed directly from the RF time series and are combined with textural and spectral features from B-mode and elasticity images. The Random Forest permutation importance ranking and the Quadratic Mutual Information methods are used to reduce the number of features from 377 to 20. Support Vector Machines and Random Forest classifiers are used with leave-one-patient-out and Monte Carlo cross-validations. Classification results obtained for different feature sets are presented. Our best results (95% confidence interval, Area Under Curve = 95%1.45%, sensitivity = 95%, and specificity = 93%) outperform the state-of-the-art reported S-WAVE breast cancer classification performance. The effect of feature selection and the sensitivity of the above classification results to changes in breast lesion contours is also studied. We demonstrate that time-series analysis of externally vibrated tissue as an elastography technique, even if the elasticity is not explicitly computed, has promise and should be pursued with larger patient datasets. Our study proposes novel directions in the field of elasticity imaging for tissue classification.

Estimating Feature-Label Dependence Using Gini Distance Statistics

Identifying statistical dependence between the features and the label is a fundamental problem in supervised learning. This paper presents a framework for estimating dependence between numerical features and a categorical label using generalized Gini distance, an energy distance in reproducing kernel Hilbert spaces (RKHS). Two Gini distance based dependence measures are explored: Gini distance covariance and Gini distance correlation. Unlike Pearson covariance and correlation, which do not characterize independence, the above Gini distance based measures define dependence as well as independence of random variables. The test statistics are simple to calculate and do not require probability density estimation. Uniform convergence bounds and asymptotic bounds are derived for the test statistics. Comparisons with distance covariance statistics are provided. It is shown that Gini distance statistics converge faster than distance covariance statistics in the uniform convergence bounds, hence tighter upper bounds on both Type I and Type II errors. Moreover, the probability of Gini distance covariance statistic under-performing the distance covariance statistic in Type II error decreases to 0 exponentially with the increase of the sample size. Extensive experimental results are presented to demonstrate the performance of the proposed method.

A novel hybrid filter/wrapper method for feature selection in archaeological ceramics classification by laser-induced breakdown spectroscopy

Laser-induced breakdown spectroscopy (LIBS) has been appreciated as a valuable analytical tool in the cultural heritage field owing to its unique technological superiority, particularly in combination with chemometric methods. Feature selection (FS) as an indispensable pre-processing step in data optimization, for eliminating the redundant or irrelevant features from high-dimensional data to enhance the predictive capacity and result comprehensibility of multivariate classification based on LIBS technology. In this paper, a novel hybrid filter/wrapper method based on the MI-DBS algorithm was proposed to enhance the qualitative analysis performance of the LIBS technique. The proposed method combines the advantages of the mutual information (MI) algorithm based filter method and bi-directional selection (DBS) algorithm based wrapper method. The MI algorithm is the first to remove the redundant or uncorrelated features so that a simplified input subset can be established. Then, the DBS algorithm is used to further select the retained features and hence to seek an optimal feature subset with good predictive performance. To benefit the above feature selection process, the wavelet transform denoising (WTD) method was used to reduce the noise from LIBS spectra. LIBS experiments were performed using 35 archaeological ceramic samples. Besides, the proposed hybrid filter/wrapper method was implemented through a random forest (RF) based nonlinear multivariate classification method. Through a comparison between several other feature selection methods and the proposed method, it has been seen that the proposed method is the best regarding the predictive performance and number of the selected features. Finally, the MI-DBS algorithm is used to seek the optimal features from the full spectrum (220-720 nm); the corresponding sensitivity, specificity and accuracy acquired through the RF classifier for the test set were 0.9722, 0.9956 and 0.9850. It is shown from the general results that the MI-DBS algorithm is more effective in terms of improving the model performance and decreasing the redundant or uncorrelated features and computational time and serves as a good alternative for FS in multivariate classification.

Improving prostate cancer classification in H&E tissue micro arrays using Ki67 and P63 histopathology

Histopathology of Hematoxylin and Eosin (H&E)-stained tissue obtained from biopsy is commonly used in prostate cancer (PCa) diagnosis. Automatic PCa classification of digitized H&E slides has been developed before, but no attempts have been made to classify PCa using additional tissue stains registered to H&E. In this paper, we demonstrate that using H&E, Ki67 and p63-stained (3-stain) tissue improves PCa classification relative to H&E alone. We also show that we can infer PCa-relevant Ki67 and p63 information from the H&E slides alone, and use it to achieve H&E-based PCa classification that is comparable to the 3-stain classification. Reported improvements apply to classifying benign vs. malignant tissue, and low grade (Gleason group 2) vs. high grade (Gleason groups 3,4,5) cancer. Specifically, we conducted four classification tasks using 333 tissue samples extracted from 231 radical prostatectomy patients: regression tree-based classification using either (i) 3-stain features, with a benign vs malignant area under the curve (AUC = 92.9%), or (ii) real H&E features and H&E features learned from Ki67 and p63 stains (AUC = 92.4%), as well as deep learning classification using either (iii) real 3-stain tissue patches (AUC = 94.3%) and (iv) real H&E patches and generated Ki67 and p63 patches (AUC = 93.0%) using a deep convolutional generative adversarial network. Classification performance was assessed with Monte Carlo cross validation and quantified in terms of the Area Under the Curve, Brier score, sensitivity, and specificity. Our results are interpretable and indicate that the standard H&E classification could be improved by mimicking other stain types.

Performance of ultra-high-frequency ultrasound in the evaluation of skin involvement in systemic sclerosis: a preliminary report

Objective

High frequency ultrasound allows visualization of epidermis, dermis and hypodermis, precise measurement of skin thickness, as well as assessment of skin oedema, fibrosis and atrophy. The aim of this pilot cross-sectional observational study was to assess the performance and multiobserver variability of ultra-high-frequency (UHF) (50 MHz) ultrasound (US) in measuring skin thickness as well as the capacity of UHF-derived skin features to differentiate SSc patients from healthy controls.

Methods

Twenty-one SSc patients (16 limited and five diffuse SSc) and six healthy controls were enrolled. All subjects underwent US evaluation by three experts at three anatomical sites (forearm, hand and finger). Dermal thickness was measured and two rectangular regions of interest, one in dermis and one in hypodermis, were established for texture feature analysis.

Results

UHF-US allowed a precise identification and measurement of the thickness of the dermis. The dermal thickness in the finger was significantly higher in patients than in controls (P < 0.05), while in the forearm it was significantly lower in patients than in controls (P < 0.001). Interobserver variability for dermal thickness was good to excellent [forearm intraclass correlation coefficient (ICC) = 0.754; finger ICC = 0.699; hand ICC = 0.602]. Texture computed analysis of dermis and hypodermis was able to discriminate between SSc and healthy subjects (area under the curve >0.7).

Conclusion

These preliminary data show that skin UHF-US allows a very detailed imaging of skin layers, a reliable measurement of dermal thickness, and a discriminative capacity between dermis and hypodermis texture features in SSc and healthy subjects.

Classification of Perceived Mental Stress Using A Commercially Available EEG Headband

Human stress is a serious health concern, which must be addressed with appropriate actions for a healthy society. This paper presents an experimental study to ascertain the appropriate phase, when electroencephalography (EEG) based data should be recorded for classification of perceived mental stress. The process involves data acquisition, pre-processing, feature extraction and selection, and classification. The stress level of each subject is recorded by using a standard perceived stress scale questionnaire, which is then used to label the EEG data. The data are divided into two (stressed and non-stressed) and three (non-stressed, mildly stressed, and stressed) classes. The EEG data of 28 participants are recorded using a commercially available four channel Muse EEG headband in two phases i.e., pre-activity and post-activity. Five feature groups, which include power spectral density, correlation, differential asymmetry, rational asymmetry, and power spectrum are extracted from five bands of each EEG channel. We propose a new feature selection algorithm, which selects features from appropriate EEG frequency band based on classification accuracy. Three classifiers i.e., support vector machine, the Naive Bayes, and multi-layer perceptron are used to classify stress level of the participants. It is evident from our results that EEG recording during the pre-activity phase is better for classifying the perceived stress. An accuracy of [Formula: see text] and [Formula: see text] is achieved for two- and three-class stress classification, respectively, while utilizing five groups of features from theta band. Our proposed feature selection algorithm is compared with existing algorithms and gives better classification results.

Data preprocessing in predictive data mining

A large variety of issues influence the success of data mining on a given problem. Two primary and important issues are the representation and the quality of the dataset. Specifically, if much redundant and unrelated or noisy and unreliable information is presented, then knowledge discovery becomes a very difficult problem. It is well-known that data preparation steps require significant processing time in machine learning tasks. It would be very helpful and quite useful if there were various preprocessing algorithms with the same reliable and effective performance across all datasets, but this is impossible. To this end, we present the most well-known and widely used up-to-date algorithms for each step of data preprocessing in the framework of predictive data mining.

Diurnal variation in expired breath volatiles in malaria-infected and healthy volunteers

We previously showed that thioether levels in the exhaled breath volatiles of volunteers undergoing controlled human malaria infection (CHMI) with P. falciparum increase as infection progresses. In this study, we show that thioethers have diurnal cyclical increasing patterns and their levels are significantly higher in P. falciparum CHMI volunteers compared to those of healthy volunteers. The synchronized cycle and elevation of thioethers were not present in P. vivax-infection, therefore it is likely that the thioethers are associated with unique factors in the pathology of P. falciparum. Moreover, we found that time-of-day of breath collection is important to accurately predict (98%) P. falciparum-infection. Critically, this was achieved when the disease was asymptomatic and parasitemia was below the level detectable by microscopy. Although these findings are encouraging, they show limitations because of the limited and logistically difficult diagnostic window and its utility to P. falciparum malaria only. We looked for new biomarkers in the breath of P. vivax CHMI volunteers and found that a set of terpenes increase significantly over the course of the malaria infection. The accuracy of predicting P. vivax using breath terpenes was up to 91%. Moreover, some of the terpenes were also found in the breath of P. falciparum CHMI volunteers (accuracy up to 93.5%). The results suggest that terpenes might represent better biomarkers than thioethers to predict malaria as they were not subject to malaria pathogens diurnal changes.

Breast lesion classification based on supersonic shear-wave elastography and automated lesion segmentation from B-mode ultrasound images

Supersonic shear-wave elastography (SWE) has emerged as a useful imaging modality for breast lesion assessment. Regions of interest (ROIs) were required to be specified for extracting features that characterize malignancy of lesions. Although analyses have been performed in small rectangular ROIs identified manually by expert observers, the results were subject to observer variability and the analysis of small ROIs would potentially miss out important features available in other parts of the lesion. Recent investigations extracted features from the entire lesion segmented by B-mode ultrasound images either manually or semi-automatically, but lesion delineation using existing techniques is time-consuming and prone to variability as intensive user interactions are required. In addition, rich diagnostic features were available along the rim surrounding the lesion. The width of the rim analyzed was subjectively and empirically determined by expert observers in previous studies after intensive visual study on the images, which is time-consuming and susceptible to observer variability. This paper describes an analysis pipeline to segment and classify lesions efficiently. The lesion boundary was first initialized and then deformed based on energy fields generated by the dyadic wavelet transform. Features of the SWE images were extracted from inside and outside of a lesion for different widths of the surrounding rim. Then, feature selection was performed followed by the Support Vector Machine (SVM) classification. This strategy obviates the empirical and time-consuming selection of the surrounding rim width before the analysis. The pipeline was evaluated on 137 lesions. Feature selection was performed 20 times using different sets of 14 lesions (7 malignant, 7 benign). Leave-one-out SVM classification was performed in each of the 20 experiments with a mean sensitivity, specificity and accuracy of 95.1%, 94.6% and 94.8% respectively. The pipeline took an average of 20 s to process a lesion. The fact that this efficient pipeline generated classification accuracy superior to that of existing algorithms suggests that improved efficiency did not compromise classification accuracy. The ability to streamline the quantitative assessment of SWE images will potentially accelerate the adoption of the combined use of ultrasound and elastography in clinical practice.

Enhancement of early cervical cancer diagnosis with epithelial layer analysis of fluorescence lifetime images

This work reports the use of layer analysis to aid the fluorescence lifetime diagnosis of cervical intraepithelial neoplasia (CIN) from H&E stained cervical tissue sections. The mean and standard deviation of lifetimes in single region of interest (ROI) of cervical epithelium were previously shown to correlate to the gold standard histopathological classification of early cervical cancer. These previously defined single ROIs were evenly divided into layers for analysis. A 10-layer model revealed a steady increase in fluorescence lifetime from the inner to the outer epithelial layers of healthy tissue sections, suggesting a close association with cellular maturity. The shorter lifetime and minimal lifetime increase towards the epithelial surface of CIN-affected regions are in good agreement with the absence of cellular maturation in CIN. Mean layer lifetimes in the top-half cervical epithelium were used as feature vectors for extreme learning machine (ELM) classifier discriminations. It was found that the proposed layer analysis technique greatly improves the sensitivity and specificity to 94.6% and 84.3%, respectively, which can better supplement the traditional gold standard cervical histopathological examinations.

Online learning of a Dirichlet process mixture of Beta-Liouville distributions via variational inference

A large class of problems can be formulated in terms of the clustering process. Mixture models are an increasingly important tool in statistical pattern recognition and for analyzing and clustering complex data. Two challenging aspects that should be addressed when considering mixture models are how to choose between a set of plausible models and how to estimate the model's parameters. In this paper, we address both problems simultaneously within a unified online nonparametric Bayesian framework that we develop to learn a Dirichlet process mixture of Beta-Liouville distributions (i.e., an infinite Beta-Liouville mixture model). The proposed infinite model is used for the online modeling and clustering of proportional data for which the Beta-Liouville mixture has been shown to be effective. We propose a principled approach for approximating the intractable model's posterior distribution by a tractable one-which we develop-such that all the involved mixture's parameters can be estimated simultaneously and effectively in a closed form. This is done through variational inference that enjoys important advantages, such as handling of unobserved attributes and preventing under or overfitting; we explain that in detail. The effectiveness of the proposed work is evaluated on three challenging real applications, namely facial expression recognition, behavior modeling and recognition, and dynamic textures clustering.

A novel feature selection method and its application

In this paper, a novel feature selection method based on rough sets and mutual information is proposed. The dependency of each feature guides the selection, and mutual information is employed to reduce the features which do not favor addition of dependency significantly. So the dependency of the subset found by our method reaches maximum with small number of features. Since our method evaluates both definitive relevance and uncertain relevance by a combined selection criterion of dependency and class-based distance metric, the feature subset is more relevant than other rough sets based methods. As a result, the subset is near optimal solution. In order to verify the contribution, eight different classification applications are employed. Our method is also employed on a real Alzheimer's disease dataset, and finds a feature subset where classification accuracy arrives at 81.3%. Those present results verify the contribution of our method.

A NOVEL FEATURE SELECTION ALGORITHM WITH SUPERVISED MUTUAL INFORMATION FOR CLASSIFICATION

Feature selection is essential in data mining and pattern recognition, especially for database classification. During past years, several feature selection algorithms have been proposed to measure the relevance of various features to each class. A suitable feature selection algorithm normally maximizes the relevancy and minimizes the redundancy of the selected features. The mutual information measure can successfully estimate the dependency of features on the entire sampling space, but it cannot exactly represent the redundancies among features. In this paper, a novel feature selection algorithm is proposed based on maximum relevance and minimum redundancy criterion. The mutual information is used to measure the relevancy of each feature with class variable and calculate the redundancy by utilizing the relationship between candidate features, selected features and class variables. The effectiveness is tested with ten benchmarked datasets available in UCI Machine Learning Repository. The experimental results show better performance when compared with some existing algorithms.

Face Recognition under Illumination Variation Using Shadow Compensation and Pixel Selection

We propose a robust face recognition method under illumination variation. By using shadow compensation methods, we restore the image of a facial image taken under arbitrary illumination into an image that is similar to the image taken with frontal illumination. Then we apply a pixel selection method to these restored images in order to reduce the noise components, which can interfere with the extraction of discriminant features for face recognition. The experimental results for the CMU-PIE, Yale B and Multi-PIE databases show that the proposed method results in the improvement of recognition performance under illumination variation.

Tree-structured feature extraction using mutual information

One of the most informative measures for feature extraction (FE) is mutual information (MI). In terms of MI, the optimal FE creates new features that jointly have the largest dependency on the target class. However, obtaining an accurate estimate of a high-dimensional MI as well as optimizing with respect to it is not always easy, especially when only small training sets are available. In this paper, we propose an efficient tree-based method for FE in which at each step a new feature is created by selecting and linearly combining two features such that the MI between the new feature and the class is maximized. Both the selection of the features to be combined and the estimation of the coefficients of the linear transform rely on estimating 2-D MIs. The estimation of the latter is computationally very efficient and robust. The effectiveness of our method is evaluated on several real-world data sets. The results show that the classification accuracy obtained by the proposed method is higher than that achieved by other FE methods.

Rule extraction from minimal neural networks for credit card screening

While feedforward neural networks have been widely accepted as effective tools for solving classification problems, the issue of finding the best network architecture remains unresolved, particularly so in real-world problem settings. We address this issue in the context of credit card screening, where it is important to not only find a neural network with good predictive performance but also one that facilitates a clear explanation of how it produces its predictions. We show that minimal neural networks with as few as one hidden unit provide good predictive accuracy, while having the added advantage of making it easier to generate concise and comprehensible classification rules for the user. To further reduce model size, a novel approach is suggested in which network connections from the input units to this hidden unit are removed by a very straightaway pruning procedure. In terms of predictive accuracy, both the minimized neural networks and the rule sets generated from them are shown to compare favorably with other neural network based classifiers. The rules generated from the minimized neural networks are concise and thus easier to validate in a real-life setting.

Feature selection for MLP neural network: the use of random permutation of probabilistic outputs

This paper presents a new wrapper-based feature selection method for multilayer perceptron (MLP) neural networks. It uses a feature ranking criterion to measure the importance of a feature by computing the aggregate difference, over the feature space, of the probabilistic outputs of the MLP with and without the feature. Thus, a score of importance with respect to every feature can be provided using this criterion. Based on the numerical experiments on several artificial and real-world data sets, the proposed method performs, in general, better than several selected feature selection methods for MLP, particularly when the data set is sparse or has many redundant features. In addition, as a wrapper-based approach, the computational cost for the proposed method is modest.

A new automated method for the segmentation and characterization of breast masses on ultrasound images

Segmentation is one of the first steps in most computer-aided diagnosis systems for characterization of masses as malignant or benign. In this study, the authors designed an automated method for segmentation of breast masses on ultrasound (US) images. The method automatically estimated an initial contour based on a manually identified point approximately at the mass center. A two-stage active contour method iteratively refined the initial contour and performed self-examination and correction on the segmentation result. To evaluate the method, the authors compared it with manual segmentation by two experienced radiologists (R1 and R2) on a data set of 488 US images from 250 biopsy-proven masses (100 malignant and 150 benign). Two area overlap ratios (AOR1 and AOR2) and an area error measure were used as performance measures to evaluate the segmentation accuracy. Values for AOR1, defined as the ratio of the intersection of the computer and the reference segmented areas to the reference segmented area, were 0.82 +/- 0.16 and 0.84 +/- 0.18, respectively, when manually segmented mass regions by R1 and R2 were used as the reference. Although this indicated a high agreement between the computer and manual segmentations, the two radiologists' manual segmentation results were significantly (p < 0.03) more consistent, with AOR1 = 0.84 +/- 0.16 and 0.91 +/- 0.12, respectively, when the segmented regions by R1 and R2 were used as the reference. To evaluate the segmentation method in terms of lesion classification accuracy, feature spaces were formed by extracting texture, width-to-height, and posterior shadowing features based on either automated computer segmentation or the radiologists' manual segmentation. A linear discriminant analysis classifier was designed using stepwise feature selection and two-fold cross validation to characterize the mass as malignant or benign. For features extracted from computer segmentation, the case-based test A(z) values ranged from 0.88 +/- 0.03 to 0.92 +/- 0.02, indicating a comparable performance to those extracted from manual segmentation by radiologists (A(z) value range: 0.87 +/- 0.03 to 0.90 +/- 0.03).

Natural image statistics and low-complexity feature selection

Low-complexity feature selection is analyzed in the context of visual recognition. It is hypothesized that high-order dependences of bandpass features contain little information for discrimination of natural images. This hypothesis is characterized formally by the introduction of the concepts of conjunctive interference and decomposability order of a feature set. Necessary and sufficient conditions for the feasibility of low-complexity feature selection are then derived in terms of these concepts. It is shown that the intrinsic complexity of feature selection is determined by the decomposability order of the feature set and not its dimension. Feature selection algorithms are then derived for all levels of complexity and are shown to be approximated by existing information-theoretic methods, which they consistently outperform. The new algorithms are also used to objectively test the hypothesis of low decomposability order through comparison of classification performance. It is shown that, for image classification, the gain of modeling feature dependencies has strongly diminishing returns: best results are obtained under the assumption of decomposability order 1. This suggests a generic law for bandpass features extracted from natural images: that the effect, on the dependence of any two features, of observing any other feature is constant across image classes.

Normalized mutual information feature selection

A filter method of feature selection based on mutual information, called normalized mutual information feature selection (NMIFS), is presented. NMIFS is an enhancement over Battiti's MIFS, MIFS-U, and mRMR methods. The average normalized mutual information is proposed as a measure of redundancy among features. NMIFS outperformed MIFS, MIFS-U, and mRMR on several artificial and benchmark data sets without requiring a user-defined parameter. In addition, NMIFS is combined with a genetic algorithm to form a hybrid filter/wrapper method called GAMIFS. This includes an initialization procedure and a mutation operator based on NMIFS to speed up the convergence of the genetic algorithm. GAMIFS overcomes the limitations of incremental search algorithms that are unable to find dependencies between groups of features.

An unsupervised partition method based on association delineated revised mutual information

Background

The syndrome is the basic pathological unit and the key concept in traditional Chinese medicine (TCM) and the herbal remedy is prescribed according to the syndrome a patient catches. Nevertheless, few studies are dedicated to investigate the number of syndromes and what these syndromes are. Correlative measure based on mutual information can measure arbitrary statistical dependences between discrete and continuous variables.

Results

We presented a revised version of mutual information to discriminate positive and negative association. The entropy partition method self-organizedly discovers the effective patterns in patient data and rat data. The super-additivity of cluster by mutual information is proved and N-class association concept is introduced in our model to reduce computational complexity. Validation of the algorithm is performed by using the patient data and its diagnostic data. The partition results of patient data indicate that the algorithm achieves a high sensitivity with 96.48% and each classified pattern is of clinical significance. The partition results of rat data show the inherent relationship between vascular endothelial function related parameters and neuro-endocrine-immune (NEI) network related parameters.

Conclusion

Therefore, we conclude that the algorithm provides an excellent solution to patients and rats data problem in the context of traditional Chinese medicine.

A Backward Feature Selection by Creating Compact Neural Network Using Coherence Learning and Pruning

In this paper we propose a new backward feature selection method that generates compact classifier of a three-layered feed-forward artificial neural network (ANN). In the algorithm, that is based on the wrapper model, two techniques, coherence and pruning, are integrated together in order to find relevant features with a network of minimal numbers of hidden units and connections. Firstly, a coherence learning and a pruning technique are applied during training for removing unnecessary hidden units from the network. After that, attribute distances are measured by a straightforward computation that is not computationally expensive. An attribute is then removed based on an error-based criterion. The network is retrained after the removal of the attribute. This unnecessary attribute selection process is continued until a stopping criterion is satisfied. We applied this method to several standard benchmark classification problems such as breast cancer, diabetes, glass identification and thyroid problems. Experimental results confirmed that the proposed method generates compact network structures that can select relevant features with good classification accuracies.