Local Feature Selection for Data Classification

Maximum margin and global criterion based-recursive feature selection

In this research paper, we aim to investigate and address the limitations of recursive feature elimination (RFE) and its variants in high-dimensional feature selection tasks. We identify two main challenges associated with these methods. Firstly, the feature ranking criterion utilized in these approaches is inconsistent with the maximum-margin theory. Secondly, the computation of the criterion is performed locally, lacking the ability to measure the importance of features globally. To overcome these challenges, we propose a novel feature ranking criterion called Maximum Margin and Global (MMG) criterion. This criterion utilizes the classification margin to determine the importance of features and computes it globally, enabling a more accurate assessment of feature importance. Moreover, we introduce an optimal feature subset evaluation algorithm that leverages the MMG criterion to determine the best subset of features. To enhance the efficiency of the proposed algorithms, we provide two alpha seeding strategies that significantly reduce computational costs while maintaining high accuracy. These strategies offer a practical means to expedite the feature selection process. Through extensive experiments conducted on ten benchmark datasets, we demonstrate that our proposed algorithms outperform current state-of-the-art methods. Additionally, the alpha seeding strategies yield significant speedups, further enhancing the efficiency of the feature selection process.

Research on Industry Data Analytics on Processing Procedure of Named 3-4-8-2 Components Combination for the Application Identification in New Chain Convenience Store

With the rapid economic boom of Asian countries, the president of Country-A has made great efforts to reform in recent years. The prospect of economic development is promising, and business opportunities are emerging gradually, depicting a prosperous scene; accordingly, people’s livelihood consumption also has changed significantly. The original main point of consumption for urban and rural people was the old and traditional grocery store with poor sanitation, but due to the economic improvement, the quality of consumption has also improved, and convenience stores are gradually replacing grocery store. However, convenience store management involves performance, logistic, competition, and personnel costs. Both whether the store can create a net profit and evaluate and select a new store will be important keys that significantly influence business performance. Therefore, this study attempts to use the industry data analysis method for highlighting a concept of processing an experience procedure of named 3-4-8-2 components combination in two stages. First, in the data preprocessing stage, this research considers 22 condition attributes and two types of decision factors, that include net profit and new store selection, and use both techniques of attribute selection and data discretization through the analysis and prediction of data mining tools. Next, in the experiment execution stage, three well-known classifiers (Bayes net, logistic regression, and J48 decision tree) with past good performance and four models (without preprocessing, with attribute selection, with data discretization, and with attribute selection and data discretization) are used for eight different experiments through two data verification methods (percentage split and cross-validation). Conclusively, three key results are identified from empirical analysis: (1) It is found that the prediction accuracy of the J48 decision tree classifier is relatively high and stable among the three classifiers in this study; at the same time, the J48 decision tree can yield comprehensible knowledge-based rules to instruct interested parties. (2) The results of this study show that the important attributes for the net profit decision attribute include the store type, POS number, and cashier number, while the important attributes for the new store selection include the store type and cashier number. (3) There is a difference in the selection of important attributes. Furthermore, four key valuable contributions are addressed from the empirical results, including academic contributions, enterprise contributions, application contributions, and management contributions. It is expected that the direction of store layout expansion can be found and identified through this study, but there are still many risks hidden behind the considerable business opportunities that need to be carefully managed.

Multiclass feature selection with metaheuristic optimization algorithms: a review

Selecting relevant feature subsets is vital in machine learning, and multiclass feature selection is harder to perform since most classifications are binary. The feature selection problem aims at reducing the feature set dimension while maintaining the performance model accuracy. Datasets can be classified using various methods. Nevertheless, metaheuristic algorithms attract substantial attention to solving different problems in optimization. For this reason, this paper presents a systematic survey of literature for solving multiclass feature selection problems utilizing metaheuristic algorithms that can assist classifiers selects optima or near optima features faster and more accurately. Metaheuristic algorithms have also been presented in four primary behavior-based categories, i.e., evolutionary-based, swarm-intelligence-based, physics-based, and human-based, even though some literature works presented more categorization. Further, lists of metaheuristic algorithms were introduced in the categories mentioned. In finding the solution to issues related to multiclass feature selection, only articles on metaheuristic algorithms used for multiclass feature selection problems from the year 2000 to 2022 were reviewed about their different categories and detailed descriptions. We considered some application areas for some of the metaheuristic algorithms applied for multiclass feature selection with their variations. Popular multiclass classifiers for feature selection were also examined. Moreover, we also presented the challenges of metaheuristic algorithms for feature selection, and we identified gaps for further research studies.

A Complete Process of Text Classification System Using State-of-the-Art NLP Models

With the rapid advancement of information technology, online information has been exponentially growing day by day, especially in the form of text documents such as news events, company reports, reviews on products, stocks-related reports, medical reports, tweets, and so on. Due to this, online monitoring and text mining has become a prominent task. During the past decade, significant efforts have been made on mining text documents using machine and deep learning models such as supervised, semisupervised, and unsupervised. Our area of the discussion covers state-of-the-art learning models for text mining or solving various challenging NLP (natural language processing) problems using the classification of texts. This paper summarizes several machine learning and deep learning algorithms used in text classification with their advantages and shortcomings. This paper would also help the readers understand various subtasks, along with old and recent literature, required during the process of text classification. We believe that readers would be able to find scope for further improvements in the area of text classification or to propose new techniques of text classification applicable in any domain of their interest.

Feature selection using autoencoders with Bayesian methods to high-dimensional data

It is easy to lead to poor generalization in machine learning tasks using real-world data directly, since such data is usually high-dimensional dimensionality and limited. Through learning the low dimensional representations of high-dimensional data, feature selection can retain useful features for machine learning tasks. Using these useful features effectively trains machine learning models. Hence, it is a challenge for feature selection from high-dimensional data. To address this issue, in this paper, a hybrid approach consisted of an autoencoder and Bayesian methods is proposed for a novel feature selection. Firstly, Bayesian methods are embedded in the proposed autoencoder as a special hidden layer. This of doing is to increase the precision during selecting non-redundant features. Then, the other hidden layers of the autoencoder are used for non-redundant feature selection. Finally, compared with the mainstream approaches for feature selection, the proposed method outperforms them. We find that the way consisted of autoencoders and probabilistic correction methods is more meaningful than that of stacking architectures or adding constraints to autoencoders as regards feature selection. We also demonstrate that stacked autoencoders are more suitable for large-scale feature selection, however, sparse autoencoders are beneficial for a smaller number of feature selection. We indicate that the value of the proposed method provides a theoretical reference to analyze the optimality of feature selection.

Multiview Feature Selection for Single-View Classification

In many real-world scenarios, data from multiple modalities (sources) are collected during a development phase. Such data are referred to as multiview data. While additional information from multiple views often improves the performance, collecting data from such additional views during the testing phase may not be desired due to the high costs associated with measuring such views or, unavailability of such additional views. Therefore, in many applications, despite having a multiview training data set, it is desired to do performance testing using data from only one view. In this paper, we present a multiview feature selection method that leverages the knowledge of all views and use it to guide the feature selection process in an individual view. We realize this via a multiview feature weighting scheme such that the local margins of samples in each view are maximized and similarities of samples to some reference points in different views are preserved. Also, the proposed formulation can be used for cross-view matching when the view-specific feature weights are pre-computed on an auxiliary data set. Promising results have been achieved on nine real-world data sets as well as three biometric recognition applications. On average, the proposed feature selection method has improved the classification error rate by 31 percent of the error rate of the state-of-the-art.

A novel feature selection algorithm based on damping oscillation theory

Feature selection is an important task in big data analysis and information retrieval processing. It reduces the number of features by removing noise, extraneous data. In this paper, one feature subset selection algorithm based on damping oscillation theory and support vector machine classifier is proposed. This algorithm is called the Maximum Kendall coefficient Maximum Euclidean Distance Improved Gray Wolf Optimization algorithm (MKMDIGWO). In MKMDIGWO, first, a filter model based on Kendall coefficient and Euclidean distance is proposed, which is used to measure the correlation and redundancy of the candidate feature subset. Second, the wrapper model is an improved grey wolf optimization algorithm, in which its position update formula has been improved in order to achieve optimal results. Third, the filter model and the wrapper model are dynamically adjusted by the damping oscillation theory to achieve the effect of finding an optimal feature subset. Therefore, MKMDIGWO achieves both the efficiency of the filter model and the high precision of the wrapper model. Experimental results on five UCI public data sets and two microarray data sets have demonstrated the higher classification accuracy of the MKMDIGWO algorithm than that of other four state-of-the-art algorithms. The maximum ACC value of the MKMDIGWO algorithm is at least 0.5% higher than other algorithms on 10 data sets.

Estimating Feature-Label Dependence Using Gini Distance Statistics

Identifying statistical dependence between the features and the label is a fundamental problem in supervised learning. This paper presents a framework for estimating dependence between numerical features and a categorical label using generalized Gini distance, an energy distance in reproducing kernel Hilbert spaces (RKHS). Two Gini distance based dependence measures are explored: Gini distance covariance and Gini distance correlation. Unlike Pearson covariance and correlation, which do not characterize independence, the above Gini distance based measures define dependence as well as independence of random variables. The test statistics are simple to calculate and do not require probability density estimation. Uniform convergence bounds and asymptotic bounds are derived for the test statistics. Comparisons with distance covariance statistics are provided. It is shown that Gini distance statistics converge faster than distance covariance statistics in the uniform convergence bounds, hence tighter upper bounds on both Type I and Type II errors. Moreover, the probability of Gini distance covariance statistic under-performing the distance covariance statistic in Type II error decreases to 0 exponentially with the increase of the sample size. Extensive experimental results are presented to demonstrate the performance of the proposed method.

Trial-Specific Feature Performance on Single-Channel Auditory Mismatch Negativity Detection

Successful detection of uncommon events is vital in the survival of an organism. Specifically, the study of neuro-sensory detection lends itself widely to understanding the human brain. Mismatch Negativity (MMN) is an important Event-Related Potential (ERP) response to an oddball stimulus which is preceded by repeated homogeneous stimulation. MMN is associated with perceptual learning and medical diagnostics among other applications. Currently, MMN detection relies on visual inspection of ERPs by skilled clinicians which makes for a costly, slow and subjective tool. In this paper, we use MMN to quantify the discriminative abilities of healthy or diagnosed subjects. We introduce a novel algorithmic method to extract and select important trial-specific features for discriminating standard from deviant responses. We utilize machine learning and classification approaches to evaluate our novel model using single-subject trial data while minimizing the number of necessary selection features provided by statistical test parameters and Genetic Algorithm (GA). In this work, a large variety of methods with 27 subjects, hundreds of trials and electrode counts compete for the definitive discrimination of MMN events. Our model requires only one EEG channel, a single subject and as low as five deviant tones. The results show statistically significant detection improvement over the traditional methods while maximizing resource economy.

Pixel-Level Discrete Multiobjective Sampling for Image Matting

In sampling-based matting methods, the alpha is estimated by choosing the best pair of foreground and background color samples. The lack of true samples is the major obstacle in obtaining high-quality alpha mattes. Regrettably, several proposed approaches did not address the conflicts among multiple sampling criteria and the effects of incomplete sample spaces. To address this issue, we propose a pixel-level discrete multiobjective sampling (PDMS) method. The color sampling process at each unknown pixel is formalized as a multiobjective optimization problem (MOP). The strength of PDMS includes its ability to minimize both color difference and spatial distance between unknown and known pixels, and its capacity to adaptively make trade-offs among conflicting sampling criteria. To mitigate the effects of incomplete sample spaces, the sample space is extended to complete known regions in PDMS, which means that the colors of all known pixels can be sampled, instead of mean colors of superpixels. Our experimental results show that PDMS collects a small set of samples while achieving smaller minimum absolute difference in alpha estimation. Moreover, PDMS implements pixel-level sampling by using the proposed multiobjective optimization algorithm to efficiently solve sampling MOPs. The PDMS-based matting method provides high-quality alpha mattes with sharp boundaries and thus outperforms those prior image matting methods in terms of gradient error.

Development of a point of care system for automated coma prognosis: a prospective cohort study protocol

INTRODUCTION

Coma is a deep state of unconsciousness that can be caused by a variety of clinical conditions. Traditional tests for coma outcome prediction are based mainly on a set of clinical observations. Recently, certain event-related potentials (ERPs), which are transient electroencephalogram (EEG) responses to auditory, visual or tactile stimuli, have been introduced as useful predictors of a positive coma outcome (ie, emergence). However, such tests require the skills of clinical neurophysiologists, who are not commonly available in many clinical settings. Additionally, none of the current standard clinical approaches have sufficient predictive accuracies to provide definitive prognoses.

OBJECTIVE

The objective of this study is to develop improved machine learning procedures based on EEG/ERP for determining emergence from coma.

METHODS AND ANALYSIS

Data will be collected from 50 participants in coma. EEG/ERP data will be recorded for 24 consecutive hours at a maximum of five time points spanning 30 days from the date of recruitment to track participants' progression. The study employs paradigms designed to elicit brainstem potentials, middle-latency responses, N100, mismatch negativity, P300 and N400. In the case of patient emergence, data are recorded on that occasion to form an additional basis for comparison. A relevant data set will be developed from the testing of 20 healthy controls, each spanning a 15-hour recording period in order to formulate a baseline. Collected data will be used to develop an automated procedure for analysis and detection of various ERP components that are salient to prognosis. Salient features extracted from the ERP and resting-state EEG will be identified and combined to give an accurate indicator of prognosis.

ETHICS AND DISSEMINATION

This study is approved by the Hamilton Integrated Research Ethics Board (project number 4840). Results will be disseminated through peer-reviewed journal articles and presentations at scientific conferences.

TRIAL REGISTRATION NUMBER

NCT03826407.

A Machine Learning Framework for Automatic and Continuous MMN Detection With Preliminary Results for Coma Outcome Prediction

Mismatch negativity (MMN) is a component of the event-related potential (ERP) that is elicited through an odd-ball paradigm. The existence of the MMN in a coma patient has a good correlation with coma emergence; however, this component can be difficult to detect. Previously, MMN detection was based on visual inspection of the averaged ERPs by a skilled clinician, a process that is expensive and not always feasible in practice. In this paper, we propose a practical machine learning (ML) based approach for detection of MMN component, thus, improving the accuracy of prediction of emergence from coma. Furthermore, the method can operate on an automatic and continuous basis thus alleviating the need for clinician involvement. The proposed method is capable of the MMN detection over intervals as short as two minutes. This finer time resolution enables identification of waxing and waning cycles of a conscious state. An auditory odd-ball paradigm was applied to 22 healthy subjects and 2 coma patients. A coma patient is tested by measuring the similarity of the patient's ERP responses with the aggregate healthy responses. Because the training process for measuring similarity requires only healthy subjects, the complexity and practicality of training procedure of the proposed method are greatly improved relative to training on coma patients directly. Since there are only two coma patients involved with this study, the results are reported on a very preliminary basis. Preliminary results indicate we can detect the MMN component with an accuracy of 92.7% on healthy subjects. The method successfully predicted emergence in both coma patients when conventional methods failed. The proposed method for collecting training data using exclusively healthy subjects is a novel approach that may prove useful in future, unrelated studies where ML methods are used.

Local-Nearest-Neighbors-Based Feature Weighting for Gene Selection

Selecting functional genes is essential for analyzing microarray data. Among many available feature (gene) selection approaches, the ones on the basis of the large margin nearest neighbor receive more attention due to their low computational costs and high accuracies in analyzing the high-dimensional data. Yet, there still exist some problems that hamper the existing approaches in sifting real target genes, including selecting erroneous nearest neighbors, high sensitivity to irrelevant genes, and inappropriate evaluation criteria. Previous pioneer works have partly addressed some of the problems, but none of them are capable of solving these problems simultaneously. In this paper, we propose a new local-nearest-neighbors-based feature weighting approach to alleviate the above problems. The proposed approach is based on the trick of locally minimizing the within-class distances and maximizing the between-class distances with the nearest neighbors rule. We further define a feature weight vector, and construct it by minimizing the cost function with a regularization term. The proposed approach can be applied naturally to the multi-class problems and does not require extra modification. Experimental results on the UCI and the open microarray data sets validate the effectiveness and efficiency of the new approach.

Image-Specific Classification With Local and Global Discriminations

Most image classification methods try to learn classifiers for each class using training images alone. Due to the interclass and intraclass variations, it would be more effective to take the testing images into consideration for classifier learning. In this brief, we propose a novel image-specific classification method by combing the local and global discriminations of training images. We adaptively train classifier for each testing image instead of generating classifiers for each class with training images alone. For each testing image, we first select its ${k}$ nearest neighbors in the training set with the corresponding labels for local classifier training. This helps to model the distinctive characters of each testing image. Besides, we also use all the training images for global discrimination modeling. The local and global discriminations are combined for final classification. In this way, we could not only model the specific character of each testing image but also avoid the local optimum by jointly considering all the training images. To evaluate the usefulness of the proposed image-specific classification with local and global discrimination (ISC-LG) method, we conduct image classification experiments on several public image data sets. The superior performances over other baseline methods prove the effectiveness of the proposed ISC-LG method.

Object Categorization Using Class-Specific Representations

Object categorization refers to the task of automatically classifying objects based on the visual content. Existing approaches simply represent each image with the visual features without considering the specific characters of images within the same class. However, objects of the same class may exhibit unique characters, which should be represented accordingly. In this brief, we propose a novel class-specific representation strategy for object categorization. For each class, we first model the characters of images within the same class using Gaussian mixture model (GMM). We then represent each image by calculating the Euclidean distance and relative Euclidean distance between the image and the GMM model for each class. We concatenate the representations of each class for joint representation. In this way, we can represent an image by not only considering the visual contents but also combining the class-specific characters. Experiments on several public available data sets validate the superiority of the proposed class-specific representation method over well-established algorithms for object category predictions.

Exploring diagnosis and imaging biomarkers of Parkinson's disease via iterative canonical correlation analysis based feature selection

Parkinson's disease (PD) is a neurodegenerative disorder that progressively hampers the brain functions and leads to various movement and non-motor symptoms. However, it is difficult to attain early-stage PD diagnosis based on the subjective judgment of physicians in clinical routines. Therefore, automatic and accurate diagnosis of PD is highly demanded, so that the corresponding treatment can be implemented more appropriately. In this paper, we focus on finding the most discriminative features from different brain regions in PD through T1-weighted MR images, which can help the subsequent PD diagnosis. Specifically, we proposed a novel iterative canonical correlation analysis (ICCA) feature selection method, aiming at exploiting MR images in a more comprehensive manner and fusing features of different types into a common space. To state succinctly, we first extract the feature vectors from the gray matter and the white matter tissues separately, represented as insights of two different anatomical feature spaces for the subject's brain. The ICCA feature selection method aims at iteratively finding the optimal feature subset from two sets of features that have inherent high correlation with each other. In experiments we have conducted thorough investigations on the optimal feature set extracted by our ICCA method. We also demonstrate that using the proposed feature selection method, the PD diagnosis performance is further improved, and also outperforms many state-of-the-art methods.

Logistic Localized Modeling of the Sample Space for Feature Selection and Classification

Conventional feature selection algorithms assign a single common feature set to all regions of the sample space. In contrast, this paper proposes a novel algorithm for localized feature selection for which each region of the sample space is characterized by its individual distinct feature subset that may vary in size and membership. This approach can therefore select an optimal feature subset that adapts to local variations of the sample space, and hence offer the potential for improved performance. Feature subsets are computed by choosing an optimal coordinate space so that, within a localized region, within-class distances and between-class distances are, respectively, minimized and maximized. Distances are measured using a logistic function metric within the corresponding region. This enables the optimization process to focus on a localized region within the sample space. A local classification approach is utilized for measuring the similarity of a new input data point to each class. The proposed logistic localized feature selection (lLFS) algorithm is invariant to the underlying probability distribution of the data; hence, it is appropriate when the data are distributed on a nonlinear or disjoint manifold. lLFS is efficiently formulated as a joint convex/increasing quasi-convex optimization problem with a unique global optimum point. The method is most applicable when the number of available training samples is small. The performance of the proposed localized method is successfully demonstrated on a large variety of data sets. We demonstrate that the number of features selected by the lLFS method saturates at the number of available discriminative features. In addition, we have shown that the Vapnik-Chervonenkis dimension of the localized classifier is finite. Both these factors suggest that the lLFS method is insensitive to the overfitting issue, relative to other methods.

Differential Evolutionary Superpixel Segmentation

Superpixel segmentation has been of increasing importance in many computer vision applications recently. To handle the problem, most state-of-the-art algorithms either adopt a local color variance model or a local optimization algorithm. This paper develops a new approach, named differential evolutionary superpixels, which is able to optimize the global properties of segmentation by means of a global optimizer. We design a comprehensive objective function aggregating within-superpixel error, boundary gradient, and a regularization term. Minimizing the within-superpixel error enforces the homogeneity of superpixels. In addition, the introduction of boundary gradient drives the superpixel boundaries to capture the natural image boundaries, so as to make each superpixel overlaps with a single object. The regularizer further encourages producing similarly sized superpixels that are friendly to human vision. The optimization is then accomplished by a powerful global optimizer-differential evolution. The algorithm constantly evolves the superpixels by mimicking the process of natural evolution, while using a linear complexity to the image size. Experimental results and comparisons with eleven state-of-the-art peer algorithms verify the promising performance of our algorithm.