Ensemble of feature selection algorithms: a multi-criteria decision-making approach

Rough hypervolume-driven feature selection with groupwise intelligent sampling for detecting clinical characterization of lupus nephritis

Systemic lupus erythematosus (SLE) is an autoimmune inflammatory disease. Lupus nephritis (LN) is a major risk factor for morbidity and mortality in SLE. Proliferative and pure membranous LN have different prognoses and may require different treatments. This study proposes a binary rough hypervolume-driven spherical evolution algorithm with groupwise intelligent sampling (bRGSE). The efficient dimensionality reduction capability of the bRGSE is verified across twelve datasets. These datasets are from the public datasets, with feature dimensions ranging from seven hundred to fifty thousand. The experimental results indicate that bRGSE performs better than seven high-performing alternatives. Then, the bRGSE was combined with adaptive boosting (AdaBoost) to form a new model (bRGSE_AdaBoost), which analyzed clinical records collected from 110 patients with LN. Experimental results show that the proposed bRGSE_AdaBoost can identify the most critical indicators, including urine latent blood, white blood cells, endogenous creatinine clearing rate, and age. These indicators may help differentiate between proliferative LN and membranous LN. The proposed bRGSE algorithm is an efficient dimensionality reduction method. The developed bRGSE_AdaBoost model, a computer-aided model, achieved an accuracy of 96.687 % and is expected to provide early warning for the treatment and diagnosis of LN.

Adap-BDCM: Adaptive Bilinear Dynamic Cascade Model for Classification Tasks on CNV Datasets

Copy number variation (CNV) is an essential genetic driving factor of cancer formation and progression, making intelligent classification based on CNV feasible. However, there are a few challenges in the current machine learning and deep learning methods, such as the design of base classifier combination schemes in ensemble methods and the selection of layers of neural networks, which often result in low accuracy. Therefore, an adaptive bilinear dynamic cascade model (Adap-BDCM) is developed to further enhance the accuracy and applicability of these methods for intelligent classification on CNV datasets. In this model, a feature selection module is introduced to mitigate the interference of redundant information, and a bilinear model based on the gated attention mechanism is proposed to extract more beneficial deep fusion features. Furthermore, an adaptive base classifier selection scheme is designed to overcome the difficulty of manually designing base classifier combinations and enhance the applicability of the model. Lastly, a novel feature fusion scheme with an attribute recall submodule is constructed, effectively avoiding getting stuck in local solutions and missing some valuable information. Numerous experiments have demonstrated that our Adap-BDCM model exhibits optimal performance in cancer classification, stage prediction, and recurrence on CNV datasets. This study can assist physicians in making diagnoses faster and better.

A novel parallel feature rank aggregation algorithm for gene selection applied to microarray data classification

Microarray data often comprises numerous genes, yet not all genes are relevant for predicting cancer. Feature selection becomes a crucial step to reduce the high dimensionality in these kinds of data. While no single feature selection method consistently outperforms others across diverse domains, the combination of multiple feature selectors or rankers tends to produce more effective results compared to relying on a single ranker alone. However, this approach can be computationally expensive, particularly when handling a large quantity of features. Hence, this paper presents a parallel feature rank aggregation that utilizes borda count as the rank aggregator. The concept of vertically partitioning the data along feature space was adapted to ease the parallel execution of the aggregation task. Features were selected based on the final aggregated rank list, and their classification performances were evaluated. The model's execution time was also observed across multiple worker nodes of the cluster. The experiment was conducted on six benchmark microarray datasets. The results show the capability of the proposed distributed framework compared to the sequential version in all the cases. It also illustrated the improved accuracy performance of the proposed method and its ability to select a minimal number of genes.

Correlating process parameters and print accuracy of 3D-printable heat acid coagulated milk semisolids and polyol matrix: implications for testing methods

The primary additive manufacturing (AM) technique for all high-viscosity food composites is extrusion-based. Therefore, understanding the impact of process parameters involved is crucial in fulfilling the demand characteristics of the printed constructs. In this regard, a correlation between print accuracy and critical 3D printing (3DP) process variables as a strategy for expediting the selection of 3D printable food inks has the most potential for success. This paper studies the effectiveness of using heat-acid coagulated milk semisolids and polyol matrix as 3D printable food ink for high-quality prints. The study focused on the critical material properties and conducted rheological characterization and particle size distribution analysis. The study obtained the effective range of printing parameters for various process variables using a mathematical model that employed finite element analysis (FEA) to define the flow field characteristics. The dimensional accuracy of the printed constructs under different process variables was determined by utilizing image processing methods. A multi-objective optimization was carried out using the desirability function method to obtain the key correlations between the process parameters for the best-printed construct.

New Classification Method for Independent Data Sources Using Pawlak Conflict Model and Decision Trees

The research concerns data collected in independent sets-more specifically, in local decision tables. A possible approach to managing these data is to build local classifiers based on each table individually. In the literature, many approaches toward combining the final prediction results of independent classifiers can be found, but insufficient efforts have been made on the study of tables' cooperation and coalitions' formation. The importance of such an approach was expected on two levels. First, the impact on the quality of classification-the ability to build combined classifiers for coalitions of tables should allow for the learning of more generalized concepts. In turn, this should have an impact on the quality of classification of new objects. Second, combining tables into coalitions will result in reduced computational complexity-a reduced number of classifiers will be built. The paper proposes a new method for creating coalitions of local tables and generating an aggregated classifier for each coalition. Coalitions are generated by determining certain characteristics of attribute values occurring in local tables and applying the Pawlak conflict analysis model. In the study, the classification and regression trees with Gini index are built based on the aggregated table for one coalition. The system bears a hierarchical structure, as in the next stage the decisions generated by the classifiers for coalitions are aggregated using majority voting. The classification quality of the proposed system was compared with an approach that does not use local data cooperation and coalition creation. The structure of the system is parallel and decision trees are built independently for local tables. In the paper, it was shown that the proposed approach provides a significant improvement in classification quality and execution time. The Wilcoxon test confirmed that differences in accuracy rate of the results obtained for the proposed method and results obtained without coalitions are significant, with a p level = 0.005. The average accuracy rate values obtained for the proposed approach and the approach without coalitions are, respectively: 0.847 and 0.812; so the difference is quite large. Moreover, the algorithm implementing the proposed approach performed up to 21-times faster than the algorithm implementing the approach without using coalitions.

Minimizing features while maintaining performance in data classification problems

High dimensional classification problems have gained increasing attention in machine learning, and feature selection has become essential in executing machine learning algorithms. In general, most feature selection methods compare the scores of several feature subsets and select the one that gives the maximum score. There may be other selections of a lower number of features with a lower score, yet the difference is negligible. This article proposes and applies an extended version of such feature selection methods, which selects a smaller feature subset with similar performance to the original subset under a pre-defined threshold. It further validates the suggested extended version of the Principal Component Loading Feature Selection (PCLFS-ext) results by simulating data for several practical scenarios with different numbers of features and different imbalance rates on several classification methods. Our simulated results show that the proposed method outperforms the original PCLFS and existing Recursive Feature Elimination (RFE) by giving reasonable feature reduction on various data sets, which is important in some applications.

Using Feature Selection with Machine Learning for Generation of Insurance Insights

Insurance is a data-rich sector, hosting large volumes of customer data that is analysed to evaluate risk. Machine learning techniques are increasingly used in the effective management of insurance risk. Insurance datasets by their nature, however, are often of poor quality with noisy subsets of data (or features). Choosing the right features of data is a significant pre-processing step in the creation of machine learning models. The inclusion of irrelevant and redundant features has been demonstrated to affect the performance of learning models. In this article, we propose a framework for improving predictive machine learning techniques in the insurance sector via the selection of relevant features. The experimental results, based on five publicly available real insurance datasets, show the importance of applying feature selection for the removal of noisy features before performing machine learning techniques, to allow the algorithm to focus on influential features. An additional business benefit is the revelation of the most and least important features in the datasets. These insights can prove useful for decision making and strategy development in areas/business problems that are not limited to the direct target of the downstream algorithms. In our experiments, machine learning techniques based on a set of selected features suggested by feature selection algorithms outperformed the full feature set for a set of real insurance datasets. Specifically, 20% and 50% of features in our five datasets had improved downstream clustering and classification performance when compared to whole datasets. This indicates the potential for feature selection in the insurance sector to both improve model performance and to highlight influential features for business insights.

Genetic Programming-Based Feature Selection for Emotion Classification Using EEG Signal

The COVID-19 has resulted in one of the world's most significant worldwide lock-downs, affecting human mental health. Therefore, emotion recognition is becoming one of the essential research areas among various world researchers. Treatment that is efficacious and diagnosed early for negative emotions is the only way to save people from mental health problems. Genetic programming, a very important research area of artificial intelligence, proves its potential in almost every field. Therefore, in this study, a genetic program-based feature selection (FSGP) technique is proposed. A fourteen-channel EEG device gives 70 features for the input brain signal; with the help of GP, all the irrelevant and redundant features are separated, and 32 relevant features are selected. The proposed model achieves a classification accuracy of 85% that outmatches other prior works.

Evaluation of Feature Selection Methods on Psychosocial Education Data Using Additive Ratio Assessment

Artificial intelligence, particularly machine learning, is the fastest-growing research trend in educational fields. Machine learning shows an impressive performance in many prediction models, including psychosocial education. The capability of machine learning to discover hidden patterns in large datasets encourages researchers to invent data with high-dimensional features. In contrast, not all features are needed by machine learning, and in many cases, high-dimensional features decrease the performance of machine learning. The feature selection method is one of the appropriate approaches to reducing the features to ensure machine learning works efficiently. Various selection methods have been proposed, but research to determine the essential subset feature in psychosocial education has not been established thus far. This research investigated and proposed methods to determine the best feature selection method in the domain of psychosocial education. We used a multi-criteria decision system (MCDM) approach with Additive Ratio Assessment (ARAS) to rank seven feature selection methods. The proposed model evaluated the best feature selection method using nine criteria from the performance metrics provided by machine learning. The experimental results showed that the ARAS is promising for evaluating and recommending the best feature selection method for psychosocial education data using the teacher’s psychosocial risk levels dataset.