Particle swarm optimization for feature selection in classification: a multi-objective approach

Population characteristic exploitation-based multi-orientation multi-objective gene selection for microarray data classification

Gene selection is a process of selecting discriminative genes from microarray data that helps to diagnose and classify cancer samples effectively. Swarm intelligence evolution-based gene selection algorithms can never circumvent the problem that the population is prone to local optima in the process of gene selection. To tackle this challenge, previous research has focused primarily on two aspects: mitigating premature convergence to local optima and escaping from local optima. In contrast to these strategies, this paper introduces a novel perspective by adopting reverse thinking, where the issue of local optima is seen as an opportunity rather than an obstacle. Building on this foundation, we propose MOMOGS-PCE, a novel gene selection approach that effectively exploits the advantageous characteristics of populations trapped in local optima to uncover global optimal solutions. Specifically, MOMOGS-PCE employs a novel population initialization strategy, which involves the initialization of multiple populations that explore diverse orientations to foster distinct population characteristics. The subsequent step involved the utilization of an enhanced NSGA-II algorithm to amplify the advantageous characteristics exhibited by the population. Finally, a novel exchange strategy is proposed to facilitate the transfer of characteristics between populations that have reached near maturity in evolution, thereby promoting further population evolution and enhancing the search for more optimal gene subsets. The experimental results demonstrated that MOMOGS-PCE exhibited significant advantages in comprehensive indicators compared with six competitive multi-objective gene selection algorithms. It is confirmed that the "reverse-thinking" approach not only avoids local optima but also leverages it to uncover superior gene subsets for cancer diagnosis.

Multi-Objective Self-Adaptive Particle Swarm Optimization for Large-Scale Feature Selection in Classification

Feature selection (FS) is recognized for its role in enhancing the performance of learning algorithms, especially for high-dimensional datasets. In recent times, FS has been framed as a multi-objective optimization problem, leading to the application of various multi-objective evolutionary algorithms (MOEAs) to address it. However, the solution space expands exponentially with the dataset's dimensionality. Simultaneously, the extensive search space often results in numerous local optimal solutions due to a large proportion of unrelated and redundant features [H. Adeli and H. S. Park, Fully automated design of super-high-rise building structures by a hybrid ai model on a massively parallel machine, AI Mag. 17 (1996) 87-93]. Consequently, existing MOEAs struggle with local optima stagnation, particularly in large-scale multi-objective FS problems (LSMOFSPs). Different LSMOFSPs generally exhibit unique characteristics, yet most existing MOEAs rely on a single candidate solution generation strategy (CSGS), which may be less efficient for diverse LSMOFSPs [H. S. Park and H. Adeli, Distributed neural dynamics algorithms for optimization of large steel structures, J. Struct. Eng. ASCE 123 (1997) 880-888; M. Aldwaik and H. Adeli, Advances in optimization of highrise building structures, Struct. Multidiscip. Optim. 50 (2014) 899-919; E. G. González, J. R. Villar, Q. Tan, J. Sedano and C. Chira, An efficient multi-robot path planning solution using a* and coevolutionary algorithms, Integr. Comput. Aided Eng. 30 (2022) 41-52]. Moreover, selecting an appropriate MOEA and determining its corresponding parameter values for a specified LSMOFSP is time-consuming. To address these challenges, a multi-objective self-adaptive particle swarm optimization (MOSaPSO) algorithm is proposed, combined with a rapid nondominated sorting approach. MOSaPSO employs a self-adaptive mechanism, along with five modified efficient CSGSs, to generate new solutions. Experiments were conducted on ten datasets, and the results demonstrate that the number of features is effectively reduced by MOSaPSO while lowering the classification error rate. Furthermore, superior performance is observed in comparison to its counterparts on both the training and test sets, with advantages becoming increasingly evident as the dimensionality increases.

Emotional arousal pattern (EMAP): A new database for modeling momentary subjective and psychophysiological responding to affective stimuli

This article describes a new database (named "EMAP") of 145 individuals' reactions to emotion-provoking film clips. It includes electroencephalographic and peripheral physiological data as well as moment-by-moment ratings for emotional arousal in addition to overall and categorical ratings. The resulting variation in continuous ratings reflects inter-individual variability in emotional responding. To make use of the moment-by-moment data for ratings as well as neurophysiological activity, we used a machine learning approach. The results show that algorithms that are based on temporal information improve predictions compared to algorithms without a temporal component, both within and across participant modeling. Although predicting moment-by-moment changes in emotional experiences by analyzing neurophysiological activity was more difficult than using aggregated experience ratings, selecting a subset of predictors improved the prediction. This also showed that not only single features, for example, skin conductance, but a range of neurophysiological parameters explain variation in subjective fluctuations of subjective experience.

CDMO: Chaotic Dwarf Mongoose Optimization Algorithm for feature selection

In this paper, a modified version of Dwarf Mongoose Optimization Algorithm (DMO) for feature selection is proposed. DMO is a novel technique of the swarm intelligence algorithms which mimic the foraging behavior of the Dwarf Mongoose. The developed method, named Chaotic DMO (CDMO), is considered a wrapper-based model which selects optimal features that give higher classification accuracy. To speed up the convergence and increase the effectiveness of DMO, ten chaotic maps were used to modify the key elements of Dwarf Mongoose movement during the optimization process. To evaluate the efficiency of the CDMO, ten different UCI datasets are used and compared against the original DMO and other well-known Meta-heuristic techniques, namely Ant Colony optimization (ACO), Whale optimization algorithm (WOA), Artificial rabbit optimization (ARO), Harris hawk optimization (HHO), Equilibrium optimizer (EO), Ring theory based harmony search (RTHS), Random switching serial gray-whale optimizer (RSGW), Salp swarm algorithm based on particle swarm optimization (SSAPSO), Binary genetic algorithm (BGA), Adaptive switching gray-whale optimizer (ASGW) and Particle Swarm optimization (PSO). The experimental results show that the CDMO gives higher performance than the other methods used in feature selection. High value of accuracy (91.9-100%), sensitivity (77.6-100%), precision (91.8-96.08%), specificity (91.6-100%) and F-Score (90-100%) for all ten UCI datasets are obtained. In addition, the proposed method is further assessed against CEC'2022 benchmarks functions.

Automated marker-free longitudinal infrared breast image registration by GA-PSO

The automated marker-free longitudinal Infrared (IR) breast image registration overcomes several challenges like no anatomic fiducial markers on the body surface, blurry boundaries, heat pattern variation by environmental and physiological factors, nonrigid deformation, etc, has the ability of quantitative pixel-wise analysis with the heat energy and patterns change in a time course study. To achieve the goal, scale-invariant feature transform, Harris corner, and Hessian matrix were employed to generate the feature points as anatomic fiducial markers, and hybrid genetic algorithm and particle swarm optimization minimizing the matching errors was used to find the appropriate corresponding pairs between the 1st IR image and thenth IR image. Moreover, the mechanism of the IR spectrogram hardware system has a high level of reproducibility. The performance of the proposed longitudinal image registration system was evaluated by the simulated experiments and the clinical trial. In the simulated experiments, the mean difference of our system is 1.64 mm, which increases 57.58% accuracy than manual determination and makes a 17.4% improvement than the previous study. In the clinical trial, 80 patients were captured several times of IR breast images during chemotherapy. Most of them were well aligned in the spatiotemporal domain. In the few cases with evident heat pattern dissipation and spatial deviation, it still provided a reliable comparison of vascular variation. Therefore, the proposed system is accurate and robust, which could be considered as a reliable tool for longitudinal approaches to breast cancer diagnosis.

scFseCluster: a feature selection-enhanced clustering for single-cell RNA-seq data

Single-cell RNA sequencing (scRNA-seq) enables researchers to reveal previously unknown cell heterogeneity and functional diversity, which is impossible with bulk RNA sequencing. Clustering approaches are widely used for analyzing scRNA-seq data and identifying cell types and states. In the past few years, various advanced computational strategies emerged. However, the low generalization and high computational cost are the main bottlenecks of existing methods. In this study, we established a novel computational framework, scFseCluster, for scRNA-seq clustering analysis. scFseCluster incorporates a metaheuristic algorithm (Feature Selection based on Quantum Squirrel Search Algorithm) to extract the optimal gene set, which largely guarantees the performance of cell clustering. We conducted simulation experiments in several aspects to verify the performance of the proposed approach. scFseCluster performed very well on eight benchmark scRNA-seq datasets because of the optimal gene sets obtained using the Feature Selection based on Quantum Squirrel Search Algorithm. The comparative study demonstrated the significant advantages of scFseCluster over seven State-of-the-Art algorithms. In addition, our analysis shows that feature selection on high-variable genes can significantly improve clustering performance. In conclusion, our study demonstrates that scFseCluster is a highly versatile tool for enhancing scRNA-seq data clustering analysis.

Benefiting From Single-Objective Feature Selection to Multiobjective Feature Selection: A Multiform Approach

Evolutionary multiobjective feature selection (FS) has gained increasing attention in recent years. However, it still faces some challenges, for example, the frequently appeared duplicated solutions in either the search space or the objective space lead to the diversity loss of the population, and the huge search space results in the low search efficiency of the algorithm. Minimizing the number of selected features and maximizing the classification performance are two major objectives in FS. Usually, the fitness function of a single-objective FS problem linearly aggregates these two objectives through a weighted sum method. Given a predefined direction (weight) vector, the single-objective FS task can explore the specified direction or area extensively. Different direction vectors result in different search directions in the objective space. Motivated by this, this article proposes a multiform framework, which solves a multiobjective FS task combined with its auxiliary single-objective FS tasks in a multitask environment. By setting different direction vectors, promising feature subsets from single-objective FS tasks can be utilized, to boost the evolutionary search of the multiobjective FS task. By comparing with five classical and state-of-the-art multiobjective evolutionary algorithms, as well as four well-performing FS algorithms, the effectiveness and efficiency of the proposed method are verified via extensive experiments on 18 classification datasets. Furthermore, the effectiveness of the proposed method is also investigated in a noisy environment.

A Specific Emitter Identification System Design for Crossing Signal Modes in the Air Traffic Control Radar Beacon System and Wireless Devices

To improve communication stability, more wireless devices transmit multi-modal signals while operating. The term 'modal' refers to signal waveforms or signal types. This poses challenges to traditional specific emitter identification (SEI) systems, e.g., unknown modal signals require extra open-set mode identification; different modes require different radio frequency fingerprint (RFF) extractors and SEI classifiers; and it is hard to collect and label all signals. To address these issues, we propose an enhanced SEI system consisting of a universal RFF extractor, denoted as multiple synchrosqueezed wavelet transformation of energy unified (MSWTEu), and a new generative adversarial network for feature transferring (FTGAN). MSWTEu extracts uniform RFF features for different modal signals, FTGAN transfers different modal features to a recognized distribution in an unsupervised manner, and a novel training strategy is proposed to achieve emitter identification across multi-modal signals using a single clustering method. To evaluate the system, we built a hybrid dataset, which consists of multi-modal signals transmitted by various emitters, and built a complete civil air traffic control radar beacon system (ATCRBS) dataset for airplanes. The experiments show that our enhanced SEI system can resolve the SEI problems associated with crossing signal modes. It directly achieves 86% accuracy in cross-modal emitter identification using an unsupervised classifier, and simultaneously obtains 99% accuracy in open-set recognition of signal mode.

Video action recognition collaborative learning with dynamics via PSO-ConvNet Transformer

Recognizing human actions in video sequences, known as Human Action Recognition (HAR), is a challenging task in pattern recognition. While Convolutional Neural Networks (ConvNets) have shown remarkable success in image recognition, they are not always directly applicable to HAR, as temporal features are critical for accurate classification. In this paper, we propose a novel dynamic PSO-ConvNet model for learning actions in videos, building on our recent work in image recognition. Our approach leverages a framework where the weight vector of each neural network represents the position of a particle in phase space, and particles share their current weight vectors and gradient estimates of the Loss function. To extend our approach to video, we integrate ConvNets with state-of-the-art temporal methods such as Transformer and Recurrent Neural Networks. Our experimental results on the UCF-101 dataset demonstrate substantial improvements of up to 9% in accuracy, which confirms the effectiveness of our proposed method. In addition, we conducted experiments on larger and more variety of datasets including Kinetics-400 and HMDB-51 and obtained preference for Collaborative Learning in comparison with Non-Collaborative Learning (Individual Learning). Overall, our dynamic PSO-ConvNet model provides a promising direction for improving HAR by better capturing the spatio-temporal dynamics of human actions in videos. The code is available at https://github.com/leonlha/Video-Action-Recognition-Collaborative-Learning-with-Dynamics-via-PSO-ConvNet-Transformer .

An improved multi-objective marine predator algorithm for gene selection in classification of cancer microarray data

Gene selection (GS) is an important branch of interest within the field of feature selection, which is widely used in cancer classification. It provides essential insights into the pathogenesis of cancer and enables a deeper understanding of cancer data. In cancer classification, GS is essentially a multi-objective optimization problem, which aims to simultaneously optimize the two objectives of classification accuracy and the size of the gene subset. The marine predator algorithm (MPA) has been successfully employed in practical applications, however, its random initialization can lead to blindness, which may adversely affect the convergence of the algorithm. Furthermore, the elite individuals in guiding evolution are randomly chosen from the Pareto solutions, which may degrade the good exploration performance of the population. To overcome these limitations, a multi-objective improved MPA with continuous mapping initialization and leader selection strategies is proposed. In this work, a new continuous mapping initialization with ReliefF overwhelms the defects with less information in late evolution. Moreover, an improved elite selection mechanism with Gaussian distribution guides the population to evolve towards a better Pareto front. Finally, an efficient mutation method is adopted to prevent evolutionary stagnation. To evaluate its effectiveness, the proposed algorithm was compared with 9 famous algorithms. The experimental results on 16 datasets demonstrate that the proposed algorithm can significantly reduce the data dimension and obtain the highest classification accuracy on most of high-dimension cancer microarray datasets.

A Non-Dominated Sorting Based Multi-Objective Neural Network Algorithm

Neural Network Algorithm (NNA) is a recently proposed Metaheuristic that is inspired by the idea of artificial neural networks. The performance of NNA on single-objective optimization problems is very promising and effective. In this article, a maiden attempt is made to restructure NNA for its possible use to address multi-objective optimization problems. To make NNA suitable for MOPs several fundamental changes in the original NNA are proposed. A novel concept is proposed to initialize the candidate solution, position update, and selection of target solution. To examine the optimization ability of the proposed scheme, it is tested on several benchmark problems and the results are compared with eight state-of-the-art multi-objective optimization algorithms. Inverse generational distance(IGD) and hypervolume (HV) metrics are also calculated to understand the optimization ability of the proposed scheme. The results are statistically validated using Wilcoxon signed rank test. It is observed that the overall optimization ability of the proposed scheme to solve MOPs is very good.•This paper proposes a method to solve multi-objective optimization problems.•A multi-objective Neural Network Algorithm method is proposed.•The proposed method solves difficult multi-objective optimization problems.

Disease biomarker identification based on sample network optimization

A large amount of evidence shows that biomarkers are discriminant features related to disease development. Thus, the identification of disease biomarkers has become a basic problem in the analysis of complex diseases in the medical fields, such as disease stage judgment, disease diagnosis and treatment. Research based on networks have become one of the most popular methods. Several algorithms based on networks have been proposed to identify biomarkers, however the networks of genes or molecules ignored the similarities and associations among the samples. It is essential to further understand how to construct and optimize the networks to make the identified biomarkers more accurate. On this basis, more effective strategies can be developed to improve the performance of biomarkers identification. In this study, a multi-objective evolution algorithm based on sample similarity networks has been proposed for disease biomarker identification. Specifically, we design the sample similarity networks to extract the structural characteristic information among samples, which used to calculate the influence of the sample to each class. Besides, based on the networks and the group of biomarkers we choose in every iteration, we can divide samples into different classes by the importance for each class. Then, in the process of evolution algorithm population iteration, we develop the elite guidance strategy and fusion selection strategy to select the biomarkers which make the sample classification more accurate. The experiment results on the five gene expression datasets suggests that the algorithm we proposed is superior over some state-of-the-art disease biomarker identification methods.

A multi-objective based radiomics feature selection method for response prediction following radiotherapy

Objective.Radiomics contains a large amount of mineable information extracted from medical images, which has important significance in treatment response prediction for personalized treatment. Radiomics analyses generally involve high dimensions and redundant features, feature selection is essential for construction of prediction models.Approach.We proposed a novel multi-objective based radiomics feature selection method (MRMOPSO), where the number of features, sensitivity, and specificity are jointly considered as optimization objectives in feature selection. The MRMOPSO innovated in the following three aspects: (1) Fisher score to initialize the population to speed up the convergence; (2) Min-redundancy particle generation operations to reduce the redundancy between radiomics features, a truncation strategy was introduced to further reduce the number of features effectively; (3) Particle selection operations guided by elitism strategies to improve local search ability of the algorithm. We evaluated the effectiveness of the MRMOPSO by using a multi-institution oropharyngeal cancer dataset from The Cancer Imaging Archive. 357 patients were used for model training and cross validation, an additional 64 patients were used for evaluation.Main results.The area under the curve (AUC) of our method achieved AUCs of 0.82 and 0.84 for cross validation and independent dataset, respectively. Compared with classical feature selection methods, the AUC of MRMOPSO is significantly higher than the Lasso (AUC = 0.74,p-value = 0.02), minimal-redundancy-maximal-relevance criterion (mRMR) (AUC = 0.73,p-value = 0.05), F-score (AUC = 0.48,p-value < 0.01), and mutual information (AUC = 0.69,p-value < 0.01) methods. Compared to single-objective methods, the AUC of MRMOPSO is 12% higher than those of the genetic algorithm (GA) (AUC = 0.68,p-value = 0.02) and particle swarm optimization algorithm (AUC = 0.72,p-value = 0.05) methods. Compared to other multi-objective feature selection methods, the AUC of MRMOPSO is 14% higher than those of multiple objective particle swarm optimization (MOPSO) (AUC = 0.68,p-value = 0.02) and nondominated sorting genetic algorithm II (NSGA2) (AUC = 0.70,p-value = 0.03).Significance.We proposed a multi-objective based radiomics feature selection method. Compared to conventional feature reduction algorithms, the proposed algorithm effectively reduced feature dimension, and achieved superior performance, with improved sensitivity and specificity, for response prediction in radiotherapy.

Semisupervised Bacterial Heuristic Feature Selection Algorithm for High-Dimensional Classification with Missing Labels

Feature selection is a crucial method for discovering relevant features in high-dimensional data. However, most studies primarily focus on completely labeled data, ignoring the frequent occurrence of missing labels in real-world problems. To address high-dimensional and label-missing problems in data classification simultaneously, we proposed a semisupervised bacterial heuristic feature selection algorithm. To track the label-missing problem, a k-nearest neighbor semisupervised learning strategy is designed to reconstruct missing labels. In addition, the bacterial heuristic algorithm is improved using hierarchical population initialization, dynamic learning, and elite population evolution strategies to enhance the search capacity for various feature combinations. To verify the effectiveness of the proposed algorithm, three groups of comparison experiments based on eight datasets are employed, including two traditional feature selection methods, four bacterial heuristic feature selection algorithms, and two swarm-based heuristic feature selection algorithms. Experimental results demonstrate that the proposed algorithm has obvious advantages in terms of classification accuracy and selected feature numbers.

A self-adaptive quantum equilibrium optimizer with artificial bee colony for feature selection

Feature selection (FS) is a popular data pre-processing technique in machine learning to extract the optimal features to maintain or increase the classification accuracy of the dataset, which is a combinatorial optimization problem, requiring a powerful optimizer to obtain the optimum subset. The equilibrium optimizer (EO) is a recent physical-based metaheuristic algorithm with good performance for various optimization problems, but it may encounter premature or the local convergence in feature selection. This work presents a self-adaptive quantum EO with artificial bee colony for feature selection, named SQEOABC. In the proposed algorithm, the quantum theory and the self-adaptive mechanism are employed into the updating rule of EO to enhance convergence, and the updating mechanism from the artificial bee colony is also incorporated into EO to achieve appropriate FS solutions. In the experiments, 25 benchmark datasets from the UCI repository are investigated to verify SQEOABC, which is compared with several state-of-the-art metaheuristic algorithms and the variants of EO. The statistical results of fitness values and accuracy demonstrate that SQEOABC has better performance than the compared algorithms and the variants of EO. Finally, a real-world FS problem from COVID-19 illustrates the effectiveness and superiority of SQEOABC.

Applying machine learning techniques to detect the deployment of spatial working memory from the spiking activity of MT neurons

Neural signatures of working memory have been frequently identified in the spiking activity of different brain areas. However, some studies reported no memory-related change in the spiking activity of the middle temporal (MT) area in the visual cortex. However, recently it was shown that the content of working memory is reflected as an increase in the dimensionality of the average spiking activity of the MT neurons. This study aimed to find the features that can reveal memory-related changes with the help of machine-learning algorithms. In this regard, different linear and nonlinear features were obtained from the neuronal spiking activity during the presence and absence of working memory. To select the optimum features, the Genetic algorithm, Particle Swarm Optimization, and Ant Colony Optimization methods were employed. The classification was performed using the Support Vector Machine (SVM) and the K-Nearest Neighbor (KNN) classifiers. Our results suggest that the deployment of spatial working memory can be perfectly detected from spiking patterns of MT neurons with an accuracy of 99.65±0.12 using the KNN and 99.50±0.26 using the SVM classifiers.

Solving Periodic Investment Portfolio Selection Problems by a Data-Assisted Multiobjective Evolutionary Approach

Classic portfolio selection problems mainly focus on high-risk financial markets with tradeoffs between returns and risk. However, more risk-averse investors pursue long-term portfolio planning with the objectives of maximizing final returns and maximizing flexibility. This article addresses a new type of the portfolio problem, called periodic investment portfolio selection problems (PIPSPs), in which investors periodically allocate resources to financial products with different periods. A multiobjective model for PIPSPs is first presented. With a mechanism for utilizing the data generated during the implementation of multiobjective evolutionary algorithms (MOEAs), a data-assisted MOEA (DA-MOEA) is proposed to solve PIPSPs. The main idea of a DA-MOEA is to combine a MOEA with a data-assisted process that consists of three components: 1) feature construction; 2) data fusion model development; and 3) obtained information utilization. To solve the addressed PIPSPs, two versions of DA-MOEAs with baselines of nondominated sorting and decomposition-based mechanisms are implemented, namely, the data-assisted NSGA-II (DA-NSGA-II) and data-assisted MOEA/D (DA-MOEA/D). In the developed DA-MOEAs for PIPSPs, a feature construction process and a data fusion model are well designed for mining data with different formats. To validate the algorithms, two sets of test instances are generated. The experimental results demonstrate the efficacy of the data-assisted process. Furthermore, the effects of the algorithm components, such as the data source sizes, information types, and information utilization strategies, are investigated.

Quantum Chaotic Honey Badger Algorithm for Feature Selection

Determining the most relevant features is a critical pre-processing step in various fields to enhance prediction. To address this issue, a set of feature selection (FS) techniques have been proposed; however, they still have certain limitations. For example, they may focus on nearby points, which lowers classification accuracy because the chosen features may include noisy features. To take advantage of the benefits of the quantum-based optimization technique and the 2D chaotic Hénon map, we provide a modified version of the honey badger algorithm (HBA) called QCHBA. The ability of such strategies to strike a balance between exploitation and exploration while identifying the workable subset of pertinent features is the basis for employing them to enhance HBA. The effectiveness of QCHBA was evaluated in a series of experiments conducted using eighteen datasets involving comparison with recognized FS techniques. The results indicate high efficiency of the QCHBA among the datasets using various performance criteria.

Binary dwarf mongoose optimizer for solving high-dimensional feature selection problems

Selecting appropriate feature subsets is a vital task in machine learning. Its main goal is to remove noisy, irrelevant, and redundant feature subsets that could negatively impact the learning model's accuracy and improve classification performance without information loss. Therefore, more advanced optimization methods have been employed to locate the optimal subset of features. This paper presents a binary version of the dwarf mongoose optimization called the BDMO algorithm to solve the high-dimensional feature selection problem. The effectiveness of this approach was validated using 18 high-dimensional datasets from the Arizona State University feature selection repository and compared the efficacy of the BDMO with other well-known feature selection techniques in the literature. The results show that the BDMO outperforms other methods producing the least average fitness value in 14 out of 18 datasets which means that it achieved 77.77% on the overall best fitness values. The result also shows BDMO demonstrating stability by returning the least standard deviation (SD) value in 13 of 18 datasets (72.22%). Furthermore, the study achieved higher validation accuracy in 15 of the 18 datasets (83.33%) over other methods. The proposed approach also yielded the highest validation accuracy attainable in the COIL20 and Leukemia datasets which vividly portray the superiority of the BDMO.

A Steering-Matrix-Based Multiobjective Evolutionary Algorithm for High-Dimensional Feature Selection

In recent years, multiobjective evolutionary algorithms (MOEAs) have been demonstrated to show promising performance in feature selection (FS) tasks. However, designing an MOEA for high-dimensional FS is more challenging due to the curse of dimensionality. To address this problem, in this article, a steering-matrix-based multiobjective evolutionary algorithm, called SM-MOEA, is proposed. In SM-MOEA, a steering matrix is suggested and harnessed to guide the evolution of the population, which not only improves the search efficiency greatly but also obtains the feature subsets with high quality. Specifically, each element SM (i, j) in the steering matrix SM reflects the probability of the j th feature that is selected in the i th individual (feature subset), which is generated by considering the importance of both the feature j and the individual i . Based on the suggested steering matrix, two important operators referred to as dimensionality reduction and individual repairing operators are developed to effectively steer the population evolution in each generation. In addition, an effective initialization and update strategy for the steering matrix is also designed to further improve the performance of SM-MOEA. The experimental results on 12 high-dimensional datasets with the number of features ranging from 3000 to 13 000 demonstrate the superiority of the proposed algorithm over several state-of-the-art algorithms (including single-objective and MOEAs for high-dimensional FS) in terms of both the number and quality of the selected features.

A Fast Hybrid Feature Selection Based on Correlation-Guided Clustering and Particle Swarm Optimization for High-Dimensional Data

The "curse of dimensionality" and the high computational cost have still limited the application of the evolutionary algorithm in high-dimensional feature selection (FS) problems. This article proposes a new three-phase hybrid FS algorithm based on correlation-guided clustering and particle swarm optimization (PSO) (HFS-C-P) to tackle the above two problems at the same time. To this end, three kinds of FS methods are effectively integrated into the proposed algorithm based on their respective advantages. In the first and second phases, a filter FS method and a feature clustering-based method with low computational cost are designed to reduce the search space used by the third phase. After that, the third phase applies oneself to finding an optimal feature subset by using an evolutionary algorithm with the global searchability. Moreover, a symmetric uncertainty-based feature deletion method, a fast correlation-guided feature clustering strategy, and an improved integer PSO are developed to improve the performance of the three phases, respectively. Finally, the proposed algorithm is validated on 18 publicly available real-world datasets in comparison with nine FS algorithms. Experimental results show that the proposed algorithm can obtain a good feature subset with the lowest computational cost.

An electronic transition-based bare bones particle swarm optimization algorithm for high dimensional optimization problems

An electronic transition-based bare bones particle swarm optimization (ETBBPSO) algorithm is proposed in this paper. The ETBBPSO is designed to present high precision results for high dimensional single-objective optimization problems. Particles in the ETBBPSO are divided into different orbits. A transition operator is proposed to enhance the global search ability of ETBBPSO. The transition behavior of particles gives the swarm more chance to escape from local minimums. In addition, an orbit merge operator is proposed in this paper. An orbit with low search ability will be merged by an orbit with high search ability. Extensive experiments with CEC2014 and CEC2020 are evaluated with ETBBPSO. Four famous population-based algorithms are also selected in the control group. Experimental results prove that ETBBPSO can present high precision results for high dimensional single-objective optimization problems.

An Evolutionary Multitasking-Based Feature Selection Method for High-Dimensional Classification

Feature selection (FS) is an important data preprocessing technique in data mining and machine learning, which aims to select a small subset of information features to increase the performance and reduce the dimensionality. Particle swarm optimization (PSO) has been successfully applied to FS due to being efficient and easy to implement. However, most of the existing PSO-based FS methods face the problems of trapping into local optima and computationally expensive high-dimensional data. Multifactorial optimization (MFO), as an effective evolutionary multitasking paradigm, has been widely used for solving complex problems through implicit knowledge transfer between related tasks. Inspired by MFO, this study proposes a novel PSO-based FS method to solve high-dimensional classification via information sharing between two related tasks generated from a dataset. To be specific, two related tasks about the target concept are established by evaluating the importance of features. A new crossover operator, called assortative mating, is applied to share information between these two related tasks. In addition, two mechanisms, which are variable-range strategy and subset updating mechanism, are also developed to reduce the search space and maintain the diversity of the population, respectively. The results show that the proposed FS method can achieve higher classification accuracy with a smaller feature subset in a reasonable time than the state-of-the-art FS methods on the examined high-dimensional classification problems.

A Pattern Mining-Based Evolutionary Algorithm for Large-Scale Sparse Multiobjective Optimization Problems

In real-world applications, there exist a lot of multiobjective optimization problems whose Pareto-optimal solutions are sparse, that is, most variables of these solutions are 0. Generally, many sparse multiobjective optimization problems (SMOPs) contain a large number of variables, which pose grand challenges for evolutionary algorithms to find the optimal solutions efficiently. To address the curse of dimensionality, this article proposes an evolutionary algorithm for solving large-scale SMOPs, which aims to mine the sparse distribution of the Pareto-optimal solutions and, thus, considerably reduces the search space. More specifically, the proposed algorithm suggests an evolutionary pattern mining approach to detect the maximum and minimum candidate sets of the nonzero variables in the Pareto-optimal solutions, and uses them to limit the dimensions in generating offspring solutions. For further performance enhancement, a binary crossover operator and a binary mutation operator are designed to ensure the sparsity of solutions. According to the results on eight benchmark problems and four real-world problems, the proposed algorithm is superior over existing evolutionary algorithms in solving large-scale SMOPs.

From Cognitive Bias Toward Advanced Computational Intelligence for Smart Infrastructure Monitoring

Visual inspections have been typically used in condition assessment of infrastructure. However, they are based on human judgment and their interpretation of data can differ from acquired results. In psychology, this difference is called cognitive bias which directly affects Structural Health Monitoring (SHM)-based decision making. Besides, the confusion between condition state and safety of a bridge is another example of cognitive bias in bridge monitoring. Therefore, integrated computer-based approaches as powerful tools can be significantly applied in SHM systems. This paper explores the relationship between the use of advanced computational intelligence and the development of SHM solutions through conducting an infrastructure monitoring methodology. Artificial Intelligence (AI)-based algorithms, i.e., Artificial Neural Network (ANN), hybrid ANN-based Imperial Competitive Algorithm, and hybrid ANN-based Genetic Algorithm, are developed for damage assessment using a lab-scale composite bridge deck structure. Based on the comparison of the results, the employed evolutionary algorithms could improve the prediction error of the pre-developed network by enhancing the learning procedure of the ANN.