Online Causal Feature Selection for Streaming Features

Feature importance feedback with Deep Q process in ensemble-based metaheuristic feature selection algorithms

Feature selection is an indispensable aspect of modern machine learning, especially for high-dimensional datasets where overfitting and computational inefficiencies are common concerns. Traditional methods often employ either filter, wrapper, or embedded approaches, which have limitations in terms of robustness, computational load, or capability to capture complex interactions among features. Despite the utility of metaheuristic algorithms like Particle Swarm Optimization (PSO), Firefly Algorithm (FA), and Whale Optimization (WOA) in feature selection, there still exists a gap in efficiently incorporating feature importance feedback into these processes. This paper presents a novel approach that integrates the strengths of PSO, FA, and WOA algorithms into an ensemble model and further enhances its performance by incorporating a Deep Q-Learning framework for relevance feedbacks. The Deep Q-Learning module intelligently updates feature importance based on model performance, thereby fine-tuning the selection process iteratively. Our ensemble model demonstrates substantial gains in effectiveness over traditional and individual metaheuristic approaches. Specifically, the proposed model achieved a 9.5% higher precision, an 8.5% higher accuracy, an 8.3% higher recall, a 4.9% higher AUC, and a 5.9% higher specificity across multiple software bug prediction datasets and samples. By resolving some of the key issues in existing feature selection methods and achieving superior performance metrics, this work paves the way for more robust and efficient machine learning models in various applications, from healthcare to natural language processing scenarios. This research provides an innovative framework for feature selection that promises not only superior performance but also offers a flexible architecture that can be adapted for a variety of machine learning challenges.

Online Multi-Label Streaming Feature Selection Based on Label Group Correlation and Feature Interaction

Multi-label streaming feature selection has received widespread attention in recent years because the dynamic acquisition of features is more in line with the needs of practical application scenarios. Most previous methods either assume that the labels are independent of each other, or, although label correlation is explored, the relationship between related labels and features is difficult to understand or specify. In real applications, both situations may occur where the labels are correlated and the features may belong specifically to some labels. Moreover, these methods treat features individually without considering the interaction between features. Based on this, we present a novel online streaming feature selection method based on label group correlation and feature interaction (OSLGC). In our design, we first divide labels into multiple groups with the help of graph theory. Then, we integrate label weight and mutual information to accurately quantify the relationships between features under different label groups. Subsequently, a novel feature selection framework using sliding windows is designed, including online feature relevance analysis and online feature interaction analysis. Experiments on ten datasets show that the proposed method outperforms some mature MFS algorithms in terms of predictive performance, statistical analysis, stability analysis, and ablation experiments.

Causality analysis in type 1 diabetes mellitus with application to blood glucose level prediction

Effective control of blood glucose level (BGL) is the key factor in the management of type 1 diabetes mellitus (T1D). BGL prediction is an important tool to help maximise the time BGL is in the target range and thus minimise both acute and chronic diabetes-related complications. To predict future BGL, histories of variables known to affect BGL, such as carbohydrate intake, injected bolus insulin, and physical activity, are utilised. Due to these identified cause and effect relationships, T1D management can be examined via the causality context. In this respect, this work initially investigates these relations and quantifies the causality strengths of each variable with BGL using the convergent cross mapping method (CCM). Then, considering the extended CCM, the causality strengths of each variable for different lags are quantified. After that, the optimal time lag for each variable is determined according to the quantified causality effects. Subsequently, the feasibility of leveraging causality information as prior knowledge for BGL prediction is investigated by proposing two approaches. In the first approach, causality strengths are used as weights for relevant affecting variables. In the second approach, the optimal causal lags and the corresponding causality strengths are considered the shifts and weights for the variables, respectively. Overall, the evaluation criteria and statistical analysis used for comparing results show the effectiveness of using causality analysis in T1D management.

Online group streaming feature selection using entropy-based uncertainty measures for fuzzy neighborhood rough sets

Online group streaming feature selection, as an essential online processing method, can deal with dynamic feature selection tasks by considering the original group structure information of the features. Due to the fuzziness and uncertainty of the feature stream, some existing methods are unstable and yield low predictive accuracy. To address these issues, this paper presents a novel online group streaming feature selection method (FNE-OGSFS) using fuzzy neighborhood entropy-based uncertainty measures. First, a separability measure integrating the dependency degree with the coincidence degree is proposed and introduced into the fuzzy neighborhood rough sets model to define a new fuzzy neighborhood entropy. Second, inspired by both algebra and information views, some fuzzy neighborhood entropy-based uncertainty measures are investigated and some properties are derived. Furthermore, the optimal features in the group are selected to flow into the feature space according to the significance of features, and the features with interactions are left. Then, all selected features are re-evaluated by the Lasso model to discard the redundant features. Finally, an online group streaming feature selection algorithm is designed. Experimental results compared with eight representative methods on thirteen datasets show that FNE-OGSFS can achieve better comprehensive performance.