StackGenVis: Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics

Temporomandibular joint CBCT image segmentation via multi-view ensemble learning network

Accurate segmentation of the temporomandibular joint (TMJ) from cone beam CT (CBCT) images holds significant clinical value for diagnosing temporomandibular joint osteoarthrosis (TMJOA) and related conditions. Convolutional neural network-based medical image segmentation methods have achieved state-of-the-art performance in various segmentation tasks. However, 3D medical images segmentation requires substantial global context and rich spatial semantic information, demanding much more GPU memory and computational resources. To address these challenges in 3D medical image segmentation, we propose a novel network- the MVEL-Net (Multi-view Ensemble Learning Network) for TMJ CBCT image segmentation. By resampling images along three dimensions, we generate multiple weak learners with different spatial semantic information. A subsequent strong learning network effectively integrates the outputs from these weak learners to achieve more accurate segmentation results. We evaluated our network model using a clinical dataset comprising 88 subjects with TMJ CBCT images. The average Dice similarity coefficient (DSC) was 0.9817 ± 0.0049, the average surface distance was 0.0540 ± 0.0179 mm, and the 95% Hausdorff distance was 0.1743 ± 0.0550 mm. Our proposed MVEL-Net demonstrates excellent segmentation performance on TMJ from CBCT images, while using fewer GPU memory resources compared to other 3D networks. The effectiveness of this method in capturing spatial context could be leveraged for tasks like organ segmentation from volumetric scans. This may facilitate wider adoption of AI-based solutions for automated analysis of 3D medical images.

Dual-Stage Stacking Machine Learning Method Considering Virtual Sample Generation for the Prediction of ZIF-8' BET Specific Surface Area with Experimental Validation

The widespread application of metal-organic frameworks (MOFs) in wastewater and gas treatment has created an increasing demand for accurate and rapid assessment of their BET specific surface area. However, experimental methods for acquiring sufficient statistical data are often costly and time-consuming. Therefore, this study proposes a dual-stage stacking model with Gaussian mixture model-virtual sample generation (GMM-VSG) technology for the BET specific surface area prediction. In this study, 90 real samples were selected from the MOF database and 300 virtual samples were generated. The performance on both real and virtual samples was evaluated by using four machine learning models, including Bayesian regression (Bayes), adaptive boosting (AdaBoost), random forest (RF), and extreme gradient boosting (XGBoost). Subsequently, three best-performing models and a linear regression model were selected for constructing a two-stage stacking model, with R2 value of 0.974. Finally, experimental conditions were adjusted based on feature importance analysis during the validation process, and the result shows that the prediction accuracy of the BET specific surface area is 0.943. This study contributes to the development of more efficient and accurate evaluation methods.

Construction of interpretable ensemble learning models for predicting bioaccumulation parameters of organic chemicals in fish

Accurate prediction of bioaccumulation parameters is essential for assessing exposure, hazards, and risks of chemicals. However, the majority of prediction models on bioaccumulation parameters are individual models based on a single algorithm and lack model interpretation, resulting in unsatisfactory prediction accuracy due to inherent constraints of the algorithm and weak interpretability. Ensemble learning (EL) that combine multiple algorithms, coupled with SHapley Additive exPlanation (SHAP) method, may overcome the limitations. Herein, EL models were constructed for three bioaccumulation parameters using datasets covering 2496 chemicals. The EL models demonstrated superior prediction accuracy compared to both individual models developed in this study and those from previous research, achieving a coefficient of determination of up to 0.861 on the validation sets. Applicability domains were characterized using a structure-activity landscape-based (abbreviated as ADSAL) methodology. The optimal EL models, together with the ADSAL, were successfully used to predict bioaccumulation parameters for 4374 chemicals included in the Inventory of Existing Chemical Substances of China. Model interpretation using the SHAP method offered insight into key features influencing bioaccumulation potential, including hydrophobicity, water solubility, polarizability, ionization potential, weight, and volume of molecules. Overall, the study provides data and models to support the sound management and risk assessment of chemicals.

Fluoride contamination in African groundwater: Predictive modeling using stacking ensemble techniques

Fluoride contamination of groundwater is a severe public health problem in Africa due to natural factors that include geological weathering of fluoride-bearing minerals and climatic conditions characterized by high evaporation rates that highly elevate fluoride levels. Anthropogenic activities further aggravate the problem and have affected millions of people in countries such as; South Africa, Tanzania, Nigeria, Ethiopia, Ghana, Kenya, Mauritania, Botswana, and Egypt. High fluoride levels of up to 10 mg/L have been encountered in parts of the East African Rift Valley, above the WHO's recommended limit of 1.5 mg/L, causing serious dental and skeletal fluorosis among the affected people. In this study, the distributions of F- in groundwater of Africa were forecast using an advanced stacking ensemble learning model based on 11 crucial groundwater physiochemical variables and 6270 accessible statistics of observed concentrations. The enhanced algorithm incorporates randomized trees, Tree-Bag, RF, DT, XGB, and ET Machine as base trainees, with a simple Naïve Bayes as the meta-analyzer. The model's AUC score of 0.86 accurately represented the uneven distributions of groundwater fluoride. The results showed that 20-35 % of the continent's eastern part and 10 % of its western region are at risk of having fluoride levels exceeding WHO limits, with an expected population of around 80 million. Regionally, fluoride contamination ranges from 0.1 to 3 mg/L in West Africa was range from 0.0 to 13.29 mg/L, 0.01-588 mg/L in East Africa, 0.04-65.9 mg/L in South Africa, and 0.1-10.5 mg/L in North and 0.01-1.9 mg/L in Central Africa. Na+ and HCO3- are Africa's leading primary causes of fluoride contamination, with Ca2+ and Cl- contributing to fluoride influence in some parts of the continent. This study helped identify health concerns linked to groundwater fluoride and offered guidance on assessing health risks in areas with sparse sample sizes.

An efficient interpretable stacking ensemble model for lung cancer prognosis

Lung cancer significantly contributes to global cancer mortality, posing challenges in clinical management. Early detection and accurate prognosis are crucial for improving patient outcomes. This study develops an interpretable stacking ensemble model (SEM) for lung cancer prognosis prediction and identifies key risk factors. Using a Kaggle dataset of 1000 patients with 22 variables, the model classifies prognosis into Low, Medium, and High-risk categories. The bootstrap method was employed for evaluation metrics, while SHAP (Shapley Additive Explanations) and LIME (Local Interpretable Model-agnostic Explanations) assessed model interpretability. Results showed SEM's superior interpretability over traditional models, such as Random Forest, Logistic Regression, Decision Tree, Gradient Boosting Machine, Extreme Gradient Boosting Machine, and Light Gradient Boosting Machine. SEM achieved an accuracy of 98.90 %, precision of 98.70 %, F1 score of 98.85 %, sensitivity of 98.77 %, specificity of 95.45 %, Cohen's kappa value of 94.56 %, and an AUC of 98.10 %. The SEM demonstrated robust performance in lung cancer prognosis, revealing chronic lung cancer and genetic risk as major factors.

Advanced Predictive Modeling of Concrete Compressive Strength and Slump Characteristics: A Comparative Evaluation of BPNN, SVM, and RF Models Optimized via PSO

This study presents the development of predictive models for concrete performance, specifically targeting the compressive strength and slump value, utilizing the quantities of individual raw materials in the concrete mix design as input variables. Three distinct machine learning approaches-Backpropagation Neural Network (BPNN), Support Vector Machine (SVM), and Random Forest (RF)-were employed to establish the prediction models independently. In the model construction process, the Particle Swarm Optimization (PSO) algorithm was integrated with cross-validation to fine-tune the hyperparameters of each model, ensuring optimal performance. Following the completion of training and modeling, a comprehensive comparison of the predictive accuracy among the three models was conducted, with the aim of selecting the most suitable model for incorporation into an optimized objective function. The findings reveal that among the chosen machine learning techniques, BPNN exhibited superior predictive capabilities for the compressive strength of concrete. Specifically, in the validation set, BPNN achieved a high correlation coefficient (R) of 0.9531 between the predicted and actual outputs, accompanied by a low Root Mean Square Error (RMSE) of 4.2568 and a Mean Absolute Error (MAE) of 2.6627, indicating a precise and reliable prediction. Conversely, for the prediction of the concrete slump value, RF outperformed the other two models, demonstrating a correlation coefficient (R) of 0.8986, an RMSE of 9.4906, and an MAE of 5.5034 in the validation set. This underscores the effectiveness of RF in capturing the complexity and variability inherent in slump behavior. Overall, this research highlights the potential of integrating advanced machine learning algorithms with optimization techniques for enhancing the accuracy and efficiency of concrete performance predictions. The identified optimal models, BPNN for compressive strength and RF for slump, can serve as valuable tools for engineers and researchers in the field of construction materials, facilitating the design of concrete mixes tailored to specific performance requirements.

Twistable and Stretchable Nasal Patch for Monitoring Sleep-Related Breathing Disorders Based on a Stacking Ensemble Learning Model

Obstructive sleep apnea syndrome disrupts sleep, destroys the homeostasis of biological systems such as metabolism and the immune system, and reduces learning ability and memory. The existing polysomnography used to measure sleep disorders is executed in an unfamiliar environment, which may result in sleep patterns that are different from usual, reducing accuracy. This study reports a machine learning-based personalized twistable patch system that can simply measure obstructive sleep apnea syndrome in daily life. The stretchable patch attaches directly to the nose in an integrated form factor, detecting sleep-disordered breathing by simultaneously sensing microscopic vibrations and airflow in the nasal cavity and paranasal sinuses. The highly sensitive multichannel patch, which can detect airflow at the level of 0.1 m/s, has flexibility via a unique slit pattern and fabric layer. It has linearity with an R2 of 0.992 in the case of the amount of change according to its curvature. The stacking ensemble learning model predicted the degree of sleep-disordered breathing with an accuracy of 92.9%. The approach demonstrates high sleep disorder detection capacity and proactive visual notification. It is expected to help as a diagnostic platform for the early detection of chronic diseases such as cerebrovascular disease and diabetes.

Dynamic stacking ensemble for cross-language code smell detection

Code smells refer to poor design and implementation choices by software engineers that might affect the overall software quality. Code smells detection using machine learning models has become a popular area to build effective models that are capable of detecting different code smells in multiple programming languages. However, the process of building of such effective models has not reached a state of stability, and most of the existing research focuses on Java code smells detection. The main objective of this article is to propose dynamic ensembles using two strategies, namely greedy search and backward elimination, which are capable of accurately detecting code smells in two programming languages (i.e., Java and Python), and which are less complex than full stacking ensembles. The detection performance of dynamic ensembles were investigated within the context of four Java and two Python code smells. The greedy search and backward elimination strategies yielded different base models lists to build dynamic ensembles. In comparison to full stacking ensembles, dynamic ensembles yielded less complex models when they were used to detect most of the investigated Java and Python code smells, with the backward elimination strategy resulting in less complex models. Dynamic ensembles were able to perform comparably against full stacking ensembles with no significant detection loss. This article concludes that dynamic stacking ensembles were able to facilitate the effective and stable detection performance of Java and Python code smells over all base models and with less complexity than full stacking ensembles.

Prediction of arsenic and fluoride in groundwater of the North China Plain using enhanced stacking ensemble learning

Chronic exposure to elevated geogenic arsenic (As) and fluoride (F-) concentrations in groundwater poses a significant global health risk. In regions around the world where regular groundwater quality assessments are limited, the presence of harmful levels of As and F- in shallow groundwater extracted from specific wells remains uncertain. This study utilized an enhanced stacking ensemble learning model to predict the distributions of As and F- in shallow groundwater based on 4,393 available datasets of observed concentrations and forty relevant environmental factors. The enhanced model was obtained by fusing well-suited Extreme Gradient Boosting, Random Forest, and Support Vector Machine as the base learners and a structurally simple Linear Discriminant Analysis as the meta-learner. The model precisely captured the patchy distributions of groundwater As and F- with an AUC value of 0.836 and 0.853, respectively. The findings revealed that 9.0% of the study area was characterized by a high As risk in shallow groundwater, while 21.2% was at high F- risk identified as having a high risk of fluoride contamination. About 0.2% of the study area shows elevated levels of both of them. The affected populations are estimated at approximately 7.61 million, 34.1 million, and 0.2 million, respectively. Furthermore, sedimentary environment exerted the greatest influence on distribution of groundwater As, with human activities and climate following closely behind at 29.5%, 28.1%, and 21.9%, respectively. Likewise, sedimentary environment was the primary factor affecting groundwater F- distribution, followed by hydrogeology and soil physicochemical properties, contributing 27.8%, 24.0%, and 23.3%, respectively. This study contributed to the identification of health risks associated with shallow groundwater As and F-, and provided insights into evaluating health risks in regions with limited samples.

Optimal computed tomography-based biomarkers for prediction of incisional hernia formation

PURPOSE

Unstructured data are an untapped source for surgical prediction. Modern image analysis and machine learning (ML) can harness unstructured data in medical imaging. Incisional hernia (IH) is a pervasive surgical disease, well-suited for prediction using image analysis. Our objective was to identify optimal biomarkers (OBMs) from preoperative abdominopelvic computed tomography (CT) imaging which are most predictive of IH development.

METHODS

Two hundred and twelve rigorously matched colorectal surgery patients at our institution were included. Preoperative abdominopelvic CT scans were segmented to derive linear, volumetric, intensity-based, and textural features. These features were analyzed to find a small subset of OBMs, which are maximally predictive of IH. Three ML classifiers (Ensemble Boosting, Random Forest, SVM) trained on these OBMs were used for prediction of IH.

RESULTS

Altogether, 279 features were extracted from each CT scan. The most predictive OBMs found were: (1) abdominopelvic visceral adipose tissue (VAT) volume, normalized for height; (2) abdominopelvic skeletal muscle tissue volume, normalized for height; and (3) pelvic VAT volume to pelvic outer aspect of body wall skeletal musculature (OAM) volume ratio. Among ML prediction models, Ensemble Boosting produced the best performance with an AUC of 0.85, accuracy of 0.83, sensitivity of 0.86, and specificity of 0.81.

CONCLUSION

These OBMs suggest increased intra-abdominopelvic volume/pressure as the salient pathophysiologic driver and likely mechanism for IH formation. ML models using these OBMs are highly predictive for IH development. The next generation of surgical prediction will maximize the utility of unstructured data using advanced image analysis and ML.

A comparative study of fracture conductivity prediction using ensemble methods in the acid fracturing treatment in oil wells

The study of acid fracture conductivity stands as a pivotal aspect of petroleum engineering, offering a well-established technique to amplify production rates in carbonate reservoirs. This research delves into the intricate dynamics influencing the conductivity of acid fractures, particularly under varying closure stresses and in diverse rock formations. The conductivity of acid fractures is intricately interconnected with the dissolution of rock, etching patterns on fracture surfaces, rock strength, and closure stress. To accurately predict fracture conductivity under different closure stresses, a robust model is necessary. This model involves assessing both the baseline fracture conductivity under zero closure stress and the rate of conductivity variation as closure stress fluctuates. Key among the influential factors affecting fracture conductivity is the type of rock within the reservoir. Understanding and predicting the behavior of different formations under disparate closure stresses poses a significant challenge, as does deciphering the diverse effects of treatment parameters such as acid injection rate and strength on fracture conductivity. In this study, the predictive power of XGBoost, a machine learning algorithm, was explored in assessing acid fracture conductivity in dolomite and limestone formations. The findings revealed XGBoost's ability to outperform previous studies in predicting fracture conductivity in both types of formations. Notably, it exhibited superior accuracy in forecasting fracture conductivity under varying treatment conditions, underscoring its robustness and versatility. The research underscores the pivotal role of closure stress, dissolution rate of rock (DREC), and rock strength in influencing fracture conductivity. By integrating these parameters into the design of acid fracturing operations, accurate predictions can be achieved, allowing for the optimization of treatment designs. This study illuminates the potential of XGBoost in optimizing acid fracturing treatments, ultimately bolstering well productivity in carbonate reservoirs. Furthermore, it advocates for the essential nature of separate modeling and analysis based on rock types to comprehend and optimize fracturing processes. The comparison between dolomite and limestone formations unveiled distinct conductivity behaviors, underlining the significance of tailored analyses based on rock type for precise operational optimization.

BowelNet: Joint Semantic-Geometric Ensemble Learning for Bowel Segmentation From Both Partially and Fully Labeled CT Images

Accurate bowel segmentation is essential for diagnosis and treatment of bowel cancers. Unfortunately, segmenting the entire bowel in CT images is quite challenging due to unclear boundary, large shape, size, and appearance variations, as well as diverse filling status within the bowel. In this paper, we present a novel two-stage framework, named BowelNet, to handle the challenging task of bowel segmentation in CT images, with two stages of 1) jointly localizing all types of the bowel, and 2) finely segmenting each type of the bowel. Specifically, in the first stage, we learn a unified localization network from both partially- and fully-labeled CT images to robustly detect all types of the bowel. To better capture unclear bowel boundary and learn complex bowel shapes, in the second stage, we propose to jointly learn semantic information (i.e., bowel segmentation mask) and geometric representations (i.e., bowel boundary and bowel skeleton) for fine bowel segmentation in a multi-task learning scheme. Moreover, we further propose to learn a meta segmentation network via pseudo labels to improve segmentation accuracy. By evaluating on a large abdominal CT dataset, our proposed BowelNet method can achieve Dice scores of 0.764, 0.848, 0.835, 0.774, and 0.824 in segmenting the duodenum, jejunum-ileum, colon, sigmoid, and rectum, respectively. These results demonstrate the effectiveness of our proposed BowelNet framework in segmenting the entire bowel from CT images.

SliceTeller: A Data Slice-Driven Approach for Machine Learning Model Validation

Real-world machine learning applications need to be thoroughly evaluated to meet critical product requirements for model release, to ensure fairness for different groups or individuals, and to achieve a consistent performance in various scenarios. For example, in autonomous driving, an object classification model should achieve high detection rates under different conditions of weather, distance, etc. Similarly, in the financial setting, credit-scoring models must not discriminate against minority groups. These conditions or groups are called as "Data Slices". In product MLOps cycles, product developers must identify such critical data slices and adapt models to mitigate data slice problems. Discovering where models fail, understanding why they fail, and mitigating these problems, are therefore essential tasks in the MLOps life-cycle. In this paper, we present SliceTeller, a novel tool that allows users to debug, compare and improve machine learning models driven by critical data slices. SliceTeller automatically discovers problematic slices in the data, helps the user understand why models fail. More importantly, we present an efficient algorithm, SliceBoosting, to estimate trade-offs when prioritizing the optimization over certain slices. Furthermore, our system empowers model developers to compare and analyze different model versions during model iterations, allowing them to choose the model version best suitable for their applications. We evaluate our system with three use cases, including two real-world use cases of product development, to demonstrate the power of SliceTeller in the debugging and improvement of product-quality ML models.

An evaluation model for children's foot & ankle deformity severity using sparse multi-objective feature selection algorithm

Foot & ankle deformity is a chronic disease with high incidence and is best treated in childhood. However, the current diagnostic procedures rely on doctor's consultation and empirical judgment, and lack objective and quantitative evaluation methods, resulting in low screening rates. To solve this problem, this paper aims to construct an evaluation model for children's foot & ankle deformity through data mining and machine learning technologies. Firstly, it proposes the grading rules for children's foot & ankle deformity severity based on analyzing the existing quantitative indexes and expert experience. Then the 3D foot scanner is used to collect the sample data including 30 foot structure indexes. Finally, an advanced sparse multi-objective evolutionary algorithm (sparse MO-FS) is present for feature selection. The effectiveness of the proposed sparse MO-FS and its search efficiency are proved by comparing 8 feature selection methods and 7 search strategies. Using sparse MO-FS, foot length, arch index, ankle index, and hallux valgus index are selected, which not only simplifies the evaluation model but also improves the average classification accuracy of random forest to more than 98%.

Assessment of Machine Learning Techniques for Oil Rig Classification in C-Band SAR Images

This article aims at performing maritime target classification in SAR images using machine learning (ML) and deep learning (DL) techniques. In particular, the targets of interest are oil platforms and ships located in the Campos Basin, Brazil. Two convolutional neural networks (CNNs), VGG-16 and VGG-19, were used for attribute extraction. The logistic regression (LR), random forest (RF), support vector machine (SVM), k-nearest neighbours (kNN), decision tree (DT), naive Bayes (NB), neural networks (NET), and AdaBoost (ADBST) schemes were considered for classification. The target classification methods were evaluated using polarimetric images obtained from the C-band synthetic aperture radar (SAR) system Sentinel-1. Classifiers are assessed by the accuracy indicator. The LR, SVM, NET, and stacking results indicate better performance, with accuracy ranging from 84.1% to 85.5%. The Kruskal–Wallis test shows a significant difference with the tested classifier, indicating that some classifiers present different accuracy results. The optimizations provide results with more significant accuracy gains, making them competitive with those shown in the literature. There is no exact combination of methods for SAR image classification that will always guarantee the best accuracy. The optimizations performed in this article were for the specific data set of the Campos Basin, and results may change depending on the data set format and the number of images.

FeatureEnVi: Visual Analytics for Feature Engineering Using Stepwise Selection and Semi-Automatic Extraction Approaches

The machine learning (ML) life cycle involves a series of iterative steps, from the effective gathering and preparation of the data-including complex feature engineering processes-to the presentation and improvement of results, with various algorithms to choose from in every step. Feature engineering in particular can be very beneficial for ML, leading to numerous improvements such as boosting the predictive results, decreasing computational times, reducing excessive noise, and increasing the transparency behind the decisions taken during the training. Despite that, while several visual analytics tools exist to monitor and control the different stages of the ML life cycle (especially those related to data and algorithms), feature engineering support remains inadequate. In this paper, we present FeatureEnVi, a visual analytics system specifically designed to assist with the feature engineering process. Our proposed system helps users to choose the most important feature, to transform the original features into powerful alternatives, and to experiment with different feature generation combinations. Additionally, data space slicing allows users to explore the impact of features on both local and global scales. FeatureEnVi utilizes multiple automatic feature selection techniques; furthermore, it visually guides users with statistical evidence about the influence of each feature (or subsets of features). The final outcome is the extraction of heavily engineered features, evaluated by multiple validation metrics. The usefulness and applicability of FeatureEnVi are demonstrated with two use cases and a case study. We also report feedback from interviews with two ML experts and a visualization researcher who assessed the effectiveness of our system.

Use of metamodels for rapid discovery of narrow bandgap oxide photocatalysts

New photocatalysts are traditionally identified through trial-and-error methods. Machine learning has shown considerable promise for improving the efficiency of photocatalyst discovery from a large potential pool. Here, we describe a multi-step, target-driven consensus method using a stacking meta-learning algorithm that robustly predicts bandgaps and H2 evolution activities of photocatalysts. Trained on small datasets, these models can rapidly screen a large space (>10 million materials) to identify promising, non-toxic compounds as candidate water splitting photocatalysts. Two effective compounds and two controls possessing optimal bandgap values (∼2 eV) but not photoactivity as predicted by the models were synthesized. Their experimentally measured bandgaps and H2 evolution activities were consistent with the predictions. Conspicuously, the two compounds with strong photoactivities under UV and visible light are promising visible-light-driven water splitting photocatalysts. This study demonstrates the power of machine learning and the potential of big data to accelerate discovery of next-generation photocatalysts.

Machine learning-based approach for disease severity classification of carpal tunnel syndrome

Identifying the severity of carpal tunnel syndrome (CTS) is essential to providing appropriate therapeutic interventions. We developed and validated machine-learning (ML) models for classifying CTS severity. Here, 1037 CTS hands with 11 variables each were retrospectively analyzed. CTS was confirmed using electrodiagnosis, and its severity was classified into three grades: mild, moderate, and severe. The dataset was randomly split into a training (70%) and test (30%) set. A total of 507 mild, 276 moderate, and 254 severe CTS hands were included. Extreme gradient boosting (XGB) showed the highest external validation accuracy in the multi-class classification at 76.6% (95% confidence interval [CI] 71.2–81.5). XGB also had an optimal model training accuracy of 76.1%. Random forest (RF) and k-nearest neighbors had the second-highest external validation accuracy of 75.6% (95% CI 70.0–80.5). For the RF and XGB models, the numeric rating scale of pain was the most important variable, and body mass index was the second most important. The one-versus-rest classification yielded improved external validation accuracies for each severity grade compared with the multi-class classification (mild, 83.6%; moderate, 78.8%; severe, 90.9%). The CTS severity classification based on the ML model was validated and is readily applicable to aiding clinical evaluations.