Mechanistic and data-driven perspectives on plant uptake of organic pollutants

Establishing reliable predictive models for plant uptake of organic pollutants is crucial for environmental risk assessment and guiding phytoremediation efforts. This study compiled an expanded dataset of plant cuticle-water partition coefficients (Kcw), a useful indicator for plant uptake, for 371 data points of 148 unique compounds and various plant species. Quantum/computational chemistry software and tools were utilized to compute various molecular descriptors, aiming to comprehensively characterize the properties and structures of each compound. Three types of models were developed to predict Kcw: a mechanism-driven pp-LFER model, a data-driven machine learning model, and an integrated mechanism-data-driven model. The mechanism-data-driven GBRT-ppLFER model exhibited superior performance, achieving RMSEtrain = 0.133 and RMSEtest = 0.301 while maintaining interpretability. The Shapley Additive Explanation analysis indicated that pp-LFER parameters, ESPI, FwRadicalmax, ExtFP607, and RDF70s are the key factors influencing plant uptake in the GBRT-ppLFER model. Overall, pp-LFER parameter, ESPI, and ExtFP607 show positive effects, while the remaining factors exhibit negative effects. Partial dependency analysis further indicated that plant uptake is not solely determined by individual factors but rather by the combined interactions of multiple factors. Specifically, compounds with ppLFER parameter >4, ESPI > -25.5, 0.098 < FwRadicalmax <0.132, and 2 < RFD70s < 3, are generally more readily taken up by plants. Besides, the predicted Kcw values from the GBRT-ppLFER model were effectively employed to estimate the plant-water partition coefficients and bioconcentration factors across different plant species and growth media (water, sand, and soil), achieving an outstanding performance with an RMSE of 0.497. This study provides effective tools for assessing plant uptake of organic pollutants and deepens our understanding of plant-environment-compound interactions.

On relevant features for the recurrence prediction of urothelial carcinoma of the bladder

BACKGROUND

Urothelial bladder cancer (UBC) is characterized by a high recurrence rate, which is predicted by scoring systems. However, recent studies show the superiority of Machine Learning (ML) models. Nevertheless, these ML approaches are rarely used in medical practice because most of them are black-box models, that cannot adequately explain how a prediction is made.

OBJECTIVE

We investigate the global feature importance of different ML models. By providing information on the most relevant features, we can facilitate the use of ML in everyday medical practice.

DESIGN, SETTING, AND PARTICIPANTS

The data is provided by the cancer registry Rhineland-Palatinate gGmbH, Germany. It consists of numerical and categorical features of 1,944 patients with UBC. We retrospectively predict 2-year recurrence through ML models using Support Vector Machine, Gradient Boosting, and Artificial Neural Network. We then determine the global feature importance using performance-based Permutation Feature Importance (PFI) and variance-based Feature Importance Ranking Measure (FIRM).

RESULTS

We show reliable recurrence prediction of UBC with 82.02% to 83.89% F1-Score, 83.95% to 84.49% Precision, and an overall performance of 69.20% to 70.82% AUC on testing data, depending on the model. Gradient Boosting performs best among all black-box models with an average F1-Score (83.89%), AUC (70.82%), and Precision (83.95%). Furthermore, we show consistency across PFI and FIRM by identifying the same features as relevant across the different models. These features are exclusively therapeutic measures and are consistent with findings from both medical research and clinical trials.

CONCLUSIONS

We confirm the superiority of ML black-box models in predicting UBC recurrence compared to more traditional logistic regression. In addition, we present an approach that increases the explanatory power of black-box models by identifying the underlying influence of input features, thus facilitating the use of ML in clinical practice and therefore providing improved recurrence prediction through the application of black-box models.

Validation of causal inference data using DirectLiNGAM in an environmental small-scale model and calculation settings

The development of data science has been needed in environmental fields such as marine, weather, and soil data. In general, the datasets are large in some cases, but they are often small because they contain observation data that the analyses themselves are limited. In such a case, the data are statistically evaluated by increasing or decreasing the levels of factors using differential analysis, resulting in the essential factors are estimated. However, there is no consistent approach to the means of assessing strong associations as a group between factors. Causal inference method has the possibility to output effective results for small data, and the results are expected to provide important information for understanding the potential highly association between factors, not necessarily the inference with big data. Here, we describe essential checkpoints and settings for the calculation by a direct method for learning a linear non-Gaussian structural equation model (DirectLiNGAM) and validation methods for the calculation results by using DirectLiNGAM with small-scale model data as an additional discussion of DirectLiNGAM portion of the related research article. Thus, this study provides the statistical validation methods for the association networks, treatments, and interventions for structural inference as a group of essential factors.•Causal inference with DirectLiNGAM•Validation of correlation coefficient and feature importance•Validation using causal effect object and propensity scores.

Exploring risk factors and their differences on suicidal ideation and suicide attempts among depressed adolescents based on decision tree model

BACKGROUND

Suicide has been recognized as a major global public health issue. Depressed adolescents are more prone to experiencing it. We explore risk factors and their differences on suicidal ideation and suicide attempts to further enhance our understanding of suicidal behavior.

METHODS

2343 depressed adolescents aged 12-18 from 9 provinces/cities in China participated in this cross-sectional study. We utilized decision tree model, incorporating 32 factors encompassing participants' suicidal behavior. The feature importance of each factor was measured using Gini coefficients.

RESULTS

The decision tree model demonstrated a good fit with high accuracy (SI = 0.86, SA = 0.85 and F-Score (SI = 0.85, SA = 0.83). The predictive importance of each factor varied between groups with suicidal ideation and with suicide attempts. The most significant risk factor in both groups was depression (SI = 16.7 %, SA = 19.8 %). However, factors such as academic stress (SI = 7.2 %, SA = 1.6 %), hopelessness (SI = 9.1 %, SA = 5.0 %), and age (SI = 7.1 %, SA = 3.2 %) were more closely associated with suicidal ideation than suicide attempts. Factors related to the schooling status (SI = 3.5 %, SA = 10.1 %), total years of education (SI = 2.6 %, SA = 8.6 %), and loneliness (SI = 2.3 %, SA = 7.4 %) were relatively more important in the suicide attempt stage compared to suicidal ideation.

LIMITATIONS

The cross-sectional design limited the ability to capture changes in suicidal behavior among depressed adolescents over time. Possible bias may exist in the measurement of suicidal ideation.

CONCLUSION

The relative importance of each risk factor for suicidal ideation and attempted suicide varies. These findings provide further empirical evidence for understanding suicide behavior. Targeted treatment measures should be taken for different stages of suicide in clinical interventions.

Using machine learning and partial dependence to evaluate robustness of best linear unbiased prediction (BLUP) for phenotypic values

Best linear unbiased prediction (BLUP) is widely used in plant research to address experimental variation. For phenotypic values, BLUP accuracy is largely dependent on properly controlled experimental repetition and how variable components are outlined in the model. Thus, determining BLUP robustness implies the need to evaluate contributions from each repetition. Here, we assessed the robustness of BLUP values for simulated or empirical phenotypic datasets, where the BLUP value and each experimental repetition served as dependent and independent (feature) variables, respectively. Our technique incorporated machine learning and partial dependence. First, we compared the feature importance estimated with the neural networks. Second, we compared estimated average marginal effects of individual repetitions, calculated with a partial dependence analysis. We showed that contributions of experimental repetitions are unequal in a phenotypic dataset, suggesting that the calculated BLUP value is likely to be influenced by some repetitions more than others (such as failing to detect simulated true positive associations). To resolve disproportionate sources, variable components in the BLUP model must be further outlined.

Explainable AI-based Deep-SHAP for mapping the multivariate relationships between regional neuroimaging biomarkers and cognition

BACKGROUND

Mild cognitive impairment (MCI)/Alzheimer's disease (AD) is associated with cognitive decline beyond normal aging and linked to the alterations of brain volume quantified by magnetic resonance imaging (MRI) and amyloid-beta (Aβ) quantified by positron emission tomography (PET). Yet, the complex relationships between these regional imaging measures and cognition in MCI/AD remain unclear. Explainable artificial intelligence (AI) may uncover such relationships.

METHOD

We integrate the AI-based deep learning neural network and Shapley additive explanations (SHAP) approaches and introduce the Deep-SHAP method to investigate the multivariate relationships between regional imaging measures and cognition. After validating this approach on simulated data, we apply it to real experimental data from MCI/AD patients.

RESULTS

Deep-SHAP significantly predicted cognition using simulated regional features and identified the ground-truth simulated regions as the most significant multivariate predictors. When applied to experimental MRI data, Deep-SHAP revealed that the insula, lateral occipital, medial frontal, temporal pole, and occipital fusiform gyrus are the primary contributors to global cognitive decline in MCI/AD. Furthermore, when applied to experimental amyloid Pittsburgh compound B (PiB)-PET data, Deep-SHAP identified the key brain regions for global cognitive decline in MCI/AD as the inferior temporal, parahippocampal, inferior frontal, supratemporal, and lateral frontal gray matter.

CONCLUSION

Deep-SHAP method uncovered the multivariate relationships between regional brain features and cognition, offering insights into the most critical modality-specific brain regions involved in MCI/AD mechanisms.

Machine learning models for predicting biochar properties from lignocellulosic biomass torrefaction

This study developed six machine learning models to predict the biochar properties from the dry torrefaction of lignocellulosic biomass by using biomass characteristics and torrefaction conditions as input variables. After optimization, gradient boosting machines were the optimal model, with the highest coefficient of determination ranging from 0.89 to 0.94. Torrefaction conditions exhibited a higher relative contribution to the yield and higher heating value (HHV) of biochar than biomass characteristics. Temperature was the dominant contributor to the elemental and proximate composition and the yield and HHV of biochar. Feature importance and SHapley Additive exPlanations revealed the effect of each influential factor on the target variables and the interactions between these factors in torrefaction. Software that can accurately predict the element, yield, and HHV of biochar was developed. These findings provide a comprehensive understanding of the key factors and their interactions influencing the torrefaction process and biochar properties.

Predicting treatment outcome based on resting-state functional connectivity in internalizing mental disorders: A systematic review and meta-analysis

Predicting treatment outcome in internalizing mental disorders prior to treatment initiation is pivotal for precision mental healthcare. In this regard, resting-state functional connectivity (rs-FC) and machine learning have often shown promising prediction accuracies. This systematic review and meta-analysis evaluates these studies, considering their risk of bias through the Prediction Model Study Risk of Bias Assessment Tool (PROBAST). We examined the predictive performance of features derived from rs-FC, identified features with the highest predictive value, and assessed the employed machine learning pipelines. We searched the electronic databases Scopus, PubMed and PsycINFO on the 12th of December 2022, which resulted in 13 included studies. The mean balanced accuracy for predicting treatment outcome was 77% (95% CI: [72%- 83%]). rs-FC of the dorsolateral prefrontal cortex had high predictive value in most studies. However, a high risk of bias was identified in all studies, compromising interpretability. Methodological recommendations are provided based on a comprehensive exploration of the studies' machine learning pipelines, and potential fruitful developments are discussed.

SEGAL time series classification - Stable explanations using a generative model and an adaptive weighting method for LIME

Local Interpretability Model-agnostic Explanations (LIME) is a well-known post-hoc technique for explaining black-box models. While very useful, recent research highlights challenges around the explanations generated. In particular, there is a potential lack of stability, where the explanations provided vary over repeated runs of the algorithm, casting doubt on their reliability. This paper investigates the stability of LIME when applied to multivariate time series classification. We demonstrate that the traditional methods for generating neighbours used in LIME carry a high risk of creating 'fake' neighbours, which are out-of-distribution in respect to the trained model and far away from the input to be explained. This risk is particularly pronounced for time series data because of their substantial temporal dependencies. We discuss how these out-of-distribution neighbours contribute to unstable explanations. Furthermore, LIME weights neighbours based on user-defined hyperparameters which are problem-dependent and hard to tune. We show how unsuitable hyperparameters can impact the stability of explanations. We propose a two-fold approach to address these issues. First, a generative model is employed to approximate the distribution of the training data set, from which within-distribution samples and thus meaningful neighbours can be created for LIME. Second, an adaptive weighting method is designed in which the hyperparameters are easier to tune than those of the traditional method. Experiments on real-world data sets demonstrate the effectiveness of the proposed method in providing more stable explanations using the LIME framework. In addition, in-depth discussions are provided on the reasons behind these results.

A comparative study of 11 non-linear regression models highlighting autoencoder, DBN, and SVR, enhanced by SHAP importance analysis in soybean branching prediction

To explore a robust tool for advancing digital breeding practices through an artificial intelligence-driven phenotype prediction expert system, we undertook a thorough analysis of 11 non-linear regression models. Our investigation specifically emphasized the significance of Support Vector Regression (SVR) and SHapley Additive exPlanations (SHAP) in predicting soybean branching. By using branching data (phenotype) of 1918 soybean accessions and 42 k SNP (Single Nucleotide Polymorphism) polymorphic data (genotype), this study systematically compared 11 non-linear regression AI models, including four deep learning models (DBN (deep belief network) regression, ANN (artificial neural network) regression, Autoencoders regression, and MLP (multilayer perceptron) regression) and seven machine learning models (e.g., SVR (support vector regression), XGBoost (eXtreme Gradient Boosting) regression, Random Forest regression, LightGBM regression, GPs (Gaussian processes) regression, Decision Tree regression, and Polynomial regression). After being evaluated by four valuation metrics: R2 (R-squared), MAE (Mean Absolute Error), MSE (Mean Squared Error), and MAPE (Mean Absolute Percentage Error), it was found that the SVR, Polynomial Regression, DBN, and Autoencoder outperformed other models and could obtain a better prediction accuracy when they were used for phenotype prediction. In the assessment of deep learning approaches, we exemplified the SVR model, conducting analyses on feature importance and gene ontology (GO) enrichment to provide comprehensive support. After comprehensively comparing four feature importance algorithms, no notable distinction was observed in the feature importance ranking scores across the four algorithms, namely Variable Ranking, Permutation, SHAP, and Correlation Matrix, but the SHAP value could provide rich information on genes with negative contributions, and SHAP importance was chosen for feature selection. The results of this study offer valuable insights into AI-mediated plant breeding, addressing challenges faced by traditional breeding programs. The method developed has broad applicability in phenotype prediction, minor QTL (quantitative trait loci) mining, and plant smart-breeding systems, contributing significantly to the advancement of AI-based breeding practices and transitioning from experience-based to data-based breeding.

SHAP value-based ERP analysis (SHERPA): Increasing the sensitivity of EEG signals with explainable AI methods

Conventionally, event-related potential (ERP) analysis relies on the researcher to identify the sensors and time points where an effect is expected. However, this approach is prone to bias and may limit the ability to detect unexpected effects or to investigate the full range of the electroencephalography (EEG) signal. Data-driven approaches circumvent this limitation, however, the multiple comparison problem and the statistical correction thereof affect both the sensitivity and specificity of the analysis. In this study, we present SHERPA - a novel approach based on explainable artificial intelligence (XAI) designed to provide the researcher with a straightforward and objective method to find relevant latency ranges and electrodes. SHERPA is comprised of a convolutional neural network (CNN) for classifying the conditions of the experiment and SHapley Additive exPlanations (SHAP) as a post hoc explainer to identify the important temporal and spatial features. A classical EEG face perception experiment is employed to validate the approach by comparing it to the established researcher- and data-driven approaches. Likewise, SHERPA identified an occipital cluster close to the temporal coordinates for the N170 effect expected. Most importantly, SHERPA allows quantifying the relevance of an ERP for a psychological mechanism by calculating an "importance score". Hence, SHERPA suggests the presence of a negative selection process at the early and later stages of processing. In conclusion, our new method not only offers an analysis approach suitable in situations with limited prior knowledge of the effect in question but also an increased sensitivity capable of distinguishing neural processes with high precision.

Computer-Aided Diagnosis of Complications After Liver Transplantation Based on Transfer Learning

Liver transplantation is one of the most effective treatments for acute liver failure, cirrhosis, and even liver cancer. The prediction of postoperative complications is of great significance for liver transplantation. However, the existing prediction methods based on traditional machine learning are often unavailable or unreliable due to the insufficient amount of real liver transplantation data. Therefore, we propose a new framework to increase the accuracy of computer-aided diagnosis of complications after liver transplantation with transfer learning, which can handle small-scale but high-dimensional data problems. Furthermore, since data samples are often high dimensional in the real world, capturing key features that influence postoperative complications can help make the correct diagnosis for patients. So, we also introduce the SHapley Additive exPlanation (SHAP) method into our framework for exploring the key features of postoperative complications. We used data obtained from 425 patients with 456 features in our experiments. Experimental results show that our approach outperforms all compared baseline methods in predicting postoperative complications. In our work, the average precision, the mean recall, and the mean F1 score reach 91.22%, 91.70%, and 91.18%, respectively.

Predicting effluent quality parameters for wastewater treatment plant: A machine learning-based methodology

Wastewater Treatment Plants (WWTPs) present complex biochemical processes of high variability and difficult prediction. This study presents an innovative approach using Machine Learning (ML) models to predict wastewater quality parameters. In particular, the models are applied to datasets from both a simulated wastewater treatment plant (WWTP), using DHI WEST software (WEST WWTP), and a real-world WWTP database from Santa Catarina Brewery AMBEV, located in Lages/SC - Brazil (AMBEV WWTP). A distinctive aspect is the evaluation of predictive performance in continuous data scenarios and the impact of changes in WWTP operations on predictive model performance, including changes in plant layout. For both plants, three different scenarios were addressed, and the quality of predictions by random forest (RF), support vector machine (SVM), and multilayer perceptron (MLP) models were evaluated. The prediction quality by the MLP model reached an R2 of 0.72 for TN prediction in the WEST WWTP output, and the RF model better adapted to the real data of the AMBEV WWTP, despite the significant discrepancy observed between the real and the predicted data. Techniques such as Partial Dependence Plots (PDP) and Permutation Importance (PI) were used to assess the importance of features, particularly in the simulated WEST tool scenario, showing a strong correlation of prediction results with influent parameters related to nitrogen content. The results of this study highlight the importance of collecting and storing high-quality data and the need for information on changes in WWTP operation for predictive model performance. These contributions advance the understanding of predictive modeling for wastewater quality and provide valuable insights for future practice in wastewater treatment.

An interpretable clustering approach to safety climate analysis: Examining driver group distinctions

The transportation industry, particularly the trucking sector, is prone to workplace accidents and fatalities. Accidents involving large trucks accounted for a considerable percentage of overall traffic fatalities. Recognizing the crucial role of safety climate in accident prevention, researchers have sought to understand its factors and measure its impact within organizations. While existing data-driven safety climate studies have made remarkable progress, clustering employees based on their safety climate perception is innovative and has not been extensively utilized in research. Identifying clusters of drivers based on their safety climate perception allows the organization to profile its workforce and devise more impactful interventions. The lack of utilizing the clustering approach could be due to difficulties interpreting or explaining the factors influencing employees' cluster membership. Moreover, existing safety-related studies did not compare multiple clustering algorithms, resulting in potential bias. To address these problems, this study introduces an interpretable clustering approach for safety climate analysis. This study compares five algorithms for clustering truck drivers based on their safety climate perceptions. It also proposes a novel method for quantitatively evaluating partial dependence plots (QPDP). Then, to better interpret the clustering results, this study introduces different interpretable machine learning measures (Shapley additive explanations, permutation feature importance, and QPDP). The Python code used in this study is available at https://github.com/NUS-DBE/truck-driver-safety-climate. This study explains the clusters based on the importance of different safety climate factors. Drawing on data collected from more than 7,000 American truck drivers, this study significantly contributes to the scientific literature. It highlights the critical role of supervisory care promotion in distinguishing various driver groups. Moreover, it showcases the advantages of employing machine learning techniques, such as cluster analysis, to enrich the scientific knowledge in this field. Future studies could involve experimental methods to assess strategies for enhancing supervisory care promotion, as well as integrating deep learning clustering techniques with safety climate evaluation.

Explainable machine learning model based on clinical factors for predicting the disappearance of indeterminate pulmonary nodules

BACKGROUND

During lung cancer screening, indeterminate pulmonary nodules (IPNs) are a frequent finding. We aim to predict whether IPNs are resolving or non-resolving to reduce follow-up examinations, using machine learning (ML) models. We incorporated dedicated techniques to enhance prediction explainability.

METHODS

In total, 724 IPNs (size 50-500 mm3, 575 participants) from the Dutch-Belgian Randomized Lung Cancer Screening Trial were used. We implemented six ML models and 14 factors to predict nodule disappearance. Random search was applied to determine the optimal hyperparameters on the training set (579 nodules). ML models were trained using 5-fold cross-validation and tested on the test set (145 nodules). Model predictions were evaluated by utilizing the recall, precision, F1 score, and the area under the receiver operating characteristic curve (AUC). The best-performing model was used for three feature importance techniques: mean decrease in impurity (MDI), permutation feature importance (PFI), and SHAPley Additive exPlanations (SHAP).

RESULTS

The random forest model outperformed the other ML models with an AUC of 0.865. This model achieved a recall of 0.646, a precision of 0.816, and an F1 score of 0.721. The evaluation of feature importance achieved consistent ranking across all three methods for the most crucial factors. The MDI, PFI, and SHAP methods highlighted volume, maximum diameter, and minimum diameter as the top three factors. However, the remaining factors revealed discrepant ranking across methods.

CONCLUSION

ML models effectively predict IPN disappearance using participant demographics and nodule characteristics. Explainable techniques can assist clinicians in developing understandable preliminary assessments.

InterDILI: interpretable prediction of drug-induced liver injury through permutation feature importance and attention mechanism

Safety is one of the important factors constraining the distribution of clinical drugs on the market. Drug-induced liver injury (DILI) is the leading cause of safety problems produced by drug side effects. Therefore, the DILI risk of approved drugs and potential drug candidates should be assessed. Currently, in vivo and in vitro methods are used to test DILI risk, but both methods are labor-intensive, time-consuming, and expensive. To overcome these problems, many in silico methods for DILI prediction have been suggested. Previous studies have shown that DILI prediction models can be utilized as prescreening tools, and they achieved a good performance. However, there are still limitations in interpreting the prediction results. Therefore, this study focused on interpreting the model prediction to analyze which features could potentially cause DILI. For this, five publicly available datasets were collected to train and test the model. Then, various machine learning methods were applied using substructure and physicochemical descriptors as inputs and the DILI label as the output. The interpretation of feature importance was analyzed by recognizing the following general-to-specific patterns: (i) identifying general important features of the overall DILI predictions, and (ii) highlighting specific molecular substructures which were highly related to the DILI prediction for each compound. The results indicated that the model not only captured the previously known properties to be related to DILI but also proposed a new DILI potential substructural of physicochemical properties. The models for the DILI prediction achieved an area under the receiver operating characteristic (AUROC) of 0.88-0.97 and an area under the Precision-Recall curve (AUPRC) of 0.81-0.95. From this, we hope the proposed models can help identify the potential DILI risk of drug candidates at an early stage and offer valuable insights for drug development.

Techniques for Developing Reliable Machine Learning Classifiers Applied to Understanding and Predicting Protein:Protein Interaction Hot Spots

With machine learning now transforming the sciences, successful prediction of biological structure or activity is mainly limited by the extent and quality of data available for training, the astute choice of features for prediction, and thorough assessment of the robustness of prediction on a variety of new cases. In this chapter, we address these issues while developing and sharing protocols to build a robust dataset and rigorously compare several predictive classifiers using the open-source Python machine learning library, scikit-learn. We show how to evaluate whether enough data has been used for training and whether the classifier has been overfit to training data. The most telling experiment is 500-fold repartitioning of the training and test sets, followed by prediction, which gives a good indication of whether a classifier performs consistently well on different datasets. An intuitive method is used to quantify which features are most important for correct prediction.The resulting well-trained classifier, hotspotter, can robustly predict the small subset of amino acid residues on the surface of a protein that are energetically most important for binding a protein partner: the interaction hot spots. Hotspotter has been trained and tested here on a curated dataset assembled from 1046 non-redundant alanine scanning mutation sites with experimentally measured change in binding free energy values from 97 different protein complexes; this dataset is available to download. The accessible surface area of the wild-type residue at a given site and its degree of evolutionary conservation proved the most important features to identify hot spots. A variant classifier was trained and validated for proteins where only the amino acid sequence is available, augmented by secondary structure assignment. This version of hotspotter requiring fewer features is almost as robust as the structure-based classifier. Application to the ACE2 (angiotensin converting enzyme 2) receptor, which mediates COVID-19 virus entry into human cells, identified the critical hot spot triad of ACE2 residues at the center of the small interface with the CoV-2 spike protein. Hotspotter results can be used to guide the strategic design of protein interfaces and ligands and also to identify likely interfacial residues for protein:protein docking.

MS-CPFI: A model-agnostic Counterfactual Perturbation Feature Importance algorithm for interpreting black-box Multi-State models

Multi-state processes (Webster, 2019) are commonly used to model the complex clinical evolution of diseases where patients progress through different states. In recent years, machine learning and deep learning algorithms have been proposed to improve the accuracy of these models' predictions (Wang et al., 2019). However, acceptability by patients and clinicians, as well as for regulatory compliance, require interpretability of these algorithms's predictions. Existing methods, such as the Permutation Feature Importance algorithm, have been adapted for interpreting predictions in black-box models for 2-state processes (corresponding to survival analysis). For generalizing these methods to multi-state models, we introduce a novel model-agnostic interpretability algorithm called Multi-State Counterfactual Perturbation Feature Importance (MS-CPFI) that computes feature importance scores for each transition of a general multi-state model, including survival, competing-risks, and illness-death models. MS-CPFI uses a new counterfactual perturbation method that allows interpreting feature effects while capturing the non-linear effects and potentially capturing time-dependent effects. Experimental results on simulations show that MS-CPFI increases model interpretability in the case of non-linear effects. Additionally, results on a real-world dataset for patients with breast cancer confirm that MS-CPFI can detect clinically important features and provide information on the disease progression by displaying features that are protective factors versus features that are risk factors for each stage of the disease. Overall, MS-CPFI is a promising model-agnostic interpretability algorithm for multi-state models, which can improve the interpretability of machine learning and deep learning algorithms in healthcare.

Development and validation of a web-based artificial intelligence prediction model to assess massive intraoperative blood loss for metastatic spinal disease using machine learning techniques

BACKGROUND CONTEXT

Intraoperative blood loss is a significant concern in patients with metastatic spinal disease. Early identification of patients at high risk of experiencing massive intraoperative blood loss is crucial as it allows for the development of appropriate surgical plans and facilitates timely interventions. However, accurate prediction of intraoperative blood loss remains limited based on prior studies.

PURPOSE

The purpose of this study was to develop and validate a web-based artificial intelligence (AI) model to predict massive intraoperative blood loss during surgery for metastatic spinal disease.

STUDY DESIGN/SETTING

An observational cohort study.

PATIENT SAMPLE

Two hundred seventy-six patients with metastatic spinal tumors undergoing decompressive surgery from two hospitals were included for analysis. Of these, 200 patients were assigned to the derivation cohort for model development and internal validation, while the remaining 76 were allocated to the external validation cohort.

OUTCOME MEASURES

The primary outcome was massive intraoperative blood loss defined as an estimated blood loss of 2,500 cc or more.

METHODS

Data on patients' demographics, tumor conditions, oncological therapies, surgical strategies, and laboratory examinations were collected in the derivation cohort. SMOTETomek resampling (which is a combination of Synthetic Minority Oversampling Technique and Tomek Links Undersampling) was performed to balance the classes of the dataset and obtain an expanded dataset. The patients were randomly divided into two groups in a proportion of 7:3, with the most used for model development and the remaining for internal validation. External validation was performed in another cohort of 76 patients with metastatic spinal tumors undergoing decompressive surgery from a teaching hospital. The logistic regression (LR) model, and five machine learning models, including K-Nearest Neighbor (KNN), Decision Tree (DT), XGBoosting Machine (XGBM), Random Forest (RF), and Support Vector Machine (SVM), were used to develop prediction models. Model prediction performance was evaluated using area under the curve (AUC), recall, specificity, F1 score, Brier score, and log loss. A scoring system incorporating 10 evaluation metrics was developed to comprehensively evaluate the prediction performance.

RESULTS

The incidence of massive intraoperative blood loss was 23.50% (47/200). The model features were comprised of five clinical variables, including tumor type, smoking status, Eastern Cooperative Oncology Group (ECOG) score, surgical process, and preoperative platelet level. The XGBM model performed the best in AUC (0.857 [95% CI: 0.827, 0.877]), accuracy (0.771), recall (0.854), F1 score (0.787), Brier score (0.150), and log loss (0.461), and the RF model ranked second in AUC (0.826 [95% CI: 0.793, 0.861]) and precise (0.705), whereas the AUC of the LR model was only 0.710 (95% CI: 0.665, 0.771), the accuracy was 0.627, the recall was 0.610, and the F1 score was 0.617. According to the scoring system, the XGBM model obtained the highest total score of 55, which signifies the best predictive performance among the evaluated models. External validation showed that the AUC of the XGBM model was also up to 0.809 (95% CI: 0.778, 0.860) and the accuracy was 0.733. The XGBM model, was further deployed online, and can be freely accessed at https://starxueshu-massivebloodloss-main-iudy71.streamlit.app/.

CONCLUSIONS

The XGBM model may be a useful AI tool to assess the risk of intraoperative blood loss in patients with metastatic spinal disease undergoing decompressive surgery.

Predicting cognitive dysfunction and regional hubs using Braak staging amyloid-beta biomarkers and machine learning

Mild cognitive impairment (MCI) is a transitional stage between normal aging and early Alzheimer's disease (AD). The presence of extracellular amyloid-beta (Aβ) in Braak regions suggests a connection with cognitive dysfunction in MCI/AD. Investigating the multivariate predictive relationships between regional Aβ biomarkers and cognitive function can aid in the early detection and prevention of AD. We introduced machine learning approaches to estimate cognitive dysfunction from regional Aβ biomarkers and identify the Aβ-related dominant brain regions involved with cognitive impairment. We employed Aβ biomarkers and cognitive measurements from the same individuals to train support vector regression (SVR) and artificial neural network (ANN) models and predict cognitive performance solely based on Aβ biomarkers on the test set. To identify Aβ-related dominant brain regions involved in cognitive prediction, we built the local interpretable model-agnostic explanations (LIME) model. We found elevated Aβ in MCI compared to controls and a stronger correlation between Aβ and cognition, particularly in Braak stages III-IV and V-VII (p < 0.05) biomarkers. Both SVR and ANN, especially ANN, showed strong predictive relationships between regional Aβ biomarkers and cognitive impairment (p < 0.05). LIME integrated with ANN showed that the parahippocampal gyrus, inferior temporal gyrus, and hippocampus were the most decisive Braak regions for predicting cognitive decline. Consistent with previous findings, this new approach suggests relationships between Aβ biomarkers and cognitive impairment. The proposed analytical framework can estimate cognitive impairment from Braak staging Aβ biomarkers and delineate the dominant brain regions collectively involved in AD pathophysiology.

Hybrid visualization-based framework for depressive state detection and characterization of atypical patients

INTRODUCTION

Depression is a global concern, with a significant number of people affected worldwide, particularly in low- and middle-income countries. The rising prevalence of depression emphasizes the importance of early detection and understanding the origins of such conditions.

OBJECTIVE

This paper proposes a framework for detecting depression using a hybrid visualization approach that combines local and global interpretation. This approach aims to assist in model adaptation, provide insights into patient characteristics, and evaluate prediction model suitability in a different environment.

METHODS

This study utilizes R programming language with the Caret, ggplot2, Plotly, and Dalex libraries for model training, visualization, and interpretation. Data from the NHANES repository was used for secondary data analysis. The NHANES repository is a comprehensive source for examining health and nutrition of individuals in the United States, and covers demographic, dietary, medication use, lifestyle choices, reproductive and mental health data. Penalized logistic regression models were built using NHANES 2015-2018 data, while NHANES 2019-March 2020 data was used for evaluation at the global-specific and local level interpretation.

RESULTS

The prediction model that supports this framework achieved an average AUC score of 0.748 (95% CI: 0.743-0.752), with minimal variability in sensitivity and specificity.

CONCLUSION

The built-in prediction model highlights chest pain, the ratio of family income to poverty, and smoking status as crucial features for predicting depressive states in both the original and local environments.

Single-label machine learning classification revealed some hidden but inter-related causes of five psychotic disorder diseases

Psychotic disorder diseases (PDD) or mental illnesses are group of illnesses that affect the minds and impair the cognitive ability, retard emotional ability and obstruct the process of communication and relationship with others and are characterized by delusions, hallucinations and disoriented or disordered pattern of thinking. Prognosis of PDD is not sufficient because of the nature of the diseases and as such adequate form of diagnosis is required to detect, manage and treat the illness. This paper applied the single-label classification (SLC) machine learning approach in mining of electronic health records of people with PDD in Nigeria using eleven independent (demographic) variables and five PDD as target variables. The five PDDs are Insomnia, Schizophrenia, Minimal Brain dysfunction (MBD), which is also known as Attention-Deficit/Hyperactivity Disorder (ADHD), Vascular Dementia (VD) and Bipolar Disorder (BD). The aim of using SLC is that it would be easier to detect some PDDs that are related to each other without the loss of information, which is a plus over multi-label classification (MLC). ReliefF algorithm was used at each experiment to precipitate the order of importance of the independent variables and redundant variables were excluded from the analysis. The order of the variables in feature selection was matched with feature importance after the classifications and quantified using the Spearman rank correlation coefficient. The data was divided into: 70% for training and 30% for testing. Four new performance metrics adapted from the root mean square (RMSE) were proposed and used to measure the differences between the performance results of the 10 Machine learning models in terms of the training and testing and secondly, feature and without feature selection. The new metrics are close to zero which is an indication that the use of feature selection and cross validation may not greatly affects the accuracy of the SLC. When the PDDs are included as predictors for classifying others, there was a tremendous improvement as revealed by the four new metrics for classification accuracy (CA), precision and recall. Analysis of variance showed the four different metrics differs significantly for classification accuracy (CA) and precision. However, there were no significant difference between the CA and precision when the duo are compared together across the four evaluation metrics at p value less than 0.05.

Survival Prediction After Transarterial Chemoembolization for Hepatocellular Carcinoma: a Deep Multitask Survival Analysis Approach

The accurate prediction of postoperative survival time of patients with Barcelona Clinic Liver Cancer (BCLC) stage B hepatocellular carcinoma (HCC) is important for postoperative health care. Survival analysis is a common method used to predict the occurrence time of events of interest in the medical field. At present, the mainstream survival analysis models, such as the Cox proportional risk model, should make strict assumptions about the potential random process to solve the censored data, thus potentially limiting their application in clinical practice. In this paper, we propose a novel deep multitask survival model (DMSM) to analyze HCC survival data. Specifically, DMSM transforms the traditional survival time prediction problem of patients with HCC into a survival probability prediction problem at multiple time points and applies entropy regularization and ranking loss to optimize a multitask neural network. Compared with the traditional methods of deleting censored data and strong hypothesis, DMSM makes full use of all the information in the censored data but does not need to make any assumption. In addition, we identify the risk factors affecting the prognosis of patients with HCC and visualize the importance of ranking these factors. On the basis of the analysis of a real dataset of patients with BCLC stage B HCC, experimental results on three different validation datasets show that the DMSM achieves competitive performance with concordance index of 0.779, 0.727, and 0.780 and integrated Brier score (IBS) of 0.172, 0.138, and 0.135, respectively. Our DMSM has a comparatively small standard deviation (0.002, 0.002, and 0.003) for IBS of bootstrapping 100 times. The DMSM we proposed can be utilized as an effective survival analysis model and provide an important means for the accurate prediction of postoperative survival time of patients with BCLC stage B HCC.

Using machine learning-based binary classifiers for predicting organizational members' user satisfaction with collaboration software

Background

In today's digital economy, enterprises are adopting collaboration software to facilitate digital transformation. However, if employees are not satisfied with the collaboration software, it can hinder enterprises from achieving the expected benefits. Although existing literature has contributed to user satisfaction after the introduction of collaboration software, there are gaps in predicting user satisfaction before its implementation. To address this gap, this study offers a machine learning-based forecasting method.

Methods

We utilized national public data provided by the national information society agency of South Korea. To enable the data to be used in a machine learning-based binary classifier, we discretized the predictor variable. We then validated the effectiveness of our prediction model by calculating feature importance scores and prediction accuracy.

Results

We identified 10 key factors that can predict user satisfaction. Furthermore, our analysis indicated that the naive Bayes (NB) classifier achieved the highest prediction accuracy rate of 0.780, followed by logistic regression (LR) at 0.767, extreme gradient boosting (XGBoost) at 0.744, support vector machine (SVM) at 0.744, K-nearest neighbor (KNN) at 0.707, and decision tree (DT) at 0.637.

Conclusions

This research identifies essential indicators that can predict user satisfaction with collaboration software across four levels: institutional guidance, information and communication technology (ICT) environment, company culture, and demographics. Enterprises can use this information to evaluate their current collaboration status and develop strategies for introducing collaboration software. Furthermore, this study presents a novel approach to predicting user satisfaction and confirm the effectiveness of the machine learning-based prediction method proposed in this study, adding to the existing knowledge on the subject.

Quantitative prediction of the hydraulic performance of free water surface constructed wetlands by integrating numerical simulation and machine learning

Quantitative prediction of the design parameter-influenced hydraulic performance is significant for optimizing free water surface constructed wetlands (FWS CWs) to reduce point and non-point source pollution and improve land utilization. However, owing to limitations of the test conditions and data scale, a quantitative prediction model of the hydraulic performance under multiple design parameters has not yet been established. In this study, we integrated field test data, mechanism model, statistical regression, and machine learning (ML) to construct such quantitative prediction models. A FWS CW numerical model was established by integrating 13 groups of trace data from field tests. Subsequently, training, test and extension datasets comprising 125 (5^3), 25 (L₂₅(5⁶)) and 16 (L₁₆(4⁴)) data points, respectively, were generated via numerical simulation of multi-level value combination of three quantitative design parameters, namely, water depth, hydraulic loading rate (HLR), and aspect ratio. The short circuit index (φ₁₀), Morrill dispersion index (MDI), hydraulic efficiency (λ) and moment index (MI) were used as representative hydraulic performance indicators. Training set with large samples were analyzed to determine the variation rules of different hydraulic indicators. Based on the control variable method, φ₁₀, λ, and MI grew exponentially with increasing aspect ratio whereas MDI showed a decreasing trend; with increasing water depth, φ₁₀, λ, and MI showed polynomial decreases whereas MDI increased; with increasing HLR, φ₁₀, λ, and MI slowly increased linearly whereas MDI showed the opposite trend. Finally, we constructed models based on multivariate nonlinear regression (MNLR) and ML (random forest (RF), multilayer perceptron (MLP), and support vector regression. The coefficients of determination (R²) of the MNLR and ML models fitting the training and test sets were all greater than 0.9; however, the generalization abilities of different models in the extension set were different. The most robust MLP, MNLR without interaction term, and RF models were recommended as the preferred models to hydraulic performance prediction. The extreme importance of aspect ratio in hydraulic performance was revealed. Thus, gaps in the current understanding of multivariate quantitative prediction of the hydraulic performance of FWS CWs are addressed while providing an avenue for researching FWS CWs in different regions according to local conditions.

Use and misuse of random forest variable importance metrics in medicine: demonstrations through incident stroke prediction

BACKGROUND

Machine learning tools such as random forests provide important opportunities for modeling large, complex modern data generated in medicine. Unfortunately, when it comes to understanding why machine learning models are predictive, applied research continues to rely on 'out of bag' (OOB) variable importance metrics (VIMPs) that are known to have considerable shortcomings within the statistics community. After explaining the limitations of OOB VIMPs - including bias towards correlated features and limited interpretability - we describe a modern approach called 'knockoff VIMPs' and explain its advantages.

METHODS

We first evaluate current VIMP practices through an in-depth literature review of 50 recent random forest manuscripts. Next, we recommend organized and interpretable strategies for analysis with knockoff VIMPs, including computing them for groups of features and considering multiple model performance metrics. To demonstrate methods, we develop a random forest to predict 5-year incident stroke in the Sleep Heart Health Study and compare results based on OOB and knockoff VIMPs.

RESULTS

Nearly all papers in the literature review contained substantial limitations in their use of VIMPs. In our demonstration, using OOB VIMPs for individual variables suggested two highly correlated lung function variables (forced expiratory volume, forced vital capacity) as the best predictors of incident stroke, followed by age and height. Using an organized analytic approach that considered knockoff VIMPs of both groups of features and individual features, the largest contributions to model sensitivity were medications (especially cardiovascular) and measured medical risk factors, while the largest contributions to model specificity were age, diastolic blood pressure, self-reported medical risk factors, polysomnography features, and pack-years of smoking. Thus, we reach very different conclusions about stroke risk factors using OOB VIMPs versus knockoff VIMPs.

CONCLUSIONS

The near-ubiquitous reliance on OOB VIMPs may provide misleading results for researchers who use such methods to guide their research. Given the rapid pace of scientific inquiry using machine learning, it is essential to bring modern knockoff VIMPs that are interpretable and unbiased into widespread applied practice to steer researchers using random forest machine learning toward more meaningful results.

A hybrid model for estimating the number concentration of ultrafine particles based on machine learning algorithms in central Taiwan

Modeling is a cost-effective measure to estimate ultrafine particle (UFP) levels. Previous UFP estimates generally relied on land-use regression with insufficient temporal resolution. We carried out in-situ measurements for UFP in central Taiwan and developed a model incorporating satellite-based measurements, meteorological variables, and land-use data to estimate daily UFP levels at a 1-km resolution. Two sampling campaigns were conducted for measuring hourly UFP concentrations at six sites between 2008-2010 and 2017-2021, respectively, using scanning mobility particle sizers. Three machine learning algorithms, namely random forest, eXtreme gradient boosting (XGBoost), and deep neural network, were used to develop UFP estimation models. The performances were evaluated with a 10-fold cross-validation, temporal, and spatial validation. A total of 1,022 effective sampling days were conducted. The XGBoost model had the best performance with a training coefficient of determination (R²) of 0.99 [normalized root mean square error (nRMSE): 6.52%] and a cross-validation R² of 0.78 (nRMSE: 31.0%). The ten most important variables were surface pressure, distance to the nearest road, temperature, calendar year, day of the year, NO₂, meridional wind, the total length of roads, PM_2.5, and zonal wind. The UFP levels were elevated along the main roads across different seasons, suggesting that traffic emission is an important contributor to UFP. This hybrid model outperformed prior land use regression models and thus can provide more accurate estimates of UFP for epidemiological studies.

From local counterfactuals to global feature importance: efficient, robust, and model-agnostic explanations for brain connectivity networks

BACKGROUND

Explainable artificial intelligence (XAI) is a technology that can enhance trust in mental state classifications by providing explanations for the reasoning behind artificial intelligence (AI) models outputs, especially for high-dimensional and highly-correlated brain signals. Feature importance and counterfactual explanations are two common approaches to generate these explanations, but both have drawbacks. While feature importance methods, such as shapley additive explanations (SHAP), can be computationally expensive and sensitive to feature correlation, counterfactual explanations only explain a single outcome instead of the entire model.

METHODS

To overcome these limitations, we propose a new procedure for computing global feature importance that involves aggregating local counterfactual explanations. This approach is specifically tailored to fMRI signals and is based on the hypothesis that instances close to the decision boundary and their counterfactuals mainly differ in the features identified as most important for the downstream classification task. We refer to this proposed feature importance measure as Boundary Crossing Solo Ratio (BoCSoR), since it quantifies the frequency with which a change in each feature in isolation leads to a change in classification outcome, i.e., the crossing of the model's decision boundary.

RESULTS AND CONCLUSIONS

Experimental results on synthetic data and real publicly available fMRI data from the Human Connect project show that the proposed BoCSoR measure is more robust to feature correlation and less computationally expensive than state-of-the-art methods. Additionally, it is equally effective in providing an explanation for the behavior of any AI model for brain signals. These properties are crucial for medical decision support systems, where many different features are often extracted from the same physiological measures and a gold standard is absent. Consequently, computing feature importance may become computationally expensive, and there may be a high probability of mutual correlation among features, leading to unreliable results from state-of-the-art XAI methods.

A deep neural network framework to derive interpretable decision rules for accurate traumatic brain injury identification of infants

OBJECTIVE

We aimed to develop a robust framework to model the complex association between clinical features and traumatic brain injury (TBI) risk in children under age two, and identify significant features to derive clinical decision rules for triage decisions.

METHODS

In this retrospective study, four frequently used machine learning models, i.e., support vector machine (SVM), random forest (RF), deep neural network (DNN), and XGBoost (XGB), were compared to identify significant clinical features from 24 input features associated with the TBI risk in children under age two under the permutation feature importance test (PermFIT) framework by using the publicly available data set from the Pediatric Emergency Care Applied Research Network (PECARN) study. The prediction accuracy was determined by comparing the predicted TBI status with the computed tomography (CT) scan results since CT scan is the gold standard for diagnosing TBI.

RESULTS

At a significance level of [Formula: see text], DNN, RF, XGB, and SVM identified 9, 1, 2,  and 4 significant features, respectively. In a comparison of accuracy (Accuracy), the area under the curve (AUC), and the precision-recall area under the curve (PR-AUC), the permutation feature importance test for DNN model was the most powerful framework for identifying significant features and outperformed other methods, i.e., RF, XGB, and SVM, with Accuracy, AUC, and PR-AUC as 0.915, 0.794, and 0.974, respectively.

CONCLUSION

These results indicate that the PermFIT-DNN framework robustly identifies significant clinical features associated with TBI status and improves prediction performance. The findings could be used to inform the development of clinical decision tools designed to inform triage decisions.

Using machine learning models to predict the effects of seasonal fluxes on Plesiomonas shigelloides population density

Seasonal variations (SVs) affect the population density (PD), fate, and fitness of pathogens in environmental water resources and the public health impacts. Therefore, this study is aimed at applying machine learning intelligence (MLI) to predict the impacts of SVs on P. shigelloides population density (PDP) in the aquatic milieu. Physicochemical events (PEs) and PDP from three rivers acquired via standard microbiological and instrumental techniques across seasons were fitted to MLI algorithms (linear regression (LR), multiple linear regression (MR), random forest (RF), gradient boosted machine (GBM), neural network (NN), K-nearest neighbour (KNN), boosted regression tree (BRT), extreme gradient boosting (XGB) regression, support vector regression (SVR), decision tree regression (DTR), M5 pruned regression (M5P), artificial neural network (ANN) regression (with one 10-node hidden layer (ANN10), two 6- and 4-node hidden layers (ANN64), and two 5- and 5-node hidden layers (ANN55)), and elastic net regression (ENR)) to assess the implications of the SVs of PEs on aquatic PDP. The results showed that SVs significantly influenced PDP and PEs in the water (p < 0.0001), exhibiting a site-specific pattern. While MLI algorithms predicted PDP with differing absolute flux magnitudes for the contributing variables, DTR predicted the highest PDP value of 1.707 log unit, followed by XGB (1.637 log unit), but XGB (mean-squared-error (MSE) = 0.0025; root-mean-squared-error (RMSE) = 0.0501; R² =0.998; medium absolute deviation (MAD) = 0.0275) outperformed other models in terms of regression metrics. Temperature and total suspended solids (TSS) ranked first and second as significant factors in predicting PDP in 53.3% (8/15) and 40% (6/15), respectively, of the models, based on the RMSE loss after permutations. Additionally, season ranked third among the 7 models, and turbidity (TBS) ranked fourth at 26.7% (4/15), as the primary significant factor for predicting PDP in the aquatic milieu. The results of this investigation demonstrated that MLI predictive modelling techniques can promisingly be exploited to complement the repetitive laboratory-based monitoring of PDP and other pathogens, especially in low-resource settings, in response to seasonal fluxes and can provide insights into the potential public health risks of emerging pathogens and TSS pollution (e.g., nanoparticles and micro- and nanoplastics) in the aquatic milieu. The model outputs provide low-cost and effective early warning information to assist watershed managers and fish farmers in making appropriate decisions about water resource protection, aquaculture management, and sustainable public health protection.

Integrated radiomics, dose-volume histogram criteria and clinical features for early prediction of saliva amount reduction after radiotherapy in nasopharyngeal cancer patients

PURPOSE

Previously, the evaluation of xerostomia depended on subjective grading systems, rather than the accurate saliva amount reduction. Our aim was to quantify acute xerostomia with reduced saliva amount, and apply radiomics, dose-volume histogram (DVH) criteria and clinical features to predict saliva amount reduction by machine learning techniques.

MATERIAL AND METHODS

Computed tomography (CT) of parotid glands, DVH, and clinical data of 52 patients were collected to extract radiomics, DVH criteria and clinical features, respectively. Firstly, radiomics, DVH criteria and clinical features were divided into 3 groups for feature selection, in order to alleviate the masking effect of the number of features in different groups. Secondly, the top features in the 3 groups composed integrated features, and features selection was performed again for integrated features. In this study, feature selection was used as a combination of eXtreme Gradient Boosting (XGBoost) and SHapley Additive exPlanations (SHAP) to alleviate multicollinearity. Finally, 6 machine learning techniques were used for predicting saliva amount reduction. Meanwhile, top radiomics features were modeled using the same machine learning techniques for comparison.

RESULT

17 integrated features (10 radiomics, 4 clinical, 3 DVH criteria) were selected to predict saliva amount reduction, with a mean square error (MSE) of 0.6994 and a R² score of 0.9815. Top 17 and 10 selected radiomics features predicted saliva amount reduction, with MSE of 0.7376, 0.7519, and R² score of 0.9805, 0.9801, respectively.

CONCLUSION

With the same number of features, integrated features (radiomics + DVH criteria + clinical) performed better than radiomics features alone. The important DVH criteria and clinical features mainly included, white blood cells (WBC), parotid_glands_Dmax, Age, parotid_glands_V15, hemoglobin (Hb), BMI and parotid_glands_V45.

Identifying the acute toxicity of contaminated sediments using machine learning models

Ecological risk assessment of contaminated sediment has become a fundamental component of water quality management programs, supporting decision-making for management actions or prompting additional investigations. In this study, we proposed a machine learning (ML)-based approach to assess the ecological risk of contaminated sediment as an alternative to existing index-based methods and costly toxicity testing. The performance of three widely used index-based methods (the pollution load index, potential ecological risk index, and mean probable effect concentration) and three ML algorithms (random forest, support vector machine, and extreme gradient boosting [XGB]) were compared in their prediction of sediment toxicity using 327 nationwide data sets from Korea consisting of 14 sediment quality parameters and sediment toxicity testing data. We also compared the performances of classifiers and regressors in predicting the toxicity for each of RF, SVM, and XGB algorithms. For all algorithms, the classifiers poorly classified toxic and non-toxic samples due to limited information on the sediment composition and the small training dataset. The regressors with a given classification threshold provided better classification, with the XGB regressor outperforming the other models in the classification. A permutation feature importance analysis revealed that Cr, Cu, Pb, and Zn were major contributors to toxicity prediction. The ML-based approach has the potential to be even more useful in the future with the expected increase in available sediment data.

Machine learning predictions of chlorophyll-a in the Han river basin, Korea

This study developed a model to predict concentrations of chlorophyll-a ([Chl-a]) as a proxy for algal population with data from multiple monitoring stations in the Han river basin, by using machine-learning predictive models, then analyzed the relationship between [Chl-a] and the input variables of the optimized model. Daily water quality and meteorological data from 2012 to 2020 were collected from the real-time water quality information system and the meteorological administration of Korea. To quantify model accuracy, the coefficient of determination, root mean square error, and mean absolute error were applied. Among random forest (RF), support vector machine, and artificial neural network, the RF with random dataset showed the highest accuracy. The RF was optimized when 78 trees were applied to the model. Input variables for the best RF model were total organic carbon (feature importance: 27%), total nitrogen (19%), pH (13%), water temperature (8%), total phosphorus (8%), electrical conductivity (7%), dissolved oxygen (6%), minimum air temperature (AT) (4%), mean AT (3%), and maximum AT (3%). The feature-importance analysis showed that total organic carbon was the most important variable to predict [Chl-a] in the Han river basin. Total nitrogen was a more important variable than total phosphorus.

Prediction of soil water contents and erodibility indices based on artificial neural networks: using topography and remote sensing

This study aimed to predict some soil water contents and soil erodibility indices with a multilayer perceptron (MLP) artificial neural network (ANN) using remote sensing data (Landsat 8 OLI TIRS) and topographic variables from a digital elevation model (DEM) in a semi-arid ecosystem. In models, the input variables were derived from remote sensing imaging and DEM. The output variables were field capacity, wilting point, aggregate stability index, structural stability index, dispersion ratio, and clay flocculation index. This study was realized in the watersheds of the Koruluk dam, the Kızlarkalesi, and the Telme ponds built for agricultural irrigation in Gümüşhane-Şiran. The soil samples were obtained from two depths (0-10 cm and 10-20 cm) from 59 soil profiles. Besides field capacity, wilting point, and aggregate stability analysis, undispersed/dispersed sand, silt, clay contents, and organic matter analysis were performed due to their strong effect on soil moisture, soil water content, and erodibility indices. The correlation analysis results showed significant relationships between soil characteristics and soil water contents/soil erodibility indices. The remote sensing variables were derived from three Landsat images of 2015 (June, July, and September). The performance results of MLP ANN models predicted for soil water contents and erodibility indices ranged from 0.75 to 0.90 for R², 0.046-4.115 for root mean square error (RMSE), 4.46-6.54 for normalized root mean square error (NRMSE), and 0.042-0.186 for mean absolute error (MAE). Topography was a more significant group of variables that affected soil water contents and soil erodibility indices and the feature importance of topography in the prediction was over 55%. The results showed that the use of topographic variables together with remote sensing variables in MLP ANN modeling increased the performance of the models.

A revised application of cognitive presence automatic classifiers for MOOCs: a new set of indicators revealed?

Automatic analysis of the myriad discussion messages in large online courses can support effective educator-learner interaction at scale. Robust classifiers are an essential foundation for the use of automatic analysis of cognitive presence in practice. This study reports on the application of a revised machine learning approach, which was originally developed from traditional, small-scale, for-credit, online courses, to automatically identify the phases of cognitive presence in the discussions from a Philosophy Massive Open Online Course (MOOC). The classifier performed slightly better on the MOOC discussions than similar previous studies have found. A new set of indicators to identify cognitive presence was revealed in the MOOC discussions, unlike those in the traditional courses. This study also cross-validated the classifier using MOOC discussion data from three other disciplines: Medicine, Education, and Humanities. Our results suggest that the cognitive classifier trained using MOOC data in only one discipline cannot yet be applied to other disciplines with sufficient accuracy.

Analysis of changes in geographical factors affecting sales in commercial alleys after COVID-19 using machine learning techniques

Social restrictions, such as social distancing and self-isolation, imposed owing to the coronavirus disease-19 (COVID-19) pandemic have resulted in a decreased demand of commodities and manufactured products. However, the factors influencing sales in commercial districts in the pre- and post-COVID-19 periods have not yet been fully understood. Thus, this study uses machine learning techniques to identify the changes in important geographical factors among both periods that have affected sales in commercial alleys. It was discovered that, in the post-COVID-19 period, the number of pharmacies, age groups of the working population, average monthly income, and number of families living in apartments priced higher than $600k in the catchment areas had relatively high importance after COVID-19 in the prediction of a high level of sales. Moreover, the percentage of deciduous forests appeared to be a important factor in the post-COVID-19 period. As the average monthly income and worker population in their 60s and numbers of pharmacies and banks increased after the pandemic, sales in commercial alleys also increased. The survival of commercial alleys has become a critical social problem in the post-COVID-19 era; therefore, this study is meaningful in that it suggests a policy direction that could contribute to the revitalization of commercial alley sales in the future and boost the local economy.

Risk assessment in COVID-19 patients: A multiclass classification approach

Understanding SARS-CoV-2 infection that causes COVID-19 disease among the population was fundamental to determine the risk factors associated with severe cases or even death. Amidst the study of the pandemic, Artificial Intelligence (AI) and Machine Learning (ML) have been successfully applied in many areas such as biomedicine. Using a dataset from the Mexican Ministry of Health, we performed a multiclass classification scheme for the detection of risks in COVID-19 patients and implemented three Machine Learning algorithms achieving the following accuracy measures: Random Forest (89.86%), GBM (89.37%) XGBoost (89.97%). The key findings are the identification of relevant components associated with different severities of COVID-19 disease. Among these factors, we found sex, age, days elapsed from the beginning of symptoms, symptoms such as dyspnea and polypnea; and other comorbidities such as diabetes and hypertension. This setting allows us to establish predicting algorithms to model the risk that an individual or a specific group of people face after contracting COVID-19 and the factors associated with developing complications or receiving appropriate treatment.

Machine learning exhibited excellent advantages in the performance simulation and prediction of free water surface constructed wetlands

Optimizing the design and operation parameters of free water surface constructed wetlands (FWS CWs) in runoff regulation and wastewater treatment is necessary to improve the comprehensive performance. In this study, nine machine learning (ML) algorithms were successfully developed to optimize the parameter combinations for FWS CWs. The scale effect of surface area on wetland performance was determined based on consistently smaller predictions (-6.2% to -28.9%) of the nine well-established ML algorithms. The models most suitable for FWS CW performance simulation and prediction were random forest and extra trees algorithms because of their high R² values (0.818 in both) with the training set and low mean absolute relative errors (4.7% and 3.8%, respectively) with the test set. Results from feature analysis of the six tree-based algorithms emphasized the importance of water depth and layout of inlet and outlet, and revealed the negligible effect of the aspect ratio. Feature importance and partial dependence analysis enhanced the interpretability of the tree-based algorithms. The proposed ML algorithms enabled the implementation of an extended scenario at a low cost in real time. Therefore, ML algorithms are suitable for expressing the complex and uncertain effects of the design and operation parameters on the performance of FWS CWs. Acquiring datasets consisting of more extensive, uniform, and unbiased parameter combinations is crucial for developing more robust and practical ML algorithms for the optimal design of FWS CWs.

Prediction of metabolic and pre-metabolic syndromes using machine learning models with anthropometric, lifestyle, and biochemical factors from a middle-aged population in Korea

Background

Metabolic syndrome (MetS) is a complex condition that appears as a cluster of metabolic abnormalities, and is closely associated with the prevalence of various diseases. Early prediction of the risk of MetS in the middle-aged population provides greater benefits for cardiovascular disease-related health outcomes. This study aimed to apply the latest machine learning techniques to find the optimal MetS prediction model for the middle-aged Korean population.

Methods

We retrieved 20 data types from the Korean Medicine Daejeon Citizen Cohort, a cohort study on a community-based population of adults aged 30–55 years. The data included sex, age, anthropometric data, lifestyle-related data, and blood indicators of 1991 individuals. Participants satisfying two (pre-MetS) or ≥ 3 (MetS) of the five NECP-ATP III criteria were included in the MetS group. MetS prediction used nine machine learning models based on the following algorithms: Decision tree, Gaussian Naïve Bayes, K-nearest neighbor, eXtreme gradient boosting (XGBoost), random forest, logistic regression, support vector machine, multi-layer perceptron, and 1D convolutional neural network. All analyses were performed by sequentially inputting the features in three steps according to their characteristics. The models’ performances were compared after applying the synthetic minority oversampling technique (SMOTE) to resolve data imbalance.

Results

MetS was detected in 33.85% of the subjects. Among the MetS prediction models, the tree-based random forest and XGBoost models showed the best performance, which improved with the number of features used. As a measure of the models’ performance, the area under the receiver operating characteristic curve (AUC) increased by up to 0.091 when the SMOTE was applied, with XGBoost showing the highest AUC of 0.851. Body mass index and waist-to-hip ratio were identified as the most important features in the MetS prediction models for this population.

Conclusions

Tree-based machine learning models were useful in identifying MetS with high accuracy in middle-aged Koreans. Early diagnosis of MetS is important and requires a multidimensional approach that includes self-administered questionnaire, anthropometric, and biochemical measurements.

Ongoing neural oscillations predict the post-stimulus outcome of closed loop auditory stimulation during slow-wave sleep

Large slow oscillations (SO, 0.5-2 Hz) characterise slow-wave sleep and are crucial to memory consolidation and other physiological functions. Manipulating slow oscillations may enhance sleep and memory, as well as benefitting the immune system. Closed-loop auditory stimulation (CLAS) has been demonstrated to increase the SO amplitude and to boost fast sleep spindle activity (11-16 Hz). Nevertheless, not all such stimuli are effective in evoking SOs, even when they are precisely phase locked. Here, we studied what factors of the ongoing activity patterns may help to determine what oscillations to stimulate to effectively enhance SOs or SO-locked spindle activity. Hence, we trained classifiers using the morphological characteristics of the ongoing SO, as measured by electroencephalography (EEG), to predict whether stimulation would lead to a benefit in terms of the resulting SO and spindle amplitude. Separate classifiers were trained using trials from spontaneous control and stimulated datasets, and we evaluated their performance by applying them to held-out data both within and across conditions. We were able to predict both when large SOs occurred spontaneously, and whether a phase-locked auditory click effectively enlarged them with good accuracy for predicting the SO trough (∼70%) and SO peak values (∼80%). Also, we were able to predict when stimulation would elicit spindle activity with an accuracy of ∼60%. Finally, we evaluate the importance of the various SO features used to make these predictions. Our results offer new insight into SO and spindle dynamics and may suggest techniques for developing future methods for online optimization of stimulation.

Explanation of machine learning models using shapley additive explanation and application for real data in hospital

BACKGROUND AND OBJECTIVE

When using machine learning techniques in decision-making processes, the interpretability of the models is important. In the present paper, we adopted the Shapley additive explanation (SHAP), which is based on fair profit allocation among many stakeholders depending on their contribution, for interpreting a gradient-boosting decision tree model using hospital data.

METHODS

For better interpretability, we propose two novel techniques as follows: (1) a new metric of feature importance using SHAP and (2) a technique termed feature packing, which packs multiple similar features into one grouped feature to allow an easier understanding of the model without reconstruction of the model. We then compared the explanation results between the SHAP framework and existing methods using cerebral infarction data from our hospital.

RESULTS

The interpretation by SHAP was mostly consistent with that by the existing methods. We showed how the A/G ratio works as an important prognostic factor for cerebral infarction using proposed techniques.

CONCLUSION

Our techniques are useful for interpreting machine learning models and can uncover the underlying relationships between features and outcome.

AI in predicting COPD in the Canadian population

Chronic obstructive pulmonary disease (COPD) is a progressive lung disease that produces non-reversible airflow limitations. Approximately 10% of Canadians aged 35 years or older are living with COPD. Primary care is often the first contact an individual will have with the healthcare system providing acute care, chronic disease management, and services aimed at health maintenance. This study used Electronic Medical Record (EMR) data from primary care clinics in seven provinces across Canada to develop predictive models to identify COPD in the Canadian population. The comprehensive nature of this primary care EMR data containing structured numeric, categorical, hybrid, and unstructured text data, enables the predictive models to capture symptoms of COPD and discriminate it from diseases with similar symptoms. We applied two supervised machine learning models, a Multilayer Neural Networks (MLNN) model and an Extreme Gradient Boosting (XGB) to identify COPD patients. The XGB model achieved an accuracy of 86% in the test dataset compared to 83% achieved by the MLNN. Utilizing feature importance, we identified a set of key symptoms from the EMR for diagnosing COPD, which included medications, health conditions, risk factors, and patient age. Application of this XGB model to primary care structured EMR data can identify patients with COPD from others having similar chronic conditions for disease surveillance, and improve evidence-based care delivery.

A comprehensive study of macro factors related to traffic fatality rates by XGBoost-based model and GIS techniques

With the fast development of economics, road safety is becoming a serious problem. Exploring macro factors is effective to improve road safety. However, the existing studies have some limitations: (1) The existing studies only considered one aspect of macro factors and constructed models based on a few data samples. (2) The methods commonly used cannot address the non-linear relationship or calculate the feature importance. The findings obtained from such models may be limited and biased. To address the limitations, this study proposes a BO-CV-XGBoost framework to explore the macro factors related to traffic fatality rate classes based on a high-dimensional dataset that fully considers the impact of multi-factor interaction with adequate data samples. The proposed framework is applied to a dataset in the US. 453 county-level macro factors are collected from various data sources, covering ten macro aspects, including topography, transportation, etc. The optimized BO-CV-XGBoost model obtains the best classification performance with an AUC of 0.8977 and an accuracy of 85.02%. Compared with other methods, the proposed model has superiority on fatality rate classification. Ten macro factors are identified, including 'Current-dollar GDP', 'highway miles per person', etc. The ten factors contain four aspects of information, including economics, transportation, education, and medical condition. Geographic information system (GIS) techniques are further used for spatial analysis of the identified macro factors. Therefore, targeted and effective measures are accordingly proposed to prevent traffic fatalities and improve road safety.

Automated detection and explainability of pathological gait patterns using a one-class support vector machine trained on inertial measurement unit based gait data

BACKGROUND

Machine learning approaches for the classification of pathological gait based on kinematic data, e.g. derived from inertial sensors, are commonly used in terms of a multi-class classification problem. However, there is a lack of research regarding one-class classifiers that are independent of certain pathologies. Therefore, it was the aim of this work to design a one-class classifier based on healthy norm-data that provides not only a prediction probability but rather an explanation of the classification decision, increasing the acceptance of this machine learning approach.

METHODS

The inertial sensor based gait kinematics of 25 healthy subjects was employed to train a one-class support vector machine. 25 healthy subjects, 20 patients after total hip arthroplasty and one transfemoral amputee served to validate the classifier. Prediction probabilities and feature importance scores were estimated for each subject.

FINDINGS

The support vector machine predicted 100% of the patients as outliers from the healthy group. Three healthy subjects were predicted as outliers. The feature importance calculation revealed the hip in the sagittal plane as most relevant feature concerning the group after total hip arthroplasty. For the misclassified healthy subject with the lowest probability score the knee flexion and the pelvis obliquity were identified.

INTERPRETATION

The support vector machine seems a useful tool to identify outliers from a healthy norm-group. The feature importance examination proved to provide valuable information on the musculoskeletal status of the subjects. In this combination, the present approach could be employed in various disciplines to identify abnormal gait and suggest subsequent training.

Utilizing a multilayer perceptron artificial neural network to assess a virtual reality surgical procedure

BACKGROUND

Virtual reality surgical simulators are a safe and efficient technology for the assessment and training of surgical skills. Simulators allow trainees to improve specific surgical techniques in risk-free environments. Recently, machine learning has been coupled to simulators to classify performance. However, most studies fail to extract meaningful observations behind the classifications and the impact of specific surgical metrics on the performance. One benefit from integrating machine learning algorithms, such as Artificial Neural Networks, to simulators is the ability to extract novel insights into the composites of the surgical performance that differentiate levels of expertise.

OBJECTIVE

This study aims to demonstrate the benefits of artificial neural network algorithms in assessing and analyzing virtual surgical performances. This study applies the algorithm on a virtual reality simulated annulus incision task during an anterior cervical discectomy and fusion scenario.

DESIGN

An artificial neural network algorithm was developed and integrated. Participants performed the simulated surgical procedure on the Sim-Ortho simulator. Data extracted from the annulus incision task were extracted to generate 157 surgical performance metrics that spanned three categories (motion, safety, and efficiency).

SETTING

Musculoskeletal Biomechanics Research Lab; Neurosurgical Simulation and Artificial Intelligence Learning Center, McGill University, Montreal, Canada.

PARTICIPANTS

Twenty-three participants were recruited and divided into 3 groups: 11 post-residents, 5 senior and 7 junior residents.

RESULTS

An artificial neural network model was trained on nine selected surgical metrics, spanning all three categories and achieved 80% testing accuracy.

CONCLUSIONS

This study outlines the benefits of integrating artificial neural networks to virtual reality surgical simulators in understanding composites of expertise performance.

Prediction of weaning from mechanical ventilation using Convolutional Neural Networks

Weaning from mechanical ventilation covers the process of liberating the patient from mechanical support and removing the associated endotracheal tube. The management of weaning from mechanical ventilation comprises a significant proportion of the care of critically ill intubated patients in Intensive Care Units (ICUs). Both prolonged dependence on mechanical ventilation and premature extubation expose patients to an increased risk of complications and increased health care costs. This work aims to develop a decision support model using routinely-recorded patient information to predict extubation readiness. In order to do so, we have deployed Convolutional Neural Networks (CNN) to predict the most appropriate treatment action in the next hour for a given patient state, using historical ICU data extracted from MIMIC-III. The model achieved 86% accuracy and 0.94 area under the receiver operating characteristic curve (AUC-ROC). We also performed feature importance analysis for the CNN model and interpreted these features using the DeepLIFT method. The results of the feature importance assessment show that the CNN model makes predictions using clinically meaningful and appropriate features. Finally, we implemented counterfactual explanations for the CNN model. This can help clinicians understand what feature changes for a particular patient would lead to a desirable outcome, i.e. readiness to extubate.

GT-Finder: Classify the family of glucose transporters with pre-trained BERT language models

Recently, language representation models have drawn a lot of attention in the field of natural language processing (NLP) due to their remarkable results. Among them, BERT (Bidirectional Encoder Representations from Transformers) has proven to be a simple, yet powerful language model that has achieved novel state-of-the-art performance. BERT adopted the concept of contextualized word embeddings to capture the semantics and context in which words appear. We utilized pre-trained BERT models to extract features from protein sequences for discriminating three families of glucose transporters: the major facilitator superfamily of glucose transporters (GLUTs), the sodium-glucose linked transporters (SGLTs), and the sugars will eventually be exported transporters (SWEETs). We treated protein sequences as sentences and transformed them into fixed-length meaningful vectors where a 768- or 1024-dimensional vector represents each amino acid. We observed that BERT-Base and BERT-Large models improved the performance by more than 4% in terms of average sensitivity and Matthews correlation coefficient (MCC), indicating the efficiency of this approach. We also developed a bidirectional transformer-based protein model (TransportersBERT) for comparison with existing pre-trained BERT models.

Interpreting a recurrent neural network's predictions of ICU mortality risk

Deep learning has demonstrated success in many applications; however, their use in healthcare has been limited due to the lack of transparency into how they generate predictions. Algorithms such as Recurrent Neural Networks (RNNs) when applied to Electronic Medical Records (EMR) introduce additional barriers to transparency because of the sequential processing of the RNN and the multi-modal nature of EMR data. This work seeks to improve transparency by: 1) introducing Learned Binary Masks (LBM) as a method for identifying which EMR variables contributed to an RNN model's risk of mortality (ROM) predictions for critically ill children; and 2) applying KernelSHAP for the same purpose. Given an individual patient, LBM and KernelSHAP both generate an attribution matrix that shows the contribution of each input feature to the RNN's sequence of predictions for that patient. Attribution matrices can be aggregated in many ways to facilitate different levels of analysis of the RNN model and its predictions. Presented are three methods of aggregations and analyses: 1) over volatile time periods within individual patient predictions, 2) over populations of ICU patients sharing specific diagnoses, and 3) across the general population of critically ill children.

Lung cancer survival period prediction and understanding: Deep learning approaches

INTRODUCTION

Survival period prediction through early diagnosis of cancer has many benefits. It allows both patients and caregivers to plan resources, time and intensity of care to provide the best possible treatment path for the patients. In this paper, by focusing on lung cancer patients, we build several survival prediction models using deep learning techniques to tackle both cancer survival classification and regression problems. We also conduct feature importance analysis to understand how lung cancer patients' relevant factors impact their survival periods. We contribute to identifying an approach to estimate survivability that are commonly and practically appropriate for medical use.

METHODOLOGIES

We have compared the performance across three of the most popular deep learning architectures - Artificial Neural Networks (ANN), Convolutional Neural Networks (CNN), and Recurrent Neural Networks (RNN) while comparing the performing of deep learning models against traditional machine learning models. The data was obtained from the lung cancer section of Surveillance, Epidemiology, and End Results (SEER) cancer registry.

RESULTS

The deep learning models outperformed traditional machine learning models across both classification and regression approaches. We obtained a best of 71.18 % accuracy for the classification approach when patients' survival periods are segmented into classes of '<=6 months',' 0.5 - 2 years' and '>2 years' and Root Mean Squared Error (RMSE) of 13.5 % andR² value of 0.5 for the regression approach for the deep learning models while the traditional machine learning models saturated at 61.12 % classification accuracy and 14.87 % RMSE in regression.

CONCLUSIONS

This approach can be a baseline for early prediction with predictions that can be further improved with more temporal treatment information collected from treated patients. In addition, we evaluated the feature importance to investigate the model interpretability, gaining further insight into the survival analysis models and the factors that are important in cancer survival period prediction.

Comparative evaluation of machine learning models for groundwater quality assessment

Contamination from pesticides and nitrate in groundwater is a significant threat to water quality in general and agriculturally intensive regions in particular. Three widely used machine learning models, namely, artificial neural networks (ANN), support vector machines (SVM), and extreme gradient boosting (XGB), were evaluated for their efficacy in predicting contamination levels using sparse data with non-linear relationships. The predictive ability of the models was assessed using a dataset consisting of 303 wells across 12 Midwestern states in the USA. Multiple hydrogeologic, water quality, and land use features were chosen as the independent variables, and classes were based on measured concentration ranges of nitrate and pesticide. This study evaluates the classification performance of the models for two, three, and four class scenarios and compares them with the corresponding regression models. The study also examines the issue of class imbalance and tests the efficacy of three class imbalance mitigation techniques: oversampling, weighting, and oversampling and weighting, for all the scenarios. The models' performance is reported using multiple metrics, both insensitive to class imbalance (accuracy) and sensitive to class imbalance (F1 score and MCC). Finally, the study assesses the importance of features using game-theoretic Shapley values to rank features consistently and offer model interpretability.