From Clustering to Cluster Explanations via Neural Networks

Machine learning and multi-omics integration: advancing cardiovascular translational research and clinical practice

The global burden of cardiovascular diseases continues to rise, making their prevention, diagnosis and treatment increasingly critical. With advancements and breakthroughs in omics technologies such as high-throughput sequencing, multi-omics approaches can offer a closer reflection of the complex physiological and pathological changes in the body from a molecular perspective, providing new microscopic insights into cardiovascular diseases research. However, due to the vast volume and complexity of data, accurately describing, utilising, and translating these biomedical data demands substantial effort. Researchers and clinicians are actively developing artificial intelligence (AI) methods for data-driven knowledge discovery and causal inference using various omics data. These AI approaches, integrated with multi-omics research, have shown promising outcomes in cardiovascular studies. In this review, we outline the methods for integrating machine learning, one of the most successful applications of AI, with omics data and summarise representative AI models developed that leverage various omics data to facilitate the exploration of cardiovascular diseases from underlying mechanisms to clinical practice. Particular emphasis is placed on the effectiveness of using AI to extract potential molecular information to address current knowledge gaps. We discuss the challenges and opportunities of integrating omics with AI into routine diagnostic and therapeutic practices and anticipate the future development of novel AI models for wider application in the field of cardiovascular diseases.

Adaptive multilevel thresholding for SVD-based clutter filtering in ultrafast transthoracic coronary flow imaging

BACKGROUND AND OBJECTIVE

The integration of ultrafast Doppler imaging with singular value decomposition clutter filtering has demonstrated notable enhancements in flow measurement and Doppler sensitivity, surpassing conventional Doppler techniques. However, in the context of transthoracic coronary flow imaging, additional challenges arise due to factors such as the utilization of unfocused diverging waves, constraints in spatial and temporal resolution for achieving deep penetration, and rapid tissue motion. These challenges pose difficulties for ultrafast Doppler imaging and singular value decomposition in determining optimal tissue-blood (TB) and blood-noise (BN) thresholds, thereby limiting their ability to deliver high-contrast Doppler images.

METHODS

This study introduces a novel local blood subspace detection method that utilizes multilevel thresholding by the valley-emphasized Otsu's method to estimate the TB and BN thresholds on a pixel-based level, operating under the assumption that the magnitude of the spatial singular vector curve of each pixel resembles the shape of a trimodal Gaussian. Upon obtaining the local TB and BN thresholds, a weighted mask (WM) is generated to assess the blood content in each pixel. To enhance the computational efficiency of this pixel-based algorithm, a dedicated tree-structure k-means clustering approach, further enhanced by noise rejection (NR) at each singular vector order, is proposed to group pixels with similar spatial singular vector curves, subsequently applying local thresholding (LT) on a cluster-based (CB) level.

RESULTS

The effectiveness of the proposed method was evaluated using an ex-vivo setup featuring a Langendorff swine heart. Comparative analysis with power Doppler images filtered using the conventional global thresholding method, which uniformly applies TB and BN thresholds to all pixels, revealed noteworthy enhancements. Specifically, our proposed CBLT+NR+WM approach demonstrated an average 10.8-dB and 11.2-dB increase in Contrast-to-Noise ratio and Contrast in suppressing the tissue signal, paralleled by an average 5-dB (Contrast-to-Noise ratio) and 9-dB (Contrast) increase in suppressing the noise signal.

CONCLUSIONS

These results clearly indicate the capability of our method to attenuate residual tissue and noise signals compared to the global thresholding method, suggesting its promising utility in challenging transthoracic settings for coronary flow measurement.

Feature graphs for interpretable unsupervised tree ensembles: centrality, interaction, and application in disease subtyping

Explainable and interpretable machine learning has emerged as essential in leveraging artificial intelligence within high-stakes domains such as healthcare to ensure transparency and trustworthiness. Feature importance analysis plays a crucial role in improving model interpretability by pinpointing the most relevant input features, particularly in disease subtyping applications, aimed at stratifying patients based on a small set of signature genes and biomarkers. While clustering methods, including unsupervised random forests, have demonstrated good performance, approaches for evaluating feature contributions in an unsupervised regime are notably scarce. To address this gap, we introduce a novel methodology to enhance the interpretability of unsupervised random forests by elucidating feature contributions through the construction of feature graphs, both over the entire dataset and individual clusters, that leverage parent-child node splits within the trees. Feature selection strategies to derive effective feature combinations from these graphs are presented and extensively evaluated on synthetic and benchmark datasets against state-of-the-art methods, standing out for performance, computational efficiency, reliability, versatility and ability to provide cluster-specific insights. In a disease subtyping application, clustering kidney cancer gene expression data over a feature subset selected with our approach reveals three patient groups with different survival outcomes. Cluster-specific analysis identifies distinctive feature contributions and interactions, essential for devising targeted interventions, conducting personalised risk assessments, and enhancing our understanding of the underlying molecular complexities.

Towards Explainable Artificial Intelligence for GNSS Multipath LSTM Training Models

This paper addresses the critical challenge of understanding and interpreting deep learning models in Global Navigation Satellite System (GNSS) applications, specifically focusing on multipath effect detection and analysis. As GNSS systems become increasingly reliant on deep learning for signal processing, the lack of model interpretability poses significant risks for safety-critical applications. We propose a novel approach combining Recurrent Neural Networks (RNNs) with Long Short-Term Memory (LSTM) cells with Layer-wise Relevance Propagation (LRP) to create an explainable framework for multipath detection. Our key contributions include: (1) the development of an interpretable LSTM architecture for processing GNSS observables, including multipath variables, carrier-to-noise ratios, and satellite elevation angles; (2) the adaptation of the LRP technique for GNSS signal analysis, enabling attribution of model decisions to specific input features; and (3) the discovery of a correlation between LRP relevance scores and signal anomalies, leading to a new method for anomaly detection. Through systematic experimental validation, we demonstrate that our LSTM model achieves high prediction accuracy across all GNSS parameters while maintaining interpretability. A significant finding emerges from our controlled experiments: LRP relevance scores consistently increase during anomalous signal conditions, with growth rates varying from 7.34% to 32.48% depending on the feature type. In our validation experiments, we systematically introduced signal anomalies in specific time segments of the data sequence and observed corresponding increases in LRP scores: multipath parameters showed increases of 7.34-8.81%, carrier-to-noise ratios exhibited changes of 12.50-32.48%, and elevation angle parameters increased by 16.10%. These results demonstrate the potential of LRP-based analysis for enhancing GNSS signal quality monitoring and integrity assessment. Our approach not only improves the interpretability of deep learning models in GNSS applications but also provides a practical framework for detecting and analyzing signal anomalies, contributing to the development of more reliable and trustworthy navigation systems.

Analyzing Atomic Interactions in Molecules as Learned by Neural Networks

While machine learning (ML) models have been able to achieve unprecedented accuracies across various prediction tasks in quantum chemistry, it is now apparent that accuracy on a test set alone is not a guarantee for robust chemical modeling such as stable molecular dynamics (MD). To go beyond accuracy, we use explainable artificial intelligence (XAI) techniques to develop a general analysis framework for atomic interactions and apply it to the SchNet and PaiNN neural network models. We compare these interactions with a set of fundamental chemical principles to understand how well the models have learned the underlying physicochemical concepts from the data. We focus on the strength of the interactions for different atomic species, how predictions for intensive and extensive quantum molecular properties are made, and analyze the decay and many-body nature of the interactions with interatomic distance. Models that deviate too far from known physical principles produce unstable MD trajectories, even when they have very high energy and force prediction accuracy. We also suggest further improvements to the ML architectures to better account for the polynomial decay of atomic interactions.

Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces

Explainable AI aims to overcome the black-box nature of complex ML models like neural networks by generating explanations for their predictions. Explanations often take the form of a heatmap identifying input features (e.g. pixels) that are relevant to the model's decision. These explanations, however, entangle the potentially multiple factors that enter into the overall complex decision strategy. We propose to disentangle explanations by extracting at some intermediate layer of a neural network, subspaces that capture the multiple and distinct activation patterns (e.g. visual concepts) that are relevant to the prediction. To automatically extract these subspaces, we propose two new analyses, extending principles found in PCA or ICA to explanations. These novel analyses, which we call principal relevant component analysis (PRCA) and disentangled relevant subspace analysis (DRSA), maximize relevance instead of e.g. variance or kurtosis. This allows for a much stronger focus of the analysis on what the ML model actually uses for predicting, ignoring activations or concepts to which the model is invariant. Our approach is general enough to work alongside common attribution techniques such as Shapley Value, Integrated Gradients, or LRP. Our proposed methods show to be practically useful and compare favorably to the state of the art as demonstrated on benchmarks and three use cases.

Deep Neural Networks and Tabular Data: A Survey

Heterogeneous tabular data are the most commonly used form of data and are essential for numerous critical and computationally demanding applications. On homogeneous datasets, deep neural networks have repeatedly shown excellent performance and have therefore been widely adopted. However, their adaptation to tabular data for inference or data generation tasks remains highly challenging. To facilitate further progress in the field, this work provides an overview of state-of-the-art deep learning methods for tabular data. We categorize these methods into three groups: data transformations, specialized architectures, and regularization models. For each of these groups, our work offers a comprehensive overview of the main approaches. Moreover, we discuss deep learning approaches for generating tabular data and also provide an overview over strategies for explaining deep models on tabular data. Thus, our first contribution is to address the main research streams and existing methodologies in the mentioned areas while highlighting relevant challenges and open research questions. Our second contribution is to provide an empirical comparison of traditional machine learning methods with 11 deep learning approaches across five popular real-world tabular datasets of different sizes and with different learning objectives. Our results, which we have made publicly available as competitive benchmarks, indicate that algorithms based on gradient-boosted tree ensembles still mostly outperform deep learning models on supervised learning tasks, suggesting that the research progress on competitive deep learning models for tabular data is stagnating. To the best of our knowledge, this is the first in-depth overview of deep learning approaches for tabular data; as such, this work can serve as a valuable starting point to guide researchers and practitioners interested in deep learning with tabular data.

Using of artificial neural networks and different evolutionary algorithms to predict the viscosity and thermal conductivity of silica-alumina-MWCN/water nanofluid

This study predicts the parameters such as viscosity and thermal conductivity in silica-alumina-MWCN/water nanofluid using the artificial intelligence method and using design variables such as solid volume fraction and temperature. In this study, 6 optimization algorithms were used to predict and numerically model the μnf and TC of silica-alumina-MWCNT/water-NF. In this study, six measurement criteria were used to evaluate the estimates obtained from the coupling process of GMDH ANN with each of these 6 optimization algorithms. The results reveal that the influence of the φ is notably higher on both μnf and TC with values of 0.83 for μnf and 0.92 for TC, while Temp has a relatively weaker impact with -0.5 for μnf and 0.38 for TC. Among various algorithms, the coupling of the evolutionary algorithm NSGA II with ANN and GMDH performs best in predicting μnf and TC for the NF, with a maximum margin of deviation of -0.108 and an R2 evaluation criterion of 0.99996 for μnf and 1 for TC, indicating exceptional model accuracy. In the subsequent phase, a meta-heuristic Genetic Algorithm minimizes μnf and TC values. Four points (A, B, C, and D) along the Pareto front are selected, with point A representing the optimal state characterized by low values of φ and Temp (0.0002 and 50.8772, respectively) and corresponding target function values of 0.9988 for μnf and 0.6344 for TC. In contrast, point D represents the highest values of φ and Temp (0.49986 and 59.9775, respectively) and yields target function values of 2.382 for μnf and 0.8517 for TC. This analysis aids in identifying the optimal operating conditions for maximizing NF performance.

Explanation of clustering result based on multi-objective optimization

Clustering is an unsupervised machine learning technique whose goal is to cluster unlabeled data. But traditional clustering methods only output a set of results and do not provide any explanations of the results. Although in the literature a number of methods based on decision tree have been proposed to explain the clustering results, most of them have some disadvantages, such as too many branches and too deep leaves, which lead to complex explanations and make it difficult for users to understand. In this paper, a hypercube overlay model based on multi-objective optimization is proposed to achieve succinct explanations of clustering results. The model designs two objective functions based on the number of hypercubes and the compactness of instances and then uses multi-objective optimization to find a set of nondominated solutions. Finally, an Utopia point is defined to determine the most suitable solution, in which each cluster can be covered by as few hypercubes as possible. Based on these hypercubes, an explanations of each cluster is provided. Upon verification on synthetic and real datasets respectively, it shows that the model can provide a concise and understandable explanations to users.

Predictive Modeling and Integrated Risk Assessment of Postoperative Mortality and Pneumonia in Traumatic Brain Injury Patients through Clustering and Machine Learning: Retrospective Study

This study harnessed machine learning to forecast postoperative mortality (POM) and postoperative pneumonia (PPN) among surgical traumatic brain injury (TBI) patients. Our analysis centered on the following key variables: Glasgow Coma Scale (GCS), midline brain shift (MSB), and time from injury to emergency room arrival (TIE). Additionally, we introduced innovative clustered variables to enhance predictive accuracy and risk assessment. Exploring data from 617 patients spanning 2012 to 2022, we observed that 22.9% encountered postoperative mortality, while 30.0% faced postoperative pneumonia (PPN). Sensitivity for POM and PPN prediction, before incorporating clustering, was in the ranges of 0.43-0.82 (POM) and 0.54-0.76 (PPN). Following clustering, sensitivity values were 0.47-0.76 (POM) and 0.61-0.77 (PPN). Accuracy was in the ranges of 0.67-0.76 (POM) and 0.70-0.81 (PPN) prior to clustering and 0.42-0.73 (POM) and 0.55-0.73 (PPN) after clustering. Clusters characterized by low GCS, small MSB, and short TIE exhibited a 3.2-fold higher POM risk compared to clusters with high GCS, small MSB, and short TIE. In summary, leveraging clustered variables offers a novel avenue for predicting POM and PPN in TBI patients. Assessing the amalgamated impact of GCS, MSB, and TIE characteristics provides valuable insights for clinical decision making.