Biologically informed NeuralODEs for genome-wide regulatory dynamics

BACKGROUND

Gene regulatory network (GRN) models that are formulated as ordinary differential equations (ODEs) can accurately explain temporal gene expression patterns and promise to yield new insights into important cellular processes, disease progression, and intervention design. Learning such gene regulatory ODEs is challenging, since we want to predict the evolution of gene expression in a way that accurately encodes the underlying GRN governing the dynamics and the nonlinear functional relationships between genes. Most widely used ODE estimation methods either impose too many parametric restrictions or are not guided by meaningful biological insights, both of which impede either scalability, explainability, or both.

RESULTS

We developed PHOENIX, a modeling framework based on neural ordinary differential equations (NeuralODEs) and Hill-Langmuir kinetics, that overcomes limitations of other methods by flexibly incorporating prior domain knowledge and biological constraints to promote sparse, biologically interpretable representations of GRN ODEs. We tested the accuracy of PHOENIX in a series of in silico experiments, benchmarking it against several currently used tools. We demonstrated PHOENIX's flexibility by modeling regulation of oscillating expression profiles obtained from synchronized yeast cells. We also assessed the scalability of PHOENIX by modeling genome-scale GRNs for breast cancer samples ordered in pseudotime and for B cells treated with Rituximab.

CONCLUSIONS

PHOENIX uses a combination of user-defined prior knowledge and functional forms from systems biology to encode biological "first principles" as soft constraints on the GRN allowing us to predict subsequent gene expression patterns in a biologically explainable manner.

Biologically informed NeuralODEs for genome-wide regulatory dynamics

Modeling dynamics of gene regulatory networks using ordinary differential equations (ODEs) allow a deeper understanding of disease progression and response to therapy, thus aiding in intervention optimization. Although there exist methods to infer regulatory ODEs, these are generally limited to small networks, rely on dimensional reduction, or impose non-biological parametric restrictions - all impeding scalability and explainability. PHOENIX is a neural ODE framework incorporating prior domain knowledge as soft constraints to infer sparse, biologically interpretable dynamics. Extensive experiments - on simulated and real data - demonstrate PHOENIX's unique ability to learn key regulatory dynamics while scaling to the whole genome.

Biologically informed NeuralODEs for genome-wide regulatory dynamics

Models that are formulated as ordinary differential equations (ODEs) can accurately explain temporal gene expression patterns and promise to yield new insights into important cellular processes, disease progression, and intervention design. Learning such ODEs is challenging, since we want to predict the evolution of gene expression in a way that accurately encodes the causal gene-regulatory network (GRN) governing the dynamics and the nonlinear functional relationships between genes. Most widely used ODE estimation methods either impose too many parametric restrictions or are not guided by meaningful biological insights, both of which impedes scalability and/or explainability. To overcome these limitations, we developed PHOENIX, a modeling framework based on neural ordinary differential equations (NeuralODEs) and Hill-Langmuir kinetics, that can flexibly incorporate prior domain knowledge and biological constraints to promote sparse, biologically interpretable representations of ODEs. We test accuracy of PHOENIX in a series of in silico experiments benchmarking it against several currently used tools for ODE estimation. We also demonstrate PHOENIX's flexibility by studying oscillating expression data from synchronized yeast cells and assess its scalability by modelling genome-scale breast cancer expression for samples ordered in pseudotime. Finally, we show how the combination of user-defined prior knowledge and functional forms from systems biology allows PHOENIX to encode key properties of the underlying GRN, and subsequently predict expression patterns in a biologically explainable way.

The Network Zoo: a multilingual package for the inference and analysis of gene regulatory networks

Inference and analysis of gene regulatory networks (GRNs) require software that integrates multi-omic data from various sources. The Network Zoo (netZoo; netzoo.github.io) is a collection of open-source methods to infer GRNs, conduct differential network analyses, estimate community structure, and explore the transitions between biological states. The netZoo builds on our ongoing development of network methods, harmonizing the implementations in various computing languages and between methods to allow better integration of these tools into analytical pipelines. We demonstrate the utility using multi-omic data from the Cancer Cell Line Encyclopedia. We will continue to expand the netZoo to incorporate additional methods.

DRAGON: Determining Regulatory Associations using Graphical models on multi-Omic Networks

The increasing quantity of multi-omic data, such as methylomic and transcriptomic profiles collected on the same specimen or even on the same cell, provides a unique opportunity to explore the complex interactions that define cell phenotype and govern cellular responses to perturbations. We propose a network approach based on Gaussian Graphical Models (GGMs) that facilitates the joint analysis of paired omics data. This method, called DRAGON (Determining Regulatory Associations using Graphical models on multi-Omic Networks), calibrates its parameters to achieve an optimal trade-off between the network's complexity and estimation accuracy, while explicitly accounting for the characteristics of each of the assessed omics 'layers.' In simulation studies, we show that DRAGON adapts to edge density and feature size differences between omics layers, improving model inference and edge recovery compared to state-of-the-art methods. We further demonstrate in an analysis of joint transcriptome - methylome data from TCGA breast cancer specimens that DRAGON can identify key molecular mechanisms such as gene regulation via promoter methylation. In particular, we identify Transcription Factor AP-2 Beta (TFAP2B) as a potential multi-omic biomarker for basal-type breast cancer. DRAGON is available as open-source code in Python through the Network Zoo package (netZooPy v0.8; netzoo.github.io).

Assessment of Artificial Intelligence in Echocardiography Diagnostics in Differentiating Takotsubo Syndrome From Myocardial Infarction

Importance

Machine learning algorithms enable the automatic classification of cardiovascular diseases based on raw cardiac ultrasound imaging data. However, the utility of machine learning in distinguishing between takotsubo syndrome (TTS) and acute myocardial infarction (AMI) has not been studied.

Objectives

To assess the utility of machine learning systems for automatic discrimination of TTS and AMI.

Design, Settings, and Participants

This cohort study included clinical data and transthoracic echocardiogram results of patients with AMI from the Zurich Acute Coronary Syndrome Registry and patients with TTS obtained from 7 cardiovascular centers in the International Takotsubo Registry. Data from the validation cohort were obtained from April 2011 to February 2017. Data from the training cohort were obtained from March 2017 to May 2019. Data were analyzed from September 2019 to June 2021.

Exposure

Transthoracic echocardiograms of 224 patients with TTS and 224 patients with AMI were analyzed.

Main Outcomes and Measures

Area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity of the machine learning system evaluated on an independent data set and 4 practicing cardiologists for comparison. Echocardiography videos of 228 patients were used in the development and training of a deep learning model. The performance of the automated echocardiogram video analysis method was evaluated on an independent data set consisting of 220 patients. Data were matched according to age, sex, and ST-segment elevation/non-ST-segment elevation (1 patient with AMI for each patient with TTS). Predictions were compared with echocardiographic-based interpretations from 4 practicing cardiologists in terms of sensitivity, specificity, and AUC calculated from confidence scores concerning their binary diagnosis.

Results

In this cohort study, apical 2-chamber and 4-chamber echocardiographic views of 110 patients with TTS (mean [SD] age, 68.4 [12.1] years; 103 [90.4%] were female) and 110 patients with AMI (mean [SD] age, 69.1 [12.2] years; 103 [90.4%] were female) from an independent data set were evaluated. This approach achieved a mean (SD) AUC of 0.79 (0.01) with an overall accuracy of 74.8 (0.7%). In comparison, cardiologists achieved a mean (SD) AUC of 0.71 (0.03) and accuracy of 64.4 (3.5%) on the same data set. In a subanalysis based on 61 patients with apical TTS and 56 patients with AMI due to occlusion of the left anterior descending coronary artery, the model achieved a mean (SD) AUC score of 0.84 (0.01) and an accuracy of 78.6 (1.6%), outperforming the 4 practicing cardiologists (mean [SD] AUC, 0.72 [0.02]) and accuracy of 66.9 (2.8%).

Conclusions and Relevance

In this cohort study, a real-time system for fully automated interpretation of echocardiogram videos was established and trained to differentiate TTS from AMI. While this system was more accurate than cardiologists in echocardiography-based disease classification, further studies are warranted for clinical application.

GRAND: a database of gene regulatory network models across human conditions

Gene regulation plays a fundamental role in shaping tissue identity, function, and response to perturbation. Regulatory processes are controlled by complex networks of interacting elements, including transcription factors, miRNAs and their target genes. The structure of these networks helps to determine phenotypes and can ultimately influence the development of disease or response to therapy. We developed GRAND (https://grand.networkmedicine.org) as a database for computationally-inferred, context-specific gene regulatory network models that can be compared between biological states, or used to predict which drugs produce changes in regulatory network structure. The database includes 12 468 genome-scale networks covering 36 human tissues, 28 cancers, 1378 unperturbed cell lines, as well as 173 013 TF and gene targeting scores for 2858 small molecule-induced cell line perturbation paired with phenotypic information. GRAND allows the networks to be queried using phenotypic information and visualized using a variety of interactive tools. In addition, it includes a web application that matches disease states to potentially therapeutic small molecule drugs using regulatory network properties.

Glycowork: A Python package for glycan data science and machine learning

While glycans are crucial for biological processes, existing analysis modalities make it difficult for researchers with limited computational background to include these diverse carbohydrates into workflows. Here, we present glycowork, an open-source Python package designed for glycan-related data science and machine learning by end users. Glycowork includes functions to, for instance, automatically annotate glycan motifs and analyze their distributions via heatmaps and statistical enrichment. We also provide visualization methods, routines to interact with stored databases, trained machine learning models and learned glycan representations. We envision that glycowork can extract further insights from glycan datasets and demonstrate this with workflows that analyze glycan motifs in various biological contexts. Glycowork can be freely accessed at https://github.com/BojarLab/glycowork/.

Using graph convolutional neural networks to learn a representation for glycans

As the only nonlinear and the most diverse biological sequence, glycans offer substantial challenges for computational biology. These carbohydrates participate in nearly all biological processes—from protein folding to viral cell entry—yet are still not well understood. There are few computational methods to link glycan sequences to functions, and they do not fully leverage all available information about glycans. SweetNet is a graph convolutional neural network that uses graph representation learning to facilitate a computational understanding of glycobiology. SweetNet explicitly incorporates the nonlinear nature of glycans and establishes a framework to map any glycan sequence to a representation. We show that SweetNet outperforms other computational methods in predicting glycan properties on all reported tasks. More importantly, we show that glycan representations, learned by SweetNet, are predictive of organismal phenotypic and environmental properties. Finally, we use glycan-focused machine learning to predict viral glycan binding, which can be used to discover viral receptors.

Burkholz et al. develop an analysis platform for glycans, using graph convolutional neural networks, that considers the branched nature of these carbohydrates. They demonstrate that glycan-focused machine learning can be employed for various purposes, such as to cluster species according to their glycomic similarity or to identify viral receptors.

Improving 1-year mortality prediction in ACS patients using machine learning

BACKGROUND

The Global Registry of Acute Coronary Events (GRACE) score is an established clinical risk stratification tool for patients with acute coronary syndromes (ACS). We developed and internally validated a model for 1-year all-cause mortality prediction in ACS patients.

METHODS

Between 2009 and 2012, 2'168 ACS patients were enrolled into the Swiss SPUM-ACS Cohort. Biomarkers were determined in 1'892 patients and follow-up was achieved in 95.8% of patients. 1-year all-cause mortality was 4.3% (n = 80). In our analysis we consider all linear models using combinations of 8 out of 56 variables to predict 1-year all-cause mortality and to derive a variable ranking.

RESULTS

1.3% of 1'420'494'075 models outperformed the GRACE 2.0 Score. The SPUM-ACS Score includes age, plasma glucose, NT-proBNP, left ventricular ejection fraction (LVEF), Killip class, history of peripheral artery disease (PAD), malignancy, and cardio-pulmonary resuscitation. For predicting 1-year mortality after ACS, the SPUM-ACS Score outperformed the GRACE 2.0 Score which achieves a 5-fold cross-validated AUC of 0.81 (95% CI 0.78-0.84). Ranking individual features according to their importance across all multivariate models revealed age, trimethylamine N-oxide, creatinine, history of PAD or malignancy, LVEF, and haemoglobin as the most relevant variables for predicting 1-year mortality.

CONCLUSIONS

The variable ranking and the selection for the SPUM-ACS Score highlight the relevance of age, markers of heart failure, and comorbidities for prediction of all-cause death. Before application, this score needs to be externally validated and refined in larger cohorts.

CLINICAL TRIAL REGISTRATION

NCT01000701.

Gene regulatory network inference as relaxed graph matching

Bipartite network inference is a ubiquitous problem across disciplines. One important example in the field molecular biology is gene regulatory network inference. Gene regulatory networks are an instrumental tool aiding in the discovery of the molecular mechanisms driving diverse diseases, including cancer. However, only noisy observations of the projections of these regulatory networks are typically assayed. In an effort to better estimate regulatory networks from their noisy projections, we formulate a non-convex but analytically tractable optimization problem called OTTER. This problem can be interpreted as relaxed graph matching between the two projections of the bipartite network. OTTER's solutions can be derived explicitly and inspire a spectral algorithm, for which we provide network recovery guarantees. We also provide an alternative approach based on gradient descent that is more robust to noise compared to the spectral algorithm. Interestingly, this gradient descent approach resembles the message passing equations of an established gene regulatory network inference method, PANDA. Using three cancer-related data sets, we show that OTTER outperforms state-of-the-art inference methods in predicting transcription factor binding to gene regulatory regions. To encourage new graph matching applications to this problem, we have made all networks and validation data publicly available.

Artificial intelligence in echocardiography diagnostics – detection of takotsubo syndrome

Background

Machine learning allows classifying diseases based only on raw echocardiographic imaging data and is therefore a landmark in the development of computer-assisted decision support systems in echocardiography.

Purpose

The present study sought to determine the value of deep (machine) learning systems for automatic discrimination of takotsubo syndrome and acute myocardial infarction.

Methods

Apical 2- and 4-chamber echocardiographic views of 110 patients with takotsubo syndrome and 110 patients with acute myocardial infarction were used in the development, training and validation of a deep learning approach, i.e. a convolutional autoencoder (CAE) for feature extraction followed by classical machine learning models for classification of the diseases.

Results

The deep learning model achieved an area under the receiver operating curve (AUC) of 0.801 with an overall accuracy of 74.5% for 5-fold cross validation evaluated on a clinically relevant dataset. In comparison, experienced cardiologists achieved AUCs in the range 0.678–0.740 and an average accuracy of 64.5% on the same dataset.

Conclusions

A real-time system for fully automated interpretation of echocardiographic videos was established and trained to differentiate takotsubo syndrome from acute myocardial infarction. The framework provides insight into the algorithms' decision process for physicians and yields new and valuable information on the manifestation of disease patterns in echocardiographic data. While our system was superior to cardiologists in echocardiography-based disease classification, further studies should be conducted in a larger patient population to prove its clinical application.

Funding Acknowledgement

Type of funding source: None

Correlations between thresholds and degrees: An analytic approach to model attacks and failure cascades

Two node variables determine the evolution of cascades in random networks: a node's degree and threshold. Correlations between both fundamentally change the robustness of a network, yet they are disregarded in standard analytic methods as local tree or heterogeneous mean field approximations, since order statistics are difficult to capture analytically because of their combinatorial nature. We show how they become tractable in the thermodynamic limit of infinite network size. This enables the analytic description of node attacks that are characterized by threshold allocations based on node degree. Using two examples, we discuss possible implications of irregular phase transitions and different speeds of cascade evolution for the control of cascades.

Framework for cascade size calculations on random networks

We present a framework to calculate the cascade size evolution for a large class of cascade models on random network ensembles in the limit of infinite network size. Our method is exact and applies to network ensembles with almost arbitrary degree distribution, degree-degree correlations, and, in case of threshold models, for arbitrary threshold distribution. With our approach, we shift the perspective from the known branching process approximations to the iterative update of suitable probability distributions. Such distributions are key to capture cascade dynamics that involve possibly continuous quantities and that depend on the cascade history, e.g., if load is accumulated over time. As a proof of concept, we provide two examples: (a) Constant load models that cover many of the analytically tractable casacade models, and, as a highlight, (b) a fiber bundle model that was not tractable by branching process approximations before. Our derivations cover the whole cascade dynamics, not only their steady state. This allows us to include interventions in time or further model complexity in the analysis.

How damage diversification can reduce systemic risk

We study the influence of risk diversification on cascading failures in weighted complex networks, where weighted directed links represent exposures between nodes. These weights result from different diversification strategies and their adjustment allows us to reduce systemic risk significantly by topological means. As an example, we contrast a classical exposure diversification (ED) approach with a damage diversification (DD) variant. The latter reduces the loss that the failure of high degree nodes generally inflict to their network neighbors and thus hampers the cascade amplification. To quantify the final cascade size and obtain our results, we develop a branching process approximation taking into account that inflicted losses cannot only depend on properties of the exposed, but also of the failing node. This analytic extension is a natural consequence of the paradigm shift from individual to system safety. To deepen our understanding of the cascade process, we complement this systemic perspective by a mesoscopic one: an analysis of the failure risk of nodes dependent on their degree. Additionally, we ask for the role of these failures in the cascade amplification.