AMICI: High-Performance Sensitivity Analysis for Large Ordinary Differential Equation Models

Using SED-ML for reproducible curation: Verifying BioModels across multiple simulation engines

The BioModels Repository contains over 1000 manually curated mechanistic models drawn from published literature, most of which are encoded in the Systems Biology Markup Language (SBML). This community-based standard formally specifies each model, but does not describe the computational experimental conditions to run a simulation. Therefore, it can be challenging to reproduce any given figure or result from a publication with an SBML model alone. The Simulation Experiment Description Markup Language (SED-ML) provides a solution: a standard way to specify exactly how to run a specific experiment that corresponds to a specific figure or result. BioModels was established years before SED-ML, and both systems evolved over time, both in content and acceptance. Hence, only about half of the entries in BioModels contained SED-ML files, and these files reflected the version of SED-ML that was available at the time. Additionally, almost all of these SED-ML files had at least one minor mistake that made them invalid. To make these models and their results more reproducible, we report here on our work updating, correcting and providing new SED-ML files for 1055 curated mechanistic models in BioModels. In addition, because SED-ML is implementation-independent, it can be used for verification, demonstrating that results hold across multiple simulation engines. Here, we use a wrapper architecture for interpreting SED-ML, and report verification results across five different ODE-based biosimulation engines. Our work with SED-ML and the BioModels collection aims to improve the utility of these models by making them more reproducible and credible.

Mathematical multi-compartment modeling of chronic lymphocytic leukemia cell kinetics under ibrutinib

The Bruton tyrosine kinase inhibitor ibrutinib is an effective treatment for patients with chronic lymphocytic leukemia (CLL). While it rapidly reduces lymph node and spleen size, it initially increases the number of lymphocytes in the blood due to cell redistribution. A previously published mathematical model described and quantified those cell kinetics. Here, we propose an alternative mechanistic model that outperforms the previous model in 26 of 29 patients. Our model introduces constant subcompartments for healthy lymphocytes and benign tissue and treats spleen and lymph nodes as separate compartments. This three-compartment model (comprising blood, spleen, and lymph nodes) performed significantly better in patients without a mutation in the IGHV gene, indicating a diverse response to ibrutinib for cells residing in lymph nodes and spleen. Additionally, high ZAP-70 expression was linked to less cell death in the spleen. Overall, our study enhances understanding of CLL genetics and patient response to ibrutinib and provides a framework applicable to the study of similar drugs.

Benchmarking methods for computing local sensitivities in ordinary differential equation models at dynamic and steady states

Estimating parameters of dynamic models from experimental data is a challenging, and often computationally-demanding task. It requires a large number of model simulations and objective function gradient computations, if gradient-based optimization is used. In many cases, steady-state computation is a part of model simulation, either due to steady-state data or an assumption that the system is at steady state at the initial time point. Various methods are available for steady-state and gradient computation. Yet, the most efficient pair of methods (one for steady states, one for gradients) for a particular model is often not clear. In order to facilitate the selection of methods, we explore six method pairs for computing the steady state and sensitivities at steady state using six real-world problems. The method pairs involve numerical integration or Newton's method to compute the steady-state, and-for both forward and adjoint sensitivity analysis-numerical integration or a tailored method to compute the sensitivities at steady-state. Our evaluation shows that all method pairs provide accurate steady-state and gradient values, and that the two method pairs that combine numerical integration for the steady-state with a tailored method for the sensitivities at steady-state were the most robust, and amongst the most computationally-efficient. We also observed that while Newton's method for steady-state computation yields a substantial speedup compared to numerical integration, it may lead to a large number of simulation failures. Overall, our study provides a concise overview across current methods for computing sensitivities at steady state. While our study shows that there is no universally-best method pair, it also provides guidance to modelers in choosing the right methods for a problem at hand.

A Model-Driven Meta-Analysis Supports the Emerging Consensus View that Inhibitory Neurons Dominate BOLD-fMRI Responses

Functional magnetic resonance imaging (fMRI) is a pivotal tool for mapping neuronal activity in the brain. Traditionally, the observed hemodynamic changes are assumed to reflect the activity of the most common neuronal type: excitatory neurons. In contrast, recent experiments, using optogenetic techniques, suggest that the fMRI-signal instead reflects the activity of inhibitory interneurons. However, these data paint a complex picture, with numerous regulatory interactions, and where the different experiments display many qualitative differences. It is therefore not trivial how to quantify the relative contributions of the different cell types and to combine all observations into a unified theory. To address this, we present a new model-driven meta-analysis, which provides a unified and quantitative explanation for all data. This model-driven analysis allows for quantification of the relative contribution of different cell types: the contribution to the BOLD-signal from the excitatory cells is <20 % and 50-80 % comes from the interneurons. Our analysis also provides a mechanistic explanation for the observed experiment-to-experiment differences, e.g. a biphasic vascular response dependent on different stimulation intensities and an emerging secondary post-stimulation peak during longer stimulations. In summary, our study provides a new, emerging consensus-view supporting the larger role of interneurons in fMRI.

Computational Model Predicts Patient Outcomes in Luminal B Breast Cancer Treated with Endocrine Therapy and CDK4/6 Inhibition

PURPOSE

Development of a computational biomarker to predict, prior to treatment, the response to CDK4/6 inhibition (CDK4/6i) in combination with endocrine therapy in patients with breast cancer.

EXPERIMENTAL DESIGN

A mechanistic mathematical model that accounts for protein signaling and drug mechanisms of action was developed and trained on extensive, publicly available data from breast cancer cell lines. The model was built to provide a patient-specific response score based on the expression of six genes (CCND1, CCNE1, ESR1, RB1, MYC, and CDKN1A). The model was validated in five independent cohorts of 148 patients in total with early-stage or advanced breast cancer treated with endocrine therapy and CDK4/6i. Response was measured either by evaluating Ki67 levels and PAM50 risk of relapse (ROR) after neoadjuvant treatment or by evaluating progression-free survival (PFS).

RESULTS

The model showed significant association with patient's outcomes in all five cohorts. The model predicted high Ki67 [area under the curve; AUC (95% confidence interval, CI) of 0.80 (0.64-0.92), 0.81 (0.60-1.00) and 0.80 (0.65-0.93)] and high PAM50 ROR [AUC of 0.78 (0.64-0.89)]. This observation was not obtained in patients treated with chemotherapy. In the other cohorts, patient stratification based on the model prediction was significantly associated with PFS [hazard ratio (HR) = 2.92 (95% CI, 1.08-7.86), P = 0.034 and HR = 2.16 (1.02 4.55), P = 0.043].

CONCLUSIONS

A mathematical modeling approach accurately predicts patient outcome following CDK4/6i plus endocrine therapy that marks a step toward more personalized treatments in patients with Luminal B breast cancer.

Quantitative Online Monitoring of an Immobilized Enzymatic Network by Ion Mobility-Mass Spectrometry

The forward design of in vitro enzymatic reaction networks (ERNs) requires a detailed analysis of network kinetics and potentially hidden interactions between the substrates and enzymes. Although flow chemistry allows for a systematic exploration of how the networks adapt to continuously changing conditions, the analysis of the reaction products is often a bottleneck. Here, we report on the interface between a continuous stirred-tank reactor, in which an immobilized enzymatic network made of 12 enzymes is compartmentalized, and an ion mobility-mass spectrometer. Feeding uniformly 13C-labeled inputs to the enzymatic network generates all isotopically labeled reaction intermediates and products, which are individually detected by ion mobility-mass spectrometry (IMS-MS) based on their mass-to-charge ratios and inverse ion mobilities. The metabolic flux can be continuously and quantitatively monitored by diluting the ERN output with nonlabeled standards of known concentrations. The real-time quantitative data obtained by IMS-MS are then harnessed to train a model of network kinetics, which proves sufficiently predictive to control the ERN output after a single optimally designed experiment. The high resolution of the time-course data provided by this approach is an important stepping stone to design and control sizable and intricate ERNs.

Chemical reservoir computation in a self-organizing reaction network

Chemical reaction networks, such as those found in metabolism and signalling pathways, enable cells to process information from their environment1,2. Current approaches to molecular information processing and computation typically pursue digital computation models and require extensive molecular-level engineering3. Despite considerable advances, these approaches have not reached the level of information processing capabilities seen in living systems. Here we report on the discovery and implementation of a chemical reservoir computer based on the formose reaction4. We demonstrate how this complex, self-organizing chemical reaction network can perform several nonlinear classification tasks in parallel, predict the dynamics of other complex systems and achieve time-series forecasting. This in chemico information processing system provides proof of principle for the emergent computational capabilities of complex chemical reaction networks, paving the way for a new class of biomimetic information processing systems.

Efficient parameter estimation for ODE models of cellular processes using semi-quantitative data

MOTIVATION

Quantitative dynamical models facilitate the understanding of biological processes and the prediction of their dynamics. The parameters of these models are commonly estimated from experimental data. Yet, experimental data generated from different techniques do not provide direct information about the state of the system but a nonlinear (monotonic) transformation of it. For such semi-quantitative data, when this transformation is unknown, it is not apparent how the model simulations and the experimental data can be compared.

RESULTS

We propose a versatile spline-based approach for the integration of a broad spectrum of semi-quantitative data into parameter estimation. We derive analytical formulas for the gradients of the hierarchical objective function and show that this substantially increases the estimation efficiency. Subsequently, we demonstrate that the method allows for the reliable discovery of unknown measurement transformations. Furthermore, we show that this approach can significantly improve the parameter inference based on semi-quantitative data in comparison to available methods.

AVAILABILITY AND IMPLEMENTATION

Modelers can easily apply our method by using our implementation in the open-source Python Parameter EStimation TOolbox (pyPESTO) available at https://github.com/ICB-DCM/pyPESTO.

Processivity and specificity of histone acetylation by the male-specific lethal complex

Acetylation of lysine 16 of histone H4 (H4K16ac) stands out among the histone modifications, because it decompacts the chromatin fiber. The metazoan acetyltransferase MOF (KAT8) regulates transcription through H4K16 acetylation. Antibody-based studies had yielded inconclusive results about the selectivity of MOF to acetylate the H4 N-terminus. We used targeted mass spectrometry to examine the activity of MOF in the male-specific lethal core (4-MSL) complex on nucleosome array substrates. This complex is part of the Dosage Compensation Complex (DCC) that activates X-chromosomal genes in male Drosophila. During short reaction times, MOF acetylated H4K16 efficiently and with excellent selectivity. Upon longer incubation, the enzyme progressively acetylated lysines 12, 8 and 5, leading to a mixture of oligo-acetylated H4. Mathematical modeling suggests that MOF recognizes and acetylates H4K16 with high selectivity, but remains substrate-bound and continues to acetylate more N-terminal H4 lysines in a processive manner. The 4-MSL complex lacks non-coding roX RNA, a critical component of the DCC. Remarkably, addition of RNA to the reaction non-specifically suppressed H4 oligo-acetylation in favor of specific H4K16 acetylation. Because RNA destabilizes the MSL-nucleosome interaction in vitro we speculate that RNA accelerates enzyme-substrate turn-over in vivo, thus limiting the processivity of MOF, thereby increasing specific H4K16 acetylation.

Long-term monitoring of SARS-CoV-2 seroprevalence and variants in Ethiopia provides prediction for immunity and cross-immunity

Under-reporting of COVID-19 and the limited information about circulating SARS-CoV-2 variants remain major challenges for many African countries. We analyzed SARS-CoV-2 infection dynamics in Addis Ababa and Jimma, Ethiopia, focusing on reinfection, immunity, and vaccination effects. We conducted an antibody serology study spanning August 2020 to July 2022 with five rounds of data collection across a population of 4723, sequenced PCR-test positive samples, used available test positivity rates, and constructed two mathematical models integrating this data. A multivariant model explores variant dynamics identifying wildtype, alpha, delta, and omicron BA.4/5 as key variants in the study population, and cross-immunity between variants, revealing risk reductions between 24% and 69%. An antibody-level model predicts slow decay leading to sustained high antibody levels. Retrospectively, increased early vaccination might have substantially reduced infections during the delta and omicron waves in the considered group of individuals, though further vaccination now seems less impactful.

Iterative design of training data to control intricate enzymatic reaction networks

Kinetic modeling of in vitro enzymatic reaction networks is vital to understand and control the complex behaviors emerging from the nonlinear interactions inside. However, modeling is severely hampered by the lack of training data. Here, we introduce a methodology that combines an active learning-like approach and flow chemistry to efficiently create optimized datasets for a highly interconnected enzymatic reactions network with multiple sub-pathways. The optimal experimental design (OED) algorithm designs a sequence of out-of-equilibrium perturbations to maximize the information about the reaction kinetics, yielding a descriptive model that allows control of the output of the network towards any cost function. We experimentally validate the model by forcing the network to produce different product ratios while maintaining a minimum level of overall conversion efficiency. Our workflow scales with the complexity of the system and enables the optimization of previously unobtainable network outputs.

Reusable rule-based cell cycle model explains compartment-resolved dynamics of 16 observables in RPE-1 cells

The mammalian cell cycle is regulated by a well-studied but complex biochemical reaction system. Computational models provide a particularly systematic and systemic description of the mechanisms governing mammalian cell cycle control. By combining both state-of-the-art multiplexed experimental methods and powerful computational tools, this work aims at improving on these models along four dimensions: model structure, validation data, validation methodology and model reusability. We developed a comprehensive model structure of the full cell cycle that qualitatively explains the behaviour of human retinal pigment epithelial-1 cells. To estimate the model parameters, time courses of eight cell cycle regulators in two compartments were reconstructed from single cell snapshot measurements. After optimisation with a parallel global optimisation metaheuristic we obtained excellent agreements between simulations and measurements. The PEtab specification of the optimisation problem facilitates reuse of model, data and/or optimisation results. Future perturbation experiments will improve parameter identifiability and allow for testing model predictive power. Such a predictive model may aid in drug discovery for cell cycle-related disorders.

pyPESTO: a modular and scalable tool for parameter estimation for dynamic models

SUMMARY

Mechanistic models are important tools to describe and understand biological processes. However, they typically rely on unknown parameters, the estimation of which can be challenging for large and complex systems. pyPESTO is a modular framework for systematic parameter estimation, with scalable algorithms for optimization and uncertainty quantification. While tailored to ordinary differential equation problems, pyPESTO is broadly applicable to black-box parameter estimation problems. Besides own implementations, it provides a unified interface to various popular simulation and inference methods.

AVAILABILITY AND IMPLEMENTATION

pyPESTO is implemented in Python, open-source under a 3-Clause BSD license. Code and documentation are available on GitHub (https://github.com/icb-dcm/pypesto).

BioNetGMMFit: estimating parameters of a BioNetGen model from time-stamped snapshots of single cells

Mechanistic models are commonly employed to describe signaling and gene regulatory kinetics in single cells and cell populations. Recent advances in single-cell technologies have produced multidimensional datasets where snapshots of copy numbers (or abundances) of a large number of proteins and mRNA are measured across time in single cells. The availability of such datasets presents an attractive scenario where mechanistic models are validated against experiments, and estimated model parameters enable quantitative predictions of signaling or gene regulatory kinetics. To empower the systems biology community to easily estimate parameters accurately from multidimensional single-cell data, we have merged a widely used rule-based modeling software package BioNetGen, which provides a user-friendly way to code for mechanistic models describing biochemical reactions, and the recently introduced CyGMM, that uses cell-to-cell differences to improve parameter estimation for such networks, into a single software package: BioNetGMMFit. BioNetGMMFit provides parameter estimates of the model, supplied by the user in the BioNetGen markup language (BNGL), which yield the best fit for the observed single-cell, time-stamped data of cellular components. Furthermore, for more precise estimates, our software generates confidence intervals around each model parameter. BioNetGMMFit is capable of fitting datasets of increasing cell population sizes for any mechanistic model specified in the BioNetGen markup language. By streamlining the process of developing mechanistic models for large single-cell datasets, BioNetGMMFit provides an easily-accessible modeling framework designed for scale and the broader biochemical signaling community.

Haemodynamic effects of hypertension and type 2 diabetes: Insights from a 4D flow MRI-based personalized cardiovascular mathematical model

Type 2 diabetes (T2D) and hypertension increase the risk of cardiovascular diseases mediated by whole-body changes to metabolism, cardiovascular structure and haemodynamics. The haemodynamic changes related to hypertension and T2D are complex and subject-specific, however, and not fully understood. We aimed to investigate the haemodynamic mechanisms in T2D and hypertension by comparing the haemodynamics between healthy controls and subjects with T2D, hypertension, or both. For all subjects, we combined 4D flow magnetic resonance imaging data, brachial blood pressure and a cardiovascular mathematical model to create a comprehensive subject-specific analysis of central haemodynamics. When comparing the subject-specific haemodynamic parameters between the four groups, the predominant haemodynamic difference is impaired left ventricular relaxation in subjects with both T2D and hypertension compared to subjects with only T2D, only hypertension and controls. The impaired relaxation indicates that, in this cohort, the long-term changes in haemodynamic load of co-existing T2D and hypertension cause diastolic dysfunction demonstrable at rest, whereas either disease on its own does not. However, through subject-specific predictions of impaired relaxation, we show that altered relaxation alone is not enough to explain the subject-specific and group-related differences; instead, a combination of parameters is affected in T2D and hypertension. These results confirm previous studies that reported more adverse effects from the combination of T2D and hypertension compared to either disease on its own. Furthermore, this shows the potential of personalized cardiovascular models in providing haemodynamic mechanistic insights and subject-specific predictions that could aid in the understanding and treatment planning of patients with T2D and hypertension. KEY POINTS: The combination of 4D flow magnetic resonance imaging data and a cardiovascular mathematical model allows for a comprehensive analysis of subject-specific haemodynamic parameters that otherwise cannot be derived non-invasively. Using this combination, we show that diastolic dysfunction in subjects with both type 2 diabetes (T2D) and hypertension is the main group-level difference between controls, subjects with T2D, subjects with hypertension, and subjects with both T2D and hypertension. These results suggest that, in this relatively healthy population, the additional load of both hypertension and T2D affects the haemodynamic function of the left ventricle, whereas each disease on its own is not enough to cause significant effects under resting conditions. Finally, using the subject-specific model, we show that the haemodynamic effects of diastolic dysfunction alone are not sufficient to explain all the observed haemodynamic differences. Instead, additional subject-specific variations in cardiac and vascular function combine to explain the complex haemodynamics of subjects affected by hypertension and/or T2D.

Progressive disruption of hematopoietic architecture from clonal hematopoiesis to MDS

Clonal hematopoiesis of indeterminate potential (CHIP) describes the age-related acquisition of somatic mutations in hematopoietic stem/progenitor cells (HSPC) leading to clonal blood cell expansion. Although CHIP mutations drive myeloid malignancies like myelodysplastic syndromes (MDS) it is unknown if clonal expansion is attributable to changes in cell type kinetics, or involves reorganization of the hematopoietic hierarchy. Using computational modeling we analyzed differentiation and proliferation kinetics of cultured hematopoietic stem cells (HSC) from 8 healthy individuals, 7 CHIP, and 10 MDS patients. While the standard hematopoietic hierarchy explained HSPC kinetics in healthy samples, 57% of CHIP and 70% of MDS samples were best described with alternative hierarchies. Deregulated kinetics were found at various HSPC compartments with high inter-individual heterogeneity in CHIP and MDS, while altered HSC rates were most relevant in MDS. Quantifying kinetic heterogeneity in detail, we show that reorganization of the HSPC compartment is already detectable in the premalignant CHIP state.

A pathway model of glucose-stimulated insulin secretion in the pancreatic β-cell

The pancreas plays a critical role in maintaining glucose homeostasis through the secretion of hormones from the islets of Langerhans. Glucose-stimulated insulin secretion (GSIS) by the pancreatic β-cell is the main mechanism for reducing elevated plasma glucose. Here we present a systematic modeling workflow for the development of kinetic pathway models using the Systems Biology Markup Language (SBML). Steps include retrieval of information from databases, curation of experimental and clinical data for model calibration and validation, integration of heterogeneous data including absolute and relative measurements, unit normalization, data normalization, and model annotation. An important factor was the reproducibility and exchangeability of the model, which allowed the use of various existing tools. The workflow was applied to construct a novel data-driven kinetic model of GSIS in the pancreatic β-cell based on experimental and clinical data from 39 studies spanning 50 years of pancreatic, islet, and β-cell research in humans, rats, mice, and cell lines. The model consists of detailed glycolysis and phenomenological equations for insulin secretion coupled to cellular energy state, ATP dynamics and (ATP/ADP ratio). Key findings of our work are that in GSIS there is a glucose-dependent increase in almost all intermediates of glycolysis. This increase in glycolytic metabolites is accompanied by an increase in energy metabolites, especially ATP and NADH. One of the few decreasing metabolites is ADP, which, in combination with the increase in ATP, results in a large increase in ATP/ADP ratios in the β-cell with increasing glucose. Insulin secretion is dependent on ATP/ADP, resulting in glucose-stimulated insulin secretion. The observed glucose-dependent increase in glycolytic intermediates and the resulting change in ATP/ADP ratios and insulin secretion is a robust phenomenon observed across data sets, experimental systems and species. Model predictions of the glucose-dependent response of glycolytic intermediates and biphasic insulin secretion are in good agreement with experimental measurements. Our model predicts that factors affecting ATP consumption, ATP formation, hexokinase, phosphofructokinase, and ATP/ADP-dependent insulin secretion have a major effect on GSIS. In conclusion, we have developed and applied a systematic modeling workflow for pathway models that allowed us to gain insight into key mechanisms in GSIS in the pancreatic β-cell.

Reproducibility and FAIR principles: the case of a segment polarity network model

The issue of reproducibility of computational models and the related FAIR principles (findable, accessible, interoperable, and reusable) are examined in a specific test case. I analyze a computational model of the segment polarity network in Drosophila embryos published in 2000. Despite the high number of citations to this publication, 23 years later the model is barely accessible, and consequently not interoperable. Following the text of the original publication allowed successfully encoding the model for the open source software COPASI. Subsequently saving the model in the SBML format allowed it to be reused in other open source software packages. Submission of this SBML encoding of the model to the BioModels database enables its findability and accessibility. This demonstrates how the FAIR principles can be successfully enabled by using open source software, widely adopted standards, and public repositories, facilitating reproducibility and reuse of computational cell biology models that will outlive the specific software used.

Integrative modelling of reported case numbers and seroprevalence reveals time-dependent test efficiency and infectious contacts

Mathematical models have been widely used during the ongoing SARS-CoV-2 pandemic for data interpretation, forecasting, and policy making. However, most models are based on officially reported case numbers, which depend on test availability and test strategies. The time dependence of these factors renders interpretation difficult and might even result in estimation biases. Here, we present a computational modelling framework that allows for the integration of reported case numbers with seroprevalence estimates obtained from representative population cohorts. To account for the time dependence of infection and testing rates, we embed flexible splines in an epidemiological model. The parameters of these splines are estimated, along with the other parameters, from the available data using a Bayesian approach. The application of this approach to the official case numbers reported for Munich (Germany) and the seroprevalence reported by the prospective COVID-19 Cohort Munich (KoCo19) provides first estimates for the time dependence of the under-reporting factor. Furthermore, we estimate how the effectiveness of non-pharmaceutical interventions and of the testing strategy evolves over time. Overall, our results show that the integration of temporally highly resolved and representative data is beneficial for accurate epidemiological analyses.

Mechanistic model of MAPK signaling reveals how allostery and rewiring contribute to drug resistance

BRAF is prototypical of oncogenes that can be targeted therapeutically and the treatment of BRAF^V600E melanomas with RAF and MEK inhibitors results in rapid tumor regression. However, drug-induced rewiring generates a drug adapted state thought to be involved in acquired resistance and disease recurrence. In this article, we study mechanisms of adaptive rewiring in BRAF^V600E melanoma cells using an energy-based implementation of ordinary differential equation (ODE) modeling in combination with proteomic, transcriptomic and imaging data. We develop a method for causal tracing of ODE models and identify two parallel MAPK reaction channels that are differentially sensitive to RAF and MEK inhibitors due to differences in protein oligomerization and drug binding. We describe how these channels, and timescale separation between immediate-early signaling and transcriptional feedback, create a state in which the RAS-regulated MAPK channel can be activated by growth factors under conditions in which the BRAF^V600E -driven channel is fully inhibited. Further development of the approaches in this article is expected to yield a unified model of adaptive drug resistance in melanoma.

Industrial ion-exchange chromatography development using discontinuous Galerkin methods coupled with forward sensitivity analysis

In this work, a discontinuous Galerkin method coupled with forward sensitivity analysis (DG-FSA) is presented. The DG-FSA method is used to reduce computational cost required for model-based ion-exchange chromatography development using industrial load samples. As an example, the design of an anion-exchange chromatography step is considered. This step is used to purify an experimental peptide product called Protein G from Novo Nordisk A/S (Bagsværd, Denmark). The results demonstrate, that a fourth order DG-FSA method can reduce computational cost of inverse problems by a factor ×16 compared to a second (low) order DG-FSA method. Furthermore, the fourth-order DG-FSA method enable the computation of probability distributions of optimized processing conditions given uncertainty in model parameters or inputs. This analysis is not possible within a reasonable timeframe when applying the second (low) order DG-FSA method. The design procedure facilitates the optimization of the Protein G purification step. In an experimental validation run, the productivity is increased by 70% while sacrificing 4% yield at a similar purity constraint compared to an experiment with baseline performance.

Efficient computation of adjoint sensitivities at steady-state in ODE models of biochemical reaction networks

Dynamical models in the form of systems of ordinary differential equations have become a standard tool in systems biology. Many parameters of such models are usually unknown and have to be inferred from experimental data. Gradient-based optimization has proven to be effective for parameter estimation. However, computing gradients becomes increasingly costly for larger models, which are required for capturing the complex interactions of multiple biochemical pathways. Adjoint sensitivity analysis has been pivotal for working with such large models, but methods tailored for steady-state data are currently not available. We propose a new adjoint method for computing gradients, which is applicable if the experimental data include steady-state measurements. The method is based on a reformulation of the backward integration problem to a system of linear algebraic equations. The evaluation of the proposed method using real-world problems shows a speedup of total simulation time by a factor of up to 4.4. Our results demonstrate that the proposed approach can achieve a substantial improvement in computation time, in particular for large-scale models, where computational efficiency is critical.

Design and application of a kinetic model of lipid metabolism in Saccharomyces cerevisiae

Lipid biosynthesis plays a vital role in living cells and has been increasingly engineered to overproduce various lipid-based chemicals. However, owing to the tightly constrained and interconnected nature of lipid biosynthesis, both understanding and engineering of lipid metabolism remain challenging, even with the help of mathematical models. Here we report the development of a kinetic metabolic model of lipid metabolism in Saccharomyces cerevisiae that integrates fatty acid biosynthesis, glycerophospholipid metabolism, sphingolipid metabolism, storage lipids, lumped sterol synthesis, and the synthesis and transport of relevant target-chemicals, such as fatty acids and fatty alcohols. The model was trained on lipidomic data of a reference S. cerevisiae strain, single knockout mutants, and lipid overproduction strains reported in literature. The model was used to design mutants for fatty alcohol overproduction and the lipidomic analysis of the resultant mutant strains coupled with model-guided hypothesis led to discovery of a futile cycle in the triacylglycerol biosynthesis pathway. In addition, the model was used to explain successful and unsuccessful mutant designs in metabolic engineering literature. Thus, this kinetic model of lipid metabolism can not only enable the discovery of new phenomenon in lipid metabolism but also the engineering of mutant strains for overproduction of lipids.

A Practical Guide for the Efficient Formulation and Calibration of Large, Energy- and Rule-Based Models of Cellular Signal Transduction

Aberrant signal transduction leads to complex diseases such as cancer. To rationally design treatment strategies with small molecule inhibitors, computational models have to be employed. Energy- and rule-based models allow the construction of mechanistic ordinary differential equation models based on structural insights. The detailed, energy-based description often generates large models, which are difficult to calibrate on experimental data. In this chapter, we provide a detailed, interactive protocol for the programmatic formulation and calibration of such large, energy- and rule-based models of cellular signal transduction based on an example model describing the action of RAF inhibitors on MAPK signaling. An interactive version of this chapter is available as Jupyter Notebook at github.com/FFroehlich/energy_modeling_chapter .

Periportal steatosis in mice affects distinct parameters of pericentral drug metabolism

Little is known about the impact of morphological disorders in distinct zones on metabolic zonation. It was described recently that periportal fibrosis did affect the expression of CYP proteins, a set of pericentrally located drug-metabolizing enzymes. Here, we investigated whether periportal steatosis might have a similar effect. Periportal steatosis was induced in C57BL6/J mice by feeding a high-fat diet with low methionine/choline content for either two or four weeks. Steatosis severity was quantified using image analysis. Triglycerides and CYP activity were quantified in photometric or fluorometric assay. The distribution of CYP3A4, CYP1A2, CYP2D6, and CYP2E1 was visualized by immunohistochemistry. Pharmacokinetic parameters of test drugs were determined after injecting a drug cocktail (caffeine, codeine, and midazolam). The dietary model resulted in moderate to severe mixed steatosis confined to periportal and midzonal areas. Periportal steatosis did not affect the zonal distribution of CYP expression but the activity of selected CYPs was associated with steatosis severity. Caffeine elimination was accelerated by microvesicular steatosis, whereas midazolam elimination was delayed in macrovesicular steatosis. In summary, periportal steatosis affected parameters of pericentrally located drug metabolism. This observation calls for further investigations of the highly complex interrelationship between steatosis and drug metabolism and underlying signaling mechanisms.

Improving dynamic predictions with ensembles of observable models

MOTIVATION

Dynamic mechanistic modelling in systems biology has been hampered by the complexity and variability associated with the underlying interactions, and by uncertain and sparse experimental measurements. Ensemble modelling, a concept initially developed in statistical mechanics, has been introduced in biological applications with the aim of mitigating those issues. Ensemble modelling uses a collection of different models compatible with the observed data to describe the phenomena of interest. However, since systems biology models often suffer from a lack of identifiability and observability, ensembles of models are particularly unreliable when predicting non-observable states.

RESULTS

We present a strategy to assess and improve the reliability of a class of model ensembles. In particular, we consider kinetic models described using ordinary differential equations with a fixed structure. Our approach builds an ensemble with a selection of the parameter vectors found when performing parameter estimation with a global optimization metaheuristic. This technique enforces diversity during the sampling of parameter space and it can quantify the uncertainty in the predictions of state trajectories. We couple this strategy with structural identifiability and observability analysis, and when these tests detect possible prediction issues we obtain model reparameterizations that surmount them. The end result is an ensemble of models with the ability to predict the internal dynamics of a biological process. We demonstrate our approach with models of glucose regulation, cell division, circadian oscillations and the JAK-STAT signalling pathway.

AVAILABILITY AND IMPLEMENTATION

The code that implements the methodology and reproduces the results is available at https://doi.org/10.5281/zenodo.6782638.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Fides: Reliable trust-region optimization for parameter estimation of ordinary differential equation models

Ordinary differential equation (ODE) models are widely used to study biochemical reactions in cellular networks since they effectively describe the temporal evolution of these networks using mass action kinetics. The parameters of these models are rarely known a priori and must instead be estimated by calibration using experimental data. Optimization-based calibration of ODE models on is often challenging, even for low-dimensional problems. Multiple hypotheses have been advanced to explain why biochemical model calibration is challenging, including non-identifiability of model parameters, but there are few comprehensive studies that test these hypotheses, likely because tools for performing such studies are also lacking. Nonetheless, reliable model calibration is essential for uncertainty analysis, model comparison, and biological interpretation. We implemented an established trust-region method as a modular Python framework (fides) to enable systematic comparison of different approaches to ODE model calibration involving a variety of Hessian approximation schemes. We evaluated fides on a recently developed corpus of biologically realistic benchmark problems for which real experimental data are available. Unexpectedly, we observed high variability in optimizer performance among different implementations of the same mathematical instructions (algorithms). Analysis of possible sources of poor optimizer performance identified limitations in the widely used Gauss-Newton, BFGS and SR1 Hessian approximation schemes. We addressed these drawbacks with a novel hybrid Hessian approximation scheme that enhances optimizer performance and outperforms existing hybrid approaches. When applied to the corpus of test models, we found that fides was on average more reliable and efficient than existing methods using a variety of criteria. We expect fides to be broadly useful for ODE constrained optimization problems in biochemical models and to be a foundation for future methods development.

A microfluidic optimal experimental design platform for forward design of cell-free genetic networks

Cell-free protein synthesis has been widely used as a “breadboard” for design of synthetic genetic networks. However, due to a severe lack of modularity, forward engineering of genetic networks remains challenging. Here, we demonstrate how a combination of optimal experimental design and microfluidics allows us to devise dynamic cell-free gene expression experiments providing maximum information content for subsequent non-linear model identification. Importantly, we reveal that applying this methodology to a library of genetic circuits, that share common elements, further increases the information content of the data resulting in higher accuracy of model parameters. To show modularity of model parameters, we design a pulse decoder and bistable switch, and predict their behaviour both qualitatively and quantitatively. Finally, we update the parameter database and indicate that network topology affects parameter estimation accuracy. Utilizing our methodology provides us with more accurate model parameters, a necessity for forward engineering of complex genetic networks.

Characterization of cell-free genetic networks is inherently difficult. Here the authors use optimal experimental design and microfluidics to improve characterization, demonstrating modularity and predictability of parts in applied test cases.

Kinetic Modeling of Saccharomyces cerevisiae Central Carbon Metabolism: Achievements, Limitations, and Opportunities

Central carbon metabolism comprises the metabolic pathways in the cell that process nutrients into energy, building blocks and byproducts. To unravel the regulation of this network upon glucose perturbation, several metabolic models have been developed for the microorganism Saccharomyces cerevisiae. These dynamic representations have focused on glycolysis and answered multiple research questions, but no commonly applicable model has been presented. This review systematically evaluates the literature to describe the current advances, limitations, and opportunities. Different kinetic models have unraveled key kinetic glycolytic mechanisms. Nevertheless, some uncertainties regarding model topology and parameter values still limit the application to specific cases. Progressive improvements in experimental measurement technologies as well as advances in computational tools create new opportunities to further extend the model scale. Notably, models need to be made more complex to consider the multiple layers of glycolytic regulation and external physiological variables regulating the bioprocess, opening new possibilities for extrapolation and validation. Finally, the onset of new data representative of individual cells will cause these models to evolve from depicting an average cell in an industrial fermenter, to characterizing the heterogeneity of the population, opening new and unseen possibilities for industrial fermentation improvement.

Mini-batch optimization enables training of ODE models on large-scale datasets

Quantitative dynamic models are widely used to study cellular signal processing. A critical step in modelling is the estimation of unknown model parameters from experimental data. As model sizes and datasets are steadily growing, established parameter optimization approaches for mechanistic models become computationally extremely challenging. Mini-batch optimization methods, as employed in deep learning, have better scaling properties. In this work, we adapt, apply, and benchmark mini-batch optimization for ordinary differential equation (ODE) models, thereby establishing a direct link between dynamic modelling and machine learning. On our main application example, a large-scale model of cancer signaling, we benchmark mini-batch optimization against established methods, achieving better optimization results and reducing computation by more than an order of magnitude. We expect that our work will serve as a first step towards mini-batch optimization tailored to ODE models and enable modelling of even larger and more complex systems than what is currently possible.

Efficient gradient-based parameter estimation for dynamic models using qualitative data

MOTIVATION

Unknown parameters of dynamical models are commonly estimated from experimental data. However, while various efficient optimization and uncertainty analysis methods have been proposed for quantitative data, methods for qualitative data are rare and suffer from bad scaling and convergence.

RESULTS

Here, we propose an efficient and reliable framework for estimating the parameters of ordinary differential equation models from qualitative data. In this framework, we derive a semi-analytical algorithm for gradient calculation of the optimal scaling method developed for qualitative data. This enables the use of efficient gradient-based optimization algorithms. We demonstrate that the use of gradient information improves performance of optimization and uncertainty quantification on several application examples. On average, we achieve a speedup of more than one order of magnitude compared to gradient-free optimization. In addition, in some examples, the gradient-based approach yields substantially improved objective function values and quality of the fits. Accordingly, the proposed framework substantially improves the parameterization of models from qualitative data.

AVAILABILITY AND IMPLEMENTATION

The proposed approach is implemented in the open-source Python Parameter EStimation TOolbox (pyPESTO). pyPESTO is available at https://github.com/ICB-DCM/pyPESTO. All application examples and code to reproduce this study are available at https://doi.org/10.5281/zenodo.4507613.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Combination treatment optimization using a pan-cancer pathway model

The design of efficient combination therapies is a difficult key challenge in the treatment of complex diseases such as cancers. The large heterogeneity of cancers and the large number of available drugs renders exhaustive in vivo or even in vitro investigation of possible treatments impractical. In recent years, sophisticated mechanistic, ordinary differential equation-based pathways models that can predict treatment responses at a molecular level have been developed. However, surprisingly little effort has been put into leveraging these models to find novel therapies. In this paper we use for the first time, to our knowledge, a large-scale state-of-the-art pan-cancer signaling pathway model to identify candidates for novel combination therapies to treat individual cancer cell lines from various tissues (e.g., minimizing proliferation while keeping dosage low to avoid adverse side effects) and populations of heterogeneous cancer cell lines (e.g., minimizing the maximum or average proliferation across the cell lines while keeping dosage low). We also show how our method can be used to optimize the drug combinations used in sequential treatment plans-that is, optimized sequences of potentially different drug combinations-providing additional benefits. In order to solve the treatment optimization problems, we combine the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) algorithm with a significantly more scalable sampling scheme for truncated Gaussian distributions, based on a Hamiltonian Monte-Carlo method. These optimization techniques are independent of the signaling pathway model, and can thus be adapted to find treatment candidates for other complex diseases than cancers as well, as long as a suitable predictive model is available.

Benchmarking of numerical integration methods for ODE models of biological systems

Ordinary differential equation (ODE) models are a key tool to understand complex mechanisms in systems biology. These models are studied using various approaches, including stability and bifurcation analysis, but most frequently by numerical simulations. The number of required simulations is often large, e.g., when unknown parameters need to be inferred. This renders efficient and reliable numerical integration methods essential. However, these methods depend on various hyperparameters, which strongly impact the ODE solution. Despite this, and although hundreds of published ODE models are freely available in public databases, a thorough study that quantifies the impact of hyperparameters on the ODE solver in terms of accuracy and computation time is still missing. In this manuscript, we investigate which choices of algorithms and hyperparameters are generally favorable when dealing with ODE models arising from biological processes. To ensure a representative evaluation, we considered 142 published models. Our study provides evidence that most ODEs in computational biology are stiff, and we give guidelines for the choice of algorithms and hyperparameters. We anticipate that our results will help researchers in systems biology to choose appropriate numerical methods when dealing with ODE models.