Validation Through Collaboration: Encouraging Team Efforts to Ensure Internal and External Validity of Computational Models of Biochemical Pathways

Computational modelling of biochemical reaction pathways is an increasingly important part of neuroscience research. In order to be useful, computational models need to be valid in two senses: First, they need to be consistent with experimental data and able to make testable predictions (external validity). Second, they need to be internally consistent and independently reproducible (internal validity). Here, we discuss both types of validity and provide a brief overview of tools and technologies used to ensure they are met. We also suggest the introduction of new collaborative technologies to ensure model validity: an incentivised experimental database for external validity and reproducibility audits for internal validity. Both rely on FAIR principles and on collaborative science practices.

JD419, a Staphylococcus aureus Phage With a Unique Morphology and Broad Host Range

Phage therapy represents a possible treatment option to cure infections caused by multidrug-resistant bacteria, including methicillin and vancomycin-resistant Staphylococcus aureus, to which most antibiotics have become ineffective. In the present study, we report the isolation and complete characterization of a novel phage named JD219 exhibiting a broad host range able to infect 61 of 138 clinical strains of S. aureus tested, which included MRSA strains as well. The phage JD419 exhibits a unique morphology with an elongated capsid and a flexible tail. To evaluate the potential of JD419 to be used as a therapeutic phage, we tested the ability of the phage particles to remain infectious after treatment exceeding physiological pH or temperature. The activity was retained at pH values of 6.0–8.0 and below 50°C. As phages can contain virulence genes, JD419’s complete genome was sequenced. The 45509 bp genome is predicted to contain 65 ORFs, none of which show homology to any known virulence or antibiotic resistance genes. Genome analysis indicates that JD419 is a temperate phage, despite observing rapid replication and lysis of host strains. Following the recent advances in synthetic biology, JD419 can be modified by gene engineering to remove prophage-related genes, preventing potential lysogeny, in order to be deployed as a therapeutic phage.

High neural activity accelerates the decline of cognitive plasticity with age in Caenorhabditis elegans

The ability to learn progressively declines with age. Neural hyperactivity has been implicated in impairing cognitive plasticity with age, but the molecular mechanisms remain elusive. Here, we show that chronic excitation of the Caenorhabditis elegans O₂-sensing neurons during ageing causes a rapid decline of experience-dependent plasticity in response to environmental O₂ concentration, whereas sustaining lower activity of O₂-sensing neurons retains plasticity with age. We demonstrate that neural activity alters the ageing trajectory in the transcriptome of O₂-sensing neurons, and our data suggest that high-activity neurons redirect resources from maintaining plasticity to sustaining continuous firing. Sustaining plasticity with age requires the K⁺-dependent Na⁺/Ca²⁺ (NCKX) exchanger, whereas the decline of plasticity with age in high-activity neurons acts through calmodulin and the scaffold protein Kidins220. Our findings demonstrate directly that the activity of neurons alters neuronal homeostasis to govern the age-related decline of neural plasticity and throw light on the mechanisms involved.

The first 10 years of the international coordination network for standards in systems and synthetic biology (COMBINE)

This paper presents a report on outcomes of the 10th Computational Modeling in Biology Network (COMBINE) meeting that was held in Heidelberg, Germany, in July of 2019. The annual event brings together researchers, biocurators and software engineers to present recent results and discuss future work in the area of standards for systems and synthetic biology. The COMBINE initiative coordinates the development of various community standards and formats for computational models in the life sciences. Over the past 10 years, COMBINE has brought together standard communities that have further developed and harmonized their standards for better interoperability of models and data. COMBINE 2019 was co-located with a stakeholder workshop of the European EU-STANDS4PM initiative that aims at harmonized data and model standardization for in silico models in the field of personalized medicine, as well as with the FAIRDOM PALs meeting to discuss findable, accessible, interoperable and reusable (FAIR) data sharing. This report briefly describes the work discussed in invited and contributed talks as well as during breakout sessions. It also highlights recent advancements in data, model, and annotation standardization efforts. Finally, this report concludes with some challenges and opportunities that this community will face during the next 10 years.

A multi-state model of the CaMKII dodecamer suggests a role for calmodulin in maintenance of autophosphorylation

Ca2+/calmodulin-dependent protein kinase II (CaMKII) accounts for up to 2 percent of all brain protein and is essential to memory function. CaMKII activity is known to regulate dynamic shifts in the size and signaling strength of neuronal connections, a process known as synaptic plasticity. Increasingly, computational models are used to explore synaptic plasticity and the mechanisms regulating CaMKII activity. Conventional modeling approaches may exclude biophysical detail due to the impractical number of state combinations that arise when explicitly monitoring the conformational changes, ligand binding, and phosphorylation events that occur on each of the CaMKII holoenzyme's subunits. To manage the combinatorial explosion without necessitating bias or loss in biological accuracy, we use a specialized syntax in the software MCell to create a rule-based model of a twelve-subunit CaMKII holoenzyme. Here we validate the rule-based model against previous experimental measures of CaMKII activity and investigate molecular mechanisms of CaMKII regulation. Specifically, we explore how Ca2+/CaM-binding may both stabilize CaMKII subunit activation and regulate maintenance of CaMKII autophosphorylation. Noting that Ca2+/CaM and protein phosphatases bind CaMKII at nearby or overlapping sites, we compare model scenarios in which Ca2+/CaM and protein phosphatase do or do not structurally exclude each other's binding to CaMKII. Our results suggest a functional mechanism for the so-called "CaM trapping" phenomenon, wherein Ca2+/CaM may structurally exclude phosphatase binding and thereby prolong CaMKII autophosphorylation. We conclude that structural protection of autophosphorylated CaMKII by Ca2+/CaM may be an important mechanism for regulation of synaptic plasticity.

FindSim: A Framework for Integrating Neuronal Data and Signaling Models

Current experiments touch only small but overlapping parts of very complex subcellular signaling networks in neurons. Even with modern optical reporters and pharmacological manipulations, a given experiment can only monitor and control a very small subset of the diverse, multiscale processes of neuronal signaling. We have developed FindSim (Framework for Integrating Neuronal Data and SIgnaling Models) to anchor models to structured experimental datasets. FindSim is a framework for integrating many individual electrophysiological and biochemical experiments with large, multiscale models so as to systematically refine and validate the model. We use a structured format for encoding the conditions of many standard physiological and pharmacological experiments, specifying which parts of the model are involved, and comparing experiment outcomes with model output. A database of such experiments is run against successive generations of composite cellular models to iteratively improve the model against each experiment, while retaining global model validity. We suggest that this toolchain provides a principled and scalable way to tackle model complexity and diversity of data sources.

Cooperative binding mitigates the high-dose hook effect

Background

The high-dose hook effect (also called prozone effect) refers to the observation that if a multivalent protein acts as a linker between two parts of a protein complex, then increasing the amount of linker protein in the mixture does not always increase the amount of fully formed complex. On the contrary, at a high enough concentration range the amount of fully formed complex actually decreases. It has been observed that allosterically regulated proteins seem less susceptible to this effect. The aim of this study was two-fold: First, to investigate the mathematical basis of how allostery mitigates the prozone effect. And second, to explore the consequences of allostery and the high-dose hook effect using the example of calmodulin, a calcium-sensing protein that regulates the switch between long-term potentiation and long-term depression in neurons.

Results

We use a combinatorial model of a “perfect linker protein” (with infinite binding affinity) to mathematically describe the hook effect and its behaviour under allosteric conditions. We show that allosteric regulation does indeed mitigate the high-dose hook effect. We then turn to calmodulin as a real-life example of an allosteric protein. Using kinetic simulations, we show that calmodulin is indeed subject to a hook effect. We also show that this effect is stronger in the presence of the allosteric activator Ca ²⁺/calmodulin-dependent kinase II (CaMKII), because it reduces the overall cooperativity of the calcium-calmodulin system. It follows that, surprisingly, there are conditions where increased amounts of allosteric activator actually decrease the activity of a protein.

Conclusions

We show that cooperative binding can indeed act as a protective mechanism against the hook effect. This will have implications in vivo where the extent of cooperativity of a protein can be modulated, for instance, by allosteric activators or inhibitors. This can result in counterintuitive effects of decreased activity with increased concentrations of both the allosteric protein itself and its allosteric activators.

Electronic supplementary material

The online version of this article (doi:10.1186/s12918-017-0447-8) contains supplementary material, which is available to authorized users.

Cooperativity: a competition of definitions

Cooperativity as a concept is easy to grasp intuitively, but surprisingly hard to define. Two recent papers shed light on the issue and continue the debate on how best to define cooperative binding.

The systems biology format converter

BACKGROUND

Interoperability between formats is a recurring problem in systems biology research. Many tools have been developed to convert computational models from one format to another. However, they have been developed independently, resulting in redundancy of efforts and lack of synergy.

RESULTS

Here we present the System Biology Format Converter (SBFC), which provide a generic framework to potentially convert any format into another. The framework currently includes several converters translating between the following formats: SBML, BioPAX, SBGN-ML, Matlab, Octave, XPP, GPML, Dot, MDL and APM. This software is written in Java and can be used as a standalone executable or web service.

CONCLUSIONS

The SBFC framework is an evolving software project. Existing converters can be used and improved, and new converters can be easily added, making SBFC useful to both modellers and developers. The source code and documentation of the framework are freely available from the project web site.

The quantitative methods boot camp: teaching quantitative thinking and computing skills to graduate students in the life sciences

The past decade has seen a rapid increase in the ability of biologists to collect large amounts of data. It is therefore vital that research biologists acquire the necessary skills during their training to visualize, analyze, and interpret such data. To begin to meet this need, we have developed a “boot camp” in quantitative methods for biology graduate students at Harvard Medical School. The goal of this short, intensive course is to enable students to use computational tools to visualize and analyze data, to strengthen their computational thinking skills, and to simulate and thus extend their intuition about the behavior of complex biological systems. The boot camp teaches basic programming using biological examples from statistics, image processing, and data analysis. This integrative approach to teaching programming and quantitative reasoning motivates students’ engagement by demonstrating the relevance of these skills to their work in life science laboratories. Students also have the opportunity to analyze their own data or explore a topic of interest in more detail. The class is taught with a mixture of short lectures, Socratic discussion, and in-class exercises. Students spend approximately 40% of their class time working through both short and long problems. A high instructor-to-student ratio allows students to get assistance or additional challenges when needed, thus enhancing the experience for students at all levels of mastery. Data collected from end-of-course surveys from the last five offerings of the course (between 2012 and 2014) show that students report high learning gains and feel that the course prepares them for solving quantitative and computational problems they will encounter in their research. We outline our course here which, together with the course materials freely available online under a Creative Commons License, should help to facilitate similar efforts by others.

Multi-state modeling of biomolecules

Multi-state modeling of biomolecules refers to a series of techniques used to represent and compute the behavior of biological molecules or complexes that can adopt a large number of possible functional states. Biological signaling systems often rely on complexes of biological macromolecules that can undergo several functionally significant modifications that are mutually compatible. Thus, they can exist in a very large number of functionally different states. Modeling such multi-state systems poses two problems: the problem of how to describe and specify a multi-state system (the “specification problem”) and the problem of how to use a computer to simulate the progress of the system over time (the “computation problem”). To address the specification problem, modelers have in recent years moved away from explicit specification of all possible states and towards rule-based formalisms that allow for implicit model specification, including the κ-calculus [1], BioNetGen [2]–[5], the Allosteric Network Compiler [6], and others [7], [8]. To tackle the computation problem, they have turned to particle-based methods that have in many cases proved more computationally efficient than population-based methods based on ordinary differential equations, partial differential equations, or the Gillespie stochastic simulation algorithm[9], [10]. Given current computing technology, particle-based methods are sometimes the only possible option. Particle-based simulators fall into two further categories: nonspatial simulators, such as StochSim [11], DYNSTOC [12], RuleMonkey [9], [13], and the Network-Free Stochastic Simulator (NFSim) [14], and spatial simulators, including Meredys [15], SRSim [16], [17], and MCell [18]–[20]. Modelers can thus choose from a variety of tools, the best choice depending on the particular problem. Development of faster and more powerful methods is ongoing, promising the ability to simulate ever more complex signaling processes in the future.

Biophysical properties of presynaptic short-term plasticity in hippocampal neurons: insights from electrophysiology, imaging and mechanistic models

Hippocampal neurons show different types of short-term plasticity (STP). Some exhibit facilitation of their synaptic responses and others depression. In this review we discuss presynaptic biophysical properties behind heterogeneity in STP in hippocampal neurons such as alterations in vesicle priming and docking, fusion, neurotransmitter filling and vesicle replenishment. We look into what types of information electrophysiology, imaging and mechanistic models have given about the time scales and relative impact of the different properties on STP with an emphasis on the use of mechanistic models as complementary tools to experimental procedures. Taken together this tells us that it is possible for a multitude of different mechanisms to underlie the same STP pattern, even though some are more important in specific cases, and that mechanistic models can be used to integrate the biophysical properties to see which mechanisms are more important in specific cases of STP.

Calcium input frequency, duration and amplitude differentially modulate the relative activation of calcineurin and CaMKII

NMDA receptor dependent long-term potentiation (LTP) and long-term depression (LTD) are two prominent forms of synaptic plasticity, both of which are triggered by post-synaptic calcium elevation. To understand how calcium selectively stimulates two opposing processes, we developed a detailed computational model and performed simulations with different calcium input frequencies, amplitudes, and durations. We show that with a total amount of calcium ions kept constant, high frequencies of calcium pulses stimulate calmodulin more efficiently. Calcium input activates both calcineurin and Ca²⁺/calmodulin-dependent protein kinase II (CaMKII) at all frequencies, but increased frequencies shift the relative activation from calcineurin to CaMKII. Irrespective of amplitude and duration of the inputs, the total amount of calcium ions injected adjusts the sensitivity of the system to calcium input frequencies. At a given frequency, the quantity of CaMKII activated is proportional to the total amount of calcium. Thus, an input of a small amount of calcium at high frequencies can induce the same activation of CaMKII as a larger amount, at lower frequencies. Finally, the extent of activation of CaMKII signals with high calcium frequency is further controlled by other factors, including the availability of calmodulin, and by the potency of phosphatase inhibitors.

Structural analysis and stochastic modelling suggest a mechanism for calmodulin trapping by CaMKII

Activation of CaMKII by calmodulin and the subsequent maintenance of constitutive activity through autophosphorylation at threonine residue 286 (Thr286) are thought to play a major role in synaptic plasticity. One of the effects of autophosphorylation at Thr286 is to increase the apparent affinity of CaMKII for calmodulin, a phenomenon known as "calmodulin trapping". It has previously been suggested that two binding sites for calmodulin exist on CaMKII, with high and low affinities, respectively. We built structural models of calmodulin bound to both of these sites. Molecular dynamics simulation showed that while binding of calmodulin to the supposed low-affinity binding site on CaMKII is compatible with closing (and hence, inactivation) of the kinase, and could even favour it, binding to the high-affinity site is not. Stochastic simulations of a biochemical model showed that the existence of two such binding sites, one of them accessible only in the active, open conformation, would be sufficient to explain calmodulin trapping by CaMKII. We can explain the effect of CaMKII autophosphorylation at Thr286 on calmodulin trapping: It stabilises the active state and therefore makes the high-affinity binding site accessible. Crucially, a model with only one binding site where calmodulin binding and CaMKII inactivation are strictly mutually exclusive cannot reproduce calmodulin trapping. One of the predictions of our study is that calmodulin binding in itself is not sufficient for CaMKII activation, although high-affinity binding of calmodulin is.

BioModels Database: An enhanced, curated and annotated resource for published quantitative kinetic models

Background

Quantitative models of biochemical and cellular systems are used to answer a variety of questions in the biological sciences. The number of published quantitative models is growing steadily thanks to increasing interest in the use of models as well as the development of improved software systems and the availability of better, cheaper computer hardware. To maximise the benefits of this growing body of models, the field needs centralised model repositories that will encourage, facilitate and promote model dissemination and reuse. Ideally, the models stored in these repositories should be extensively tested and encoded in community-supported and standardised formats. In addition, the models and their components should be cross-referenced with other resources in order to allow their unambiguous identification.

Description

BioModels Database http://www.ebi.ac.uk/biomodels/ is aimed at addressing exactly these needs. It is a freely-accessible online resource for storing, viewing, retrieving, and analysing published, peer-reviewed quantitative models of biochemical and cellular systems. The structure and behaviour of each simulation model distributed by BioModels Database are thoroughly checked; in addition, model elements are annotated with terms from controlled vocabularies as well as linked to relevant data resources. Models can be examined online or downloaded in various formats. Reaction network diagrams generated from the models are also available in several formats. BioModels Database also provides features such as online simulation and the extraction of components from large scale models into smaller submodels. Finally, the system provides a range of web services that external software systems can use to access up-to-date data from the database.

Conclusions

BioModels Database has become a recognised reference resource for systems biology. It is being used by the community in a variety of ways; for example, it is used to benchmark different simulation systems, and to study the clustering of models based upon their annotations. Model deposition to the database today is advised by several publishers of scientific journals. The models in BioModels Database are freely distributed and reusable; the underlying software infrastructure is also available from SourceForge https://sourceforge.net/projects/biomodels/ under the GNU General Public License.

BioModels Database: An enhanced, curated and annotated resource for published quantitative kinetic models

BACKGROUND

Quantitative models of biochemical and cellular systems are used to answer a variety of questions in the biological sciences. The number of published quantitative models is growing steadily thanks to increasing interest in the use of models as well as the development of improved software systems and the availability of better, cheaper computer hardware. To maximise the benefits of this growing body of models, the field needs centralised model repositories that will encourage, facilitate and promote model dissemination and reuse. Ideally, the models stored in these repositories should be extensively tested and encoded in community-supported and standardised formats. In addition, the models and their components should be cross-referenced with other resources in order to allow their unambiguous identification.

DESCRIPTION

BioModels Database http://www.ebi.ac.uk/biomodels/ is aimed at addressing exactly these needs. It is a freely-accessible online resource for storing, viewing, retrieving, and analysing published, peer-reviewed quantitative models of biochemical and cellular systems. The structure and behaviour of each simulation model distributed by BioModels Database are thoroughly checked; in addition, model elements are annotated with terms from controlled vocabularies as well as linked to relevant data resources. Models can be examined online or downloaded in various formats. Reaction network diagrams generated from the models are also available in several formats. BioModels Database also provides features such as online simulation and the extraction of components from large scale models into smaller submodels. Finally, the system provides a range of web services that external software systems can use to access up-to-date data from the database.

CONCLUSIONS

BioModels Database has become a recognised reference resource for systems biology. It is being used by the community in a variety of ways; for example, it is used to benchmark different simulation systems, and to study the clustering of models based upon their annotations. Model deposition to the database today is advised by several publishers of scientific journals. The models in BioModels Database are freely distributed and reusable; the underlying software infrastructure is also available from SourceForge https://sourceforge.net/projects/biomodels/ under the GNU General Public License.

Computing phenomenologic Adair-Klotz constants from microscopic MWC parameters

BACKGROUND

Modellers using the MWC allosteric framework have often found it difficult to validate their models. Indeed many experiments are not conducted with the notion of alternative conformations in mind and therefore do not (or cannot) measure relevant microscopic constant and parameters. Instead, experimentalists widely use the Adair-Klotz approach in order to describe their experimental data.

RESULTS

We propose a way of computing apparent Adair-Klotz constants from microscopic association constants and allosteric parameters of a generalised concerted model with two different states (R and T), with an arbitrary number of non-equivalent ligand binding sites. We apply this framework to compute Adair-Klotz constants from existing models of calmodulin and hemoglobin, two extreme cases of the general framework.

CONCLUSION

The validation of computational models requires methods to relate model parameters to experimentally observable quantities. We provide such a method for comparing generalised MWC allosteric models to experimentally determined Adair-Klotz constants.