Optimization and Control of Agent-Based Models in Biology: A Perspective

Chemotaxis in heterogeneous environments: A multi-agent model of decentralized gathering past obstacles

Chemotaxis, cell migration in response to chemical gradients, is known to promote self-organization of microbiological populations. However, the modeling of chemotaxis in heterogeneous environments is still limited. This study analyzes a decentralized gathering process in environments with physical as well as chemical barriers, using a multi-agent model for Disctyostelium discoideum colonies. Employing a topology-independent metric to quantify the system evolution, we study dynamical features emerging from complex social interactions. The results show that obstacles may hamper the gathering process by altering the flux of chemical signals among amoebas, acting as local topological perturbations. We also find that a minimal set of agent's rules for robust gathering does not require explicit mechanisms for obstacle sensing and avoidance; moreover, random cell movements concur in preventing multiple stable clusters and improve the gathering efficacy. Hence, we speculate that chemotactic cells can avoid obstacles without needing specialized mechanisms: tradeoffs of social interactions and individual fluctuations are sufficient to guarantee the aggregation of the whole colony past numerous obstacles.

Surrogate modeling and control of medical digital twins

The vision of personalized medicine is to identify interventions that maintain or restore a person's health based on their individual biology. Medical digital twins, computational models that integrate a wide range of health-related data about a person and can be dynamically updated, are a key technology that can help guide medical decisions. Such medical digital twin models can be high-dimensional, multi-scale, and stochastic. To be practical for healthcare applications, they often need to be simplified into low-dimensional surrogate models that can be used for optimal design of interventions. This paper introduces surrogate modeling algorithms for the purpose of optimal control applications. As a use case, we focus on agent-based models (ABMs), a common model type in biomedicine for which there are no readily available optimal control algorithms. By deriving surrogate models that are based on systems of ordinary differential equations, we show how optimal control methods can be employed to compute effective interventions, which can then be lifted back to a given ABM. The relevance of the methods introduced here extends beyond medical digital twins to other complex dynamical systems.

A design specification for Critical Illness Digital Twins (CIDTs) to cure sepsis: responding to the National Academies of Sciences, Engineering and Medicine Report "Foundational Research Gaps and Future Directions for Digital Twins"

On December 15, 2023, The National Academies of Sciences, Engineering and Medicine (NASEM) released a report entitled: "Foundational Research Gaps and Future Directions for Digital Twins." The ostensible purpose of this report was to bring some structure to the burgeoning field of digital twins by providing a working definition and a series of research challenges that need to be addressed to allow this technology to fulfill its full potential. In the work presented herein we focus on five specific findings from the NASEM Report: 1) definition of a Digital Twin, 2) using "fit-for-purpose" guidance, 3) developing novel approaches to Verification, Validation and Uncertainty Quantification (VVUQ) of Digital Twins, 4) incorporating control as an explicit purpose for a Digital Twin and 5) using a Digital Twin to guide data collection and sensor development, and describe how these findings are addressed through the design specifications for a Critical Illness Digital Twin (CIDT) aimed at curing sepsis.

Quantifying collective motion patterns in mesenchymal cell populations using topological data analysis and agent-based modeling

Fibroblasts in a confluent monolayer are known to adopt elongated morphologies in which cells are oriented parallel to their neighbors. We collected and analyzed new microscopy movies to show that confluent fibroblasts are motile and that neighboring cells often move in anti-parallel directions in a collective motion phenomenon we refer to as "fluidization" of the cell population. We used machine learning to perform cell tracking for each movie and then leveraged topological data analysis (TDA) to show that time-varying point-clouds generated by the tracks contain significant topological information content that is driven by fluidization, i.e., the anti-parallel movement of individual neighboring cells and neighboring groups of cells over long distances. We then utilized the TDA summaries extracted from each movie to perform Bayesian parameter estimation for the D'Orsgona model, an agent-based model (ABM) known to produce a wide array of different patterns, including patterns that are qualitatively similar to fluidization. Although the D'Orsgona ABM is a phenomenological model that only describes inter-cellular attraction and repulsion, the estimated region of D'Orsogna model parameter space was consistent across all movies, suggesting that a specific level of inter-cellular repulsion force at close range may be a mechanism that helps drive fluidization patterns in confluent mesenchymal cell populations.

Control of Medical Digital Twins with Artificial Neural Networks

The objective of personalized medicine is to tailor interventions to an individual patient's unique characteristics. A key technology for this purpose involves medical digital twins, computational models of human biology that can be personalized and dynamically updated to incorporate patient-specific data collected over time. Certain aspects of human biology, such as the immune system, are not easily captured with physics-based models, such as differential equations. Instead, they are often multi-scale, stochastic, and hybrid. This poses a challenge to existing model-based control and optimization approaches that cannot be readily applied to such models. Recent advances in automatic differentiation and neural-network control methods hold promise in addressing complex control problems. However, the application of these approaches to biomedical systems is still in its early stages. This work introduces dynamics-informed neural-network controllers as an alternative approach to control of medical digital twins. As a first use case for this method, the focus is on agent-based models, a versatile and increasingly common modeling platform in biomedicine. The effectiveness of the proposed neural-network control method is illustrated and benchmarked against other methods with two widely-used agent-based model types. The relevance of the method introduced here extends beyond medical digital twins to other complex dynamical systems.

SimService: a lightweight library for building simulation services in Python

SUMMARY

Integrative biological modeling requires software infrastructure to launch, interconnect, and execute simulation software components without loss of functionality. SimService is a software library that enables deploying simulations in integrated applications as memory-isolated services with interactive proxy objects in the Python programming language. SimService supports customizing the interface of proxies so that simulation developers and users alike can tailor generated simulation instances according to model, method, and integrated application.

AVAILABILITY AND IMPLEMENTATION

SimService is written in Python, is freely available on GitHub under the MIT license at https://github.com/tjsego/simservice, and is available for download via the Python Package Index (package name "simservice") and conda (package name "simservice" on the conda-forge channel).

Delayed skeletal muscle repair following inflammatory damage in simulated agent-based models of muscle regeneration

Healthy skeletal muscle undergoes repair in response to mechanically localised strains during activities such as exercise. The ability of cells to transduce the external stimuli into a cascade of cell signalling responses is important to the process of muscle repair and regeneration. In chronic myopathies such as Duchenne muscular dystrophy and inflammatory myopathies, muscle is often subject to chronic necrosis and inflammation that perturbs tissue homeostasis and leads to non-localised, widespread damage across the tissue. Here we present an agent-based model that simulates muscle repair in response to both localised eccentric contractions similar to what would be experienced during exercise, and non-localised widespread inflammatory damage that is present in chronic disease. Computational modelling of muscle repair allows for in silico exploration of phenomena related to muscle disease. In our model, widespread inflammation led to delayed clearance of tissue damage, and delayed repair for recovery of initial fibril counts at all damage levels. Macrophage recruitment was delayed and significantly higher in widespread compared to localised damage. At higher damage percentages of 10%, widespread damage led to impaired muscle regeneration and changes in muscle geometry that represented alterations commonly observed in chronic myopathies, such as fibrosis. This computational work offers insight into the progression and aetiology of inflammatory muscle diseases, and suggests a focus on the muscle regeneration cascade in understanding the progression of muscle damage in inflammatory myopathies.

Extensions of mean-field approximations for environmentally-transmitted pathogen networks

Many pathogens spread via environmental transmission, without requiring host-to-host direct contact. While models for environmental transmission exist, many are simply constructed intuitively with structures analogous to standard models for direct transmission. As model insights are generally sensitive to the underlying model assumptions, it is important that we are able understand the details and consequences of these assumptions. We construct a simple network model for an environmentally-transmitted pathogen and rigorously derive systems of ordinary differential equations (ODEs) based on different assumptions. We explore two key assumptions, namely homogeneity and independence, and demonstrate that relaxing these assumptions can lead to more accurate ODE approximations. We compare these ODE models to a stochastic implementation of the network model over a variety of parameters and network structures, demonstrating that with fewer restrictive assumptions we are able to achieve higher accuracy in our approximations and highlighting more precisely the errors produced by each assumption. We show that less restrictive assumptions lead to more complicated systems of ODEs and the potential for unstable solutions. Due to the rigour of our derivation, we are able to identify the reason behind these errors and propose potential resolutions.

Preparing for the next pandemic: Simulation-based deep reinforcement learning to discover and test multimodal control of systemic inflammation using repurposed immunomodulatory agents

Background

Preparation to address the critical gap in a future pandemic between non-pharmacological measures and the deployment of new drugs/vaccines requires addressing two factors: 1) finding virus/pathogen-agnostic pathophysiological targets to mitigate disease severity and 2) finding a more rational approach to repurposing existing drugs. It is increasingly recognized that acute viral disease severity is heavily driven by the immune response to the infection ("cytokine storm" or "cytokine release syndrome"). There exist numerous clinically available biologics that suppress various pro-inflammatory cytokines/mediators, but it is extremely difficult to identify clinically effective treatment regimens with these agents. We propose that this is a complex control problem that resists standard methods of developing treatment regimens and accomplishing this goal requires the application of simulation-based, model-free deep reinforcement learning (DRL) in a fashion akin to training successful game-playing artificial intelligences (AIs). This proof-of-concept study determines if simulated sepsis (e.g. infection-driven cytokine storm) can be controlled in the absence of effective antimicrobial agents by targeting cytokines for which FDA-approved biologics currently exist.

Methods

We use a previously validated agent-based model, the Innate Immune Response Agent-based Model (IIRABM), for control discovery using DRL. DRL training used a Deep Deterministic Policy Gradient (DDPG) approach with a clinically plausible control interval of 6 hours with manipulation of six cytokines for which there are existing drugs: Tumor Necrosis Factor (TNF), Interleukin-1 (IL-1), Interleukin-4 (IL-4), Interleukin-8 (IL-8), Interleukin-12 (IL-12) and Interferon-γ(IFNg).

Results

DRL trained an AI policy that could improve outcomes from a baseline Recovered Rate of 61% to one with a Recovered Rate of 90% over ~21 days simulated time. This DRL policy was then tested on four different parameterizations not seen in training representing a range of host and microbe characteristics, demonstrating a range of improvement in Recovered Rate by +33% to +56.

Discussion

The current proof-of-concept study demonstrates that significant disease severity mitigation can potentially be accomplished with existing anti-mediator drugs, but only through a multi-modal, adaptive treatment policy requiring implementation with an AI. While the actual clinical implementation of this approach is a projection for the future, the current goal of this work is to inspire the development of a research ecosystem that marries what is needed to improve the simulation models with the development of the sensing/assay technologies to collect the data needed to iteratively refine those models.

ChemChaste: Simulating spatially inhomogeneous biochemical reaction-diffusion systems for modeling cell-environment feedbacks

Background

Spatial organization plays an important role in the function of many biological systems, from cell fate specification in animal development to multistep metabolic conversions in microbial communities. The study of such systems benefits from the use of spatially explicit computational models that combine a discrete description of cells with a continuum description of one or more chemicals diffusing within a surrounding bulk medium. These models allow the in silico testing and refinement of mechanistic hypotheses. However, most existing models of this type do not account for concurrent bulk and intracellular biochemical reactions and their possible coupling.

Conclusions

Here, we describe ChemChaste, an extension for the open-source C++ computational biology library Chaste. ChemChaste enables the spatial simulation of both multicellular and bulk biochemistry by expanding on Chaste’s existing capabilities. In particular, ChemChaste enables (i) simulation of an arbitrary number of spatially diffusing chemicals, (ii) spatially heterogeneous chemical diffusion coefficients, and (iii) inclusion of both bulk and intracellular biochemical reactions and their coupling. ChemChaste also introduces a file-based interface that allows users to define the parameters relating to these functional features without the need to interact directly with Chaste’s core C++ code. We describe ChemChaste and demonstrate its functionality using a selection of chemical and biochemical exemplars, with a focus on demonstrating increased ability in modeling bulk chemical reactions and their coupling with intracellular reactions.

Availability and implementation

ChemChaste version 1.0 is a free, open-source C++ library, available via GitHub at https://github.com/OSS-Lab/ChemChaste under the BSD license, on the Zenodo archive at zendodo doi, as well as on BioTools (biotools:chemchaste) and SciCrunch (RRID:SCR022208) databases.

Preparing for the next COVID: Deep Reinforcement Learning trained Artificial Intelligence discovery of multi-modal immunomodulatory control of systemic inflammation in the absence of effective anti-microbials

Background

Despite a great deal of interest in the application of artificial intelligence (AI) to sepsis/critical illness, most current approaches are limited in their potential impact: prediction models do not (and cannot) address the lack of effective therapeutics and current approaches to enhancing the treatment of sepsis focus on optimizing the application of existing interventions, and thus cannot address the development of new treatment options/modalities. The inability to test new therapeutic applications was highlighted by the generally unsatisfactory results from drug repurposing efforts in COVID-19.

Hypothesis

Addressing this challenge requires the application of simulation-based, model-free deep reinforcement learning (DRL) in a fashion akin to training the game-playing AIs. We have previously demonstrated the potential of this method in the context of bacterial sepsis in which the microbial infection is responsive to antibiotic therapy. The current work addresses the control problem of multi-modal, adaptive immunomodulation in the circumstance where there is no effective anti-pathogen therapy (e.g., in a novel viral pandemic or in the face of resistant microbes).

Methods

This is a proof-of-concept study that determines the controllability of sepsis without the ability to pharmacologically suppress the pathogen. We use as a surrogate system a previously validated agent-based model, the Innate Immune Response Agent-based Model (IIRABM), for control discovery using DRL. The DRL algorithm 'trains' an AI on simulations of infection where both the control and observation spaces are limited to operating upon the defined immune mediators included in the IIRABM (a total of 11). Policies were learned using the Deep Deterministic Policy Gradient approach, with the objective function being a return to baseline system health.

Results

DRL trained an AI policy that improved system mortality from 85% to 10.4%. Control actions affected every one of the 11 targetable cytokines and could be divided into those with static/unchanging controls and those with variable/adaptive controls. Adaptive controls primarily targeted 3 different aspects of the immune response: 2nd order pro-inflammation governing TH1/TH2 balance, primary anti-inflammation, and inflammatory cell proliferation.

Discussion

The current treatment of sepsis is hampered by limitations in therapeutic options able to affect the biology of sepsis. This is heightened in circumstances where no effective antimicrobials exist, as was the case for COVID-19. Current AI methods are intrinsically unable to address this problem; doing so requires training AIs in contexts that fully represent the counterfactual space of potential treatments. The synthetic data needed for this task is only possible through the use of high-resolution, mechanism-based simulations. Finally, being able to treat sepsis will require a reorientation as to the sensing and actuating requirements needed to develop these simulations and bring them to the bedside.

In silico design of recombinant multi-epitope vaccine against influenza A virus

Background

Influenza A virus is one of the leading causes of annual mortality. The emerging of novel escape variants of the influenza A virus is still a considerable challenge in the annual process of vaccine production. The evolution of vaccines ranks among the most critical successes in medicine and has eradicated numerous infectious diseases. Recently, multi-epitope vaccines, which are based on the selection of epitopes, have been increasingly investigated.

Results

This study utilized an immunoinformatic approach to design a recombinant multi-epitope vaccine based on a highly conserved epitope of hemagglutinin, neuraminidase, and membrane matrix proteins with fewer changes or mutate over time. The potential B cells, cytotoxic T lymphocytes (CTL), and CD4 T cell epitopes were identified. The recombinant multi-epitope vaccine was designed using specific linkers and a proper adjuvant. Moreover, some bioinformatics online servers and datasets were used to evaluate the immunogenicity and chemical properties of selected epitopes. In addition, Universal Immune System Simulator (UISS) in silico trial computational framework was run after influenza exposure and recombinant multi-epitope vaccine administration, showing a good immune response in terms of immunoglobulins of class G (IgG), T Helper 1 cells (TH1), epithelial cells (EP) and interferon gamma (IFN-g) levels. Furthermore, after a reverse translation (i.e., convertion of amino acid sequence to nucleotide one) and codon optimization phase, the optimized sequence was placed between the two EcoRV/MscI restriction sites in the PET32a⁺ vector.

Conclusions

The proposed “Recombinant multi-epitope vaccine” was predicted with unique and acceptable immunological properties. This recombinant multi-epitope vaccine can be successfully expressed in the prokaryotic system and accepted for immunogenicity studies against the influenza virus at the in silico level. The multi-epitope vaccine was then tested with the Universal Immune System Simulator (UISS) in silico trial platform. It revealed slight immune protection against the influenza virus, shedding the light that a multistep bioinformatics approach including molecular and cellular level is mandatory to avoid inappropriate vaccine efficacy predictions.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12859-022-04581-6.

A laboratory and simulation platform to integrate individual life history traits and population dynamics

Understanding populations is important because they are a fundamental level of biological organization. Individual traits such as aging and lifespan interact in complex ways to determine birth and death and thereby influence population dynamics. However, we lack a deep understanding of the relationships between individual traits and population dynamics. To address this challenge, we established a laboratory population using the model organism C. elegans and an individual-based computational simulation informed by measurements of real worms. The simulation realistically models individual worms and the behavior of the laboratory population. To elucidate the role of aging in population dynamics, we analyzed old age as a cause of death and showed, using computer simulations, that it was influenced by maximum lifespan, rate of adult culling, and progeny number/food stability. Notably, populations displayed a tipping point for aging as the primary cause of adult death. Our work establishes a conceptual framework that could be used for better understanding why certain animals die of old age in the wild.

Coarse-grained Stochastic Model of Myosin-Driven Vesicles into Dendritic Spines

We study the dynamics of membrane vesicle motor transport into dendritic spines, which are bulbous intracellular compartments in neurons that play a key role in transmitting signals between neurons. We consider the stochastic analog of the vesicle transport model in [Park and Fai, The Dynamics of Vesicles Driven Into Closed Constrictions by Molecular Motors. Bull. Math. Biol. 82, 141 (2020)]. The stochastic version, which may be considered as an agent-based model, relies mostly on the action of individual myosin motors to produce vesicle motion. To aid in our analysis, we coarse-grain this agent-based model using a master equation combined with a partial differential equation describing the probability of local motor positions. We confirm through convergence studies that the coarse-graining captures the essential features of bistability in velocity (observed in experiments) and waiting-time distributions to switch between steady-state velocities. Interestingly, these results allow us to reformulate the translocation problem in terms of conditional mean first passage times for a run-and-tumble particle moving on a finite domain with absorbing boundaries at the two ends. We conclude by presenting numerical and analytical calculations of vesicle translocation.

Derivation of continuum models from discrete models of mechanical forces in cell populations

In certain discrete models of populations of biological cells, the mechanical forces between the cells are center based or vertex based on the microscopic level where each cell is individually represented. The cells are circular or spherical in a center based model and polygonal or polyhedral in a vertex based model. On a higher, macroscopic level, the time evolution of the density of the cells is described by partial differential equations (PDEs). We derive relations between the modelling on the micro and macro levels in one, two, and three dimensions by regarding the micro model as a discretization of a PDE for conservation of mass on the macro level. The forces in the micro model correspond on the macro level to a gradient of the pressure scaled by quantities depending on the cell geometry. The two levels of modelling are compared in numerical experiments in one and two dimensions.

The Use of Artificial Neural Networks to Forecast the Behavior of Agent-Based Models of Pathophysiology: An Example Utilizing an Agent-Based Model of Sepsis

Introduction: Disease states are being characterized at finer and finer levels of resolution via biomarker or gene expression profiles, while at the same time. Machine learning (ML) is increasingly used to analyze and potentially classify or predict the behavior of biological systems based on such characterization. As ML applications are extremely data-intensive, given the relative sparsity of biomedical data sets ML training of artificial neural networks (ANNs) often require the use of synthetic training data. Agent-based models (ABMs) that incorporate known biological mechanisms and their associated stochastic properties are a potential means of generating synthetic data. Herein we present an example of ML used to train an artificial neural network (ANN) as a surrogate system used to predict the time evolution of an ABM focusing on the clinical condition of sepsis. Methods: The disease trajectories for clinical sepsis, in terms of temporal cytokine and phenotypic dynamics, can be interpreted as a random dynamical system. The Innate Immune Response Agent-based Model (IIRABM) is a well-established model that utilizes known cellular and molecular rules to simulate disease trajectories corresponding to clinical sepsis. We have utilized two distinct neural network architectures, Long Short-Term Memory and Multi-Layer Perceptron, to take a time sequence of five measurements of eleven IIRABM simulated serum cytokine concentrations as input and to return both the future cytokine trajectories as well as an aggregate metric representing the patient's state of health. Results: The ANNs predicted model trajectories with the expected amount of error, due to stochasticity in the simulation, and recognizing that the mapping from a specific cytokine profile to a state-of-health is not unique. The Multi-Layer Perceptron neural network, generated predictions with a more accurate forecasted trajectory cone. Discussion: This work serves as a proof-of-concept for the use of ANNs to predict disease progression in sepsis as represented by an ABM. The findings demonstrate that multicellular systems with intrinsic stochasticity can be approximated with an ANN, but that forecasting a specific trajectory of the system requires sequential updating of the system state to provide a rolling forecast horizon.

Bayesian uncertainty quantification for data-driven equation learning

Equation learning aims to infer differential equation models from data. While a number of studies have shown that differential equation models can be successfully identified when the data are sufficiently detailed and corrupted with relatively small amounts of noise, the relationship between observation noise and uncertainty in the learned differential equation models remains unexplored. We demonstrate that for noisy datasets there exists great variation in both the structure of the learned differential equation models and their parameter values. We explore how to exploit multiple datasets to quantify uncertainty in the learned models, and at the same time draw mechanistic conclusions about the target differential equations. We showcase our results using simulation data from a relatively straightforward agent-based model (ABM) which has a well-characterized partial differential equation description that provides highly accurate predictions of averaged ABM behaviours in relevant regions of parameter space. Our approach combines equation learning methods with Bayesian inference approaches so that a quantification of uncertainty can be given by the posterior parameter distribution of the learned model.

Nested active learning for efficient model contextualization and parameterization: pathway to generating simulated populations using multi-scale computational models

There is increasing interest in the use of mechanism-based multi-scale computational models (such as agent-based models (ABMs)) to generate simulated clinical populations in order to discover and evaluate potential diagnostic and therapeutic modalities. The description of the environment in which a biomedical simulation operates (model context) and parameterization of internal model rules (model content) requires the optimization of a large number of free parameters. In this work, we utilize a nested active learning (AL) workflow to efficiently parameterize and contextualize an ABM of systemic inflammation used to examine sepsis. Contextual parameter space was examined using four parameters external to the model's rule set. The model's internal parameterization, which represents gene expression and associated cellular behaviors, was explored through the augmentation or inhibition of signaling pathways for 12 signaling mediators associated with inflammation and wound healing. We have implemented a nested AL approach in which the clinically relevant (CR) model environment space for a given internal model parameterization is mapped using a small Artificial Neural Network (ANN). The outer AL level workflow is a larger ANN that uses AL to efficiently regress the volume and centroid location of the CR space given by a single internal parameterization. We have reduced the number of simulations required to efficiently map the CR parameter space of this model by approximately 99%. In addition, we have shown that more complex models with a larger number of variables may expect further improvements in efficiency.

Learning differential equation models from stochastic agent-based model simulations

Agent-based models provide a flexible framework that is frequently used for modelling many biological systems, including cell migration, molecular dynamics, ecology and epidemiology. Analysis of the model dynamics can be challenging due to their inherent stochasticity and heavy computational requirements. Common approaches to the analysis of agent-based models include extensive Monte Carlo simulation of the model or the derivation of coarse-grained differential equation models to predict the expected or averaged output from the agent-based model. Both of these approaches have limitations, however, as extensive computation of complex agent-based models may be infeasible, and coarse-grained differential equation models can fail to accurately describe model dynamics in certain parameter regimes. We propose that methods from the equation learning field provide a promising, novel and unifying approach for agent-based model analysis. Equation learning is a recent field of research from data science that aims to infer differential equation models directly from data. We use this tutorial to review how methods from equation learning can be used to learn differential equation models from agent-based model simulations. We demonstrate that this framework is easy to use, requires few model simulations, and accurately predicts model dynamics in parameter regions where coarse-grained differential equation models fail to do so. We highlight these advantages through several case studies involving two agent-based models that are broadly applicable to biological phenomena: a birth-death-migration model commonly used to explore cell biology experiments and a susceptible-infected-recovered model of infectious disease spread.

Optimization of Agent-Based Models Through Coarse-Graining: A Case Study in Microbial Ecology

Optimization and control are important objectives across biology and biomedicine, and mathematical models are a key enabling technology. This paper reports a computational study of model-based multi-objective optimization in the setting of microbial ecology, using agent-based models. This modeling framework is well-suited to the field, but is not amenable to standard control-theoretic approaches. Furthermore, due to computational complexity, simulation-based optimization approaches are often challenging to implement. This paper presents the results of an approach that combines control-dependent coarse-graining with Pareto optimization, applied to two models of multi-species bacterial biofilms. It shows that this approach can be successful for models whose computational complexity prevents effective simulation-based optimization.

Receptor Models of Phagocytosis: The Effect of Target Shape

Phagocytosis is a remarkably complex process, requiring simultaneous organisation of the cell membrane, the cytoskeleton, receptors and various signalling molecules. As can often be the case, mathematical modelling is able to penetrate some of this complexity, identifying the key biophysical components and generating understanding that would take far longer with a purely experimental approach. This chapter will review a particularly important class of phagocytosis model, championed in recent years, that primarily focuses on the role of receptors during the engulfment process. These models are pertinent to a host of unsolved questions in the subject, including the rate of cup growth during uptake, the role of both intra- and extracellular noise, and the precise differences between phagocytosis and other forms of endocytosis. In particular, this chapter will focus on the effect of target shape and orientation, including how these influence the rate and final outcome of phagocytic engulfment.

Applying optimal control theory to complex epidemiological models to inform real-world disease management

Mathematical models provide a rational basis to inform how, where and when to control disease. Assuming an accurate spatially explicit simulation model can be fitted to spread data, it is straightforward to use it to test the performance of a range of management strategies. However, the typical complexity of simulation models and the vast set of possible controls mean that only a small subset of all possible strategies can ever be tested. An alternative approach-optimal control theory-allows the best control to be identified unambiguously. However, the complexity of the underpinning mathematics means that disease models used to identify this optimum must be very simple. We highlight two frameworks for bridging the gap between detailed epidemic simulations and optimal control theory: open-loop and model predictive control. Both these frameworks approximate a simulation model with a simpler model more amenable to mathematical analysis. Using an illustrative example model, we show the benefits of using feedback control, in which the approximation and control are updated as the epidemic progresses. Our work illustrates a new methodology to allow the insights of optimal control theory to inform practical disease management strategies, with the potential for application to diseases of humans, animals and plants. This article is part of the theme issue 'Modelling infectious disease outbreaks in humans, animals and plants: epidemic forecasting and control'. This theme issue is linked with the earlier issue 'Modelling infectious disease outbreaks in humans, animals and plants: approaches and important themes'.

Applying optimal control theory to a spatial simulation model of sudden oak death: ongoing surveillance protects tanoak while conserving biodiversity

Sudden oak death has devastated tree populations across California. However, management might still slow disease spread at local scales. We demonstrate how to unambiguously characterize effective, local management strategies using a detailed, spatially explicit simulation model of spread in a single forest stand. This pre-existing, parameterized simulation is approximated here by a carefully calibrated, non-spatial model, explicitly constructed to be sufficiently simple to allow optimal control theory (OCT) to be applied. By lifting management strategies from the approximate model to the detailed simulation, effective time-dependent controls can be identified. These protect tanoak-a culturally and ecologically important species-while conserving forest biodiversity within a limited budget. We also consider model predictive control, in which both the approximating model and optimal control are repeatedly updated as the epidemic progresses. This allows management which is robust to both parameter uncertainty and systematic differences between simulation and approximate models. Including the costs of disease surveillance then introduces an optimal intensity of surveillance. Our study demonstrates that successful control of sudden oak death is likely to rely on adaptive strategies updated via ongoing surveillance. More broadly, it illustrates how OCT can inform effective real-world management, even when underpinning disease spread models are highly complex.

Trends in mathematical modeling of host-pathogen interactions

Pathogenic microorganisms entail enormous problems for humans, livestock, and crop plants. A better understanding of the different infection strategies of the pathogens enables us to derive optimal treatments to mitigate infectious diseases or develop vaccinations preventing the occurrence of infections altogether. In this review, we highlight the current trends in mathematical modeling approaches and related methods used for understanding host-pathogen interactions. Since these interactions can be described on vastly different temporal and spatial scales as well as abstraction levels, a variety of computational and mathematical approaches are presented. Particular emphasis is placed on dynamic optimization, game theory, and spatial modeling, as they are attracting more and more interest in systems biology. Furthermore, these approaches are often combined to illuminate the complexities of the interactions between pathogens and their host. We also discuss the phenomena of molecular mimicry and crypsis as well as the interplay between defense and counter defense. As a conclusion, we provide an overview of method characteristics to assist non-experts in their decision for modeling approaches and interdisciplinary understanding.

Macroprudential Policy in a Heterogeneous Environment-An Application of Agent-Based Approach in Systemic Risk Modelling

Assessment of welfare effects of macroprudential policy seems the most important application of the Dynamic Stochastic General Equilibrium (DSGE) framework of macro-modelling. In particular, the DSGE-3D model, with three layers of default (3D), was developed and used by the European Systemic Risk Board and European Central Bank as a reference tool to formally model the financial cycle as well as to analyze effects of macroprudential policies. Despite the extreme importance of incorporating financial constraints in Real Business Cycle (RBC) models, the resulting DSGE-3D construct still embraces the representative agent idea, making serious analyses of diversity of economic entities impossible. In this paper, we present an alternative to DSGE modelling that seriously departs from the assumption of the representativeness of agents. Within an Agent Based Modelling (ABM) framework, we build an environment suitable for performing counterfactual simulations of the impact of macroprudential policy on the economy, financial system and society. We contribute to the existing literature by presenting an ABM model with broad insight into heterogeneity of agents. We show the stabilizing effects of macroprudential policies in the case of economic or financial distress.

Predicting the artificial immunity induced by RUTI® vaccine against tuberculosis using universal immune system simulator (UISS)

BACKGROUND

Tuberculosis (TB) represents a worldwide cause of mortality (it infects one third of the world's population) affecting mostly developing countries, including India, and recently also developed ones due to the increased mobility of the world population and the evolution of different new bacterial strains capable to provoke multi-drug resistance phenomena. Currently, antitubercular drugs are unable to eradicate subpopulations of Mycobacterium tuberculosis (MTB) bacilli and therapeutic vaccinations have been postulated to overcome some of the critical issues related to the increase of drug-resistant forms and the difficult clinical and public health management of tuberculosis patients. The Horizon 2020 EC funded project "In Silico Trial for Tuberculosis Vaccine Development" (STriTuVaD) to support the identification of new therapeutic interventions against tuberculosis through novel in silico modelling of human immune responses to disease and vaccines, thereby drastically reduce the cost of clinical trials in this critical sector of public healthcare.

RESULTS

We present the application of the Universal Immune System Simulator (UISS) computational modeling infrastructure as a disease model for TB. The model is capable to simulate the main features and dynamics of the immune system activities i.e., the artificial immunity induced by RUTI® vaccine, a polyantigenic liposomal therapeutic vaccine made of fragments of Mycobacterium tuberculosis cells (FCMtb). Based on the available data coming from phase II Clinical Trial in subjects with latent tuberculosis infection treated with RUTI® and isoniazid, we generated simulation scenarios through validated data in order to tune UISS accordingly to STriTuVaD objectives. The first case simulates the establishment of MTB latent chronic infection with some typical granuloma formation; the second scenario deals with a reactivation phase during latent chronic infection; the third represents the latent chronic disease infection scenario during RUTI® vaccine administration.

CONCLUSIONS

The application of this computational modeling strategy helpfully contributes to simulate those mechanisms involved in the early stages and in the progression of tuberculosis infection and to predict how specific therapeutical strategies will act in this scenario. In view of these results, UISS owns the capacity to open the door for a prompt integration of in silico methods within the pipeline of clinical trials, supporting and guiding the testing of treatments in patients affected by tuberculosis.

Agent-based models of inflammation in translational systems biology: A decade later

Agent-based modeling is a rule-based, discrete-event, and spatially explicit computational modeling method that employs computational objects that instantiate the rules and interactions among the individual components ("agents") of system. Agent-based modeling is well suited to translating into a computational model the knowledge generated from basic science research, particularly with respect to translating across scales the mechanisms of cellular behavior into aggregated cell population dynamics manifesting at the tissue and organ level. This capacity has made agent-based modeling an integral method in translational systems biology (TSB), an approach that uses multiscale dynamic computational modeling to explicitly represent disease processes in a clinically relevant fashion. The initial work in the early 2000s using agent-based models (ABMs) in TSB focused on examining acute inflammation and its intersection with wound healing; the decade since has seen vast growth in both the application of agent-based modeling to a wide array of disease processes as well as methodological advancements in the use and analysis of ABM. This report presents an update on an earlier review of ABMs in TSB and presents examples of exciting progress in the modeling of various organs and diseases that involve inflammation. This review also describes developments that integrate the use of ABMs with cutting-edge technologies such as high-performance computing, machine learning, and artificial intelligence, with a view toward the future integration of these methodologies. This article is categorized under: Translational, Genomic, and Systems Medicine > Translational Medicine Models of Systems Properties and Processes > Mechanistic Models Models of Systems Properties and Processes > Organ, Tissue, and Physiological Models Models of Systems Properties and Processes > Organismal Models.

Computational and Systems Immunology: A Student's Perspective

The big data revolution has transformed the landscape of immunology research. As inaugural students of Stanford's new Computational and Systems Immunology PhD track, we share our experiences and advice with other institutions considering a similar program.

Applying optimal control theory to complex epidemiological models to inform real-world disease management

Mathematical models provide a rational basis to inform how, where and when to control disease. Assuming an accurate spatially explicit simulation model can be fitted to spread data, it is straightforward to use it to test the performance of a range of management strategies. However, the typical complexity of simulation models and the vast set of possible controls mean that only a small subset of all possible strategies can ever be tested. An alternative approach-optimal control theory-allows the best control to be identified unambiguously. However, the complexity of the underpinning mathematics means that disease models used to identify this optimum must be very simple. We highlight two frameworks for bridging the gap between detailed epidemic simulations and optimal control theory: open-loop and model predictive control. Both these frameworks approximate a simulation model with a simpler model more amenable to mathematical analysis. Using an illustrative example model, we show the benefits of using feedback control, in which the approximation and control are updated as the epidemic progresses. Our work illustrates a new methodology to allow the insights of optimal control theory to inform practical disease management strategies, with the potential for application to diseases of humans, animals and plants. This article is part of the theme issue 'Modelling infectious disease outbreaks in humans, animals and plants: epidemic forecasting and control'. This theme issue is linked with the earlier issue 'Modelling infectious disease outbreaks in humans, animals and plants: approaches and important themes'.

Deep Reinforcement Learning and Simulation as a Path Toward Precision Medicine

Traditionally, precision medicine involves classifying patients to identify subpopulations that respond favorably to specific therapeutics. We pose precision medicine as a dynamic feedback control problem, where treatment administered to a patient is guided by measurements taken during the course of treatment. We consider sepsis, a life-threatening condition in which dysregulation of the immune system causes tissue damage. We leverage an existing simulation of the innate immune response to infection and apply deep reinforcement learning (DRL) to discover an adaptive personalized treatment policy that specifies effective multicytokine therapy to simulated sepsis patients based on systemic measurements. The learned policy achieves a dramatic reduction in mortality rate over a set of 500 simulated patients relative to standalone antibiotic therapy. Advantages of our approach are threefold: (1) the use of simulation allows exploring therapeutic strategies beyond clinical practice and available data, (2) advances in DRL accommodate learning complex therapeutic strategies for complex biological systems, and (3) optimized treatments respond to a patient's individual disease progression over time, therefore, capturing both differences across patients and the inherent randomness of disease progression within a single patient. We hope that this work motivates both considering adaptive personalized multicytokine mediation therapy for sepsis and exploiting simulation with DRL for precision medicine more broadly.

Issues in Reproducible Simulation Research

In recent years, serious concerns have arisen about reproducibility in science. Estimates of the cost of irreproducible preclinical studies range from 28 billion USD per year in the USA alone (Freedman et al. in PLoS Biol 13(6):e1002165, 2015) to over 200 billion USD per year worldwide (Chalmers and Glasziou in Lancet 374:86-89, 2009). The situation in the social sciences is not very different: Reproducibility in psychological research, for example, has been estimated to be below 50% as well (Open Science Collaboration in Science 349:6251, 2015). Less well studied is the issue of reproducibility of simulation research. A few replication studies of agent-based models, however, suggest the problem for computational modeling may be more severe than for laboratory experiments (Willensky and Rand in JASSS 10(4):2, 2007; Donkin et al. in Environ Model Softw 92:142-151, 2017; Bajracharya and Duboz in: Proceedings of the symposium on theory of modeling and simulation-DEVS integrative M&S symposium, pp 6-11, 2013). In this perspective, we discuss problems of reproducibility in agent-based simulations of life and social science problems, drawing on best practices research in computer science and in wet-lab experiment design and execution to suggest some ways to improve simulation research practice.

Logistical constraints lead to an intermediate optimum in outbreak response vaccination

Dynamic models in disease ecology have historically evaluated vaccination strategies under the assumption that they are implemented homogeneously in space and time. However, this approach fails to formally account for operational and logistical constraints inherent in the distribution of vaccination to the population at risk. Thus, feedback between the dynamic processes of vaccine distribution and transmission might be overlooked. Here, we present a spatially explicit, stochastic Susceptible-Infected-Recovered-Vaccinated model that highlights the density-dependence and spatial constraints of various diffusive strategies of vaccination during an outbreak. The model integrates an agent-based process of disease spread with a partial differential process of vaccination deployment. We characterize the vaccination response in terms of a diffusion rate that describes the distribution of vaccination to the population at risk from a central location. This generates an explicit trade-off between slow diffusion, which concentrates effort near the central location, and fast diffusion, which spreads a fixed vaccination effort thinly over a large area. We use stochastic simulation to identify the optimum vaccination diffusion rate as a function of population density, interaction scale, transmissibility, and vaccine intensity. Our results show that, conditional on a timely response, the optimal strategy for minimizing outbreak size is to distribute vaccination resource at an intermediate rate: fast enough to outpace the epidemic, but slow enough to achieve local herd immunity. If the response is delayed, however, the optimal strategy for minimizing outbreak size changes to a rapidly diffusive distribution of vaccination effort. The latter may also result in significantly larger outbreaks, thus suggesting a benefit of allocating resources to timely outbreak detection and response.

It has long been recognized that an epidemic of infectious disease can be prevented if a sufficient proportion of the susceptible population is vaccinated in advance. This logic also holds for vaccine-based outbreak response to stop an outbreak of a novel, or re-emerging pathogen, but with an important caveat. If vaccination is used in response to an outbreak, then it will necessarily take time to achieve the required level of vaccination coverage, during which time the outbreak may continue to spread. Thus, one must consider the logistical and operational constraints of vaccine distribution to assess the ability of outbreak response vaccination to slow or stop an advancing epidemic. We develop a simple mathematical framework for representing vaccine distribution in response to an epidemic and solve for the optimal distribution strategy under realistic constraints of total vaccination effort. Focused deployment near the outbreak epicenter concentrates resources in the area most in need, but may allow the outbreak to spread outside of the response zone. Broad deployment over the whole population may spread vaccination resources too thin, creating shortages and delays at the local scale that fail to prevent the advancing epidemic. Thus we found that, in general, the best strategy is an intermediate optimum that deploys vaccine neither too slow to prevent escape from the outbreak epicenter, nor too fast to spread resources too thin. The specific optimum rate for any given outbreak depends on the infectiousness of the pathogen, the population density, the range of contacts amongst individuals, the timeliness of the response, and the vaccine intensity. This insight only emerges from linking an epidemic model with a realistic model of outbreak response and highlights the need for further work to merge operations research with epidemic models to develop operationally relevant response strategies.

Examining the controllability of sepsis using genetic algorithms on an agent-based model of systemic inflammation

Sepsis, a manifestation of the body's inflammatory response to injury and infection, has a mortality rate of between 28%-50% and affects approximately 1 million patients annually in the United States. Currently, there are no therapies targeting the cellular/molecular processes driving sepsis that have demonstrated the ability to control this disease process in the clinical setting. We propose that this is in great part due to the considerable heterogeneity of the clinical trajectories that constitute clinical "sepsis," and that determining how this system can be controlled back into a state of health requires the application of concepts drawn from the field of dynamical systems. In this work, we consider the human immune system to be a random dynamical system, and investigate its potential controllability using an agent-based model of the innate immune response (the Innate Immune Response ABM or IIRABM) as a surrogate, proxy system. Simulation experiments with the IIRABM provide an explanation as to why single/limited cytokine perturbations at a single, or small number of, time points is unlikely to significantly improve the mortality rate of sepsis. We then use genetic algorithms (GA) to explore and characterize multi-targeted control strategies for the random dynamical immune system that guide it from a persistent, non-recovering inflammatory state (functionally equivalent to the clinical states of systemic inflammatory response syndrome (SIRS) or sepsis) to a state of health. We train the GA on a single parameter set with multiple stochastic replicates, and show that while the calculated results show good generalizability, more advanced strategies are needed to achieve the goal of adaptive personalized medicine. This work evaluating the extent of interventions needed to control a simplified surrogate model of sepsis provides insight into the scope of the clinical challenge, and can serve as a guide on the path towards true "precision control" of sepsis.

Stochastic Individual-Based Modeling of Bacterial Growth and Division Using Flow Cytometry

A realistic description of the variability in bacterial growth and division is critical to produce reliable predictions of safety risks along the food chain. Individual-based modeling of bacteria provides the theoretical framework to deal with this variability, but it requires information about the individual behavior of bacteria inside populations. In this work, we overcome this problem by estimating the individual behavior of bacteria from population statistics obtained with flow cytometry. For this objective, a stochastic individual-based modeling framework is defined based on standard assumptions during division and exponential growth. The unknown single-cell parameters required for running the individual-based modeling simulations, such as cell size growth rate, are estimated from the flow cytometry data. Instead of using directly the individual-based model, we make use of a modified Fokker-Plank equation. This only equation simulates the population statistics in function of the unknown single-cell parameters. We test the validity of the approach by modeling the growth and division of Pediococcus acidilactici within the exponential phase. Estimations reveal the statistics of cell growth and division using only data from flow cytometry at a given time. From the relationship between the mother and daughter volumes, we also predict that P. acidilactici divide into two successive parallel planes.

Applying optimization algorithms to tuberculosis antibiotic treatment regimens

Introduction

Tuberculosis (TB), one of the most common infectious diseases, requires treatment with multiple antibiotics taken over at least 6 months. This long treatment often results in poor patient-adherence, which can lead to the emergence of multi-drug resistant TB. New antibiotic treatment strategies are sorely needed. New antibiotics are being developed or repurposed to treat TB, but as there are numerous potential antibiotics, dosing sizes and potential schedules, the regimen design space for new treatments is too large to search exhaustively. Here we propose a method that combines an agent-based multi-scale model capturing TB granuloma formation with algorithms for mathematical optimization to identify optimal TB treatment regimens.

Methods

We define two different single-antibiotic treatments to compare the efficiency and accuracy in predicting optimal treatment regimens of two optimization algorithms: genetic algorithms (GA) and surrogate-assisted optimization through radial basis function (RBF) networks. We also illustrate the use of RBF networks to optimize double-antibiotic treatments.

Results

We found that while GAs can locate optimal treatment regimens more accurately, RBF networks provide a more practical strategy to TB treatment optimization with fewer simulations, and successfully estimated optimal double-antibiotic treatment regimens.

Conclusions

Our results indicate surrogate-assisted optimization can locate optimal TB treatment regimens from a larger set of antibiotics, doses and schedules, and could be applied to solve optimization problems in other areas of research using systems biology approaches. Our findings have important implications for the treatment of diseases like TB that have lengthy protocols or for any disease that requires multiple drugs.

Sepsis reconsidered: Identifying novel metrics for behavioral landscape characterization with a high-performance computing implementation of an agent-based model

OBJECTIVES

Sepsis affects nearly 1 million people in the United States per year, has a mortality rate of 28-50% and requires more than $20 billion a year in hospital costs. Over a quarter century of research has not yielded a single reliable diagnostic test or a directed therapeutic agent for sepsis. Central to this insufficiency is the fact that sepsis remains a clinical/physiological diagnosis representing a multitude of molecularly heterogeneous pathological trajectories. Advances in computational capabilities offered by High Performance Computing (HPC) platforms call for an evolution in the investigation of sepsis to attempt to define the boundaries of traditional research (bench, clinical and computational) through the use of computational proxy models. We present a novel investigatory and analytical approach, derived from how HPC resources and simulation are used in the physical sciences, to identify the epistemic boundary conditions of the study of clinical sepsis via the use of a proxy agent-based model of systemic inflammation.

DESIGN

Current predictive models for sepsis use correlative methods that are limited by patient heterogeneity and data sparseness. We address this issue by using an HPC version of a system-level validated agent-based model of sepsis, the Innate Immune Response ABM (IIRBM), as a proxy system in order to identify boundary conditions for the possible behavioral space for sepsis. We then apply advanced analysis derived from the study of Random Dynamical Systems (RDS) to identify novel means for characterizing system behavior and providing insight into the tractability of traditional investigatory methods.

RESULTS

The behavior space of the IIRABM was examined by simulating over 70 million sepsis patients for up to 90 days in a sweep across the following parameters: cardio-respiratory-metabolic resilience; microbial invasiveness; microbial toxigenesis; and degree of nosocomial exposure. In addition to using established methods for describing parameter space, we developed two novel methods for characterizing the behavior of a RDS: Probabilistic Basins of Attraction (PBoA) and Stochastic Trajectory Analysis (STA). Computationally generated behavioral landscapes demonstrated attractor structures around stochastic regions of behavior that could be described in a complementary fashion through use of PBoA and STA. The stochasticity of the boundaries of the attractors highlights the challenge for correlative attempts to characterize and classify clinical sepsis.

CONCLUSIONS

HPC simulations of models like the IIRABM can be used to generate approximations of the behavior space of sepsis to both establish "boundaries of futility" with respect to existing investigatory approaches and apply system engineering principles to investigate the general dynamic properties of sepsis to provide a pathway for developing control strategies. The issues that bedevil the study and treatment of sepsis, namely clinical data sparseness and inadequate experimental sampling of system behavior space, are fundamental to nearly all biomedical research, manifesting in the "Crisis of Reproducibility" at all levels. HPC-augmented simulation-based research offers an investigatory strategy more consistent with that seen in the physical sciences (which combine experiment, theory and simulation), and an opportunity to utilize the leading advances in HPC, namely deep machine learning and evolutionary computing, to form the basis of an iterative scientific process to meet the full promise of Precision Medicine (right drug, right patient, right time).