FLAMINGO: calibrating large cosmological hydrodynamical simulations with machine learning

To fully take advantage of the data provided by large-scale structure surveys, we need to quantify the potential impact of baryonic effects, such as feedback from active galactic nuclei (AGN) and star formation, on cosmological observables. In simulations, feedback processes originate on scales that remain unresolved. Therefore, they need to be sourced via subgrid models that contain free parameters. We use machine learning to calibrate the AGN and stellar feedback models for the FLAMINGO (Fullhydro Large-scale structure simulations with All-sky Mapping for the Interpretation of Next Generation Observations) cosmological hydrodynamical simulations. Using Gaussian process emulators trained on Latin hypercubes of 32 smaller volume simulations, we model how the galaxy stellar mass function (SMF) and cluster gas fractions change as a function of the subgrid parameters. The emulators are then fit to observational data, allowing for the inclusion of potential observational biases. We apply our method to the three different FLAMINGO resolutions, spanning a factor of 64 in particle mass, recovering the observed relations within the respective resolved mass ranges. We also use the emulators, which link changes in subgrid parameters to changes in observables, to find models that skirt or exceed the observationally allowed range for cluster gas fractions and the SMF. Our method enables us to define model variations in terms of the data that they are calibrated to rather than the values of specific subgrid parameters. This approach is useful, because subgrid parameters are typically not directly linked to particular observables, and predictions for a specific observable are influenced by multiple subgrid parameters.

Author Correction: Ab initio predictions link the neutron skin of 208Pb to nuclear forces

[This corrects the article DOI: 10.1038/s41567-022-01715-8.].

First observation of 28O

Subjecting a physical system to extreme conditions is one of the means often used to obtain a better understanding and deeper insight into its organization and structure. In the case of the atomic nucleus, one such approach is to investigate isotopes that have very different neutron-to-proton (N/Z) ratios than in stable nuclei. Light, neutron-rich isotopes exhibit the most asymmetric N/Z ratios and those lying beyond the limits of binding, which undergo spontaneous neutron emission and exist only as very short-lived resonances (about 10-21 s), provide the most stringent tests of modern nuclear-structure theories. Here we report on the first observation of 28O and 27O through their decay into 24O and four and three neutrons, respectively. The 28O nucleus is of particular interest as, with the Z = 8 and N = 20 magic numbers1,2, it is expected in the standard shell-model picture of nuclear structure to be one of a relatively small number of so-called 'doubly magic' nuclei. Both 27O and 28O were found to exist as narrow, low-lying resonances and their decay energies are compared here to the results of sophisticated theoretical modelling, including a large-scale shell-model calculation and a newly developed statistical approach. In both cases, the underlying nuclear interactions were derived from effective field theories of quantum chromodynamics. Finally, it is shown that the cross-section for the production of 28O from a 29F beam is consistent with it not exhibiting a closed N = 20 shell structure.

Demonstrating multi-country calibration of a tuberculosis model using new history matching and emulation package - hmer

Infectious disease models are widely used by epidemiologists to improve the understanding of transmission dynamics and disease natural history, and to predict the possible effects of interventions. As the complexity of such models increases, however, it becomes increasingly challenging to robustly calibrate them to empirical data. History matching with emulation is a calibration method that has been successfully applied to such models, but has not been widely used in epidemiology partly due to the lack of available software. To address this issue, we developed a new, user-friendly R package hmer to simply and efficiently perform history matching with emulation. In this paper, we demonstrate the first use of hmer for calibrating a complex deterministic model for the country-level implementation of tuberculosis vaccines to 115 low- and middle-income countries. The model was fit to 9-13 target measures, by varying 19-22 input parameters. Overall, 105 countries were successfully calibrated. Among the remaining countries, hmer visualisation tools, combined with derivative emulation methods, provided strong evidence that the models were misspecified and could not be calibrated to the target ranges. This work shows that hmer can be used to simply and rapidly calibrate a complex model to data from over 100 countries, making it a useful addition to the epidemiologist's calibration tool-kit.

Bayesian emulation and history matching of JUNE

We analyze JUNE: a detailed model of COVID-19 transmission with high spatial and demographic resolution, developed as part of the RAMP initiative. JUNE requires substantial computational resources to evaluate, making model calibration and general uncertainty analysis extremely challenging. We describe and employ the uncertainty quantification approaches of Bayes linear emulation and history matching to mimic JUNE and to perform a global parameter search, hence identifying regions of parameter space that produce acceptable matches to observed data, and demonstrating the capability of such methods. This article is part of the theme issue 'Technical challenges of modelling real-life epidemics and examples of overcoming these'.

Bayesian emulation and history matching of JUNE

We analyze JUNE: a detailed model of COVID-19 transmission with high spatial and demographic resolution, developed as part of the RAMP initiative. JUNE requires substantial computational resources to evaluate, making model calibration and general uncertainty analysis extremely challenging. We describe and employ the uncertainty quantification approaches of Bayes linear emulation and history matching to mimic JUNE and to perform a global parameter search, hence identifying regions of parameter space that produce acceptable matches to observed data, and demonstrating the capability of such methods. This article is part of the theme issue 'Technical challenges of modelling real-life epidemics and examples of overcoming these'.

Ab initio predictions link the neutron skin of 208Pb to nuclear forces

Heavy atomic nuclei have an excess of neutrons over protons, which leads to the formation of a neutron skin whose thickness is sensitive to details of the nuclear force. This links atomic nuclei to properties of neutron stars, thereby relating objects that differ in size by orders of magnitude. The nucleus 208Pb is of particular interest because it exhibits a simple structure and is experimentally accessible. However, computing such a heavy nucleus has been out of reach for ab initio theory. By combining advances in quantum many-body methods, statistical tools and emulator technology, we make quantitative predictions for the properties of 208Pb starting from nuclear forces that are consistent with symmetries of low-energy quantum chromodynamics. We explore 109 different nuclear force parameterizations via history matching, confront them with data in select light nuclei and arrive at an importance-weighted ensemble of interactions. We accurately reproduce bulk properties of 208Pb and determine the neutron skin thickness, which is smaller and more precise than a recent extraction from parity-violating electron scattering but in agreement with other experimental probes. This work demonstrates how realistic two- and three-nucleon forces act in a heavy nucleus and allows us to make quantitative predictions across the nuclear landscape.

Complex model calibration through emulation, a worked example for a stochastic epidemic model

Uncertainty quantification is a formal paradigm of statistical estimation that aims to account for all uncertainties inherent in the modelling process of real-world complex systems. The methods are directly applicable to stochastic models in epidemiology, however they have thus far not been widely used in this context. In this paper, we provide a tutorial on uncertainty quantification of stochastic epidemic models, aiming to facilitate the use of the uncertainty quantification paradigm for practitioners with other complex stochastic simulators of applied systems. We provide a formal workflow including the important decisions and considerations that need to be taken, and illustrate the methods over a simple stochastic epidemic model of UK SARS-CoV-2 transmission and patient outcome. We also present new approaches to visualisation of outputs from sensitivity analyses and uncertainty quantification more generally in high input and/or output dimensions.

June: open-source individual-based epidemiology simulation

We introduce June, an open-source framework for the detailed simulation of epidemics on the basis of social interactions in a virtual population constructed from geographically granular census data, reflecting age, sex, ethnicity and socio-economic indicators. Interactions between individuals are modelled in groups of various sizes and properties, such as households, schools and workplaces, and other social activities using social mixing matrices. June provides a suite of flexible parametrizations that describe infectious diseases, how they are transmitted and affect contaminated individuals. In this paper, we apply June to the specific case of modelling the spread of COVID-19 in England. We discuss the quality of initial model outputs which reproduce reported hospital admission and mortality statistics at national and regional levels as well as by age strata.

June: open-source individual-based epidemiology simulation

We introduce June, an open-source framework for the detailed simulation of epidemics on the basis of social interactions in a virtual population constructed from geographically granular census data, reflecting age, sex, ethnicity and socio-economic indicators. Interactions between individuals are modelled in groups of various sizes and properties, such as households, schools and workplaces, and other social activities using social mixing matrices. June provides a suite of flexible parametrizations that describe infectious diseases, how they are transmitted and affect contaminated individuals. In this paper, we apply June to the specific case of modelling the spread of COVID-19 in England. We discuss the quality of initial model outputs which reproduce reported hospital admission and mortality statistics at national and regional levels as well as by age strata.

Understanding hormonal crosstalk in Arabidopsis root development via emulation and history matching

A major challenge in plant developmental biology is to understand how plant growth is coordinated by interacting hormones and genes. To meet this challenge, it is important to not only use experimental data, but also formulate a mathematical model. For the mathematical model to best describe the true biological system, it is necessary to understand the parameter space of the model, along with the links between the model, the parameter space and experimental observations. We develop sequential history matching methodology, using Bayesian emulation, to gain substantial insight into biological model parameter spaces. This is achieved by finding sets of acceptable parameters in accordance with successive sets of physical observations. These methods are then applied to a complex hormonal crosstalk model for Arabidopsis root growth. In this application, we demonstrate how an initial set of 22 observed trends reduce the volume of the set of acceptable inputs to a proportion of 6.1 × 10-7 of the original space. Additional sets of biologically relevant experimental data, each of size 5, reduce the size of this space by a further three and two orders of magnitude respectively. Hence, we provide insight into the constraints placed upon the model structure by, and the biological consequences of, measuring subsets of observations.

Choice of time horizon critical in estimating costs and effects of changes to HIV programmes

BACKGROUND

Uganda changed its antiretroviral therapy guidelines in 2014, increasing the CD4 threshold for antiretroviral therapy initiation from 350 cells/μl to 500 cells/μl. We investigate what effect this change in policy is likely to have on HIV incidence, morbidity, and programme costs, and estimate the cost-effectiveness of the change over different time horizons.

METHODS

We used a complex individual-based model of HIV transmission and antiretroviral therapy scale-up in Uganda. 100 model fits were generated by fitting the model to 51 demographic, sexual behaviour, and epidemiological calibration targets, varying 96 input parameters, using history matching with model emulation. An additional 19 cost and disability weight parameters were varied during the analysis of the model results. For each model fit, the model was run to 2030, with and without the change in threshold to 500 cells/μl.

RESULTS

The change in threshold led to a 9.7% (90% plausible range: 4.3%-15.0%) reduction in incidence in 2030, and averted 278,944 (118,452-502,790) DALYs, at a total cost of $28M (-$142M to +$195M). The cost per disability adjusted life year (DALY) averted fell over time, from $3238 (-$125 to +$29,969) in 2014 to $100 (-$499 to +$785) in 2030. The change in threshold was cost-effective (cost <3×Uganda's per capita GDP per DALY averted) by 2018, and highly cost-effective (cost

CONCLUSIONS

Model results suggest that the change in threshold is unlikely to have been cost-effective to date, but is likely to be highly cost-effective in Uganda by 2030. The time horizon needs to be chosen carefully when projecting intervention effects. Large amounts of uncertainty in our results demonstrates the need to comprehensively incorporate uncertainties in model parameterisation.

Collapse

Key Words Collapse

MESH Headings

• Anti-HIV Agents/economics
• Anti-HIV Agents/therapeutic use
• CD4 Lymphocyte Count
• Cost-Benefit Analysis
• Female
• HIV Infections/drug therapy
• HIV Infections/economics
• HIV Infections/epidemiology
• Health Policy/economics
• Humans
• Incidence
• Male
• Models, Economic
• Quality-Adjusted Life Years
• Time Factors
• Uganda/epidemiology

Collapse

Grants

• G0601721 Medical Research Council
• MR/N014693/1 Medical Research Council
• PDF-2012-05-258 Department of Health
• MR/K012126/1 Medical Research Council
• MR/J005088/1 Medical Research Council

Collapse

Affiliation(s)

Nicky McCreesh London School of Hygiene and Tropical Medicine, London, United Kingdom
Ioannis Andrianakis London School of Hygiene and Tropical Medicine, London, United Kingdom
Rebecca N. Nsubuga MRC/UVRI Research Unit on AIDS, Entebbe, Uganda
Mark Strong Sheffield University, Sheffield, United Kingdom
Ian Vernon Durham University, Durham, United Kingdom
Trevelyan J. McKinley Exeter University, Exeter, United Kingdom
Jeremy E. Oakley Sheffield University, Sheffield, United Kingdom
Michael Goldstein Durham University, Durham, United Kingdom
Richard Hayes London School of Hygiene and Tropical Medicine, London, United Kingdom
Richard G. White London School of Hygiene and Tropical Medicine, London, United Kingdom

Collapse

Bayesian uncertainty analysis for complex systems biology models: emulation, global parameter searches and evaluation of gene functions

Background

Many mathematical models have now been employed across every area of systems biology. These models increasingly involve large numbers of unknown parameters, have complex structure which can result in substantial evaluation time relative to the needs of the analysis, and need to be compared to observed data of various forms. The correct analysis of such models usually requires a global parameter search, over a high dimensional parameter space, that incorporates and respects the most important sources of uncertainty. This can be an extremely difficult task, but it is essential for any meaningful inference or prediction to be made about any biological system. It hence represents a fundamental challenge for the whole of systems biology.

Methods

Bayesian statistical methodology for the uncertainty analysis of complex models is introduced, which is designed to address the high dimensional global parameter search problem. Bayesian emulators that mimic the systems biology model but which are extremely fast to evaluate are embeded within an iterative history match: an efficient method to search high dimensional spaces within a more formal statistical setting, while incorporating major sources of uncertainty.

Results

The approach is demonstrated via application to a model of hormonal crosstalk in Arabidopsis root development, which has 32 rate parameters, for which we identify the sets of rate parameter values that lead to acceptable matches between model output and observed trend data. The multiple insights into the model’s structure that this analysis provides are discussed. The methodology is applied to a second related model, and the biological consequences of the resulting comparison, including the evaluation of gene functions, are described.

Conclusions

Bayesian uncertainty analysis for complex models using both emulators and history matching is shown to be a powerful technique that can greatly aid the study of a large class of systems biology models. It both provides insight into model behaviour and identifies the sets of rate parameters of interest.

Electronic supplementary material

The online version of this article (doi:10.1186/s12918-017-0484-3) contains supplementary material, which is available to authorized users.

Improving ART programme retention and viral suppression are key to maximising impact of treatment as prevention - a modelling study

Background

UNAIDS calls for fewer than 500,000 new HIV infections/year by 2020, with treatment-as-prevention being a key part of their strategy for achieving the target. A better understanding of the contribution to transmission of people at different stages of the care pathway can help focus intervention services at populations where they may have the greatest effect. We investigate this using Uganda as a case study.

Methods

An individual-based HIV/ART model was fitted using history matching. 100 model fits were generated to account for uncertainties in sexual behaviour, HIV epidemiology, and ART coverage up to 2015 in Uganda. A number of different ART scale-up intervention scenarios were simulated between 2016 and 2030. The incidence and proportion of transmission over time from people with primary infection, post-primary ART-naïve infection, and people currently or previously on ART was calculated.

Results

In all scenarios, the proportion of transmission by ART-naïve people decreases, from 70% (61%–79%) in 2015 to between 23% (15%–40%) and 47% (35%–61%) in 2030. The proportion of transmission by people on ART increases from 7.8% (3.5%–13%) to between 14% (7.0%–24%) and 38% (21%–55%). The proportion of transmission by ART dropouts increases from 22% (15%–33%) to between 31% (23%–43%) and 56% (43%–70%).

Conclusions

People who are currently or previously on ART are likely to play an increasingly large role in transmission as ART coverage increases in Uganda. Improving retention on ART, and ensuring that people on ART remain virally suppressed, will be key in reducing HIV incidence in Uganda.

Universal test, treat, and keep: improving ART retention is key in cost-effective HIV control in Uganda

Background

With ambitious new UNAIDS targets to end AIDS by 2030, and new WHO treatment guidelines, there is increased interest in the best way to scale-up ART coverage. We investigate the cost-effectiveness of various ART scale-up options in Uganda.

Methods

Individual-based HIV/ART model of Uganda, calibrated using history matching. 22 ART scale-up strategies were simulated from 2016 to 2030, comprising different combinations of six single interventions (1. increased HIV testing rates, 2. no CD4 threshold for ART initiation, 3. improved ART retention, 4. increased ART restart rates, 5. improved linkage to care, 6. improved pre-ART care). The incremental net monetary benefit (NMB) of each intervention was calculated, for a wide range of different willingness/ability to pay (WTP) per DALY averted (health-service perspective, 3% discount rate).

Results

For all WTP thresholds above $210, interventions including removing the CD4 threshold were likely to be most cost-effective. At a WTP of $715 (1 × per-capita-GDP) interventions to improve linkage to and retention/re-enrolment in HIV care were highly likely to be more cost-effective than interventions to increase rates of HIV testing. At higher WTP (> ~ $1690), the most cost-effective option was ‘Universal Test, Treat, and Keep’ (UTTK), which combines interventions 1–5 detailed above.

Conclusions

Our results support new WHO guidelines to remove the CD4 threshold for ART initiation in Uganda. With additional resources, this could be supplemented with interventions aimed at improving linkage to and/or retention in HIV care. To achieve the greatest reductions in HIV incidence, a UTTK policy should be implemented.

Electronic supplementary material

The online version of this article (doi:10.1186/s12879-017-2420-y) contains supplementary material, which is available to authorized users.

History matching of a complex epidemiological model of human immunodeficiency virus transmission by using variance emulation

Complex stochastic models are commonplace in epidemiology, but their utility depends on their calibration to empirical data. History matching is a (pre)calibration method that has been applied successfully to complex deterministic models. In this work, we adapt history matching to stochastic models, by emulating the variance in the model outputs, and therefore accounting for its dependence on the model's input values. The method proposed is applied to a real complex epidemiological model of human immunodeficiency virus in Uganda with 22 inputs and 18 outputs, and is found to increase the efficiency of history matching, requiring 70% of the time and 43% fewer simulator evaluations compared with a previous variant of the method. The insight gained into the structure of the human immunodeficiency virus model, and the constraints placed on it, are then discussed.