Disease transmission and control modelling at the science-policy interface

The future of pandemic modeling in support of decision making: lessons learned from COVID-19

The devastating global impacts of the COVID-19 pandemic are a stark reminder of the need for proactive and effective pandemic response. Disease modeling and forecasting are key in this response, as they enable forward-looking assessment and strategic planning. Via 85 interviews spanning 14 countries with disease modelers and those they support, conducted amid the COVID-19 pandemic response, we offer a qualitative overview of challenges faced, lessons learned, and readiness for future pandemics. The interviewees highlighted several key challenges and considerations in forecasting, particularly emphasizing the complications introduced by human behavior and various data-related issues (including data availability, quality, and standardization). They underscored the importance of effective communication among those who create models, those who make decisions based on these models, and the general public. Additionally, they pointed out the necessity for addressing global equity, debated the merits of centralized versus decentralized responses to crises, and stressed the need for establishing measures for sustainable preparedness. Their verdicts on future pandemic readiness were mixed, with only 43% of respondents saying we are better prepared for a future pandemic. We conclude by providing our vision for how modeling can and should look in the context of a successful pandemic response, in light of the insights gleaned via the interview process. These interviews and their synthesis offer crucial perspectives to shape future responses and preparedness for global health crises.

Policy impact of the Imperial College COVID-19 Response Team: global perspective and United Kingdom case study

BACKGROUND

Mathematical models and advanced analytics play an important role in policy decision making and mobilizing action. The Imperial College Coronavirus Disease 2019 (COVID-19) Response Team (ICCRT) provided continuous, timely and robust epidemiological analyses to inform the policy responses of governments and public health agencies around the world. This study aims to quantify the policy impact of ICCRT outputs, and understand which evidence was considered policy-relevant during the COVID-19 pandemic.

METHODS

We collated all outputs published by the ICCRT between 01-01-2020 and 24-02-2022 and conducted inductive thematic analysis. A systematic search of the Overton database identified policy document references, as an indicator of policy impact.

RESULTS

We identified 620 outputs including preprints (16%), reports (29%), journal articles (37%) and news items (18%). More than half (56%) of all reports and preprints were subsequently peer-reviewed and published as a journal article after 202 days on average. Reports and preprints were crucial during the COVID-19 pandemic to the timely distribution of important research findings. One-fifth of ICCRT outputs (21%) were available to or considered by United Kingdom government meetings. Policy documents from 41 countries in 26 different languages referenced 43% of ICCRT outputs, with a mean time between publication and reference in the policy document of 256 days. We analysed a total of 1746 policy document references. Two-thirds (61%) of journal articles, 39% of preprints, 31% of reports and 16% of news items were referenced in one or more policy documents (these 217 outputs had a mean of 8 policy document references per output). The most frequent themes of the evidence produced by the ICCRT reflected the evidence-need for policy decision making, and evolved accordingly from the pre-vaccination phase [severity, healthcare demand and capacity, and non-pharmaceutical interventions (NPIs)] to the vaccination phase of the epidemic (variants and genomics).

CONCLUSION

The work produced by the ICCRT affected global and domestic policy during the COVID-19 pandemic. The focus of evidence produced by the ICCRT corresponded with changing policy needs over time. The policy impact from ICCRT news items highlights the effectiveness of this unique communication strategy in addition to traditional research outputs, ensuring research informs policy decisions more effectively.

Inference on spatiotemporal dynamics for coupled biological populations

Mathematical models in ecology and epidemiology must be consistent with observed data in order to generate reliable knowledge and evidence-based policy. Metapopulation systems, which consist of a network of connected sub-populations, pose technical challenges in statistical inference owing to nonlinear, stochastic interactions. Numerical difficulties encountered in conducting inference can obstruct the core scientific questions concerning the link between the mathematical models and the data. Recently, an algorithm has been proposed that enables computationally tractable likelihood-based inference for high-dimensional partially observed stochastic dynamic models of metapopulation systems. We use this algorithm to build a statistically principled data analysis workflow for metapopulation systems. Via a case study of COVID-19, we show how this workflow addresses the limitations of previous approaches. The COVID-19 pandemic provides a situation where mathematical models and their policy implications are widely visible, and we revisit an influential metapopulation model used to inform basic epidemiological understanding early in the pandemic. Our methods support self-critical data analysis, enabling us to identify and address model weaknesses, leading to a new model with substantially improved statistical fit and parameter identifiability. Our results suggest that the lockdown initiated on 23 January 2020 in China was more effective than previously thought.

Informing policy via dynamic models: Cholera in Haiti

Public health decisions must be made about when and how to implement interventions to control an infectious disease epidemic. These decisions should be informed by data on the epidemic as well as current understanding about the transmission dynamics. Such decisions can be posed as statistical questions about scientifically motivated dynamic models. Thus, we encounter the methodological task of building credible, data-informed decisions based on stochastic, partially observed, nonlinear dynamic models. This necessitates addressing the tradeoff between biological fidelity and model simplicity, and the reality of misspecification for models at all levels of complexity. We assess current methodological approaches to these issues via a case study of the 2010-2019 cholera epidemic in Haiti. We consider three dynamic models developed by expert teams to advise on vaccination policies. We evaluate previous methods used for fitting these models, and we demonstrate modified data analysis strategies leading to improved statistical fit. Specifically, we present approaches for diagnosing model misspecification and the consequent development of improved models. Additionally, we demonstrate the utility of recent advances in likelihood maximization for high-dimensional nonlinear dynamic models, enabling likelihood-based inference for spatiotemporal incidence data using this class of models. Our workflow is reproducible and extendable, facilitating future investigations of this disease system.

Improving modelling for epidemic responses: reflections from members of the UK infectious disease modelling community on their experiences during the COVID-19 pandemic

Background

The COVID-19 pandemic both relied and placed significant burdens on the experts involved from research and public health sectors. The sustained high pressure of a pandemic on responders, such as healthcare workers, can lead to lasting psychological impacts including acute stress disorder, post-traumatic stress disorder, burnout, and moral injury, which can impact individual wellbeing and productivity.

Methods

As members of the infectious disease modelling community, we convened a reflective workshop to understand the professional and personal impacts of response work on our community and to propose recommendations for future epidemic responses. The attendees represented a range of career stages, institutions, and disciplines. This piece was collectively produced by those present at the session based on our collective experiences.

Results

Key issues we identified at the workshop were lack of institutional support, insecure contracts, unequal credit and recognition, and mental health impacts. Our recommendations include rewarding impactful work, fostering academia-public health collaboration, decreasing dependence on key individuals by developing teams, increasing transparency in decision-making, and implementing sustainable work practices.

Conclusions

Despite limitations in representation, this workshop provided valuable insights into the UK COVID-19 modelling experience and guidance for future public health crises. Recognising and addressing the issues highlighted is crucial, in our view, for ensuring the effectiveness of epidemic response work in the future.

Public awareness of and opinions on the use of mathematical transmission modelling to inform public health policy in the United Kingdom

Mathematical modelling is used to inform public health policy, particularly so during the COVID-19 pandemic. As the public are key stakeholders, understanding the public perceptions of these tools is vital. To complement our previous study on the science-policy interface, novel survey data were collected via an online panel ('representative' sample) and social media ('non-probability' sample). Many questions were asked twice, in reference to the period 'prior to' (retrospectively) and 'during' the COVID-19 pandemic. Respondents reported being increasingly aware of modelling in informing policy during the pandemic, with higher levels of awareness among social media respondents. Modelling informing policy was perceived as more reliable during the pandemic than in reference to the pre-pandemic period in both samples. Trust in government public health advice remained high within both samples but was lower during the pandemic in comparison with the (retrospective) pre-pandemic period. The decay in trust was greater among social media respondents. Many respondents explicitly made the distinction that their trust was reserved for 'scientists' and not 'politicians'. Almost all respondents believed governments have responsibility for communicating modelling to the public. These results provide a reminder of the skewed conclusions that could be drawn from non-representative samples.

An evaluation of prospective COVID-19 modelling studies in the USA: from data to science translation

Infectious disease modelling can serve as a powerful tool for situational awareness and decision support for policy makers. However, COVID-19 modelling efforts faced many challenges, from poor data quality to changing policy and human behaviour. To extract practical insight from the large body of COVID-19 modelling literature available, we provide a narrative review with a systematic approach that quantitatively assessed prospective, data-driven modelling studies of COVID-19 in the USA. We analysed 136 papers, and focused on the aspects of models that are essential for decision makers. We have documented the forecasting window, methodology, prediction target, datasets used, and geographical resolution for each study. We also found that a large fraction of papers did not evaluate performance (25%), express uncertainty (50%), or state limitations (36%). To remedy some of these identified gaps, we recommend the adoption of the EPIFORGE 2020 model reporting guidelines and creating an information-sharing system that is suitable for fast-paced infectious disease outbreak science.

Public perceptions and interactions with UK COVID-19 Test, Trace and Isolate policies, and implications for pandemic infectious disease modelling

Background The efforts to contain SARS-CoV-2 and reduce the impact of the COVID-19 pandemic have been supported by Test, Trace and Isolate (TTI) systems in many settings, including the United Kingdom. Mathematical models of transmission and TTI interventions, used to inform design and policy choices, make assumptions about the public’s behaviour in the context of a rapidly unfolding and changeable emergency. This study investigates public perceptions and interactions with UK TTI policy in July 2021, assesses them against how TTI processes are conceptualised and represented in models, and then interprets the findings with modellers who have been contributing evidence to TTI policy. Methods 20 members of the public recruited via social media were interviewed for one hour about their perceptions and interactions with the UK TTI system. Thematic analysis identified key themes, which were then presented back to a workshop of pandemic infectious disease modellers who assessed these findings against assumptions made in TTI intervention modelling. Workshop members co-drafted this report. Results Themes included education about SARS-CoV-2, perceived risks, trust, mental health and practical concerns. Findings covered testing practices, including the uses of and trust in different types of testing, and the challenges of testing and isolating faced by different demographic groups. This information was judged as consequential to the modelling process, from guiding the selection of research questions, influencing choice of model structure, informing parameter ranges and validating or challenging assumptions, to highlighting where model assumptions are reasonable or where their poor reflection of practice might lead to uninformative results. Conclusions We conclude that deeper engagement with members of the public should be integrated at regular stages of public health intervention modelling.

The African swine fever modelling challenge: Model comparison and lessons learnt

Robust epidemiological knowledge and predictive modelling tools are needed to address challenging objectives, such as: understanding epidemic drivers; forecasting epidemics; and prioritising control measures. Often, multiple modelling approaches can be used during an epidemic to support effective decision making in a timely manner. Modelling challenges contribute to understanding the pros and cons of different approaches and to fostering technical dialogue between modellers. In this paper, we present the results of the first modelling challenge in animal health - the ASF Challenge - which focused on a synthetic epidemic of African swine fever (ASF) on an island. The modelling approaches proposed by five independent international teams were compared. We assessed their ability to predict temporal and spatial epidemic expansion at the interface between domestic pigs and wild boar, and to prioritise a limited number of alternative interventions. We also compared their qualitative and quantitative spatio-temporal predictions over the first two one-month projection phases of the challenge. Top-performing models in predicting the ASF epidemic differed according to the challenge phase, host species, and in predicting spatial or temporal dynamics. Ensemble models built using all team-predictions outperformed any individual model in at least one phase. The ASF Challenge demonstrated that accounting for the interface between livestock and wildlife is key to increasing our effectiveness in controlling emerging animal diseases, and contributed to improving the readiness of the scientific community to face future ASF epidemics. Finally, we discuss the lessons learnt from model comparison to guide decision making.

Disease transmission and control modelling at the science-policy interface

The coronavirus disease 2019 (COVID-19) pandemic has disrupted the lives of billions across the world. Mathematical modelling has been a key tool deployed throughout the pandemic to explore the potential public health impact of an unmitigated epidemic. The results of such studies have informed governments' decisions to implement non-pharmaceutical interventions to control the spread of the virus. In this article, we explore the complex relationships between models, decision-making, the media and the public during the COVID-19 pandemic in the United Kingdom of Great Britain and Northern Ireland (UK). Doing so not only provides an important historical context of COVID-19 modelling and how it has shaped the UK response, but as the pandemic continues and looking towards future pandemic preparedness, understanding these relationships and how they might be improved is critical. As such, we have synthesized information gathered via three methods: a survey to publicly list attendees of the Scientific Advisory Group for Emergencies, the Scientific Pandemic Influenza Group on Modelling and other comparable advisory bodies, interviews with science communication experts and former scientific advisors, and reviewing some of the key COVID-19 modelling literature from 2020. Our research highlights the desire for increased bidirectional communication between modellers, decision-makers and the public, as well as the need to convey uncertainty inherent in transmission models in a clear manner. These aspects should be considered carefully ahead of the next emergency response.