Validating model output in the absence of ground truth data: A COVID-19 case study using the Simulator of Infectious Disease Dynamics in North Carolina (SIDD-NC) model

As the COVID-19 pandemic has progressed, various models have been developed to forecast changes in the outbreak and assess intervention strategies. In this study we validate the Simulator of Infectious Disease Dynamics in North Carolina (SIDD-NC) model against an ensemble of proxy-ground truth infections datasets. We assess the performance of SIDD-NC using Spearman Rank Correlation, RMSE, and percent RMSE at a state and county level. We conduct the analysis for the period of March 2020 through November 2020 as well as in shorter time increments to assess both the recreation of the pandemic curve as well as day-to-day transmission of SARS-CoV-2 within the population. We find that SIDD-NC performs well against the datasets in the ensemble, generating an estimate of infections that is robust both spatially and temporally.

NC-COVID: A Time-Varying Compartmental Model for Estimating SARS-CoV-2 Infection Dynamics in North Carolina, US

Efforts to track and model SARS-CoV-2 infection dynamics in the population have been complicated by certain aspects of the transmission characteristics, which include a pre-symptomatic infectious phase as well as asymptomatic infectious individuals. Another problem is that many models focus on case count, as there has been (and is) limited data regarding infection status of members of the population, which is the most important aspect for constructing transmission models. This paper describes and explains the parameterization, calibration, and revision of the NC-COVID model, a compartmental model to estimate SARS-CoV-2 infection dynamics for the state of North Carolina, US. The model was developed early in the pandemic to provide rapid, up-to-date state-level estimates of the number of people who were currently infected, were immune from a prior infection, and remained susceptible to infection. As a post modeling exercise, we assessed the veracity of the model by comparing its output to SARS-CoV-2 viral particle concentrations detected in wastewater data and to estimates of people infected using COVID-19 deaths. The NC-COVID model was highly correlated with these independently derived estimates, suggesting that it produced accurate estimates of SARS-CoV-2 infection dynamics in North Carolina.