Evaluation of FluSight influenza forecasting in the 2021-22 and 2022-23 seasons with a new target laboratory-confirmed influenza hospitalizations

Accurate forecasts can enable more effective public health responses during seasonal influenza epidemics. Forecasting teams were asked to provide national and jurisdiction-specific probabilistic predictions of weekly confirmed influenza hospital admissions for one through four weeks ahead for the 2021-22 and 2022-23 influenza seasons. Across both seasons, 26 teams submitted forecasts, with the submitting teams varying between seasons. Forecast skill was evaluated using the Weighted Interval Score (WIS), relative WIS, and coverage. Six out of 23 models outperformed the baseline model across forecast weeks and locations in 2021-22 and 12 out of 18 models in 2022-23. Averaging across all forecast targets, the FluSight ensemble was the 2nd most accurate model measured by WIS in 2021-22 and the 5th most accurate in the 2022-23 season. Forecast skill and 95% coverage for the FluSight ensemble and most component models degraded over longer forecast horizons and during periods of rapid change. Current influenza forecasting efforts help inform situational awareness, but research is needed to address limitations, including decreased performance during periods of changing epidemic dynamics.

Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations

Background: Short-term forecasts of infectious disease contribute to situational awareness and capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise forecasts' predictive performance by combining independent models into an ensemble. Here we report the performance of ensemble predictions of COVID-19 cases and deaths across Europe from March 2021 to March 2022.

Methods: We created the European COVID-19 Forecast Hub, an online open-access platform where modellers upload weekly forecasts for 32 countries with results publicly visualised and evaluated. We created a weekly ensemble forecast from the equally-weighted average across individual models' predictive quantiles. We measured forecast accuracy using a baseline and relative Weighted Interval Score (rWIS). We retrospectively explored ensemble methods, including weighting by past performance.

Results: We collected weekly forecasts from 48 models, of which we evaluated 29 models alongside the ensemble model. The ensemble had a consistently strong performance across countries over time, performing better on rWIS than 91% of forecasts for deaths (N=763 predictions from 20 models), and 83% forecasts for cases (N=886 predictions from 23 models). Performance remained stable over a 4-week horizon for death forecasts but declined with longer horizons for cases. Among ensemble methods, the most influential choice came from using a median average instead of the mean, regardless of weighting component models.

Conclusions: Our results support combining independent models into an ensemble forecast to improve epidemiological predictions, and suggest that median averages yield better performance than methods based on means. We highlight that forecast consumers should place more weight on incident death forecasts than case forecasts at horizons greater than two weeks.

Funding: European Commission, Ministerio de Ciencia, Innovación y Universidades, FEDER; Agència de Qualitat i Avaluació Sanitàries de Catalunya; Netzwerk Universitätsmedizin; Health Protection Research Unit; Wellcome Trust; European Centre for Disease Prevention and Control; Ministry of Science and Higher Education of Poland; Federal Ministry of Education and Research; Los Alamos National Laboratory; German Free State of Saxony; NCBiR; FISR 2020 Covid-19 I Fase; Spanish Ministry of Health / REACT-UE (FEDER); National Institutes of General Medical Sciences; Ministerio de Sanidad/ISCIII; PERISCOPE European H2020; PERISCOPE European H2021; InPresa; National Institutes of Health, NSF, US Centers for Disease Control and Prevention, Google, University of Virginia, Defense Threat Reduction Agency.

Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-wk horizon three to five times larger than when predicting at a 1-wk horizon. This project underscores the role that collaboration and active coordination between governmental public-health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks.

Melioidosis diagnostic workshop, 2013

Melioidosis is a severe disease that can be difficult to diagnose because of its diverse clinical manifestations and a lack of adequate diagnostic capabilities for suspected cases. There is broad interest in improving detection and diagnosis of this disease not only in melioidosis-endemic regions but also outside these regions because melioidosis may be underreported and poses a potential bioterrorism challenge for public health authorities. Therefore, a workshop of academic, government, and private sector personnel from around the world was convened to discuss the current state of melioidosis diagnostics, diagnostic needs, and future directions.

Workshop on treatment of and postexposure prophylaxis for Burkholderia pseudomallei and B. mallei Infection, 2010

The US Public Health Emergency Medical Countermeasures Enterprise convened subject matter experts at the 2010 HHS Burkholderia Workshop to develop consensus recommendations for postexposure prophylaxis against and treatment for Burkholderia pseudomallei and B. mallei infections, which cause melioidosis and glanders, respectively. Drugs recommended by consensus of the participants are ceftazidime or meropenem for initial intensive therapy, and trimethoprim/sulfamethoxazole or amoxicillin/clavulanic acid for eradication therapy. For postexposure prophylaxis, recommended drugs are trimethoprim/sulfamethoxazole or co-amoxiclav. To improve the timely diagnosis of melioidosis and glanders, further development and wide distribution of rapid diagnostic assays were also recommended. Standardized animal models and B. pseudomallei strains are needed for further development of therapeutic options. Training for laboratory technicians and physicians would facilitate better diagnosis and treatment options.

Mass prophylaxis dispensing concerns: traffic and public access to PODs

The ability to quickly dispense postexposure prophylaxis (PEP) using multiple points of dispensing (PODs) following a bioterrorism event could potentially save a large proportion of those who were exposed, while failure in PEP dispensing could have dire public health consequences. A Monte Carlo simulation was developed to explore the traffic flow and parking around PODs under different arrival rates and how these factors might affect the utilization rate of POD workers. The results demonstrate that the public can reasonably access the PODs under ideal conditions assuming a stationary (uniform) arrival rate. For the 5 nonstationary arrival rates tested, however, the available parking spaces quickly become filled, causing long traffic queues and resulting in total processing times that range from 1 hour to over 6 hours. Basic planning considerations should include the use of physical barriers, signage, and traffic control officers to help direct vehicular and pedestrian access to the PODs. Furthermore, the parking and traffic surrounding PODs creates long queues of people waiting to access the PODs. Thus, POD staff are fully used approximately 90% of the time, which can lead to worker fatigue and burn out.

Public health response to an anthrax attack: an evaluation of vaccination policy options

A discrete-time, deterministic, compartmental model was developed and analyzed to provide insight into how the use of anthrax vaccine before or after a large-scale attack can reduce casualties. The model accounts for important response and protection factors such as antibiotic and vaccine efficacy, the protective effects of buildings, the timing of emergency response, and antibiotic adherence and vaccine coverage in the population prior to the attack. The relative benefit of pre- versus post-exposure vaccination is influenced by the timing of the post-exposure antibiotic distribution campaign as well as assumptions of antibiotic adherence. The results indicate that, regardless of which vaccination policy is adopted, a rapid and effective post-attack medical response has a large impact on the number of lives that can be saved by a post-exposure prophylaxis (PEP) campaign. A sensitivity analysis of the model indicates that uncertainty in medical efficacy and the time to initiate a PEP campaign are the model parameters that have the greatest impact on the number of predicted deaths. It is shown that for each day that a mass prophylaxis campaign is delayed, more casualties and deaths result than for each day that the completion of the campaign is delayed.

Kinetics of influenza A virus infection in humans

Currently, little is known about the viral kinetics of influenza A during infection within an individual. We utilize a series of mathematical models of increasing complexity, which incorporate target cell limitation and the innate interferon response, to examine influenza A virus kinetics in the upper respiratory tracts of experimentally infected adults. The models were fit to data from an experimental H1N1 influenza A/Hong Kong/123/77 infection and suggest that it is important to include the eclipse phase of the viral life cycle in viral dynamic models. Doing so, we estimate that after a delay of approximately 6 h, infected cells begin producing influenza virus and continue to do so for approximately 5 h. The average lifetime of infected cells is approximately 11 h, and the half-life of free infectious virus is approximately 3 h. We calculated the basic reproductive number, R(0), which indicated that a single infected cell could produce approximately 22 new productive infections. This suggests that antiviral treatments have a large hurdle to overcome in moderating symptoms and limiting infectiousness and that treatment has to be initiated as early as possible. For about 50% of patients, the curve of viral titer versus time has two peaks. This bimodal behavior can be explained by incorporating the antiviral effects of interferon into the model. Our model also compared well to an additional data set on viral titer after experimental infection and treatment with the neuraminidase inhibitor zanamivir, which suggests that such models may prove useful in estimating the efficacies of different antiviral therapies for influenza A infection.

Evolution of the equine infectious anemia virus long terminal repeat during the alteration of cell tropism

Equine infectious anemia virus (EIAV) is a lentivirus with in vivo cell tropism primarily for tissue macrophages; however, in vitro the virus can be adapted to fibroblasts and other cell types. Tropism adaptation is associated with both envelope and long terminal repeat (LTR) changes, and findings strongly suggest that these regions of the genome influence cell tropism and virulence. Furthermore, high levels of genetic variation have been well documented in both of these genomic regions. However, specific EIAV nucleotide or amino acid changes that are responsible for cell tropism changes have not been identified. A study was undertaken with the highly virulent, macrophage-tropic strain of virus EIAV(wyo) to identify LTR changes associated with alterations in cell tropism. We found the stepwise generation of a new transcription factor binding motif within the enhancer that was associated with adaptation of EIAV to endothelial cells and fibroblasts. An LTR that contained the new motif had enhanced transcriptional activity in fibroblasts, whereas the new site did not alter LTR activity in a macrophage cell line. This finding supports a previous prediction that selection for new LTR genetic variants may be a consequence of cell-specific selective pressures. Additional investigations of the EIAV(wyo) LTR were performed in vivo to determine if LTR evolution could be detected over the course of a 3-year infection. Consistent with previous in vivo findings, we observed no changes in the enhancer region of the LTR over that time period, indicating that the EIAV(wyo) LTR was evolutionarily stable in vivo.

Isolation and characterization of RNase LTR sequences of Ty1-copia retrotransposons in common bean (Phaseolus vulgaris L)

Retroelements have proved useful for molecular marker studies and play an important role in genome evolution. Ty1-copia retrotransposons are ubiquitous and heterogeneous in plant genomes, and although many elements have been isolated and characterized, almost no information about them is available in the literature for Phaseolus vulgaris L. We report here the isolation and characterization of new RNase long terminal repeat (LTR) sections of the Ty1-copia group for this crop plant. RNAse sections showed conserved amino acids with the downstream sections corresponding to the polypurine-tract and 5' sections of 3' LTRs. The RNase sections were aligned using ClustalX to find potential relationships between sequences. A comparison with this analysis was made using the partition analysis of quasispecies package (PAQ), which is specific for quasispecies-like populations. The analysis revealed eight distinct groups. To uncover LTR variability and potential conserved promoter motifs, we also designed new primers from the presumed polypurine-tract regions. A similarity search found short stretches similar to upstream and downstream regions of some genes. Conserved motifs, corresponding to transcription factor binding sites, were discovered through MatInspector software and two sequences characterized. From a putative LTR fragment, we then designed a new primer, which, through sequence-specific amplification polymorphism (SSAP), showed numerous polymorphic bands between two distinct P. vulgaris accessions.

Subpopulations of equine infectious anemia virus Rev coexist in vivo and differ in phenotype

Lentiviruses exist in vivo as a population of related, nonidentical genotypes, commonly referred to as quasispecies. The quasispecies structure is characteristic of complex adaptive systems and contributes to the high rate of evolution in lentiviruses that confounds efforts to develop effective vaccines and antiviral therapies. Here, we describe analyses of genetic data from longitudinal studies of genetic variation in a lentivirus regulatory protein, Rev, over the course of disease in ponies experimentally infected with equine infectious anemia virus. As observed with other lentivirus data, the Rev variants exhibited a quasispecies character. Phylogenetic and partition analyses suggested that the Rev quasispecies comprised two distinct subpopulations that coexisted during infection. One subpopulation appeared to accumulate changes in a linear, time-dependent manner, while the other evolved radially from a common variant. Over time, the two subpopulations cycled in predominance coincident with changes in the disease state, suggesting that the two groups differed in selective advantage. Transient expression assays indicated the two populations differed significantly in Rev nuclear export activity. Chimeric proviral clones containing Rev genotypes representative of each population differed in rate and overall level of virus replication in vitro. The coexistence of genetically distinct viral subpopulations that differ in phenotype provides great adaptability to environmental changes within the infected host. A quasispecies model with multiple subpopulations may provide additional insight into the nature of lentivirus reservoirs and the evolution of antigenic and drug-resistant variants.

Genetic and biological variation in equine infectious anemia virus Rev correlates with variable stages of clinical disease in an experimentally infected pony

Genetic and biological variation in the regulatory protein Rev of equine infectious anemia virus (EIAV) were examined throughout a clinically dynamic disease course of an experimentally infected pony. Following infection with the virulent EIAV(Wyo), the pony underwent a variable disease course, including an acute fever episode at 12 days postinfection (DPI), multiple recurrent fever episodes until 135 DPI, a prolonged subclinical period, and two late fever episodes. Viral RNA was isolated from the inoculum and sequential sera samples, and the rev exon 2/gp45 overlapping ORFs were amplified, cloned, and sequenced. Novel variants were found throughout infection, and genetic analyses indicated that both the Rev and gp45 ORFs were under selective pressure. The Rev variant predominant in the inoculum, R1, remained predominant during the early periods following infection (until 35 DPI); however, R1 was replaced by new predominant variants during the recurrent fever period (67-135 DPI). R1 reemerged as the predominant variant during the afebrile period, but a new predominant variant, R93, was associated with the late fever episodes. Rev variants predominant during recurrent febrile and late-febrile periods had significantly higher Rev-mediated nuclear export activity than the variants predominant during the acute and afebrile periods. Statistical correlation was found between Rev activity and different stages of clinical disease. Together, these results suggest that genetic and biological variation in rev may be a contributing factor in EIAV disease progression.

PAQ: Partition Analysis of Quasispecies

MOTIVATION

The complexities of genetic data may not be accurately described by any single analytical tool. Phylogenetic analysis is often used to study the genetic relationship among different sequences. Evolutionary models and assumptions are invoked to reconstruct trees that describe the phylogenetic relationship among sequences. Genetic databases are rapidly accumulating large amounts of sequences. Newly acquired sequences, which have not yet been characterized, may require preliminary genetic exploration in order to build models describing the evolutionary relationship among sequences. There are clustering techniques that rely less on models of evolution, and thus may provide nice exploratory tools for identifying genetic similarities. Some of the more commonly used clustering methods perform better when data can be grouped into mutually exclusive groups. Genetic data from viral quasispecies, which consist of closely related variants that differ by small changes, however, may best be partitioned by overlapping groups.

RESULTS

We have developed an intuitive exploratory program, Partition Analysis of Quasispecies (PAQ), which utilizes a non-hierarchical technique to partition sequences that are genetically similar. PAQ was used to analyze a data set of human immunodeficiency virus type 1 (HIV-1) envelope sequences isolated from different regions of the brain and another data set consisting of the equine infectious anemia virus (EIAV) regulatory gene rev. Analysis of the HIV-1 data set by PAQ was consistent with phylogenetic analysis of the same data, and the EIAV rev variants were partitioned into two overlapping groups. PAQ provides an additional tool which can be used to glean information from genetic data and can be used in conjunction with other tools to study genetic similarities and genetic evolution of viral quasispecies.

Antigenic and genetic stability of bovine immunodeficiency virus during long-term persistence in cattle experimentally infected with the BIV(R29) isolate

Experimental infection of cattle with bovine immunodeficiency virus (BIV) is characterized by persistent, low levels of virus replication in the absence of clinical disease. A virus neutralization (VN) assay was developed to examine the role of VN antibodies in controlling virus replication in cattle experimentally infected with the BIV(R29) isolate of BIV. All animals developed VN antibody, but there was no correlation between VN titres and restriction of virus replication in vivo. BIV infection did not induce high-titred, cross-neutralizing antibody and there was no evidence for antigenic variation through more than 4 years in vivo. Genetic comparisons among the BIV(R29) inoculum virus and viruses isolated from infected animals identified only limited genetic variation during 4 years in vivo. Moreover, there was no evidence that the observed variation was due to selection. Analyses of genetic diversity in the virus stock used for inoculation indicated a fairly homogeneous population. In the absence of high levels of virus replication and overt clinical disease, there appeared to be little selection of virus variants, resulting in antigenic and genetic stability of BIV(R29) during long-term, persistent infection.