Evaluating probabilistic ecological forecasts

Context-dependent representation of within- and between-model uncertainty: aggregating probabilistic predictions in infectious disease epidemiology

Probabilistic predictions support public health planning and decision making, especially in infectious disease emergencies. Aggregating outputs from multiple models yields more robust predictions of outcomes and associated uncertainty. While the selection of an aggregation method can be guided by retrospective performance evaluations, this is not always possible. For example, if predictions are conditional on assumptions about how the future will unfold (e.g. possible interventions), these assumptions may never materialize, precluding any direct comparison between predictions and observations. Here, we summarize literature on aggregating probabilistic predictions, illustrate various methods for infectious disease predictions via simulation, and present a strategy for choosing an aggregation method when empirical validation cannot be used. We focus on the linear opinion pool (LOP) and Vincent average, common methods that make different assumptions about between-prediction uncertainty. We contend that assumptions of the aggregation method should align with a hypothesis about how uncertainty is expressed within and between predictions from different sources. The LOP assumes that between-prediction uncertainty is meaningful and should be retained, while the Vincent average assumes that between-prediction uncertainty is akin to sampling error and should not be preserved. We provide an R package for implementation. Given the rising importance of multi-model infectious disease hubs, our work provides useful guidance on aggregation and a deeper understanding of the benefits and risks of different approaches.

Managing host-parasite interactions in humans and wildlife in times of global change

Global change in the Anthropocene has modified the environment of almost any species on earth, be it through climate change, habitat modifications, pollution, human intervention in the form of mass drug administration (MDA), or vaccination. This can have far-reaching consequences on all organisational levels of life, including eco-physiological stress at the cell and organism level, individual fitness and behaviour, population viability, species interactions and biodiversity. Host-parasite interactions often require highly adapted strategies by the parasite to survive and reproduce within the host environment and ensure efficient transmission among hosts. Yet, our understanding of the system-level outcomes of the intricate interplay of within host survival and among host parasite spread is in its infancy. We shed light on how global change affects host-parasite interactions at different organisational levels and address challenges and opportunities to work towards better-informed management of parasite control. We argue that global change affects host-parasite interactions in wildlife inhabiting natural environments rather differently than in humans and invasive species that benefit from anthropogenic environments as habitat and more deliberate rather than erratic exposure to therapeutic drugs and other control efforts.

A niche for null models in adaptive resource management

As global systems rapidly change, our collective ability to predict future ecological dynamics will become increasingly important for successful natural resource management. By merging stakeholder objectives with system uncertainty, and by adapting actions to changing systems and knowledge, adaptive resource management (ARM) provides a rigorous platform for making sound decisions in a changing world. Critically, however, applications of ARM could be improved by employing benchmarks (i.e., points of reference) for determining when learning is occurring through the cycle of monitoring, modeling, and decision-making steps in ARM. Many applications of ARM use multiple model-based hypotheses to identify and reduce systematic uncertainty over time, but generally lack benchmarks for gauging discovery of scientific evidence and learning. This creates the danger of thinking that directional changes in model weights or rankings are indicative of evidence for hypotheses, when possibly all competing models are inadequate. There is thus a somewhat obvious, but yet to be filled niche for including benchmarks for learning in ARM. We contend that carefully designed "ecological null models," which are structured to produce an expected ecological pattern in the absence of a hypothesized mechanism, can serve as suitable benchmarks. Using a classic case study of mallard harvest management that is often used to demonstrate the successes of ARM for learning about ecological mechanisms, we show that simple ecological null models, such as population persistence (Nt +1 = Nt ), provide more robust near-term forecasts of population abundance than the currently used mechanistic models. More broadly, ecological null models can be used as benchmarks for learning in ARM that trigger the need for discarding model parameterizations and developing new ones when prevailing models underperform the ecological null model. Identifying mechanistic models that surpass these benchmarks will improve learning through ARM and help decision-makers keep pace with a rapidly changing world.