Methodological challenges to multivariate syndromic surveillance: a case study using Swiss animal health data

Outbreak detection algorithms based on generalized linear model: a review with new practical examples

Public health surveillance serves a crucial function within health systems, enabling the monitoring, early detection, and warning of infectious diseases. Recently, outbreak detection algorithms have gained significant importance across various surveillance systems, particularly in light of the COVID-19 pandemic. These algorithms are approached from both theoretical and practical perspectives. The theoretical aspect entails the development and introduction of novel statistical methods that capture the interest of statisticians. In contrast, the practical aspect involves designing outbreak detection systems and employing diverse methodologies for monitoring syndromes, thus drawing the attention of epidemiologists and health managers. Over the past three decades, considerable efforts have been made in the field of surveillance, resulting in valuable publications that introduce new statistical methods and compare their performance. The generalized linear model (GLM) family has undergone various advancements in comparison to other statistical methods and models. This study aims to present and describe GLM-based methods, providing a coherent comparison between them. Initially, a historical overview of outbreak detection algorithms based on the GLM family is provided, highlighting commonly used methods. Furthermore, real data from Measles and COVID-19 are utilized to demonstrate examples of these methods. This study will be useful for researchers in both theoretical and practical aspects of outbreak detection methods, enabling them to familiarize themselves with the key techniques within the GLM family and facilitate comparisons, particularly for those with limited mathematical expertise.

Data-fed, needs-driven: Designing analytical workflows fit for disease surveillance

Syndromic surveillance has been an important driver for the incorporation of "big data analytics" into animal disease surveillance systems over the past decade. As the range of data sources to which automated data digitalization can be applied continues to grow, we discuss how to move beyond questions around the means to handle volume, variety and velocity, so as to ensure that the information generated is fit for disease surveillance purposes. We make the case that the value of data-driven surveillance depends on a "needs-driven" design approach to data digitalization and information delivery and highlight some of the current challenges and research frontiers in syndromic surveillance.

A Unifying Framework and Comparative Evaluation of Statistical and Machine Learning Approaches to Non-Specific Syndromic Surveillance

Monitoring the development of infectious diseases is of great importance for the prevention of major outbreaks. Syndromic surveillance aims at developing algorithms which can detect outbreaks as early as possible by monitoring data sources which allow to capture the occurrences of a certain disease. Recent research mainly concentrates on the surveillance of specific, known diseases, putting the focus on the definition of the disease pattern under surveillance. Until now, only little effort has been devoted to what we call non-specific syndromic surveillance, i.e., the use of all available data for detecting any kind of infectious disease outbreaks. In this work, we give an overview of non-specific syndromic surveillance from the perspective of machine learning and propose a unified framework based on global and local modeling techniques. We also present a set of statistical modeling techniques which have not been used in a local modeling context before and can serve as benchmarks for the more elaborate machine learning approaches. In an experimental comparison of different approaches to non-specific syndromic surveillance we found that these simple statistical techniques already achieve competitive results and sometimes even outperform more elaborate approaches. In particular, applying common syndromic surveillance methods in a non-specific setting seems to be promising.

Factors associated with cattle necropsy submissions in Switzerland, and their importance for surveillance

Pathology data have been reported to be important for surveillance, as they are crucial for correctly recognizing and identifying new or re-emerging diseases in animal populations. However, there are no reports in the literature of necropsy data being compared or complemented with other data. In our study, we compared cattle necropsy reports extracted from 3 laboratories with the Swiss fallen stock data and clinical data collected by the association of Swiss Cattle Breeders. The objective was to assess the completeness, validity and representativeness of the necropsy data, as well as evaluate potential factors for necropsy submission and how they can benefit animal health surveillance. Our results showed that, on average, 1% of Swiss cattle that die are submitted for post-mortem examinations. However, different factors influence cattle necropsy submissions, such as the age of the animal, the geographical location and the number of sick and/or dead animals on the farm. There was a median of five animals reported sick and two animals reported dead within 30 days prior to a necropsy submission, providing quantitative evidence of a correlation between on farm morbidity/mortality and post-mortem examination. Our results also showed that necropsy data can help improve the accuracy and completeness of health data for surveillance systems. In this study, we were able to demonstrate the importance of veterinary pathology data for AHS by providing quantitative evidence that necropsied animals are indicative of farms with important disease problems and are therefore critically important for surveillance. Furthermore, thanks to the amount of information provided by combined data sources, the epidemiology (e.g. season, geographic region, risk factors) of potential diseases can be analysed more precisely and help supporting animal health surveillance systems.

Autoregressive Models Applied to Time-Series Data in Veterinary Science

A time-series is any set of N time-ordered observations of a process. In veterinary epidemiology, our focus is generally on disease occurrence (the “process”) over time, but animal production, welfare or other traits might also be of interest. A common source of time-series datasets are animal disease monitoring and surveillance systems. Here, we scan the application of methods to analyse time-series data in the peer-reviewed, published literature. Based on this literature scan we focus on autocorrelation and illustrate the recommended steps using ARIMA (Autoregressive Integrated Moving Average Models) methods via analysis of a time-series of canine parvovirus (CPV) events in a pet dog population in Australia, 2009 to 2015. We conclude by identifying the barriers to the application of ARIMA methods in veterinary epidemiology and suggest some possible solutions. In the literature scan the selected 37 studies focused mostly on infectious and parasitic diseases, predominantly for analytical, rather than descriptive or predictive, purposes. Trends and seasonality were investigated, and autocorrelation analyzed, in most studies, most commonly using R software. An approach to analyzing autocorrelation using ARIMA methods was then illustrated using a time-series (week and month units) of CPV events in a pet dog population in Australia, reported to a national companion animal disease surveillance system. This time-series was derived by summing veterinarian reports of confirmed CPV diagnoses. We present data analysis output generated via the R statistical environment, and make this code available for the reader to apply to this or other time-series datasets. We also illustrate prediction of CPV events by rainfall as a covariate. Time-series analysis using ARIMA methods to understand and explore autocorrelation appears to be relatively uncommon in veterinary epidemiology. Some of the reasons might include limited availability of data of sufficient time unit length, lack of familiarity with analytical methods and available software, and how to best use the information generated. We recommend that wherever feasible, such time-series data be made available both for analysis and for methods development.

Comparison of statistical algorithms for daily syndromic surveillance aberration detection

Motivation

Public health authorities can provide more effective and timely interventions to protect populations during health events if they have effective multi-purpose surveillance systems. These systems rely on aberration detection algorithms to identify potential threats within large datasets. Ensuring the algorithms are sensitive, specific and timely is crucial for protecting public health. Here, we evaluate the performance of three detection algorithms extensively used for syndromic surveillance: the ‘rising activity, multilevel mixed effects, indicator emphasis’ (RAMMIE) method and the improved quasi-Poisson regression-based method known as ‘Farrington Flexible’ both currently used at Public Health England, and the ‘Early Aberration Reporting System’ (EARS) method used at the US Centre for Disease Control and Prevention. We model the wide range of data structures encountered within the daily syndromic surveillance systems used by PHE. We undertake extensive simulations to identify which algorithms work best across different types of syndromes and different outbreak sizes. We evaluate RAMMIE for the first time since its introduction. Performance metrics were computed and compared in the presence of a range of simulated outbreak types that were added to baseline data.

Results

We conclude that amongst the algorithm variants that have a high specificity (i.e. >90%), Farrington Flexible has the highest sensitivity and specificity, whereas RAMMIE has the highest probability of outbreak detection and is the most timely, typically detecting outbreaks 2–3 days earlier.

Availability and implementation

R codes developed for this project are available through https://github.com/FelipeJColon/AlgorithmComparison

Supplementary information

Supplementary data are available at Bioinformatics online.

Multichart Schemes for Detecting Changes in Disease Incidence

Several methods have been proposed in open literatures for detecting changes in disease outbreak or incidence. Most of these methods are likelihood-based as well as the direct application of Shewhart, CUSUM and EWMA schemes. We use CUSUM, EWMA and EWMA-CUSUM multi-chart schemes to detect changes in disease incidence. Multi-chart is a combination of several single charts that detects changes in a process and have been shown to have elegant properties in the sense that they are fast in detecting changes in a process as well as being computationally less expensive. Simulation results show that the multi-CUSUM chart is faster than EWMA and EWMA-CUSUM multi-charts in detecting shifts in the rate parameter. A real illustration with health data is used to demonstrate the efficiency of the schemes.

An integer-valued time series model for multivariate surveillance

In recent days, different types of surveillance data are becoming available for public health purposes. In most cases, several variables are monitored and events of different types are reported. As the amount of surveillance data increases, statistical methods that can effectively address multivariate surveillance scenarios are demanded. Even though research activity in this field is increasing rapidly in recent years, only a few approaches have simultaneously addressed the integer-valued property of the data and its correlation (both time correlation and cross-correlation) structure. In this article, we suggest a multivariate integer-valued autoregressive model that allows for both serial and cross-correlations between the series and can easily accommodate overdispersion and covariate information. Moreover, its structure implies a natural decomposition into an endemic and an epidemic component, a common distinction in dynamic models for infectious disease counts. Detection of disease outbreaks is achieved through the comparison of surveillance data with one-step-ahead predictions obtained after fitting the suggested model to a set of clean historical data. The performance of the suggested model is illustrated on a trivariate series of syndromic surveillance data collected during Athens 2004 Olympic Games.

Simulation Based Evaluation of Time Series for Syndromic Surveillance of Cattle in Switzerland

Choosing the syndrome time series to monitor in a syndromic surveillance system is not a straight forward process. Defining which syndromes to monitor in order to maximize detection performance has been recently identified as one of the research priorities in Syndromic surveillance. Estimating the minimum size of an epidemic that could potentially be detected in a specific syndrome could be used as a criteria for comparing the performance of different syndrome time series, and could provide some guidance for syndrome selection. The aim of our study was to estimate the potential value of different time series for building a national syndromic surveillance system for cattle in Switzerland. Simulations were used to produce outbreaks of different size and shape and to estimate the ability of each time series and aberration detection algorithm to detect them with high sensitivity, specificity and timeliness. Two temporal aberration detection algorithms were also compared: Holt-Winters generalized exponential smoothing (HW) and Exponential Weighted Moving Average (EWMA). Our results indicated that a specific aberration detection algorithm should be used for each time series. In addition, time series with high counts per unit of time had good overall detection performance, but poor detection performance for small epidemics making them of limited use for an early detection system. Estimating the minimum size of simulated epidemics that could potentially be detected in syndrome TS-event detection pairs can help surveillance system designers choosing the most appropriate syndrome TS to include in their early epidemic surveillance system.

Multivariate syndromic surveillance for cattle diseases: Epidemic simulation and algorithm performance evaluation

Multivariate Syndromic Surveillance (SyS) systems that simultaneously assess and combine information from different data sources are especially useful for strengthening surveillance systems for early detection of infectious disease epidemics. Despite the strong motivation for implementing multivariate SyS and there being numerous methods reported, the number of operational multivariate SyS systems in veterinary medicine is still very small. One possible reason is that assessing the performance of such surveillance systems remains challenging because field epidemic data are often unavailable. The objective of this study is to demonstrate a practical multivariate event detection method (directionally sensitive multivariate control charts) that can be easily applied in livestock disease SyS, using syndrome time series data from the Swiss cattle population as an example. We present a standardized method for simulating multivariate epidemics of different diseases using four diseases as examples: Bovine Virus Diarrhea (BVD), Infectious Bovine Rhinotracheitis (IBR), Bluetongue virus (BTV) and Schmallenberg virus (SV). Two directional multivariate control chart algorithms, Multivariate Exponentially Weighted Moving Average (MEWMA) and Multivariate Cumulative Sum (MCUSUM) were compared. The two algorithms were evaluated using 12 syndrome time series extracted from two Swiss national databases. The two algorithms were able to detect all simulated epidemics around 4.5 months after the start of the epidemic, with a specificity of 95%. However, the results varied depending on the algorithm and the disease. The MEWMA algorithm always detected epidemics earlier than the MCUSUM, and epidemics of IBR and SV were detected earlier than epidemics of BVD and BTV. Our results show that the two directional multivariate control charts are promising methods for combining information from multiple time series for early detection of subtle changes in time series from a population without producing an unreasonable amount of false alarms. The approach that we used for simulating multivariate epidemics is relatively easy to implement and could be used in other situations where real epidemic data are unavailable. We believe that our study results can support the implementation and assessment of multivariate SyS systems in animal health.

Value of evidence from syndromic surveillance with cumulative evidence from multiple data streams with delayed reporting

Delayed reporting of health data may hamper the early detection of infectious diseases in surveillance systems. Furthermore, combining multiple data streams, e.g. aiming at improving a system's sensitivity, can be challenging. In this study, we used a Bayesian framework where the result is presented as the value of evidence, i.e. the likelihood ratio for the evidence under outbreak versus baseline conditions. Based on a historical data set of routinely collected cattle mortality events, we evaluated outbreak detection performance (sensitivity, time to detection, in-control run length) under the Bayesian approach among three scenarios: presence of delayed data reporting, but not accounting for it; presence of delayed data reporting accounted for; and absence of delayed data reporting (i.e. an ideal system). Performance on larger and smaller outbreaks was compared with a classical approach, considering syndromes separately or combined. We found that the Bayesian approach performed better than the classical approach, especially for the smaller outbreaks. Furthermore, the Bayesian approach performed similarly well in the scenario where delayed reporting was accounted for to the scenario where it was absent. We argue that the value of evidence framework may be suitable for surveillance systems with multiple syndromes and delayed reporting of data.