Case study in evaluating time series prediction models using the relative mean absolute error

New double decomposition deep learning methods for river water level forecasting

Forecasting river water levels or streamflow water levels (SWL) is vital to optimising the practical and sustainable use of available water resources. We propose a new deep learning hybrid model for SWL forecasting using convolutional neural networks (CNN), bi-directional long-short term memory (BiLSTM), and ant colony optimisation (ACO) with a two-phase decomposition approach at the 7-day, 14-day, and 28-day forecast horizons. The newly developed CBILSTM method is coupled with complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and variational mode decomposition (VMD) methods to extract the most significant features within predictor variables to build a hybrid CVMD-CBiLSTM model. We integrate three distinct datasets (satellite-derived, climate mode indices, and ground-based meteorological observations) to improve the forecasting capability of the CVMD-CBiLSTM model, applied at nineteen different gauging stations in the Australian Murray River system. This proposed model returns a significantly accurate performance with ~98% of all prediction errors within less than ±0.020 m and a low relative root mean square of ~0.08%, demonstrating its superiority over several benchmark models. The results show that using the new hybrid deep learning algorithm with ACO feature selection can significantly improve the accuracy of forecasted river water levels, and therefore, the method is attractive for adopting remote sensing data to the model ground-based river flow for strategic water savings planning initiatives and dealing with climate change-induced extreme events such as drought events.

Novel Epigenetic Clock Biomarkers of Age-Related Macular Degeneration

Age-Related Macular Degeneration (AMD) is a bilateral ocular condition resulting in irreversible vision impairment caused by the progressive loss of photoreceptors in the macula, a region at the center of the retina. The progressive loss of photoreceptor is a key feature of dry AMD but not always wet AMD, though both forms of AMD can lead to loss of vision. Regression-based biological age clocks are one of the most promising biomarkers of aging but have not yet been used in AMD. Here we conducted analyses to identify regression-based biological age clocks for the retina and explored their use in AMD using transcriptomic data consisting of a total of 453 retina samples including 105 Minnesota Grading System (MGS) level 1 samples, 175 MGS level 2, 112 MGS level 3 and 61 MGS level 4 samples, as well as 167 fibroblast samples. The clocks yielded good separation among AMD samples with increasing severity score viz., MGS1-4, regardless of whether clocks were trained in retina tissue, dermal fibroblasts, or in combined datasets. Clock application to cultured fibroblasts, embryonic stem cells, and induced Pluripotent Stem Cells (iPSCs) were consistent with age reprograming in iPSCs. Moreover, clock application to in vitro neuronal differentiation suggests broader applications. Interesting, many of the age clock genes identified include known targets mechanistically linked to AMD and aging, such as GDF11, C16ORF72, and FBN2. This study provides new observations for retina age clocks and suggests new applications for monitoring in vitro neuronal differentiation. These clocks could provide useful markers for AMD monitoring and possible intervention, as well as potential targets for in vitro screens.

A Trust Management Model for IoT Devices and Services Based on the Multi-Criteria Decision-Making Approach and Deep Long Short-Term Memory Technique

Recently, Internet of Things (IoT) technology has emerged in many aspects of life, such as transportation, healthcare, and even education. IoT technology incorporates several tasks to achieve the goals for which it was developed through smart services. These services are intelligent activities that allow devices to interact with the physical world to provide suitable services to users anytime and anywhere. However, the remarkable advancement of this technology has increased the number and the mechanisms of attacks. Attackers often take advantage of the IoTs' heterogeneity to cause trust problems and manipulate the behavior to delude devices' reliability and the service provided through it. Consequently, trust is one of the security challenges that threatens IoT smart services. Trust management techniques have been widely used to identify untrusted behavior and isolate untrusted objects over the past few years. However, these techniques still have many limitations like ineffectiveness when dealing with a large amount of data and continuously changing behaviors. Therefore, this paper proposes a model for trust management in IoT devices and services based on the simple multi-attribute rating technique (SMART) and long short-term memory (LSTM) algorithm. The SMART is used for calculating the trust value, while LSTM is used for identifying changes in the behavior based on the trust threshold. The effectiveness of the proposed model is evaluated using accuracy, loss rate, precision, recall, and F-measure on different data samples with different sizes. Comparisons with existing deep learning and machine learning models show superior performance with a different number of iterations. With 100 iterations, the proposed model achieved 99.87% and 99.76% of accuracy and F-measure, respectively.

Predicting virologically confirmed influenza using school absences in Allegheny County, Pennsylvania, USA during the 2007-2015 influenza seasons

Background

Children are important in community‐level influenza transmission. School‐based monitoring may inform influenza surveillance.

Methods

We used reported weekly confirmed influenza in Allegheny County during the 2007 and 2010‐2015 influenza seasons using Pennsylvania's Allegheny County Health Department all‐age influenza cases from health facilities, and all‐cause and influenza‐like illness (ILI)‐specific absences from nine county school districts. Negative binomial regression predicted influenza cases using all‐cause and illness‐specific absence rates, calendar week, average weekly temperature, and relative humidity, using four cross‐validations.

Results

School districts reported 2 184 220 all‐cause absences (2010‐2015). Three one‐season studies reported 19 577 all‐cause and 3012 ILI‐related absences (2007, 2012, 2015). Over seven seasons, 11 946 confirmed influenza cases were reported. Absences improved seasonal model fits and predictions. Multivariate models using elementary school absences outperformed middle and high school models (relative mean absolute error (relMAE) = 0.94, 0.98, 0.99). K‐5 grade‐specific absence models had lowest mean absolute errors (MAE) in cross‐validations. ILI‐specific absences performed marginally better than all‐cause absences in two years, adjusting for other covariates, but markedly worse one year.

Conclusions

Our findings suggest seasonal models including K‐5th grade absences predict all‐age‐confirmed influenza and may serve as a useful surveillance tool.

Regime shifts in the COVID-19 case fatality rate dynamics: A Markov-switching autoregressive model analysis

The 2019 novel coronavirus disease (COVID-19) has spread rapidly to many countries around the world from Wuhan, the capital of China’s Hubei province since December 2019. It has now a huge effect on the global economy. As of 13 September 2020, more than 28, 802, 775, and 920, 931 people are infected and dead, respectively. The mortality of COVID-19 infections is increasing as the number of infections increase. Many countries published control measures to contain its spread. Even though there are many drugs and vaccines under trial by pharmaceutical companies and research groups, no specific vaccine or drug has yet been found. Therefore, it is necessary to explain the behaviour of the case fatality rate (CFR) of COVID-19 using the most updated COVID-19 epidemiological data before 13 September 2020. The dynamics in the CFR were analyzed using the Markov-switching autoregressive (MSAR) models. Results showed that the two-regime and three-regime MSAR approach better captured the non-linear dynamics in the CFR time series data for each of the top heavily infected countries including the world. The results also showed that rises in CFRs are more volatile than drops. We believe that this information can be useful for the government to establish appropriate policies in a timely manner.

Incorporating human mobility data improves forecasts of Dengue fever in Thailand

Over 390 million people worldwide are infected with dengue fever each year. In the absence of an effective vaccine for general use, national control programs must rely on hospital readiness and targeted vector control to prepare for epidemics, so accurate forecasting remains an important goal. Many dengue forecasting approaches have used environmental data linked to mosquito ecology to predict when epidemics will occur, but these have had mixed results. Conversely, human mobility, an important driver in the spatial spread of infection, is often ignored. Here we compare time-series forecasts of dengue fever in Thailand, integrating epidemiological data with mobility models generated from mobile phone data. We show that geographically-distant provinces strongly connected by human travel have more highly correlated dengue incidence than weakly connected provinces of the same distance, and that incorporating mobility data improves traditional time-series forecasting approaches. Notably, no single model or class of model always outperformed others. We propose an adaptive, mosaic forecasting approach for early warning systems.

Machine learning and dengue forecasting: Comparing random forests and artificial neural networks for predicting dengue burden at national and sub-national scales in Colombia

The robust estimate and forecast capability of random forests (RF) has been widely recognized, however this ensemble machine learning method has not been widely used in mosquito-borne disease forecasting. In this study, two sets of RF models were developed at the national (pooled department-level data) and department level in Colombia to predict weekly dengue cases for 12-weeks ahead. A pooled national model based on artificial neural networks (ANN) was also developed and used as a comparator to the RF models. The various predictors included historic dengue cases, satellite-derived estimates for vegetation, precipitation, and air temperature, as well as population counts, income inequality, and education. Our RF model trained on the pooled national data was more accurate for department-specific weekly dengue cases estimation compared to a local model trained only on the department's data. Additionally, the forecast errors of the national RF model were smaller to those of the national pooled ANN model and were increased with the forecast horizon increasing from one-week-ahead (mean absolute error, MAE: 9.32) to 12-weeks ahead (MAE: 24.56). There was considerable variation in the relative importance of predictors dependent on forecast horizon. The environmental and meteorological predictors were relatively important for short-term dengue forecast horizons while socio-demographic predictors were relevant for longer-term forecast horizons. This study demonstrates the potential of RF in dengue forecasting with a feasible approach of using a national pooled model to forecast at finer spatial scales. Furthermore, including sociodemographic predictors is likely to be helpful in capturing longer-term dengue trends.

Electronically Available Patient Claims Data Improve Models for Comparing Antibiotic Use Across Hospitals: Results from 576 U.S. Facilities

Background

The Centers for Disease Control and Prevention (CDC) uses standardized antimicrobial administration ratios (SAARs)—that is, observed-to-predicted ratios—to compare antibiotic use across facilities. CDC models adjust for facility characteristics when predicting antibiotic use but do not include patient diagnoses and comorbidities that may also affect utilization. This study aimed to identify comorbidities causally related to appropriate antibiotic use and to compare models that include these comorbidities and other patient-level claims variables to a facility model for risk-adjusting inpatient antibiotic utilization.

Methods

The study included adults discharged from Premier Database hospitals in 2016–2017. For each admission, we extracted facility, claims, and antibiotic data. We evaluated 7 models to predict an admission’s antibiotic days of therapy (DOTs): a CDC facility model, models that added patient clinical constructs in varying layers of complexity, and an external validation of a published patient-variable model. We calculated hospital-specific SAARs to quantify effects on hospital rankings. Separately, we used Delphi Consensus methodology to identify Elixhauser comorbidities associated with appropriate antibiotic use.

Results

The study included 11 701 326 admissions across 576 hospitals. Compared to a CDC-facility model, a model that added Delphi-selected comorbidities and a bacterial infection indicator was more accurate for all antibiotic outcomes. For total antibiotic use, it was 24% more accurate (respective mean absolute errors: 3.11 vs 2.35 DOTs), resulting in 31–33% more hospitals moving into bottom or top usage quartiles postadjustment.

Conclusions

Adding electronically available patient claims data to facility models consistently improved antibiotic utilization predictions and yielded substantial movement in hospitals’ utilization rankings.

Hydrostats: A Python Package for Characterizing Errors between Observed and Predicted Time Series

Hydrologists use a number of tools to compare model results to observed flows. These include tools to pre-process the data, data frames to store and access data, visualization and plotting routines, error metrics for single realizations, and ensemble metrics for stochastic realizations to calibrate and evaluate hydrologic models. We present an open-source Python package to help characterize predicted and observed hydrologic time series data called hydrostats which has three main capabilities: Data storage and retrieval based on the Python Data Analysis Library (pandas), visualization and plotting routines using Matplotlib, and a metrics library that currently contains routines to compute over 70 different error metrics and routines for ensemble forecast skill scores. Hydrostats data storage and retrieval functions allow hydrologists to easily compare all, or portions of, a time series. For example, it makes it easy to compare observed and modeled data only during April over a 30-year period. The package includes literature references, explanations, examples, and source code. In this note, we introduce the hydrostats package, provide short examples of the various capabilities, and provide some background on programming issues and practices. The hydrostats package provides a range of tools to make characterizing and analyzing model data easy and efficient. The electronic supplement provides working hydrostats examples.

Prospective forecasts of annual dengue hemorrhagic fever incidence in Thailand, 2010-2014

Dengue hemorrhagic fever (DHF), a severe manifestation of dengue viral infection that can cause severe bleeding, organ impairment, and even death, affects between 15,000 and 105,000 people each year in Thailand. While all Thai provinces experience at least one DHF case most years, the distribution of cases shifts regionally from year to year. Accurately forecasting where DHF outbreaks occur before the dengue season could help public health officials prioritize public health activities. We develop statistical models that use biologically plausible covariates, observed by April each year, to forecast the cumulative DHF incidence for the remainder of the year. We perform cross-validation during the training phase (2000-2009) to select the covariates for these models. A parsimonious model based on preseason incidence outperforms the 10-y median for 65% of province-level annual forecasts, reduces the mean absolute error by 19%, and successfully forecasts outbreaks (area under the receiver operating characteristic curve = 0.84) over the testing period (2010-2014). We find that functions of past incidence contribute most strongly to model performance, whereas the importance of environmental covariates varies regionally. This work illustrates that accurate forecasts of dengue risk are possible in a policy-relevant timeframe.

Challenges in Real-Time Prediction of Infectious Disease: A Case Study of Dengue in Thailand

Epidemics of communicable diseases place a huge burden on public health infrastructures across the world. Producing accurate and actionable forecasts of infectious disease incidence at short and long time scales will improve public health response to outbreaks. However, scientists and public health officials face many obstacles in trying to create such real-time forecasts of infectious disease incidence. Dengue is a mosquito-borne virus that annually infects over 400 million people worldwide. We developed a real-time forecasting model for dengue hemorrhagic fever in the 77 provinces of Thailand. We created a practical computational infrastructure that generated multi-step predictions of dengue incidence in Thai provinces every two weeks throughout 2014. These predictions show mixed performance across provinces, out-performing seasonal baseline models in over half of provinces at a 1.5 month horizon. Additionally, to assess the degree to which delays in case reporting make long-range prediction a challenging task, we compared the performance of our real-time predictions with predictions made with fully reported data. This paper provides valuable lessons for the implementation of real-time predictions in the context of public health decision making.