Nonpooling Convolutional Neural Network Forecasting for Seasonal Time Series With Trends

Prediction of quality traits in packaged mango by NIR spectroscopy

Packaging with paper bags is essential for protecting mangoes during growth, but strongly berries near infrared (NIR) spectral signals used for non-invasive analysis of their internal quality. This study focused on eliminating or minimizing the interference of paper bags on the NIR spectra of mangoes and developed innovative solutions to accurately assess mango firmness (FI), dry matter content (DMC), soluble solids content (SSC), and titratable acidity (TA). Specific NIR signals at around 1150-1250 nm and 2100-2400 nm were highlighted, which significantly reduced the precision of NIR predictions for these quality traits. A deep learning-based fully connected neural network (FNN) combined with Gaussian spatial (GS) filtering were applied as an effective strategy to mitigate the spectral interferences of packaged mangoes. Additionally, partial least squares regression (PLSR) consistently outperformed principal components regression (PCR) across all quality traits based on the spectra of packaged mangoes after FNN correction and GS filtering (PMs-FNN-GS), with Rp2 and RMSEP values of 0.847 and 10.705 N, 0.932 and 0.320 % for DMC, 0.821 and 1.211 % for SSC, and 0.907 and 0.032 % for TA, demonstrating that reliable predictive accuracy for packaged mangoes was effectively achieved through the combination of deep learning and GS filtering.

Multivariate Time Series Forecasting Using Multiscale Recurrent Networks With Scale Attention and Cross-Scale Guidance

Multivariate time series (MTS) forecasting is considered as a challenging task due to complex and nonlinear interdependencies between time steps and series. With the advance of deep learning, significant efforts have been made to model long-term and short-term temporal patterns hidden in historical information by recurrent neural networks (RNNs) with a temporal attention mechanism. Although various forecasting models have been developed, most of them are single-scale oriented, resulting in scale information loss. In this article, we seamlessly integrate multiscale analysis into deep learning frameworks to build scale-aware recurrent networks and propose two multiscale recurrent network (MRN) models for MTS forecasting. The first model called MRN-SA adopts a scale attention mechanism to dynamically select the most relevant information from different scales and simultaneously employs input attention and temporal attention to make predictions. The second one named as MRN-CSG introduces a novel cross-scale guidance mechanism to exploit the information from coarse scale to guide the decoding process at fine scale, which results in a lightweight and more easily trained model without obvious loss of accuracy. Extensive experimental results demonstrate that both MRN-SA and MRN-CSG can achieve state-of-the-art performance on five typical MTS datasets in different domains. The source codes will be publicly available at https://github.com/qguo2010/MRN.

Representation-Learning-Based CNN for Intelligent Attack Localization and Recovery of Cyber-Physical Power Systems

Enabled by the advances in communication networks, computational units, and control systems, cyber-physical power systems (CPPSs) are anticipated to be complex and smart systems in which a large amount of data are generated, exchanged, and processed for various purposes. Due to these strong interactions, CPPSs will introduce new security vulnerabilities. To ensure secure operation and control of CPPSs, it is essential to detect the locations of the attacked measurements and remove the state bias caused by malicious cyber-attacks such as false data inject attack, jamming attack, denial of service attack, or hybrid attack. Accordingly, this article makes the first contribution concerning the representation-learning-based convolutional neural network (RL-CNN) for intelligent attack localization and system recovery of CPPSs. In the proposed method, the cyber-attacks' locational detection problem is formulated as a multilabel classification problem for CPPSs. An RL-CNN is originally adopted as the multilabel classifier to explore and exploit the implicit information of measurements. By comparing with previous multilabel classifiers, the RL-CNN improves the performance of attack localization for complex CPPSs. Then, to automatically filter out the cyber-attacks for system recovery, a mean-squared estimator is used to handle the difficulty in state estimation with the removal of contaminated measurements. In this scheme, prior knowledge of the system state is obtained based on the outputs of the stochastic power flow or historical measurements. The extensive simulation results in three IEEE bus systems show that the proposed method is able to provide high accuracy for attack localization and perform automatic attack filtering for system recovery under various cyber-attacks.

An IoT framework for quality analysis of aquatic water data using time-series convolutional neural network

Water quality monitoring and analysis in fish farms are of paramount importance for the aquaculture sector; however, traditional methods can pose difficulties. To address this challenge, this study proposes an IoT-based deep learning model using a time-series convolution neural network (TMS-CNN) for monitoring and analyzing water quality in fish farms. The proposed TMS-CNN model can handle spatial-temporal data effectively by considering temporal and spatial dependencies between data points, which allows it to capture patterns and trends that would not be possible with traditional models. The model calculates the water quality index (WQI) using correlation analysis and assigns class labels to the data based on the WQI. Then, the TMS-CNN model analyzed the time-series data. It produces high accuracy of 96.2% in analysis of water quality parameters for fish growth and mortality conditions. The proposed model accuracy is higher than the current best model MANN, which has only had an accuracy of 91%.

Forecast evaluation for data scientists: common pitfalls and best practices

Recent trends in the Machine Learning (ML) and in particular Deep Learning (DL) domains have demonstrated that with the availability of massive amounts of time series, ML and DL techniques are competitive in time series forecasting. Nevertheless, the different forms of non-stationarities associated with time series challenge the capabilities of data-driven ML models. Furthermore, due to the domain of forecasting being fostered mainly by statisticians and econometricians over the years, the concepts related to forecast evaluation are not the mainstream knowledge among ML researchers. We demonstrate in our work that as a consequence, ML researchers oftentimes adopt flawed evaluation practices which results in spurious conclusions suggesting methods that are not competitive in reality to be seemingly competitive. Therefore, in this work we provide a tutorial-like compilation of the details associated with forecast evaluation. This way, we intend to impart the information associated with forecast evaluation to fit the context of ML, as means of bridging the knowledge gap between traditional methods of forecasting and adopting current state-of-the-art ML techniques.We elaborate the details of the different problematic characteristics of time series such as non-normality and non-stationarities and how they are associated with common pitfalls in forecast evaluation. Best practices in forecast evaluation are outlined with respect to the different steps such as data partitioning, error calculation, statistical testing, and others. Further guidelines are also provided along selecting valid and suitable error measures depending on the specific characteristics of the dataset at hand.

A Hybrid System Based on Dynamic Selection for Time Series Forecasting

Hybrid systems, which combine statistical and machine learning (ML) techniques using residual (error forecasting) modeling, have been highlighted in the literature due to their accuracy and ability to forecast time series with different characteristics. In these architectures, a crucial task is the proper modeling of the residuals since they may present random fluctuations, complex nonlinear patterns, and heteroscedastic behavior. Hence, the selection, specification, and training of one ML model to forecast the residuals are costly and challenging tasks since issues, such as underfitting, overfitting, and misspecification, can lead to a system with low accuracy or even deteriorate the linear forecast of the time series. This article proposes a hybrid system, named dynamic residual forecasting (DReF), that employs a modified dynamic selection (DS) algorithm to decide: the most suitable ML model to forecast a pattern of the residual series and if it is a promising candidate to increase the accuracy of the time series forecast from the linear combination. Thus, the DReF aims to reduce the uncertainty of the ML model selection and avoid the deterioration of the time series forecast. Furthermore, the proposed system searches for the most suitable parameters of the DS algorithm for each data set. In this article, the proposed method uses a pool of five ML models widely adopted in the literature: multilayer perceptron, support vector regression, radial basis function, long short-term memory, and convolutional neural network. An experimental evaluation was conducted using ten well-known time series. The results show that the DReF obtains superior results for the majority of the data sets compared with single and hybrid models of the literature.

Unsupervised Domain Adaptive 1D-CNN for Fault Diagnosis of Bearing

Fault diagnosis (FD) plays a vital role in building a smart factory regarding system reliability improvement and cost reduction. Recent deep learning-based methods have been applied for FD and have obtained excellent performance. However, most of them require sufficient historical labeled data to train the model which is difficult and sometimes not available. Moreover, the big size model increases the difficulties for real-time FD. Therefore, this article proposed a domain adaptive and lightweight framework for FD based on a one-dimension convolutional neural network (1D-CNN). Particularly, 1D-CNN is designed with a structure of autoencoder to extract the rich, robust hidden features with less noise from source and target data. The extracted features are processed by correlation alignment (CORAL) to minimize domain shifts. Thus, the proposed method could learn robust and domain-invariance features from raw signals without any historical labeled target domain data for FD. We designed, trained, and tested the proposed method on CRWU bearing data sets. The sufficient comparative analysis confirmed its effectiveness for FD.

A Novel Encoder-Decoder Model for Multivariate Time Series Forecasting

The time series is a kind of complex structure data, which contains some special characteristics such as high dimension, dynamic, and high noise. Moreover, multivariate time series (MTS) has become a crucial study in data mining. The MTS utilizes the historical data to forecast its variation trend and has turned into one of the hotspots. In the era of rapid information development and big data, accurate prediction of MTS has attracted much attention. In this paper, a novel deep learning architecture based on the encoder-decoder framework is proposed for MTS forecasting. In this architecture, firstly, the gated recurrent unit (GRU) is taken as the main unit structure of both the procedures in encoding and decoding to extract the useful successive feature information. Then, different from the existing models, the attention mechanism (AM) is introduced to exploit the importance of different historical data for reconstruction at the decoding stage. Meanwhile, feature reuse is realized by skip connections based on the residual network for alleviating the influence of previous features on data reconstruction. Finally, in order to enhance the performance and the discriminative ability of the new MTS, the convolutional structure and fully connected module are established. Furthermore, to better validate the effectiveness of MTS forecasting, extensive experiments are executed on two different types of MTS such as stock data and shared bicycle data, respectively. The experimental results adequately demonstrate the effectiveness and the feasibility of the proposed method.

Accurate Deep Model for Electricity Consumption Forecasting Using Multi-channel and Multi-Scale Feature Fusion CNN–LSTM

Electricity consumption forecasting is a vital task for smart grid building regarding the supply and demand of electric power. Many pieces of research focused on the factors of weather, holidays, and temperatures for electricity forecasting that requires to collect those data by using kinds of sensors, which raises the cost of time and resources. Besides, most of the existing methods only focused on one or two types of forecasts, which cannot satisfy the actual needs of decision-making. This paper proposes a novel hybrid deep model for multiple forecasts by combining Convolutional Neural Networks (CNN) and Long-Short Term Memory (LSTM) algorithm without additional sensor data, and also considers the corresponding statistics. Different from the conventional stacked CNN–LSTM, in the proposed hybrid model, CNN and LSTM extracted features in parallel, which can obtain more robust features with less loss of original information. Chiefly, CNN extracts multi-scale robust features by various filters at three levels and wide convolution technology. LSTM extracts the features which think about the impact of different time-steps. The features extracted by CNN and LSTM are combined with six statistical components as comprehensive features. Therefore, comprehensive features are the fusion of multi-scale, multi-domain (time and statistic domain) and robust due to the utilization of wide convolution technology. We validate the effectiveness of the proposed method on three natural subsets associated with electricity consumption. The comparative study shows the state-of-the-art performance of the proposed hybrid deep model with good robustness for very short-term, short-term, medium-term, and long-term electricity consumption forecasting.