CreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Tasks

Effective uncertainty estimation is becoming increasingly attractive for enhancing the reliability of neural networks. This work presents a novel approach, termed Credal-Set Interval Neural Networks (CreINNs), for classification. CreINNs retain the fundamental structure of traditional Interval Neural Networks, capturing weight uncertainty through deterministic intervals. CreINNs are designed to predict an upper and a lower probability bound for each class, rather than a single probability value. The probability intervals can define a credal set, facilitating estimating different types of uncertainties associated with predictions. Experiments on standard multiclass and binary classification tasks demonstrate that the proposed CreINNs can achieve superior or comparable quality of uncertainty estimation compared to variational Bayesian Neural Networks (BNNs) and Deep Ensembles. Furthermore, CreINNs significantly reduce the computational complexity of variational BNNs during inference. Moreover, the effective uncertainty quantification of CreINNs is also verified when the input data are intervals.

NEXT-FBA: A hybrid stoichiometric/data-driven approach to improve intracellular flux predictions

Genome-scale metabolic models (GEMs) have been widely utilized to understand cellular metabolism. The application of GEMs has been advanced by computational methods that enable the prediction and analysis of intracellular metabolic states. However, the accuracy and biological relevance of these predictions often suffer from the many degrees of freedom and scarcity of available data to constrain the models adequately. Here, we introduce Neural-net EXtracellular Trained Flux Balance Analysis, (NEXT-FBA), a novel computational methodology that addresses these limitations by utilizing exometabolomic data to derive biologically relevant constraints for intracellular fluxes in GEMs. We achieve this by training artificial neural networks (ANNs) with exometabolomic data from Chinese hamster ovary (CHO) cells and correlating it with 13C-labeled intracellular fluxomic data. By capturing the underlying relationships between exometabolomics and cell metabolism, NEXT-FBA predicts upper and lower bounds for intracellular reaction fluxes to constrain GEMs. We demonstrate the efficacy of NEXT-FBA across several validation experiments, where it outperforms existing methods in predicting intracellular flux distributions that align closely with experimental observations. Furthermore, a case study demonstrates how NEXT-FBA can guide bioprocess optimization by identifying key metabolic shifts and refining flux predictions to yield actionable process and metabolic engineering targets. Overall, NEXT-FBA aims to improve the accuracy and biological relevance of intracellular flux predictions in metabolic modelling, with minimal input data requirements for pre-trained models.

Shapley value-driven superior subset selection algorithm for carbon price interval forecast combination

Interval prediction requires not only accuracy but also the consideration of interval width and coverage, making model selection complex. However, research rarely addresses this challenge in interval combination forecasting. To address this issue, this study introduces a model selection for interval forecast combination based on the Shapley value (MSIFC-SV). This algorithm calculates Shapley values to measure each model's marginal contribution and establishes a redundancy criterion on the basis of changes in interval scores. If the removal of a model does not decrease the interval score, it is considered redundant and excluded. The selection process starts with all the models and ranks them by their Shapley values. Models are then assessed for retention or removal according to the redundancy criterion, which continues until all redundant models are excluded. The remaining subset is used to generate interval forecast combinations through interval Bayesian weighting. Empirical analysis of carbon price shows that MSIFC-SV outperforms individual models and derived subsets across metrics such as prediction interval coverage probability (PICP), mean prediction interval width (MPIW), coverage width criterion (CWC), and interval score (IS). Comparisons with benchmark methods further demonstrate the superiority of MSIFC-SV. Furthermore, MSIFC-SV is also successfully extended to the public dataset-housing price dataset, this indicates its universality. In summary, MSIFC-SV provides reliable model selection and delivers high-quality interval forecasts.

Dual Accuracy-Quality-Driven Neural Network for Prediction Interval Generation

Accurate uncertainty quantification is necessary to enhance the reliability of deep learning (DL) models in real-world applications. In the case of regression tasks, prediction intervals (PIs) should be provided along with the deterministic predictions of DL models. Such PIs are useful or "high-quality (HQ)" as long as they are sufficiently narrow and capture most of the probability density. In this article, we present a method to learn PIs for regression-based neural networks (NNs) automatically in addition to the conventional target predictions. In particular, we train two companion NNs: one that uses one output, the target estimate, and another that uses two outputs, the upper and lower bounds of the corresponding PI. Our main contribution is the design of a novel loss function for the PI-generation network that takes into account the output of the target-estimation network and has two optimization objectives: minimizing the mean PI width and ensuring the PI integrity using constraints that maximize the PI probability coverage implicitly. Furthermore, we introduce a self-adaptive coefficient that balances both objectives within the loss function, which alleviates the task of fine-tuning. Experiments using a synthetic dataset, eight benchmark datasets, and a real-world crop yield prediction dataset showed that our method was able to maintain a nominal probability coverage and produce significantly narrower PIs without detriment to its target estimation accuracy when compared to those PIs generated by three state-of-the-art neural-network-based methods. In other words, our method was shown to produce higher quality PIs.

Uncertainty-aware probabilistic graph neural networks for road-level traffic crash prediction

Traffic crashes present substantial challenges to human safety and socio-economic development in urban areas. Developing a reliable and responsible traffic crash prediction model is crucial to address growing public safety concerns and improve the safety of urban mobility systems. Traditional methods face limitations at fine spatiotemporal scales due to the sporadic nature of high-risk crashes and the predominance of non-crash characteristics. Furthermore, while most current models show promising occurrence prediction, they overlook the uncertainties arising from the inherent nature of crashes, and then fail to adequately map the hierarchical ranking of crash risk values for more precise insights. To address these issues, we introduce the Spatiotemporal Zero-Inflated Tweedie Graph Neural Networks (STZITD-GNN), the first uncertainty-aware probabilistic graph deep learning model in road-level daily-basis traffic crash prediction for multi-steps. Our model combines the interpretability of the statistical Tweedie family with the predictive power of graph neural networks, excelling in predicting a comprehensive range of crash risks. The decoder employs a compound Tweedie model, handling the non-Gaussian distribution inherent in crash data, with a zero-inflated component for accurately identifying non-crash cases and low-risk roads. The model accurately predicts and differentiates between high-risk, low-risk, and no-risk scenarios, providing a holistic view of road safety that accounts for the full spectrum of probability and severity of crashes. Empirical tests using real-world traffic data from London, UK, demonstrate that the STZITD-GNN surpasses other baseline models across multiple benchmarks, including a reduction in regression error of up to 34.60% in point estimation metrics and an improvement of above 47% in interval-based uncertainty metrics.

Energy-water management system based on robust predictive control for open-field cultivation

Food availability has been endangered by recent global events, where agriculture, the main food source for the global population, is expected to increase even more to fulfill the growing food demand. Along with food production, water and energy consumption are also increased, leading to over-extraction of groundwater and an excess emission of greenhouse gases due to fossil fuel consumption. In this context, a balance of these three resources is crucial; therefore, the water-energy-food nexus is considered to address the previous issues by designing an energy-water management system based on robust predictive control. This controller estimates the future worst-case scenario for multiple climatic conditions, such as solar radiation, ambient temperature, wind speed, precipitation, and groundwater recharge, to define an optimal irrigation volume, maximize crop growth, and minimize water consumption. At the same time, the controller schedules daily irrigation and groundwater extraction, considering energy availability from solar generation and storage, to fulfill the previously defined irrigation volume while minimizing operating costs. Climate prediction is done through fuzzy prediction intervals, whose lower or upper bound are used as worst-case to include climate uncertainty on the controller design. The energy-water management system is tested in different experiments, where results show that considering a robust approach ensures maximum crop development, avoids over-extraction of groundwater, and prioritizes renewable energy sources. This work proposes a robust energy-water management system designed to be sustainable. Considering the water-energy-food nexus, the system ensures food security and proper resource allocation, tackling global starvation, water availability, and energy access.

Stream salinity prediction in data-scarce regions: Application of transfer learning and uncertainty quantification

Scarcity of stream salinity data poses a challenge to understanding salinity dynamics and its implications for water supply management in water-scarce salt-prone regions around the world. This paper introduces a framework for generating continuous daily stream salinity estimates using instance-based transfer learning (TL) and assessing the reliability of the synthetic salinity data through uncertainty quantification via prediction intervals (PIs). The framework was developed using two temporally distinct specific conductance (SC) datasets from the Upper Red River Basin (URRB) located in southwestern Oklahoma and Texas Panhandle, United States. The instance-based TL approach was implemented by calibrating Feedforward Neural Networks (FFNNs) on a source SC dataset of around 1200 instantaneous grab samples collected by United States Geological Survey (USGS) from 1959 to 1993. The trained FFNNs were subsequently tested on a target dataset (1998-present) of 220 instantaneous grab samples collected by the Oklahoma Water Resources Board (OWRB). The framework's generalizability was assessed in the data-rich Bird Creek watershed in Oklahoma by manipulating continuous SC data to simulate data-scarce conditions for training the models and using the complete Bird Creek dataset for model evaluation. The Lower Upper Bound Estimation (LUBE) method was used with FFNNs to estimate PIs for uncertainty quantification. Autoregressive SC prediction methods via FFNN were found to be reliable with Nash Sutcliffe Efficiency (NSE) values of 0.65 and 0.45 on in-sample and out-of-sample test data, respectively. The same modeling scenario resulted in an NSE of 0.54 for the Bird Creek data using a similar missing data ratio, whereas a higher ratio of observed data increased the accuracy (NSE = 0.84). The relatively narrow estimated PIs for the North Fork Red River in the URRB indicated satisfactory stream salinity predictions, showing an average width equivalent to 25 % of the observed range and a confidence level of 70 %.

Uncertainty quantification in neural-network based pain intensity estimation

Improper pain management leads to severe physical or mental consequences, including suffering, a negative impact on quality of life, and an increased risk of opioid dependency. Assessing the presence and severity of pain is imperative to prevent such outcomes and determine the appropriate intervention. However, the evaluation of pain intensity is a challenging task because different individuals experience pain differently. To overcome this, many researchers in the field have employed machine learning models to evaluate pain intensity objectively using physiological signals. However, these efforts have primarily focused on pain point estimation, disregarding inherent uncertainty and variability in the data and model. A point estimate, which provides only partial information, is not sufficient for sound clinical decision-making. This study proposes a neural network-based method for objective pain interval estimation, and quantification of uncertainty. Our approach, which enables objective pain intensity estimation with desired confidence probabilities, affords clinicians a better understanding of a person's pain intensity. We explored three distinct algorithms: the bootstrap method, lower and upper bound estimation (LossL) optimized by genetic algorithm, and modified lower and upper bound estimation (LossS) optimized by gradient descent algorithm. Our empirical results demonstrate that LossS outperforms the other two by providing narrower prediction intervals. For 50%, 75%, 85%, and 95% prediction interval coverage probability, LossS provides average interval widths that are 22.4%, 7.9%, 16.7%, and 9.1% narrower than those of LossL, and 19.3%, 21.1%, 23.6%, and 26.9% narrower than those of bootstrap. As LossS outperforms, we assessed its performance in three different model-building approaches: (1) a generalized approach using a single model for the entire population, (2) a personalized approach with separate models for each individual, and (3) a hybrid approach with models for clusters of individuals. Results demonstrate that the hybrid model-building approach provides the best performance.

A Stochastic Recurrent Encoder Decoder Network for Multistep Probabilistic Wind Power Predictions

In this article, a stochastic recurrent encoder decoder neural network (SREDNN), which considers latent random variables in its recurrent structures, is developed for the first time for the generative multistep probabilistic wind power predictions (MPWPPs). The SREDNN enables the stochastic recurrent model under the encoder-decoder framework to engage exogenous covariates to produce better MPWPP. The SREDNN consists of five components, the prior network, the inference network, the generative network, the encoder recurrent network, and the decoder recurrent network. The SREDNN is equipped with two critical advantages compared with conventional RNN-based methods. First, the integration over the latent random variable builds an infinite Gaussian mixture model (IGMM) as the observation model, which drastically increases the expressiveness of the wind power distribution. Secondly, hidden states of the SREDNN are updated in a stochastic way, which builds an infinite mixture of the IGMM for describing the ultimate wind power distribution and enables the SREDNN to model complex patterns across wind speed and wind power sequences. Computational experiments are conducted on a dataset of a commercial wind farm having 25 wind turbines (WTs) and two publicly assessable WT datasets to verify the advantages and effectiveness of the SREDNN for MPWPP. Experimental results show that the SREDNN achieves a lower negative form of the continuously ranked probability score (CRPS*) as well as a superior sharpness and comparable reliability of prediction intervals by comparing against considered benchmarking models. Results also show the clear benefit gained from considering latent random variables in SREDNN.

Predicting mechanical properties lower upper bound for cold-rolling strip by machine learning-based artificial intelligence

The mechanical properties serve as crucial quality indicators for cold-rolled strips. For a long time, the mechanical properties mechanism and data-driven models can't comprehensively consider sufficient factors to achieve high-accuracy prediction due to the "data-isolated island" between production lines. In this research, we introduce a multi-process collaborative platform based on the industrial internet system. This platform is designed to enable real-time collection of diverse and heterogeneous data from both upstream and downstream processes of cold rolling. On this basis, a novel mechanical properties interval prediction model is proposed using the sparrow search algorithm to optimize fast learning network under the LUBE framework. We trained the model by using a dataset collected from a large steel plant. Based on the rolling theory and Pearson correlation coefficient, 25 features are selected as the inputs of the prediction model. The experimental results and comparison show that the proposed model is feasible and outperforms other machine learning models, such as the artificial bee colony algorithm optimized extreme learning machine and back propagation neural network model.

Mining human periodic behaviors via tensor factorization and entropy

Understanding human periodic behaviors is crucial in many applications. Existing research has shown the existence of periodicity in human behaviors, but has achieved limited success in leveraging location periodicity and obtaining satisfactory accuracy for oscillations in human periodic behaviors. In this article, we propose the Mobility Intention and Relative Entropy (MIRE) model to address these challenges. We employ tensor decomposition to extract mobility intentions from spatiotemporal datasets, thereby revealing hidden structures in users' historical records. Subsequently, we utilize subsequences associated with the same mobility intention to mine human periodic behaviors. Furthermore, we introduce a novel periodicity detection algorithm based on relative entropy. Our experimental results, conducted on real-world datasets, demonstrate the effectiveness of the MIRE model in accurately uncovering human periodic behaviors. Comparative analysis further reveals that the MIRE model significantly outperforms baseline periodicity detection algorithms.

An interval water demand prediction method to reduce uncertainty: A case study of Sichuan Province, China

Effective prediction of water demand is a prerequisite for decision makers to achieve reliable management of water supply. Currently, the research on water demand prediction focuses on point prediction method. In this study, we constructed a GA-BP-KDE hybrid interval water demand prediction model by combining non-parametric estimation and point prediction. Multiple metaheuristic algorithms were used to optimize the Back-Propagation Neural Network (BP) and Kernel Extreme Learning Machine (KELM) network structures. The performance of the water demand point prediction models was compared by the Root Mean Squared Error (RMSE), Mean Absolute Percentage Error (MAPE), Kling-Gupta Efficiency (KGE), computation time, and fitness convergence curves. The kernel density estimation method (KDE) and the normal distribution method were used to fit the distribution of errors. The probability density function with the best fitting degree was selected based on the index G. The shortest confidence interval under 95% confidence was calculated according to the asymmetry of the error distribution. We predicted the impact indicator values for 2025 using the exponential smoothing method, and obtained water demand prediction intervals for various water use sectors. The results showed that the GA-BP model was the optimal model as it exhibited the highest computational efficiency, algorithmic stability, and prediction accuracy. The three prediction intervals estimated after adjusting the KDE bandwidth parameter covered most of the sample points in the test set. The prediction intervals of the four water use sectors were evaluated as F values of 1.6845, 1.3294, 1.6237, and 1.3600, which indicates high accuracy and quality of the prediction intervals. The mixed water demand interval prediction based on GA-BP-KDE reduces the uncertainty of the point prediction results and can provide a basis for water resource management by decision makers.

Point and interval forecasting for carbon trading price: a case of 8 carbon trading markets in China

Carbon trading price (CTP) prediction accuracy is critical for both market participants and policymakers. As things stand, most previous studies have only focused on one or a few carbon trading markets, implying that the models' universality is insufficient to be validated. By employing a case study of all carbon trading markets in China, this study proposes a hybrid point and interval CTP forecasting model. First, the Pearson correlation method is used to identify the key influencing factors of CTP. The original CTP data is then decomposed into multiple series using complete ensemble empirical mode decomposition with adaptive noise. Following that, the sample entropy method is used to reconstruct the series to reduce computational time and avoid overdecomposition. Following that, a long short-term memory method optimized by the Adam algorithm is established to achieve the point forecasting of CTP. Finally, the kernel density estimation method is used to predict CTP intervals. On the one hand, the results demonstrate the proposed model's validity and superiority. The interval prediction model, on the other hand, reflects the uncertainty of market participants' behavior, which is more practical in the operation of carbon trading markets.

A real-time prediction interval correction method with an unscented Kalman filter for settlement monitoring of a power station dam

A prediction interval (PI) method is developed to quantify the model uncertainty of embankment settlement prediction. Traditional PIs are constructed based on specific past period information and remain unchanged; hence, they neglect discrepancies between previous calculations and new monitoring data. In this paper, a real-time prediction interval correction method is proposed. Time-varying PIs are built by continuously incorporating new measurements into model uncertainty calculations. The method consists of trend identification, PI construction, and real-time correction. Primarily, trend identification is carried out by wavelet analysis to eliminate early unstable noise and determine the settlement trend. Then, the Delta method is applied to construct PIs based on the characterized trend, and a comprehensive evaluation index is introduced. The model output and the upper and lower bounds of the PIs are updated by the unscented Kalman filter (UKF). The effect of the UKF is compared with that of the Kalman filter (KF) and extended Kalman filter (EKF). The method was demonstrated in the Qingyuan power station dam. The results show that the time-varying PIs based on trend data are smoother than those based on original data with better evaluation index scores. Also, the PIs are not affected by local anomalies. The proposed PIs are consistent with the actual measurements, and the UKF performs better than the KF and EKF. The approach has the potential to provide more reliable embankment safety assessments.

Hybrid attention-based temporal convolutional bidirectional LSTM approach for wind speed interval prediction

Precise wind speed prediction is crucial for the management of the wind power generation systems. However, the stochastic nature of the wind speed makes optimal interval prediction very complicated. In this paper, a hybrid approach consisting of improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN), temporal convolutional network with attention mechanism (ATCN), and bidirectional long short-term memory network (Bi-LSTM) is proposed for wind speed interval prediction (WSIP). First, ICEEMDAN is used to pre-process the raw data by decomposing the wind signal to several intrinsic mode functions. ATCN is used to reduce the uncertainty from the denoised data and extract the important temporal and spatial characteristics. Then, Bi-LSTM is used to forecast the high-quality intervals for the wind speed. Existing approaches observe a decline in the forecasting performance when the time ahead increases. As a result, the hybrid approach is evaluated using 5-min, 10-min, and 30-min ahead WSIP. To evaluate the novelty of the proposed approach, an experiment is conducted utilising wind speed data from the Garden City, Manhattan wind farm. The experimental results demonstrate that the proposed framework outperformed the comparison models with percentage improvements of 36%, 47%, and 17% for 5-min, 10-min, and 30-min ahead WSIP.

An overview of deterministic and probabilistic forecasting methods of wind energy

In recent years, a variety of wind forecasting models have been developed, prompting necessity to review the abundant methods to gain insights of the state-of-the-art development status. However, existing literature reviews only focus on a subclass of methods, such as multi-objective optimization and machine learning methods while lacking the full particulars of wind forecasting field. Furthermore, the classification of wind forecasting methods is unclear and incomplete, especially considering the rapid development of this field. Therefore, this article aims to provide a systematic review of the existing deterministic and probabilistic wind forecasting methods, from the perspectives of data source, model evaluation framework, technical background, theoretical basis, and model performance. It is expected that this work will provide junior researchers with broad and detailed information on wind forecasting for their future development of more accurate and practical wind forecasting models.

Hierarchical-Bayesian-Based Sparse Stochastic Configuration Networks for Construction of Prediction Intervals

To address the architecture complexity and ill-posed problems of neural networks when dealing with high-dimensional data, this article presents a Bayesian-learning-based sparse stochastic configuration network (SCN) (BSSCN). The BSSCN inherits the basic idea of training an SCN in the Bayesian framework but replaces the common Gaussian distribution with a Laplace one as the prior distribution of the output weights of SCN. Meanwhile, a lower bound of the Laplace sparse prior distribution using a two-level hierarchical prior is adopted based on which an approximate Gaussian posterior with sparse property is obtained. It leads to the facilitation of training the BSSCN, and the analytical solution for output weights of BSSCN can be obtained. Furthermore, the hyperparameter estimation process is derived by maximizing the corresponding lower bound of the marginal likelihood function based on the expectation-maximization algorithm. In addition, considering the uncertainties caused by both noises in the real-world data and model mismatch, a bootstrap ensemble strategy using BSSCN is designed to construct the prediction intervals (PIs) of the target variables. The experimental results on three benchmark data sets and two real-world high-dimensional data sets demonstrate the effectiveness of the proposed method in terms of both prediction accuracy and quality of the constructed PIs.

Interval prediction of bending force in the hot strip rolling process based on neural network and whale optimization algorithm

In the hot strip rolling process, accurate prediction of bending force is beneficial to improve the accuracy of strip crown and flatness, and further improve the strip shape quality. Due to outliers and noise are commonly present in the data generated in the rolling process, not only the prediction accuracy should be considered, but also the uncertainty of prediction results should be described quantitatively. Therefore, for the first time, the authors establish an interval prediction model for bending force in hot strip rolling process. In this paper, we use Artificial Neural Network (ANN) and whale optimization algorithm (WOA) to produce a prediction interval model (WOA-ANN) for bending force in hot strip rolling. Based on the point prediction by ANN, interval prediction is completed by using lower upper bound estimation (LUBE) and WOA, and three indexes are used to evaluate the performance of the model. This paper uses real world data from steel factory to determine the optimal network structure and parameters of the interval prediction model. Furthermore, the proposed WOA-ANN model is compared with other interval prediction models established by other three optimization algorithms. The experimental results show that the proposed WOA-ANN model has high reliability and narrow interval width, and can well complete the interval prediction of bending force in hot strip rolling. This study provides a more detailed and rigorous basis for setting bending force in hot strip rolling process.

Assessing the Impact of Features on Probabilistic Modeling of Photovoltaic Power Generation

Photovoltaic power generation has high variability and uncertainty because it is affected by uncertain factors such as weather conditions. Therefore, probabilistic forecasting is useful for optimal operation and risk hedging in power systems with large amounts of photovoltaic power generation. However, deterministic forecasting is the mainstay of photovoltaic generation forecasting; there are few studies on probabilistic forecasting and feature selection from weather or time-oriented features in such forecasting. In this study, prediction intervals were generated by the lower upper bound estimation (LUBE) using neural networks with two outputs to make probabilistic modeling for predictions. The objective was to improve prediction interval coverage probability (PICP), mean prediction interval width (MPIW), continuous ranked probability score (CRPS), and loss, which is the integration of PICP and MPIW, by removing unnecessary features through feature selection. When features with high gain were selected by random forest (RF), in the modeling of 14.7 kW PV systems, loss improved by 1.57 kW, CRPS by 0.03 kW, PICP by 0.057 kW, and MPIW by 0.12 kW on average over two weeks compared to the case where all features were used without feature selection. Therefore, the low gain features from RF act as noise and reduce the modeling accuracy.

Comparison of Bayesian, k-Nearest Neighbor and Gaussian process regression methods for quantifying uncertainty of suspended sediment concentration prediction

Suspended sediment transport in river system is a complex process influenced by many factors that their interactions lead to nonlinear and high scatter of concentration-discharge relationships. This makes the model prediction subject to high uncertainty and providing one value as the model prediction is somehow useless and cannot provide adequate information about the model accuracy and associated uncertainty. Current study compares the efficiency of Bayesian (i.e. Bayesian segmented linear regression (BSLR) and Bayesian linear model (BLR)), Gaussian Process Regression (GPR) and k-Nearest Neighbor (k-NN) in quantifying uncertainty of the suspended sediment concentration prediction in three watersheds namely Arazkoseh, Oghan and Jajrood located in Iran. Three input combinations including, contemporary discharge, slow and quick flow components and contemporary, one and two antecedent days discharge, were used. The BSLR model was able to identify threshold value, furthermore, pre-threshold and post-threshold slopes of BSLR model indicated that for Arazkoseh watershed channel and for Oghan and Jajrood watersheds, upland area are dominate sediment sources. In all three studied cases, given prediction interval width and the percent of enclosed observed data by prediction interval, k-NN model provided more reliable prediction interval. Moreover, separation stream flow into slow and quick flow components lead to improved performance of GPR and k-NN models in the studied watersheds, and the best results for Arazkoseh and Oghan watersheds were obtained when slow and quick flow components were used as the model input.

Stacked LSTM Sequence-to-Sequence Autoencoder with Feature Selection for Daily Solar Radiation Prediction: A Review and New Modeling Results

We review the latest modeling techniques and propose new hybrid SAELSTM framework based on Deep Learning (DL) to construct prediction intervals for daily Global Solar Radiation (GSR) using the Manta Ray Foraging Optimization (MRFO) feature selection to select model parameters. Features are employed as potential inputs for Long Short-Term Memory and a seq2seq SAELSTM autoencoder Deep Learning (DL) system in the final GSR prediction. Six solar energy farms in Queensland, Australia are considered to evaluate the method with predictors from Global Climate Models and ground-based observation. Comparisons are carried out among DL models (i.e., Deep Neural Network) and conventional Machine Learning algorithms (i.e., Gradient Boosting Regression, Random Forest Regression, Extremely Randomized Trees, and Adaptive Boosting Regression). The hyperparameters are deduced with grid search, and simulations demonstrate that the DL hybrid SAELSTM model is accurate compared with the other models as well as the persistence methods. The SAELSTM model obtains quality solar energy prediction intervals with high coverage probability and low interval errors. The review and new modelling results utilising an autoencoder deep learning method show that our approach is acceptable to predict solar radiation, and therefore is useful in solar energy monitoring systems to capture the stochastic variations in solar power generation due to cloud cover, aerosols, ozone changes, and other atmospheric attenuation factors.

Uncertainty Prediction for Machine Learning Models of Material Properties

Uncertainty quantification in artificial intelligence (AI)-based predictions of material properties is of immense importance for the success and reliability of AI applications in materials science. While confidence intervals are commonly reported for machine learning (ML) models, prediction intervals, i.e., the evaluation of the uncertainty on each prediction, are not as frequently available. In this work, we compare three different approaches to obtain such individual uncertainty, testing them on 12 ML-physical properties. Specifically, we investigated using the quantile loss function, machine learning the prediction intervals directly, and using Gaussian processes. We identify each approach's advantages and disadvantages and end up slightly favoring the modeling of the individual uncertainties directly, as it is the easiest to fit and, in most of the cases, minimizes over- and underestimation of the predicted errors. All data for training and testing were taken from the publicly available JARVIS-DFT database, and the codes developed for computing the prediction intervals are available through the JARVIS-tools package.

Intra-hour irradiance forecasting techniques for solar power integration: a review

The ever-growing installation of solar power systems imposes severe challenges on the operations of local and regional power grids due to the inherent intermittency and variability of ground-level solar irradiance. In recent decades, solar forecasting methodologies for intra-hour, intra-day and day-ahead energy markets have been extensively explored as cost-effective technologies to mitigate the negative effects on the power grids caused by solar power instability. In this work, the progress in intra-hour solar forecasting methodologies are comprehensively reviewed and concisely summarized. The theories behind the forecasting methodologies and how these theories are applied in various forecasting models are presented. The reviewed mathematical tools include regressive methods, stochastic learning methods, deep learning methods, and genetic algorithm. The reviewed forecasting methodologies include data-driven methods, local-sensing methods, hybrid forecasting methods, and application orientated methods that generate probabilistic forecasts and spatial forecasts. Furthermore, suggestions to accelerate the development of future intra-hour forecasting methods are provided.

Global Surface HCHO Distribution Derived from Satellite Observations with Neural Networks Technique

Formaldehyde (HCHO) is one of the most important carcinogenic air contaminants in outdoor air. However, the lack of monitoring of the global surface concentration of HCHO is currently hindering research on outdoor HCHO pollution. Traditional methods are either restricted to small areas or, for research on a global scale, too data-demanding. To alleviate this issue, we adopted neural networks to estimate the 2019 global surface HCHO concentration with confidence intervals, utilizing HCHO vertical column density data from TROPOMI, and in-situ data from HAPs (harmful air pollutants) monitoring networks and the ATom mission. Our results show that the global surface HCHO average concentration is 2.30 μg/m3. Furthermore, in terms of regions, the concentrations in the Amazon Basin, Northern China, South-east Asia, the Bay of Bengal, and Central and Western Africa are among the highest. The results from our study provide the first dataset on global surface HCHO concentration. In addition, the derived confidence intervals of surface HCHO concentration add an extra layer of confidence to our results. As a pioneering work in adopting confidence interval estimation to AI-driven atmospheric pollutant research and the first global HCHO surface distribution dataset, our paper paves the way for rigorous study of global ambient HCHO health risk and economic loss, thus providing a basis for pollution control policies worldwide.

Preventing food waste in subsidy-based university dining systems: An artificial neural network-aided model under uncertainty

Food waste planning at universities is often a complex matter due to the large volume of food and variety of services. A major portion of university food waste arises from dining systems including meal booking and distribution. Although dining systems have a significant role in generating food wastes, few studies have designed prediction models that could control such wastes based on reservation data and behavior of students at meal delivery times. To fill this gap, analyzing meal booking systems at universities, the present study proposed a new model based on machine learning to reduce the food waste generated at major universities that provide food subsidies. Students' reservation and their presence or absence at the dining hall (show/no-show rate) at mealtime were incorporated in data analysis. Given the complexity of the relationship between the attributes and the uncertainty observed in user behavior, a model was designed to analyze definite and random components of demand. An artificial neural network-based model designed for demand prediction provided a two-step prediction approach to dealing with uncertainty in actual demand. In order to estimate the lowest total cost based on the cost of waste and the shortage penalty cost, an uncertainty-based analysis was conducted at the final step of the research. This study formed a framework that could reduce the food waste volume by up to 79% and control the penalty and waste cost in the case study. The model was investigated with cost analysis and the results proved its efficiency in reducing total cost.

Uncertain Interval Forecasting for Combined Electricity-Heat-Cooling-Gas Loads in the Integrated Energy System Based on Multi-Task Learning and Multi-Kernel Extreme Learning Machine

The accurate prediction of electricity-heat-cooling-gas loads on the demand side in the integrated energy system (IES) can provide significant reference for multiple energy planning and stable operation of the IES. This paper combines the multi-task learning (MTL) method, the Bootstrap method, the improved Salp Swarm Algorithm (ISSA) and the multi-kernel extreme learning machine (MKELM) method to establish the uncertain interval prediction model of electricity-heat-cooling-gas loads. The ISSA introduces the dynamic inertia weight and chaotic local searching mechanism into the basic SSA to improve the searching speed and avoid falling into local optimum. The MKELM model is established by combining the RBF kernel function and the Poly kernel function to integrate the superior learning ability and generalization ability of the two functions. Based on the established model, weather, calendar information, social–economic factors, and historical load are selected as the input variables. Through empirical analysis and comparison discussion, we can obtain: (1) the prediction results of workday are better than those on holiday. (2) The Bootstrap-ISSA-MKELM based on the MTL method has superior performance than that based on the STL method. (3) Through comparing discussion, we discover the established uncertain interval prediction model has the superior performance in combined electricity-heat-cooling-gas loads prediction.

Bootstrapped Ensemble of Artificial Neural Networks Technique for Quantifying Uncertainty in Prediction of Wind Energy Production

The accurate prediction of wind energy production is crucial for an affordable and reliable power supply to consumers. Prediction models are used as decision-aid tools for electric grid operators to dynamically balance the energy production provided by a pool of diverse sources in the energy mix. However, different sources of uncertainty affect the predictions, providing the decision-makers with non-accurate and possibly misleading information for grid operation. In this regard, this work aims to quantify the possible sources of uncertainty that affect the predictions of wind energy production provided by an ensemble of Artificial Neural Network (ANN) models. The proposed Bootstrap (BS) technique for uncertainty quantification relies on estimating Prediction Intervals (PIs) for a predefined confidence level. The capability of the proposed BS technique is verified, considering a 34 MW wind plant located in Italy. The obtained results show that the BS technique provides a more satisfactory quantification of the uncertainty of wind energy predictions than that of a technique adopted by the wind plant owner and the Mean-Variance Estimation (MVE) technique of literature. The PIs obtained by the BS technique are also analyzed in terms of different weather conditions experienced by the wind plant and time horizons of prediction.

A Comparative Study of Machine Learning-Based Methods for Global Horizontal Irradiance Forecasting

The proliferation of photovoltaic (PV) power generation in power distribution grids induces increasing safety and service quality concerns for grid operators. The inherent variability, essentially due to meteorological conditions, of PV power generation affects the power grid reliability. In order to develop efficient monitoring and control schemes for distribution grids, reliable forecasting of the solar resource at several time horizons that are related to regulation, scheduling, dispatching, and unit commitment, is necessary. PV power generation forecasting can result from forecasting global horizontal irradiance (GHI), which is the total amount of shortwave radiation received from above by a surface horizontal to the ground. A comparative study of machine learning methods is given in this paper, with a focus on the most widely used: Gaussian process regression (GPR), support vector regression (SVR), and artificial neural networks (ANN). Two years of GHI data with a time step of 10 min are used to train the models and forecast GHI at varying time horizons, ranging from 10 min to 4 h. Persistence on the clear-sky index, also known as scaled persistence model, is included in this paper as a reference model. Three criteria are used for in-depth performance estimation: normalized root mean square error (nRMSE), dynamic mean absolute error (DMAE) and coverage width-based criterion (CWC). Results confirm that machine learning-based methods outperform the scaled persistence model. The best-performing machine learning-based methods included in this comparative study are the long short-term memory (LSTM) neural network and the GPR model using a rational quadratic kernel with automatic relevance determination.

A Novel Machine Learning-Based Framework for Optimal and Secure Operation of Static VAR Compensators in EAFs

A static VAR compensator (SVC) is a critical component for reactive power compensation in electric arc furnaces (EAFs) that is used to relieve the flicker impacts and maintain the voltage level. A weak voltage profile can not only reduce the power-quality services, but can also result in system instability in severe cases. The cybersecurity of EAFs is becoming a significant concern due to their cyber-physical structure. The reliance of SVC controllers on reactive power measurement and network communications has resulted in a cyber-vulnerability point for unauthorized access to the EAF, which can affect its normal operation. This paper addresses concerns about cyber attacks on EAFs, which can cause network communication issues in measurement data for SVCs. Three significant and different types of cyber attacks that are launched on SVC controllers—a replay attack, delay attack, and false data injection attack (FDIA)—were simulated and investigated. In order to stop the activities of cyber attacks, a secured anomaly detection model (ADM) based on a prediction interval is proposed. The proposed model is dependent on a support vector regression and a new smooth cost function for constructing the optimal and symmetrical intervals. A modified algorithm based on teaching–learning-based optimization was developed to adapt the ADM’s parameters during training. The simulation’s outcomes on a genuine dataset showed the strong capability of the proposed model against cyber attacks in EAFs.

Towards a mathematical framework to inform neural network modelling via polynomial regression

Even when neural networks are widely used in a large number of applications, they are still considered as black boxes and present some difficulties for dimensioning or evaluating their prediction error. This has led to an increasing interest in the overlapping area between neural networks and more traditional statistical methods, which can help overcome those problems. In this article, a mathematical framework relating neural networks and polynomial regression is explored by building an explicit expression for the coefficients of a polynomial regression from the weights of a given neural network, using a Taylor expansion approach. This is achieved for single hidden layer neural networks in regression problems. The validity of the proposed method depends on different factors like the distribution of the synaptic potentials or the chosen activation function. The performance of this method is empirically tested via simulation of synthetic data generated from polynomials to train neural networks with different structures and hyperparameters, showing that almost identical predictions can be obtained when certain conditions are met. Lastly, when learning from polynomial generated data, the proposed method produces polynomials that approximate correctly the data locally.