New parameter-free simplified swarm optimization for artificial neural network training and its application in the prediction of time series

Variable Gain Impulsive Synchronization for Discrete-Time Delayed Neural Networks and Its Application in Digital Secure Communication

This article revisits the problems of impulsive stabilization and impulsive synchronization of discrete-time delayed neural networks (DDNNs) in the presence of disturbance in the input channel. A new Lyapunov approach based on double Lyapunov functionals is introduced for analyzing exponential input-to-state stability (EISS) of discrete impulsive delayed systems. In the framework of double Lyapunov functionals, a pair of timer-dependent Lyapunov functionals are constructed for impulsive DDNNs. The pair of Lyapunov functionals can introduce more degrees of freedom that not only can be exploited to reduce the conservatism of the previous methods, but also make it possible to design variable gain impulsive controllers. New design criteria for impulsive stabilization and impulsive synchronization are derived in terms of linear matrix inequalities. Numerical results show that compared with the constant gain design technique, the proposed variable gain design technique can accept larger impulse intervals and equip the impulsive controllers with a stronger disturbance attenuation ability. Applications to digital signal encryption and image encryption are provided which validate the effectiveness of the theoretical results.

Solving Dual-Channel Supply Chain Pricing Strategy Problem with Multi-Level Programming Based on Improved Simplified Swarm Optimization

With the evolution of the Internet and the introduction of third-party platforms, a diversified supply chain has gradually emerged. In contrast to the traditional single sales channel, companies can also increase their revenue by selling through multiple channels, such as dual-channel sales: adding a sales channel for direct sales through online third-party platforms. However, due to the complexity of the supply chain structure, previous studies have rarely discussed and analyzed the capital-constrained dual-channel supply chain model, which is more relevant to the actual situation. To solve more complex and realistic supply chain decision problems, this paper uses the concept of game theory to describe the pricing negotiation procedures among the capital-constrained manufacturers and other parties in the dual-channel supply chain by applying the Stackelberg game theory to describe the supply chain structure as a hierarchical multi-level mathematical model to solve the optimal pricing strategy for different financing options to achieve the common benefit of the supply chain. In this study, we propose a Multi-level Improved Simplified Swarm Optimization (MLiSSO) method, which uses the improved, simplified swarm optimization (iSSO) for the Multi-level Programming Problem (MLPP). It is applied to this pricing strategy model of the supply chain and experiments with three related MLPPs in the past studies to verify the effectiveness of the method. The results show that the MLiSSO algorithm is effective, qualitative, and stable and can be used to solve the pricing strategy problem for supply chain models; furthermore, the algorithm can also be applied to other MLPPs.

On Adaptive Learning Framework for Deep Weighted Sparse Autoencoder: A Multiobjective Evolutionary Algorithm

In this article, an adaptive learning framework is established for a deep weighted sparse autoencoder (AE) by resorting to the multiobjective evolutionary algorithm (MOEA). The weighted sparsity is introduced to facilitate the design of the varying degrees of the sparsity constraints imposed on the hidden units of the AE. The MOEA is exploited to adaptively seek appropriate hyperparameters, where the divide-and-conquer strategy is implemented to enhance the MOEA's performance in the context of deep neural networks. Moreover, a sharing scheme is proposed to further reduce the time complexity of the learning process at the slight expense of the learning precision. It is shown via extensive experiments that the established adaptive learning framework is effective, where different sparse models are utilized to demonstrate the generality of the proposed results. Then, the generality of the proposed framework is examined on the convolutional AE and VGG-16 network. Finally, the developed framework is applied to the blind image quantity assessment that illustrates the applicability of the established algorithms.

A new automatic forecasting method based on a new input significancy test of a single multiplicative neuron model artificial neural network

The model adequacy and input significance tests have not been proposed as features for the specification of a single multiplicative neuron model artificial neural networks in the literature. Moreover, there is no systematic approach based on hypothesis tests for using single multiplicative neuron model artificial neural networks for forecasting purposes like classical time series forecasting methods. In this study, new methods are proposed to solve these problems. The performance of the proposed test procedures is investigated in a simulation study. According to simulation results, the proposed tests have very good performance. Moreover, the test procedures are illustrated by using two real-world examples. The second contribution of the paper is that an automatic forecasting method is proposed based on input significance and model adequacy tests and the particle swarm optimization-based learning algorithm. The proposed automatic forecasting method is applied to M4 competition hourly data sets, and it is the best pure machine learning method among others in the competition. The proposed automatic forecasting method is more accurate than all benchmarks, such as MLP, RNN, and ETS, which were proposed by the competition organizers.

Artificial Neural Networks Based Optimization Techniques: A Review

In the last few years, intensive research has been done to enhance artificial intelligence (AI) using optimization techniques. In this paper, we present an extensive review of artificial neural networks (ANNs) based optimization algorithm techniques with some of the famous optimization techniques, e.g., genetic algorithm (GA), particle swarm optimization (PSO), artificial bee colony (ABC), and backtracking search algorithm (BSA) and some modern developed techniques, e.g., the lightning search algorithm (LSA) and whale optimization algorithm (WOA), and many more. The entire set of such techniques is classified as algorithms based on a population where the initial population is randomly created. Input parameters are initialized within the specified range, and they can provide optimal solutions. This paper emphasizes enhancing the neural network via optimization algorithms by manipulating its tuned parameters or training parameters to obtain the best structure network pattern to dissolve the problems in the best way. This paper includes some results for improving the ANN performance by PSO, GA, ABC, and BSA optimization techniques, respectively, to search for optimal parameters, e.g., the number of neurons in the hidden layers and learning rate. The obtained neural net is used for solving energy management problems in the virtual power plant system.

Simplified Swarm Optimization for the Heterogeneous Fleet Vehicle Routing Problem with Time-Varying Continuous Speed Function

Transportation planning has been established as a key topic in the literature and practices of social production, especially in urban contexts. To consider traffic environment factors, more and more researchers are taking time-varying factors into account when scheduling their logistic activities. The time-dependent vehicle routing problem (TDVRP) is an extension of the classical Vehicle Routing Problem with Time Windows (VRPTW) by determining a set of optimal routes serving a set of customers within specific time windows. However, few of them use the continuous speed function to express the time-varying. In practice, many vehicle routing problems are addressed by a fleet of heterogeneous vehicles with different capacities and travel costs including fix costs and variable costs. In this study, a Heterogeneous Fleet Vehicle Routing Problem (HFPRP) Time-Varying Continuous Speed Function has been proposed. The objective is to minimize distribution costs, which contained fixed costs of acquiring and variable fuel costs. To address this problem, our research developed a mathematical model and proposed a Simplified Swarm Optimization (SSO) heuristic for HFVRP with Time-Varying Continuous Speed Function.

Recurrent Broad Learning Systems for Time Series Prediction

The broad learning system (BLS) is an emerging approach for effective and efficient modeling of complex systems. The inputs are transferred and placed in the feature nodes, and then sent into the enhancement nodes for nonlinear transformation. The structure of a BLS can be extended in a wide sense. Incremental learning algorithms are designed for fast learning in broad expansion. Based on the typical BLSs, a novel recurrent BLS (RBLS) is proposed in this paper. The nodes in the enhancement units of the BLS are recurrently connected, for the purpose of capturing the dynamic characteristics of a time series. A sparse autoencoder is used to extract the features from the input instead of the randomly initialized weights. In this way, the RBLS retains the merit of fast computing and fits for processing sequential data. Motivated by the idea of "fine-tuning" in deep learning, the weights in the RBLS can be updated by conjugate gradient methods if the prediction errors are large. We exhibit the merits of our proposed model on several chaotic time series. Experimental results substantiate the effectiveness of the RBLS. For chaotic benchmark datasets, the RBLS achieves very small errors, and for the real-world dataset, the performance is satisfactory.

Short-Term Wind Speed Forecasting Based on Hybrid Variational Mode Decomposition and Least Squares Support Vector Machine Optimized by Bat Algorithm Model

With the integration of wind energy into electricity grids, wind speed forecasting plays an important role in energy generation planning, power grid integration and turbine maintenance scheduling. This study proposes a hybrid wind speed forecasting model to enhance prediction performance. Variational mode decomposition (VMD) was applied to decompose the original wind speed series into different sub-series with various frequencies. A least squares support vector machine (LSSVM) model with the pertinent parameters being optimized by a bat algorithm (BA) was established to forecast those sub-series extracted from VMD. The ultimate forecast of wind speed can be obtained by accumulating the prediction values of each sub-series. The results show that: (a) VMD-BA-LSSVM displays better capacity for the prediction of ultra short-term (15 min) and short-term (1 h) wind speed forecasting; (b) the proposed forecasting model was compared with wavelet decomposition (WD) and ensemble empirical mode decomposition (EEMD), and the results indicate that VMD has stronger decomposition ability than WD and EEMD, thus, significant improvements in forecasting accuracy were obtained with the proposed forecasting models compared with other forecasting methods.

An Ant-Lion Optimizer-Trained Artificial Neural Network System for Chaotic Electroencephalogram (EEG) Prediction

The prediction of future events based on available time series measurements is a relevant research area specifically for healthcare, such as prognostics and assessments of intervention applications. A measure of brain dynamics, electroencephalogram time series, are routinely analyzed to obtain information about current, as well as future, mental states, and to detect and diagnose diseases or environmental factors. Due to their chaotic nature, electroencephalogram time series require specialized techniques for effective prediction. The objective of this study was to introduce a hybrid system developed by artificial intelligence techniques to deal with electroencephalogram time series. Both artificial neural networks and the ant-lion optimizer, which is a recent intelligent optimization technique, were employed to comprehend the related system and perform some prediction applications over electroencephalogram time series. According to the obtained findings, the system can successfully predict the future states of target time series and it even outperforms some other hybrid artificial neural network-based systems and alternative time series prediction approaches from the literature.

Solving Multi-Objective Fuzzy Optimization in Wireless Smart Sensor Networks under Uncertainty Using a Hybrid of IFR and SSO Algorithm

Wireless (smart) sensor networks (WSNs), networks made up of embedded wireless smart sensors, are an important paradigm with a wide range of applications, including the internet of things (IoT), smart grids, smart production systems, smart buildings and many others. WSNs achieve better execution efficiency if their energy consumption can be better controlled, because their component sensors are either difficult or impossible to recharge, and have a finite battery life. In addition, transmission cost must be minimized, and signal transmission quantity must be maximized to improve WSN performance. Thus, a multi-objective involving energy consumption, cost and signal transmission quantity in WSNs needs to be studied. Energy consumption, cost and signal transmission quantity usually have uncertain characteristics, and can often be represented by fuzzy numbers. Therefore, this work suggests a fuzzy simplified swarm optimization algorithm (fSSO) to resolve the multi-objective optimization problem consisting of energy consumption, cost and signal transmission quantity of the transmission process in WSNs under uncertainty. Finally, an experiment of ten benchmarks from smaller to larger scale WSNs is conducted to demonstrate the effectiveness and efficiency of the proposed fSSO algorithm.

A Squeezed Artificial Neural Network for the Symbolic Network Reliability Functions of Binary-State Networks

Network reliability is an important index to the provision of useful information for decision support in the modern world. There is always a need to calculate symbolic network reliability functions (SNRFs) due to dynamic and rapid changes in network parameters. In this brief, the proposed squeezed artificial neural network (SqANN) approach uses the Monte Carlo simulation to estimate the corresponding reliability of a given designed matrix from the Box-Behnken design, and then the Taguchi method is implemented to find the appropriate number of neurons and activation functions of the hidden layer and the output layer in ANN to evaluate SNRFs. According to the experimental results of the benchmark networks, the comparison appears to support the superiority of the proposed SqANN method over the traditional ANN-based approach with at least 16.6% improvement in the median absolute deviation in the cost of extra 2 s on average for all experiments.Network reliability is an important index to the provision of useful information for decision support in the modern world. There is always a need to calculate symbolic network reliability functions (SNRFs) due to dynamic and rapid changes in network parameters. In this brief, the proposed squeezed artificial neural network (SqANN) approach uses the Monte Carlo simulation to estimate the corresponding reliability of a given designed matrix from the Box-Behnken design, and then the Taguchi method is implemented to find the appropriate number of neurons and activation functions of the hidden layer and the output layer in ANN to evaluate SNRFs. According to the experimental results of the benchmark networks, the comparison appears to support the superiority of the proposed SqANN method over the traditional ANN-based approach with at least 16.6% improvement in the median absolute deviation in the cost of extra 2 s on average for all experiments.

A New Soft Computing Method for K-Harmonic Means Clustering

The K-harmonic means clustering algorithm (KHM) is a new clustering method used to group data such that the sum of the harmonic averages of the distances between each entity and all cluster centroids is minimized. Because it is less sensitive to initialization than K-means (KM), many researchers have recently been attracted to studying KHM. In this study, the proposed iSSO-KHM is based on an improved simplified swarm optimization (iSSO) and integrates a variable neighborhood search (VNS) for KHM clustering. As evidence of the utility of the proposed iSSO-KHM, we present extensive computational results on eight benchmark problems. From the computational results, the comparison appears to support the superiority of the proposed iSSO-KHM over previously developed algorithms for all experiments in the literature.

Impact of Noise on a Dynamical System: Prediction and Uncertainties from a Swarm-Optimized Neural Network

An artificial neural network (ANN) based on particle swarm optimization (PSO) was developed for the time series prediction. The hybrid ANN+PSO algorithm was applied on Mackey-Glass chaotic time series in the short-term x(t + 6). The performance prediction was evaluated and compared with other studies available in the literature. Also, we presented properties of the dynamical system via the study of chaotic behaviour obtained from the predicted time series. Next, the hybrid ANN+PSO algorithm was complemented with a Gaussian stochastic procedure (called stochastic hybrid ANN+PSO) in order to obtain a new estimator of the predictions, which also allowed us to compute the uncertainties of predictions for noisy Mackey-Glass chaotic time series. Thus, we studied the impact of noise for several cases with a white noise level (σ_N) from 0.01 to 0.1.

Sequential projection pursuit with kernel matrix update and symbolic model selection

This paper proposes a novel way for generating reliable low-dimensional features with improved class separability in a kernel-induced feature space. The feature projections rely on a very efficient sequential projection pursuit method, adapted to support nonlinear projections using a new kernel matrix update scheme. This enables the gradual removal of structure from the space of residual dimensions to allow the recovery of multiple projections. An adaptive kernel function is employed to unfold different types of data characteristics. We follow a holistic model selection procedure that, together with the optimal projections, dimensionality, and kernel parameters, additionally optimizes symbolically the projection index that controls the actual measurement of the data interestingness without user interaction. We tackle the underlying complex bi-level optimization model as a mixture of evolutionary and gradient search. The effectiveness of the proposed algorithm over existing approaches is demonstrated with benchmark evaluations and comparisons.

Modeling of batch processes using explicitly time-dependent artificial neural networks

A neural network architecture incorporating time dependency explicitly, proposed recently, for modeling nonlinear nonstationary dynamic systems is further developed in this paper, and three alternate configurations are proposed to represent the dynamics of batch chemical processes. The first configuration consists of L subnets, each having M inputs representing the past samples of process inputs and output; each subnet has a hidden layer with polynomial activation function; the outputs of the hidden layer are combined and acted upon by an explicitly time-dependent modulation function. The outputs of all the subnets are summed to obtain the output prediction. In the second configuration, additional weights are incorporated to obtain a more generalized model. In the third configuration, the subnets are eliminated by incorporating an additional hidden layer consisting of L nodes. Backpropagation learning algorithm is formulated for each of the proposed neural network configuration to determine the weights, the polynomial coefficients, and the modulation function parameters. The modeling capability of the proposed neural network configuration is evaluated by employing it to represent the dynamics of a batch reactor in which a consecutive reaction takes place. The results show that all the three time-varying neural networks configurations are able to represent the batch reactor dynamics accurately, and it is found that the third configuration is exhibiting comparable or better performance over the other two configurations while requiring much smaller number of parameters. The modeling ability of the third configuration is further validated by applying to modeling a semibatch polymerization reactor challenge problem. This paper illustrates that the proposed approach can be applied to represent dynamics of any batch/semibatch process.

Short-term load and wind power forecasting using neural network-based prediction intervals

Electrical power systems are evolving from today's centralized bulk systems to more decentralized systems. Penetrations of renewable energies, such as wind and solar power, significantly increase the level of uncertainty in power systems. Accurate load forecasting becomes more complex, yet more important for management of power systems. Traditional methods for generating point forecasts of load demands cannot properly handle uncertainties in system operations. To quantify potential uncertainties associated with forecasts, this paper implements a neural network (NN)-based method for the construction of prediction intervals (PIs). A newly introduced method, called lower upper bound estimation (LUBE), is applied and extended to develop PIs using NN models. A new problem formulation is proposed, which translates the primary multiobjective problem into a constrained single-objective problem. Compared with the cost function, this new formulation is closer to the primary problem and has fewer parameters. Particle swarm optimization (PSO) integrated with the mutation operator is used to solve the problem. Electrical demands from Singapore and New South Wales (Australia), as well as wind power generation from Capital Wind Farm, are used to validate the PSO-based LUBE method. Comparative results show that the proposed method can construct higher quality PIs for load and wind power generation forecasts in a short time.