An Improved Animal Migration Optimization Algorithm to Train the Feed-Forward Artificial Neural Networks

Prediction of Surface Subsidence of Deep Foundation Pit Based on Wavelet Analysis

Predicting surface settlement in deep foundation pit engineering plays a central role in the safety of foundation pit construction. Recently, static or dynamic methods are usually applied to predict ground settlement in deep foundation pit projects. In this work, we propose a model combining wavelet noise reduction and radial basis neural network (XW-RBF) to reduce noise interference in monitoring data. The results show that the XW-RBF model predicts an average relative error of 0.77 and a root average square error of 0.13. The prediction performance is better than the original data prediction results with noise structure and has higher prediction accuracy. The noise data caused by the interference of construction and the surrounding environment in the original data can be removed via the wavelet noise reduction method, with the discreteness of the original data reducing by 30%. More importantly, our results show that the XW-RBF model can reflect the law of data change to predict the future data trend with high credibility. The findings of this study indicate that the XW-RBF model could optimize the deep foundation pit settlement prediction model for high accuracy during the prediction, which inspires the potential application in deep foundation pit engineering.

Orthogonal pinhole-imaging-based learning salp swarm algorithm with self-adaptive structure for global optimization

Salp swarm algorithm (SSA) is a simple and effective bio-inspired algorithm that is gaining popularity in global optimization problems. In this paper, first, based on the pinhole imaging phenomenon and opposition-based learning mechanism, a new strategy called pinhole-imaging-based learning (PIBL) is proposed. Then, the PIBL strategy is combined with orthogonal experimental design (OED) to propose an OPIBL mechanism that helps the algorithm to jump out of the local optimum. Second, a novel effective adaptive conversion parameter method is designed to enhance the balance between exploration and exploitation ability. To validate the performance of OPLSSA, comparative experiments are conducted based on 23 widely used benchmark functions and 30 IEEE CEC2017 benchmark problems. Compared with some well-established algorithms, OPLSSA performs better in most of the benchmark problems.

Application of Meta-Heuristic Algorithms for Training Neural Networks and Deep Learning Architectures: A Comprehensive Review

The learning process and hyper-parameter optimization of artificial neural networks (ANNs) and deep learning (DL) architectures is considered one of the most challenging machine learning problems. Several past studies have used gradient-based back propagation methods to train DL architectures. However, gradient-based methods have major drawbacks such as stucking at local minimums in multi-objective cost functions, expensive execution time due to calculating gradient information with thousands of iterations and needing the cost functions to be continuous. Since training the ANNs and DLs is an NP-hard optimization problem, their structure and parameters optimization using the meta-heuristic (MH) algorithms has been considerably raised. MH algorithms can accurately formulate the optimal estimation of DL components (such as hyper-parameter, weights, number of layers, number of neurons, learning rate, etc.). This paper provides a comprehensive review of the optimization of ANNs and DLs using MH algorithms. In this paper, we have reviewed the latest developments in the use of MH algorithms in the DL and ANN methods, presented their disadvantages and advantages, and pointed out some research directions to fill the gaps between MHs and DL methods. Moreover, it has been explained that the evolutionary hybrid architecture still has limited applicability in the literature. Also, this paper classifies the latest MH algorithms in the literature to demonstrate their effectiveness in DL and ANN training for various applications. Most researchers tend to extend novel hybrid algorithms by combining MHs to optimize the hyper-parameters of DLs and ANNs. The development of hybrid MHs helps improving algorithms performance and capable of solving complex optimization problems. In general, the optimal performance of the MHs should be able to achieve a suitable trade-off between exploration and exploitation features. Hence, this paper tries to summarize various MH algorithms in terms of the convergence trend, exploration, exploitation, and the ability to avoid local minima. The integration of MH with DLs is expected to accelerate the training process in the coming few years. However, relevant publications in this way are still rare.

An improved butterfly optimization algorithm for training the feed-forward artificial neural networks

Artificial neural network (ANN) which is an information processing technique developed by modeling the nervous system of the human brain is one of the most powerful learning methods today. One of the factors that make ANN successful is its training algorithm. In this paper, an improved butterfly optimization algorithm (IBOA) based on the butterfly optimization algorithm was proposed for training the feed-forward artificial neural networks. The IBOA algorithm has the chaotic property which helps optimization algorithms to explore the search space more dynamically and globally. In the experiments, ten chaotic maps were used. The success of the IBOA algorithm was tested on 13 benchmark functions which are well known to those working on global optimization and are frequently used for testing and analysis of optimization algorithms. The Tent-mapped IBOA algorithm outperformed the other algorithms in most of the benchmark functions. Moreover, the success of the IBOA-MLP algorithm also has been tested on five classification datasets (xor, balloon, iris, breast cancer, and heart) and the IBOA-MLP algorithm was compared with four algorithms in the literature. According to the statistical performance metrics (sensitivity, specificity, precision, F1-score, and Friedman test), the IBOA-MLP outperformed the other algorithms and proved to be successful in training the feed-forward artificial neural networks.