A genetic algorithm to obtain the optimal recurrent neural network

Artificial Neural Network Modeling and Genetic Algorithm Multiobjective Optimization of Process of Drying-Assisted Walnut Breaking

This study combined an artificial neural network (ANN) with a genetic algorithm (GA) to obtain the model and optimal process parameters of drying-assisted walnut breaking. Walnuts were dried at different IR temperatures (40 °C, 45 °C, 50 °C, and 55 °C) and air velocities (1, 2, 3, and 4 m/s) to different moisture contents (10%, 15%, 20%, and 25%) by using air-impingement technology. Subsequently, the dried walnuts were broken in different loading directions (sutural, longitudinal, and vertical). The drying time (DT), specific energy consumption (SEC), high kernel rate (HR), whole kernel rate (WR), and shell-breaking rate (SR) were determined as response variables. An ANN optimized by a GA was applied to simulate the influence of IR temperature, air velocity, moisture content, and loading direction on the five response variables, from which the objective functions of DT, SEC, HR, WR, and SR were developed. A GA was applied for the simultaneous maximization of HR, WR, and SR and minimization of DT and SEC to determine the optimized process parameters. The ANN model had a satisfactory prediction ability, with the coefficients of determination of 0.996, 0.998, 0.990, 0.991, and 0.993 for DT, SEC, HR, WR, and SR, respectively. The optimized process parameters were found to be 54.9 °C of IR temperature, 3.66 m/s of air velocity, 10.9% of moisture content, and vertical loading direction. The model combining an ANN and a GA was proven to be an effective method for predicting and optimizing the process parameters of walnut breaking. The predicted values under optimized process parameters fitted the experimental data well, with a low relative error value of 2.51-3.96%. This study can help improve the quality of walnut breaking, processing efficiency, and energy conservation. The ANN modeling and GA multiobjective optimization method developed in this study provide references for the process optimization of walnut and other similar commodities.

Designing an optimal neural network architecture: an application to property valuation

Purpose

The success of a neural network depends on, among others, an architecture that is appropriate for the task at hand. This study attempts to identify an optimal architecture of a neural network in the context of property valuation, and aims to test the ability of connecting related neural networks to reduce the property valuation error.

Design/methodology/approach

This study explores efficient network architectures to estimate land and house prices in Seoul, South Korea. The input is structured data, and the embedding technique is used to process high-cardinality categorical variables.

Findings

The shared architecture of a network for simultaneous estimation of both land and houses was revealed to be the best performing network. Through weight sharing between relevant layers in networks, the root-mean-square error (RMSE) for land price estimation was reduced significantly, from 0.55–0.68 using the baseline architecture, to 0.44–0.47 using the shared architecture.

Originality/value

The study results are expected to encourage active investigation of efficient architectures by using domain knowledge, and to promote interest in using structured data, which is still the dominant type in most industries.

Performance assessment of artificial neural networks and support vector regression models for stream flow predictions

Water resources planning, development, and management need reliable forecasts of river flows. In past few decades, an important dimension has been introduced in the prediction of the hydrologic phenomenon through artificial intelligence-based modeling. In this paper, the performance of three artificial neural network (ANN) and four support vector regression (SVR) models was investigated to predict streamflows in the Upper Indus River. Results from ANN models using three different optimization techniques, namely Broyden-Fletcher-Goldfarb-Shannon, Conjugate Gradient, and Back Propagation algorithms, were compared with one another. A further comparison was made between these ANNs and four types of SVR models which were based on linear, polynomial, radial basis function, and sigmoid kernels. Past 30 years' monthly data for precipitation, temperature, and streamflow obtained from Pakistan Surface Water Hydrology Department Lahore were used for this purpose. Three types of input combinations with respect to the main input variables (temperature, precipitation, and stream flow) and several types of input combinations with respect to time lag were tested. The best input for ANN and SVR models was identified using correlation coefficient analysis and genetic algorithm. The performance of the ANN and SVR models was evaluated by mean bias error, Nash-Sutcliffe efficiency, root mean square error, and correlation coefficient. The efficiency of the Broyden-Fletcher-Goldfarb-Shannon-ANN model was found to be much better than that of other models, while the SVR model based on radial basis function kernel predicted stream flows with comparatively higher accuracy than the other kernels. Finally, long-term predictions of streamflow have been made by the best ANN model. It was found that stream flow of Upper Indus River has a decreasing trend.

OPTIMIZING FEEDFORWARD NEURAL NETWORKS FOR CONTROL CHART PATTERN RECOGNITION THROUGH GENETIC ALGORITHMS

Pattern recognition is an important issue in statistical process control (SPC) because unnatural patterns exhibited by control charts can be associated with specific assignable causes adversely affecting the process. Artificial neural networks have been widely investigated as an effective approach to control chart pattern (CCP) recognition in recent years. However, an overwhelming majority of these applications has used trial-and-error experiments to determine the network architecture and training parameters, which are crucial to the performance of the network. In this paper, the genetic algorithm (GA) is used to evolve the configuration and the training parameter set of the neural network to solve the online CCP recognition problem. Numerical results are provided that indicate that the proposed GA can evolve neural network architecture while simultaneously determining training parameters to maximize efficiently the performance of the online CCP recognizers. Because the population size is a major parameter of GA processing speed, an investigation was also conducted to identify the effects of the population size on the performance of the proposed GA. This research further confirms the feasibility of using GA to evolve neural networks. Although a back-propagation-based CCP recognizer is the particular application presented here, the proposed GA methodology can be applied to neural networks in general.

Multiobjective hybrid optimization and training of recurrent neural networks

The application of neural networks to solve a problem involves tasks with a high computational cost until a suitable network is found, and these tasks mainly involve the selection of the network topology and the training step. We usually select the network structure by means of a trial-and-error procedure, and we then train the network. In the case of recurrent neural networks (RNNs), the lack of suitable training algorithms sometimes hampers these procedures due to vanishing gradient problems. This paper addresses the simultaneous training and topology optimization of RNNs using multiobjective hybrid procedures. The proposal is based on the SPEA2 and NSGA2 algorithms for making hybrid methods using the Baldwinian hybridization strategy. We also study the effects of the selection of the objectives, crossover, and mutation in the diversity during evolution. The proposals are tested in the experimental section to train and optimize the networks in the competition on artificial time-series (CATS) benchmark.

Classification of neck movement patterns related to whiplash-associated disorders using neural networks

This paper presents a new method for classification of neck movement patterns related to Whiplash-associated disorders (WAD) using a resilient backpropagation neural network (BPNN). WAD are a common diagnosis after neck trauma, typically caused by rear-end car accidents. Since physical injuries seldom are found with present imaging techniques, the diagnosis can be difficult to make. The active range of the neck is often visually inspected in patients with neck pain, but this is a subjective measure, and a more objective decision support system, that gives a reliable and more detailed analysis of neck movement pattern, is needed. The objective of this study was to evaluate the predictive ability of a BPNN, using neck movement variables as input. Three-dimensional (3-D) neck movement data from 59 subjects with WAD and 56 control subjects were collected with a ProReflex system. Rotation angle and angle velocity were calculated using the instantaneous helical axis method and motion variables were extracted. A principal component analysis was performed in order to reduce data and improve the BPNN performance. BPNNs with six hidden nodes had a predictivity of 0.89, a sensitivity of 0.90 and a specificity of 0.88, which are very promising results. This shows that neck movement analysis combined with a neural network could build the basis of a decision support system for classifying suspected WAD, even though further evaluation of the method is needed.

High-Order Neural Network Structure Selection for Function Approximation Applications Using Genetic Algorithms

Neural network literature for function approximation is by now sufficiently rich. In its complete form, the problem entails both parametric (i.e., weights determination) and structural learning (i.e., structure selection). The majority of works deal with parametric uncertainty assuming knowledge of the appropriate neural structure. In this paper we present an algorithmic approach to determine the structure of High Order Neural Networks (HONNs), to solve function approximation problems. The method is based on a Genetic Algorithm (GA) and is equipped with a stable update law to guarantee parametric learning. Simulation results on an illustrative example highlight the performance and give some insight of the proposed approach.

A real-coded genetic algorithm for training recurrent neural networks

The use of Recurrent Neural Networks is not as extensive as Feedforward Neural Networks. Training algorithms for Recurrent Neural Networks, based on the error gradient, are very unstable in their search for a minimum and require much computational time when the number of neurons is high. The problems surrounding the application of these methods have driven us to develop new training tools. In this paper, we present a Real-Coded Genetic Algorithm that uses the appropriate operators for this encoding type to train Recurrent Neural Networks. We describe the algorithm and we also experimentally compare our Genetic Algorithm with the Real-Time Recurrent Learning algorithm to perform the fuzzy grammatical inference.