A L-BFGS Based Learning Algorithm for Complex-Valued Feedforward Neural Networks

Adaptive CL-BFGS Algorithms for Complex-Valued Neural Networks

Complex-valued limited-memory BFGS (CL-BFGS) algorithm is efficient for the training of complex-valued neural networks (CVNNs). As an important parameter, the memory size represents the number of saved vector pairs and would essentially affect the performance of the algorithm. However, the determination of a suitable memory size for the CL-BFGS algorithm remains challenging. To deal with this issue, an adaptive method is proposed in which the memory size is allowed to vary during the iteration process. Basically, at each iteration, with the help of multistep quasi-Newton method, an appropriate memory size is chosen from a variable set {1,2, ... , M} by approximating complex Hessian matrix as close as possible. To reduce the computational complexity and ensure desired performance, the upper bound M is adjustable according to the moving average of memory sizes found in previous iterations. The proposed adaptive CL-BFGS (ACL-BFGS) algorithm can be efficiently applied for the training of CVNNs. Moreover, it is suggested to take multiple memory sizes to construct the search direction, which further improves the performance of the ACL-BFGS algorithm. Experimental results on some benchmark problems including the pattern classification, complex function approximation, and nonlinear channel equalization problems are given to illustrate the advantages of the developed algorithms over some previous ones.

Utilization of multilayer perceptron for determining the inverse kinematics of an industrial robotic manipulator

Inverse kinematic equations allow the determination of the joint angles necessary for the robotic manipulator to place a tool into a predefined position. Determining this equation is vital but a complex work. In this article, an artificial neural network, more specifically, a feed-forward type, multilayer perceptron (MLP), is trained, so that it could be used to calculate the inverse kinematics for a robotic manipulator. First, direct kinematics of a robotic manipulator are determined using Denavit–Hartenberg method and a dataset of 15,000 points is generated using the calculated homogenous transformation matrices. Following that, multiple MLPs are trained with 10,240 different hyperparameter combinations to find the best. Each trained MLP is evaluated using the R 2 and mean absolute error metrics and the architectures of the MLPs that achieved the best results are presented. Results show a successful regression for the first five joints (percentage error being less than 0.1%) but a comparatively poor regression for the final joint due to the configuration of the robotic manipulator.

Circular Complex-Valued GMDH-Type Neural Network for Real-Valued Classification Problems

Recently, applications of complex-valued neural networks (CVNNs) to real-valued classification problems have attracted significant attention. However, most existing CVNNs are black-box models with poor explanation performance. This study extends the real-valued group method of data handling (RGMDH)-type neural network to the complex field and constructs a circular complex-valued group method of data handling (C-CGMDH)-type neural network, which is a white-box model. First, a complex least squares method is proposed for parameter estimation. Second, a new complex-valued symmetric regularity criterion is constructed with a logarithmic function to represent explicitly the magnitude and phase of the actual and predicted complex output to evaluate and select the middle candidate models. Furthermore, the property of this new complex-valued external criterion is proven to be similar to that of the real external criterion. Before training this model, a circular transformation is used to transform the real-valued input features to the complex field. Twenty-five real-valued classification data sets from the UCI Machine Learning Repository are used to conduct the experiments. The results show that both RGMDH and C-CGMDH models can select the most important features from the complete feature space through a self-organizing modeling process. Compared with RGMDH, the C-CGMDH model converges faster and selects fewer features. Furthermore, its classification performance is statistically significantly better than the benchmark complex-valued and real-valued models. Regarding time complexity, the C-CGMDH model is comparable with other models in dealing with the data sets that have few features. Finally, we demonstrate that the GMDH-type neural network can be interpretable.