A Novel Fast Feedforward Neural Networks Training Algorithm

Fast Computational Approach to the Levenberg-Marquardt Algorithm for Training Feedforward Neural Networks

This paper presents a parallel approach to the Levenberg-Marquardt algorithm (LM). The use of the Levenberg-Marquardt algorithm to train neural networks is associated with significant computational complexity, and thus computation time. As a result, when the neural network has a big number of weights, the algorithm becomes practically ineffective. This article presents a new parallel approach to the computations in Levenberg-Marquardt neural network learning algorithm. The proposed solution is based on vector instructions to effectively reduce the high computational time of this algorithm. The new approach was tested on several examples involving the problems of classification and function approximation, and next it was compared with a classical computational method. The article presents in detail the idea of parallel neural network computations and shows the obtained acceleration for different problems.

Detecting Anomalies in Advertising Web Traffic with the Use of the Variational Autoencoder

This paper presents a neural network model for identifying non-human traffic to a web-site, which is significantly different from visits made by regular users. Such visits are undesirable from the point of view of the website owner as they are not human activity, and therefore do not bring any value, and, what is more, most often involve costs incurred in connection with the handling of advertising. They are made most often by dishonest publishers using special software (bots) to generate profits. Bots are also used in scraping, which is automatic scanning and downloading of website content, which actually is not in the interest of website authors. The model proposed in this work is learnt by data extracted directly from the web browser during website visits. This data is acquired by using a specially prepared JavaScript that monitors the behavior of the user or bot. The appearance of a bot on a website generates parameter values that are significantly different from those collected during typical visits made by human website users. It is not possible to learn more about the software controlling the bots and to know all the data generated by them. Therefore, this paper proposes a variational autoencoder (VAE) neural network model with modifications to detect the occurrence of abnormal parameter values that deviate from data obtained from human users’ Internet traffic. The algorithm works on the basis of a popular autoencoder method for detecting anomalies, however, a number of original improvements have been implemented. In the study we used authentic data extracted from several large online stores.

A Novel Approach to Type-Reduction and Design of Interval Type-2 Fuzzy Logic Systems

Fuzzy logic systems, unlike black-box models, are known as transparent artificial intelligence systems that have explainable rules of reasoning. Type 2 fuzzy systems extend the field of application to tasks that require the introduction of uncertainty in the rules, e.g. for handling corrupted data. Most practical implementations use interval type-2 sets and process interval membership grades. The key role in the design of type-2 interval fuzzy logic systems is played by the type-2 inference defuzzification method. In type-2 systems this generally takes place in two steps: type-reduction first, then standard defuzzification. The only precise type-reduction method is the iterative method known as Karnik-Mendel (KM) algorithm with its enhancement modifications. The known non-iterative methods deliver only an approximation of the boundaries of a type-reduced set and, in special cases, they diminish the profits that result from the use of type-2 fuzzy logic systems. In this paper, we propose a novel type-reduction method based on a smooth approximation of maximum/minimum, and we call this method a smooth type-reduction. Replacing the iterative KM algorithm by the smooth type-reduction, we obtain a structure of an adaptive interval type-2 fuzzy logic which is non-iterative and as close to an approximation of the KM algorithm as we like.

Towards a Very Fast Feedforward Multilayer Neural Networks Training Algorithm

**This paper presents a novel fast algorithm for feedforward neural networks training. It is based on the Recursive Least Squares (RLS) method commonly used for designing adaptive filters. Besides, it utilizes two techniques of linear algebra, namely the orthogonal transformation method, called the Givens Rotations (GR), and the QR decomposition, creating the GQR (symbolically we write GR + QR = GQR) procedure for solving the normal equations in the weight update process. In this paper, a novel approach to the GQR algorithm is presented. The main idea revolves around reducing the computational cost of a single rotation by eliminating the square root calculation and reducing the number of multiplications. The proposed modification is based on the scaled version of the Givens rotations, denoted as SGQR. This modification is expected to bring a significant training time reduction comparing to the classic GQR algorithm. The paper begins with the introduction and the classic Givens rotation description. Then, the scaled rotation and its usage in the QR decomposition is discussed. The main section of the article presents the neural network training algorithm which utilizes scaled Givens rotations and QR decomposition in the weight update process. Next, the experiment results of the proposed algorithm are presented and discussed. The experiment utilizes several benchmarks combined with neural networks of various topologies. It is shown that the proposed algorithm outperforms several other commonly used methods, including well known Adam optimizer.