The Growing Hierarchical Neural Gas Self-Organizing Neural Network

An Improved Deep Belief Network Prediction Model Based on Knowledge Transfer

A deep belief network (DBN) is a powerful generative model based on unlabeled data. However, it is difficult to quickly determine the best network structure and gradient dispersion in traditional DBN. This paper proposes an improved deep belief network (IDBN): first, the basic DBN structure is pre-trained and the learned weight parameters are fixed; secondly, the learned weight parameters are transferred to the new neuron and hidden layer through the method of knowledge transfer, thereby constructing the optimal network width and depth of DBN; finally, the top-down layer-by-layer partial least squares regression method is used to fine-tune the weight parameters obtained by the pre-training, which avoids the traditional fine-tuning problem based on the back-propagation algorithm. In order to verify the prediction performance of the model, this paper conducts benchmark experiments on the Movielens-20M (ML-20M) and Last.fm-1k (LFM-1k) public data sets. Compared with other traditional algorithms, IDBN is better than other fixed models in terms of prediction performance and training time. The proposed IDBN model has higher prediction accuracy and convergence speed.

The GH-EXIN neural network for hierarchical clustering

Hierarchical clustering is an important tool for extracting information from data in a multi-resolution way. It is more meaningful if driven by data, as in the case of divisive algorithms, which split data until no more division is allowed. However, they have the drawback of the splitting threshold setting. The neural networks can address this problem, because they basically depend on data. The growing hierarchical GH-EXIN neural network builds a hierarchical tree in an incremental (data-driven architecture) and self-organized way. It is a top-down technique which defines the horizontal growth by means of an anisotropic region of influence, based on the novel idea of neighborhood convex hull. It also reallocates data and detects outliers by using a novel approach on all the leaves, simultaneously. Its complexity is estimated and an analysis of its user-dependent parameters is given. The advantages of the proposed approach, with regard to the best existing networks, are shown and analyzed, qualitatively and quantitatively, both in benchmark synthetic problems and in a real application (image recognition from video), in order to test the performance in building hierarchical trees. Furthermore, an important and very promising application of GH-EXIN in two-way hierarchical clustering, for the analysis of gene expression data in the study of the colorectal cancer is described.