Multilayer Batch Learning Growing Neural Gas for Learning Multiscale Topologies

Hierarchical topological structure learning methods are expected to be developed in the field of data mining for extracting multiscale topological structures from an unknown dataset. However, most methods require user-defined parameters, and it is difficult for users to determine these parameters and effectively utilize the method. In this paper, we propose a new parameter-less hierarchical topological structure learning method based on growing neural gas (GNG). First, we propose batch learning GNG (BL-GNG) to improve the learning convergence and reduce the user-designed parameters in GNG. BL-GNG uses an objective function based on fuzzy C-means to improve the learning convergence. Next, we propose multilayer BL-GNG (MBL-GNG), which is a parameter-less unsupervised learning algorithm based on hierarchical topological structure learning. In MBL-GNG, the input data of each layer uses parent nodes to learn more abstract topological structures from the dataset. Furthermore, MBL-GNG can automatically determine the number of nodes and layers according to the data distribution. Finally, we conducted several experiments to evaluate our proposed method by comparing it with other hierarchical approaches and discuss the effectiveness of our proposed method.

The Growing Hierarchical Neural Gas Self-Organizing Neural Network

The growing neural gas (GNG) self-organizing neural network stands as one of the most successful examples of unsupervised learning of a graph of processing units. Despite its success, little attention has been devoted to its extension to a hierarchical model, unlike other models such as the self-organizing map, which has many hierarchical versions. Here, a hierarchical GNG is presented, which is designed to learn a tree of graphs. Moreover, the original GNG algorithm is improved by a distinction between a growth phase where more units are added until no significant improvement in the quantization error is obtained, and a convergence phase where no unit creation is allowed. This means that a principled mechanism is established to control the growth of the structure. Experiments are reported, which demonstrate the self-organization and hierarchy learning abilities of our approach and its performance for vector quantization applications.

Learning Topologies with the Growing Neural Forest

In this work, a novel self-organizing model called growing neural forest (GNF) is presented. It is based on the growing neural gas (GNG), which learns a general graph with no special provisions for datasets with separated clusters. On the contrary, the proposed GNF learns a set of trees so that each tree represents a connected cluster of data. High dimensional datasets often contain large empty regions among clusters, so this proposal is better suited to them than other self-organizing models because it represents these separated clusters as connected components made of neurons. Experimental results are reported which show the self-organization capabilities of the model. Moreover, its suitability for unsupervised clustering and foreground detection applications is demonstrated. In particular, the GNF is shown to correctly discover the connected component structure of some datasets. Moreover, it outperforms some well-known foreground detectors both in quantitative and qualitative terms.

Grid topologies for the self-organizing map

The original Self-Organizing Feature Map (SOFM) has been extended in many ways to suit different goals and application domains. However, the topologies of the map lattice that we can found in literature are nearly always square or, more rarely, hexagonal. In this paper we study alternative grid topologies, which are derived from the geometrical theory of tessellations. Experimental results are presented for unsupervised clustering, color image segmentation and classification tasks, which show that the differences among the topologies are statistically significant in most cases, and that the optimal topology depends on the problem at hand. A theoretical interpretation of these results is also developed.

Bregman divergences for growing hierarchical self-organizing networks

Growing hierarchical self-organizing models are characterized by the flexibility of their structure, which can easily accommodate for complex input datasets. However, most proposals use the Euclidean distance as the only error measure. Here we propose a way to introduce Bregman divergences in these models, which is based on stochastic approximation principles, so that more general distortion measures can be employed. A procedure is derived to compare the performance of networks using different divergences. Moreover, a probabilistic interpretation of the model is provided, which enables its use as a Bayesian classifier. Experimental results are presented for classification and data visualization applications, which show the advantages of these divergences with respect to the classical Euclidean distance.

Variational learning for finite Dirichlet mixture models and applications

In this paper, we focus on the variational learning of finite Dirichlet mixture models. Compared to other algorithms that are commonly used for mixture models (such as expectation-maximization), our approach has several advantages: first, the problem of over-fitting is prevented; furthermore, the complexity of the mixture model (i.e., the number of components) can be determined automatically and simultaneously with the parameters estimation as part of the Bayesian inference procedure; finally, since the whole inference process is analytically tractable with closed-form solutions, it may scale well to large applications. Both synthetic and real data, generated from real-life challenging applications namely image databases categorization and anomaly intrusion detection, are experimented to verify the effectiveness of the proposed approach.

Tackling learning intractability through topological organization and regulation of cortical networks

A key challenge in evolving control systems for robots using neural networks is training tractability. Evolving monolithic fixed topology neural networks is shown to be intractable with limited supervision in high dimensional search spaces. Common strategies to overcome this limitation are to provide more supervision by encouraging particular solution strategies, manually decomposing the task and segmenting the search space and network. These strategies require a supervisor with domain knowledge and may not be feasible for difficult tasks where novel concepts are required. The alternate strategy is to use self-organized task decomposition to solve difficult tasks with limited supervision. The artificial neural tissue (ANT) approach presented here uses self-organized task decomposition to solve tasks. ANT inspired by neurobiology combines standard neural networks with a novel wireless signaling scheme modeling chemical diffusion of neurotransmitters. These chemicals are used to dynamically activate and inhibit wired network of neurons using a coarse-coding framework. Using only a global fitness function that does not encourage a predefined solution, modular networks of neurons are shown to self-organize and perform task decomposition. This approach solves the sign-following task found to be intractable with conventional fixed and variable topology networks. In this paper, key attributes of the ANT architecture that perform self-organized task decomposition are shown. The architecture is robust and scalable to number of neurons, synaptic connections, and initialization parameters.