Conventional displays of structures in data compared with interactive projection-based clustering (IPBC)

Clustering is an important task in knowledge discovery with the goal to identify structures of similar data points in a dataset. Here, the focus lies on methods that use a human-in-the-loop, i.e., incorporate user decisions into the clustering process through 2D and 3D displays of the structures in the data. Some of these interactive approaches fall into the category of visual analytics and emphasize the power of such displays to identify the structures interactively in various types of datasets or to verify the results of clustering algorithms. This work presents a new method called interactive projection-based clustering (IPBC). IPBC is an open-source and parameter-free method using a human-in-the-loop for an interactive 2.5D display and identification of structures in data based on the user’s choice of a dimensionality reduction method. The IPBC approach is systematically compared with accessible visual analytics methods for the display and identification of cluster structures using twelve clustering benchmark datasets and one additional natural dataset. Qualitative comparison of 2D, 2.5D and 3D displays of structures and empirical evaluation of the identified cluster structures show that IPBC outperforms comparable methods. Additionally, IPBC assists in identifying structures previously unknown to domain experts in an application.

The experimental study of the effectiveness of Kohonen maps and autoassociative neural networks in the qualitative analysis of multidimensional data by the example of real data describing coal susceptibility to fluidal gasification

The qualitative analysis of multidimensional data using their visualization allows to observe some characteristics of data in a way which is the most natural for a human, through the sense of sight. Thanks to such an approach, some characteristics of the analyzed data are simply visible. This allows to avoid using often complex algorithms allowing to examine specific data properties. Visualization of multidimensional data consists in using the representation transforming a multidimensional space into a two-dimensional space representing a computer screen. The important information which can be obtained in this way is the possibility to separate points belonging to different classes in the multidimensional space. Such information can be directly obtained if images of points belonging to different classes occupy other areas of the picture presenting these data. The paper presents the effectiveness of the qualitative analysis of multidimensional data conducted in this way through their visualization with the application of Kohonen maps and autoassociative neural networks. The obtained results were compared with results obtained using the perspective-based observational tunnels method, PCA, multidimensional scaling and relevance maps. Effectiveness tests of the above methods were performed using real seven-dimensional data describing coal samples in terms of their susceptibility to fluidal gasification. The methods’ effectiveness was compared using the criterion for the readability of the multidimensional visualization results, introduced in earlier papers.

A multivariate-based variable selection framework for clustering traffic conflicts in a brazilian freeway

More than one million people die or suffer non-fatal injuries annually due to road accidents around the world. Understanding the causes that give rise to different types of conflict events, as well as their characteristics, can help researchers and traffic authorities to draw up strategies aimed at mitigating collision risks. This paper proposes a framework for grouping traffic conflicts relying on similar profiles and factors that contribute to conflict occurrence using self-organizing maps (SOM). In order to improve the quality of the formed groups, we developed a novel variable importance index relying on the outputs of the nonlinear principal component analysis (NLPCA) that intends to identify the most informative variables for grouping collision events. Such index guides a backward variable selection procedure in which less relevant variables are removed one-by-one; after each removal, the clustering quality is assessed via the Davies-Bouldin (DB) index. The proposed framework was applied to a real-time dataset collected from a Brazilian highway aimed at allocating traffic conflicts into groups presenting similar profiles. The selected variables suggest that lower average speeds, which are typically verified during congestion events, contribute to conflict occurrence. Higher variability on speed (denoted by high standard deviation, and speed's coefficient of variation levels on that variable), which are also perceived in the assessed freeway near to congestion periods, also contribute to conflicts.

Self-Organizing Hidden Markov Model Map (SOHMMM): Biological Sequence Clustering and Cluster Visualization

The present study devises mapping methodologies and projection techniques that visualize and demonstrate biological sequence data clustering results. The Sequence Data Density Display (SDDD) and Sequence Likelihood Projection (SLP) visualizations represent the input symbolical sequences in a lower-dimensional space in such a way that the clusters and relations of data elements are depicted graphically. Both operate in combination/synergy with the Self-Organizing Hidden Markov Model Map (SOHMMM). The resulting unified framework is in position to analyze automatically and directly raw sequence data. This analysis is carried out with little, or even complete absence of, prior information/domain knowledge.

Kohonen's Map Approach for the Belief Mass Modeling

In the framework of the evidence theory, several approaches for estimating belief functions are proposed. However, they generally suffer from the problem of masses attribution in the case of compound hypotheses that lose much conceptual contribution of the theory. In this paper, an original method for estimating mass functions using Kohonen's map derived from the initial feature space and an initial classifier is proposed. Our approach allows a smart mass belief assignment, not only for simple hypotheses but also for disjunctions and conjunctions of hypotheses. Thus, it can model at the same time ignorance, imprecision, and paradox. The proposed method for a basic belief assignment (BBA) is of interest for solving estimation mass functions problems where a large quantity of multivariate data is available. Indeed, the use of Kohonen's map simplifies the process of assigning mass functions. The proposed method has been compared with the state-of-the-art BBA technique on benchmark database and applied on remote sensing data for image classification purpose. Experimentation shows that our approach gives similar or better results than other methods presented in the literature so far, with an ability to handle a large amount of data.

Depth-Sensor-Based Monitoring of Therapeutic Exercises

In this paper, we propose a self-organizing feature map-based (SOM) monitoring system which is able to evaluate whether the physiotherapeutic exercise performed by a patient matches the corresponding assigned exercise. It allows patients to be able to perform their physiotherapeutic exercises on their own, but their progress during exercises can be monitored. The performance of the proposed the SOM-based monitoring system is tested on a database consisting of 12 different types of physiotherapeutic exercises. An average 98.8% correct rate was achieved.

Unsupervised spatiotemporal analysis of fMRI data using graph-based visualizations of self-organizing maps

We present novel graph-based visualizations of self-organizing maps for unsupervised functional magnetic resonance imaging (fMRI) analysis. A self-organizing map is an artificial neural network model that transforms high-dimensional data into a low-dimensional (often a 2-D) map using unsupervised learning. However, a postprocessing scheme is necessary to correctly interpret similarity between neighboring node prototypes (feature vectors) on the output map and delineate clusters and features of interest in the data. In this paper, we used graph-based visualizations to capture fMRI data features based upon 1) the distribution of data across the receptive fields of the prototypes (density-based connectivity); and 2) temporal similarities (correlations) between the prototypes (correlation-based connectivity). We applied this approach to identify task-related brain areas in an fMRI reaction time experiment involving a visuo-manual response task, and we correlated the time-to-peak of the fMRI responses in these areas with reaction time. Visualization of self-organizing maps outperformed independent component analysis and voxelwise univariate linear regression analysis in identifying and classifying relevant brain regions. We conclude that the graph-based visualizations of self-organizing maps help in advanced visualization of cluster boundaries in fMRI data enabling the separation of regions with small differences in the timings of their brain responses.

Clustering of the self-organizing map

The self-organizing map (SOM) is an excellent tool in exploratory phase of data mining. It projects input space on prototypes of a low-dimensional regular grid that can be effectively utilized to visualize and explore properties of the data. When the number of SOM units is large, to facilitate quantitative analysis of the map and the data, similar units need to be grouped, i.e., clustered. In this paper, different approaches to clustering of the SOM are considered. In particular, the use of hierarchical agglomerative clustering and partitive clustering using k-means are investigated. The two-stage procedure--first using SOM to produce the prototypes that are then clustered in the second stage--is found to perform well when compared with direct clustering of the data and to reduce the computation time.

Data-driven cluster reinforcement and visualization in sparsely-matched self-organizing maps

A self-organizing map (SOM) is a self-organized projection of high-dimensional data onto a typically 2-dimensional (2-D) feature map, wherein vector similarity is implicitly translated into topological closeness in the 2-D projection. However, when there are more neurons than input patterns, it can be challenging to interpret the results, due to diffuse cluster boundaries and limitations of current methods for displaying interneuron distances. In this brief, we introduce a new cluster reinforcement (CR) phase for sparsely-matched SOMs. The CR phase amplifies within-cluster similarity in an unsupervised, data-driven manner. Discontinuities in the resulting map correspond to between-cluster distances and are stored in a boundary (B) matrix. We describe a new hierarchical visualization of cluster boundaries displayed directly on feature maps, which requires no further clustering beyond what was implicitly accomplished during self-organization in SOM training. We use a synthetic benchmark problem and previously published microbial community profile data to demonstrate the benefits of the proposed methods.

A SOM PROJECTION TECHNIQUE WITH THE GROWING STRUCTURE FOR VISUALIZING HIGH-DIMENSIONAL DATA

The Self-Organizing Map (SOM) is an efficient tool for visualizing high-dimensional data. In this paper, an intuitive and effective SOM projection method is proposed for mapping high-dimensional data onto the two-dimensional grid structure with a growing self-organizing mechanism. In the learning phase, a growing SOM is trained and the growing cell structure is used as the baseline framework. In the ordination phase, the new projection method is used to map the input vector so that the input data is mapped to the structure of the SOM without having to plot the weight values, resulting in easy visualization of the data. The projection method is demonstrated on four different data sets, including a 118 patent data set and a 399 checical abstract data set related to polymer cements, with promising results and a significantly reduced network size.

Graph based representations of density distribution and distances for self-organizing maps

The self-organizing map (SOM) is a powerful method for manifold learning because of producing a 2-D spatially ordered quantization of a higher dimensional data space on a rigid lattice and adaptively determining optimal approximation of the (unknown) density distribution of the data. However, a postprocessing visualization scheme is often required to capture the data manifold. A recent visualization scheme CONNvis, which is shown effective for clustering, uses a topology representing graph that shows detailed local data distribution within receptive fields. This brief proposes that this graph representation can be adapted to show local distances. The proposed graphs of local density and local distances provide tools to analyze the correlation between these two information and to merge them in various ways to achieve an advanced visualization. The brief also gives comparisons for several synthetic data sets.

Exploiting data topology in visualization and clustering of self-organizing maps

The self-organizing map (SOM) is a powerful method for visualization, cluster extraction, and data mining. It has been used successfully for data of high dimensionality and complexity where traditional methods may often be insufficient. In order to analyze data structure and capture cluster boundaries from the SOM, one common approach is to represent the SOM's knowledge by visualization methods. Different aspects of the information learned by the SOM are presented by existing methods, but data topology, which is present in the SOM's knowledge, is greatly underutilized. We show in this paper that data topology can be integrated into the visualization of the SOM and thereby provide a more elaborate view of the cluster structure than existing schemes. We achieve this by introducing a weighted Delaunay triangulation (a connectivity matrix) and draping it over the SOM. This new visualization, CONNvis, also shows both forward and backward topology violations along with the severity of forward ones, which indicate the quality of the SOM learning and the data complexity. CONNvis greatly assists in detailed identification of cluster boundaries. We demonstrate the capabilities on synthetic data sets and on a real 8-D remote sensing spectral image.

Automatic cluster detection in Kohonen's SOM

Kohonen's self-organizing map (SOM) is a popular neural network architecture for solving problems in the field of explorative data analysis, clustering, and data visualization. One of the major drawbacks of the SOM algorithm is the difficulty for nonexpert users to interpret the information contained in a trained SOM. In this paper, this problem is addressed by introducing an enhanced version of the Clusot algorithm. This algorithm consists of two main steps: 1) the computation of the Clusot surface utilizing the information contained in a trained SOM and 2) the automatic detection of clusters in this surface. In the Clusot surface, clusters present in the underlying SOM are indicated by the local maxima of the surface. For SOMs with 2-D topology, the Clusot surface can, therefore, be considered as a convenient visualization technique. Yet, the presented approach is not restricted to a certain type of 2-D SOM topology and it is also applicable for SOMs having an n-dimensional grid topology.

Asynchronous self-organizing maps

A recently defined energy function which leads to a self-organizing map is used as a foundation for an asynchronous neural-network algorithm. We generalize the existing stochastic gradient approach to an asynchronous parallel stochastic gradient method for generating a topological map on a distributed computer system (MIMD). A convergence proof is presented and simulation results on a set of problems are included. A practical problem using the energy function approach is that a summation over the entire network is required during the computation of updates. Using simulations we demonstrate effective algorithms that use efficient sampling for the approximation of these sums.

Visualizing and clustering high throughput sub-cellular localization imaging

Background

The expansion of automatic imaging technologies has created a need to be able to efficiently compare and review large sets of image data. To enable comparisons of image data between samples we need to define the normal variation within distinct images of the same sample. Even with tightly controlled experimental conditions, protein expression can vary widely between cells, and because of the difficulty in viewing and comparing large image sets this might not be observed. Here we introduce a novel methodology, iCluster, for visualizing, clustering and comparing large sub-cellular localization image sets. For each member of an image set, iCluster generates statistics that have been found to be useful in distinguishing sub-cellular localization. The statistics are mapped into two or three dimensions such as to preserve distances between the statistics vectors. The complete image set is then visualized in two or three dimensions using the coordinates so determined. The result is images that are statistically similar are spatially close in the visualization allowing for easy comparison of images that are similar and distinguishment of dissimilar images into distinct clusters.

Results

The methodology was tested on a set of 502 previously published images containing 10 known sub-cellular localizations. The clustering of images of like type was evaluated both by examining the classes of nearest neighbors to each image and by visual inspection. In three dimensions, 3-neighbor classification accuracy was 83.2%. Visually, each class clustered well with the majority of classes localizing to distinct regions of the space. In two dimensions, 3-neighbor classification accuracy was 68.9%, though visually clustering into classes could be readily discerned. Computational expense was found to be relatively low, and sets of up to 1400 images visualized and interacted with in real time.

Conclusion

The feasibility of automated spatial layout to allow comparison and discrimination of high throughput sub-cellular imaging has been demonstrated. There are many potential applications such as image database curation, semi-automated interactive classification, outlier detection and reference image comparison. By allowing the observation of the full range of imaging data available using modern microscopes these methods will provide an invaluable tool for cell biologists.

PRSOM: A New Visualization Method by Hybridizing Multidimensional Scaling and Self-Organizing Map

Self-organizing map (SOM) is an approach of nonlinear dimension reduction and can be used for visualization. It only preserves topological structures of input data on the projected output space. The interneuron distances of SOM are not preserved from input space into output space such that the visualization of SOM can be degraded. Visualization-induced SOM (ViSOM) has been proposed to overcome this problem. However, ViSOM is derived from heuristic and no cost function is assigned to it. In this paper, a probabilistic regularized SOM (PRSOM) is proposed to give a better visualization effect. It is associated with a cost function and gives a principled rule for weight-updating. The advantages of both multidimensional scaling (MDS) and SOM are incorporated in PRSOM. Like MDS, The interneuron distances of PRSOM in input space resemble those in output space, which are predefined before training. Instead of the hard assignment by ViSOM, the soft assignment by PRSOM can be further utilized to enhance the visualization effect. Experimental results demonstrate the effectiveness of the proposed PRSOM method compared with other dimension reduction methods.

Class distributions on SOM surfaces for feature extraction and object retrieval

A Self-Organizing Map (SOM) is typically trained in unsupervised mode, using a large batch of training data. If the data contain semantically related object groupings or classes, subsets of vectors belonging to such user-defined classes can be mapped on the SOM by finding the best matching unit for each vector in the set. The distribution of the data vectors over the map forms a two-dimensional discrete probability density. Even from the same data, qualitatively different distributions can be obtained by using different feature extraction techniques. We used such feature distributions for comparing different classes and different feature representations of the data in the context of our content-based image retrieval system PicSOM. The information-theoretic measures of entropy and mutual information are suggested to evaluate the compactness of a distribution and the independence of two distributions. Also, the effect of low-pass filtering the SOM surfaces prior to the calculation of the entropy is studied.

Impact of descriptor vector scaling on the classification of drugs and nondrugs with artificial neural networks

The influence of preprocessing of molecular descriptor vectors for solving classification tasks was analyzed for drug/nondrug classification by artificial neural networks. Molecular properties were used to form descriptor vectors. Two types of neural networks were used, supervised multilayer neural nets trained with the back-propagation algorithm, and unsupervised self-organizing maps (Kohonen maps). Data were preprocessed by logistic scaling and histogram equalization. For both types of neural networks, the preprocessing step significantly improved classification compared to nonstandardized data. Classification accuracy was measured as prediction mean square error and Matthews correlation coefficient in the case of supervised learning, and quantization error in the case of unsupervised learning. The results demonstrate that appropriate data preprocessing is an essential step in solving classification tasks.

Self-organizing maps with recursive neighborhood adaptation

Self-organizing maps (SOMs) are widely used in several fields of application, from neurobiology to multivariate data analysis. In that context, this paper presents variants of the classic SOM algorithm. With respect to the traditional SOM, the modifications regard the core of the algorithm, (the learning rule), but do not alter the two main tasks it performs, i.e. vector quantization combined with topology preservation. After an intuitive justification based on geometrical considerations, three new rules are defined in addition to the original one. They develop interesting properties such as recursive neighborhood adaptation and non-radial neighborhood adaptation. In order to assess the relative performances and speeds of convergence, the four rules are used to train several maps and the results are compared according to several error measures (quantization error and topology preservation criterions).

How to make large self-organizing maps for nonvectorial data

The self-organizing map (SOM) represents an open set of input samples by a topologically organized, finite set of models. In this paper, a new version of the SOM is used for the clustering, organization, and visualization of a large database of symbol sequences (viz. protein sequences). This method combines two principles: the batch computing version of the SOM, and computation of the generalized median of symbol strings.

Data visualisation and manifold mapping using the ViSOM

The self-organising map (SOM) has been successfully employed as a nonparametric method for dimensionality reduction and data visualisation. However, for visualisation the SOM requires a colouring scheme to imprint the distances between neurons so that the clustering and boundaries can be seen. Even though the distributions of the data and structures of the clusters are not faithfully portrayed on the map. Recently an extended SOM, called the visualisation-induced SOM (ViSOM) has been proposed to directly preserve the distance information on the map, along with the topology. The ViSOM constrains the lateral contraction forces between neurons and hence regularises the interneuron distances so that distances between neurons in the data space are in proportion to those in the map space. This paper shows that it produces a smooth and graded mesh in the data space and captures the nonlinear manifold of the data. The relationships between the ViSOM and the principal curve/surface are analysed. The ViSOM represents a discrete principal curve or surface and is a natural algorithm for obtaining principal curves/surfaces. Guidelines for applying the ViSOM constraint and setting the resolution parameter are also provided, together with experimental results and comparisons with the SOM, Sammon mapping and principal curve methods.

Mapping and fuzzy classification of macromolecular images using self-organizing neural networks

In this work the effectiveness of the fuzzy kohonen clustering network (FKCN) in the unsupervised classification of electron microscopic images of biological macromolecules is studied. The algorithm combines Kohonen's self-organizing feature maps (SOFM) and Fuzzy c-means (FCM) in order to obtain a powerful clustering technique with the best properties inherited from both. Exploratory data analysis using SOFM is also presented as a step previous to final clustering. Two different data sets obtained from the G40P helicase from B. Subtilis bacteriophage SPP1 have been used for testing the proposed method, one composed of 2458 rotational power spectra of individual images and the other composed by 338 images from the same macromolecule. Results of FKCN are compared with self-organizing feature maps (SOFM) and manual classification. Experimental results prove that this new technique is suitable for working with large, high-dimensional and noisy data sets and, thus, it is proposed to be used as a classification tool in electron microscopy.

Neuro-fuzzy feature evaluation with theoretical analysis

The article provides a fuzzy set theoretic feature evaluation index and a connectionist model for its evaluation along with their theoretical analysis. A concept of weighted membership function is introduced which makes the modeling of the class structures more appropriate. A neuro-fuzzy algorithm is developed for determining the optimum weighting coefficients representing the feature importance. It is shown theoretically that the evaluation index has a fixed upper bound and a varying lower bound, and it monotonically increases with the lower bound. A relation between the evaluation index, interclass distance and weighting coefficients is established. Effectiveness of the algorithms for evaluating features both individually and in a group (considering their independence and dependency) is demonstrated along with comparisons on speech, Iris, medical and mango-leaf data. The results are also validated using scatter diagram and k-NN classifier.