Topology-based hierarchical clustering of self-organizing maps

Automatic Modulation Classification for MASK, MPSK, and MQAM Signals Based on Hierarchical Self-Organizing Map

Automatic modulation classification (AMC) plays a fundamental role in common communication systems. Existing clustering models typically handle fewer modulation types with lower classification accuracies and more computational resources. This paper proposes a hierarchical self-organizing map (SOM) based on a feature space composed of high-order cumulants (HOC) and amplitude moment features. This SOM with two stacked layers can identify intrinsic differences among samples in the feature space without the need to set thresholds. This model can roughly cluster the multiple amplitude-shift keying (MASK), multiple phase-shift keying (MPSK), and multiple quadrature amplitude keying (MQAM) samples in the root layer and then finely distinguish the samples with different orders in the leaf layers. We creatively implement a discrete transformation method based on modified activation functions. This method causes MQAM samples to cluster in the leaf layer with more distinct boundaries between clusters and higher classification accuracies. The simulation results demonstrate the superior performance of the proposed hierarchical SOM on AMC problems when compared with other clustering models. Our proposed method can manage more categories of modulation signals and obtain higher classification accuracies while using fewer computational resources.

Approximate spectral clustering using both reference vectors and topology of the network generated by growing neural gas

Spectral clustering (SC) is one of the most popular clustering methods and often outperforms traditional clustering methods. SC uses the eigenvectors of a Laplacian matrix calculated from a similarity matrix of a dataset. SC has serious drawbacks: the significant increases in the time complexity derived from the computation of eigenvectors and the memory space complexity to store the similarity matrix. To address the issues, I develop a new approximate spectral clustering using the network generated by growing neural gas (GNG), called ASC with GNG in this study. ASC with GNG uses not only reference vectors for vector quantization but also the topology of the network for extraction of the topological relationship between data points in a dataset. ASC with GNG calculates the similarity matrix from both the reference vectors and the topology of the network generated by GNG. Using the network generated from a dataset by GNG, ASC with GNG achieves to reduce the computational and space complexities and improve clustering quality. In this study, I demonstrate that ASC with GNG effectively reduces the computational time. Moreover, this study shows that ASC with GNG provides equal to or better clustering performance than SC.

Self-Supervised Learning of Satellite-Derived Vegetation Indices for Clustering and Visualization of Vegetation Types

Vegetation indices are commonly used techniques for the retrieval of biophysical and chemical attributes of vegetation. This paper presents the potential of an Autoencoders (AEs) and Convolutional Autoencoders (CAEs)-based self-supervised learning approach for the decorrelation and dimensionality reduction of high-dimensional vegetation indices derived from satellite observations. This research was implemented in Mt. Zao and its base in northeast Japan with a cool temperate climate by collecting the ground truth points belonging to 16 vegetation types (including some non-vegetation classes) in 2018. Monthly median composites of 16 vegetation indices were generated by processing all Sentinel-2 scenes available for the study area from 2017 to 2019. The performance of AEs and CAEs-based compressed images for the clustering and visualization of vegetation types was quantitatively assessed by computing the bootstrap resampling-based confidence interval. The AEs and CAEs-based compressed images with three features showed around 4% and 9% improvements in the confidence intervals respectively over the classical method. CAEs using convolutional neural networks showed better feature extraction and dimensionality reduction capacity than the AEs. The class-wise performance analysis also showed the superiority of the CAEs. This research highlights the potential of AEs and CAEs for attaining a fine clustering and visualization of vegetation types.

The Applications of Clustering Methods in Predicting Protein Functions

Background:

The understanding of protein function is essential to the study of biological processes. However, the prediction of protein function has been a difficult task for bioinformatics to overcome. This has resulted in many scholars focusing on the development of computational methods to address this problem.

Objective:

In this review, we introduce the recently developed computational methods of protein function prediction and assess the validity of these methods. We then introduce the applications of clustering methods in predicting protein functions.

Generalized Self-Organizing Maps for Automatic Determination of the Number of Clusters and Their Multiprototypes in Cluster Analysis

This paper presents a generalization of self-organizing maps with 1-D neighborhoods (neuron chains) that can be effectively applied to complex cluster analysis problems. The essence of the generalization consists in introducing mechanisms that allow the neuron chain-during learning-to disconnect into subchains, to reconnect some of the subchains again, and to dynamically regulate the overall number of neurons in the system. These features enable the network-working in a fully unsupervised way (i.e., using unlabeled data without a predefined number of clusters)-to automatically generate collections of multiprototypes that are able to represent a broad range of clusters in data sets. First, the operation of the proposed approach is illustrated on some synthetic data sets. Then, this technique is tested using several real-life, complex, and multidimensional benchmark data sets available from the University of California at Irvine (UCI) Machine Learning repository and the Knowledge Extraction based on Evolutionary Learning data set repository. A sensitivity analysis of our approach to changes in control parameters and a comparative analysis with an alternative approach are also performed.

An Information-Theoretic-Cluster Visualization for Self-Organizing Maps

Improved data visualization will be a significant tool to enhance cluster analysis. In this paper, an information-theoretic-based method for cluster visualization using self-organizing maps (SOMs) is presented. The information-theoretic visualization (IT-vis) has the same structure as the unified distance matrix, but instead of depicting Euclidean distances between adjacent neurons, it displays the similarity between the distributions associated with adjacent neurons. Each SOM neuron has an associated subset of the data set whose cardinality controls the granularity of the IT-vis and with which the first- and second-order statistics are computed and used to estimate their probability density functions. These are used to calculate the similarity measure, based on Renyi's quadratic cross entropy and cross information potential (CIP). The introduced visualizations combine the low computational cost and kernel estimation properties of the representative CIP and the data structure representation of a single-linkage-based grouping algorithm to generate an enhanced SOM-based visualization. The visual quality of the IT-vis is assessed by comparing it with other visualization methods for several real-world and synthetic benchmark data sets. Thus, this paper also contains a significant literature survey. The experiments demonstrate the IT-vis cluster revealing capabilities, in which cluster boundaries are sharply captured. Additionally, the information-theoretic visualizations are used to perform clustering of the SOM. Compared with other methods, IT-vis of large SOMs yielded the best results in this paper, for which the quality of the final partitions was evaluated using external validity indices.

K-MEAP: Multiple Exemplars Affinity Propagation With Specified $K$ Clusters

Recently, an attractive clustering approach named multiexemplar affinity propagation (MEAP) has been proposed as an extension to the single exemplar-based AP. MEAP is able to automatically identify multiple exemplars for each cluster associated with a superexemplar. However, if the cluster number is a prior knowledge and can be specified by the user, MEAP is unable to make use of such knowledge directly in its learning process. Instead, it has to rely on rerunning the process as many times as it takes by tuning parameters until it generates the desired number of clusters. The process of MEAP rerunning may be very time-consuming. In this paper, we propose a new clustering algorithm called Multiple Exemplars Affinity Propagation with Specified K Clusters which is able to generate specified K clusters directly while retaining the advantages of MEAP. Two kinds of new additional messages are introduced in K-MEAP in order to control the number of clusters in the process of message passing. Detailed problem formulation, derived messages, and in-depth analysis of the proposed K-MEAP are provided. Experimental studies on 11 real-world data sets with different kinds of applications demonstrate that K-MEAP not only generates K clusters directly and efficiently without tuning parameters but also outperforms related approaches in terms of clustering accuracy.

Topology-based clustering using polar self-organizing map

Cluster analysis of unlabeled data sets has been recognized as a key research topic in varieties of fields. In many practical cases, no a priori knowledge is specified, for example, the number of clusters is unknown. In this paper, grid clustering based on the polar self-organizing map (PolSOM) is developed to automatically identify the optimal number of partitions. The data topology consisting of both the distance and density is exploited in the grid clustering. The proposed clustering method also provides a visual representation as PolSOM allows the characteristics of clusters to be presented as a 2-D polar map in terms of the data feature and value. Experimental studies on synthetic and real data sets demonstrate that the proposed algorithm provides higher clustering accuracy and lower computational cost compared with six conventional methods.

Confabulation-inspired association rule mining for rare and frequent itemsets

A new confabulation-inspired association rule mining (CARM) algorithm is proposed using an interestingness measure inspired by cogency. Cogency is only computed based on pairwise item conditional probability, so the proposed algorithm mines association rules by only one pass through the file. The proposed algorithm is also more efficient for dealing with infrequent items due to its cogency-inspired approach. The problem of associative classification is used here for evaluating the proposed algorithm. We evaluate CARM over both synthetic and real benchmark data sets obtained from the UC Irvine machine learning repository. Experiments show that the proposed algorithm is consistently faster due to its one time file access and consumes less memory space than the Conditional Frequent Patterns growth algorithm. In addition, statistical analysis reveals the superiority of the approach for classifying minority classes in unbalanced data sets.

Segmentation of brain MRI using SOM-FCM-based method and 3D statistical descriptors

Current medical imaging systems provide excellent spatial resolution, high tissue contrast, and up to 65535 intensity levels. Thus, image processing techniques which aim to exploit the information contained in the images are necessary for using these images in computer-aided diagnosis (CAD) systems. Image segmentation may be defined as the process of parcelling the image to delimit different neuroanatomical tissues present on the brain. In this paper we propose a segmentation technique using 3D statistical features extracted from the volume image. In addition, the presented method is based on unsupervised vector quantization and fuzzy clustering techniques and does not use any a priori information. The resulting fuzzy segmentation method addresses the problem of partial volume effect (PVE) and has been assessed using real brain images from the Internet Brain Image Repository (IBSR).

Data-driven cluster reinforcement and visualization in sparsely-matched self-organizing maps

A self-organizing map (SOM) is a self-organized projection of high-dimensional data onto a typically 2-dimensional (2-D) feature map, wherein vector similarity is implicitly translated into topological closeness in the 2-D projection. However, when there are more neurons than input patterns, it can be challenging to interpret the results, due to diffuse cluster boundaries and limitations of current methods for displaying interneuron distances. In this brief, we introduce a new cluster reinforcement (CR) phase for sparsely-matched SOMs. The CR phase amplifies within-cluster similarity in an unsupervised, data-driven manner. Discontinuities in the resulting map correspond to between-cluster distances and are stored in a boundary (B) matrix. We describe a new hierarchical visualization of cluster boundaries displayed directly on feature maps, which requires no further clustering beyond what was implicitly accomplished during self-organization in SOM training. We use a synthetic benchmark problem and previously published microbial community profile data to demonstrate the benefits of the proposed methods.

Tackling learning intractability through topological organization and regulation of cortical networks

A key challenge in evolving control systems for robots using neural networks is training tractability. Evolving monolithic fixed topology neural networks is shown to be intractable with limited supervision in high dimensional search spaces. Common strategies to overcome this limitation are to provide more supervision by encouraging particular solution strategies, manually decomposing the task and segmenting the search space and network. These strategies require a supervisor with domain knowledge and may not be feasible for difficult tasks where novel concepts are required. The alternate strategy is to use self-organized task decomposition to solve difficult tasks with limited supervision. The artificial neural tissue (ANT) approach presented here uses self-organized task decomposition to solve tasks. ANT inspired by neurobiology combines standard neural networks with a novel wireless signaling scheme modeling chemical diffusion of neurotransmitters. These chemicals are used to dynamically activate and inhibit wired network of neurons using a coarse-coding framework. Using only a global fitness function that does not encourage a predefined solution, modular networks of neurons are shown to self-organize and perform task decomposition. This approach solves the sign-following task found to be intractable with conventional fixed and variable topology networks. In this paper, key attributes of the ANT architecture that perform self-organized task decomposition are shown. The architecture is robust and scalable to number of neurons, synaptic connections, and initialization parameters.

Visualized analysis of mixed numeric and categorical data via extended self-organizing map

Many real-world datasets are of mixed types, having numeric and categorical attributes. Even though difficult, analyzing mixed-type datasets is important. In this paper, we propose an extended self-organizing map (SOM), called MixSOM, which utilizes a data structure distance hierarchy to facilitate the handling of numeric and categorical values in a direct, unified manner. Moreover, the extended model regularizes the prototype distance between neighboring neurons in proportion to their map distance so that structures of the clusters can be portrayed better on the map. Extensive experiments on several synthetic and real-world datasets are conducted to demonstrate the capability of the model and to compare MixSOM with several existing models including Kohonen's SOM, the generalized SOM and visualization-induced SOM. The results show that MixSOM is superior to the other models in reflecting the structure of the mixed-type data and facilitates further analysis of the data such as exploration at various levels of granularity.