Cluster validity indices for automatic clustering: A comprehensive review

The Cluster Validity Index is an integral part of clustering algorithms. It evaluates inter-cluster separation and intra-cluster cohesion of candidate clusters to determine the quality of potential solutions. Several cluster validity indices have been suggested for both classical clustering algorithms and automatic metaheuristic-based clustering algorithms. Different cluster validity indices exhibit different characteristics based on the mathematical models they employ in determining the values for the various cluster attributes. Metaheuristic-based automatic clustering algorithms use cluster validity index as a fitness function in its optimization procedure to evaluate the candidate cluster solution's quality. A systematic review of the cluster validity indices used as fitness functions in metaheuristic-based automatic clustering algorithms is presented in this study. Identifying, reporting, and analysing various cluster validity indices is important in classifying the best CVIs for optimum performance of a metaheuristic-based automatic clustering algorithm. This review also includes an experimental study on the performance of some common cluster validity indices on some synthetic datasets with varied characteristics as well as real-life datasets using the SOSK-means automatic clustering algorithm. This review aims to assist researchers in identifying and selecting the most suitable cluster validity indices (CVIs) for their specific application areas.

iCVI-ARTMAP: Using Incremental Cluster Validity Indices and Adaptive Resonance Theory Reset Mechanism to Accelerate Validation and Achieve Multiprototype Unsupervised Representations

This article presents an adaptive resonance theory predictive mapping (ARTMAP) model, which uses incremental cluster validity indices (iCVIs) to perform unsupervised learning, namely, iCVI-ARTMAP. Incorporating iCVIs to the decision-making and many-to-one mapping capabilities of this adaptive resonance theory (ART)-based model can improve the choices of clusters to which samples are incrementally assigned. These improvements are accomplished by intelligently performing the operations of swapping sample assignments between clusters, splitting and merging clusters, and caching the values of variables when iCVI values need to be recomputed. Using recursive formulations enables iCVI-ARTMAP to considerably reduce the computational burden associated with cluster validity index (CVI)-based offline clustering. In this work, six iCVI-ARTMAP variants were realized via the integration of one information-theoretic and five sum-of-squares-based iCVIs into fuzzy ARTMAP. With proper choice of iCVI, iCVI-ARTMAP either outperformed or performed comparably to three ART-based and four non-ART-based clustering algorithms in experiments using benchmark datasets of different natures. Naturally, the performance of iCVI-ARTMAP is subject to the selected iCVI and its suitability to the data at hand; fortunately, it is a general model in which other iCVIs can be easily embedded.

Incremental Cluster Validity Index-Guided Online Learning for Performance and Robustness to Presentation Order

In streaming data applications, the incoming samples are processed and discarded, and therefore, intelligent decision-making is crucial for the performance of lifelong learning systems. In addition, the order in which the samples arrive may heavily affect the performance of incremental learners. The recently introduced incremental cluster validity indices (iCVIs) provide valuable aid in addressing such class of problems. Their primary use case has been cluster quality monitoring; nonetheless, they have been recently integrated in a streaming clustering method. In this context, the work presented, here, introduces the first adaptive resonance theory (ART)-based model that uses iCVIs for unsupervised and semi-supervised online learning. Moreover, it shows how to use iCVIs to regulate ART vigilance via an iCVI-based match tracking mechanism. The model achieves improved accuracy and robustness to ordering effects by integrating an online iCVI module as module B of a topological ART predictive mapping (TopoARTMAP)-thereby being named iCVI-TopoARTMAP-and using iCVI-driven postprocessing heuristics at the end of each learning step. The online iCVI module provides assignments of input samples to clusters at each iteration in accordance to any of the several iCVIs. The iCVI-TopoARTMAP maintains useful properties shared by the ART predictive mapping (ARTMAP) models, such as stability, immunity to catastrophic forgetting, and the many-to-one mapping capability via the map field module. The performance and robustness to the presentation order of iCVI-TopoARTMAP were evaluated via experiments with synthetic and real-world datasets.

Modeling Soil Temperature for Different Days Using Novel Quadruplet Loss-Guided LSTM

Soil temperature (Ts), a key variable in geosciences study, has generated growing interest among researchers. There are many factors affecting the spatiotemporal variation of Ts, which poses immense challenges for the Ts estimation. To enrich processing information on loss function and achieve better performance in estimation, the paper designed a new long short-term memory model using quadruplet loss function as an intelligence tool for data processing (QL-LSTM). The model in this paper combined the traditional squared-error loss function with distance metric learning between the sample features. It can zoom analyze the samples accurately to optimize the estimation accuracy. We applied the meteorological data from Laegern and Fluehli stations at 5, 10, and 15 cm depth on the 1st, 5th, and 15th day separately to verify the performance of the proposed soil temperature estimation model. Meanwhile, this paper inputs the variables into the proposed model including radiation, air temperature, vapor pressure deficit, wind speed, air pressure, and past Ts data. The performance of the model was tested by several error evaluation indices, including root mean square error (RMSE), mean absolute error (MAE), Nash-Sutcliffe model efficiency coefficient (NS), Willmott Index of Agreement (WI), and Legates and McCabe index (LMI). As the test results at different soil depths show, our model generally outperformed the four existing advanced estimation models, namely, backpropagation neural networks, extreme learning machines, support vector regression, and LSTM. Furthermore, as experiments show, the proposed model achieved the best performance at the 15 cm depth of soil on the 1st day at Laegern station, which achieved higher WI (0.998), NS (0.995), and LMI (0.938) values, and got lower RMSE (0.312) and MAE (0.239) values. Consequently, the QL-LSTM model is recommended to estimate daily Ts profiles estimation on the 1st, 5th, and 15th days.

Elastic Differential Evolution for Automatic Data Clustering

In many practical applications, it is crucial to perform automatic data clustering without knowing the number of clusters in advance. The evolutionary computation paradigm is good at dealing with this task, but the existing algorithms encounter several deficiencies, such as the encoding redundancy and the cross-dimension learning error. In this article, we propose a novel elastic differential evolution algorithm to solve automatic data clustering. Unlike traditional methods, the proposed algorithm considers each clustering layout as a whole and adapts the cluster number and cluster centroids inherently through the variable-length encoding and the evolution operators. The encoding scheme contains no redundancy. To enable the individuals of different lengths to exchange information properly, we develop a subspace crossover and a two-phase mutation operator. The operators employ the basic method of differential evolution and, in addition, they consider the spatial information of cluster layouts to generate offspring solutions. Particularly, each dimension of the parameter vector interacts with its correlated dimensions, which not only adapts the cluster number but also avoids the cross-dimension learning error. The experimental results show that our algorithm outperforms the state-of-the-art algorithms that it is able to identify the correct number of clusters and obtain a good cluster validation value.

An Algorithm for the Evolutionary-Fuzzy Generation of on-Line Signature Hybrid Descriptors

In biometrics, methods which are able to precisely adapt to the biometric features of users are much sought after. They use various methods of artificial intelligence, in particular methods from the group of soft computing. In this paper, we focus on on-line signature verification. Such signatures are complex objects described not only by the shape but also by the dynamics of the signing process. In standard devices used for signature acquisition (with an LCD touch screen) this dynamics may include pen velocity, but sometimes other types of signals are also available, e.g. pen pressure on the screen surface (e.g. in graphic tablets), the angle between the pen and the screen surface, etc. The precision of the on-line signature dynamics processing has been a motivational springboard for developing methods that use signature partitioning. Partitioning uses a well-known principle of decomposing the problem into smaller ones. In this paper, we propose a new partitioning algorithm that uses capabilities of the algorithms based on populations and fuzzy systems. Evolutionary-fuzzy partitioning eliminates the need to average dynamic waveforms in created partitions because it replaces them. Evolutionary separation of partitions results in a better matching of partitions with reference signatures, eliminates dispro-portions between the number of points describing dynamics in partitions, eliminates the impact of random values, separates partitions related to the signing stage and its dynamics (e.g. high and low velocity of signing, where high and low are imprecise-fuzzy concepts). The operation of the presented algorithm has been tested using the well-known BioSecure DS2 database of real dynamic signatures.

Long-term activity recognition from wristwatch accelerometer data

With the development of wearable devices that have several embedded sensors, it is possible to collect data that can be analyzed in order to understand the user's needs and provide personalized services. Examples of these types of devices are smartphones, fitness-bracelets, smartwatches, just to mention a few. In the last years, several works have used these devices to recognize simple activities like running, walking, sleeping, and other physical activities. There has also been research on recognizing complex activities like cooking, sporting, and taking medication, but these generally require the installation of external sensors that may become obtrusive to the user. In this work we used acceleration data from a wristwatch in order to identify long-term activities. We compare the use of Hidden Markov Models and Conditional Random Fields for the segmentation task. We also added prior knowledge into the models regarding the duration of the activities by coding them as constraints and sequence patterns were added in the form of feature functions. We also performed subclassing in order to deal with the problem of intra-class fragmentation, which arises when the same label is applied to activities that are conceptually the same but very different from the acceleration point of view.

Foraging swarms as Nash equilibria of dynamic games

The question of whether foraging swarms can form as a result of a noncooperative game played by individuals is shown here to have an affirmative answer. A dynamic game played by N agents in 1-D motion is introduced and models, for instance, a foraging ant colony. Each agent controls its velocity to minimize its total work done in a finite time interval. The game is shown to have a unique Nash equilibrium under two different foraging location specifications, and both equilibria display many features of a foraging swarm behavior observed in biological swarms. Explicit expressions are derived for pairwise distances between individuals of the swarm, swarm size, and swarm center location during foraging.

Grid topologies for the self-organizing map

The original Self-Organizing Feature Map (SOFM) has been extended in many ways to suit different goals and application domains. However, the topologies of the map lattice that we can found in literature are nearly always square or, more rarely, hexagonal. In this paper we study alternative grid topologies, which are derived from the geometrical theory of tessellations. Experimental results are presented for unsupervised clustering, color image segmentation and classification tasks, which show that the differences among the topologies are statistically significant in most cases, and that the optimal topology depends on the problem at hand. A theoretical interpretation of these results is also developed.

PSO-MISMO modeling strategy for multistep-ahead time series prediction

Multistep-ahead time series prediction is one of the most challenging research topics in the field of time series modeling and prediction, and is continually under research. Recently, the multiple-input several multiple-outputs (MISMO) modeling strategy has been proposed as a promising alternative for multistep-ahead time series prediction, exhibiting advantages compared with the two currently dominating strategies, the iterated and the direct strategies. Built on the established MISMO strategy, this paper proposes a particle swarm optimization (PSO)-based MISMO modeling strategy, which is capable of determining the number of sub-models in a self-adaptive mode, with varying prediction horizons. Rather than deriving crisp divides with equal-size s prediction horizons from the established MISMO, the proposed PSO-MISMO strategy, implemented with neural networks, employs a heuristic to create flexible divides with varying sizes of prediction horizons and to generate corresponding sub-models, providing considerable flexibility in model construction, which has been validated with simulated and real datasets.

Improving the quality of self-organizing maps by self-intersection avoidance

The quality of self-organizing maps is always a key issue to practitioners. Smooth maps convey information about input data sets in a clear manner. Here a method is presented to modify the learning algorithm of self-organizing maps to reduce the number of topology errors, hence the obtained map has better quality at the expense of increased quantization error. It is based on avoiding maps that self-intersect or nearly so, as these states are related to low quality. Our approach is tested with synthetic data and real data from visualization, pattern recognition and computer vision applications, with satisfactory results.