Extreme Learning Machine for Joint Embedding and Clustering

A review on extreme learning machine

Extreme learning machine (ELM) is a training algorithm for single hidden layer feedforward neural network (SLFN), which converges much faster than traditional methods and yields promising performance. In this paper, we hope to present a comprehensive review on ELM. Firstly, we will focus on the theoretical analysis including universal approximation theory and generalization. Then, the various improvements are listed, which help ELM works better in terms of stability, efficiency, and accuracy. Because of its outstanding performance, ELM has been successfully applied in many real-time learning tasks for classification, clustering, and regression. Besides, we report the applications of ELM in medical imaging: MRI, CT, and mammogram. The controversies of ELM were also discussed in this paper. We aim to report these advances and find some future perspectives.

Cerebral Microbleed Detection via Convolutional Neural Network and Extreme Learning Machine

Aim: Cerebral microbleeds (CMBs) are small round dots distributed over the brain which contribute to stroke, dementia, and death. The early diagnosis is significant for the treatment. Method: In this paper, a new CMB detection approach was put forward for brain magnetic resonance images. We leveraged a sliding window to obtain training and testing samples from input brain images. Then, a 13-layer convolutional neural network (CNN) was designed and trained. Finally, we proposed to utilize an extreme learning machine (ELM) to substitute the last several layers in the CNN for detection. We carried out an experiment to decide the optimal number of layers to be substituted. The parameters in ELM were optimized by a heuristic algorithm named bat algorithm. The evaluation of our approach was based on hold-out validation, and the final predictions were generated by averaging the performance of five runs. Results: Through the experiments, we found replacing the last five layers with ELM can get the optimal results. Conclusion: We offered a comparison with state-of-the-art algorithms, and it can be revealed that our method was accurate in CMB detection.

Correntropy induced loss based sparse robust graph regularized extreme learning machine for cancer classification

Background

As a machine learning method with high performance and excellent generalization ability, extreme learning machine (ELM) is gaining popularity in various studies. Various ELM-based methods for different fields have been proposed. However, the robustness to noise and outliers is always the main problem affecting the performance of ELM.

Results

In this paper, an integrated method named correntropy induced loss based sparse robust graph regularized extreme learning machine (CSRGELM) is proposed. The introduction of correntropy induced loss improves the robustness of ELM and weakens the negative effects of noise and outliers. By using the L_2,1-norm to constrain the output weight matrix, we tend to obtain a sparse output weight matrix to construct a simpler single hidden layer feedforward neural network model. By introducing the graph regularization to preserve the local structural information of the data, the classification performance of the new method is further improved. Besides, we design an iterative optimization method based on the idea of half quadratic optimization to solve the non-convex problem of CSRGELM.

Conclusions

The classification results on the benchmark dataset show that CSRGELM can obtain better classification results compared with other methods. More importantly, we also apply the new method to the classification problems of cancer samples and get a good classification effect.

ELM embedded discriminative dictionary learning for image classification

Dictionary learning is a widely adopted approach for image classification. Existing methods focus either on finding a dictionary that produces discriminative sparse representation, or on enforcing priors that best describe the dataset distribution. In many cases, the dataset size is often small with large intra-class variability and nondiscriminative feature space. In this work we propose a simple and effective framework called ELM-DDL to address these issues. Specifically, we represent input features with Extreme Learning Machine (ELM) with orthogonal output projection, which enables diverse representation on nonlinear hidden space and task specific feature learning on output space. The embeddings are further regularized via a maximum margin criterion (MMC) to maximize the inter-class variance and minimize intra-class variance. For dictionary learning, we design a novel weighted class specific ℓ_1,2 norm to regularize the sparse coding vectors, which promotes uniformity of the sparse patterns of samples belonging to the same class and suppresses support overlaps of different classes. We show that such regularization is robust, discriminative and easy to optimize. The proposed method is combined with a sparse representation classifier (SRC) to evaluate on benchmark datasets. Results show that our approach achieves state-of-the-art performance compared to other dictionary learning methods.

A distributed semi-supervised learning algorithm based on manifold regularization using wavelet neural network

This paper aims to propose a distributed semi-supervised learning (D-SSL) algorithm to solve D-SSL problems, where training samples are often extremely large-scale and located on distributed nodes over communication networks. Training data of each node consists of labeled and unlabeled samples whose output values or labels are unknown. These nodes communicate in a distributed way, where each node has only access to its own data and can only exchange local information with its neighboring nodes. In some scenarios, these distributed data cannot be processed centrally. As a result, D-SSL problems cannot be centrally solved by using traditional semi-supervised learning (SSL) algorithms. The state-of-the-art D-SSL algorithm, denoted as Distributed Laplacian Regularization Least Square (D-LapRLS), is a kernel based algorithm. It is essential for the D-LapRLS algorithm to estimate the global Euclidian Distance Matrix (EDM) with respect to total samples, which is time-consuming especially when the scale of training data is large. In order to solve D-SSL problems and overcome the common drawback of kernel based D-SSL algorithms, we propose a novel Manifold Regularization (MR) based D-SSL algorithm using Wavelet Neural Network (WNN) and Zero-Gradient-Sum (ZGS) distributed optimization strategy. Accordingly, each node is assigned an individual WNN with the same basis functions. In order to initialize the proposed D-SSL algorithm, we propose a centralized MR based SSL algorithm using WNN. We denote the proposed SSL and D-SSL algorithms as Laplacian WNN (LapWNN) and distributed LapWNN (D-LapWNN), respectively. The D-LapWNN algorithm works in a fully distributed fashion by using ZGS strategy, whose convergence is guaranteed by the Lyapunov method. During the learning process, each node only exchanges local coefficients with its neighbors rather than raw data. It means that the D-LapWNN algorithm is a privacy preserving method. At last, several illustrative simulations are presented to show the efficiency and advantage of the proposed algorithm.

Multi-View Fusion with Extreme Learning Machine for Clustering

Unlabeled, multi-view data presents a considerable challenge in many real-world data analysis tasks. These data are worth exploring because they often contain complementary information that improves the quality of the analysis results. Clustering with multi-view data is a particularly challenging problem as revealing the complex data structures between many feature spaces demands discriminative features that are specific to the task and, when too few of these features are present, performance suffers. Extreme learning machines (ELMs) are an emerging form of learning model that have shown an outstanding representation ability and superior performance in a range of different learning tasks. Motivated by the promise of this advancement, we have developed a novel multi-view fusion clustering framework based on an ELM, called MVEC. MVEC learns the embeddings from each view of the data via the ELM network, then constructs a single unified embedding according to the correlations and dependencies between each embedding and automatically weighting the contribution of each. This process exposes the underlying clustering structures embedded within multi-view data with a high degree of accuracy. A simple yet efficient solution is also provided to solve the optimization problem within MVEC. Experiments and comparisons on eight different benchmarks from different domains confirm MVEC’s clustering accuracy.