Sparse Bayesian extreme learning machine for multi-classification

Robustly federated learning model for identifying high-risk patients with postoperative gastric cancer recurrence

The prediction of patient disease risk via computed tomography (CT) images and artificial intelligence techniques shows great potential. However, training a robust artificial intelligence model typically requires large-scale data support. In practice, the collection of medical data faces obstacles related to privacy protection. Therefore, the present study aims to establish a robust federated learning model to overcome the data island problem and identify high-risk patients with postoperative gastric cancer recurrence in a multicentre, cross-institution setting, thereby enabling robust treatment with significant value. In the present study, we collect data from four independent medical institutions for experimentation. The robust federated learning model algorithm yields area under the receiver operating characteristic curve (AUC) values of 0.710, 0.798, 0.809, and 0.869 across four data centres. Additionally, the effectiveness of the algorithm is evaluated, and both adaptive and common features are identified through analysis.

A novel correlation Gaussian process regression-based extreme learning machine

An obvious defect of extreme learning machine (ELM) is that its prediction performance is sensitive to the random initialization of input-layer weights and hidden-layer biases. To make ELM insensitive to random initialization, GPRELM adopts the simple an effective strategy of integrating Gaussian process regression into ELM. However, there is a serious overfitting problem in kernel-based GPRELM (kGPRELM). In this paper, we investigate the theoretical reasons for the overfitting of kGPRELM and further propose a correlation-based GPRELM (cGPRELM), which uses a correlation coefficient to measure the similarity between two different hidden-layer output vectors. cGPRELM reduces the likelihood that the covariance matrix becomes an identity matrix when the number of hidden-layer nodes is increased, effectively controlling overfitting. Furthermore, cGPRELM works well for improper initialization intervals where ELM and kGPRELM fail to provide good predictions. The experimental results on real classification and regression data sets demonstrate the feasibility and superiority of cGPRELM, as it not only achieves better generalization performance but also has a lower computational complexity.

Extreme Learning Machine Design for Dealing with Unrepresentative Features

Extreme Learning Machines (ELMs) have become a popular tool for the classification of electroencephalography (EEG) signals for Brain Computer Interfaces. This is so mainly due to their very high training speed and generalization capabilities. Another important advantage is that they have only one hyperparameter that must be calibrated: the number of hidden nodes. While most traditional approaches dictate that this parameter should be chosen smaller than the number of available training examples, in this article we argue that, in the case of problems in which the data contain unrepresentative features, such as in EEG classification problems, it is beneficial to choose a much larger number of hidden nodes. We characterize this phenomenon, explain why this happens and exhibit several concrete examples to illustrate how ELMs behave. Furthermore, as searching for the optimal number of hidden nodes could be time consuming in enlarged ELMs, we propose a new training scheme, including a novel pruning method. This scheme provides an efficient way of finding the optimal number of nodes, making ELMs more suitable for dealing with real time EEG classification problems. Experimental results using synthetic data and real EEG data show a major improvement in the training time with respect to most traditional and state of the art ELM approaches, without jeopardising classification performance and resulting in more compact networks.

Multiclass Probabilistic Classification Vector Machine

The probabilistic classification vector machine (PCVM) synthesizes the advantages of both the support vector machine and the relevant vector machine, delivering a sparse Bayesian solution to classification problems. However, the PCVM is currently only applicable to binary cases. Extending the PCVM to multiclass cases via heuristic voting strategies such as one-vs-rest or one-vs-one often results in a dilemma where classifiers make contradictory predictions, and those strategies might lose the benefits of probabilistic outputs. To overcome this problem, we extend the PCVM and propose a multiclass PCVM (mPCVM). Two learning algorithms, i.e., one top-down algorithm and one bottom-up algorithm, have been implemented in the mPCVM. The top-down algorithm obtains the maximum a posteriori (MAP) point estimates of the parameters based on an expectation-maximization algorithm, and the bottom-up algorithm is an incremental paradigm by maximizing the marginal likelihood. The superior performance of the mPCVMs, especially when the investigated problem has a large number of classes, is extensively evaluated on the synthetic and benchmark data sets.

Relaxation of the Radio-Frequency Linewidth for Coherent-Optical Orthogonal Frequency-Division Multiplexing Schemes by Employing the Improved Extreme Learning Machine

A coherent optical (CO) orthogonal frequency division multiplexing (OFDM) scheme gives a scalable and flexible solution for increasing the transmission rate, being extremely robust to chromatic dispersion as well as polarization mode dispersion. Nevertheless, as any coherent-detection OFDM system, the overall system performance is limited by laser phase noises. On the other hand, extreme learning machines (ELMs) have gained a lot of attention from the machine learning community owing to good generalization performance, negligible learning speed, and minimum human intervention. In this manuscript, a phase-error mitigation method based on the single-hidden layer feedforward network prone to the improved ELM algorithm for CO-OFDM systems is introduced for the first time. In the training step, two steps are distinguished. Firstly, pilots are used, which is very common in OFDM-based systems, to diminish laser phase noises as well as to correct frequency-selective impairments and, therefore, the bandwidth efficiency can be maximized. Secondly, the regularization parameter is included in the ELM to balance the empirical and structural risks, namely to minimize the root mean square error in the test stage and, consequently, the bit error rate (BER) metric. The operational principle of the real-complex (RC) ELM is analytically explained, and then, its sub-parameters (number of hidden neurons, regularization parameter, and activation function) are numerically found in order to enhance the system performance. For binary and quadrature phase-shift keying modulations, the RC-ELM outperforms the benchmark pilot-assisted equalizer as well as the fully-real ELM, and almost matches the common phase error (CPE) compensation and the ELM defined in the complex domain (C-ELM) in terms of the BER over an additive white Gaussian noise channel and different laser oscillators. However, both techniques are characterized by the following disadvantages: the CPE compensator reduces the transmission rate since an additional preamble is mandatory for channel estimation purposes, while the C-ELM requires a bounded and differentiable activation function in the complex domain and can not follow semi-supervised training. In the same context, the novel ELM algorithm can not compete with the CPE compensator and C-ELM for the 16-ary quadrature amplitude modulation. On the other hand, the novel ELM exposes a negligible computational cost with respect to the C-ELM and PAE methods.

Multimodal Sparse Classifier for Adolescent Brain Age Prediction

The study of healthy brain development helps to better understand both brain transformation and connectivity patterns, which happen during childhood to adulthood. This study presents a sparse machine learning solution across whole-brain functional connectivity measures of three datasets, derived from resting state functional magnetic resonance imaging (rs-fMRI) and two task fMRI data including a working memory n-back task (nb-fMRI) and an emotion identification task (em-fMRI). The fMRI data are collected from the Philadelphia Neurodevelopmental Cohort (PNC) for the prediction of brain age in adolescents. Due to extremely large variable-to-instance ratio of PNC data, a high-dimensional matrix with several irrelevant and highly correlated features is generated, and hence a sparse learning approach is necessary to extract effective features from fMRI data. We propose a sparse learner based on the residual errors along the estimation of an inverse problem for extreme learning machine (ELM). Our proposed method is able to overcome the overlearning problem by pruning several redundant features and their corresponding output weights. The proposed multimodal sparse ELM classifier based on residual errors is highly competitive in terms of classification accuracy compared to its counterparts such as conventional ELM, and sparse Bayesian learning ELM.

Adaptive sparse learning using multi-template for neurodegenerative disease diagnosis

Neurodegenerative diseases are excessively affecting millions of patients, especially elderly people. Early detection and management of these diseases are crucial as the clinical symptoms take years to appear after the onset of neuro-degeneration. This paper proposes an adaptive feature learning framework using multiple templates for early diagnosis. A multi-classification scheme is developed based on multiple brain parcellation atlases with various regions of interest. Different sets of features are extracted and then fused, and a feature selection is applied with an adaptively chosen sparse degree. In addition, both linear discriminative analysis and locally preserving projections are integrated to construct a least square regression model. Finally, we propose a feature space to predict the severity of the disease by the guidance of clinical scores. Our proposed method is validated on both Alzheimer's disease neuroimaging initiative and Parkinson's progression markers initiative databases. Extensive experimental results suggest that the proposed method outperforms the state-of-the-art methods, such as the multi-modal multi-task learning or joint sparse learning. Our method demonstrates that accurate feature learning facilitates the identification of the highly relevant brain regions with significant contribution in the prediction of disease progression. This may pave the way for further medical analysis and diagnosis in practical applications.

Extreme Learning Machine With Affine Transformation Inputs in an Activation Function

The extreme learning machine (ELM) has attracted much attention over the past decade due to its fast learning speed and convincing generalization performance. However, there still remains a practical issue to be approached when applying the ELM: the randomly generated hidden node parameters without tuning can lead to the hidden node outputs being nonuniformly distributed, thus giving rise to poor generalization performance. To address this deficiency, a novel activation function with an affine transformation (AT) on its input is introduced into the ELM, which leads to an improved ELM algorithm that is referred to as an AT-ELM in this paper. The scaling and translation parameters of the AT activation function are computed based on the maximum entropy principle in such a way that the hidden layer outputs approximately obey a uniform distribution. Application of the AT-ELM algorithm in nonlinear function regression shows its robustness to the range scaling of the network inputs. Experiments on nonlinear function regression, real-world data set classification, and benchmark image recognition demonstrate better performance for the AT-ELM compared with the original ELM, the regularized ELM, and the kernel ELM. Recognition results on benchmark image data sets also reveal that the AT-ELM outperforms several other state-of-the-art algorithms in general.

Hierarchical Feature Selection for Random Projection

Random projection is a popular machine learning algorithm, which can be implemented by neural networks and trained in a very efficient manner. However, the number of features should be large enough when applied to a rather large-scale data set, which results in slow speed in testing procedure and more storage space under some circumstances. Furthermore, some of the features are redundant and even noisy since they are randomly generated, so the performance may be affected by these features. To remedy these problems, an effective feature selection method is introduced to select useful features hierarchically. Specifically, a novel criterion is proposed to select useful neurons for neural networks, which establishes a new way for network architecture design. The testing time and accuracy of the proposed method are improved compared with traditional methods and some variations on both classification and regression tasks. Extensive experiments confirm the effectiveness of the proposed method.

Bundle Extreme Learning Machine for Power Quality Analysis in Transmission Networks

This paper presents a novel method for online power quality data analysis in transmission networks using a machine learning-based classifier. The proposed classifier has a bundle structure based on the enhanced version of the Extreme Learning Machine (ELM). Due to its fast response and easy-to-build architecture, the ELM is an appropriate machine learning model for power quality analysis. The sparse Bayesian ELM and weighted ELM have been embedded into the proposed bundle learning machine. The case study includes real field signals obtained from the Turkish electricity transmission system. Most actual events like voltage sag, voltage swell, interruption, and harmonics have been detected using the proposed algorithm. For validation purposes, the ELM algorithm is compared with state-of-the-art methods such as artificial neural network and least squares support vector machine.

Short-Term Electric Load and Price Forecasting Using Enhanced Extreme Learning Machine Optimization in Smart Grids

A Smart Grid (SG) is a modernized grid to provide efficient, reliable and economic energy to the consumers. Energy is the most important resource in the world. An efficient energy distribution is required as smart devices are increasing dramatically. The forecasting of electricity consumption is supposed to be a major constituent to enhance the performance of SG. Various learning algorithms have been proposed to solve the forecasting problem. The sole purpose of this work is to predict the price and load efficiently. The first technique is Enhanced Logistic Regression (ELR) and the second technique is Enhanced Recurrent Extreme Learning Machine (ERELM). ELR is an enhanced form of Logistic Regression (LR), whereas, ERELM optimizes weights and biases using a Grey Wolf Optimizer (GWO). Classification and Regression Tree (CART), Relief-F and Recursive Feature Elimination (RFE) are used for feature selection and extraction. On the basis of selected features, classification is performed using ELR. Cross validation is done for ERELM using Monte Carlo and K-Fold methods. The simulations are performed on two different datasets. The first dataset, i.e., UMass Electric Dataset is multi-variate while the second dataset, i.e., UCI Dataset is uni-variate. The first proposed model performed better with UMass Electric Dataset than UCI Dataset and the accuracy of second model is better with UCI than UMass. The prediction accuracy is analyzed on the basis of four different performance metrics: Mean Absolute Percentage Error (MAPE), Mean Absolute Error (MAE), Mean Square Error (MSE) and Root Mean Square Error (RMSE). The proposed techniques are then compared with four benchmark schemes. The comparison is done to verify the adaptivity of the proposed techniques. The simulation results show that the proposed techniques outperformed benchmark schemes. The proposed techniques efficiently increased the prediction accuracy of load and price. However, the computational time is increased in both scenarios. ELR achieved almost 5% better results than Convolutional Neural Network (CNN) and almost 3% than LR. While, ERELM achieved almost 6% better results than ELM and almost 5% than RELM. However, the computational time is almost 20% increased with ELR and 50% with ERELM. Scalability is also addressed for the proposed techniques using half-yearly and yearly datasets. Simulation results show that ELR gives 5% better results while, ERELM gives 6% better results when used for yearly dataset.

Nonparametric Bayesian Correlated Group Regression With Applications to Image Classification

Sparse Bayesian learning has emerged as a powerful tool to tackle various image classification tasks. The existing sparse Bayesian models usually use independent Gaussian distribution as the prior knowledge for the noise. However, this assumption often contradicts to the practical observations in which the noise is long tail and pixels containing noise are spatially correlated. To handle the practical noise, this paper proposes to partition the noise image into several 2-D groups and adopt the long-tail distribution, i.e., the scale mixture of the matrix Gaussian distribution, to model each group to capture the intragroup correlation of the noise. Under the nonparametric Bayesian estimation, the low-rank-induced prior and the matrix Gamma distribution prior are imposed on the covariance matrix of each group, respectively, to induce two Bayesian correlated group regression (BCGR) methods. Moreover, the proposed methods are extended to the case with unknown group structure. Our BCGR method provides an effective way to automatically fit the noise distribution and integrates the long-tail attribute and structure information of the practical noise into model. Therefore, the estimated coefficients are better for reconstructing the desired data. We apply BCGR to address image classification task and utilize the learned covariance matrices to construct a grouped Mahalanobis distance to measure the reconstruction residual of each class in the design of a classifier. Experimental results demonstrate the effectiveness of our new BCGR model.

Spatial extreme learning machines: An application on prediction of disease counts

Extreme learning machines have gained a lot of attention by the machine learning community because of its interesting properties and computational advantages. With the increase in collection of information nowadays, many sources of data have missing information making statistical analysis harder or unfeasible. In this paper, we present a new model, coined spatial extreme learning machine, that combine spatial modeling with extreme learning machines keeping the nice properties of both methodologies and making it very flexible and robust. As explained throughout the text, the spatial extreme learning machines have many advantages in comparison with the traditional extreme learning machines. By a simulation study and a real data analysis we present how the spatial extreme learning machine can be used to improve imputation of missing data and uncertainty prediction estimation.

Evolutionary Cost-Sensitive Extreme Learning Machine

Conventional extreme learning machines (ELMs) solve a Moore-Penrose generalized inverse of hidden layer activated matrix and analytically determine the output weights to achieve generalized performance, by assuming the same loss from different types of misclassification. The assumption may not hold in cost-sensitive recognition tasks, such as face recognition-based access control system, where misclassifying a stranger as a family member may result in more serious disaster than misclassifying a family member as a stranger. Though recent cost-sensitive learning can reduce the total loss with a given cost matrix that quantifies how severe one type of mistake against another, in many realistic cases, the cost matrix is unknown to users. Motivated by these concerns, this paper proposes an evolutionary cost-sensitive ELM, with the following merits: 1) to the best of our knowledge, it is the first proposal of ELM in evolutionary cost-sensitive classification scenario; 2) it well addresses the open issue of how to define the cost matrix in cost-sensitive learning tasks; and 3) an evolutionary backtracking search algorithm is induced for adaptive cost matrix optimization. Experiments in a variety of cost-sensitive tasks well demonstrate the effectiveness of the proposed approaches, with about 5%-10% improvements.

Urban Traffic Congestion Evaluation Based on Kernel the Semi-Supervised Extreme Learning Machine

There is always an asymmetric phenomenon between traffic data quantity and unit information content. Labeled data is more effective but scarce, while unlabeled data is large but weaker in sample information. In an urban transportation assessment system, semi-supervised extreme learning machine (SSELM) can unite manual observed data and extensively collected data cooperatively to build connection between congestion condition and road information. In our method, semi-supervised learning can integrate both small-scale labeled data and large-scale unlabeled data, so that they can play their respective advantages, while the ELM can process large scale data at high speed. Optimized by kernel function, Kernel-SSELM can achieve higher classification accuracy and robustness than original SSELM. Both the experiment and the real-time application show that the evaluation system can precisely reflect the traffic condition.

Highly Efficient Framework for Predicting Interactions Between Proteins

Protein-protein interactions (PPIs) play a central role in many biological processes. Although a large amount of human PPI data has been generated by high-throughput experimental techniques, they are very limited compared to the estimated 130 000 protein interactions in humans. Hence, automatic methods for human PPI-detection are highly desired. This work proposes a novel framework, i.e., Low-rank approximation-kernel Extreme Learning Machine (LELM), for detecting human PPI from a protein's primary sequences automatically. It has three main steps: 1) mapping each protein sequence into a matrix built on all kinds of adjacent amino acids; 2) applying the low-rank approximation model to the obtained matrix to solve its lowest rank representation, which reflects its true subspace structures; and 3) utilizing a powerful kernel extreme learning machine to predict the probability for PPI based on this lowest rank representation. Experimental results on a large-scale human PPI dataset demonstrate that the proposed LELM has significant advantages in accuracy and efficiency over the state-of-art approaches. Hence, this work establishes a new and effective way for the automatic detection of PPI.

A new framework for intelligent simultaneous-fault diagnosis of rotating machinery using pairwise-coupled sparse Bayesian extreme learning committee machine

This paper proposes a new diagnostic framework, namely, probabilistic committee machine, to diagnose simultaneous-fault in the rotating machinery. The new framework combines a feature extraction method with ensemble empirical mode decomposition and singular value decomposition, multiple pairwise-coupled sparse Bayesian extreme learning machines (PCSBELM), and a parameter optimization algorithm to create an intelligent diagnostic framework. The feature extraction method is employed to find the features of single faults in a simultaneous-fault pattern. Multiple PCSBELM networks are built as different signal committee members, and each member is trained using vibration or sound signals respectively. The individual diagnostic result from each fault detection member is then combined by a new probabilistic ensemble method, which can improve the overall diagnostic accuracy and increase the number of detectable fault as compared to individual classifier acting alone. The effectiveness of the proposed framework is verified by a case study on a gearbox fault detection. Experimental results show the proposed framework is superior to the existing single probabilistic classifier. Moreover, the proposed system can diagnose both single- and simultaneous-faults for the rotating machinery while the framework is trained by single-fault patterns only.

Extreme learning machine based optimal embedding location finder for image steganography

In image steganography, determining the optimum location for embedding the secret message precisely with minimum distortion of the host medium remains a challenging issue. Yet, an effective approach for the selection of the best embedding location with least deformation is far from being achieved. To attain this goal, we propose a novel approach for image steganography with high-performance, where extreme learning machine (ELM) algorithm is modified to create a supervised mathematical model. This ELM is first trained on a part of an image or any host medium before being tested in the regression mode. This allowed us to choose the optimal location for embedding the message with best values of the predicted evaluation metrics. Contrast, homogeneity, and other texture features are used for training on a new metric. Furthermore, the developed ELM is exploited for counter over-fitting while training. The performance of the proposed steganography approach is evaluated by computing the correlation, structural similarity (SSIM) index, fusion matrices, and mean square error (MSE). The modified ELM is found to outperform the existing approaches in terms of imperceptibility. Excellent features of the experimental results demonstrate that the proposed steganographic approach is greatly proficient for preserving the visual information of an image. An improvement in the imperceptibility as much as 28% is achieved compared to the existing state of the art methods.

Extreme Learning Machine With Subnetwork Hidden Nodes for Regression and Classification

As demonstrated earlier, the learning effectiveness and learning speed of single-hidden-layer feedforward neural networks are in general far slower than required, which has been a major bottleneck for many applications. Huang et al. proposed extreme learning machine (ELM) which improves the training speed by hundreds of times as compared to its predecessor learning techniques. This paper offers an ELM-based learning method that can grow subnetwork hidden nodes by pulling back residual network error to the hidden layer. Furthermore, the proposed method provides a similar or better generalization performance with remarkably fewer hidden nodes as compared to other ELM methods employing huge number of hidden nodes. Thus, the learning speed of the proposed technique is hundred times faster compared to other ELMs as well as to back propagation and support vector machines. The experimental validations for all methods are carried out on 32 data sets.

Multilayer Extreme Learning Machine With Subnetwork Nodes for Representation Learning

The extreme learning machine (ELM), which was originally proposed for "generalized" single-hidden layer feedforward neural networks, provides efficient unified learning solutions for the applications of clustering, regression, and classification. It presents competitive accuracy with superb efficiency in many applications. However, ELM with subnetwork nodes architecture has not attracted much research attentions. Recently, many methods have been proposed for supervised/unsupervised dimension reduction or representation learning, but these methods normally only work for one type of problem. This paper studies the general architecture of multilayer ELM (ML-ELM) with subnetwork nodes, showing that: 1) the proposed method provides a representation learning platform with unsupervised/supervised and compressed/sparse representation learning and 2) experimental results on ten image datasets and 16 classification datasets show that, compared to other conventional feature learning methods, the proposed ML-ELM with subnetwork nodes performs competitively or much better than other feature learning methods.