Bidirectional extreme learning machine for regression problem and its learning effectiveness

Sequential Order-Aware Coding-Based Robust Subspace Clustering for Human Action Recognition in Untrimmed Videos

Human action recognition (HAR) is one of most important tasks in video analysis. Since video clips distributed on networks are usually untrimmed, it is required to accurately segment a given untrimmed video into a set of action segments for HAR. As an unsupervised temporal segmentation technology, subspace clustering learns the codes from each video to construct an affinity graph, and then cuts the affinity graph to cluster the video into a set of action segments. However, most of the existing subspace clustering schemes not only ignore the sequential information of frames in code learning, but also the negative effects of noises when cutting the affinity graph, which lead to inferior performance. To address these issues, we propose a sequential order-aware coding-based robust subspace clustering (SOAC-RSC) scheme for HAR. By feeding the motion features of video frames into multi-layer neural networks, two expressive code matrices are learned in a sequential order-aware manner from unconstrained and constrained videos, respectively, to construct the corresponding affinity graphs. Then, with the consideration of the existence of noise effects, a simple yet robust cutting algorithm is proposed to cut the constructed affinity graphs to accurately obtain the action segments for HAR. The extensive experiments demonstrate the proposed SOAC-RSC scheme achieves the state-of-the-art performance on the datasets of Keck Gesture and Weizmann, and provides competitive performance on the other 6 public datasets such as UCF101 and URADL for HAR task, compared to the recent related approaches.

Multimodal Vigilance Estimation Using Deep Learning

The phenomenon of increasing accidents caused by reduced vigilance does exist. In the future, the high accuracy of vigilance estimation will play a significant role in public transportation safety. We propose a multimodal regression network that consists of multichannel deep autoencoders with subnetwork neurons (MCDAE _sn ). After we define two thresholds of "0.35" and "0.70" from the percentage of eye closure, the output values are in the continuous range of 0-0.35, 0.36-0.70, and 0.71-1 representing the awake state, the tired state, and the drowsy state, respectively. To verify the efficiency of our strategy, we first applied the proposed approach to a single modality. Then, for the multimodality, since the complementary information between forehead electrooculography and electroencephalography features, we found the performance of the proposed approach using features fusion significantly improved, demonstrating the effectiveness and efficiency of our method.

Improvement of Cyber-Attack Detection Accuracy from Urban Water Systems Using Extreme Learning Machine

This study proposes a novel detection model for the detection of cyber-attacks using remote sensing data on water distribution systems (i.e., pipe flow sensor, nodal pressure sensor, tank water level sensor, and programmable logic controllers) by machine learning approaches. The most commonly used and well-known machine learning algorithms (i.e., k-nearest neighbor, support vector machine, artificial neural network, and extreme learning machine) were compared to determine the one with the best detection performance. After identifying the best algorithm, several improved versions of the algorithm are compared and analyzed according to their characteristics. Their quantitative performances and abilities to correctly classify the state of the urban water system under cyber-attack were measured using various performance indices. Among the algorithms tested, the extreme learning machine (ELM) was found to exhibit the best performance. Moreover, this study not only has identified excellent algorithm among the compared algorithms but also has considered an improved version of the outstanding algorithm. Furthermore, the comparison was performed using various representative performance indices to quantitatively measure the prediction accuracy and select the most appropriate model. Therefore, this study provides a new perspective on the characteristics of various versions of machine learning algorithms and their application to different problems, and this study may be referenced as a case study for future cyber-attack detection fields.

Improving the Accuracy of Low-Cost Sensor Measurements for Freezer Automation

In this work, a regression method is implemented on a low-cost digital temperature sensor to improve the sensor's accuracy; thus, following the EN12830 European standard. This standard defines that the maximum acceptable error regarding temperature monitoring devices should not exceed 1 °C for the refrigeration and freezer areas. The purpose of the proposed method is to improve the accuracy of a low-cost digital temperature sensor by correcting its nonlinear response using simple linear regression (SLR). In the experimental part of this study, the proposed method's outcome (in a custom created dataset containing values taken from a refrigerator) is compared against the values taken from a sensor complying with the EN12830 standard. The experimental results confirmed that the proposed method reduced the mean absolute error (MAE) by 82% for the refrigeration area and 69% for the freezer area-resulting in the accuracy improvement of the low-cost digital temperature sensor. Moreover, it managed to achieve a lower generalization error on the test set when compared to three other machine learning algorithms (SVM, B-ELM, and OS-ELM).

Coal Classification Method Based on Improved Local Receptive Field-Based Extreme Learning Machine Algorithm and Visible-Infrared Spectroscopy

In the process of using coal, if the type of coal cannot be accurately determined, it will have a significant impact on production efficiency, environmental pollution, and economic loss. At present, the traditional classification method of coal mainly relies on technician's experience. This requires a lot of manpower and time, and it is difficult to automate. This paper mainly studies the application of visible-infrared spectroscopy and machine learning methods in coal mine identification and analysis to provide guidance for coal mining and production. This paper explores a fast and high-precision method for coal identification. In this paper, for the characteristics of high dimensionality, strong correlation, and large redundancy of spectral data, the local receptive field (LRF) is used to extract the advanced features of spectral data, which is combined with the extreme learning machine (ELM). We improved the coyote optimization algorithm (COA). The improved coyote optimization algorithm (I-COA) and local receptive field-based extreme learning machine (ELM-LRF) are used to optimize the structure and training parameters of the extreme learning machine network. The experimental results show that the coal classification model based on the network and visible-infrared spectroscopy can effectively identify the coal types through the spectral data. Compared with convolutional neural networks (CNN algorithm) and principal component analysis (PCA algorithm), LRF can extract the spectral characteristics of coal more effectively.

A Hybrid Method Based on Extreme Learning Machine and Self Organizing Map for Pattern Classification

Extreme learning machine is a fast learning algorithm for single hidden layer feedforward neural network. However, an improper number of hidden neurons and random parameters have a great effect on the performance of the extreme learning machine. In order to select a suitable number of hidden neurons, this paper proposes a novel hybrid learning based on a two-step process. First, the parameters of hidden layer are adjusted by a self-organized learning algorithm. Next, the weights matrix of the output layer is determined using the Moore–Penrose inverse method. Nine classification datasets are considered to demonstrate the efficiency of the proposed approach compared with original extreme learning machine, Tikhonov regularization optimally pruned extreme learning machine, and backpropagation algorithms. The results show that the proposed method is fast and produces better accuracy and generalization performances.

Ensemble Deep Learning Models for Heart Disease Classification: A Case Study from Mexico

Heart diseases are highly ranked among the leading causes of mortality in the world. They have various types including vascular, ischemic, and hypertensive heart disease. A large number of medical features are reported for patients in the Electronic Health Records (EHR) that allow physicians to diagnose and monitor heart disease. We collected a dataset from Medica Norte Hospital in Mexico that includes 800 records and 141 indicators such as age, weight, glucose, blood pressure rate, and clinical symptoms. Distribution of the collected records is very unbalanced on the different types of heart disease, where 17% of records have hypertensive heart disease, 16% of records have ischemic heart disease, 7% of records have mixed heart disease, and 8% of records have valvular heart disease. Herein, we propose an ensemble-learning framework of different neural network models, and a method of aggregating random under-sampling. To improve the performance of the classification algorithms, we implement a data preprocessing step with features selection. Experiments were conducted with unidirectional and bidirectional neural network models and results showed that an ensemble classifier with a BiLSTM or BiGRU model with a CNN model had the best classification performance with accuracy and F1-score between 91% and 96% for the different types of heart disease. These results are competitive and promising for heart disease dataset. We showed that ensemble-learning framework based on deep models could overcome the problem of classifying an unbalanced heart disease dataset. Our proposed framework can lead to highly accurate models that are adapted for clinical real data and diagnosis use.

Angle Estimation for Knee Joint Movement Based on PCA-RELM Algorithm

Surface electromyogram (sEMG) signals are easy to record and offer valuable motion information, such as symmetric and periodic motion in human gait. Due to these characteristics, sEMG is widely used in human-computer interaction, clinical diagnosis and rehabilitation medicine, sports medicine and other fields. This paper aims to improve the estimation accuracy and real-time performance, in the case of the knee joint angle in the lower limb, using a sEMG signal, in a proposed estimation algorithm of the continuous motion, based on the principal component analysis (PCA) and the regularized extreme learning machine (RELM). First, the sEMG signals, collected during the lower limb motion, are preprocessed, while feature samples are extracted from the acquired and preconditioned sEMG signals. Next, the feature samples dimensions are reduced by the PCA, as well as the knee joint angle system is measured by the three-dimensional motion capture system, are followed by the normalization of the feature variable value. The normalized sEMG feature is used as the input layer, in the RELM model, while the joint angle is used as the output layer. After training, the RELM model estimates the knee joint angle of the lower limbs, while it uses the root mean square error (RMSE), Pearson correlation coefficient and model training time as key performance indicators (KPIs), to be further discussed. The RELM, the traditional BP neural network and the support vector machine (SVM) estimation results are compared. The conclusions prove that the RELM method, not only has ensured the validity of results, but also has greatly reduced the learning train time. The presented work is a valuable point of reference for further study of the motion estimation in lower limb.

The Computer Course Correlation between Learning Satisfaction and Learning Effectiveness of Vocational College in Taiwan

In this paper, we surveyed the influence of learn effectiveness in a computer course under the factors of learning attitude and learning problems for students in senior-high school. We followed the formula for a regression line as R = A + BX +ε and simulated on SPSS platform with symmetry to obtained the results as follows: (1) In learning attitude, both the cognitive-level and behavior-level, are positively correlated with satisfaction. This means the students have cognitive-level and behavior-level more positively correlated with satisfaction in computer subjects and have a high degree of self-learning effectiveness. (2) In learning problems, the female students had higher learning effectiveness than male students, and the students who practiced on the computer on their own initiative long-term each week had higher learning effectiveness.

Hierarchical Feature Selection for Random Projection

Random projection is a popular machine learning algorithm, which can be implemented by neural networks and trained in a very efficient manner. However, the number of features should be large enough when applied to a rather large-scale data set, which results in slow speed in testing procedure and more storage space under some circumstances. Furthermore, some of the features are redundant and even noisy since they are randomly generated, so the performance may be affected by these features. To remedy these problems, an effective feature selection method is introduced to select useful features hierarchically. Specifically, a novel criterion is proposed to select useful neurons for neural networks, which establishes a new way for network architecture design. The testing time and accuracy of the proposed method are improved compared with traditional methods and some variations on both classification and regression tasks. Extensive experiments confirm the effectiveness of the proposed method.

Hypercomplex extreme learning machine with its application in multispectral palmprint recognition

An extreme learning machine (ELM) is a novel training method for single-hidden layer feedforward neural networks (SLFNs) in which the hidden nodes are randomly assigned and fixed without iterative tuning. ELMs have earned widespread global interest due to their fast learning speed, satisfactory generalization ability and ease of implementation. In this paper, we extend this theory to hypercomplex space and attempt to simultaneously consider multisource information using a hypercomplex representation. To illustrate the performance of the proposed hypercomplex extreme learning machine (HELM), we have applied this scheme to the task of multispectral palmprint recognition. Images from different spectral bands are utilized to construct the hypercomplex space. Extensive experiments conducted on the PolyU and CASIA multispectral databases demonstrate that the HELM scheme can achieve competitive results. The source code together with datasets involved in this paper can be available for free download at https://figshare.com/s/01aef7d48840afab9d6d.

Fault Diagnosis of Tennessee-Eastman Process Using Orthogonal Incremental Extreme Learning Machine Based on Driving Amount

Fault diagnosis is important to the industrial process. This paper proposes an orthogonal incremental extreme learning machine based on driving amount (DAOI-ELM) for recognizing the faults of the Tennessee-Eastman process (TEP). The basic idea of DAOI-ELM is to incorporate the Gram-Schmidt orthogonalization method and driving amount into an incremental extreme learning machine (I-ELM). The case study for the 2-D nonlinear function and regression problems from the UCI dataset results show that DAOI-ELM can obtain better generalization ability and a more compact structure of ELM than I-ELM, convex I-ELM (CI-ELM), orthogonal I-ELM (OI-ELM), and bidirectional ELM. The experimental training and testing data are derived from the simulations of TEP. The performance of DAOI-ELM is evaluated and compared with that of the back propagation neural network, support vector machine, I-ELM, CI-ELM, and OI-ELM. The simulation results show that DAOI-ELM diagnoses the TEP faults better than other methods.

A Parallel Multiclassification Algorithm for Big Data Using an Extreme Learning Machine

As data sets become larger and more complicated, an extreme learning machine (ELM) that runs in a traditional serial environment cannot realize its ability to be fast and effective. Although a parallel ELM (PELM) based on MapReduce to process large-scale data shows more efficient learning speed than identical ELM algorithms in a serial environment, some operations, such as intermediate results stored on disks and multiple copies for each task, are indispensable, and these operations create a large amount of extra overhead and degrade the learning speed and efficiency of the PELMs. In this paper, an efficient ELM based on the Spark framework (SELM), which includes three parallel subalgorithms, is proposed for big data classification. By partitioning the corresponding data sets reasonably, the hidden layer output matrix calculation algorithm, matrix decomposition algorithm, and matrix decomposition algorithm perform most of the computations locally. At the same time, they retain the intermediate results in distributed memory and cache the diagonal matrix as broadcast variables instead of several copies for each task to reduce a large amount of the costs, and these actions strengthen the learning ability of the SELM. Finally, we implement our SELM algorithm to classify large data sets. Extensive experiments have been conducted to validate the effectiveness of the proposed algorithms. As shown, our SELM achieves an speedup on a cluster with ten nodes, and reaches a speedup with 15 nodes, an speedup with 20 nodes, a speedup with 25 nodes, a speedup with 30 nodes, and a speedup with 35 nodes.

Extreme Learning Machine With Subnetwork Hidden Nodes for Regression and Classification

As demonstrated earlier, the learning effectiveness and learning speed of single-hidden-layer feedforward neural networks are in general far slower than required, which has been a major bottleneck for many applications. Huang et al. proposed extreme learning machine (ELM) which improves the training speed by hundreds of times as compared to its predecessor learning techniques. This paper offers an ELM-based learning method that can grow subnetwork hidden nodes by pulling back residual network error to the hidden layer. Furthermore, the proposed method provides a similar or better generalization performance with remarkably fewer hidden nodes as compared to other ELM methods employing huge number of hidden nodes. Thus, the learning speed of the proposed technique is hundred times faster compared to other ELMs as well as to back propagation and support vector machines. The experimental validations for all methods are carried out on 32 data sets.

Multilayer Extreme Learning Machine With Subnetwork Nodes for Representation Learning

The extreme learning machine (ELM), which was originally proposed for "generalized" single-hidden layer feedforward neural networks, provides efficient unified learning solutions for the applications of clustering, regression, and classification. It presents competitive accuracy with superb efficiency in many applications. However, ELM with subnetwork nodes architecture has not attracted much research attentions. Recently, many methods have been proposed for supervised/unsupervised dimension reduction or representation learning, but these methods normally only work for one type of problem. This paper studies the general architecture of multilayer ELM (ML-ELM) with subnetwork nodes, showing that: 1) the proposed method provides a representation learning platform with unsupervised/supervised and compressed/sparse representation learning and 2) experimental results on ten image datasets and 16 classification datasets show that, compared to other conventional feature learning methods, the proposed ML-ELM with subnetwork nodes performs competitively or much better than other feature learning methods.