Performance analyses of five neural network classifiers on nodule classification in lung CT images using WEKA: a comparative study

In this report, we are presenting our work on performance analyses of five different neural network classifiers viz. MLP, DL4JMLP, logistic regression, SGD and simple logistic classifier in lung nodule detection using WEKA interface. To the best of our knowledge, this report demonstrates first use of WEKA for comparative performance analyses of neural network classifiers in identifying lung nodules from lung CT-images. A total of 624 handcrafted features from 52 numbers of lung CT-images collected randomly from Lung Image Database Consortium (LIDC) were fed into WEKA to evaluate the performances of the classifiers under four different categories of computation. Performances of the classifiers were observed in terms of 11 important parameters viz. accuracy, kappa statistic, root mean squared error, TPR, FPR, precision, sensitivity, F-measurement, MCC, ROC area and PRC area. Results show 86.53%, 77.77%, 55.55%, 94.44% & 88.88% accuracy as well as 0.91, 0.86, 0.68, 0.91 & 0.93 ROC area for MLP, DL4JMLP, logistic, SGD and simple logistic classifier respectively at tenfold cross-validation by taking 66% of the data set for training and 34% for testing and validation purpose. SGDClassifier has been found the best performing followed by simple logistic classifier for the purpose.

A survey of data element perspective: Application of artificial intelligence in health big data

Artificial intelligence (AI) based on the perspective of data elements is widely used in the healthcare informatics domain. Large amounts of clinical data from electronic medical records (EMRs), electronic health records (EHRs), and electroencephalography records (EEGs) have been generated and collected at an unprecedented speed and scale. For instance, the new generation of wearable technologies enables easy-collecting peoples’ daily health data such as blood pressure, blood glucose, and physiological data, as well as the application of EHRs documenting large amounts of patient data. The cost of acquiring and processing health big data is expected to reduce dramatically with the help of AI technologies and open-source big data platforms such as Hadoop and Spark. The application of AI technologies in health big data presents new opportunities to discover the relationship among living habits, sports, inheritances, diseases, symptoms, and drugs. Meanwhile, with the development of fast-growing AI technologies, many promising methodologies are proposed in the healthcare field recently. In this paper, we review and discuss the application of machine learning (ML) methods in health big data in two major aspects: (1) Special features of health big data including multimodal, incompletion, time validation, redundancy, and privacy. (2) ML methodologies in the healthcare field including classification, regression, clustering, and association. Furthermore, we review the recent progress and breakthroughs of automatic diagnosis in health big data and summarize the challenges, gaps, and opportunities to improve and advance automatic diagnosis in the health big data field.

An Integrative Computational Approach for the Prediction of Human-Plasmodium Protein-Protein Interactions

Host-pathogen molecular cross-talks are critical in determining the pathophysiology of a specific infection. Most of these cross-talks are mediated via protein-protein interactions between the host and the pathogen (HP-PPI). Thus, it is essential to know how some pathogens interact with their hosts to understand the mechanism of infections. Malaria is a life-threatening disease caused by an obligate intracellular parasite belonging to the Plasmodium genus, of which P. falciparum is the most prevalent. Several previous studies predicted human-plasmodium protein-protein interactions using computational methods have demonstrated their utility, accuracy, and efficiency to identify the interacting partners and therefore complementing experimental efforts to characterize host-pathogen interaction networks. To predict potential putative HP-PPIs, we use an integrative computational approach based on the combination of multiple OMICS-based methods including human red blood cells (RBC) and Plasmodium falciparum 3D7 strain expressed proteins, domain-domain based PPI, similarity of gene ontology terms, structure similarity method homology identification, and machine learning prediction. Our results reported a set of 716 protein interactions involving 302 human proteins and 130 Plasmodium proteins. This work provides a list of potential human-Plasmodium interacting proteins. These findings will contribute to better understand the mechanisms underlying the molecular determinism of malaria disease and potentially to identify candidate pharmacological targets.

Greedy Projected Gradient-Newton Method for Sparse Logistic Regression

Sparse logistic regression (SLR), which is widely used for classification and feature selection in many fields, such as neural networks, deep learning, and bioinformatics, is the classical logistic regression model with sparsity constraints. In this paper, we perform theoretical analysis on the existence and uniqueness of the solution to the SLR, and we propose a greedy projected gradient-Newton (GPGN) method for solving the SLR. The GPGN method is a combination of the projected gradient method and the Newton method. The following characteristics show that the GPGN method achieves not only elegant theoretical results but also a remarkable numerical performance in solving the SLR: 1) the full iterative sequence generated by the GPGN method converges to a global/local minimizer of the SLR under weaker conditions; 2) the GPGN method has the properties of afinite identification for an optimal support set and local quadratic convergence; and 3) the GPGN method achieves higher accuracy and higher speed compared with a number of state-of-the-art solvers according to numerical experiments.

Fast Multi-Objective Antenna Optimization Based on RBF Neural Network Surrogate Model Optimized by Improved PSO Algorithm

In this paper, a radial basis function neural network (RBFNN) surrogate model optimized by an improved particle swarm optimization (PSO) algorithm is developed to reduce the computation cost of traditional antenna design methods which rely on high-fidelity electromagnetic (EM) simulations. Considering parameters adjustment and update mechanism simultaneously, two modifications are proposed in this improved PSO. First, time-varying learning factors are designed to balance exploration and exploitation ability of particles in the search space. Second, the local best information is added to the updating process of particles except for personal and global best information for better population diversity. The improved PSO is applied to train RBFNN for determining optimal network parameters. As a result, the constructed improved PSO-RBFNN model can be used as a surrogate model for antenna performance prediction with better network generalization capability. By integrating the improved PSO-RBFNN surrogate model with multi-objective evolutionary algorithms (MOEAs), a fast multi-objective antenna optimization framework for multi-parameter antenna structures is then established. Finally, a Pareto-optimal planar miniaturized multiband antenna design is presented, demonstrating that the proposed model provides better prediction performance and considerable computational savings compared to those previously published approaches.

Biologically-Inspired Learning and Adaptation of Self-Evolving Control for Networked Mobile Robots

This paper presents a biologically-inspired learning and adaptation method for self-evolving control of networked mobile robots. A Kalman filter (KF) algorithm is employed to develop a self-learning RBFNN (Radial Basis Function Neural Network), called the KF-RBFNN. The structure of the KF-RBFNN is optimally initialized by means of a modified genetic algorithm (GA) in which a Lévy flight strategy is applied. By using the derived mathematical kinematic model of the mobile robots, the proposed GA-KF-RBFNN is utilized to design a self-evolving motion control law. The control parameters of the mobile robots are self-learned and adapted via the proposed GA-KF-RBFNN. This approach is extended to address the formation control problem of networked mobile robots by using a broadcast leader-follower control strategy. The proposed pragmatic approach circumvents the communication delay problem found in traditional networked mobile robot systems where consensus graph theory and directed topology are applied. The simulation results and numerical analysis are provided to demonstrate the merits and effectiveness of the developed GA-KF-RBFNN to achieve self-evolving formation control of networked mobile robots.

Detection of Oil Chestnuts Infected by Blue Mold Using Near-Infrared Hyperspectral Imaging Combined with Artificial Neural Networks

Mildew damage is a major reason for chestnut poor quality and yield loss. In this study, a near-infrared hyperspectral imaging system in the 874⁻1734 nm spectral range was applied to detect the mildew damage to chestnuts caused by blue mold. Principal component analysis (PCA) scored images were firstly employed to qualitatively and intuitively distinguish moldy chestnuts from healthy chestnuts. Spectral data were extracted from the hyperspectral images. A successive projections algorithm (SPA) was used to select 12 optimal wavelengths. Artificial neural networks, including back propagation neural network (BPNN), evolutionary neural network (ENN), extreme learning machine (ELM), general regression neural network (GRNN) and radial basis neural network (RBNN) were used to build models using the full spectra and optimal wavelengths to distinguish moldy chestnuts. BPNN and ENN models using full spectra and optimal wavelengths obtained satisfactory performances, with classification accuracies all surpassing 99%. The results indicate the potential for the rapid and non-destructive detection of moldy chestnuts by hyperspectral imaging, which would help to develop online detection system for healthy and blue mold infected chestnuts.

An Adaptive-PSO-Based Self-Organizing RBF Neural Network

In this paper, a self-organizing radial basis function (SORBF) neural network is designed to improve both accuracy and parsimony with the aid of adaptive particle swarm optimization (APSO). In the proposed APSO algorithm, to avoid being trapped into local optimal values, a nonlinear regressive function is developed to adjust the inertia weight. Furthermore, the APSO algorithm can optimize both the network size and the parameters of an RBF neural network simultaneously. As a result, the proposed APSO-SORBF neural network can effectively generate a network model with a compact structure and high accuracy. Moreover, the analysis of convergence is given to guarantee the successful application of the APSO-SORBF neural network. Finally, multiple numerical examples are presented to illustrate the effectiveness of the proposed APSO-SORBF neural network. The results demonstrate that the proposed method is more competitive in solving nonlinear problems than some other existing SORBF neural networks.

A Fast Adaptive Tunable RBF Network For Nonstationary Systems

This paper describes a novel on-line learning approach for radial basis function (RBF) neural network. Based on an RBF network with individually tunable nodes and a fixed small model size, the weight vector is adjusted using the multi-innovation recursive least square algorithm on-line. When the residual error of the RBF network becomes large despite of the weight adaptation, an insignificant node with little contribution to the overall system is replaced by a new node. Structural parameters of the new node are optimized by proposed fast algorithms in order to significantly improve the modeling performance. The proposed scheme describes a novel, flexible, and fast way for on-line system identification problems. Simulation results show that the proposed approach can significantly outperform existing ones for nonstationary systems in particular.

Radial basis function neural networks: a topical state-of-the-art survey

Radial basis function networks (RBFNs) have gained widespread appeal amongst researchers and have shown good performance in a variety of application domains. They have potential for hybridization and demonstrate some interesting emergent behaviors. This paper aims to offer a compendious and sensible survey on RBF networks. The advantages they offer, such as fast training and global approximation capability with local responses, are attracting many researchers to use them in diversified fields. The overall algorithmic development of RBF networks by giving special focus on their learning methods, novel kernels, and fine tuning of kernel parameters have been discussed. In addition, we have considered the recent research work on optimization of multi-criterions in RBF networks and a range of indicative application areas along with some open source RBFN tools.

Machine Learning Methods for Binary and Multiclass Classification of Melanoma Thickness From Dermoscopic Images

Thickness of the melanoma is the most important factor associated with survival in patients with melanoma. It is most commonly reported as a measurement of depth given in millimeters (mm) and computed by means of pathological examination after a biopsy of the suspected lesion. In order to avoid the use of an invasive method in the estimation of the thickness of melanoma before surgery, we propose a computational image analysis system from dermoscopic images. The proposed feature extraction is based on the clinical findings that correlate certain characteristics present in dermoscopic images and tumor depth. Two supervised classification schemes are proposed: a binary classification in which melanomas are classified into thin or thick, and a three-class scheme (thin, intermediate, and thick). The performance of several nominal classification methods, including a recent interpretable method combining logistic regression with artificial neural networks (Logistic regression using Initial variables and Product Units, LIPU), is compared. For the three-class problem, a set of ordinal classification methods (considering ordering relation between the three classes) is included. For the binary case, LIPU outperforms all the other methods with an accuracy of 77.6%, while, for the second scheme, although LIPU reports the highest overall accuracy, the ordinal classification methods achieve a better balance between the performances of all classes.

Negative correlation ensemble learning for ordinal regression

In this paper, two neural network threshold ensemble models are proposed for ordinal regression problems. For the first ensemble method, the thresholds are fixed a priori and are not modified during training. The second one considers the thresholds of each member of the ensemble as free parameters, allowing their modification during the training process. This is achieved through a reformulation of these tunable thresholds, which avoids the constraints they must fulfill for the ordinal regression problem. During training, diversity exists in different projections generated by each member is taken into account for the parameter updating. This diversity is promoted in an explicit way using a diversity-encouraging error function, extending the well-known negative correlation learning framework to the area of ordinal regression, and inheriting many of its good properties. Experimental results demonstrate that the proposed algorithms can achieve competitive generalization performance when considering four ordinal regression metrics.

Automated induction of heterogeneous proximity measures for supervised spectral embedding

Spectral embedding methods have played a very important role in dimensionality reduction and feature generation in machine learning. Supervised spectral embedding methods additionally improve the classification of labeled data, using proximity information that considers both features and class labels. However, these calculate the proximity information by treating all intraclass similarities homogeneously for all classes, and similarly for all interclass samples. In this paper, we propose a very novel and generic method which can treat all the intra- and interclass sample similarities heterogeneously by potentially using a different proximity function for each class and each class pair. To handle the complexity of selecting these functions, we employ evolutionary programming as an automated powerful formula induction engine. In addition, for computational efficiency and expressive power, we use a compact matrix tree representation equipped with a broad set of functions that can build most currently used similarity functions as well as new ones. Model selection is data driven, because the entire model is symbolically instantiated using only problem training data, and no user-selected functions or parameters are required. We perform thorough comparative experimentations with multiple classification datasets and many existing state-of-the-art embedding methods, which show that the proposed algorithm is very competitive in terms of classification accuracy and generalization ability.

Least square regularized regression in sum space

This paper proposes a least square regularized regression algorithm in sum space of reproducing kernel Hilbert spaces (RKHSs) for nonflat function approximation, and obtains the solution of the algorithm by solving a system of linear equations. This algorithm can approximate the low- and high-frequency component of the target function with large and small scale kernels, respectively. The convergence and learning rate are analyzed. We measure the complexity of the sum space by its covering number and demonstrate that the covering number can be bounded by the product of the covering numbers of basic RKHSs. For sum space of RKHSs with Gaussian kernels, by choosing appropriate parameters, we tradeoff the sample error and regularization error, and obtain a polynomial learning rate, which is better than that in any single RKHS. The utility of this method is illustrated with two simulated data sets and five real-life databases.

Radial basis function network training using a nonsymmetric partition of the input space and particle swarm optimization

This paper presents a novel algorithm for training radial basis function (RBF) networks, in order to produce models with increased accuracy and parsimony. The proposed methodology is based on a nonsymmetric variant of the fuzzy means (FM) algorithm, which has the ability to determine the number and locations of the hidden-node RBF centers, whereas the synaptic weights are calculated using linear regression. Taking advantage of the short computational times required by the FM algorithm, we wrap a particle swarm optimization (PSO) based engine around it, designed to optimize the fuzzy partition. The result is an integrated framework for fully determining all the parameters of an RBF network. The proposed approach is evaluated through its application on 12 real-world and synthetic benchmark datasets and is also compared with other neural network training techniques. The results show that the RBF network models produced by the PSO-based nonsymmetric FM algorithm outperform the models produced by the other techniques, exhibiting higher prediction accuracies in shorter computational times, accompanied by simpler network structures.

A STRUCTURAL DISTANCE-BASED CROSSOVER FOR NEURAL NETWORK CLASSIFIERS

This paper presents a structural distance-based crossover for neural network classifiers, which is applied as part of a Memetic Algorithm (MA) for evolving simultaneously the structure and weights of neural network models applied to multiclass problems. Previous researchers have shown that this simultaneous evolution is a way to avoid the noisy fitness evaluation. The MA incorporates a crossover operator that shows to be useful for ameliorating the permutation problem of the network representation (i.e. different genotypes can be used to represent the same neural network phenotype), increasing the structural diversity of the individuals and improving the accuracy of the results. Instead of a recombination probability, the crossover operator considers a similarity parameter (the minimum structural distance), which allows to maintain a trade-off between global and local search. The neural network models selected in this work are the product-unit neural networks (PUNNs), due to their increasing relevance in those classification problems which show a high order relationship between the input variables. The proposed MA is intended to reduce the possible overtraining problems which can raise in some datasets for this kind of models. The evolutionary system is applied to eight classification benchmarks and the results of an analysis of variance contrast (ANOVA) show the effectiveness of the structural-based crossover operator and the capacity of our algorithm to obtain evolved PUNNs with a higher classification accuracy than those obtained using other evolutionary techniques. On the other hand, the results obtained are compared with popular effective machine learning classification methods, resulting in a competitive performance.

Adaptive computation algorithm for RBF neural network

A novel learning algorithm is proposed for nonlinear modelling and identification using radial basis function neural networks. The proposed method simplifies neural network training through the use of an adaptive computation algorithm (ACA). In addition, the convergence of the ACA is analyzed by the Lyapunov criterion. The proposed algorithm offers two important advantages. First, the model performance can be significantly improved through ACA, and the modelling error is uniformly ultimately bounded. Secondly, the proposed ACA can reduce computational cost and accelerate the training speed. The proposed method is then employed to model classical nonlinear system with limit cycle and to identify nonlinear dynamic system, exhibiting the effectiveness of the proposed algorithm. Computational complexity analysis and simulation results demonstrate its effectiveness.

Nonlinear regularization path for quadratic loss support vector machines

Regularization path algorithms have been proposed to deal with model selection problem in several machine learning approaches. These algorithms allow computation of the entire path of solutions for every value of regularization parameter using the fact that their solution paths have piecewise linear form. In this paper, we extend the applicability of regularization path algorithm to a class of learning machines that have quadratic loss and quadratic penalty term. This class contains several important learning machines such as squared hinge loss support vector machine (SVM) and modified Huber loss SVM. We first show that the solution paths of this class of learning machines have piecewise nonlinear form, and piecewise segments between two breakpoints are characterized by a class of rational functions. Then we develop an algorithm that can efficiently follow the piecewise nonlinear path by solving these rational equations. To solve these rational equations, we use rational approximation technique with quadratic convergence rate, and thus, our algorithm can follow the nonlinear path much more precisely than existing approaches such as predictor-corrector type nonlinear-path approximation. We show the algorithm performance on some artificial and real data sets.