Evolutionary optimization of convolutional neural networks for cancer miRNA biomarkers classification

A new particle swarm optimization-enhanced deep neural network for automatic ECG arrhythmias classification

This study proposes an ECG classification system using particle swarm optimization (PSO) for automated deep neural network hyperparameter tuning. PSO optimizes five key parameters: neuron counts in two fully connected layers, dropout rate, learning rate, and optimizer selection. ECG signals undergo wavelet decomposition for feature extraction, with classification performed on the MIT-BIH Arrhythmia Database across five heartbeat classes. The PSO-optimized model achieves superior performance with 99.76% accuracy, 99.34% precision, and 99.21% F1 score, demonstrating PSO's effectiveness in improving model reliability while reducing manual effort.

A related convolutional neural network for cancer diagnosis using microRNA data classification

This paper develops a method for cancer classification from microRNA data using a convolutional neural network (CNN)-based model optimized by genetic algorithm. The convolutional neural network has performed well in various recognition and perception tasks. This paper contributes to the cancer classification using a union of two CNNs. The method's performance is boosted by the relationship between CNNs and exchanging knowledge between them. Besides, communication between small sizes of CNNs reduces the need for large size CNNs and, consequently, the computational time and memory usage while preserving high accuracy. The method proposed is tested on microRNA dataset containing the genomic information of 8129 patients for 29 different types of cancer with 1046 gene expression. The classification accuracy of the selected genes obtained by the proposed approach is compared with the accuracy of 22 well-known classifiers on a real-world dataset. The classification accuracy of each cancer type is also ranked with the results of 77 classifiers reported in previous works. The proposed approach shows accuracy of 100% in 24 out of 29 classes and in seven cases out of 29, the method achieved 100% accuracy that no classifier in other studies has reached. Performance analysis is performed using performance metrics.

Embedding channel pruning within the CNN architecture design using a bi-level evolutionary approach

Remarkable advancements have been achieved in machine learning and computer vision through the utilization of deep neural networks. Among the most advantageous of these networks is the convolutional neural network (CNN). It has been used in pattern recognition, medical diagnosis, and signal processing, among other things. Actually, for these networks, the challenge of choosing hyperparameters is of utmost importance. The reason behind this is that as the number of layers rises, the search space grows exponentially. In addition, all known classical and evolutionary pruning algorithms require a trained or built architecture as input. During the design phase, none of them consider the process of pruning. In order to assess the effectiveness and efficiency of any architecture created, pruning of channels must be carried out before transmitting the dataset and computing classification errors. For instance, following pruning, an architecture of medium quality in terms of classification may transform into an architecture that is both highly light and accurate, and vice versa. There exist countless potential scenarios that could occur, which prompted us to develop a bi-level optimization approach for the entire process. The upper level involves generating the architecture while the lower level optimizes channel pruning. Evolutionary algorithms (EAs) have proven effective in bi-level optimization, leading us to adopt the co-evolutionary migration-based algorithm as a search engine for our bi-level architectural optimization problem in this research. Our proposed method, CNN-D-P (bi-level CNN design and pruning), was tested on the widely used image classification benchmark datasets, CIFAR-10, CIFAR-100 and ImageNet. Our suggested technique is validated by means of a set of comparison tests with regard to relevant state-of-the-art architectures.

Application of Meta-Heuristic Algorithms for Training Neural Networks and Deep Learning Architectures: A Comprehensive Review

The learning process and hyper-parameter optimization of artificial neural networks (ANNs) and deep learning (DL) architectures is considered one of the most challenging machine learning problems. Several past studies have used gradient-based back propagation methods to train DL architectures. However, gradient-based methods have major drawbacks such as stucking at local minimums in multi-objective cost functions, expensive execution time due to calculating gradient information with thousands of iterations and needing the cost functions to be continuous. Since training the ANNs and DLs is an NP-hard optimization problem, their structure and parameters optimization using the meta-heuristic (MH) algorithms has been considerably raised. MH algorithms can accurately formulate the optimal estimation of DL components (such as hyper-parameter, weights, number of layers, number of neurons, learning rate, etc.). This paper provides a comprehensive review of the optimization of ANNs and DLs using MH algorithms. In this paper, we have reviewed the latest developments in the use of MH algorithms in the DL and ANN methods, presented their disadvantages and advantages, and pointed out some research directions to fill the gaps between MHs and DL methods. Moreover, it has been explained that the evolutionary hybrid architecture still has limited applicability in the literature. Also, this paper classifies the latest MH algorithms in the literature to demonstrate their effectiveness in DL and ANN training for various applications. Most researchers tend to extend novel hybrid algorithms by combining MHs to optimize the hyper-parameters of DLs and ANNs. The development of hybrid MHs helps improving algorithms performance and capable of solving complex optimization problems. In general, the optimal performance of the MHs should be able to achieve a suitable trade-off between exploration and exploitation features. Hence, this paper tries to summarize various MH algorithms in terms of the convergence trend, exploration, exploitation, and the ability to avoid local minima. The integration of MH with DLs is expected to accelerate the training process in the coming few years. However, relevant publications in this way are still rare.

Joint design and compression of convolutional neural networks as a Bi-level optimization problem

Over the last decade, deep neural networks have shown great success in the fields of machine learning and computer vision. Currently, the CNN (convolutional neural network) is one of the most successful networks, having been applied in a wide variety of application domains, including pattern recognition, medical diagnosis and signal processing. Despite CNNs' impressive performance, their architectural design remains a significant challenge for researchers and practitioners. The problem of selecting hyperparameters is extremely important for these networks. The reason for this is that the search space grows exponentially in size as the number of layers increases. In fact, all existing classical and evolutionary pruning methods take as input an already pre-trained or designed architecture. None of them take pruning into account during the design process. However, to evaluate the quality and possible compactness of any generated architecture, filter pruning should be applied before the communication with the data set to compute the classification error. For instance, a medium-quality architecture in terms of classification could become a very light and accurate architecture after pruning, and vice versa. Many cases are possible, and the number of possibilities is huge. This motivated us to frame the whole process as a bi-level optimization problem where: (1) architecture generation is done at the upper level (with minimum NB and NNB) while (2) its filter pruning optimization is done at the lower level. Motivated by evolutionary algorithms' (EAs) success in bi-level optimization, we use the newly suggested co-evolutionary migration-based algorithm (CEMBA) as a search engine in this research to address our bi-level architectural optimization problem. The performance of our suggested technique, called Bi-CNN-D-C (Bi-level convolution neural network design and compression), is evaluated using the widely used benchmark data sets for image classification, called CIFAR-10, CIFAR-100 and ImageNet. Our proposed approach is validated by means of a set of comparative experiments with respect to relevant state-of-the-art architectures.

Machine Learning-Based Ensemble Recursive Feature Selection of Circulating miRNAs for Cancer Tumor Classification

Circulating microRNAs (miRNA) are small noncoding RNA molecules that can be detected in bodily fluids without the need for major invasive procedures on patients. miRNAs have shown great promise as biomarkers for tumors to both assess their presence and to predict their type and subtype. Recently, thanks to the availability of miRNAs datasets, machine learning techniques have been successfully applied to tumor classification. The results, however, are difficult to assess and interpret by medical experts because the algorithms exploit information from thousands of miRNAs. In this work, we propose a novel technique that aims at reducing the necessary information to the smallest possible set of circulating miRNAs. The dimensionality reduction achieved reflects a very important first step in a potential, clinically actionable, circulating miRNA-based precision medicine pipeline. While it is currently under discussion whether this first step can be taken, we demonstrate here that it is possible to perform classification tasks by exploiting a recursive feature elimination procedure that integrates a heterogeneous ensemble of high-quality, state-of-the-art classifiers on circulating miRNAs. Heterogeneous ensembles can compensate inherent biases of classifiers by using different classification algorithms. Selecting features then further eliminates biases emerging from using data from different studies or batches, yielding more robust and reliable outcomes. The proposed approach is first tested on a tumor classification problem in order to separate 10 different types of cancer, with samples collected over 10 different clinical trials, and later is assessed on a cancer subtype classification task, with the aim to distinguish triple negative breast cancer from other subtypes of breast cancer. Overall, the presented methodology proves to be effective and compares favorably to other state-of-the-art feature selection methods.

Automatic discovery of 100-miRNA signature for cancer classification using ensemble feature selection

Background

MicroRNAs (miRNAs) are noncoding RNA molecules heavily involved in human tumors, in which few of them circulating the human body. Finding a tumor-associated signature of miRNA, that is, the minimum miRNA entities to be measured for discriminating both different types of cancer and normal tissues, is of utmost importance. Feature selection techniques applied in machine learning can help however they often provide naive or biased results.

Results

An ensemble feature selection strategy for miRNA signatures is proposed. miRNAs are chosen based on consensus on feature relevance from high-accuracy classifiers of different typologies. This methodology aims to identify signatures that are considerably more robust and reliable when used in clinically relevant prediction tasks. Using the proposed method, a 100-miRNA signature is identified in a dataset of 8023 samples, extracted from TCGA. When running eight-state-of-the-art classifiers along with the 100-miRNA signature against the original 1046 features, it could be detected that global accuracy differs only by 1.4%. Importantly, this 100-miRNA signature is sufficient to distinguish between tumor and normal tissues. The approach is then compared against other feature selection methods, such as UFS, RFE, EN, LASSO, Genetic Algorithms, and EFS-CLA. The proposed approach provides better accuracy when tested on a 10-fold cross-validation with different classifiers and it is applied to several GEO datasets across different platforms with some classifiers showing more than 90% classification accuracy, which proves its cross-platform applicability.

Conclusions

The 100-miRNA signature is sufficiently stable to provide almost the same classification accuracy as the complete TCGA dataset, and it is further validated on several GEO datasets, across different types of cancer and platforms. Furthermore, a bibliographic analysis confirms that 77 out of the 100 miRNAs in the signature appear in lists of circulating miRNAs used in cancer studies, in stem-loop or mature-sequence form. The remaining 23 miRNAs offer potentially promising avenues for future research.

Electronic supplementary material

The online version of this article (10.1186/s12859-019-3050-8) contains supplementary material, which is available to authorized users.

An Experiment on the Use of Genetic Algorithms for Topology Selection in Deep Learning

The choice of a good topology for a deep neural network is a complex task, essential for any deep learning project. This task normally demands knowledge from previous experience, as the higher amount of required computational resources makes trial and error approaches prohibitive. Evolutionary computation algorithms have shown success in many domains, by guiding the exploration of complex solution spaces in the direction of the best solutions, with minimal human intervention. In this sense, this work presents the use of genetic algorithms in deep neural networks topology selection. The evaluated algorithms were able to find competitive topologies while spending less computational resources when compared to state-of-the-art methods.