Neural network for regression problems with reduced training sets

Attention-assisted hybrid 1D CNN-BiLSTM model for predicting electric field induced by transcranial magnetic stimulation coil

Deep learning-based models such as deep neural network (DNN) and convolutional neural network (CNN) have recently been established as state-of-the-art for enumerating electric fields from transcranial magnetic stimulation coil. One of the main challenges related to this electric field enumeration is the prediction time and accuracy. Despite the low computational cost, the performance of the existing prediction models for electric field enumeration is quite inefficient. This study proposes a 1D CNN-based bi-directional long short-term memory (BiLSTM) model with an attention mechanism to predict electric field induced by a transcranial magnetic stimulation coil. The model employs three consecutive 1D CNN layers followed by the BiLSTM layer for extracting deep features. After that, the weights of the deep features are redistributed and integrated by the attention mechanism and a fully connected layer is utilized for the prediction. For the prediction purpose, six input features including coil turns of single wing, coil thickness, coil diameter, distance between two wings, distance between head and coil position, and angle between two wings of coil are mapped with the output of the electric field. The performance evaluation is conducted based on four verification metrics (e.g. R2, MSE, MAE, and RMSE) between the simulated data and predicted data. The results indicate that the proposed model outperforms existing DNN and CNN models in predicting the induced electrical field with R2 = 0.9992, MSE = 0.0005, MAE = 0.0188, and RMSE = 0.0228 in the testing stage.

Machine Learning Based Prediction for the Response of Gas Discharge Tube to Damped Sinusoid Signal

In order to predict the circuit response of a Gas Discharge Tube (GDT) to an electromagnetic pulse, a “black box” model for a GDT based on a machine learning method is proposed and validated in this paper.Firstly, the machine learning model of the Elman neural network is established by taking advantage of the existing measurement data to dampen the sinusoid signal, and then the established model is adopted to predict the response waveform of an unknown injection current grade and frequency.Without considering the complex physical parameters and dynamic behavior of GDTs, the Elman neural network modeling method is simpler than the existing physical or Pspice model.Validation experiments show a good agreement between the predicted and the measured waveforms.

Simple Breaker Index Formula Using Linear Model

Breaking waves generated by wave shoaling in coastal areas have a close relationship with various physical phenomena in coastal regions. Therefore, it is crucial to accurately predict breaker indexes such as breaking wave height and breaking depth when designing coastal structures. Many studies on wave breaking have been carried out, and many experimental data have been documented. Representative studies on wave breaking provide many empirical formulas for the prediction of breaking index, mainly through hydraulic model experiments. However, the existing empirical formulas for breaking index determine the coefficients of the assumed equation through statistical analysis of data under the assumption of a specific equation. This study presents an alternative method to estimate breaker index using representative linear-based supervised machine learning algorithms that show high predictive performance in various research fields related to regression or classification problems. Based on the used machine learning methods, a new simple linear equation for the prediction of breaker index is presented. The newly proposed breaker index formula showed similar predictive performance compared to the existing empirical formula, although it was a simple linear equation.

Multivariate Statistical Analysis for Training Process Optimization in Neural Networks-Based Forecasting Models

Data forecasting is very important for electrical analysis development, transport dimensionality, marketing strategies, etc. Hence, low error levels are required. However, in some cases data have dissimilar behaviors that can vary depending on such exogenous variables as the type of day, weather conditions, and geographical area, among others. Commonly, computational intelligence techniques (e.g., artificial neural networks) are used due to their generalization capabilities. In spite of the above, they do not have a unique way to reach optimal performance. For this reason, it is necessary to analyze the data’s behavior and their statistical features in order to identify those significant factors in the training process to guarantee a better performance. In this paper is proposed an experimental method for identifying those significant factors in the forecasting model for time series data and measure their effects on the Akaike information criterion (AIC) and the Mean Absolute Percentage Error (MAPE). Additionally, we seek to establish optimal parameters for the proper selection of the artificial neural network model.

Comparison of Diagnosis Accuracy between a Backpropagation Artificial Neural Network Model and Linear Regression in Digestive Disease Patients: an Empirical Research

Introduction

A Noninvasive diagnosis model for digestive diseases is the vital issue for the current clinical research. Our systematic review is aimed at demonstrating diagnosis accuracy between the BP-ANN algorithm and linear regression in digestive disease patients, including their activation function and data structure.

Methods

We reported the systematic review according to the PRISMA guidelines. We searched related articles from seven electronic scholarly databases for comparison of the diagnosis accuracy focusing on BP-ANN and linear regression. The characteristics, patient number, input/output marker, diagnosis accuracy, and results/conclusions related to comparison were extracted independently based on inclusion criteria.

Results

Nine articles met all the criteria and were enrolled in our review. Of those enrolled articles, the publishing year ranged from 1991 to 2017. The sample size ranged from 42 to 3222 digestive disease patients, and all of the patients showed comparable biomarkers between the BP-ANN algorithm and linear regression. According to our study, 8 literature demonstrated that the BP-ANN model is superior to linear regression in predicting the disease outcome based on AUROC results. One literature reported linear regression to be superior to BP-ANN for the early diagnosis of colorectal cancer.

Conclusion

The BP-ANN algorithm and linear regression both had high capacity in fitting the diagnostic model and BP-ANN displayed more prediction accuracy for the noninvasive diagnosis model of digestive diseases. We compared the activation functions and data structure between BP-ANN and linear regression for fitting the diagnosis model, and the data suggested that BP-ANN was a comprehensive recommendation algorithm.

Surrogate Models for Wind Turbine Electrical Power and Fatigue Loads in Wind Farm

Fatigue damage of turbine components is typically computed by running a rain-flow counting algorithm on the load signals of the components. This process is not linear and time consuming, thus, it is non-trivial for an application of wind farm control design and optimisation. To compensate this limitation, this paper will develop and compare different types of surrogate models that can predict the short term damage equivalent loads and electrical power of wind turbines, with respect to various wind conditions and down regulation set-points, in a wind farm. More specifically, Linear Regression, Artificial Neural Network and Gaussian Process Regression are the types of the developed surrogate models in this work. The results showed that Gaussian Process Regression outperforms the other types of surrogate models and can effectively estimate the aforementioned target variables.

Estimation of Coal's Sorption Parameters Using Artificial Neural Networks

This article presents research results into the application of an artificial neural network (ANN) to determine coal’s sorption parameters, such as the maximal sorption capacity and effective diffusion coefficient. Determining these parameters is currently time-consuming, and requires specialized and expensive equipment. The work was conducted with the use of feed-forward back-propagation networks (FNNs); it was aimed at estimating the values of the aforementioned parameters from information obtained through technical and densitometric analyses, as well as knowledge of the petrographic composition of the examined coal samples. Analyses showed significant compatibility between the values of the analyzed sorption parameters obtained with regressive neural models and the values of parameters determined with the gravimetric method using a sorption analyzer (prediction error for the best match was 6.1% and 0.2% for the effective diffusion coefficient and maximal sorption capacity, respectively). The established determination coefficients (0.982, 0.999) and the values of standard deviation ratios (below 0.1 in each case) confirmed very high prediction capacities of the adopted neural models. The research showed the great potential of the proposed method to describe the sorption properties of coal as a material that is a natural sorbent for methane and carbon dioxide.

An approximation algorithm for graph partitioning via deterministic annealing neural network

Graph partitioning, a classical NP-hard combinatorial optimization problem, is widely applied to industrial or management problems. In this study, an approximated solution of the graph partitioning problem is obtained by using a deterministic annealing neural network algorithm. The algorithm is a continuation method that attempts to obtain a high-quality solution by following a path of minimum points of a barrier problem as the barrier parameter is reduced from a sufficiently large positive number to 0. With the barrier parameter assumed to be any positive number, one minimum solution of the barrier problem can be found by the algorithm in a feasible descent direction. With a globally convergent iterative procedure, the feasible descent direction could be obtained by renewing Lagrange multipliers red. A distinctive feature of it is that the upper and lower bounds on the variables will be automatically satisfied on the condition that the step length is a value from 0 to 1. Four well-known algorithms are compared with the proposed one on 100 test samples. Simulation results show effectiveness of the proposed algorithm.

Rapid, Heuristic Discovery and Design of Promoter Collections in Non-Model Microbes for Industrial Applications

Well-characterized promoter collections for synthetic biology applications are not always available in industrially relevant hosts. We developed a broadly applicable method for promoter identification in atypical microbial hosts that requires no a priori understanding of cis-regulatory element structure. This novel approach combines bioinformatic filtering with rapid empirical characterization to expand the promoter toolkit and uses machine learning to improve the understanding of the relationship between DNA sequence and function. Here, we apply the method in Geobacillus thermoglucosidasius, a thermophilic organism with high potential as a synthetic biology chassis for industrial applications. Bioinformatic screening of G. kaustophilus, G. stearothermophilus, G. thermodenitrificans, and G. thermoglucosidasius resulted in the identification of 636 100 bp putative promoters, encompassing the genome-wide design space and lacking known transcription factor binding sites. Eighty of these sequences were characterized in vivo, and activities covered a 2-log range of predictable expression levels. Seven sequences were shown to function consistently regardless of the downstream coding sequence. Partition modeling identified sequence positions upstream of the canonical -35 and -10 consensus motifs that were predicted to strongly influence regulatory activity in Geobacillus, and artificial neural network and partial least squares regression models were derived to assess if there were a simple, forward, quantitative method for in silico prediction of promoter function. However, the models were insufficiently general to predict pre hoc promoter activity in vivo, most probably as a result of the relatively small size of the training data set compared to the size of the modeled design space.

An unsupervised parameter learning model for RVFL neural network

With the direct input-output connections, a random vector functional link (RVFL) network is a simple and effective learning algorithm for single-hidden layer feedforward neural networks (SLFNs). RVFL is a universal approximator for continuous functions on compact sets with fast learning property. Owing to its simplicity and effectiveness, RVFL has attracted significant interest in numerous real-world applications. In reality, the performance of RVFL is often challenged by randomly assigned network parameters. In this paper, we propose a novel unsupervised network parameter learning method for RVFL, named sparse pre-trained random vector functional link (SP-RVFL for short) network. The proposed SP-RVFL uses a sparse autoencoder with ℓ₁-norm regularization to adaptively learn superior network parameters for specific learning tasks. By doing so, the learned network parameters in SP-RVFL are embedded with the valuable information of input data, which alleviate the randomly generated parameter issue and improve the algorithmic performance. Experiments and comparisons on 16 diverse benchmarks from different domains confirm the effectiveness of the proposed SP-RVFL. The corresponding results also demonstrate that RVFL outperforms extreme learning machine (ELM).