A gated recurrent unit model based on ultrasound images of dynamic tongue movement for determining the severity of obstructive sleep apnea

Obstructive sleep apnea (OSA) presents as a respiratory disorder characterized by recurrent upper pharyngeal airway collapse during sleep. Dynamic tongue movement (DTM) analysis emerges as a promising avenue for elucidating the pathophysiological underpinnings of OSA, thereby facilitating its diagnosis. Recent endeavors have utilized artificial intelligence techniques to categorize OSA severity leveraging electrocardiography and blood oxygen saturation data. Nonetheless, the integration of ultrasound (US) imaging of the tongue remains largely untapped in the development of machine learning models aimed at determining the severity of OSA. This study endeavors to bridge this gap by capturing US images of DTM dynamics during wakefulness, encompassing transitions from normal breathing (NB) to the performance of the Müller maneuver (MM) in a cohort of 53 patients. Leveraging the modified optical flow method (MOFM), the trajectories of patients' DTM were tracked, facililtating the extraction of 27 parameters vital for model training. These parameters encompassed nine-point lateral movement, nine-point axial movement, and nine-point total displacement of the tongue, resulting in a dataset of 186,030 samples. The gated recurrent unit (GRU) method, renowned for its efficacy in motion tracking, was employed for model development in this study. Validation of the developed model was conducted via stratified k-fold cross-validation (SCV). The systems' overall performance in classifying OSA severity, as quantified by mean accuracy (MA), yielded a value of 43.49%. This pilot investigation marks an exploratory endeavor into the utilization of artificial intelligence for the classification of OSA severity based on US images and dynamic movement patterns. This novel model holds potential to assist clinicians in categorizing OSA severity and guiding the selection of pertinent treatment modalities tailored to the individual needs of patients afflicted with OSA.

Optimizing coagulant dosage using deep learning models with large-scale data

Water treatment plants are facing challenges that necessitate transition to automated processes using advanced technologies. This study introduces a novel approach to optimize coagulant dosage in water treatment processes by employing a deep learning model. The study utilized minute-by-minute data monitored in real time over a span of five years, marking the first attempt in drinking water process modeling to leverage such a comprehensive dataset. The deep learning model integrates a one-dimensional convolutional neural network (Conv1D) and gated recurrent unit (GRU) to effectively extract features and model complex time-series data. Initially, the model predicted coagulant dosage and sedimentation basin turbidity, validated against a physicochemical model. Subsequently, the model optimized coagulant dosage in two ways: 1) maintaining sedimentation basin turbidity below the 1.0 NTU guideline, and 2) analyzing changes in sedimentation basin turbidity resulting from reduced coagulant dosage (5-20%). The findings of the study highlight the effectiveness of the deep learning model in optimizing coagulant dosage with substantial reductions in coagulant dosage (approximately 22% reduction and 21 million KRW/year). The results demonstrate the potential of deep learning models in enhancing the efficiency and cost-effectiveness of water treatment processes, ultimately facilitating process automation.

An emotion recognition method based on EWT-3D-CNN-BiLSTM-GRU-AT model

This has become a significant study area in recent years because of its use in brain-machine interaction (BMI). The robustness problem of emotion classification is one of the most basic approaches for improving the quality of emotion recognition systems. One of the two main branches of these approaches deals with the problem by extracting the features using manual engineering and the other is the famous artificial intelligence approach, which infers features of EEG data. This study proposes a novel method that considers the characteristic behavior of EEG recordings and based on the artificial intelligence method. The EEG signal is a noisy signal with a non-stationary and non-linear form. Using the Empirical Wavelet Transform (EWT) signal decomposition method, the signal's frequency components are obtained. Then, frequency-based features, linear and non-linear features are extracted. The resulting frequency-based, linear, and nonlinear features are mapped to the 2-D axis according to the positions of the EEG electrodes. By merging this 2-D images, 3-D images are constructed. In this way, the multichannel brain frequency of EEG recordings, spatial and temporal relationship are combined. Lastly, 3-D deep learning framework was constructed, which was combined with convolutional neural network (CNN), bidirectional long-short term memory (BiLSTM) and gated recurrent unit (GRU) with self-attention (AT). This model is named EWT-3D-CNN-BiLSTM-GRU-AT. As a result, we have created framework comprising handcrafted features generated and cascaded from state-of-the-art deep learning models. The framework is evaluated on the DEAP recordings based on the person-independent approach. The experimental findings demonstrate that the developed model can achieve classification accuracies of 90.57 % and 90.59 % for valence and arousal axes, respectively, for the DEAP database. Compared with existing cutting-edge emotion classification models, the proposed framework exhibits superior results for classifying human emotions.

Prediction of the number of asthma patients using environmental factors based on deep learning algorithms

BACKGROUND

Air pollution, weather, pollen, and influenza are typical aggravating factors for asthma. Previous studies have identified risk factors using regression-based and ensemble models. However, studies that consider complex relationships and interactions among these factors have yet to be conducted. Although deep learning algorithms can address this problem, further research on modeling and interpreting the results is warranted.

METHODS

In this study, from 2015 to 2019, information about air pollutants, weather conditions, pollen, and influenza were utilized to predict the number of emergency room patients and outpatients with asthma using recurrent neural network, long short-term memory (LSTM), and gated recurrent unit models. The relative importance of the environmental factors in asthma exacerbation was quantified through a feature importance analysis.

RESULTS

We found that LSTM was the best algorithm for modeling patients with asthma. Our results demonstrated that influenza, temperature, PM10, NO2, CO, and pollen had a significant impact on asthma exacerbation. In addition, the week of the year and the number of holidays per week were an important factor to model the seasonality of the number of asthma patients and the effect of holiday clinic closures, respectively.

CONCLUSION

LSTM is an excellent algorithm for modeling complex epidemiological relationships, encompassing nonlinearity, lagged responses, and interactions. Our study findings can guide policymakers in their efforts to understand the environmental factors of asthma exacerbation.

Monthly runoff prediction based on variational modal decomposition combined with the dung beetle optimization algorithm for gated recurrent unit model

Highly accurate monthly runoff forecasts play a pivotal role in water resource management and utilization. This article proposes a coupling of variational modal decomposition (VMD) and the dung beetle optimization algorithm (DBO) with the gated recurrent unit (GRU) to establish a new monthly runoff forecasting model: the VMD-DBO-GRU. Initially, historical runoff data are decomposed via VMD. Subsequently, the parameters of the GRU are optimized using the DBO, and the decomposed monthly runoff components are inputted into the GRU neural network. Finally, the predictions for each component are consolidated to provide monthly runoff predictions. The model is then validated using monthly runoff data from the Ansha reservoir in Fujian, collected from 1980 to 2020. The results demonstrate a higher prediction accuracy of the VMD-DBO-GRU model compared to BP, SVM, GRU, VMD-GRU, DBO-GRU, and EMD-GRU models, providing a new alternative for conducting monthly runoff prediction.

A multi-resolution physics-informed recurrent neural network: formulation and application to musculoskeletal systems

This work presents a multi-resolution physics-informed recurrent neural network (MR PI-RNN), for simultaneous prediction of musculoskeletal (MSK) motion and parameter identification of the MSK systems. The MSK application was selected as the model problem due to its challenging nature in mapping the high-frequency surface electromyography (sEMG) signals to the low-frequency body joint motion controlled by the MSK and muscle contraction dynamics. The proposed method utilizes the fast wavelet transform to decompose the mixed frequency input sEMG and output joint motion signals into nested multi-resolution signals. The prediction model is subsequently trained on coarser-scale input-output signals using a gated recurrent unit (GRU), and then the trained parameters are transferred to the next level of training with finer-scale signals. These training processes are repeated recursively under a transfer-learning fashion until the full-scale training (i.e., with unfiltered signals) is achieved, while satisfying the underlying dynamic equilibrium. Numerical examples on recorded subject data demonstrate the effectiveness of the proposed framework in generating a physics-informed forward-dynamics surrogate, which yields higher accuracy in motion predictions of elbow flexion-extension of an MSK system compared to the case with single-scale training. The framework is also capable of identifying muscle parameters that are physiologically consistent with the subject's kinematics data.

A hybrid model of modal decomposition and gated recurrent units for short-term load forecasting

Electrical load forecasting is important to ensuring power systems are operated both economically and safely. However, accurately forecasting load is difficult because of variability and frequency aliasing. To eliminate frequency aliasing, some methods set parameters that depend on experiences. The present study proposes an adaptive hybrid model of modal decomposition and gated recurrent units (GRU) to reduce frequency aliasing and series randomness. This model uses average sample entropy and mutual correlation to jointly determine the modal number in the decomposition. Random adjustment parameters were introduced to the Adam algorithm to improve training speed. To assess the applicability and accuracy of the proposed hybrid model, it was compared with some state of the art forecasting methods. The results, which were validated by actual data sets from Shaanxi province, China, show that the proposed model had a higher accuracy and better reliability compared to the other forecasting methods.

Motion prediction using brain waves based on artificial intelligence deep learning recurrent neural network

Electroencephalogram (EEG) research has gained widespread use in various research domains due to its valuable insights into human body movements. In this study, we investigated the optimization of motion discrimination prediction by employing an artificial intelligence deep learning recurrent neural network (gated recurrent unit, GRU) on unique EEG data generated from specific movement types among EEG signals. The experiment involved participants categorized into five difficulty levels of postural control, targeting gymnasts in their twenties and college students majoring in physical education (n=10). Machine learning techniques were applied to extract brain-motor patterns from the collected EEG data, which consisted of 32 channels. The EEG data underwent spectrum analysis using fast Fourier transform conversion, and the GRU model network was utilized for machine learning on each EEG frequency domain, thereby improving the performance index of the learning operation process. Through the development of the GRU network algorithm, the performance index achieved up to a 15.92% improvement compared to the accuracy of existing models, resulting in motion recognition accuracy ranging from a minimum of 94.67% to a maximum of 99.15% between actual and predicted values. These optimization outcomes are attributed to the enhanced accuracy and cost function of the GRU network algorithm's hidden layers. By implementing motion identification optimization based on artificial intelligence machine learning results from EEG signals, this study contributes to the emerging field of exercise rehabilitation, presenting an innovative paradigm that reveals the interconnectedness between the brain and the science of exercise.

Application of deep learning approaches to predict monthly stream flows

Accurate and reliable flow estimations are of great importance for hydroelectric power generation, flood and drought risk management, and the effective use of water resources. This research carries out a comprehensive study on the application of gated recurrent unit (GRU) neural network, recurrent neural network (RNN), and long short-term memory (LSTM) to predict with river flows at three different streamflow observation stations in Erzincan, Bayburt, and Gümüshane. Monthly streamflow time series covering the years 1978 to 2015 were used to set up artificial intelligence models. During the modeling phase, 70% of the data was divided into training (October 1978-April 2004), 15% validation (May 2004-September 2009), and 15% test set (October 2010-September 2015). Model performances were made according to the correlation coefficient, root mean square error, the ratio of RMSE to the standard deviation, Nash-Sutcliffe efficiency coefficient, index of agreement, and volumetric efficiency values. The calculation results show that GRU leads efficient estimation results for estimating streamflow and can also be used in allied water resources.

Improved prediction of chlorophyll-a concentrations in reservoirs by GRU neural network based on particle swarm algorithm optimized variational modal decomposition

The accurate and reliable prediction of chlorophyll-a (Chl-a) concentration is of great significance in reservoir environment management and pollution control. To improve the accuracy of Chl-a index prediction, a novel hybrid water quality prediction method was proposed for gated recurrent unit (GRU) neural network based on particle swarm algorithm optimized variational modal decomposition (PV-GRU). The results showed that the variational mode decomposition (VMD) optimized by particle swarm optimization (PSO) in this study effectively reduced the non-smooth of water quality data. In addition, the GRU neural network reduced the risk of overfitting the deep-learning model with small sample data. Overall, the PV-GRU prediction model exhibited significant superiority in predicting non-smooth and non-linear Chl-a sequences with a relatively small sample size. The prediction errors of PV-GRU model were all less than those of other comparative models, and the fitting determination coefficient R² was 94.21%. These results indicated that the proposed PV-GRU model can effectively predict the content of Chl-a in reservoirs, which provides an alternative new method for water quality prediction to prevent and control eutrophication in reservoirs.

A gated temporal-separable attention network for EEG-based depression recognition

Depression, a common mental illness worldwide, needs to be diagnosed and cured at an early stage. To assist clinical diagnosis, an EEG-based deep learning frame, which is named the gated temporal-separable attention network (GTSAN), is proposed in this paper for depression recognition. GTSAN model extracts discriminative information from EEG recordings in two ways. On the one hand, the gated recurrent unit (GRU) is used in the GTSAN model to capture the EEG historical information to form the features. On the other hand, the model digs the multilevel information by using an improved version of temporal convolutional network (TCN), called temporal-separable convolution network (TSCN), which applies causal convolution and dilated convolution to extract features from fine to coarse scales. The TSCN and GRU features can be produced in parallel. Finally, the new model introduces the attention mechanism to give different weights to these features, allowing them to be used to identify depression more effectively. Experiments on two depression datasets have demonstrated that the proposed model can mine potential depression patterns in data and obtain high recognition accuracies. The proposed model provides the possibility of using an EEG-based system to assist for diagnosing depression.

INSnet: a method for detecting insertions based on deep learning network

BACKGROUND

Many studies have shown that structural variations (SVs) strongly impact human disease. As a common type of SV, insertions are usually associated with genetic diseases. Therefore, accurately detecting insertions is of great significance. Although many methods for detecting insertions have been proposed, these methods often generate some errors and miss some variants. Hence, accurately detecting insertions remains a challenging task.

RESULTS

In this paper, we propose a method named INSnet to detect insertions using a deep learning network. First, INSnet divides the reference genome into continuous sub-regions and takes five features for each locus through alignments between long reads and the reference genome. Next, INSnet uses a depthwise separable convolutional network. The convolution operation extracts informative features through spatial information and channel information. INSnet uses two attention mechanisms, the convolutional block attention module (CBAM) and efficient channel attention (ECA) to extract key alignment features in each sub-region. In order to capture the relationship between adjacent subregions, INSnet uses a gated recurrent unit (GRU) network to further extract more important SV signatures. After predicting whether a sub-region contains an insertion through the previous steps, INSnet determines the precise site and length of the insertion. The source code is available from GitHub at https://github.com/eioyuou/INSnet .

CONCLUSION

Experimental results show that INSnet can achieve better performance than other methods in terms of F1 score on real datasets.

Proposed algorithm for smart grid DDoS detection based on deep learning

The Smart Grid's objective is to increase the electric grid's dependability, security, and efficiency through extensive digital information and control technology deployment. As a result, it is necessary to apply real-time analysis and state estimation-based techniques to ensure efficient controls are implemented correctly. These systems are vulnerable to cyber-attacks, posing significant risks to the Smart Grid's overall availability due to their reliance on communication technology. Therefore, effective intrusion detection algorithms are required to mitigate such attacks. In dealing with these uncertainties, we propose a hybrid deep learning algorithm that focuses on Distributed Denial of Service attacks on the communication infrastructure of the Smart Grid. The proposed algorithm is hybridized by the Convolutional Neural Network and the Gated Recurrent Unit algorithms. Simulations are done using a benchmark cyber security dataset of the Canadian Institute of Cybersecurity Intrusion Detection System. According to the simulation results, the proposed algorithm outperforms the current intrusion detection algorithms, with an overall accuracy rate of 99.7%.

Air Quality Index prediction using an effective hybrid deep learning model

Environmentalism has become an intrinsic part of everyday life. One of the greatest challenge to the environment's long-term existence is the air pollution. Delhi, the capital of India, has experienced decreasing of air quality for several years. The poor air quality has a significant impact on the lives of individuals. Air Quality Index (AQI) prediction can help to its beneficiaries in taking safeguards about their health before moving to any polluted area. In this study, a variety of data forecasting approaches is evaluated to predict the AQI value for Particulate Matter (PM_2.5) μm at a particular area of Delhi and several error-prone strategies such as R-Squared (R²), Mean Absolute Error (MAE), and Root Mean Square Error (RMSE) methods are catalogued. In the proposed approach two deep learning models like Long-Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) are combined to predict the AQI of the environment. Several stand alone machine learning (ML) and deep learning (DL) models such as LSTM, Linear-Regression (LR), GRU, K-Nearest Neighbor (KNN) and Support Vector Machine (SVM) are also trained on the same dataset to compare their performances with the proposed hybrid (LSTM-GRU) model and it is found that the proposed hybrid model shows supremacy in the performance with the MAE value 36.11 and R² value 0.84.

A novel hybrid wind speed interval prediction model based on mode decomposition and gated recursive neural network

Wind energy has become one of the most efficient renewable energy sources. However, the wind has the characteristics of intermittence and uncontrollability, so it is challenging to predict wind speed accurately. Considering the shortcomings of traditional wind power point predictions, a new hybrid model comprised three main modules used for data preprocessing, deterministic point prediction, and interval prediction is proposed to predict the wind speed interval. The first module, the data preprocessing module, uses variational mode decomposition (VMD), sample entropy (SE), and singular spectrum analysis (SSA) to extract the different frequency components of the initial wind speed series. The second module, the deterministic point prediction module, uses extreme learning machines (ELM), and a gated recursive unit (GRU) model to perform point prediction on the wind speed series. The third module, the interval prediction module, uses the nonparametric kernel density estimation method to construct the upper and lower bounds of the wind speed interval. In addition, the final wind speed prediction interval is obtained by integrating the prediction results of multiple interval prediction results to improve the robustness and generalization of the wind speed interval prediction. Finally, the effectiveness of the prediction performance of the proposed hybrid model is verified based on the data of two actual wind farms. The experimental results show that the proposed hybrid model can obtain the appropriate wind speed interval with high confidence and quality with different confidence levels of 95%, 90%, and 85%.

Accurate Prediction of Anti-hypertensive Peptides Based on Convolutional Neural Network and Gated Recurrent unit

Hypertension (HT) is a general disease, and also one of the most ordinary and major causes of cardiovascular disease. Some diseases are caused by high blood pressure, including impairment of heart and kidney function, cerebral hemorrhage and myocardial infarction. Due to the limitations of laboratory methods, bioactive peptides for the treatment of HT need a long time to be identified. Therefore, it is of great immediate significance for the identification of anti-hypertensive peptides (AHTPs). With the prevalence of machine learning, it is suggested to use it as a supplementary method for AHTPs classification. Therefore, we develop a new model to identify AHTPs based on multiple features and deep learning. And the deep model is constructed by combining a convolutional neural network (CNN) and a gated recurrent unit (GRU). The unique convolution structure is used to reduce the feature dimension and running time. The data processed by CNN is input into the recurrent structure GRU, and important information is filtered out through the reset gate and update gate. Finally, the output layer adopts Sigmoid activation function. Firstly, we use Kmer, the deviation between the dipeptide frequency and the expected mean (DDE), encoding based on grouped weight (EBGW), enhanced grouped amino acid composition (EGAAC) and dipeptide binary profile and frequency (DBPF) to extract features. For Kmer, DDE, EBGW and EGAAC, it is widely used in the field of protein research. DBPF is a new feature representation method designed by us. It corresponds dipeptides to binary numbers, and finally obtains a binary coding file and a frequency file. Then these features are spliced together and input into our proposed model for prediction and analysis. After a tenfold cross-validation test, this model has a better competitive advantage than the previous methods, and the accuracy is 96.23% and 99.10%, respectively. From the results, compared with the previous methods, it has been greatly improved. It shows that the combination of convolution calculation and recurrent structure has a positive impact on the classification of AHTPs. The results show that this method is a feasible, efficient and competitive sequence analysis tool for AHTPs. Meanwhile, we design a friendly online prediction tool and it is freely accessible at http://ahtps.zhanglab.site/ .

Real-time driving risk assessment using deep learning with XGBoost

Traffic crashes typically occur in a few seconds and real-time prediction can significantly benefit traffic safety management and the development of safety countermeasures. This paper presents a novel deep learning model for crash identification based on high-frequency, high-resolution continuous driving data. The method consists of feature engineering based on Convolutional Neural Network (CNN) and Gated Recurrent Unit (GRU) and classification based on Extreme Gradient Boosting (XGBoost). The CNN-GRU architecture captures the time series characteristics of driving kinematics data. Compared to normal driving segments, safety-critical events (SCEs)-i.e., crashes and near-crashes (CNC)-are rare. The weighted categorical cross-entropy loss and oversampling methods are utilized to address this imbalance issue. An XGBoost classifier is utilized instead of the multi-layer perceptron (MLP) to achieve a high precision and recall rate. The proposed approach is applied to the Second Strategic Highway Research Program Naturalistic Driving Study (SHRP 2 NDS) data with 1,820 crashes, 6,848 near-crashes, and 59,997 normal driving segments. The results show that in a 3-class classification system (crash, near-crash, normal driving segments), the accuracy for the overall model is 97.5%, and the precision and recall for crashes are 84.7%, and 71.3% respectively, which is substantially better than benchmarks models. Furthermore, the recall of the most severe crashes is 98.0%. The proposed crash identification approach provides an accurate, highly efficient, and scalable way to identify crashes based on high frequency, high-resolution continuous driving data and has broad application prospects in traffic safety applications.

Anomalous Network Traffic Detection Method Based on an Elevated Harris Hawks Optimization Method and Gated Recurrent Unit Classifier

In recent years, network traffic contains a lot of feature information. If there are too many redundant features, the computational cost of the algorithm will be greatly increased. This paper proposes an anomalous network traffic detection method based on Elevated Harris Hawks optimization. This method is easier to identify redundant features in anomalous network traffic, reduces computational overhead, and improves the performance of anomalous traffic detection methods. By enhancing the random jump distance function, escape energy function, and designing a unique fitness function, there is a unique anomalous traffic detection method built using the algorithm and the neural network for anomalous traffic detection. This method is tested on three public network traffic datasets, namely the UNSW-NB15, NSL-KDD, and CICIDS2018. The experimental results show that the proposed method does not only significantly reduce the number of features in the dataset and computational overhead, but also gives better indicators for every test.

CMIC: predicting DNA methylation inheritance of CpG islands with embedding vectors of variable-length k-mers

Background

Epigenetic modifications established in mammalian gametes are largely reprogrammed during early development, however, are partly inherited by the embryo to support its development. In this study, we examine CpG island (CGI) sequences to predict whether a mouse blastocyst CGI inherits oocyte-derived DNA methylation from the maternal genome. Recurrent neural networks (RNNs), including that based on gated recurrent units (GRUs), have recently been employed for variable-length inputs in classification and regression analyses. One advantage of this strategy is the ability of RNNs to automatically learn latent features embedded in inputs by learning their model parameters. However, the available CGI dataset applied for the prediction of oocyte-derived DNA methylation inheritance are not large enough to train the neural networks.

Results

We propose a GRU-based model called CMIC (CGI Methylation Inheritance Classifier) to augment CGI sequence by converting it into variable-length k-mers, where the length k is randomly selected from the range \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$k_{\min }$$\end{document}kmin to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$k_{\max }$$\end{document}kmax, N times, which were then used as neural network input. N was set to 1000 in the default setting. In addition, we proposed a new embedding vector generator for k-mers called splitDNA2vec. The randomness of this procedure was higher than the previous work, dna2vec.

Conclusions

We found that CMIC can predict the inheritance of oocyte-derived DNA methylation at CGIs in the maternal genome of blastocysts with a high F-measure (0.93). We also show that the F-measure can be improved by increasing the parameter N, that is, the number of sequences of variable-length k-mers derived from a single CGI sequence. This implies the effectiveness of augmenting input data by converting a DNA sequence to N sequences of variable-length k-mers. This approach can be applied to different DNA sequence classification and regression analyses, particularly those involving a small amount of data.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12859-022-04916-3.

An integrated 3D CNN-GRU deep learning method for short-term prediction of PM2.5 concentration in urban environment

This study proposes a new model for the spatiotemporal prediction of PM_2.5 concentration at hourly and daily time intervals. It has been constructed on a combination of three-dimensional convolutional neural network and gated recurrent unit (3D CNN-GRU). The performance of the proposed model is boosted by learning spatial patterns from similar air quality (AQ) stations while maintaining long-term temporal dependencies with simultaneous learning and prediction for all stations over different time intervals. 3D CNN-GRU model was applied to air pollution observations, especially PM_2.5 level, collected from several AQ stations across the city of Tehran, the capital of Iran, from 2016 to 2019. It could achieve promising results compared to the methods such as LSTM, GRU, ANN, SVR, and ARIMA, which are recently introduced in the literature; it estimates 84% (R² = 0.84) and 78% (R² = 0.78) of PM_2.5 concentration variations for the next hour and the following day, respectively.

Adaptive windowing based recurrent neural network for drift adaption in non-stationary environment

In today's digital era, many applications generate massive data streams that must be sequenced and processed immediately. Therefore, storing large amounts of data for analysis is impractical. Now, this infinite amount of evolving data confronts concept drifts in data stream classification. Concept drift is a phenomenon in which the distribution of input data or the relationship between input data and target label changes over time. If the drifts are not addressed, the learning model's performance suffers. Non-stationary data streams must be processed as they arrive, and neural networks' built-in capabilities aid in the processing of huge non-stationary data streams. We proposed an adaptive windowing approach based on a gated recurrent unit, a variant of the recurrent neural network incrementally trained on incoming data (for the real-world airline and synthetic Streaming Ensemble Algorithm (SEA) datasets), and employed elastic weight consolidation with the Fisher information matrix to prevent forgetting. Unlike the traditional fixed window methodology, the proposed model dynamically increases the window size if the prediction is correct and reduces it if drifts occur. As a result, an adaptive recurrent neural network model can adapt to changes in the non-stationary data stream and provide consistent performance. Moreover, the findings revealed that on the airline and the SEA dataset, the proposed model outperforms state-of-the-art methods by achieving 67.74% and 91.70% accuracy, respectively. Further, the results demonstrated that the proposed model has a better accuracy of 3.6% and 1.6% for the SEA and the airline dataset, respectively.

A novel combined model for prediction of daily precipitation data using instantaneous frequency feature and bidirectional long short time memory networks

Meteorological events constantly affect human life, especially the occurrence of excessive precipitation in a short time causes important events such as floods. However, in case of insufficient precipitation for a long time, drought occurs. In recent years, significant changes in precipitation regimes have been observed and these changes cause socio-economic and ecological problems. Therefore, it is of great importance to correctly predict and analyze the precipitation data. In this study, a reliable and accurate precipitation forecasting model is proposed. For this aim, three deep neural network models, long short-time memory networks (LSTM), gated recurrent unit (GRU), and bidirectional long short time memory networks (biLSTM), were applied for one ahead forecasting of daily precipitation data and compared the performances of these models. Moreover, to increase the far ahead forecasting performance of the biLSTM model, the instantaneous frequency (IF) feature was applied as the input parameter for the first time in the literature. Therefore, a novel model ensemble of IF and biLSTM was employed for the aim of one-six ahead forecasting of daily precipitation data. The performance of the proposed IF-biLSTM model was evaluated using mean absolute error (MAE), mean square error (MSE), correlation coefficient (R), and determination coefficient (R²) performance parameter and spider charts were used to assess the model performances. According to the numerical results, the biLSTM model outperformed compared with the LSTM and GRU models. After the good score achieved with biLSTM model, IF feature applied to biLSTM and IF-biLSTM model has the best forecasting performance for daily precipitation data with R² value 0.9983, 0.9827, 0.9092, 0.8508, 0.7827, and 0.7646, respectively, for one-six ahead forecasting of daily precipitation data. It has been observed that the IF-biLSTM model has higher forecasting performance than the biLSTM model, especially in far ahead forecasting studies, and the IF feature improves the estimation performance.

Water quality prediction in sea cucumber farming based on a GRU neural network optimized by an improved whale optimization algorithm

Sea cucumber farming is an important part of China's aquaculture industry, and sea cucumbers have higher requirements for aquaculture water quality. This article proposes a sea cucumber aquaculture water quality prediction model that uses an improved whale optimization algorithm to optimize the gated recurrent unit neural network(IWOA-GRU), which provides a reference for the water quality control in the sea cucumber growth environment. This model first applies variational mode decomposition (VMD) and the wavelet threshold joint denoising method to remove mixed noise in water quality time series. Then, by optimizing the convergence factor, the convergence speed and global optimization ability of the whale optimization algorithm are strengthened. Finally, the improved whale optimization algorithm is used to construct a GRU prediction model based on optimal network weights and thresholds to predict sea cucumber farming water quality. The model was trained and tested using three water quality indices (dissolved oxygen, temperature and salinity) of sea cucumber culture waters in Shandong Peninsula, China, and compared with prediction models such as support vector regression (SVR), random forest (RF), convolutional neural network (CNN), recurrent neural network (RNN), and long short-term memory neural network (LSTM). Experimental results show that the prediction accuracy and generalization performance of this model are better than those of the other compared models.

[Research on gait recognition and prediction based on optimized machine learning algorithm]

Aiming at the problems of individual differences in the asynchrony process of human lower limbs and random changes in stride during walking, this paper proposes a method for gait recognition and prediction using motion posture signals. The research adopts an optimized gated recurrent unit (GRU) network algorithm based on immune particle swarm optimization (IPSO) to establish a network model that takes human body posture change data as the input, and the posture change data and accuracy of the next stage as the output, to realize the prediction of human body posture changes. This paper first clearly outlines the process of IPSO's optimization of the GRU algorithm. It collects human body posture change data of multiple subjects performing flat-land walking, squatting, and sitting leg flexion and extension movements. Then, through comparative analysis of IPSO optimized recurrent neural network (RNN), long short-term memory (LSTM) network, GRU network classification and prediction, the effectiveness of the built model is verified. The test results show that the optimized algorithm can better predict the changes in human posture. Among them, the root mean square error (RMSE) of flat-land walking and squatting can reach the accuracy of 10 ^-3, and the RMSE of sitting leg flexion and extension can reach the accuracy of 10 ^-2. The R ² value of various actions can reach above 0.966. The above research results show that the optimized algorithm can be applied to realize human gait movement evaluation and gait trend prediction in rehabilitation treatment, as well as in the design of artificial limbs and lower limb rehabilitation equipment, which provide a reference for future research to improve patients' limb function, activity level, and life independence ability.

[A pelvic support weight rehabilitation system tracing the human center of mass height]

The body weight support rehabilitation training system has now become an important treatment method for the rehabilitation of lower limb motor dysfunction. In this paper, a pelvic brace body weight support rehabilitation system is proposed, which follows the center of mass height (CoMH) of the human body. It aims to address the problems that the existing pelvic brace body weight support rehabilitation system with constant impedance provides a fixed motion trajectory for the pelvic mechanism during the rehabilitation training and that the patients have low participation in rehabilitation training. The system collectes human lower limb motion information through inertial measurement unit and predicts CoMH through artificial neural network to realize the tracking control of pelvic brace height. The proposed CoMH model was tested through rehabilitation training of hemiplegic patients. The results showed that the range of motion of the hip and knee joints on the affected side of the patient was improved by 25.0% and 31.4%, respectively, and the ratio of swing phase to support phase on the affected side was closer to that of the gait phase on the healthy side, as opposed to the traditional body weight support rehabilitation training model with fixed motion trajectory of pelvic brace. The motion trajectory of the pelvic brace in CoMH mode depends on the current state of the trainer so as to realize the walking training guided by active movement on the healthy side of hemiplegia patients. The strategy of dynamically adjustment of body weight support is more helpful to improve the efficiency of walking rehabilitation training.

Utilizing deep learning models in CSI-based human activity recognition

In recent years, channel state information (CSI) in WiFi 802.11n has been increasingly used to collect data pertaining to human activity. Such raw data are then used to enhance human activity recognition. Activities such as lying down, falling, walking, running, sitting down, and standing up can now be detected with the use of information collected through CSI. Human activity recognition has a multitude of applications, such as home monitoring of patients. Four deep learning models are presented in this paper, namely: a convolution neural network (CNN) with a Gated Recurrent Unit (GRU); a CNN with a GRU and attention; a CNN with a GRU and a second CNN, and a CNN with Long Short-Term Memory (LSTM) and a second CNN. Those models were trained to perform Human Activity Recognition (HAR) using CSI amplitude data collected by a CSI tool. Experiments conducted to test the efficacy of these models showed superior results compared with other recent approaches. This enhanced performance of our models may be attributable the ability of our models to make full use of available data and to extract all data features, including high dimensionality and time sequence. The highest average recognition accuracy reached by the proposed models was achieved by the CNN-GRU, and the CNN-GRU with attention models, standing at 99.31% and 99.16%, respectively. In addition, the performance of the models was evaluated for unseen CSI data by training our models using a random split-of-dataset method (70% training and 30% testing). Our models achieved impressive results with accuracies reaching 100% for nearly all activities. For the lying down activity, accuracy obtained from the CNN-GRU model stood at 99.46%; slightly higher than the 99.05% achieved by the CNN-GRU with attention model. This confirmed the robustness of our models against environmental changes.

An ensemble method to forecast 24-h ahead solar irradiance using wavelet decomposition and BiLSTM deep learning network

In recent years, the penetration of solar power at residential and utility levels has progressed exponentially. However, due to its stochastic nature, the prediction of solar global horizontal irradiance (GHI) with higher accuracy is a challenging task; but, vital for grid management: planning, scheduling & balancing. Therefore, this paper proposes an ensemble model using the extended scope of wavelet transform (WT) and bidirectional long short term memory (BiLSTM) deep learning network to forecast 24-h ahead solar GHI. The WT decomposes the input time series data into different finite intrinsic model functions (IMF) to extract the statistical features of input time series. Further, the study reduces the number of IMF series by combining the wavelet decomposed components (D1-D6) series on the basis of comprehensive experimental analysis with an aim to improve the forecasting accuracy. Next, the trained standalone BiLSTM networks are allocated to each IMF sub-series to execute the forecasting. Finally, the forecasted values of each sub-series from BiLSTM networks are reconstructed to deliver the final solar GHI forecast. The study performed monthly solar GHI forecasting for one year dataset using one month moving window mechanism for the location of Ahmedabad, Gujarat, India. For the performance comparison, the naïve predictor as a benchmark model, standalone long short term memory (LSTM), gated recurrent unit (GRU), BiLSTM and two other wavelet-based BiLSTM models are also simulated. From the results, it is observed that the proposed model outperforms other models in terms of root mean square error (RMSE) & mean absolute percentage error (MAPE), coefficient of determination (R²) and forecast skill (FS). The proposed model reduces the monthly average RMSE by range from 26.04-58.89%, 5.17-31.35%, 23.26-56.06% & 21.08-57% in comparison with benchmark, standalone BiLSTM, GRU & LSTM networks respectively. On the other hand, the monthly average MAPE is reduced by range from 9 to 51.18%, 12.59-28.14%, 30.43-59.19% & 26.54-58.92% in comparison to benchmark, standalone BiLSTM, GRU & LSTM respectively. Further, the proposed model obtained the value of R² equal to 0.94 and forecast skill (%) of 47% with reference to the benchmark model.

A simple pan-specific RNN model for predicting HLA-II binding peptides

The prediction of human leukocyte antigen (HLA) class II binding peptides plays important roles in understanding the mechanism of immune recognition and developing effective epitope-based vaccines. In this work, gated recurrent unit (GRU)-based recurrent neural network (RNN) was successfully employed to establish a pan-specific prediction model of HLA-II-binding peptides by using only the HLA and peptide sequence information. In comparison with the existing pan-specific models of HLA-II-binding peptides, the GRU-based RNN model covered a broad spectrum of HLA-II molecules including 50 HLA-DR, 47 HLA-DQ, and 19 HLA-DP molecules with peptide lengths varying from 8 to 43 mers. The results demonstrated strong discriminant capabilities of the GRU-based RNN model, of which the AUC values were 0.92, 0.88, and 0.88 for the training, validation, and test sets, respectively. Also, the GRU-based model showed state-of-the-art performances in predicting the binding peptides with the length ranging from 8-32 mers, which provides an efficient method for predicting HLA-II-binding peptides of longer lengths in comparison with the available methods. Overall, taking the advantages of the RNN architecture, the established pan-specific GRU model can be used for predicting accurately the HLA-II-binding peptides in a simple and direct manner.

Automated detection of premature ventricular contraction based on the improved gated recurrent unit network

BACKGROUND AND OBJECTIVE

Premature ventricular contraction (PVC) is the common arrhythmia disease, affecting thousands of individuals worldwide. However, the traditional PVC detection is cumbersome by visually inspecting electrocardiogram (ECG) signals.

METHODS

In this work, we specially propose an improved gated recurrent unit (IGRU) by setting a scale parameter into existing bidirectional GRU (BGRU) model for PVC signals recognition, which is used to alleviate the problem of information redundancy in BGRU. To verify the effectiveness, IGRU model will be embedded into a convolutional network frame and existing GRU and BGRU models are employed as control groups for a fair comparison.

RESULTS

The results exhibit that the model attains better model performance than control groups and several state-of-the-art algorithms with the accuracy of 98.3% and 97.9% with the MIT-BIH arrhythmia database and China Physiological Signal Challenge 2018 database. Besides, motivated from the waveform characteristics of ECG signals in PVC, the proposed model can provide certain physiological interpretability for physicians and researchers.

CONCLUSIONS

To our knowledge, this is the first attempt to re-design the existing GRU network for ECG signals classification, thus exhibiting great application potentials especially in lightweight equipment such as mobile phone and camera.

Estimating gaseous pollutants from bus emissions: A hybrid model based on GRU and XGBoost

In urban areas, traffic-related contamination is one of the main contributors to environmental deterioration, and the pollution from public transit buses is a major component. To mitigate these impacts, it is essential to estimate bus emissions and analyze their characteristics. This paper proposes a hybrid model based on gated recurrent unit (GRU) and extreme gradient boosting (XGBoost), termed GRU-XGB, to predict gaseous pollutants from bus emissions (CO, CO₂, HC, NO_X) under real conditions. On-road experimental data collected from CNG-fueled and diesel-powered buses in Zhenjiang was used as a case study to verify the model's effectiveness. A comparison between the proposed and other state-of-the-art models reveals that GRU-XGB performs best for all evaluation metrics on both microscopic and aggregative levels, with an average correlation coefficient above 0.98 and an average MAPE lower than 9%. Moreover, the results of estimation errors analysis suggest that the real conditions of bus stations are more complicated than those of intersections and road sections. In most cases, however, the emission factors produced from intersections are proven to be the highest. Furthermore, operating patterns are shown to be the most significant factors, with relative importance equal to 45.09% and 71.68% for CNG and diesel buses, respectively. Besides, the results also indicate that humidity has little impact on this issue, while the influence of temperature is obvious, with relative importance equal to 17.56% and 9.41% for CNG and diesel buses, separately. Such findings can provide theoretical guidance for both emission estimation and environmental protection. Also, it is applicable for the management of accurate monitoring from an urban-level and can be integrated into emission simulation tools.

A comparative analysis on question classification task based on deep learning approaches

Question classification is one of the essential tasks for automatic question answering implementation in natural language processing (NLP). Recently, there have been several text-mining issues such as text classification, document categorization, web mining, sentiment analysis, and spam filtering that have been successfully achieved by deep learning approaches. In this study, we illustrated and investigated our work on certain deep learning approaches for question classification tasks in an extremely inflected Turkish language. In this study, we trained and tested the deep learning architectures on the questions dataset in Turkish. In addition to this, we used three main deep learning approaches (Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN)) and we also applied two different deep learning combinations of CNN-GRU and CNN-LSTM architectures. Furthermore, we applied the Word2vec technique with both skip-gram and CBOW methods for word embedding with various vector sizes on a large corpus composed of user questions. By comparing analysis, we conducted an experiment on deep learning architectures based on test and 10-cross fold validation accuracy. Experiment results were obtained to illustrate the effectiveness of various Word2vec techniques that have a considerable impact on the accuracy rate using different deep learning approaches. We attained an accuracy of 93.7% by using these techniques on the question dataset.

Impact of climate change on snowmelt runoff in a Himalayan basin, Nepal

The Hindu Kush Himalaya (HKH) is one of the major sources of fresh water on Earth and is currently under serious threat of climate change. This study investigates the future water availability in the Langtang basin, Central Himalayas, Nepal under climate change scenarios using state-of-the-art machine learning (ML) techniques. The daily snow area for the region was derived from MODIS images. The outputs of climate models were used to project the temperature and precipitation until 2100. Three ML models, including Gated recurrent unit (GRU), Long short-term memory (LSTM), and Recurrent neural network (RNN), were developed for snowmelt runoff prediction, and their performance was compared based on statistical indicators. The result suggests that the mean temperature of the basin could rise by 4.98 °C by the end of the century. The annual average precipitation in the basin is likely to increase in the future, especially due to high monsoon rainfall, but winter precipitation could decline. The annual river discharge is projected to upsurge significantly due to increased precipitation and snowmelt, and no shift in hydrograph is expected in the future. Among three ML models, the LSTM model performed better than GRU and RNN models. In summary, this study depicts severe future climate change in the region and quantifies its effect on river discharge. Furthermore, the study demonstrates the suitability of the LSTM model in streamflow prediction in the data-scarce HKH region. The outcomes of this study will be useful for water resource managers and planners in developing strategies to harness the positive impacts and offset the negative effects of climate change in the basin.

Automated detection of arrhythmia from electrocardiogram signal based on new convolutional encoded features with bidirectional long short-term memory network classifier

Early detection of cardiac arrhythmia is needed to reduce mortality. Automatically detecting the cardiac arrhythmias is a very challenging task. In this paper, a new deep convolutional encoded feature (CEF) based on non-linear compression composition is applied to diminish the ECG signal segment size. Bidirectional long short-term memory (BLSTM) network classifier has been proposed to detect arrhythmias from the ECG signal, which is encoded by the convolutional encoder. These encoded features are used as the input to BLSTM network classifier. For performance comparison, three other classifiers, namely unidirectional long short-term memory (ULSTM) network, gated recurrent Unit (GRU) and multilayer perceptron, are designed. The experimental studies detect and classify arrhythmias present in the MIT-BIH arrhythmia database into five different heartbeat classes. These heartbeat classes are normal (N), left bundle branch block (L), right bundle branch block(R), paced (P) and premature ventricular contraction (V). Evaluation of performance and system efficiency has been done with the help of four different types of evaluation criteria which are overall accuracy, precision, recall, and F-score. The experimental results indicate that the BLSTM network has achieved an overall accuracy of 99.52% with the processing time of only 6.043 s.

Gated recurrent unit-based heart sound analysis for heart failure screening

BACKGROUND

Heart failure (HF) is a type of cardiovascular disease caused by abnormal cardiac structure and function. Early screening of HF has important implication for treatment in a timely manner. Heart sound (HS) conveys relevant information related to HF; this study is therefore based on the analysis of HS signals. The objective is to develop an efficient tool to identify subjects of normal, HF with preserved ejection fraction and HF with reduced ejection fraction automatically.

METHODS

We proposed a novel HF screening framework based on gated recurrent unit (GRU) model in this study. The logistic regression-based hidden semi-Markov model was adopted to segment HS frames. Normalized frames were taken as the input of the proposed model which can automatically learn the deep features and complete the HF screening without de-nosing and hand-crafted feature extraction.

RESULTS

To evaluate the performance of proposed model, three methods are used for comparison. The results show that the GRU model gives a satisfactory performance with average accuracy of 98.82%, which is better than other comparison models.

CONCLUSION

The proposed GRU model can learn features from HS directly, which means it can be independent of expert knowledge. In addition, the good performance demonstrates the effectiveness of HS analysis for HF early screening.

Fault diagnosis of rolling bearings with recurrent neural network-based autoencoders

As the rolling bearings being the key part of rotary machine, its healthy condition is quite important for safety production. Fault diagnosis of rolling bearing has been research focus for the sake of improving the economic efficiency and guaranteeing the operation security. However, the collected signals are mixed with ambient noise during the operation of rotary machine, which brings great challenge to the exact diagnosis results. Using signals collected from multiple sensors can avoid the loss of local information and extract more helpful characteristics. Recurrent Neural Networks (RNN) is a type of artificial neural network which can deal with multiple time sequence data. The capacity of RNN has been proved outstanding for catching time relevance about time sequence data. This paper proposed a novel method for bearing fault diagnosis with RNN in the form of an autoencoder. In this approach, multiple vibration value of the rolling bearings of the next period are predicted from the previous period by means of Gated Recurrent Unit (GRU)-based denoising autoencoder. These GRU-based non-linear predictive denoising autoencoders (GRU-NP-DAEs) are trained with strong generalization ability for each different fault pattern. Then for the given input data, the reconstruction errors between the next period data and the output data generated by different GRU-NP-DAEs are used to detect anomalous conditions and classify fault type. Classic rotating machinery datasets have been employed to testify the effectiveness of the proposed diagnosis method and its preponderance over some state-of-the-art methods. The experiment results indicate that the proposed method achieves satisfactory performance with strong robustness and high classification accuracy.