Online Training of LSTM Networks in Distributed Systems for Variable Length Data Sequences

Antimicrobial peptides: An alternative to traditional antibiotics

As antibiotic-resistant bacteria and genes continue to emerge, the identification of effective alternatives to traditional antibiotics has become a pressing issue. Antimicrobial peptides are favored for their safety, low residue, and low resistance properties, and their unique antimicrobial mechanisms show significant potential in combating antibiotic resistance. However, the high production cost and weak activity of antimicrobial peptides limit their application. Moreover, traditional laboratory methods for identifying and designing new antimicrobial peptides are time-consuming and labor-intensive, hindering their development. Currently, novel technologies, such as artificial intelligence (AI) are being employed to develop and design new antimicrobial peptide resources, offering new opportunities for the advancement of antimicrobial peptides. This article summarizes the basic characteristics and antimicrobial mechanisms of antimicrobial peptides, as well as their advantages and limitations, and explores the application of AI in antimicrobial peptides prediction amd design. This highlights the crucial role of AI in enhancing the efficiency of antimicrobial peptide research and provides a reference for antimicrobial drug development.

Identification of runner fatigue stages based on inertial sensors and deep learning

Introduction: Running is one of the most popular sports in the world, but it also increases the risk of injury. The purpose of this study was to establish a modeling approach for IMU-based subdivided action pattern evaluation and to investigate the classification performance of different deep models for predicting running fatigue. Methods: Nineteen healthy male runners were recruited for this study, and the raw time series data were recorded during the pre-fatigue, mid-fatigue, and post-fatigue states during running to construct a running fatigue dataset based on multiple IMUs. In addition to the IMU time series data, each participant's training level was monitored as an indicator of their level of physical fatigue. Results: The dataset was examined using single-layer LSTM (S_LSTM), CNN, dual-layer LSTM (D_LSTM), single-layer LSTM plus attention model (LSTM + Attention), CNN, and LSTM hybrid model (LSTM + CNN) to classify running fatigue and fatigue levels. Discussion: Based on this dataset, this study proposes a deep learning model with constant length interception of the raw IMU data as input. The use of deep learning models can achieve good classification results for runner fatigue recognition. Both CNN and LSTM can effectively complete the classification of fatigue IMU data, the attention mechanism can effectively improve the processing efficiency of LSTM on the raw IMU data, and the hybrid model of CNN and LSTM is superior to the independent model, which can better extract the features of raw IMU data for fatigue classification. This study will provide some reference for many future action pattern studies based on deep learning.

Deep-Learning Approach to Predict Survival Outcomes Using Wearable Actigraphy Device Among End-Stage Cancer Patients

Survival prediction is highly valued in end-of-life care clinical practice, and patient performance status evaluation stands as a predominant component in survival prognostication. While current performance status evaluation tools are limited to their subjective nature, the advent of wearable technology enables continual recordings of patients' activity and has the potential to measure performance status objectively. We hypothesize that wristband actigraphy monitoring devices can predict in-hospital death of end-stage cancer patients during the time of their hospital admissions. The objective of this study was to train and validate a long short-term memory (LSTM) deep-learning prediction model based on activity data of wearable actigraphy devices. The study recruited 60 end-stage cancer patients in a hospice care unit, with 28 deaths and 32 discharged in stable condition at the end of their hospital stay. The standard Karnofsky Performance Status score had an overall prognostic accuracy of 0.83. The LSTM prediction model based on patients' continual actigraphy monitoring had an overall prognostic accuracy of 0.83. Furthermore, the model performance improved with longer input data length up to 48 h. In conclusion, our research suggests the potential feasibility of wristband actigraphy to predict end-of-life admission outcomes in palliative care for end-stage cancer patients. Clinical Trial Registration: The study protocol was registered on ClinicalTrials.gov (ID: NCT04883879).

CSLM: Convertible Short-Term and Long-Term Memory in Differential Neural Computers

External memory-based neural networks, such as differentiable neural computers (DNCs), have recently gained importance and popularity to solve complex sequential learning tasks that pose challenges to conventional neural networks. However, a trained DNC usually has a low-memory utilization efficiency. This article introduces a variation of DNC architecture with a convertible short-term and long-term memory, named CSLM-DNC. Unlike the memory architecture of the original DNC, the new scheme of short-term and long-term memories offers different importance of memory locations for read and write, and they can be converted over time. This is mainly motivated by the human brain where short-term memory stores large amounts of noisy and unimportant information and decays rapidly, while long-term memory stores important information and lasts for a long time. The conversion of these two types of memory is allowed and is able to be learned according to their reading and writing frequency. We quantitatively and qualitatively evaluate the proposed CSLM-DNC architecture on the tasks of question answering, copy and repeat copy, showing that it can significantly improve memory efficiency and learning performance.

Short-Term Load Forecasting Based on Deep Learning Bidirectional LSTM Neural Network

Accurate load forecasting guarantees the stable and economic operation of power systems. With the increasing integration of distributed generations and electrical vehicles, the variability and randomness characteristics of individual loads and the distributed generation has increased the complexity of power loads in power systems. Hence, accurate and robust load forecasting results are becoming increasingly important in modern power systems. The paper presents a multi-layer stacked bidirectional long short-term memory (LSTM)-based short-term load forecasting framework; the method includes neural network architecture, model training, and bootstrapping. In the proposed method, reverse computing is combined with forward computing, and a feedback calculation mechanism is designed to solve the coupling of before and after time-series information of the power load. In order to improve the convergence of the algorithm, deep learning training is introduced to mine the correlation between historical loads, and the multi-layer stacked style of the network is established to manage the power load information. Finally, actual data are applied to test the proposed method, and a comparison of the results of the proposed method with different methods shows that the proposed method can extract dynamic features from the data as well as make accurate predictions, and the availability of the proposed method is verified with real operational data.

Forecasting PM2.5 Concentration Using a Single-Dense Layer BiLSTM Method

In recent times, particulate matter (PM2.5) is one of the most critical air quality contaminants, and the rise of its concentration will intensify the hazard of cleanrooms. The forecasting of the concentration of PM2.5 has great importance to improve the safety of the highly pollutant-sensitive electronic circuits in the factories, especially inside semiconductor industries. In this paper, a Single-Dense Layer Bidirectional Long Short-term Memory (BiLSTM) model is developed to forecast the PM2.5 concentrations in the indoor environment by using the time series data. The real-time data samples of PM2.5 concentrations were obtained by using an industrial-grade sensor based on edge computing. The proposed model provided the best results comparing with the other existing models in terms of mean absolute error, mean square error, root mean square error, and mean absolute percentage error. These results show that the low error of forecasting PM2.5 concentration in a cleanroom in a semiconductor factory using the proposed Single-Dense Layer BiLSTM method is considerably high.

Heart sound classification based on improved MFCC features and convolutional recurrent neural networks

Heart sound classification plays a vital role in the early detection of cardiovascular disorders, especially for small primary health care clinics. Despite that much progress has been made for heart sound classification in recent years, most of them are based on conventional segmented features and shallow structure based classifiers. These conventional acoustic representation and classification methods may be insufficient in characterizing heart sound, and generally suffer from a degraded performance due to the complicated and changeable cardiac acoustic environment. In this paper, we propose a new heart sound classification method based on improved Mel-frequency cepstrum coefficient (MFCC) features and convolutional recurrent neural networks. The Mel-frequency cepstrums are firstly calculated without dividing the heart sound signal. A new improved feature extraction scheme based on MFCC is proposed to elaborate the dynamic characteristics among consecutive heart sound signals. Finally, the MFCC-based features are fed to a deep convolutional and recurrent neural network (CRNN) for feature learning and later classification task. The proposed deep learning framework can take advantage of the encoded local characteristics extracted from the convolutional neural network (CNN) and the long-term dependencies captured by the recurrent neural network (RNN). Comprehensive studies on the performance of different network parameters and different network connection strategies are presented in this paper. Performance comparisons with state-of-the-art algorithms are given for discussions. Experiments show that, for the two-class classification problem (pathological or non-pathological), a classification accuracy of 98% has been achieved on the 2016 PhysioNet/CinC Challenge database.

Unsupervised Anomaly Detection With LSTM Neural Networks

We investigate anomaly detection in an unsupervised framework and introduce long short-term memory (LSTM) neural network-based algorithms. In particular, given variable length data sequences, we first pass these sequences through our LSTM-based structure and obtain fixed-length sequences. We then find a decision function for our anomaly detectors based on the one-class support vector machines (OC-SVMs) and support vector data description (SVDD) algorithms. As the first time in the literature, we jointly train and optimize the parameters of the LSTM architecture and the OC-SVM (or SVDD) algorithm using highly effective gradient and quadratic programming-based training methods. To apply the gradient-based training method, we modify the original objective criteria of the OC-SVM and SVDD algorithms, where we prove the convergence of the modified objective criteria to the original criteria. We also provide extensions of our unsupervised formulation to the semisupervised and fully supervised frameworks. Thus, we obtain anomaly detection algorithms that can process variable length data sequences while providing high performance, especially for time series data. Our approach is generic so that we also apply this approach to the gated recurrent unit (GRU) architecture by directly replacing our LSTM-based structure with the GRU-based structure. In our experiments, we illustrate significant performance gains achieved by our algorithms with respect to the conventional methods.

Debiasing and Distributed Estimation for High-Dimensional Quantile Regression

Distributed and parallel computing is becoming more important with the availability of extremely large data sets. In this article, we consider this problem for high-dimensional linear quantile regression. We work under the assumption that the coefficients in the regression model are sparse; therefore, a LASSO penalty is naturally used for estimation. We first extend the debiasing procedure, which is previously proposed for smooth parametric regression models to quantile regression. The technical challenges include dealing with the nondifferentiability of the loss function and the estimation of the unknown conditional density. In this article, the main objective is to derive a divide-and-conquer estimation approach using the debiased estimator which is useful under the big data setting. The effectiveness of distributed estimation is demonstrated using some numerical examples.

SoC Design Based on a FPGA for a Configurable Neural Network Trained by Means of an EKF

This work presents a configurable architecture for an artificial neural network implemented with a Field Programmable Gate Array (FPGA) in a System on Chip (SoC) environment. This architecture can reproduce the transfer function of different Multilayer Feedforward Neural Network (MFNN) configurations. The functionality of this configurable architecture relies on a single perceptron, multiplexers, and memory blocks that allow routing, storing, and processing information. The extended Kalman filter is the training algorithm that obtains the optimal weight values for the MFNN. The presented architecture was developed using Verilog Hardware Description Language, which permits designing hardware with a fair number of logical resources, and facilitates the portability to different FPGAs models without compatibility problems. A SoC that mainly incorporates a microprocessor and a FPGA is proposed, where the microprocessor is used for configuring the the MFNN and to enable and disable some functional blocks in the FPGA. The hardware was tested with measurements from a GaN class F power amplifier, using a 2.1 GHz Long Term Evolution signal with 5 MHz of bandwidth. In particular, a special case of an MFNN with two layers, i.e., a real-valued nonlinear autoregressive with an exogenous input neural network, was considered. The results reveal that a normalized mean square error value of −32.82 dB in steady-state was achievable, with a 71.36% generalization using unknown samples.