Deep Neural Network-Embedded Stochastic Nonlinear State-Space Models and Their Applications to Process Monitoring

Variational Discriminative Stacked Auto-Encoder: Feature Representation Using a Prelearned Discriminator, and Its Application to Industrial Process Monitoring

In deep-learning-based process monitoring, obtaining an effective feature representation is a critical step in constructing a reliable deep-learning monitoring model. Conventional deep-learning methods like stacked auto-encoders (SAEs) capture feature representation by minimizing the data reconstruction errors, which lack the expression of essential information and ultimately lead to degradation of the monitoring performance. To solve this problem, variational discriminative SAE (VDSAE) is proposed in this article. First, a variational generative discriminative structure is designed to obtain a reliable prelearned discriminator. Based on this new variational discriminator, the authenticity of the reconstructed data is evaluated as an important criterion for feature learning. Then, an SAE incorporating the prelearned discriminator is trained by both minimizing the reconstruction error and maximizing the data authenticity. In this way, the prelearned discriminator makes the network effectively capture the essential expression of the reconstructed data. The proposed approach enables SAE to learn a better feature representation owing to the excellent reconstruction performance. Finally, the feature representation and fault detection performance of VDSAE are verified in two cases. The results show that the average fault detection rates (FDRs) of the multiphase flow facility and the waste-water treatment process (WWTP) can be improved to 72% and 97%, respectively, compared with the other fault detection methods.

VAE-Based Interpretable Latent Variable Model for Process Monitoring

Latent variable-based process monitoring (PM) models have been generously developed by shallow learning approaches, such as multivariate statistical analysis and kernel techniques. Owing to their explicit projection objectives, the extracted latent variables are usually meaningful and easily interpretable in mathematical terms. Recently, deep learning (DL) has been introduced to PM and has exhibited excellent performance because of its powerful presentation capability. However, its complex nonlinearity prevents it from being interpreted as human-friendly. It is a mystery how to design a proper network structure to achieve satisfactory PM performance for DL-based latent variable models (LVMs). In this article, a variational autoencoder-based interpretable LVM (VAE-ILVM) is developed for PM. Based on Taylor expansions, two propositions are proposed to guide the design of appropriate activation functions for VAE-ILVM, allowing nondisappearing fault impact terms contained in the generated monitoring metrics (MMs). During threshold learning, the sequence of counting that test statistics exceed the threshold is considered a martingale, a representative of weakly dependent stochastic processes. A de la Peña inequality is then adopted to learn a suitable threshold. Finally, two chemical examples verify the effectiveness of the proposed method. The use of de la Peña inequality significantly reduces the minimum required sample size for modeling.

Temporal and Spatial Analysis of Alzheimer's Disease Based on an Improved Convolutional Neural Network and a Resting-State FMRI Brain Functional Network

Most current research on Alzheimer's disease (AD) is based on transverse measurements. Given the nature of neurodegeneration in AD progression, observing longitudinal changes in the structural features of brain networks over time may improve the accuracy of the predicted transformation and provide a good measure of the progression of AD. Currently, there is no cure for patients with existing AD dementia, but patients with mild cognitive impairment (MCI) in the prodromal stage of AD dementia may be diagnosed. The study of the early diagnosis of MCI and the prediction of MCI to AD transformation is of great significance for the monitoring of the MCI to AD transformation process. Despite the high rate of MCI conversion to AD, the neuropathological cause of MCI is heterogeneous. However, many people with MCI remain stable. Treatment options are different for patients with stable MCI and those with underlying dementia. Therefore, it is of great significance for clinical practice to predict whether patients with MCI will develop AD dementia. This paper proposes an improved algorithm that is based on a convolution neural network (CNN) with residuals combined with multi-layer long short-term memory (LSTM) to diagnose AD and predict MCI. Firstly, multi-time resting-state fMRI images were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database for preprocessing, and then an AAL brain partition template was used to construct a 90 × 90 functional connectivity (FC) network matrix of a whole-brain region of interest (ROI). Secondly, the diversity of training samples was increased by generating an adversarial network (GAN). Finally, a CNN with residuals and a multi-layer LSTM model were constructed to automatically classify and predict the functional adjacency matrix. This method can not only distinguish Alzheimer's disease from normal health conditions at multiple time points, but can also predict progressive MCI (pMCI) and stable MCI (sMCI) at multiple time points. The classification accuracies in AD vs. NC and sMCI vs.pMCI reached 93.5% and 75.5%, respectively.