Degradation Alignment in Remaining Useful Life Prediction Using Deep Cycle-Consistent Learning

Local-Global Correlation Fusion-Based Graph Neural Network for Remaining Useful Life Prediction

Remaining useful life (RUL) prediction is an essential component for prognostics and health management of a system. Due to the powerful ability of nonlinear modeling, deep learning (DL) models have emerged as leading solutions by capturing temporal dependencies within time series sensory data. However, in RUL prediction tasks, data are typically collected from multiple sensors, introducing spatial dependencies in the form of sensor correlations. Existing methods are limited in effectively modeling and capturing the spatial dependencies, restricting their performance to learn representative features for RUL prediction. To overcome the limitations, we propose a novel LOcal-GlObal correlation fusion-based framework (LOGO). Our approach combines both local and global information to model sensor correlations effectively. From a local perspective, we account for local correlations that represent dynamic changes of sensor relationships in local ranges. Simultaneously, from a global perspective, we capture global correlations that depict relatively stable relations between sensors. An adaptive fusion mechanism is proposed to automatically fuse the correlations from different perspectives. Subsequently, we define sequential micrographs for each sample to effectively capture the fused correlations. Graph neural network (GNN) is introduced to capture the spatial dependencies within each micrograph, and the temporal dependencies between these sequential micrographs are then captured. This approach allows us to effectively model and capture the dependency information within the data for accurate RUL prediction. Extensive experiments have been conducted, verifying the effectiveness of our method.

Bayesian Hierarchical Graph Neural Networks With Uncertainty Feedback for Trustworthy Fault Diagnosis of Industrial Processes

Deep learning (DL) methods have been widely applied to intelligent fault diagnosis of industrial processes and achieved state-of-the-art performance. However, fault diagnosis with point estimate may provide untrustworthy decisions. Recently, Bayesian inference shows to be a promising approach to trustworthy fault diagnosis by quantifying the uncertainty of the decisions with a DL model. The uncertainty information is not involved in the training process, which does not help the learning of highly uncertain samples and has little effect on improving the fault diagnosis performance. To address this challenge, we propose a Bayesian hierarchical graph neural network (BHGNN) with an uncertainty feedback mechanism, which formulates a trustworthy fault diagnosis on the Bayesian DL (BDL) framework. Specifically, BHGNN captures the epistemic uncertainty and aleatoric uncertainty via a variational dropout approach and utilizes the uncertainty information of each sample to adjust the strength of the temporal consistency (TC) constraint for robust feature learning. Meanwhile, the BHGNN method models the process data as a hierarchical graph (HG) by leveraging the interaction-aware module and physical topology knowledge of the industrial process, which integrates data with domain knowledge to learn fault representation. Moreover, the experiments on a three-phase flow facility (TFF) and secure water treatment (SWaT) show superior and competitive performance in fault diagnosis and verify the trustworthiness of the proposed method.

A Hybrid Data Preprocessing-Based Hierarchical Attention BiLSTM Network for Remaining Useful Life Prediction of Spacecraft Lithium-Ion Batteries

As a crucial energy storage for the spacecraft power system, lithium-ion batteries degradation mechanisms are complex and involved with external environmental perturbations. Hence, effective remaining useful life (RUL) prediction and model reliability assessment confronts considerable obstacles. This article develops a new RUL prediction method for spacecraft lithium-ion batteries, where a hybrid data preprocessing-based deep learning model is proposed. First, to improve the correlation between battery capacity and features, the empirically selected high-dimensional features are linearized by using the Box-Cox transformation and then denoised via the complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) method. Second, the principal component analysis (PCA) algorithm is employed to perform feature dimensionality reduction, and the output of PCA is further processed by the sliding window technique. Third, a multiscale hierarchical attention bi-directional long short-term memory (MHA-BiLSTM) model is constructed to estimate the capacity in future cycles. Specifically, the MHA-BiLSTM model can predict the RUL of lithium-ion batteries by considering the correlation and significance of each cycle's information during the degradation process on different scales. Finally, the proposed method is validated based on multiple types of experiments under two lithium-ion battery datasets, demonstrating its superior performance in terms of feature extraction and multidimensional time series prediction.

Interactive Prognosis Framework Between Deep Learning and a Stochastic Process Model for Remaining Useful Life Prediction

Uncertainty quantification of the remaining useful life (RUL) for degraded systems under the big data era has been a hot topic in recent years. A general idea is to execute two separate steps: deep-learning-based health indicator (HI) construction and stochastic process-based degradation modeling. However, there exists a critical matching defect between the constructed HI and a degradation model, which seriously affects the RUL prediction accuracy. Toward this end, this article proposes an interactive prognosis framework between deep learning and a stochastic process model for the RUL prediction. First, we resort to stacked contractive autoencoders to fuse multiple sensor information of historical systems for constructing the HI in a typical unsupervised manner. Then, considering the nonlinear characteristic of the constructed HI, an exponential-like degradation model is introduced to construct its degradation evolving model, and theoretical expressions of the prediction results are derived under the concept of the first hitting time. Furthermore, we design an optimization objective function by integrating the HI construction and degradation modeling for the RUL prediction. To minimize the designed objective function of the proposed interactive prognosis framework, a gradient descent algorithm is employed to update the model parameters. Based on the well-trained interactive prognosis model, we can obtain the HI of a field system from stacked contractive autoencoders with sensor data and the probability density function (pdf) of the predicted RUL on the basis of the estimated parameters. Finally, the effectiveness and superiority of the proposed interactive prognosis method are verified by two case studies associated with turbofan engines.

An Adaptation-Aware Interactive Learning Approach for Multiple Operational Condition-Based Degradation Modeling

Although degradation modeling has been widely applied to use multiple sensor signals to monitor the degradation process and predict the remaining useful lifetime (RUL) of operating machinery units, three challenging issues remain. One challenge is that units in engineering cases usually work under multiple operational conditions, causing the distribution of sensor signals to vary over conditions. It remains unexplored to characterize time-varying conditions as a distribution shift problem. The second challenge is that sensor signal fusion and degradation status modeling are separated into two independent steps in most of the existing methods, which ignores the intrinsic correlation between the two parts. The last challenge is how to find an accurate health index (HI) of units using previous knowledge of degradation. To tackle these issues, this article proposes an adaptation-aware interactive learning (AAIL) approach for degradation modeling. First, a condition-invariant HI is developed to handle time-varying operation conditions. Second, an interactive framework based on the fusion and degradation model is constructed, which naturally integrates a supervised learner and an unsupervised learner. To estimate the model parameters of AAIL, we propose an interactive training algorithm that shares learned degradation and fusion information during the model training process. A case study that uses the degradation data set of aircraft engines demonstrates that the proposed AAIL outperforms related benchmark methods.

DSIL-DDI: A Domain-Invariant Substructure Interaction Learning for Generalizable Drug-Drug Interaction Prediction

Drug-drug interactions (DDIs) trigger unexpected pharmacological effects in vivo, often with unknown causal mechanisms. Deep learning methods have been developed to better understand DDI. However, learning domain-invariant representations for DDI remains a challenge. Generalizable DDI predictions are closer to reality than source domain predictions. For existing methods, it is difficult to achieve out-of-distribution (OOD) predictions. In this article, focusing on substructure interaction, we propose DSIL-DDI, a pluggable substructure interaction module that can learn domain-invariant representations of DDIs from source domain. We evaluate DSIL-DDI on three scenarios: the transductive setting (all drugs in test set appear in training set), the inductive setting (test set contains new drugs that were not present in training set), and OOD generalization setting (training set and test set belong to two different datasets). The results demonstrate that DSIL-DDI improve the generalization and interpretability of DDI prediction modeling and provides valuable insights for OOD DDI predictions. DSIL-DDI can help doctors ensuring the safety of drug administration and reducing the harm caused by drug abuse.

Explainable Graph Wavelet Denoising Network for Intelligent Fault Diagnosis

Deep learning (DL)-based intelligent fault diagnosis methods have greatly promoted the development of the field of fault diagnosis due to their powerful feature extraction ability for handling massive monitoring data. However, most of them still suffer from the following three limitations. First, many existing DL-based intelligent diagnosis methods cannot extract proper discriminative features from signals with strong noise. Second, the interactions or relationships between signals are ignored, while they mainly focus on extracting temporal features from the signal. Third, owing to their black-box nature, the learned features lack interpretability, which hinders their application in the industry. To tackle these issues, an explainable graph wavelet denoising network (GWDN) is proposed to achieve intelligent fault diagnosis under noisy working conditions in this article. In GWDN, the collected signals are first transformed into graph-structured data to consider the interactions among signals. Then, the graph wavelet denoising convolution (GWDConv) is proposed based on the discrete graph wavelet frame, which allows GWDN to achieve multiscale feature extraction for graph-structured data and realize signal denoising. Extensive experiments are implemented to verify the efficacy of the proposed GWDN, and the experimental results show that GWDN can achieve state-of-the-art performance among the comparison methods. Besides, by using the square envelope spectrum to analyze the extracted features of GWDConv, we find that it can well retain the fault-related components of the signal and realize signal denoising, which further proves that GWDN is explainable.

DLformer: A Dynamic Length Transformer-Based Network for Efficient Feature Representation in Remaining Useful Life Prediction

Representation learning-based remaining useful life (RUL) prediction plays a crucial role in improving the security and reducing the maintenance cost of complex systems. Despite the superior performance, the high computational cost of deep networks hinders deploying the models on low-compute platforms. A significant reason for the high cost is the computation of representing long sequences. In contrast to most RUL prediction methods that learn features of the same sequence length, we consider that each time series has its characteristics and the sequence length should be adjusted adaptively. Our motivation is that an "easy" sample with representative characteristics can be correctly predicted even when short feature representation is provided, while "hard" samples need complete feature representation. Therefore, we focus on sequence length and propose a dynamic length transformer (DLformer) that can adaptively learn sequence representation of different lengths. Then, a feature reuse mechanism is developed to utilize previously learned features to reduce redundant computation. Finally, in order to achieve dynamic feature representation, a particular confidence strategy is designed to calculate the confidence level for the prediction results. Regarding interpretability, the dynamic architecture can help human understand which part of the model is activated. Experiments on multiple datasets show that DLformer can increase up to 90% inference speed, with less than 5% degradation in model accuracy.

Remaining Useful Life Prediction of Rolling Bearings Based on Multi-Scale Attention Residual Network

The remaining useful life (RUL) prediction of rolling bearings based on vibration signals has attracted widespread attention. It is not satisfactory to adopt information theory (such as information entropy) to realize RUL prediction for complex vibration signals. Recent research has used more deep learning methods based on the automatic extraction of feature information to replace traditional methods (such as information theory or signal processing) to obtain higher prediction accuracy. Convolutional neural networks (CNNs) based on multi-scale information extraction have demonstrated promising effectiveness. However, the existing multi-scale methods significantly increase the number of model parameters and lack efficient learning mechanisms to distinguish the importance of different scale information. To deal with the issue, the authors of this paper developed a novel feature reuse multi-scale attention residual network (FRMARNet) for the RUL prediction of rolling bearings. Firstly, a cross-channel maximum pooling layer was designed to automatically select the more important information. Secondly, a lightweight feature reuse multi-scale attention unit was developed to extract the multi-scale degradation information in the vibration signals and recalibrate the multi-scale information. Then, end-to-end mapping between the vibration signal and the RUL was established. Finally, extensive experiments were used to demonstrate that the proposed FRMARNet model can improve prediction accuracy while reducing the number of model parameters, and it outperformed other state-of-the-art methods.

Remaining Useful Life Prediction Based on Adaptive SHRINKAGE Processing and Temporal Convolutional Network

The remaining useful life (RUL) prediction is important for improving the safety, supportability, maintainability, and reliability of modern industrial equipment. The traditional data-driven rolling bearing RUL prediction methods require a substantial amount of prior knowledge to extract degraded features. A large number of recurrent neural networks (RNNs) have been applied to RUL, but their shortcomings of long-term dependence and inability to remember long-term historical information can result in low RUL prediction accuracy. To address this limitation, this paper proposes an RUL prediction method based on adaptive shrinkage processing and a temporal convolutional network (TCN). In the proposed method, instead of performing the feature extraction to preprocess the original data, the multi-channel data are directly used as an input of a prediction network. In addition, an adaptive shrinkage processing sub-network is designed to allocate the parameters of the soft-thresholding function adaptively to reduce noise-related information amount while retaining useful features. Therefore, compared with the existing RUL prediction methods, the proposed method can more accurately describe RUL based on the original historical data. Through experiments on a PHM2012 rolling bearing data set, a XJTU-SY data set and comparison with different methods, the predicted mean absolute error (MAE) is reduced by 52% at most, and the root mean square error (RMSE) is reduced by 64% at most. The experimental results show that the proposed adaptive shrinkage processing method, combined with the TCN model, can predict the RUL accurately and has a high application value.

Deep Multi-Scale Residual Connected Neural Network Model for Intelligent Athlete Balance Control Ability Evaluation

Athlete balance control ability plays an important role in different types of sports. Accurate and efficient evaluations of the balance control abilities can significantly improve the athlete management performance. With the rapid development of the athlete training field, intelligent and automatic evaluations have been highly demanded in the past years. This study proposes a deep learning-based athlete balance control ability evaluation method through processing the time-series movement pressure measurement data. An end-to-end model structure is proposed, which directly analyzes the raw data and provides the evaluation results, which largely facilitates practical utilization. A multi-scale feature extraction scheme is employed, by exploring the learned features in different scales. A residual connected neural network architecture is further proposed. By using the short-cut connection, the deep neural network model can be more efficiently trained. Experiments on the real athlete balance control ability tests are carried out for validations. Through comparisons with different related methods, the results show the proposed deep multi-scale residual connected neural network model is well suited for the athlete balance control ability evaluation problem, and promising for actual applications in the real scenarios.

An Adaptive Neuro-Fuzzy Model for Attitude Estimation and Control of a 3 DOF System

In recent decades, one of the scientists’ main concerns has been to improve the accuracy of satellite attitude, regardless of the expense. The obvious result is that a large number of control strategies have been used to address this problem. In this study, an adaptive neuro-fuzzy integrated system (ANFIS) for satellite attitude estimation and control was developed. The controller was trained with the data provided by an optimal controller. Furthermore, a pulse modulator was used to generate the right ON/OFF commands of the thruster actuator. To evaluate the performance of the proposed controller in closed-loop simulation, an ANFIS observer was also used to estimate the attitude and angular velocities of the satellite using magnetometer, sun sensor, and data gyro data. However, a new ANFIS system was proposed that can jointly control and estimate the system attitude. The performance of the proposed controller was compared to the optimal PID controller in a Monte Carlo simulation with different initial conditions, disturbance, and noise. The results show that the proposed controller can surpass the optimal PID controller in several aspects including time and smoothness. In addition, the ANFIS estimator was examined and the results demonstrate the high ability of this designated observer. Consequently, evaluating the performance of PID and the proposed controller revealed that the proposed controller consumed less control effort for satellite attitude estimation under noise and uncertainty.

Intelligent Robust Cross-Domain Fault Diagnostic Method for Rotating Machines Using Noisy Condition Labels

Cross-domain fault diagnosis methods have been successfully and widely developed in the past years, which focus on practical industrial scenarios with training and testing data from numerous machinery working regimes. Due to the remarkable effectiveness in such problems, deep learning-based domain adaptation approaches have been attracting increasing attention. However, the existing methods in the literature are generally lower compared to environmental noise and data availability, and it is difficult to achieve promising performance under harsh practical conditions. This paper proposes a new cross-domain fault diagnosis method with enhanced robustness. Noisy labels are introduced to significantly increase the generalization ability of the data-driven model. Promising diagnosis performance can be obtained with strong noise interference in testing, as well as in practical cases with low-quality data. Experiments on two rotating machinery datasets are carried out for validation. The results indicate that the proposed algorithm is well suited to be applied in real industrial environments to achieve promising performance with variations of working conditions.

Prediction of steady flows passing fixed cylinders using deep learning

Considerable attention has been given to deep-learning and machine-learning techniques in an effort to reduce the computational cost of computational fluid dynamics simulation. The present paper addresses the prediction of steady flows passing many fixed cylinders using a deep-learning model and investigates the accuracy of the predicted velocity field. The deep-learning model outputs the x- and y-components of the flow velocity field when the cylinder arrangement is input. The accuracy of the predicted velocity field is investigated, focusing on the velocity profile of the fluid flow and the fluid force acting on the cylinders. The present model accurately predicts the flow when the number of cylinders is equal to or close to that set in the training dataset. The extrapolation of the prediction to a smaller number of cylinders results in error, which can be interpreted as internal friction of the fluid. The results of the fluid force acting on the cylinders suggest that the present deep-learning model has good generalization performance for systems with a larger number of cylinders.

A Bearing Fault Diagnosis Method Using Multi-Branch Deep Neural Network

Feature extraction from a signal is the most important step in signal-based fault diagnosis. Deep learning or deep neural network (DNN) is an effective method to extract features from signals. In this paper, a novel vibration signal-based bearing fault diagnosis method using DNN is proposed. First, the measured vibration signals are transformed into a new data form called multiple-domain image-representation. By this transformation, the task of signal-based fault diagnosis is transferred into the task of image classification. After that, a DNN with a multi-branch structure is proposed to handle the multiple-domain image representation data. The multi-branch structure of the proposed DNN helps to extract features in multiple domains simultaneously, and to lead to better feature extraction. Better feature extraction leads to a better performance of fault diagnosis. The effectiveness of the proposed method was verified via the experiments conducted with actual bearing fault signals and its comparisons with well-established published methods.