GANSamples-ac4C: Enhancing ac4C site prediction via generative adversarial networks and transfer learning

RNA modification, N4-acetylcytidine (ac4C), is enzymatically catalyzed by N-acetyltransferase 10 (NAT10) and plays an essential role across tRNA, rRNA, and mRNA. It influences various cellular functions, including mRNA stability and rRNA biosynthesis. Wet-lab detection of ac4C modification sites is highly resource-intensive and costly. Therefore, various machine learning and deep learning techniques have been employed for computational detection of ac4C modification sites. The known ac4C modification sites are limited for training an accurate and stable prediction model. This study introduces GANSamples-ac4C, a novel framework that synergizes transfer learning and generative adversarial network (GAN) to generate synthetic RNA sequences to train a better ac4C modification site prediction model. Comparative analysis reveals that GANSamples-ac4C outperforms existing state-of-the-art methods in identifying ac4C sites. Moreover, our result underscores the potential of synthetic data in mitigating the issue of data scarcity for biological sequence prediction tasks. Another major advantage of GANSamples-ac4C is its interpretable decision logic. Multi-faceted interpretability analyses detect key regions in the ac4C sequences influencing the discriminating decision between positive and negative samples, a pronounced enrichment of G in this region, and ac4C-associated motifs. These findings may offer novel insights for ac4C research. The GANSamples-ac4C framework and its source code are publicly accessible at http://www.healthinformaticslab.org/supp/.

ONE3A: one-against-all authentication model for smartphone using GAN network and optimization techniques

This study focuses on addressing computational limits in smartphones by proposing an efficient authentication model that enables implicit authentication without requiring additional hardware and incurring less computational cost. The research explores various wrapper feature selection strategies and classifiers to enhance authentication accuracy while considering smartphone limitations such as hardware constraints, battery life, and memory size. However, the available dataset is small; thus, it cannot support a general conclusion. In this article, a novel implicit authentication model for smartphone users is proposed to address the one-against-all classification problem in smartphone authentication. This model depends on the integration of the conditional tabular generative adversarial network (CTGAN) to generate synthetic data to address the imbalanced dataset and a new proposed feature selection technique based on the Whale Optimization Algorithm (WOA). The model was evaluated using a public dataset (RHU touch mobile keystroke dataset), and the results showed that the WOA with the random forest (RF) classifier achieved the best reduction rate compared to the Harris Hawks Optimization (HHO) algorithm. Additionally, its classification accuracy was found to be the best in mobile user authentication from their touch behavior data. WOA-RF achieved an average accuracy of 99.62 ± 0.40% with a reduction rate averaging 87.85% across ten users, demonstrating its effectiveness in smartphone authentication.

A generative adaptive convolutional neural network with attention mechanism for driver fatigue detection with class-imbalanced and insufficient data

Over the past few years, fatigue driving has emerged as one of the main causes of traffic accidents, necessitating the development of driver fatigue detection systems. However, many existing methods involves tedious manual parameter tunings, a process that is both time-consuming and results in task-specific models. On the other hand, most of the researches on fatigue recognition are based on class-balanced and sufficient data, and effectively "mine" meaningful information from class-imbalanced and insufficient data for fatigue recognition is still a challenge. In this paper, we proposed two novel models, the attention-based residual adaptive multiscale fully convolutional network-long short term memory network (ARMFCN-LSTM), and the Generative ARMFCN-LSTM (GARMFCN-LSTM) aiming to address this issue. ARMFCN-LSTM excels at automatically extracting multiscale representations through adaptive multiscale temporal convolutions, while capturing temporal dependency features through LSTM. GARMFCN-LSTM integrates Wasserstein GAN with gradient penalty (WGAN-GP) into ARMFCN-LSTM to improve driver fatigue detection performance by alleviating data scarcity and addressing class imbalances. Experimental results show that ARMFCN-LSTM achieves the highest classification accuracy of 95.84% in driver fatigue detection on the class-balanced EEG dataset (binary classification), and GARMFCN-LSTM attained an improved classification accuracy of 84.70% on the class-imbalanced EOG dataset (triple classification), surpassing the competing methods. Therefore, the proposed models are promising for further implementations in online driver fatigue detection systems.

CT Perfusion Map Synthesis from CTP Dynamic Images Using a Learned LSTM Generative Adversarial Network for Acute Ischemic Stroke Assessment

Computed tomography perfusion (CTP) is a dynamic 4-dimensional imaging technique (3-dimensional volumes captured over approximately 1 min) in which cerebral blood flow is quantified by tracking the passage of a bolus of intravenous contrast with serial imaging of the brain. To diagnose and assess acute ischemic stroke, the standard method relies on summarizing acquired CTPs over the time axis to create maps that show different hemodynamic parameters, such as the timing of the bolus arrival and passage (Tmax and MTT), cerebral blood flow (CBF), and cerebral blood volume (CBV). However, producing accurate CTP maps requires the selection of an arterial input function (AIF), i.e. a time-concentration curve in one of the large feeding arteries of the brain, which is a highly error-prone procedure. Moreover, during approximately one minute of CT scanning, the brain is exposed to ionizing radiation that can alter tissue composition, and create free radicals that increase the risk of cancer. This paper proposes a novel end-to-end deep neural network that synthesizes CTP images to generate CTP maps using a learned LSTM Generative Adversarial Network (LSTM-GAN). Our proposed method can improve the precision and generalizability of CTP map extraction by eliminating the error-prone and expert-dependent AIF selection step. Further, our LSTM-GAN does not require the entire CTP time series and can produce CTP maps with a reduced number of time points. By reducing the scanning sequence from about 40 to 9 time points, the proposed method has the potential to minimize scanning time thereby reducing patient exposure to CT radiation. Our evaluations using the ISLES 2018 challenge dataset consisting of 63 patients showed that our model can generate CTP maps by using only 9 snapshots, without AIF selection, with an accuracy of 84.37 % .

Harmonic field extension for QSM with reduced spatial coverage using physics-informed generative adversarial network

Quantitative susceptibility mapping (QSM) is frequently employed in investigating brain iron related to brain development and diseases within deep gray matter (DGM). Nonetheless, the acquisition of whole-brain QSM data is time-intensive. An alternative approach, focusing the QSM specifically on areas of interest such as the DGM by reducing the field-of-view (FOV), can significantly decrease scan times. However, severe susceptibility value underestimations have been reported during QSM reconstruction with a limited FOV, largely attributable to artifacts from incorrect background field removal in the boundary region. This presents a considerable barrier to the clinical use of QSM with small spatial coverages using conventional methods alone. To mitigate the propagation of these errors, we proposed a harmonic field extension method based on a physics-informed generative adversarial network. Both quantitative and qualitative results demonstrate that our method outperforms conventional methods and delivers results comparable to those obtained with full FOV. Furthermore, we demonstrate the versatility of our method by applying it to data acquired prospectively with limited FOV and to data from patients with Parkinson's disease. The method has shown significant improvements in local field results, with QSM outcomes. In a clear illustration of its feasibility and effectiveness in real clinical environments, our proposed method addresses the prevalent issue of susceptibility underestimation in QSM with small spatial coverage.

Exploring the therapeutic potential of layered double hydroxides and transition metal dichalcogenides through the convergence of rheumatology and nanotechnology using generative adversarial network

Two-dimensional Layered double hydroxides (LDHs) are highly used in the biomedical domain due to their biocompatibility, biodegradability, controlled drug loading and release capabilities, and improved cellular permeability. The interaction of LDHs with biological systems could facilitate targeted drug delivery and make them an attractive option for various biomedical applications. Rheumatoid Arthritis (RA) requires targeted drug delivery for optimum therapeutic outcomes. In this study, stacked double hydroxide nanocomposites with dextran sulphate modification (LDH-DS) were developed while exhibiting both targeting and pH-sensitivity for rheumatological conditions. This research examines the loading, release kinetics, and efficiency of the therapeutics of interest in the LDH-based drug delivery system. The mean size of LDH-DS particles (300.1 ± 8.12 nm) is -12.11 ± 0.4 mV. The encapsulation efficiency was 48.52%, and the loading efficacy was 16.81%. In vitro release tests indicate that the drug's discharge is modified more rapidly in PBS at pH 5.4 compared to pH 5.6, which later reached 7.3, showing the case sensitivity to pH. A generative adversarial network (GAN) is used to analyze the drug delivery system in rheumatology. The GAN model achieved high accuracy and classification rates of 99.3% and 99.0%, respectively, and a validity of 99.5%. The second and third administrations resulted in a significant change with p-values of 0.001 and 0.05, respectively. This investigation unequivocally demonstrated that LDH functions as a biocompatible drug delivery matrix, significantly improving delivery effectiveness.

3D multi-modality Transformer-GAN for high-quality PET reconstruction

Positron emission tomography (PET) scans can reveal abnormal metabolic activities of cells and provide favorable information for clinical patient diagnosis. Generally, standard-dose PET (SPET) images contain more diagnostic information than low-dose PET (LPET) images but higher-dose scans can also bring higher potential radiation risks. To reduce the radiation risk while acquiring high-quality PET images, in this paper, we propose a 3D multi-modality edge-aware Transformer-GAN for high-quality SPET reconstruction using the corresponding LPET images and T1 acquisitions from magnetic resonance imaging (T1-MRI). Specifically, to fully excavate the metabolic distributions in LPET and anatomical structural information in T1-MRI, we first use two separate CNN-based encoders to extract local spatial features from the two modalities, respectively, and design a multimodal feature integration module to effectively integrate the two kinds of features given the diverse contributions of features at different locations. Then, as CNNs can describe local spatial information well but have difficulty in modeling long-range dependencies in images, we further apply a Transformer-based encoder to extract global semantic information in the input images and use a CNN decoder to transform the encoded features into SPET images. Finally, a patch-based discriminator is applied to ensure the similarity of patch-wise data distribution between the reconstructed and real images. Considering the importance of edge information in anatomical structures for clinical disease diagnosis, besides voxel-level estimation error and adversarial loss, we also introduce an edge-aware loss to retain more edge detail information in the reconstructed SPET images. Experiments on the phantom dataset and clinical dataset validate that our proposed method can effectively reconstruct high-quality SPET images and outperform current state-of-the-art methods in terms of qualitative and quantitative metrics.

Predicting disease-related MRI patterns of multiple sclerosis through GAN-based image editing

INTRODUCTION

Multiple sclerosis (MS) is a complex neurodegenerative disorder that affects the brain and spinal cord. In this study, we applied a deep learning-based approach using the StyleGAN model to explore patterns related to MS and predict disease progression in magnetic resonance images (MRI).

METHODS

We trained the StyleGAN model unsupervised using T1-weighted GRE MR images and diffusion-based ADC maps of MS patients and healthy controls. We then used the trained model to resample MR images from real input data and modified them by manipulations in the latent space to simulate MS progression. We analyzed the resulting simulation-related patterns mimicking disease progression by comparing the intensity profiles of the original and manipulated images and determined the brain parenchymal fraction (BPF).

RESULTS

Our results show that MS progression can be simulated by manipulating MR images in the latent space, as evidenced by brain volume loss on both T1-weighted and ADC maps and increasing lesion extent on ADC maps.

CONCLUSION

Overall, this study demonstrates the potential of the StyleGAN model in medical imaging to study image markers and to shed more light on the relationship between brain atrophy and MS progression through corresponding manipulations in the latent space.

Bone mineral density estimation from a plain X-ray image by learning decomposition into projections of bone-segmented computed tomography

Osteoporosis is a prevalent bone disease that causes fractures in fragile bones, leading to a decline in daily living activities. Dual-energy X-ray absorptiometry (DXA) and quantitative computed tomography (QCT) are highly accurate for diagnosing osteoporosis; however, these modalities require special equipment and scan protocols. To frequently monitor bone health, low-cost, low-dose, and ubiquitously available diagnostic methods are highly anticipated. In this study, we aim to perform bone mineral density (BMD) estimation from a plain X-ray image for opportunistic screening, which is potentially useful for early diagnosis. Existing methods have used multi-stage approaches consisting of extraction of the region of interest and simple regression to estimate BMD, which require a large amount of training data. Therefore, we propose an efficient method that learns decomposition into projections of bone-segmented QCT for BMD estimation under limited datasets. The proposed method achieved high accuracy in BMD estimation, where Pearson correlation coefficients of 0.880 and 0.920 were observed for DXA-measured BMD and QCT-measured BMD estimation tasks, respectively, and the root mean square of the coefficient of variation values were 3.27 to 3.79% for four measurements with different poses. Furthermore, we conducted extensive validation experiments, including multi-pose, uncalibrated-CT, and compression experiments toward actual application in routine clinical practice.

GAN-based patient information hiding for an ECG authentication system

Various biometrics such as the face, irises, and fingerprints, which can be obtained in a relatively simple way in modern society, are used in personal authentication systems to identify individuals. These biometric data are extracted from an individual's physiological data and yield high performance in identifying an individual using unique data patterns. Biometric identification is also used in portable devices such as mobile devices because it is more secure than cryptographic token-based authentication methods. However, physiological data could include personal health information such as arrhythmia related patterns in electrocardiogram (ECG) signals. To protect sensitive health information from hackers, the biomarkers of certain diseases or disorders that exist in ECG signals need to be hidden. Additionally, to implement the inference models for both arrhythmia detection and personal authentication in a mobile device, a lightweight model such as a multi-task deep learning model should be considered. This study demonstrates a multi-task neural network model that simultaneously identifies an individual's ECG and arrhythmia patterns using a small network. Finally, the computational efficiency and model size of the single-task and multi-task models were compared based on the number of parameters. Although the multi-task model has 20,000 fewer parameters than the single-task model, they yielded similar performance, which demonstrates the efficient structure of the multi-task model.

Unpaired virtual histological staining using prior-guided generative adversarial networks

Fibrosis is an inevitable stage in the development of chronic liver disease and has an irreplaceable role in characterizing the degree of progression of chronic liver disease. Histopathological diagnosis is the gold standard for the interpretation of fibrosis parameters. Conventional hematoxylin-eosin (H&E) staining can only reflect the gross structure of the tissue and the distribution of hepatocytes, while Masson trichrome can highlight specific types of collagen fiber structure, thus providing the necessary structural information for fibrosis scoring. However, the expensive costs of time, economy, and patient specimens as well as the non-uniform preparation and staining process make the conversion of existing H&E staining into virtual Masson trichrome staining a solution for fibrosis evaluation. Existing translation approaches fail to extract fiber features accurately enough, and the decoder of staining is unable to converge due to the inconsistent color of physical staining. In this work, we propose a prior-guided generative adversarial network, based on unpaired data for effective Masson trichrome stained image generation from the corresponding H&E stained image. Conducted on a small training set, our method takes full advantage of prior knowledge to set up better constraints on both the encoder and the decoder. Experiments indicate the superior performance of our method that surpasses the previous approaches. For various liver diseases, our results demonstrate a high correlation between the staging of real and virtual stains (ρ=0.82; 95% CI: 0.73-0.89). In addition, our finetuning strategy is able to standardize the staining color and release the memory and computational burden, which can be employed in clinical assessment.

A utility-based machine learning-driven personalized lifestyle recommendation for cardiovascular disease prevention

In recent decades, cardiovascular disease (CVD) has become the leading cause of death in most countries of the world. Since many types of CVD are preventable by modifying lifestyle behaviors, the objective of this paper is to develop an effective personalized lifestyle recommendation algorithm for reducing the risk of common types of CVD. However, in practice, the underlying relationships between the risk factors (e.g., lifestyles, blood pressure, etc.) and disease onset is highly complex. It is also challenging to identify effective modification recommendations for different individuals due to individual's effort-benefits consideration and uncertainties in disease progression. Therefore, to address these challenges, this study developed a novel data-driven approach for personalized lifestyle behaviors recommendation based on machine learning and a personalized exponential utility function model. The contributions of this work can be summarized into three aspects: (1) a classification-based prediction model is implemented to predict the CVD risk based on the condition of risk factors; (2) the generative adversarial network (GAN) is incorporated to learn the underlying relationship between risk factors, as well as quantifying the uncertainty of disease progression under lifestyle modifications; and (3) a novel personalized exponential utility function model is proposed to evaluate the modifications' utilities with respect to CVD risk reduction, individual's effort-benefits consideration, and disease progression uncertainty, as well as identify the optimal modification for each individual. The effectiveness of the proposed method is validated through an open-access CVD dataset. The results demonstrate that the personalized lifestyle modification recommended by the proposed methodology has the potential to effectively reduce the CVD risk. Thus, it is promising to be further applied to real-world cases for CVD prevention.

A novel GCL hybrid classification model for paddy diseases

The demand for agricultural products increased exponentially as the global population grew. The rapid development of computer vision-based artificial intelligence and deep learning-related technologies has impacted a wide range of industries, including disease detection and classification. This paper introduces a novel neural network-based hybrid model (GCL). GCL is a dataset-augmentation fusion of long-short term memory (LSTM) and convolutional neural network (CNN) with generative adversarial network (GAN). GAN is used for the augmentation of the dataset, CNN extracts the features and LSTM classifies the various paddy diseases. The GCL model is being investigated to improve the classification model's accuracy and reliability. The dataset was compiled using secondary resources such as Mendeley, Kaggle, UCI, and GitHub, having images of bacterial blight, leaf smut, and rice blast. The experimental setup for proving the efficacy of the GCL model demonstrates that the GCL is suitable for disease classification and works with 97% testing accuracy. GCL can further be used for the classification of more diseases of paddy.

Polygon generation and video-to-video translation for time-series prediction

This paper proposes an innovative method for time-series prediction in energy-intensive industrial systems characterized by highly dynamic non-linear operations. The proposed method can capture the true distributions of the inputs and outputs of such systems and map these distributions using polygon generation and video-to-video translation techniques. More specifically, the time-series data are represented as polygon streams (videos), then the video-to-video translation is used to transform the input polygon streams into the output ones. This transformation is tuned based on a model trustworthiness metric for optimal video synthesis. Finally, an image processing procedure is used for mapping the output polygon streams back to time-series outputs. The proposed method is based on cycle-consistent generative adversarial networks as an unsupervised approach. This does not need the heavy involvement of the human expert who devotes much effort to labeling the complex industrial data. The performance of the proposed method was validated successfully using a challenging industrial dataset collected from a complex heat exchanger network in a Canadian pulp mill. The results obtained using the proposed method demonstrate better performance than other comparable time-series prediction models. This allows process operators to accurately monitor process key performance indicators (KPIs) and to achieve a more energy-efficient operation.

Attention-based generative adversarial network in medical imaging: A narrative review

As a popular probabilistic generative model, generative adversarial network (GAN) has been successfully used not only in natural image processing, but also in medical image analysis and computer-aided diagnosis. Despite the various advantages, the applications of GAN in medical image analysis face new challenges. The introduction of attention mechanisms, which resemble the human visual system that focuses on the task-related local image area for certain information extraction, has drawn increasing interest. Recently proposed transformer-based architectures that leverage self-attention mechanism encode long-range dependencies and learn representations that are highly expressive. This motivates us to summarize the applications of using transformer-based GAN for medical image analysis. We reviewed recent advances in techniques combining various attention modules with different adversarial training schemes, and their applications in medical segmentation, synthesis and detection. Several recent studies have shown that attention modules can be effectively incorporated into a GAN model in detecting lesion areas and extracting diagnosis-related feature information precisely, thus providing a useful tool for medical image processing and diagnosis. This review indicates that research on the medical imaging analysis of GAN and attention mechanisms is still at an early stage despite the great potential. We highlight the attention-based generative adversarial network is an efficient and promising computational model advancing future research and applications in medical image analysis.

Enhancing deep learning based classifiers with inpainting anatomical side markers (L/R markers) for multi-center trials

BACKGROUND AND OBJECTIVE

The protocol for placing anatomical side markers (L/R markers) in chest radiographs varies from one hospital or department to another. However, the markers have strong signals that can be useful for deep learning-based classifier to predict diseases. We aimed to enhance the performance of a deep learning-based classifiers in multi-center datasets by inpainting the L/R markers.

METHODS

The L/R marker was detected with using the EfficientDet detection network; only the detected regions were inpainted using a generative adversarial network (GAN). To analyze the effect of the inpainting in detail, deep learning-based classifiers were trained using original images, marker-inpainted images, and original images clipped using the min-max value of the marker-inpainted images. Binary classification, multi-class classification, and multi-task learning with segmentation and classification were developed and evaluated. Furthermore, the performances of the network on internal and external validation datasets were compared using DeLong's test for two correlated receiver operating characteristic (ROC) curves in binary classification and Stuart-Maxwell test for marginal homogeneity in multi-class classification and multi-task learning. In addition, the qualitative results of activation maps were evaluated using the gradient-class activation map (Grad-CAM).

RESULTS

Marker-inpainting preprocessing improved the classification performances. In the binary classification based on the internal validation, the area under the curves (AUCs) and accuracies were 0.950 and 0.900 for the model trained on the min-max clipped images and 0.911 and 0.850 for the model trained on the original images, respectively (P-value=0.006). In the external validation, the AUCs and accuracies were 0.858 and 0.677 for the model trained using the inpainted images and 0.723 and 0.568 for the model trained using the original images (P-value<0.001), respectively. In addition, the models trained using the marker inpainted images showed the best performance in multi-class classification and multi-task learning. Furthermore, the activation maps obtained using the Grad-CAM improved with the proposed method. The 5-fold validation results also showed improvement trend according to the preprocessing strategies.

CONCLUSIONS

Inpainting an L/R marker significantly enhanced the classifier's performance and robustness, especially in internal and external studies, which could be useful in developing a more robust and accurate deep learning-based classifier for multi-center trials. The code for detection is available at: https://github.com/mi2rl/MI2RLNet. And the code for inpainting is available at: https://github.com/mi2rl/L-R-marker-inpainting.

Generating future fundus images for early age-related macular degeneration based on generative adversarial networks

BACKGROUND AND OBJECTIVE

Age-related macular degeneration (AMD) is one of the most common diseases that can lead to blindness worldwide. Recently, various fundus image analyzing studies are done using deep learning methods to classify fundus images to aid diagnosis and monitor AMD disease progression. But until now, to the best of our knowledge, no attempt was made to generate future synthesized fundus images that can predict AMD progression. In this paper, we developed a deep learning model using fundus images for AMD patients with different time elapses to generate synthetic future fundus images.

METHOD

We exploit generative adversarial networks (GANs) with additional drusen masks to maintain the pathological information. The dataset included 8196 fundus images from 1263 AMD patients. A proposed GAN-based model, called Multi-Modal GAN (MuMo-GAN), was trained to generate synthetic predicted-future fundus images.

RESULTS

The proposed deep learning model indicates that the additional drusen masks can help to learn the AMD progression. Our model can generate future fundus images with appropriate pathological features. The drusen development over time is depicted well. Both qualitative and quantitative experiments show that our model is more efficient to monitor the AMD disease as compared to other studies.

CONCLUSION

This study could help individualized risk prediction for AMD patients. Compared to existing methods, the experimental results show a significant improvement in terms of tracking the AMD stage in both image-level and pixel-level.

Adaptive rectification based adversarial network with spectrum constraint for high-quality PET image synthesis

Positron emission tomography (PET) is a typical nuclear imaging technique, which can provide crucial functional information for early brain disease diagnosis. Generally, clinically acceptable PET images are obtained by injecting a standard-dose radioactive tracer into human body, while on the other hand the cumulative radiation exposure inevitably raises concerns about potential health risks. However, reducing the tracer dose will increase the noise and artifacts of the reconstructed PET image. For the purpose of acquiring high-quality PET images while reducing radiation exposure, in this paper, we innovatively present an adaptive rectification based generative adversarial network with spectrum constraint, named AR-GAN, which uses low-dose PET (LPET) image to synthesize standard-dose PET (SPET) image of high-quality. Specifically, considering the existing differences between the synthesized SPET image by traditional GAN and the real SPET image, an adaptive rectification network (AR-Net) is devised to estimate the residual between the preliminarily predicted image and the real SPET image, based on the hypothesis that a more realistic rectified image can be obtained by incorporating both the residual and the preliminarily predicted PET image. Moreover, to address the issue of high-frequency distortions in the output image, we employ a spectral regularization term in the training optimization objective to constrain the consistency of the synthesized image and the real image in the frequency domain, which further preserves the high-frequency detailed information and improves synthesis performance. Validations on both the phantom dataset and the clinical dataset show that the proposed AR-GAN can estimate SPET images from LPET images effectively and outperform other state-of-the-art image synthesis approaches.

Greenhouse gas emission estimation from municipal wastewater using a hybrid approach of generative adversarial network and data-driven modelling

Greenhouse gas (GHG) emissions including carbon dioxide (CO₂), methane (CH₄), and nitrous oxide (N₂O) created via wastewater treatment processes are not easily modeled given the non-linearity and complexity of biological processes. These factors are also impacted by limited data availability making the development of artificial data generation algorithms, such as a generative adversarial network (GAN), useful for determination of GHG emission rate estimates (EREs). The main objective of this study was to develop a hybrid approach of using GAN and regression modelling to determine GHG EREs from a cold-region biological nutrient removal (BNR) municipal wastewater treatment plant (MWTP) in which the aerobic reactor has previously been established as the main GHG emission source. To our knowledge, this is the first application of GAN used for MWTP modelling purposes. The EREs were generated from laboratory-scale reactors used in conjunction with facility-monitored operating parameters to develop the GAN and regression models. Results showed that regression models provided reasonable EREs using parameters including hydraulic retention time (HRT), temperature, total organic carbon, and dissolved oxygen (DO) concentrations for CO₂ EREs; HRT, temperature, DO and phosphate (PO₄^3-) concentrations for CH₄ EREs; and temperature, DO, and nitrogen (nitrite, nitrate, and ammonium) concentrations for N₂O EREs. Additionally, the addition of 100 GAN-created virtual data points improved regression model metrics including increased correlation coefficient and index agreement values, and decreased root mean square error values. Clearly, virtual data augmentation using GAN is a valuable resource in supplementation of limited data for improved modelling outcomes. Genetic algorithm optimization was also used to determine operating parameter modifications resulting in potential for minimization (or maximization) of GHG emissions.

Synthesis of standard 12‑lead electrocardiograms using two-dimensional generative adversarial networks

This paper proposes a two-dimensional (2D) bidirectional long short-term memory generative adversarial network (GAN) to produce synthetic standard 12-lead ECGs corresponding to four types of signals-left ventricular hypertrophy (LVH), left branch bundle block (LBBB), acute myocardial infarction (ACUTMI), and Normal. It uses a fully automatic end-to-end process to generate and verify the synthetic ECGs that does not require any visual inspection. The proposed model is able to produce synthetic standard 12-lead ECG signals with success rates of 98% for LVH, 93% for LBBB, 79% for ACUTMI, and 59% for Normal. Statistical evaluation of the data confirms that the synthetic ECGs are not biased towards or overfitted to the training ECGs, and span a wide range of morphological features. This study demonstrates that it is feasible to use a 2D GAN to produce standard 12-lead ECGs suitable to augment artificially a diverse database of real ECGs, thus providing a possible solution to the demand for extensive ECG datasets.

Narrative review of generative adversarial networks in medical and molecular imaging

Recent years have witnessed a rapidly expanding use of artificial intelligence and machine learning in medical imaging. Generative adversarial networks (GANs) are techniques to synthesize images based on artificial neural networks and deep learning. In addition to the flexibility and versatility inherent in deep learning on which the GANs are based, the potential problem-solving ability of the GANs has attracted attention and is being vigorously studied in the medical and molecular imaging fields. Here this narrative review provides a comprehensive overview for GANs and discuss their usefulness in medical and molecular imaging on the following topics: (I) data augmentation to increase training data for AI-based computer-aided diagnosis as a solution for the data-hungry nature of such training sets; (II) modality conversion to complement the shortcomings of a single modality that reflects certain physical measurement principles, such as from magnetic resonance (MR) to computed tomography (CT) images or vice versa; (III) de-noising to realize less injection and/or radiation dose for nuclear medicine and CT; (IV) image reconstruction for shortening MR acquisition time while maintaining high image quality; (V) super-resolution to produce a high-resolution image from low-resolution one; (VI) domain adaptation which utilizes knowledge such as supervised labels and annotations from a source domain to the target domain with no or insufficient knowledge; and (VII) image generation with disease severity and radiogenomics. GANs are promising tools for medical and molecular imaging. The progress of model architectures and their applications should continue to be noteworthy.

Half-scan artifact correction using generative adversarial network for dental CT

Half-scan image reconstruction with Parker weighting can correct motion artifacts in dental CT images taken with a slow scan-based dental CT. Since the residual half-scan artifacts in the dental CT images appear much stronger than those in medical CT images, the artifacts often persist to the extent that they compromise the surface-rendered bone and tooth images computed from the dental CT images. We used a variation of generative adversarial network (GAN), so-called U-WGAN, to correct half-scan artifacts in dental CT images. For the generative network of GAN, we used a U-net structure of five stages to take advantage of its high computational efficiency. We trained the network using the Wasserstein loss function on the dental CT images of 40 patients. We tested the network with comparing its output images to the half-scan images corrected with other methods; Parker weighting and the other two popular GANs, that is, SRGAN and m-WGAN. For the quantitative comparison, we used the image quality metrics measuring the similarity of the corrected images to the full-scan images (reference images) and the noise level on the corrected images. We also compared the visual quality of the surface-rendered bone and tooth images. We observed that the proposed network outperformed Parker weighting and other GANs in all the image quality metrics. The computation time for the proposed network to process 336×336×336 3D images on a GPU-equipped personal computer was about 3 s, which was much shorter than those of SRGAN and m-WGAN, 50 s and 54 s, respectively.

The optimisation of deep neural networks for segmenting multiple knee joint tissues from MRIs

Automated semantic segmentation of multiple knee joint tissues is desirable to allow faster and more reliable analysis of large datasets and to enable further downstream processing e.g. automated diagnosis. In this work, we evaluate the use of conditional Generative Adversarial Networks (cGANs) as a robust and potentially improved method for semantic segmentation compared to other extensively used convolutional neural network, such as the U-Net. As cGANs have not yet been widely explored for semantic medical image segmentation, we analysed the effect of training with different objective functions and discriminator receptive field sizes on the segmentation performance of the cGAN. Additionally, we evaluated the possibility of using transfer learning to improve the segmentation accuracy. The networks were trained on i) the SKI10 dataset which comes from the MICCAI grand challenge "Segmentation of Knee Images 2010″, ii) the OAI ZIB dataset containing femoral and tibial bone and cartilage segmentations of the Osteoarthritis Initiative cohort and iii) a small locally acquired dataset (Advanced MRI of Osteoarthritis (AMROA) study) consisting of 3D fat-saturated spoiled gradient recalled-echo knee MRIs with manual segmentations of the femoral, tibial and patellar bone and cartilage, as well as the cruciate ligaments and selected peri-articular muscles. The Sørensen-Dice Similarity Coefficient (DSC), volumetric overlap error (VOE) and average surface distance (ASD) were calculated for segmentation performance evaluation. DSC ≥ 0.95 were achieved for all segmented bone structures, DSC ≥ 0.83 for cartilage and muscle tissues and DSC of ≈0.66 were achieved for cruciate ligament segmentations with both cGAN and U-Net on the in-house AMROA dataset. Reducing the receptive field size of the cGAN discriminator network improved the networks segmentation performance and resulted in segmentation accuracies equivalent to those of the U-Net. Pretraining not only increased segmentation accuracy of a few knee joint tissues of the fine-tuned dataset, but also increased the network's capacity to preserve segmentation capabilities for the pretrained dataset. cGAN machine learning can generate automated semantic maps of multiple tissues within the knee joint which could increase the accuracy and efficiency for evaluating joint health.

Performance of deep learning for differentiating pancreatic diseases on contrast-enhanced magnetic resonance imaging: A preliminary study

PURPOSE

The purpose of this study was to evaluate the ability of deep learning to differentiate pancreatic diseases on contrast-enhanced magnetic resonance (MR) images with the aid of generative adversarial network (GAN).

MATERIALS AND METHODS

A total of 504 patients who underwent T1-weighted contrast-enhanced MR examinations before any treatments were included in this retrospective study. First, the MRI examinations of 398 patients (215 men, 183 women; mean age, 59.14±12.07 [SD] years [range: 16-85 years]) from one hospital were used as the training set. Then the MRI examinations of 50 (26 men, 24women; mean age, 58.58±13.64 [SD] years [range: 24-85 years]) and 56 (30 men, 26 women; mean age, 59.13±11.35 [SD] years [range: 26-80 years]) consecutive patients from two hospitals were separately collected as the internal and external validation sets. An InceptionV4 network was trained on the training set augmented by synthetic images from GANs. Classification performance of trained InceptionV4 network for every patch and every patient were made on both validation sets, respectively. The prediction agreement between convolutional neural network (CNN) and radiologist was measured by the Cohen's kappa coefficient.

RESULTS

The patch-level average accuracy and the micro-averaging area under receiver operating characteristic curve (AUC) of InceptionV4 network were 71.56% and 0.9204 (95% confidence interval [CI]: 0.9165-0.9308) for the internal validation set, and 79.46% and 0.9451 (95%CI: 0.9320-0.9523) for the external validation set, respectively. The patient-level average accuracy and the micro-averaging AUC of InceptionV4 network were 70.00% and 0.8250 (95%CI: 0.8147-0.8326) for the internal validation, 76.79% and 0.8646 (95%CI: 0.8489-0.8772) for the external validation set, respectively. Evaluated by human reader, the average accuracy and micro-averaging AUC for internal and external validation sets were 82.00% and 0.8950 (95%CI: 0.8817-0.9083), 83.93% and 0.9063 (95%CI: 0.8968-0.9212), respectively. The Cohen's kappa coefficients between InceptionV4 network and human reader for the internal and external invalidation sets were 0.8339 (95%CI: 0.6991-0.9447) and 0.8862 (95%CI: 0.7759-0.9738), respectively.

CONCLUSION

Deep learning using CNN and GAN had the potential to differentiate pancreatic diseases on contrast-enhanced MR images.

Towards cross-modal organ translation and segmentation: A cycle- and shape-consistent generative adversarial network

Synthesized medical images have several important applications. For instance, they can be used as an intermedium in cross-modality image registration or used as augmented training samples to boost the generalization capability of a classifier. In this work, we propose a generic cross-modality synthesis approach with the following targets: 1) synthesizing realistic looking 2D/3D images without needing paired training data, 2) ensuring consistent anatomical structures, which could be changed by geometric distortion in cross-modality synthesis and 3) more importantly, improving volume segmentation by using synthetic data for modalities with limited training samples. We show that these goals can be achieved with an end-to-end 2D/3D convolutional neural network (CNN) composed of mutually-beneficial generators and segmentors for image synthesis and segmentation tasks. The generators are trained with an adversarial loss, a cycle-consistency loss, and also a shape-consistency loss (supervised by segmentors) to reduce the geometric distortion. From the segmentation view, the segmentors are boosted by synthetic data from generators in an online manner. Generators and segmentors prompt each other alternatively in an end-to-end training fashion. We validate our proposed method on three datasets, including cardiovascular CT and magnetic resonance imaging (MRI), abdominal CT and MRI, and mammography X-rays from different data domains, showing both tasks are beneficial to each other and coupling these two tasks results in better performance than solving them exclusively.