SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truth

Phase identification for dynamic CT enhancements with generative adversarial network

PURPOSE

Dynamic contrast-enhanced computed tomography (CT) is widely used to provide dynamic tissue contrast for diagnostic investigation and vascular identification. However, the phase information of contrast injection is typically recorded manually by technicians, which introduces missing or mislabeling. Hence, imaging-based contrast phase identification is appealing, but challenging, due to large variations among different contrast protocols, vascular dynamics, and metabolism, especially for clinically acquired CT scans. The purpose of this study is to perform imaging-based phase identification for dynamic abdominal CT using a proposed adversarial learning framework across five representative contrast phases.

METHODS

A generative adversarial network (GAN) is proposed as a disentangled representation learning model. To explicitly model different contrast phases, a low dimensional common representation and a class specific code are fused in the hidden layer. Then, the low dimensional features are reconstructed following a discriminator and classifier. 36 350 slices of CT scans from 400 subjects are used to evaluate the proposed method with fivefold cross-validation with splits on subjects. Then, 2216 slices images from 20 independent subjects are employed as independent testing data, which are evaluated using multiclass normalized confusion matrix.

RESULTS

The proposed network significantly improved correspondence (0.93) over VGG, ResNet50, StarGAN, and 3DSE with accuracy scores 0.59, 0.62, 0.72, and 0.90, respectively (P < 0.001 Stuart-Maxwell test for normalized multiclass confusion matrix).

CONCLUSION

We show that adversarial learning for discriminator can be benefit for capturing contrast information among phases. The proposed discriminator from the disentangled network achieves promising results.

Deep Ensemble Feature Network for Gastric Section Classification

In this paper, we propose a novel deep ensemble feature (DEF) network to classify gastric sections from endoscopic images. Different from recent deep ensemble learning methods, which need to train deep features and classifiers individually to obtain fused classification results, the proposed method can simultaneously learn the deep ensemble feature from arbitrary number of convolutional neural networks (CNNs) and the decision classifier in an end-to-end trainable manner. It comprises two sub networks, the ensemble feature network and the decision network. The former sub network learns the deep ensemble feature from multiple CNNs to represent endoscopic images. The latter sub network learns to obtain the classification labels by using the deep ensemble feature. Both sub networks are optimized based on the proposed ensemble feature loss and the decision loss which guide the learning of deep features and decisions. As shown in the experimental results, the proposed method outperforms the state-of-the-art deep learning, ensemble learning, and deep ensemble learning methods.

Validation and estimation of spleen volume via computer-assisted segmentation on clinically acquired CT scans

Purpose: Deep learning is a promising technique for spleen segmentation. Our study aims to validate the reproducibility of deep learning-based spleen volume estimation by performing spleen segmentation on clinically acquired computed tomography (CT) scans from patients with myeloproliferative neoplasms. Approach: As approved by the institutional review board, we obtained 138 de-identified abdominal CT scans. A sum of voxel volume on an expert annotator's segmentations establishes the ground truth (estimation 1). We used our deep convolutional neural network (estimation 2) alongside traditional linear estimations (estimation 3 and 4) to estimate spleen volumes independently. Dice coefficient, Hausdorff distance,R 2 coefficient, Pearson R coefficient, the absolute difference in volume, and the relative difference in volume were calculated for 2 to 4 against the ground truth to compare and assess methods' performances. We re-labeled on scan-rescan on a subset of 40 studies to evaluate method reproducibility. Results: Calculated against the ground truth, theR 2 coefficients for our method (estimation 2) and linear method (estimation 3 and 4) are 0.998, 0.954, and 0.973, respectively. The Pearson R coefficients for the estimations against the ground truth are 0.999, 0.963, and 0.978, respectively (paired t -tests produced p < 0.05 between 2 and 3, and 2 and 4). Conclusion: The deep convolutional neural network algorithm shows excellent potential in rendering more precise spleen volume estimations. Our computer-aided segmentation exhibits reasonable improvements in splenic volume estimation accuracy.

RA-SIFA: Unsupervised domain adaptation multi-modality cardiac segmentation network combining parallel attention module and residual attention unit

BACKGROUND

Convolutional neural network has achieved a profound effect on cardiac image segmentation. The diversity of medical imaging equipment brings the challenge of domain shift for cardiac image segmentation.

OBJECTIVE

In order to solve the domain shift existed in multi-modality cardiac image segmentation, this study aims to investigate and test an unsupervised domain adaptation network RA-SIFA, which combines a parallel attention module (PAM) and residual attention unit (RAU).

METHODS

First, the PAM is introduced in the generator of RA-SIFA to fuse global information, which can reduce the domain shift from the respect of image alignment. Second, the shared encoder adopts the RAU, which has residual block based on the spatial attention module to alleviate the problem that the convolution layer is insensitive to spatial position. Therefore, RAU enables to further reduce the domain shift from the respect of feature alignment. RA-SIFA model can realize the unsupervised domain adaption (UDA) through combining the image and feature alignment, and then solve the domain shift of cardiac image segmentation in a complementary manner.

RESULTS

The model is evaluated using MM-WHS2017 datasets. Compared with SIFA, the Dice of our new RA-SIFA network is improved by 8.4%and 3.2%in CT and MR images, respectively, while, the average symmetric surface distance (ASD) is reduced by 3.4 and 0.8mm in CT and MR images, respectively.

CONCLUSION

The study results demonstrate that our new RA-SIFA network can effectively improve the accuracy of whole-heart segmentation from CT and MR images.

Anatomy-Regularized Representation Learning for Cross-Modality Medical Image Segmentation

An increasing number of studies are leveraging unsupervised cross-modality synthesis to mitigate the limited label problem in training medical image segmentation models. They typically transfer ground truth annotations from a label-rich imaging modality to a label-lacking imaging modality, under an assumption that different modalities share the same anatomical structure information. However, since these methods commonly use voxel/pixel-wise cycle-consistency to regularize the mappings between modalities, high-level semantic information is not necessarily preserved. In this paper, we propose a novel anatomy-regularized representation learning approach for segmentation-oriented cross-modality image synthesis. It learns a common feature encoding across different modalities to form a shared latent space, where 1) the input and its synthesis present consistent anatomical structure information, and 2) the transformation between two images in one domain is preserved by their syntheses in another domain. We applied our method to the tasks of cross-modality skull segmentation and cardiac substructure segmentation. Experimental results demonstrate the superiority of our method in comparison with state-of-the-art cross-modality medical image segmentation methods.

PSIGAN: Joint Probabilistic Segmentation and Image Distribution Matching for Unpaired Cross-Modality Adaptation-Based MRI Segmentation

We developed a new joint probabilistic segmentation and image distribution matching generative adversarial network (PSIGAN) for unsupervised domain adaptation (UDA) and multi-organ segmentation from magnetic resonance (MRI) images. Our UDA approach models the co-dependency between images and their segmentation as a joint probability distribution using a new structure discriminator. The structure discriminator computes structure of interest focused adversarial loss by combining the generated pseudo MRI with probabilistic segmentations produced by a simultaneously trained segmentation sub-network. The segmentation sub-network is trained using the pseudo MRI produced by the generator sub-network. This leads to a cyclical optimization of both the generator and segmentation sub-networks that are jointly trained as part of an end-to-end network. Extensive experiments and comparisons against multiple state-of-the-art methods were done on four different MRI sequences totalling 257 scans for generating multi-organ and tumor segmentation. The experiments included, (a) 20 T1-weighted (T1w) in-phase mdixon and (b) 20 T2-weighted (T2w) abdominal MRI for segmenting liver, spleen, left and right kidneys, (c) 162 T2-weighted fat suppressed head and neck MRI (T2wFS) for parotid gland segmentation, and (d) 75 T2w MRI for lung tumor segmentation. Our method achieved an overall average DSC of 0.87 on T1w and 0.90 on T2w for the abdominal organs, 0.82 on T2wFS for the parotid glands, and 0.77 on T2w MRI for lung tumors.

A Multifeature Extraction Method Using Deep Residual Network for MR Image Denoising

In order to improve the resolution of magnetic resonance (MR) image and reduce the interference of noise, a multifeature extraction denoising algorithm based on a deep residual network is proposed. First, the feature extraction layer is constructed by combining three different sizes of convolution kernels, which are used to obtain multiple shallow features for fusion and increase the network's multiscale perception ability. Then, it combines batch normalization and residual learning technology to accelerate and optimize the deep network, while solving the problem of internal covariate transfer in deep learning. Finally, the joint loss function is defined by combining the perceptual loss and the traditional mean square error loss. When the network is trained, it can not only be compared at the pixel level but also be learned at a higher level of semantic features to generate a clearer target image. Based on the MATLAB simulation platform, the TCGA-GBM and CH-GBM datasets are used to experimentally demonstrate the proposed algorithm. The results show that when the image size is set to 190 × 215 and the optimization algorithm is Adam, the performance of the proposed algorithm is the best, and its denoising effect is significantly better than other comparison algorithms. Especially under high-intensity noise levels, the denoising advantage is more prominent.

Self-derived organ attention for unpaired CT-MRI deep domain adaptation based MRI segmentation

To develop and evaluate a deep learning method to segment parotid glands from MRI using unannotated MRI and unpaired expert-segmented CT datasets. We introduced a new self-derived organ attention deep learning network for combined CT to MRI image-to-image translation (I2I) and MRI segmentation, all trained as an end-to-end network. The expert segmentations available on CT scans were combined with the I2I translated pseudo MR images to train the MRI segmentation network. Once trained, the MRI segmentation network alone is required. We introduced an organ attention discriminator that constrains the CT to MR generator to synthesize pseudo MR images that preserve organ geometry and appearance statistics as in real MRI. The I2I translation network training was regularized using the organ attention discriminator, global image-matching discriminator, and cycle consistency losses. MRI segmentation training was regularized by using cross-entropy loss. Segmentation performance was compared against multiple domain adaptation-based segmentation methods using the Dice similarity coefficient (DSC) and Hausdorff distance at the 95th percentile (HD95). All networks were trained using 85 unlabeled T2-weighted fat suppressed (T2wFS) MRIs and 96 expert-segmented CT scans. Performance upper-limit was based on fully supervised MRI training done using the 85 T2wFS MRI with expert segmentations. Independent evaluation was performed on 77 MRIs never used in training. The proposed approach achieved the highest accuracy (left parotid: DSC 0.82 ± 0.03, HD95 2.98 ± 1.01 mm; right parotid: 0.81 ± 0.05, HD95 3.14 ± 1.17 mm) compared to other methods. This accuracy was close to the reference fully supervised MRI segmentation (DSC of 0.84 ± 0.04, a HD95 of 2.24 ± 0.77 mm for the left parotid, and a DSC of 0.84 ± 0.06 and HD95 of 2.32 ± 1.37 mm for the right parotid glands).

Unified cross-modality feature disentangler for unsupervised multi-domain MRI abdomen organs segmentation

Our contribution is a unified cross-modality feature disentagling approach for multi-domain image translation and multiple organ segmentation. Using CT as the labeled source domain, our approach learns to segment multi-modal (T1-weighted and T2-weighted) MRI having no labeled data. Our approach uses a variational auto-encoder (VAE) to disentangle the image content from style. The VAE constrains the style feature encoding to match a universal prior (Gaussian) that is assumed to span the styles of all the source and target modalities. The extracted image style is converted into a latent style scaling code, which modulates the generator to produce multi-modality images according to the target domain code from the image content features. Finally, we introduce a joint distribution matching discriminator that combines the translated images with task-relevant segmentation probability maps to further constrain and regularize image-to-image (I2I) translations. We performed extensive comparisons to multiple state-of-the-art I2I translation and segmentation methods. Our approach resulted in the lowest average multi-domain image reconstruction error of 1.34±0.04. Our approach produced an average Dice similarity coefficient (DSC) of 0.85 for T1w and 0.90 for T2w MRI for multi-organ segmentation, which was highly comparable to a fully supervised MRI multi-organ segmentation network (DSC of 0.86 for T1w and 0.90 for T2w MRI).

AMD-GAN: Attention encoder and multi-branch structure based generative adversarial networks for fundus disease detection from scanning laser ophthalmoscopy images

The scanning laser ophthalmoscopy (SLO) has become an important tool for the determination of peripheral retinal pathology, in recent years. However, the collected SLO images are easily interfered by the eyelash and frame of the devices, which heavily affect the key feature extraction of the images. To address this, we propose a generative adversarial network called AMD-GAN based on the attention encoder (AE) and multi-branch (MB) structure for fundus disease detection from SLO images. Specifically, the designed generator consists of two parts: the AE and generation flow network, where the real SLO images are encoded by the AE module to extract features and the generation flow network to handle the random Gaussian noise by a series of residual block with up-sampling (RU) operations to generate fake images with the same size as the real ones, where the AE is also used to mine features for generator. For discriminator, a ResNet network using MB is devised by copying the stage 3 and stage 4 structures of the ResNet-34 model to extract deep features. Furthermore, the depth-wise asymmetric dilated convolution is leveraged to extract local high-level contextual features and accelerate the training process. Besides, the last layer of discriminator is modified to build the classifier to detect the diseased and normal SLO images. In addition, the prior knowledge of experts is utilized to improve the detection results. Experimental results on the two local SLO datasets demonstrate that our proposed method is promising in detecting the diseased and normal SLO images with the experts labeling.

High tissue contrast image synthesis via multistage attention-GAN: Application to segmenting brain MR scans

Magnetic resonance imaging (MRI) presents a detailed image of the internal organs via a magnetic field. Given MRI's non-invasive advantage in repeated imaging, the low-contrast MR images in the target area make segmentation of tissue a challenging problem. This study shows the potential advantages of synthetic high tissue contrast (HTC) images through image-to-image translation techniques. Mainly, we use a novel cycle generative adversarial network (Cycle-GAN), which provides a mechanism of attention to increase the contrast within the tissue. The attention block and training on HTC images are beneficial to our model to enhance tissue visibility. We use a multistage architecture to concentrate on a single tissue as a preliminary and filter out the irrelevant context in every stage in order to increase the resolution of HTC images. The multistage architecture reduces the gap between source and target domains and alleviates synthetic images' artefacts. We apply our HTC image synthesising method to two public datasets. In order to validate the effectiveness of these images we use HTC MR images in both end-to-end and two-stage segmentation structures. The experiments on three segmentation baselines on BraTS'18 demonstrate that joining the synthetic HTC images in the multimodal segmentation framework develops the average Dice similarity scores (DSCs) of 0.8%, 0.6%, and 0.5% respectively on the whole tumour (WT), tumour core (TC), and enhancing tumour (ET) while removing one real MRI channels from the segmentation pipeline. Moreover, segmentation of infant brain tissue in T1w MR slices through our framework improves DSCs approximately 1% in cerebrospinal fluid (CSF), grey matter (GM), and white matter (WM) compared to state-of-the-art segmentation techniques. The source code of synthesising HTC images is publicly available.

A multimodal computational pipeline for 3D histology of the human brain

Ex vivo imaging enables analysis of the human brain at a level of detail that is not possible in vivo with MRI. In particular, histology can be used to study brain tissue at the microscopic level, using a wide array of different stains that highlight different microanatomical features. Complementing MRI with histology has important applications in ex vivo atlas building and in modeling the link between microstructure and macroscopic MR signal. However, histology requires sectioning tissue, hence distorting its 3D structure, particularly in larger human samples. Here, we present an open-source computational pipeline to produce 3D consistent histology reconstructions of the human brain. The pipeline relies on a volumetric MRI scan that serves as undistorted reference, and on an intermediate imaging modality (blockface photography) that bridges the gap between MRI and histology. We present results on 3D histology reconstruction of whole human hemispheres from two donors.

Causality matters in medical imaging

Causal reasoning can shed new light on the major challenges in machine learning for medical imaging: scarcity of high-quality annotated data and mismatch between the development dataset and the target environment. A causal perspective on these issues allows decisions about data collection, annotation, preprocessing, and learning strategies to be made and scrutinized more transparently, while providing a detailed categorisation of potential biases and mitigation techniques. Along with worked clinical examples, we highlight the importance of establishing the causal relationship between images and their annotations, and offer step-by-step recommendations for future studies.

Learning from dispersed manual annotations with an optimized data weighting policy

Purpose: Deep learning methods have become essential tools for quantitative interpretation of medical imaging data, but training these approaches is highly sensitive to biases and class imbalance in the available data. There is an opportunity to increase the available training data by combining across different data sources (e.g., distinct public projects); however, data collected under different scopes tend to have differences in class balance, label availability, and subject demographics. Recent work has shown that importance sampling can be used to guide training selection. To date, these approaches have not considered imbalanced data sources with distinct labeling protocols. Approach: We propose a sampling policy, known as adaptive stochastic policy (ASP), inspired by reinforcement learning to adapt training based on subject, data source, and dynamic use criteria. We apply ASP in the context of multiorgan abdominal computed tomography segmentation. Training was performed with cross validation on 840 subjects from 10 data sources. External validation was performed with 20 subjects from 1 data source. Results: Four alternative strategies were evaluated with the state-of-the-art baseline as upper confident bound (UCB). ASP achieves average Dice of 0.8261 compared to 0.8135 UCB ( p < 0.01 , paired t -test) across fivefold cross validation. On withheld testing datasets, the proposed ASP achieved 0.8265 mean Dice versus 0.8077 UCB ( p < 0.01 , paired t -test). Conclusions: ASP provides a flexible reweighting technique for training deep learning models. We conclude that the proposed method adapts the sample importance, which leverages the performance on a challenging multisite, multiorgan, and multisize segmentation task.

Unpaired Multi-Modal Segmentation via Knowledge Distillation

Multi-modal learning is typically performed with network architectures containing modality-specific layers and shared layers, utilizing co-registered images of different modalities. We propose a novel learning scheme for unpaired cross-modality image segmentation, with a highly compact architecture achieving superior segmentation accuracy. In our method, we heavily reuse network parameters, by sharing all convolutional kernels across CT and MRI, and only employ modality-specific internal normalization layers which compute respective statistics. To effectively train such a highly compact model, we introduce a novel loss term inspired by knowledge distillation, by explicitly constraining the KL-divergence of our derived prediction distributions between modalities. We have extensively validated our approach on two multi-class segmentation problems: i) cardiac structure segmentation, and ii) abdominal organ segmentation. Different network settings, i.e., 2D dilated network and 3D U-net, are utilized to investigate our method's general efficacy. Experimental results on both tasks demonstrate that our novel multi-modal learning scheme consistently outperforms single-modal training and previous multi-modal approaches.

Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation

Unsupervised domain adaptation has increasingly gained interest in medical image computing, aiming to tackle the performance degradation of deep neural networks when being deployed to unseen data with heterogeneous characteristics. In this work, we present a novel unsupervised domain adaptation framework, named as Synergistic Image and Feature Alignment (SIFA), to effectively adapt a segmentation network to an unlabeled target domain. Our proposed SIFA conducts synergistic alignment of domains from both image and feature perspectives. In particular, we simultaneously transform the appearance of images across domains and enhance domain-invariance of the extracted features by leveraging adversarial learning in multiple aspects and with a deeply supervised mechanism. The feature encoder is shared between both adaptive perspectives to leverage their mutual benefits via end-to-end learning. We have extensively evaluated our method with cardiac substructure segmentation and abdominal multi-organ segmentation for bidirectional cross-modality adaptation between MRI and CT images. Experimental results on two different tasks demonstrate that our SIFA method is effective in improving segmentation performance on unlabeled target images, and outperforms the state-of-the-art domain adaptation approaches by a large margin.

Unified generative adversarial networks for multimodal segmentation from unpaired 3D medical images

To fully define the target objects of interest in clinical diagnosis, many deep convolution neural networks (CNNs) use multimodal paired registered images as inputs for segmentation tasks. However, these paired images are difficult to obtain in some cases. Furthermore, the CNNs trained on one specific modality may fail on others for images acquired with different imaging protocols and scanners. Therefore, developing a unified model that can segment the target objects from unpaired multiple modalities is significant for many clinical applications. In this work, we propose a 3D unified generative adversarial network, which unifies the any-to-any modality translation and multimodal segmentation in a single network. Since the anatomical structure is preserved during modality translation, the auxiliary translation task is used to extract the modality-invariant features and generate the additional training data implicitly. To fully utilize the segmentation-related features, we add a cross-task skip connection with feature recalibration from the translation decoder to the segmentation decoder. Experiments on abdominal organ segmentation and brain tumor segmentation indicate that our method outperforms the existing unified methods.

Generative Adversarial Networks and Its Applications in Biomedical Informatics

The basic Generative Adversarial Networks (GAN) model is composed of the input vector, generator, and discriminator. Among them, the generator and discriminator are implicit function expressions, usually implemented by deep neural networks. GAN can learn the generative model of any data distribution through adversarial methods with excellent performance. It has been widely applied to different areas since it was proposed in 2014. In this review, we introduced the origin, specific working principle, and development history of GAN, various applications of GAN in digital image processing, Cycle-GAN, and its application in medical imaging analysis, as well as the latest applications of GAN in medical informatics and bioinformatics.

Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation

The medical imaging literature has witnessed remarkable progress in high-performing segmentation models based on convolutional neural networks. Despite the new performance highs, the recent advanced segmentation models still require large, representative, and high quality annotated datasets. However, rarely do we have a perfect training dataset, particularly in the field of medical imaging, where data and annotations are both expensive to acquire. Recently, a large body of research has studied the problem of medical image segmentation with imperfect datasets, tackling two major dataset limitations: scarce annotations where only limited annotated data is available for training, and weak annotations where the training data has only sparse annotations, noisy annotations, or image-level annotations. In this article, we provide a detailed review of the solutions above, summarizing both the technical novelties and empirical results. We further compare the benefits and requirements of the surveyed methodologies and provide our recommended solutions. We hope this survey article increases the community awareness of the techniques that are available to handle imperfect medical image segmentation datasets.

One-Shot Generative Adversarial Learning for MRI Segmentation of Craniomaxillofacial Bony Structures

Compared to computed tomography (CT), magnetic resonance imaging (MRI) delineation of craniomaxillofacial (CMF) bony structures can avoid harmful radiation exposure. However, bony boundaries are blurry in MRI, and structural information needs to be borrowed from CT during the training. This is challenging since paired MRI-CT data are typically scarce. In this paper, we propose to make full use of unpaired data, which are typically abundant, along with a single paired MRI-CT data to construct a one-shot generative adversarial model for automated MRI segmentation of CMF bony structures. Our model consists of a cross-modality image synthesis sub-network, which learns the mapping between CT and MRI, and an MRI segmentation sub-network. These two sub-networks are trained jointly in an end-to-end manner. Moreover, in the training phase, a neighbor-based anchoring method is proposed to reduce the ambiguity problem inherent in cross-modality synthesis, and a feature-matching-based semantic consistency constraint is proposed to encourage segmentation-oriented MRI synthesis. Experimental results demonstrate the superiority of our method both qualitatively and quantitatively in comparison with the state-of-the-art MRI segmentation methods.

Reducing image variability across OCT devices with unsupervised unpaired learning for improved segmentation of retina

Diagnosis and treatment in ophthalmology depend on modern retinal imaging by optical coherence tomography (OCT). The recent staggering results of machine learning in medical imaging have inspired the development of automated segmentation methods to identify and quantify pathological features in OCT scans. These models need to be sensitive to image features defining patterns of interest, while remaining robust to differences in imaging protocols. A dominant factor for such image differences is the type of OCT acquisition device. In this paper, we analyze the ability of recently developed unsupervised unpaired image translations based on cycle consistency losses (cycleGANs) to deal with image variability across different OCT devices (Spectralis and Cirrus). This evaluation was performed on two clinically relevant segmentation tasks in retinal OCT imaging: fluid and photoreceptor layer segmentation. Additionally, a visual Turing test designed to assess the quality of the learned translation models was carried out by a group of 18 participants with different background expertise. Results show that the learned translation models improve the generalization ability of segmentation models to other OCT-vendors/domains not seen during training. Moreover, relationships between model hyper-parameters and the realism as well as the morphological consistency of the generated images could be identified.

Data augmentation using generative adversarial networks (CycleGAN) to improve generalizability in CT segmentation tasks

Labeled medical imaging data is scarce and expensive to generate. To achieve generalizable deep learning models large amounts of data are needed. Standard data augmentation is a method to increase generalizability and is routinely performed. Generative adversarial networks offer a novel method for data augmentation. We evaluate the use of CycleGAN for data augmentation in CT segmentation tasks. Using a large image database we trained a CycleGAN to transform contrast CT images into non-contrast images. We then used the trained CycleGAN to augment our training using these synthetic non-contrast images. We compared the segmentation performance of a U-Net trained on the original dataset compared to a U-Net trained on the combined dataset of original data and synthetic non-contrast images. We further evaluated the U-Net segmentation performance on two separate datasets: The original contrast CT dataset on which segmentations were created and a second dataset from a different hospital containing only non-contrast CTs. We refer to these 2 separate datasets as the in-distribution and out-of-distribution datasets, respectively. We show that in several CT segmentation tasks performance is improved significantly, especially in out-of-distribution (noncontrast CT) data. For example, when training the model with standard augmentation techniques, performance of segmentation of the kidneys on out-of-distribution non-contrast images was dramatically lower than for in-distribution data (Dice score of 0.09 vs. 0.94 for out-of-distribution vs. in-distribution data, respectively, p < 0.001). When the kidney model was trained with CycleGAN augmentation techniques, the out-of-distribution (non-contrast) performance increased dramatically (from a Dice score of 0.09 to 0.66, p < 0.001). Improvements for the liver and spleen were smaller, from 0.86 to 0.89 and 0.65 to 0.69, respectively. We believe this method will be valuable to medical imaging researchers to reduce manual segmentation effort and cost in CT imaging.

Synthesized b0 for diffusion distortion correction (Synb0-DisCo)

Diffusion magnetic resonance images typically suffer from spatial distortions due to susceptibility induced off-resonance fields, which may affect the geometric fidelity of the reconstructed volume and cause mismatches with anatomical images. State-of-the art susceptibility correction (for example, FSL's TOPUP algorithm) typically requires data acquired twice with reverse phase encoding directions, referred to as blip-up blip-down acquisitions, in order to estimate an undistorted volume. Unfortunately, not all imaging protocols include a blip-up blip-down acquisition, and cannot take advantage of the state-of-the art susceptibility and motion correction capabilities. In this study, we aim to enable TOPUP-like processing with historical and/or limited diffusion imaging data that include only a structural image and single blip diffusion image. We utilize deep learning to synthesize an undistorted non-diffusion weighted image from the structural image, and use the non-distorted synthetic image as an anatomical target for distortion correction. We evaluate the efficacy of this approach (named Synb0-DisCo) and show that our distortion correction process results in better matching of the geometry of undistorted anatomical images, reduces variation in diffusion modeling, and is practically equivalent to having both blip-up and blip-down non-diffusion weighted images.

Coronary Calcium Detection using 3D Attention Identical Dual Deep Network Based on Weakly Supervised Learning

Coronary artery calcium (CAC) is biomarker of advanced subclinical coronary artery disease and predicts myocardial infarction and death prior to age 60 years. The slice-wise manual delineation has been regarded as the gold standard of coronary calcium detection. However, manual efforts are time and resource consuming and even impracticable to be applied on large-scale cohorts. In this paper, we propose the attention identical dual network (AID-Net) to perform CAC detection using scan-rescan longitudinal non-contrast CT scans with weakly supervised attention by only using per scan level labels. To leverage the performance, 3D attention mechanisms were integrated into the AID-Net to provide complementary information for classification tasks. Moreover, the 3D Gradient-weighted Class Activation Mapping (Grad-CAM) was also proposed at the testing stage to interpret the behaviors of the deep neural network. 5075 non-contrast chest CT scans were used as training, validation and testing datasets. Baseline performance was assessed on the same cohort. From the results, the proposed AID-Net achieved the superior performance on classification accuracy (0.9272) and AUC (0.9627).