CT Male Pelvic Organ Segmentation via Hybrid Loss Network With Incomplete Annotation

Automatic Segmentation of the Cisternal Segment of Trigeminal Nerve on MRI Using Deep Learning

Purpose: Accurate segmentation of the cisternal segment of the trigeminal nerve plays a critical role in identifying and treating different trigeminal nerve-related disorders, including trigeminal neuralgia (TN). However, the current manual segmentation process is prone to interobserver variability and consumes a significant amount of time. To overcome this challenge, we propose a deep learning-based approach, U-Net, that automatically segments the cisternal segment of the trigeminal nerve. Methods: To evaluate the efficacy of our proposed approach, the U-Net model was trained and validated on healthy control images and tested in on a separate dataset of TN patients. The methods such as Dice, Jaccard, positive predictive value (PPV), sensitivity (SEN), center-of-mass distance (CMD), and Hausdorff distance were used to assess segmentation performance. Results: Our approach achieved high accuracy in segmenting the cisternal segment of the trigeminal nerve, demonstrating robust performance and comparable results to those obtained by participating radiologists. Conclusion: The proposed deep learning-based approach, U-Net, shows promise in improving the accuracy and efficiency of segmenting the cisternal segment of the trigeminal nerve. To the best of our knowledge, this is the first fully automated segmentation method for the trigeminal nerve in anatomic MRI, and it has the potential to aid in the diagnosis and treatment of various trigeminal nerve-related disorders, such as TN.

Adaptive Region-Specific Loss for Improved Medical Image Segmentation

Defining the loss function is an important part of neural network design and critically determines the success of deep learning modeling. A significant shortcoming of the conventional loss functions is that they weight all regions in the input image volume equally, despite the fact that the system is known to be heterogeneous (i.e., some regions can achieve high prediction performance more easily than others). Here, we introduce a region-specific loss to lift the implicit assumption of homogeneous weighting for better learning. We divide the entire volume into multiple sub-regions, each with an individualized loss constructed for optimal local performance. Effectively, this scheme imposes higher weightings on the sub-regions that are more difficult to segment, and vice versa. Furthermore, the regional false positive and false negative errors are computed for each input image during a training step and the regional penalty is adjusted accordingly to enhance the overall accuracy of the prediction. Using different public and in-house medical image datasets, we demonstrate that the proposed regionally adaptive loss paradigm outperforms conventional methods in the multi-organ segmentations, without any modification to the neural network architecture or additional data preparation.

LWMA-Net: Light-weighted morphology attention learning for human embryo grading

Visual inspection of embryo morphology is routinely used in embryo assessment and selection. However, due to the complexity of morphologies and large inter- and intra-observer variances among embryologists, manual evaluations remain to be subjective and time-consuming. Thus, we proposed a light-weighted morphology attention learning network (LWMA-Net) for automatic assistance on embryo grading. The LWMA-Net integrated a morphology attention module (MAM) to seek the informative features and their locations and a multiscale fusion module (MFM) to increase the features flowing in the model. The LWMA-Net was trained with a primary set of 3599 embryos from 2318 couples that were clinically enrolled between Sep. 2016 and Dec. 2018, and generated area under the receiver operating characteristic curves (AUCs) of 96.88% and 97.58% on 4- and 3-category gradings, respectively. An independent test set comprises 691 embryos from 321 couples between Jan. 2019 and Jan. 2021 were used to test the assisted fertility values on the embryo grading. Five experienced embryologists were invited to regrade the embryos in the independent set with and without the aid of the LWMA-Net three months apart. Embryologists aided by our LWMA-Net significantly improved their grading capabilities with average AUCs improved by 4.98%-5.32% on 4- and 3-category grading tasks, respectively, which suggests good potential of our LWMA-Net on assisted human reproduction.

3D U-Net Improves Automatic Brain Extraction for Isotropic Rat Brain Magnetic Resonance Imaging Data

Brain extraction is a critical pre-processing step in brain magnetic resonance imaging (MRI) analytical pipelines. In rodents, this is often achieved by manually editing brain masks slice-by-slice, a time-consuming task where workloads increase with higher spatial resolution datasets. We recently demonstrated successful automatic brain extraction via a deep-learning-based framework, U-Net, using 2D convolutions. However, such an approach cannot make use of the rich 3D spatial-context information from volumetric MRI data. In this study, we advanced our previously proposed U-Net architecture by replacing all 2D operations with their 3D counterparts and created a 3D U-Net framework. We trained and validated our model using a recently released CAMRI rat brain database acquired at isotropic spatial resolution, including T2-weighted turbo-spin-echo structural MRI and T2*-weighted echo-planar-imaging functional MRI. The performance of our 3D U-Net model was compared with existing rodent brain extraction tools, including Rapid Automatic Tissue Segmentation, Pulse-Coupled Neural Network, SHape descriptor selected External Regions after Morphologically filtering, and our previously proposed 2D U-Net model. 3D U-Net demonstrated superior performance in Dice, Jaccard, center-of-mass distance, Hausdorff distance, and sensitivity. Additionally, we demonstrated the reliability of 3D U-Net under various noise levels, evaluated the optimal training sample sizes, and disseminated all source codes publicly, with a hope that this approach will benefit rodent MRI research community. Significant Methodological Contribution: We proposed a deep-learning-based framework to automatically identify the rodent brain boundaries in MRI. With a fully 3D convolutional network model, 3D U-Net, our proposed method demonstrated improved performance compared to current automatic brain extraction methods, as shown in several qualitative metrics (Dice, Jaccard, PPV, SEN, and Hausdorff). We trust that this tool will avoid human bias and streamline pre-processing steps during 3D high resolution rodent brain MRI data analysis. The software developed herein has been disseminated freely to the community.

Semi-supervised semantic segmentation of prostate and organs-at-risk on 3D pelvic CT images

The recent development of deep learning approaches has revoluted medical data processing, including semantic segmentation, by dramatically improving performance. Automated segmentation can assist radiotherapy treatment planning by saving manual contouring efforts and reducing intra-observer and inter-observer variations. However, training effective deep learning models usually Requires a large amount of high-quality labeled data, often costly to collect. We developed a novel semi-supervised adversarial deep learning approach for 3D pelvic CT image semantic segmentation. Unlike supervised deep learning methods, the new approach can utilize both annotated and un-annotated data for training. It generates un-annotated synthetic data by a data augmentation scheme using generative adversarial networks (GANs). We applied the new approach to segmenting multiple organs in male pelvic CT images. CT images without annotations and GAN-synthesized un-annotated images were used in semi-supervised learning. Experimental results, evaluated by three metrics (Dice similarity coefficient, average Hausdorff distance, and average surface Hausdorff distance), showed that the new method achieved comparable performance with substantially fewer annotated images or better performance with the same amount of annotated data, outperforming the existing state-of-the-art methods.

Radiotherapy planning parameters correlate with changes in the peripheral immune status of patients undergoing curative radiotherapy for localized prostate cancer

Purpose

The influence of radiotherapy on patient immune cell subsets has been established by several groups. Following a previously published analysis of immune changes during and after curative radiotherapy for prostate cancer, this analysis focused on describing correlations of changes of immune cell subsets with radiation treatment parameters.

Patients and methods

For 13 patients treated in a prospective trial with radiotherapy to the prostate region (primary analysis) and five patients treated with radiotherapy to prostate and pelvic nodal regions (exploratory analysis), already published immune monitoring data were correlated with clinical data as well as radiation planning parameters such as clinical target volume (CTV) and volumes receiving 20 Gy (V20) for newly contoured volumes of pelvic blood vessels and bone marrow.

Results

Most significant changes among immune cell subsets were observed at the end of radiotherapy. In contrast, correlations of age and CD8⁺ subsets (effector and memory cells) were observed early during and 3 months after radiotherapy. Ratios of T cells and T cell proliferation compared to baseline correlated with CTV. Early changes in regulatory T cells (Treg cells) and CD8⁺ effector T cells correlated with V20 of blood vessels and bone volumes.

Conclusions

Patient age as well as radiotherapy planning parameters correlated with immune changes during radiotherapy. Larger irradiated volumes seem to correlate with early suppression of anti-cancer immunity. For immune cell analysis during normofractionated radiotherapy and correlations with treatment planning parameters, different time points should be looked at in future projects.

Trial registration number:

NCT01376674, 20.06.2011

Supplementary Information

The online version contains supplementary material available at 10.1007/s00262-021-03002-6.

Asymmetric multi-task attention network for prostate bed segmentation in computed tomography images

Post-prostatectomy radiotherapy requires accurate annotation of the prostate bed (PB), i.e., the residual tissue after the operative removal of the prostate gland, to minimize side effects on surrounding organs-at-risk (OARs). However, PB segmentation in computed tomography (CT) images is a challenging task, even for experienced physicians. This is because PB is almost a "virtual" target with non-contrast boundaries and highly variable shapes depending on neighboring OARs. In this work, we propose an asymmetric multi-task attention network (AMTA-Net) for the concurrent segmentation of PB and surrounding OARs. Our AMTA-Net mimics experts in delineating the non-contrast PB by explicitly leveraging its critical dependency on the neighboring OARs (i.e., the bladder and rectum), which are relatively easy to distinguish in CT images. Specifically, we first adopt a U-Net as the backbone network for the low-level (or prerequisite) task of the OAR segmentation. Then, we build an attention sub-network upon the backbone U-Net with a series of cascaded attention modules, which can hierarchically transfer the OAR features and adaptively learn discriminative representations for the high-level (or primary) task of the PB segmentation. We comprehensively evaluate the proposed AMTA-Net on a clinical dataset composed of 186 CT images. According to the experimental results, our AMTA-Net significantly outperforms current clinical state-of-the-arts (i.e., atlas-based segmentation methods), indicating the value of our method in reducing time and labor in the clinical workflow. Our AMTA-Net also presents better performance than the technical state-of-the-arts (i.e., the deep learning-based segmentation methods), especially for the most indistinguishable and clinically critical part of the PB boundaries. Source code is released at https://github.com/superxuang/amta-net.

Multi-Scale Context-Guided Deep Network for Automated Lesion Segmentation With Endoscopy Images of Gastrointestinal Tract

Accurate lesion segmentation based on endoscopy images is a fundamental task for the automated diagnosis of gastrointestinal tract (GI Tract) diseases. Previous studies usually use hand-crafted features for representing endoscopy images, while feature definition and lesion segmentation are treated as two standalone tasks. Due to the possible heterogeneity between features and segmentation models, these methods often result in sub-optimal performance. Several fully convolutional networks have been recently developed to jointly perform feature learning and model training for GI Tract disease diagnosis. However, they generally ignore local spatial details of endoscopy images, as down-sampling operations (e.g., pooling and convolutional striding) may result in irreversible loss of image spatial information. To this end, we propose a multi-scale context-guided deep network (MCNet) for end-to-end lesion segmentation of endoscopy images in GI Tract, where both global and local contexts are captured as guidance for model training. Specifically, one global subnetwork is designed to extract the global structure and high-level semantic context of each input image. Then we further design two cascaded local subnetworks based on output feature maps of the global subnetwork, aiming to capture both local appearance information and relatively high-level semantic information in a multi-scale manner. Those feature maps learned by three subnetworks are further fused for the subsequent task of lesion segmentation. We have evaluated the proposed MCNet on 1,310 endoscopy images from the public EndoVis-Ab and CVC-ClinicDB datasets for abnormal segmentation and polyp segmentation, respectively. Experimental results demonstrate that MCNet achieves [Formula: see text] and [Formula: see text] mean intersection over union (mIoU) on two datasets, respectively, outperforming several state-of-the-art approaches in automated lesion segmentation with endoscopy images of GI Tract.

Automatic Skull Stripping of Rat and Mouse Brain MRI Data Using U-Net

Accurate removal of magnetic resonance imaging (MRI) signal outside the brain, a.k.a., skull stripping, is a key step in the brain image pre-processing pipelines. In rodents, this is mostly achieved by manually editing a brain mask, which is time-consuming and operator dependent. Automating this step is particularly challenging in rodents as compared to humans, because of differences in brain/scalp tissue geometry, image resolution with respect to brain-scalp distance, and tissue contrast around the skull. In this study, we proposed a deep-learning-based framework, U-Net, to automatically identify the rodent brain boundaries in MR images. The U-Net method is robust against inter-subject variability and eliminates operator dependence. To benchmark the efficiency of this method, we trained and validated our model using both in-house collected and publicly available datasets. In comparison to current state-of-the-art methods, our approach achieved superior averaged Dice similarity coefficient to ground truth T2-weighted rapid acquisition with relaxation enhancement and T2∗-weighted echo planar imaging data in both rats and mice (all p < 0.05), demonstrating robust performance of our approach across various MRI protocols.