The use of atlas registration and graph cuts for prostate segmentation in magnetic resonance images

Automated prostate gland segmentation in challenging clinical cases: comparison of three artificial intelligence methods

OBJECTIVE

Automated methods for prostate segmentation on MRI are typically developed under ideal scanning and anatomical conditions. This study evaluates three different prostate segmentation AI algorithms in a challenging population of patients with prior treatments, variable anatomic characteristics, complex clinical history, or atypical MRI acquisition parameters.

MATERIALS AND METHODS

A single institution retrospective database was queried for the following conditions at prostate MRI: prior prostate-specific oncologic treatment, transurethral resection of the prostate (TURP), abdominal perineal resection (APR), hip prosthesis (HP), diversity of prostate volumes (large ≥ 150 cc, small ≤ 25 cc), whole gland tumor burden, magnet strength, noted poor quality, and various scanners (outside/vendors). Final inclusion criteria required availability of axial T2-weighted (T2W) sequence and corresponding prostate organ segmentation from an expert radiologist. Three previously developed algorithms were evaluated: (1) deep learning (DL)-based model, (2) commercially available shape-based model, and (3) federated DL-based model. Dice Similarity Coefficient (DSC) was calculated compared to expert. DSC by model and scan factors were evaluated with Wilcox signed-rank test and linear mixed effects (LMER) model.

RESULTS

683 scans (651 patients) met inclusion criteria (mean prostate volume 60.1 cc [9.05-329 cc]). Overall DSC scores for models 1, 2, and 3 were 0.916 (0.707-0.971), 0.873 (0-0.997), and 0.894 (0.025-0.961), respectively, with DL-based models demonstrating significantly higher performance (p < 0.01). In sub-group analysis by factors, Model 1 outperformed Model 2 (all p < 0.05) and Model 3 (all p < 0.001). Performance of all models was negatively impacted by prostate volume and poor signal quality (p < 0.01). Shape-based factors influenced DL models (p < 0.001) while signal factors influenced all (p < 0.001).

CONCLUSION

Factors affecting anatomical and signal conditions of the prostate gland can adversely impact both DL and non-deep learning-based segmentation models.

Automatic segmentation of prostate zonal anatomy on MRI: a systematic review of the literature

OBJECTIVES

Accurate zonal segmentation of prostate boundaries on MRI is a critical prerequisite for automated prostate cancer detection based on PI-RADS. Many articles have been published describing deep learning methods offering great promise for fast and accurate segmentation of prostate zonal anatomy. The objective of this review was to provide a detailed analysis and comparison of applicability and efficiency of the published methods for automatic segmentation of prostate zonal anatomy by systematically reviewing the current literature.

METHODS

A Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) was conducted until June 30, 2021, using PubMed, ScienceDirect, Web of Science and EMBase databases. Risk of bias and applicability based on Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) criteria adjusted with Checklist for Artificial Intelligence in Medical Imaging (CLAIM) were assessed.

RESULTS

A total of 458 articles were identified, and 33 were included and reviewed. Only 2 articles had a low risk of bias for all four QUADAS-2 domains. In the remaining, insufficient details about database constitution and segmentation protocol provided sources of bias (inclusion criteria, MRI acquisition, ground truth). Eighteen different types of terminology for prostate zone segmentation were found, while 4 anatomic zones are described on MRI. Only 2 authors used a blinded reading, and 4 assessed inter-observer variability.

CONCLUSIONS

Our review identified numerous methodological flaws and underlined biases precluding us from performing quantitative analysis for this review. This implies low robustness and low applicability in clinical practice of the evaluated methods. Actually, there is not yet consensus on quality criteria for database constitution and zonal segmentation methodology.

A novel registration-based algorithm for prostate segmentation via the combination of SSM and CNN

PURPOSE

Precise determination of target is an essential procedure in prostate interventions, such as prostate biopsy, lesion detection, and targeted therapy. However, the prostate delineation may be tough in some cases due to tissue ambiguity or lack of partial anatomical boundary. In this study, we proposed a novel supervised registration-based algorithm for precise prostate segmentation, which combine the convolutional neural network (CNN) with a statistical shape model (SSM).

METHODS

The proposed network mainly consists of two branches. One called SSM-Net branch was exploited to predict the shape transform matrix, shape control parameters, and shape fine-tuning vector, for the generation of the prostate boundary. Furtherly, according to the inferred boundary, a normalized distance map was calculated as the output of SSM-Net. Another branch named ResU-Net was employed to predict a probability label map from the input images at the same time. Integrating the output of these two branches, the optimal weighted sum of the distance map and the probability map was regarded as the prostate segmentation.

RESULTS

Two public datasets PROMISE12 and NCI-ISBI 2013 were utilized to evaluate the performance of the proposed algorithm. The results demonstrate that the segmentation algorithm achieved the best performance with an SSM of 9500 nodes, which obtained a dice of 0.907 and an average surface distance of 1.85 mm. Compared with other methods, our algorithm delineates the prostate region more accurately and efficiently. In addition, we verified the impact of model elasticity augmentation and the fine-tuning item on the network segmentation capability. As a result, both factors have improved the delineation accuracy, with dice increased by 10% and 7% respectively.

CONCLUSIONS

Our segmentation method has the potential to be an effective and robust approach for prostate segmentation. This article is protected by copyright. All rights reserved.

Challenge of prostate MRI segmentation on T2-weighted images: inter-observer variability and impact of prostate morphology

Background

Accurate prostate zonal segmentation on magnetic resonance images (MRI) is a critical prerequisite for automated prostate cancer detection. We aimed to assess the variability of manual prostate zonal segmentation by radiologists on T2-weighted (T2W) images, and to study factors that may influence it.

Methods

Seven radiologists of varying levels of experience segmented the whole prostate gland (WG) and the transition zone (TZ) on 40 axial T2W prostate MRI images (3D T2W images for all patients, and both 3D and 2D images for a subgroup of 12 patients). Segmentation variabilities were evaluated based on: anatomical and morphological variation of the prostate (volume, retro-urethral lobe, intensity contrast between zones, presence of a PI-RADS ≥ 3 lesion), variation in image acquisition (3D vs 2D T2W images), and reader’s experience. Several metrics including Dice Score (DSC) and Hausdorff Distance were used to evaluate differences, with both a pairwise and a consensus (STAPLE reference) comparison.

Results

DSC was 0.92 (± 0.02) and 0.94 (± 0.03) for WG, 0.88 (± 0.05) and 0.91 (± 0.05) for TZ respectively with pairwise comparison and consensus reference. Variability was significantly (p < 0.05) lower for the mid-gland (DSC 0.95 (± 0.02)), higher for the apex (0.90 (± 0.06)) and the base (0.87 (± 0.06)), and higher for smaller prostates (p < 0.001) and when contrast between zones was low (p < 0.05). Impact of the other studied factors was non-significant.

Conclusions

Variability is higher in the extreme parts of the gland, is influenced by changes in prostate morphology (volume, zone intensity ratio), and is relatively unaffected by the radiologist’s level of expertise.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13244-021-01010-9.

Segmentation of prostate and prostate zones using deep learning : A multi-MRI vendor analysis

PURPOSE

Develop a deep-learning-based segmentation algorithm for prostate and its peripheral zone (PZ) that is reliable across multiple MRI vendors.

METHODS

This is a retrospective study. The dataset consisted of 550 MRIs (Siemens-330, General Electric[GE]-220). A multistream 3D convolutional neural network is used for automatic segmentation of the prostate and its PZ using T2-weighted (T2-w) MRI. Prostate and PZ were manually contoured on axial T2‑w. The network uses axial, coronal, and sagittal T2‑w series as input. The preprocessing of the input data includes bias correction, resampling, and image normalization. A dataset from two MRI vendors (Siemens and GE) is used to test the proposed network. Six different models were trained, three for the prostate and three for the PZ. Of the three, two were trained on data from each vendor separately, and a third (Combined) on the aggregate of the datasets. The Dice coefficient (DSC) is used to compare the manual and predicted segmentation.

RESULTS

For prostate segmentation, the Combined model obtained DSCs of 0.893 ± 0.036 and 0.825 ± 0.112 (mean ± standard deviation) on Siemens and GE, respectively. For PZ, the best DSCs were from the Combined model: 0.811 ± 0.079 and 0.788 ± 0.093. While the Siemens model underperformed on the GE dataset and vice versa, the Combined model achieved robust performance on both datasets.

CONCLUSION

The proposed network has a performance comparable to the interexpert variability for segmenting the prostate and its PZ. Combining images from different MRI vendors on the training of the network is of paramount importance for building a universal model for prostate and PZ segmentation.

Validation of automated magnetic resonance image segmentation for radiation therapy planning in prostate cancer

Background and purpose

Magnetic resonance imaging (MRI) is increasingly used in radiation therapy planning of prostate cancer (PC) to reduce target volume delineation uncertainty. This study aimed to assess and validate the performance of a fully automated segmentation tool (AST) in MRI based radiation therapy planning of PC.

Material and methods

Pelvic structures of 65 PC patients delineated in an MRI-only workflow according to established guidelines were included in the analysis. Automatic vs manual segmentation by an experienced oncologist was compared with geometrical parameters, such as the dice similarity coefficient (DSC). Fifteen patients had a second MRI within 15 days to assess repeatability of the AST for prostate and seminal vesicles. Furthermore, we investigated whether hormonal therapy or body mass index (BMI) affected the AST results.

Results

The AST showed high agreement with manual segmentation expressed as DSC (mean, SD) for delineating prostate (0.84, 0.04), bladder (0.92, 0.04) and rectum (0.86, 0.04). For seminal vesicles (0.56, 0.17) and penile bulb (0.69, 0.12) the respective agreement was moderate. Performance of AST was not influenced by neoadjuvant hormonal therapy, although those on treatment had significantly smaller prostates than the hormone-naïve patients (p < 0.0001). In repeat assessment, consistency of prostate delineation resulted in mean DSC of 0.89, (SD 0.03) between the paired MRI scans for AST, while mean DSC of manual delineation was 0.82, (SD 0.05).

Conclusion

Fully automated MRI segmentation tool showed good agreement and repeatability compared with manual segmentation and was found clinically robust in patients with PC. However, manual review and adjustment of some structures in individual cases remain important in clinical use.

Deeply supervised 3D fully convolutional networks with group dilated convolution for automatic MRI prostate segmentation

PURPOSE

Reliable automated segmentation of the prostate is indispensable for image-guided prostate interventions. However, the segmentation task is challenging due to inhomogeneous intensity distributions, variation in prostate anatomy, among other problems. Manual segmentation can be time-consuming and is subject to inter- and intraobserver variation. We developed an automated deep learning-based method to address this technical challenge.

METHODS

We propose a three-dimensional (3D) fully convolutional networks (FCN) with deep supervision and group dilated convolution to segment the prostate on magnetic resonance imaging (MRI). In this method, a deeply supervised mechanism was introduced into a 3D FCN to effectively alleviate the common exploding or vanishing gradients problems in training deep models, which forces the update process of the hidden layer filters to favor highly discriminative features. A group dilated convolution which aggregates multiscale contextual information for dense prediction was proposed to enlarge the effective receptive field of convolutional neural networks, which improve the prediction accuracy of prostate boundary. In addition, we introduced a combined loss function including cosine and cross entropy, which measures similarity and dissimilarity between segmented and manual contours, to further improve the segmentation accuracy. Prostate volumes manually segmented by experienced physicians were used as a gold standard against which our segmentation accuracy was measured.

RESULTS

The proposed method was evaluated on an internal dataset comprising 40 T2-weighted prostate MR volumes. Our method achieved a Dice similarity coefficient (DSC) of 0.86 ± 0.04, a mean surface distance (MSD) of 1.79 ± 0.46 mm, 95% Hausdorff distance (95%HD) of 7.98 ± 2.91 mm, and absolute relative volume difference (aRVD) of 15.65 ± 10.82. A public dataset (PROMISE12) including 50 T2-weighted prostate MR volumes was also employed to evaluate our approach. Our method yielded a DSC of 0.88 ± 0.05, MSD of 1.02 ± 0.35 mm, 95% HD of 9.50 ± 5.11 mm, and aRVD of 8.93 ± 7.56.

CONCLUSION

We developed a novel deeply supervised deep learning-based approach with a group dilated convolution to automatically segment the MRI prostate, demonstrated its clinical feasibility, and validated its accuracy against manual segmentation. The proposed technique could be a useful tool for image-guided interventions in prostate cancer.

A semiautomatic approach for prostate segmentation in MR images using local texture classification and statistical shape modeling

Segmentation of the prostate in magnetic resonance (MR) images has many applications in image-guided treatment planning and procedures such as biopsy and focal therapy. However, manual delineation of the prostate boundary is a time-consuming task with high inter-observer variation. In this study, we proposed a semiautomated, three-dimensional (3D) prostate segmentation technique for T2-weighted MR images based on shape and texture analysis. The prostate gland shape is usually globular with a smoothly curved surface that could be accurately modeled and reconstructed if the locations of a limited number of well-distributed surface points are known. For a training image set, we used an inter-subject correspondence between the prostate surface points to model the prostate shape variation based on a statistical point distribution modeling. We also studied the local texture difference between prostate and non-prostate tissues close to the prostate surface. To segment a new image, we used the learned prostate shape and texture characteristics to search for the prostate border close to an initially estimated prostate surface. We used 23 MR images for training, and 14 images for testing the algorithm performance. We compared the results to two sets of experts' manual reference segmentations. The measured mean ± standard deviation of error values for the whole gland were 1.4 ± 0.4 mm, 8.5 ± 2.0 mm, and 86 ± 3% in terms of mean absolute distance (MAD), Hausdorff distance (HDist), and Dice similarity coefficient (DSC). The average measured differences between the two experts on the same datasets were 1.5 mm (MAD), 9.0 mm (HDist), and 83% (DSC). The proposed algorithm illustrated a fast, accurate, and robust performance for 3D prostate segmentation. The accuracy of the algorithm is within the inter-expert variability observed in manual segmentation and comparable to the best performance results reported in the literature.

Automatic segmentation of cardiac substructures from noncontrast CT images: accurate enough for dosimetric analysis?

PURPOSE

We evaluated the feasibility of using an automatic segmentation tool to delineate cardiac substructures from noncontrast computed tomography (CT) images for cardiac dosimetry and toxicity analyses for patients with nonsmall cell lung cancer (NSCLC) after radiotherapy.

MATERIAL AND METHODS

We used an in-house developed multi-atlas segmentation tool to delineate 11cardiac substructures, including the whole heart, four heart chambers, and six greater vessels, automatically from the averaged 4D-CT planning images of 49 patients with NSCLC. Two experienced radiation oncologists edited the auto-segmented contours. Times for automatic segmentation and modification were recorded. The modified contours were compared with the auto-segmented contours in terms of Dice similarity coefficient (DSC) and mean surface distance (MSD) to evaluate the extent of modification. Differences in dose-volume histogram (DVH) characteristics were also evaluated for the modified versus auto-segmented contours.

RESULTS

The mean automatic segmentation time for all 11 structures was 7-9 min. For the 49 patients, the mean DSC values (±SD) ranged from .73 ± .08 to .95 ± .04, and the mean MSD values ranged from 1.3 ± .6 mm to 2.9 ± 5.1 mm. Overall, the modifications were small; the largest modifications were in the pulmonary vein and the inferior vena cava. The heart V30 (volume receiving dose ≥30 Gy) and the mean dose to the whole heart and the four heart chambers were not different for the modified versus the auto-segmented contours based on the statistically significant condition of p < .05. Also, the maximum dose to the great vessels was no different except for the pulmonary vein.

CONCLUSIONS

Automatic segmentation of cardiac substructures did not require substantial modifications. Dosimetric evaluation showed no significant difference between the auto-segmented and modified contours for most structures, which suggests that the auto-segmented contours can be used to study cardiac dose-responses in clinical practice.

Towards a universal MRI atlas of the prostate and prostate zones : Comparison of MRI vendor and image acquisition parameters

BACKGROUND AND PURPOSE

The aim of this study was to evaluate an automatic multi-atlas-based segmentation method for generating prostate, peripheral (PZ), and transition zone (TZ) contours on MRIs with and without fat saturation (±FS), and compare MRIs from different vendor MRI systems.

METHODS

T2-weighted (T2) and fat-saturated (T2FS) MRIs were acquired on 3T GE (GE, Waukesha, WI, USA) and Siemens (Erlangen, Germany) systems. Manual prostate and PZ contours were used to create atlas libraries. As a test MRI is entered, the procedure for atlas segmentation automatically identifies the atlas subjects that best match the test subject, followed by a normalized intensity-based free-form deformable registration. The contours are transformed to the test subject, and Dice similarity coefficients (DSC) and Hausdorff distances between atlas-generated and manual contours were used to assess performance.

RESULTS

Three atlases were generated based on GE_T2 (n = 30), GE_T2FS (n = 30), and Siem_T2FS (n = 31). When test images matched the contrast and vendor of the atlas, DSCs of 0.81 and 0.83 for T2 ± FS were obtained (baseline performance). Atlases performed with higher accuracy when segmenting (i) T2FS vs. T2 images, likely due to a superior contrast between prostate vs. surrounding tissue; (ii) prostate vs. zonal anatomy; (iii) in the mid-gland vs. base and apex. Atlases performance declined when tested with images with differing contrast and MRI vendor. Conversely, combined atlases showed similar performance to baseline.

CONCLUSION

The MRI atlas-based segmentation method achieved good results for prostate, PZ, and TZ compared to expert contoured volumes. Combined atlases performed similarly to matching atlas and scan type. The technique is fast, fully automatic, and implemented on commercially available clinical platform.

Transfer Learning for Multicenter Classification of Chronic Obstructive Pulmonary Disease

Chronic obstructive pulmonary disease (COPD) is a lung disease that can be quantified using chest computed tomography scans. Recent studies have shown that COPD can be automatically diagnosed using weakly supervised learning of intensity and texture distributions. However, up till now such classifiers have only been evaluated on scans from a single domain, and it is unclear whether they would generalize across domains, such as different scanners or scanning protocols. To address this problem, we investigate classification of COPD in a multicenter dataset with a total of 803 scans from three different centers, four different scanners, with heterogenous subject distributions. Our method is based on Gaussian texture features, and a weighted logistic classifier, which increases the weights of samples similar to the test data. We show that Gaussian texture features outperform intensity features previously used in multicenter classification tasks. We also show that a weighting strategy based on a classifier that is trained to discriminate between scans from different domains can further improve the results. To encourage further research into transfer learning methods for the classification of COPD, upon acceptance of this paper we will release two feature datasets used in this study on http://bigr.nl/research/projects/copd.

Automatic magnetic resonance prostate segmentation by deep learning with holistically nested networks

Accurate automatic segmentation of the prostate in magnetic resonance images (MRI) is a challenging task due to the high variability of prostate anatomic structure. Artifacts such as noise and similar signal intensity of tissues around the prostate boundary inhibit traditional segmentation methods from achieving high accuracy. We investigate both patch-based and holistic (image-to-image) deep-learning methods for segmentation of the prostate. First, we introduce a patch-based convolutional network that aims to refine the prostate contour which provides an initialization. Second, we propose a method for end-to-end prostate segmentation by integrating holistically nested edge detection with fully convolutional networks. Holistically nested networks (HNN) automatically learn a hierarchical representation that can improve prostate boundary detection. Quantitative evaluation is performed on the MRI scans of 250 patients in fivefold cross-validation. The proposed enhanced HNN model achieves a mean ± standard deviation. A Dice similarity coefficient (DSC) of [Formula: see text] and a mean Jaccard similarity coefficient (IoU) of [Formula: see text] are used to calculate without trimming any end slices. The proposed holistic model significantly ([Formula: see text]) outperforms a patch-based AlexNet model by 9% in DSC and 13% in IoU. Overall, the method achieves state-of-the-art performance as compared with other MRI prostate segmentation methods in the literature.

Automatic prostate segmentation on MR images with deep network and graph model

Automated prostate diagnoses and treatments have gained much attention due to the high mortality rate of prostate cancer. In particular, unsupervised (automatic) prostate segmentation is an active and challenging research. Most conventional works usually utilize handcrafted (low-level) features for prostate segmentation; however they often fail to extract the intrinsic structure of the prostate, especially on images with blurred boundaries. In this paper, we propose a novel automated prostate segmentation model with learned features from deep network. Specifically, we first generate a set of prostate proposals in transverse plane via recognizing the position and coarse estimate of the shape of the prostate on the global prostate image and using the deep network to extract highly effective features for the boundary refinement in a finer scale. With consideration of the correlations among different sequential images, we then construct a graph to select the best prostate proposals from proposal set for its use in 3D prostate segmentation. Experimental evaluation demonstrates that our proposed deep network and graph based method is superior to state-of-the-art couterparts, in terms of both dice similarity coefficient and Hausdorff distance, on public dataset.

Superpixel-Based Segmentation for 3D Prostate MR Images

This paper proposes a method for segmenting the prostate on magnetic resonance (MR) images. A superpixel-based 3D graph cut algorithm is proposed to obtain the prostate surface. Instead of pixels, superpixels are considered as the basic processing units to construct a 3D superpixel-based graph. The superpixels are labeled as the prostate or background by minimizing an energy function using graph cut based on the 3D superpixel-based graph. To construct the energy function, we proposed a superpixel-based shape data term, an appearance data term, and two superpixel-based smoothness terms. The proposed superpixel-based terms provide the effectiveness and robustness for the segmentation of the prostate. The segmentation result of graph cuts is used as an initialization of a 3D active contour model to overcome the drawback of the graph cut. The result of 3D active contour model is then used to update the shape model and appearance model of the graph cut. Iterations of the 3D graph cut and 3D active contour model have the ability to jump out of local minima and obtain a smooth prostate surface. On our 43 MR volumes, the proposed method yields a mean Dice ratio of 89.3 ±1.9%. On PROMISE12 test data set, our method was ranked at the second place; the mean Dice ratio and standard deviation is 87.0±3.2%. The experimental results show that the proposed method outperforms several state-of-the-art prostate MRI segmentation methods.