MRI-based automatic segmentation of rectal cancer using 2D U-Net on two independent cohorts

Improved deep learning for automatic localisation and segmentation of rectal cancer on T2-weighted MRI

INTRODUCTION

The automatic segmentation approaches of rectal cancer from magnetic resonance imaging (MRI) are very valuable to relieve physicians from heavy workloads and enhance working efficiency. This study aimed to compare the segmentation accuracy of a proposed model with the other three models and the inter-observer consistency.

METHODS

A total of 65 patients with rectal cancer who underwent MRI examination were enrolled in our cohort and were randomly divided into a training cohort (n = 45) and a validation cohort (n = 20). Two experienced radiologists independently segmented rectal cancer lesions. A novel segmentation model (AttSEResUNet) was trained on T2WI based on ResUNet and attention mechanisms. The segmentation performance of the AttSEResUNet, U-Net, ResUNet and U-Net with Attention Gate (AttUNet) was compared, using Dice similarity coefficient (DSC), Hausdorff distance (HD), mean distance to agreement (MDA) and Jaccard index. The segmentation variability of automatic segmentation models and inter-observer was also evaluated.

RESULTS

The AttSEResUNet with post-processing showed perfect lesion recognition rate (100%) and false recognition rate (0), and its evaluation metrics outperformed other three models for two independent readers (observer 1: DSC = 0.839 ± 0.112, HD = 9.55 ± 6.68, MDA = 0.556 ± 0.722, Jaccard index = 0.736 ± 0.150; observer 2: DSC = 0.856 ± 0.099, HD = 11.0 ± 10.1, MDA = 0.789 ± 1.07, Jaccard index = 0.673 ± 0.130). The segmentation performance of AttSEResUNet was comparable and similar to manual variability (DSC = 0.857 ± 0.115, HD = 10.0 ± 10.0, MDA = 0.704 ± 1.17, Jaccard index = 0.666 ± 0.139).

CONCLUSION

Comparing with other three models, the proposed AttSEResUNet model was demonstrated as a more accurate model for contouring the rectal tumours in axial T2WI images, whose variability was similar to that of inter-observer.

Machine learning based prediction of image quality in prostate MRI using rapid localizer images

Purpose

Diagnostic performance of prostate MRI depends on high-quality imaging. Prostate MRI quality is inversely proportional to the amount of rectal gas and distention. Early detection of poor-quality MRI may enable intervention to remove gas or exam rescheduling, saving time. We developed a machine learning based quality prediction of yet-to-be acquired MRI images solely based on MRI rapid localizer sequence, which can be acquired in a few seconds.

Approach

The dataset consists of 213 (147 for training and 64 for testing) prostate sagittal T2-weighted (T2W) MRI localizer images and rectal content, manually labeled by an expert radiologist. Each MRI localizer contains seven two-dimensional (2D) slices of the patient, accompanied by manual segmentations of rectum for each slice. Cascaded and end-to-end deep learning models were used to predict the quality of yet-to-be T2W, DWI, and apparent diffusion coefficient (ADC) MRI images. Predictions were compared to quality scores determined by the experts using area under the receiver operator characteristic curve and intra-class correlation coefficient.

Results

In the test set of 64 patients, optimal versus suboptimal exams occurred in 95.3% (61/64) versus 4.7% (3/64) for T2W, 90.6% (58/64) versus 9.4% (6/64) for DWI, and 89.1% (57/64) versus 10.9% (7/64) for ADC. The best performing segmentation model was 2D U-Net with ResNet-34 encoder and ImageNet weights. The best performing classifier was the radiomics based classifier.

Conclusions

A radiomics based classifier applied to localizer images achieves accurate diagnosis of subsequent image quality for T2W, DWI, and ADC prostate MRI sequences.

Performance reporting design in artificial intelligence studies using image-based TNM staging and prognostic parameters in rectal cancer: a systematic review

PURPOSE

The integration of artificial intelligence (AI) and magnetic resonance imaging in rectal cancer has the potential to enhance diagnostic accuracy by identifying subtle patterns and aiding tumor delineation and lymph node assessment. According to our systematic review focusing on convolutional neural networks, AI-driven tumor staging and the prediction of treatment response facilitate tailored treat-ment strategies for patients with rectal cancer.

METHODS

This paper summarizes the current landscape of AI in the imaging field of rectal cancer, emphasizing the performance reporting design based on the quality of the dataset, model performance, and external validation.

RESULTS

AI-driven tumor segmentation has demonstrated promising results using various convolutional neural network models. AI-based predictions of staging and treatment response have exhibited potential as auxiliary tools for personalized treatment strategies. Some studies have indicated superior performance than conventional models in predicting microsatellite instability and KRAS status, offer-ing noninvasive and cost-effective alternatives for identifying genetic mutations.

CONCLUSION

Image-based AI studies for rectal can-cer have shown acceptable diagnostic performance but face several challenges, including limited dataset sizes with standardized data, the need for multicenter studies, and the absence of oncologic relevance and external validation for clinical implantation. Overcoming these pitfalls and hurdles is essential for the feasible integration of AI models in clinical settings for rectal cancer, warranting further research.

Quantitative MRI-based radiomics analysis identifies blood flow feature associated to overall survival for rectal cancer patients

Radiomics objectively quantifies image information through numerical metrics known as features. In this study, we investigated the stability of magnetic resonance imaging (MRI)-based radiomics features in rectal cancer using both anatomical MRI and quantitative MRI (qMRI), when different methods to define the tumor volume were used. Second, we evaluated the prognostic value of stable features associated to 5-year progression-free survival (PFS) and overall survival (OS). On a 1.5 T MRI scanner, 81 patients underwent diagnostic MRI, an extended diffusion-weighted sequence with calculation of the apparent diffusion coefficient (ADC) and a multiecho dynamic contrast sequence generating both dynamic contrast-enhanced and dynamic susceptibility contrast (DSC) MR, allowing quantification of Ktrans, blood flow (BF) and area under the DSC curve (AUC). Radiomic features were extracted from T2w images and from ADC, Ktrans, BF and AUC maps. Tumor volumes were defined with three methods; machine learning, deep learning and manual delineations. The interclass correlation coefficient (ICC) assessed the stability of features. Internal validation was performed on 1000 bootstrap resamples in terms of discrimination, calibration and decisional benefit. For each combination of image and volume definition, 94 features were extracted. Features from qMRI contained higher prognostic potential than features from anatomical MRI. When stable features (> 90% ICC) were compared with clinical parameters, qMRI features demonstrated the best prognostic potential. A feature extracted from the DSC MRI parameter BF was associated with both PFS (p = 0.004) and OS (p = 0.004). In summary, stable qMRI-based radiomics features was identified, in particular, a feature based on BF from DSC MRI was associated with both PFS and OS.

RTAU-Net: A novel 3D rectal tumor segmentation model based on dual path fusion and attentional guidance

BACKGROUND AND OBJECTIVE

According to the Global Cancer Statistics 2020, colorectal cancer has the third-highest diagnosis rate (10.0 %) and the second-highest mortality rate (9.4 %) among the 36 types. Rectal cancer accounts for a large proportion of colorectal cancer. The size and shape of the rectal tumor can directly affect the diagnosis and treatment by doctors. The existing rectal tumor segmentation methods are based on two-dimensional slices, which cannot analyze a patient's tumor as a whole and lose the correlation between slices of MRI image, so the practical application value is not high.

METHODS

In this paper, a three-dimensional rectal tumor segmentation model is proposed. Firstly, image preprocessing is performed to reduce the effect caused by the unbalanced proportion of background region and target region, and improve the quality of the image. Secondly, a dual-path fusion network is designed to extract both global features and local detail features of rectal tumors. The network includes two encoders, a residual encoder for enhancing the spatial detail information and feature representation of the tumor and a transformer encoder for extracting global contour information of the tumor. In the decoding stage, we merge the information extracted from the dual paths and decode them. In addition, for the problem of the complex morphology and different sizes of rectal tumors, a multi-scale fusion channel attention mechanism is designed, which can capture important contextual information of different scales. Finally, visualize the 3D rectal tumor segmentation results.

RESULTS

The RTAU-Net is evaluated on the data set provided by Shanxi Provincial Cancer Hospital and Xinhua Hospital. The experimental results showed that the Dice of tumor segmentation reached 0.7978 and 0.6792, respectively, which improved by 2.78 % and 7.02 % compared with suboptimal model.

CONCLUSIONS

Although the morphology of rectal tumors varies, RTAU-Net can precisely localize rectal tumors and learn the contour and details of tumors, which can relieve physicians' workload and improve diagnostic accuracy.

MRI-based automatic identification and segmentation of extrahepatic cholangiocarcinoma using deep learning network

BACKGROUND

Accurate identification of extrahepatic cholangiocarcinoma (ECC) from an image is challenging because of the small size and complex background structure. Therefore, considering the limitation of manual delineation, it's necessary to develop automated identification and segmentation methods for ECC. The aim of this study was to develop a deep learning approach for automatic identification and segmentation of ECC using MRI.

METHODS

We recruited 137 ECC patients from our hospital as the main dataset (C1) and an additional 40 patients from other hospitals as the external validation set (C2). All patients underwent axial T1-weighted imaging (T1WI), T2-weighted imaging (T2WI), and diffusion-weighted imaging (DWI). Manual delineations were performed and served as the ground truth. Next, we used 3D VB-Net to establish single-mode automatic identification and segmentation models based on T1WI (model 1), T2WI (model 2), and DWI (model 3) in the training cohort (80% of C1), and compared them with the combined model (model 4). Subsequently, the generalization capability of the best models was evaluated using the testing set (20% of C1) and the external validation set (C2). Finally, the performance of the developed models was further evaluated.

RESULTS

Model 3 showed the best identification performance in the training, testing, and external validation cohorts with success rates of 0.980, 0.786, and 0.725, respectively. Furthermore, model 3 yielded an average Dice similarity coefficient (DSC) of 0.922, 0.495, and 0.466 to segment ECC automatically in the training, testing, and external validation cohorts, respectively.

CONCLUSION

The DWI-based model performed better in automatically identifying and segmenting ECC compared to T1WI and T2WI, which may guide clinical decisions and help determine prognosis.

Navigation-Assisted Surgery for Locally Advanced Primary and Recurrent Rectal Cancer

BACKGROUND

In some surgical disciplines, navigation-assisted surgery has become standard of care, but in rectal cancer, indications for navigation and the utility of different technologies remain undetermined.

METHODS

The NAVI-LARRC prospective study (NCT04512937; IDEAL Stage 2a) evaluated feasibility of navigation in patients with locally advanced primary (LARC) and recurrent rectal cancer (LRRC). Included patients had advanced tumours with high risk of incomplete (R1/R2) resection, and navigation was considered likely to improve the probability of complete resection (R0). Tumours were classified according to pelvic compartmental involvement, as suggested by the Royal Marsden group. The BrainlabTM navigation platform was used for preoperative segmentation of tumour and pelvic anatomy, and for intraoperative navigation with optical tracking. R0 resection rates, surgeons' experiences, and adherence to the preoperative resection plan were assessed.

RESULTS

Seventeen patients with tumours involving the posterior/lateral compartments underwent navigation-assisted procedures. Fifteen patients required abdominosacral resection, and 3 had resection of the sciatic nerve. R0 resection was obtained in 6/8 (75%) LARC and 6/9 (69%) LRRC cases. Preoperative segmentation was time-consuming (median 3.5 h), but intraoperative navigation was accurate. Surgeons reported navigation to be feasible, and adherence to the resection plan was satisfactory.

CONCLUSIONS

Navigation-assisted surgery using optical tracking was feasible. The preoperative planning was time-consuming, but intraoperative navigation was accurate and resulted in acceptable R0 resection rates. Selected patients are likely to benefit from navigation-assisted surgery.

Region-specific deep learning models for accurate segmentation of rectal structures on post-chemoradiation T2w MRI: a multi-institutional, multi-reader study

Introduction

For locally advanced rectal cancers, in vivo radiological evaluation of tumor extent and regression after neoadjuvant therapy involves implicit visual identification of rectal structures on magnetic resonance imaging (MRI). Additionally, newer image-based, computational approaches (e.g., radiomics) require more detailed and precise annotations of regions such as the outer rectal wall, lumen, and perirectal fat. Manual annotations of these regions, however, are highly laborious and time-consuming as well as subject to inter-reader variability due to tissue boundaries being obscured by treatment-related changes (e.g., fibrosis, edema).

Methods

This study presents the application of U-Net deep learning models that have been uniquely developed with region-specific context to automatically segment each of the outer rectal wall, lumen, and perirectal fat regions on post-treatment, T2-weighted MRI scans.

Results

In multi-institutional evaluation, region-specific U-Nets (wall Dice = 0.920, lumen Dice = 0.895) were found to perform comparably to multiple readers (wall inter-reader Dice = 0.946, lumen inter-reader Dice = 0.873). Additionally, when compared to a multi-class U-Net, region-specific U-Nets yielded an average 20% improvement in Dice scores for segmenting each of the wall, lumen, and fat; even when tested on T2-weighted MRI scans that exhibited poorer image quality, or from a different plane, or were accrued from an external institution.

Discussion

Developing deep learning segmentation models with region-specific context may thus enable highly accurate, detailed annotations for multiple rectal structures on post-chemoradiation T2-weighted MRI scans, which is critical for improving evaluation of tumor extent in vivo and building accurate image-based analytic tools for rectal cancers.

MRI-Based Artificial Intelligence in Rectal Cancer

Rectal cancer (RC) accounts for approximately one-third of colorectal cancer (CRC), with death rates increasing in patients younger than 50 years old. Magnetic resonance imaging (MRI) is routinely performed for tumor evaluation. However, the semantic features from images alone remain insufficient to guide treatment decisions. Functional MRIs are useful for revealing microstructural and functional abnormalities and nevertheless have low or modest repeatability and reproducibility. Therefore, during the preoperative evaluation and follow-up treatment of patients with RC, novel noninvasive imaging markers are needed to describe tumor characteristics to guide treatment strategies and achieve individualized diagnosis and treatment. In recent years, the development of artificial intelligence (AI) has created new tools for RC evaluation based on MRI. In this review, we summarize the research progress of AI in the evaluation of staging, prediction of high-risk factors, genotyping, response to therapy, recurrence, metastasis, prognosis, and segmentation with RC. We further discuss the challenges of clinical application, including improvement in imaging, model performance, and the biological meaning of features, which may also be major development directions in the future. EVIDENCE LEVEL: 5 TECHNICAL EFFICACY: Stage 2.

Artificial intelligence in multi-parametric magnetic resonance imaging: A review

Multi-parametric magnetic resonance imaging (mpMRI) is an indispensable tool in the clinical workflow for the diagnosis and treatment planning of various diseases. Machine learning-based artificial intelligence (AI) methods, especially those adopting the deep learning technique, have been extensively employed to perform mpMRI image classification, segmentation, registration, detection, reconstruction, and super-resolution. The current availability of increasing computational power and fast-improving AI algorithms have empowered numerous computer-based systems for applying mpMRI to disease diagnosis, imaging-guided radiotherapy, patient risk and overall survival time prediction, and the development of advanced quantitative imaging technology for magnetic resonance fingerprinting. However, the wide application of these developed systems in the clinic is still limited by a number of factors, including robustness, reliability, and interpretability. This survey aims to provide an overview for new researchers in the field as well as radiologists with the hope that they can understand the general concepts, main application scenarios, and remaining challenges of AI in mpMRI. This article is protected by copyright. All rights reserved.

Semi-automatic tumor segmentation of rectal cancer based on functional magnetic resonance imaging

•

Machine learning on magnetic resonance images (MRI) was used for tumor segmentation.

•

Voxelwise machine learning with morphological post-processing achieved good segmentation results.

•

Combining T2-weighted with functional MRI improved semi-automatic tumor segmentation.

•

Dynamic contrast enhanced MRI was the most valuable functional MRI information.

•

Tumor volume and interobserver variation were linked to measured segmentation quality.

Background and purpose

Tumor delineation is required both for radiotherapy planning and quantitative imaging biomarker purposes. It is a manual, time- and labor-intensive process prone to inter- and intraobserver variations. Semi or fully automatic segmentation could provide better efficiency and consistency. This study aimed to investigate the influence of including and combining functional with anatomical magnetic resonance imaging (MRI) sequences on the quality of automatic segmentations.

Materials and methods

T2-weighted (T2w), diffusion weighted, multi-echo T2*-weighted, and contrast enhanced dynamic multi-echo (DME) MR images of eighty-one patients with rectal cancer were used in the analysis. Four classical machine learning algorithms; adaptive boosting (ADA), linear and quadratic discriminant analysis and support vector machines, were trained for automatic segmentation of tumor and normal tissue using different combinations of the MR images as input, followed by semi-automatic morphological post-processing. Manual delineations from two experts served as ground truth. The Sørensen-Dice similarity coefficient (DICE) and mean symmetric surface distance (MSD) were used as performance metric in leave-one-out cross validation.

Results

Using T2w images alone, ADA outperformed the other algorithms, yielding a median per patient DICE of 0.67 and MSD of 3.6 mm. The performance improved when functional images were added and was highest for models based on either T2w and DME images (DICE: 0.72, MSD: 2.7 mm) or all four MRI sequences (DICE: 0.72, MSD: 2.5 mm).

Conclusion

Machine learning models using functional MRI, in particular DME, have the potential to improve automatic segmentation of rectal cancer relative to models using T2w MRI alone.