MRI and CT bladder segmentation from classical to deep learning based approaches: Current limitations and lessons

Involving logical clinical knowledge into deep neural networks to improve bladder tumor segmentation

Segmentation of bladder tumors from medical radiographic images is of great significance for early detection, diagnosis and prognosis evaluation of bladder cancer. Deep Convolution Neural Networks (DCNNs) have been successfully used for bladder tumor segmentation, but the segmentation based on DCNN is data-hungry for model training and ignores clinical knowledge. From the clinical view, bladder tumors originate from the mucosal surface of bladder and must rely on the bladder wall to survive and grow. This clinical knowledge of tumor location is helpful to improve the bladder tumor segmentation. To achieve this, we propose a novel bladder tumor segmentation method, which incorporates the clinical logic rules of bladder tumor and bladder wall into DCNNs to harness the tumor segmentation. Clinical logical rules provide a semantic and human-readable knowledge representation and are easy for knowledge acquisition from clinicians. In addition, incorporating logical rules of clinical knowledge helps to reduce the data dependency of the segmentation network, and enables precise segmentation results even with limited number of annotated images. Experiments on bladder MR images collected from the collaborating hospital validate the effectiveness of the proposed bladder tumor segmentation method.

The use of deep learning in interventional radiotherapy (brachytherapy): A review with a focus on open source and open data

Deep learning advanced to one of the most important technologies in almost all medical fields. Especially in areas, related to medical imaging it plays a big role. However, in interventional radiotherapy (brachytherapy) deep learning is still in an early phase. In this review, first, we investigated and scrutinised the role of deep learning in all processes of interventional radiotherapy and directly related fields. Additionally, we summarised the most recent developments. For better understanding, we provide explanations of key terms and approaches to solving common deep learning problems. To reproduce results of deep learning algorithms both source code and training data must be available. Therefore, a second focus of this work is on the analysis of the availability of open source, open data and open models. In our analysis, we were able to show that deep learning plays already a major role in some areas of interventional radiotherapy, but is still hardly present in others. Nevertheless, its impact is increasing with the years, partly self-propelled but also influenced by closely related fields. Open source, data and models are growing in number but are still scarce and unevenly distributed among different research groups. The reluctance in publishing code, data and models limits reproducibility and restricts evaluation to mono-institutional datasets. The conclusion of our analysis is that deep learning can positively change the workflow of interventional radiotherapy but there is still room for improvements when it comes to reproducible results and standardised evaluation methods.

Editorial for "Weakly Supervised MRI Slice-Level Deep Learning Classification of Prostate Cancer Approximates Full Voxel- and Slice-Level Annotation: Effect of Increasing Training Set Size"

Level of Evidence

5

Technical Efficacy Stage

1

Predicting central cervical lymph node metastasis in papillary thyroid microcarcinoma using deep learning

Background

The aim of this study is to design a deep learning (DL) model to preoperatively predict the occurrence of central lymph node metastasis (CLNM) in patients with papillary thyroid microcarcinoma (PTMC).

Methods

This research collected preoperative ultrasound (US) images and clinical factors of 611 PTMC patients. The clinical factors were analyzed using multivariate regression. Then, a DL model based on US images and clinical factors was developed to preoperatively predict CLNM. The model's efficacy was evaluated using the receiver operating characteristic (ROC) curve, along with accuracy, sensitivity, specificity, and the F1 score.

Results

The multivariate analysis indicated an independent correlation factors including age ≥55 (OR = 0.309, p < 0.001), tumor diameter (OR = 2.551, p = 0.010), macrocalcifications (OR = 1.832, p = 0.002), and capsular invasion (OR = 1.977, p = 0.005). The suggested DL model utilized US images achieved an average area under the curve (AUC) of 0.65, slightly outperforming the model that employed traditional clinical factors (AUC = 0.64). Nevertheless, the model that incorporated both of them did not enhance prediction accuracy (AUC = 0.63).

Conclusions

The suggested approach offers a reference for the treatment and supervision of PTMC. Among three models used in this study, the deep model relied generally more on image modalities than the data modality of clinic records when making the predictions.

Bladder Cancer and Artificial Intelligence: Emerging Applications

Bladder cancer is a common and heterogeneous disease that poses a significant burden to the patient and health care system. Major unmet needs include effective early detection strategy, imprecision of risk stratification, and treatment-associated morbidities. The existing clinical paradigm is imprecise, which results in missed tumors, suboptimal therapy, and disease progression. Artificial intelligence holds immense potential to address many unmet needs in bladder cancer, including early detection, risk stratification, treatment planning, quality assessment, and outcome prediction. Despite recent advances, extensive work remains to affirm the efficacy of artificial intelligence as a decision-making tool for bladder cancer management.

MM-SFENet: multi-scale multi-task localization and classification of bladder cancer in MRI with spatial feature encoder network

Objective. Bladder cancer is a common malignant urinary carcinoma, with muscle-invasive and non-muscle-invasive as its two major subtypes. This paper aims to achieve automated bladder cancer invasiveness localization and classification based on MRI.Approach. Different from previous efforts that segment bladder wall and tumor, we propose a novel end-to-end multi-scale multi-task spatial feature encoder network (MM-SFENet) for locating and classifying bladder cancer, according to the classification criteria of the spatial relationship between the tumor and bladder wall. First, we built a backbone with residual blocks to distinguish bladder wall and tumor; then, a spatial feature encoder is designed to encode the multi-level features of the backbone to learn the criteria.Main Results. We substitute Smooth-L1 Loss with IoU Loss for multi-task learning, to improve the accuracy of the classification task. By learning two datasets collected from bladder cancer patients at the hospital, the mAP, IoU, Acc, Sen and Spec are used as the evaluation metrics. The experimental result could reach 93.34%, 83.16%, 85.65%, 81.51%, 89.23% on test set1 and 80.21%, 75.43%, 79.52%, 71.87%, 77.86% on test set2.Significance. The experimental result demonstrates the effectiveness of the proposed MM-SFENet on the localization and classification of bladder cancer. It may provide an effective supplementary diagnosis method for bladder cancer staging.

Deep learning models for preoperative T-stage assessment in rectal cancer using MRI: exploring the impact of rectal filling

Background

The objective of this study was twofold: firstly, to develop a convolutional neural network (CNN) for automatic segmentation of rectal cancer (RC) lesions, and secondly, to construct classification models to differentiate between different T-stages of RC. Additionally, it was attempted to investigate the potential benefits of rectal filling in improving the performance of deep learning (DL) models.

Methods

A retrospective study was conducted, including 317 consecutive patients with RC who underwent MRI scans. The datasets were randomly divided into a training set (n = 265) and a test set (n = 52). Initially, an automatic segmentation model based on T2-weighted imaging (T2WI) was constructed using nn-UNet. The performance of the model was evaluated using the dice similarity coefficient (DSC), the 95th percentile Hausdorff distance (HD95), and the average surface distance (ASD). Subsequently, three types of DL-models were constructed: Model 1 trained on the total training dataset, Model 2 trained on the rectal-filling dataset, and Model 3 trained on the non-filling dataset. The diagnostic values were evaluated and compared using receiver operating characteristic (ROC) curve analysis, confusion matrix, net reclassification index (NRI), and decision curve analysis (DCA).

Results

The automatic segmentation showed excellent performance. The rectal-filling dataset exhibited superior results in terms of DSC and ASD (p = 0.006 and 0.017). The DL-models demonstrated significantly superior classification performance to the subjective evaluation in predicting T-stages for all test datasets (all p < 0.05). Among the models, Model 1 showcased the highest overall performance, with an area under the curve (AUC) of 0.958 and an accuracy of 0.962 in the filling test dataset.

Conclusion

This study highlighted the utility of DL-based automatic segmentation and classification models for preoperative T-stage assessment of RC on T2WI, particularly in the rectal-filling dataset. Compared with subjective evaluation, the models exhibited superior performance, suggesting their noticeable potential for enhancing clinical diagnosis and treatment practices.

Deep learning techniques for imaging diagnosis of renal cell carcinoma: current and emerging trends

This study summarizes the latest achievements, challenges, and future research directions in deep learning technologies for the diagnosis of renal cell carcinoma (RCC). This is the first review of deep learning in RCC applications. This review aims to show that deep learning technologies hold great promise in the field of RCC diagnosis, and we look forward to more research results to meet us for the mutual benefit of renal cell carcinoma patients. Medical imaging plays an important role in the early detection of renal cell carcinoma (RCC), as well as in the monitoring and evaluation of RCC during treatment. The most commonly used technologies such as contrast enhanced computed tomography (CECT), ultrasound and magnetic resonance imaging (MRI) are now digitalized, allowing deep learning to be applied to them. Deep learning is one of the fastest growing fields in the direction of medical imaging, with rapidly emerging applications that have changed the traditional medical treatment paradigm. With the help of deep learning-based medical imaging tools, clinicians can diagnose and evaluate renal tumors more accurately and quickly. This paper describes the application of deep learning-based imaging techniques in RCC assessment and provides a comprehensive review.

Artificial Intelligence in the Advanced Diagnosis of Bladder Cancer-Comprehensive Literature Review and Future Advancement

Artificial intelligence is highly regarded as the most promising future technology that will have a great impact on healthcare across all specialties. Its subsets, machine learning, deep learning, and artificial neural networks, are able to automatically learn from massive amounts of data and can improve the prediction algorithms to enhance their performance. This area is still under development, but the latest evidence shows great potential in the diagnosis, prognosis, and treatment of urological diseases, including bladder cancer, which are currently using old prediction tools and historical nomograms. This review focuses on highly significant and comprehensive literature evidence of artificial intelligence in the management of bladder cancer and investigates the near introduction in clinical practice.

A U-Shaped Network Based on Multi-level Feature and Dual-Attention Coordination Mechanism for Coronary Artery Segmentation of CCTA Images

PURPOSE

Computed tomography coronary angiography (CCTA) images provide optimal visualization of coronary arteries to aid in diagnosing coronary heart disease (CHD). With the deep convolutional neural network, this work aims to develop an intelligent and lightweight coronary artery segmentation algorithm that can be deployed in hospital systems to assist clinicians in quantitatively analyzing CHD.

METHODS

With the multi-level feature fusion, we proposed Dual-Attention Coordination U-Net (DAC-UNet) that achieves automated coronary artery segmentation in 2D CCTA images. The coronary artery occupies a small region, and the foreground and background are extremely unbalanced. For this reason, the more original information can be retained by fusing related features between adjacent layers, which is conducive to recovering the small coronary artery area. The dual-attention coordination mechanism can select valid information and filter redundant information. Moreover, the complementation and coordination of double attention factors can enhance the integrity of features of coronary arteries, reduce the interference of non-coronary arteries, and prevent over-learning. With gradual learning, the balanced character of double attention factors promotes the generalization ability of the model to enhance coronary artery localization and contour detail segmentation.

RESULTS

Compared with existing related segmentation methods, our method achieves a certain degree of improvement in 2D CCTA images for the segmentation accuracy of coronary arteries with a mean Dice index of 0.7920. Furthermore, the method can obtain relatively accurate results even in a small sample dataset and is easy to implement and deploy, which is promising. The code is available at: https://github.com/windfly666/Segmentation .

CONCLUSION

Our method can capture the coronary artery structure end-to-end, which can be used as a fundamental means for automatic detection of coronary artery stenosis, blood flow reserve fraction analysis, and assisting clinicians in diagnosing CHD.

Automatic segmentation of bladder cancer on MRI using a convolutional neural network and reproducibility of radiomics features: a two-center study

This study aimed to develop a versatile automatic segmentation model of bladder cancer (BC) on MRI using a convolutional neural network and investigate the robustness of radiomics features automatically extracted from apparent diffusion coefficient (ADC) maps. This two-center retrospective study used multi-vendor MR units and included 170 patients with BC, of whom 140 were assigned to training datasets for the modified U-net model with five-fold cross-validation and 30 to test datasets for assessment of segmentation performance and reproducibility of automatically extracted radiomics features. For model input data, diffusion-weighted images with b = 0 and 1000 s/mm², ADC maps, and multi-sequence images (b0-b1000-ADC maps) were used. Segmentation accuracy was compared between ours and existing models. The reproducibility of radiomics features on ADC maps was evaluated using intraclass correlation coefficient. The model with multi-sequence images achieved the highest Dice similarity coefficient (DSC) with five-fold cross-validation (mean DSC = 0.83 and 0.79 for the training and validation datasets, respectively). The median (interquartile range) DSC of the test dataset model was 0.81 (0.70-0.88). Radiomics features extracted from manually and automatically segmented BC exhibited good reproducibility. Thus, our U-net model performed highly accurate segmentation of BC, and radiomics features extracted from the automatic segmentation results exhibited high reproducibility.

Automated periodontitis bone loss diagnosis in panoramic radiographs using a bespoke two-stage detector

Periodontitis is a serious oral disease that can lead to severe conditions such as bone loss and teeth falling out if left untreated. Diagnosis of radiographic bone loss (RBL) is critical for the staging and treatment of periodontitis. Unfortunately, the RBL diagnosis by examining the panoramic radiographs is time-consuming. The demand for automated image analysis is urgent. However, existing deep learning methods have limited performances in diagnosis accuracy and have certain difficulties in implementation. Hence, we propose a novel two-stage periodontitis detection convolutional neural network (PDCNN), where we optimize the detector with an anchor-free encoding that allows fast and accurate prediction. We also introduce a proposal-connection module in our detector that excludes less relevant regions of interests (ROIs), making the network focus on more relevant ROIs to improve detection accuracy. Furthermore, we introduced a large-scale, high-resolution panoramic radiograph dataset that captures various complex cases with professional periodontitis annotations. Experiments on our panoramic-image dataset show that the proposed approach achieved an RBL classification accuracy of 0.762. This result shows that our approach outperforms state-of-the-art detectors such as Faster R-CNN and YOLO-v4. We can conclude that the proposed method successfully improves the RBL detection performance. The dataset and our code have been released on GitHub. (https://github.com/PuckBlink/PDCNN).

Computed Tomography slice interpolation in the longitudinal direction based on deep learning techniques: To reduce slice thickness or slice increment without dose increase

Large slice thickness or slice increment causes information insufficiency of Computed Tomography (CT) data in the longitudinal direction, which degrades the quality of CT-based diagnosis. Traditional approaches such as high-resolution computed tomography (HRCT) and linear interpolation can solve this problem. However, HRCT suffers from dose increase, and linear interpolation causes artifacts. In this study, we propose a deep-learning-based approach to reconstruct densely sliced CT from sparsely sliced CT data without any dose increase. The proposed method reconstructs CT images from neighboring slices using a U-net architecture. To prevent multiple reconstructed slices from influencing one another, we propose a parallel architecture in which multiple U-net architectures work independently. Moreover, for a specific organ (i.e., the liver), we propose a range-clip technique to improve reconstruction quality, which enhances the learning of CT values within this organ by enlarging the range of the training data. CT data from 130 patients were collected, with 80% used for training and the remaining 20% used for testing. Experiments showed that our parallel U-net architecture reduced the mean absolute error of CT values in the reconstructed slices by 22.05%, and also reduced the incidence of artifacts around the boundaries of target organs, compared with linear interpolation. Further improvements of 15.12%, 11.04%, 10.94%, and 10.63% were achieved for the liver, left kidney, right kidney, and stomach, respectively, using the proposed range-clip algorithm. Also, we compared the proposed architecture with original U-net method, and the experimental results demonstrated the superiority of our approach.

Deep learning in bladder cancer imaging: A review

Deep learning (DL) is a rapidly developing field in machine learning (ML). The concept of deep learning originates from research on artificial neural networks and is an upgrade of traditional neural networks. It has achieved great success in various domains and has shown potential in solving medical problems, particularly when using medical images. Bladder cancer (BCa) is the tenth most common cancer in the world. Imaging, as a safe, noninvasive, and relatively inexpensive technique, is a powerful tool to aid in the diagnosis and treatment of bladder cancer. In this review, we provide an overview of the latest progress in the application of deep learning to the imaging assessment of bladder cancer. First, we review the current deep learning approaches used for bladder segmentation. We then provide examples of how deep learning helps in the diagnosis, staging, and treatment management of bladder cancer using medical images. Finally, we summarize the current limitations of deep learning and provide suggestions for future improvements.

LARNet-STC: Spatio-temporal orthogonal region selection network for laryngeal closure detection in endoscopy videos

The vocal folds (VFs) are a pair of muscles in the larynx that play a critical role in breathing, swallowing, and speaking. VF function can be adversely affected by various medical conditions including head or neck injuries, stroke, tumor, and neurological disorders. In this paper, we propose a deep learning system for automated detection of laryngeal adductor reflex (LAR) events in laryngeal endoscopy videos to enable objective, quantitative analysis of VF function. The proposed deep learning system incorporates our novel orthogonal region selection network and temporal context. This network learns to directly map its input to a VF open/close state without first segmenting or tracking the VF region. This one-step approach drastically reduces manual annotation needs from labor-intensive segmentation masks or VF motion tracks to frame-level class labels. The proposed spatio-temporal network with an orthogonal region selection subnetwork allows integration of local image features, global image features, and VF state information in time for robust LAR event detection. The proposed network is evaluated against several network variations that incorporate temporal context and is shown to lead to better performance. The experimental results show promising performance for automated, objective, and quantitative analysis of LAR events from laryngeal endoscopy videos with over 90% and 99% F1 scores for LAR and non-LAR frames respectively.

Cascade Path Augmentation Unet for Bladder Cancer Segmentation in MRI

BACKGROUND

Treatment choices for patients with bladder cancer are determined by the presence of muscular invasion. The precise segmentation of the inner and outer walls (IW and OW), as well as the bladder tumor (BT), is crucial for improving computer-aided diagnosis of muscle-invasive bladder cancer.

PURPOSE

To propose a novel deep learning-based model to improve the segmentation accuracy of the IW, OW, and BT, which can be useful in clinical practice.

METHODS

We proposed a Cascade Path Augmentation Unet (CPA-Unet) network to conduct multi-regional segmentation of the bladder using 1545 T2 weighted MRI scans. The model employs a cascade strategy to eliminate the redundant information in the background. Unet is used to segment the bladder from the background in the rough segmentation. The path augmentation structure is used in the fine segmentation to mine multi-scale features. Additionally, the partial dense connection is adopted as the skip connection module to concatenate the low and high-level sematic features.

RESULTS

The CPA-Unet is trained using 1391 T2WI slices and tested using 154 T2WI slices. In comparison to previous deep learning-based methods, the CPA-Unet achieves superior segmentation results in terms of dice similarity coefficient (DSC) and Hausdorff distance (HD). (IW: DSC = 98.19%, HD = 2.07mm; OW: DSC = 82.24%, HD = 2.62mm; BT: DSC = 87.40%, HD = 0.76mm).

CONCLUSIONS

Our proposed CPA-Unet network is capable of segmenting the bladder into its IW and OW, as well as tumors. The segmentation results provide a reliable and effective foundation for computer-assisted clinical diagnosis of muscle-invasive bladder cancer. This article is protected by copyright. All rights reserved.

Machine Learning for Auto-Segmentation in Radiotherapy Planning

Manual segmentation of target structures and organs at risk is a crucial step in the radiotherapy workflow. It has the disadvantages that it can require several hours of clinician time per patient and is prone to inter- and intra-observer variability. Automatic segmentation (auto-segmentation), using computer algorithms, seeks to address these issues. Advances in machine learning and computer vision have led to the development of methods for accurate and efficient auto-segmentation. This review surveys auto-segmentation techniques and applications in radiotherapy planning. It provides an overview of traditional approaches to auto-segmentation, including intensity analysis, shape modelling and atlas-based methods. The focus, though, is on uses of machine learning and deep learning, including convolutional neural networks. Finally, the future of machine-learning-driven auto-segmentation in clinical settings is considered, and the barriers that must be overcome for it to be widely accepted into routine practice are highlighted.

MRI as a Tool to Assess Interstitial Cystitis Associated Bladder and Brain Pathologies

Interstitial cystitis/bladder pain syndrome (IC/BPS) is a chronic, often incapacitating condition characterized by pain seeming to originate in the bladder in conjunction with lower urinary tract symptoms of frequency and urgency, and consists of a wide range of clinical phenotypes with diverse etiologies. There are currently no diagnostic tests for IC/BPS. Magnetic resonance imaging (MRI) is a relatively new tool to assess IC/BPS. There are several methodologies that can be applied to assess either bladder wall or brain-associated alterations in tissue morphology and/or pain. IC/BPS is commonly associated with bladder wall hyperpermeability (BWH), particularly in severe cases. Our group developed a contrast-enhanced magnetic resonance imaging (CE-MRI) approach to assess BWH in preclinical models for IC/BPS, as well as for a pilot study for IC/BPS patients. We have also used the CE-MRI approach to assess possible therapies to alleviate the BWH in preclinical models for IC/BPS, which will hopefully pave the way for future clinical trials. In addition, we have used molecular-targeted MRI (mt-MRI) to quantitatively assess BWH biomarkers. Biomarkers, such as claudin-2, may be important to assess and determine the severity of BWH, as well as to assess therapeutic efficacy. Others have also used other MRI approaches to assess the bladder wall structural alterations with diffusion-weighted imaging (DWI), by measuring changes in the apparent diffusion coefficient (ADC), diffusion tensor imaging (DTI), as well as using functional MRI (fMRI) to assess pain and morphological MRI or DWI to assess anatomical or structural changes in the brains of patients with IC/BPS. It would be beneficial if MRI-based diagnostic tests could be routinely used for these patients and possibly used to assess potential therapeutics.