3D Fast Automatic Segmentation of Kidney Based on Modified AAM and Random Forest

Pre-screening of endomysial microvessel density by fast random forest image processing machine learning algorithm accelerates recognition of a modified vascular network in idiopathic inflammatory myopathies

Biomarkers for discrimination among different subgroups of idiopathic inflammatory myopathies (IIM) are difficult to identify and may involve multiple laboratory tests and time-consuming procedures. We assessed the potential for artificial intelligence (AI) to extract features such as density of endomysial microvessels based on automatic analysis of the CD31+ vascular network on muscle biopsy images. We also assessed the potential of this technique to save time and its agreement rate with analyses based on the manual selection of microvessels from the same images. A total of 84 images from 84 patients with IIM, diagnosed between 2014 and 2020, were retrieved and analyzed using the Fast Random Forest (FRF) technique. We built a lightweight and explainable algorithm for calculating the pixel percentage of CD31+ endomysial capillaries. The FRF technique applied on images of CD31-stained muscle sections achieved a good performance in the recognition of microvessels by estimating their density over a standard area corresponding to a sample of microscope image. The time spent for this analysis was 90% less than the manual choice of microvessels (estimated time considering the computational time and the time spent to manually detecting the microvessels features). The good performance of the FRF demonstrates that the CD31 pixel percentage of endomysial capillaries is sufficient for a correct estimation. Finally, the paper proposes a procedure to integrate AI in the pre-screening process.

Efficient Kidney Tumor Classification and Segmentation with U-Net

A novel approach to kidney tumor identification is introduced, which integrates kidney tumor classification and segmentation algorithms. The multi-faceted approach commences with the classification of kidney images, adaptly discerning between normal and tumor instances. For individuals identified as tumor-positive, a sophisticated UNet-based architecture is intricately employed to achieve precise segmentation, capturing nuanced details of both kidney and tumor regions. Many models, including VGG16, MobileNetV3, DenseNet50, and others, were tested in order to achieve this. Among these, MobilenetV3 performs better than the others in terms of accuracy, with a 99.1% accuracy rate and a 99% precision rate for classification. In this research, we applied a novel U-Net model to accurately segregate kidney and kidney tumor from CT scan data. With this, an average dice coefficient score of 0.9445 is obtained. In advancing the landscape of kidney tumor analysis, this proposed strategy not only bridges classification and segmentation but also showcases a significant leap toward refined clinical applications.

Imaging Markers Derived From MRI-Based Automated Kidney Segmentation—an Analysis of Data From the German National Cohort (NAKO Gesundheitsstudie)

BACKGROUND

Population-wide research on potential new imaging biomarkers of the kidney depends on accurate automated segmentation of the kidney and its compartments (cortex, medulla, and sinus).

METHODS

We developed a robust deep-learning framework for kidney (sub-)segmentation based on a hierarchical, three-dimensional convolutional neural network (CNN) that was optimized for multiscale problems of combined localization and segmentation. We applied the CNN to abdominal magnetic resonance images from the population-based German National Cohort (NAKO) study.

RESULTS

There was good to excellent agreement between the model predictions and manual segmentations. The median values for the body-surface normalized total kidney, cortex, medulla, and sinus volumes of 9934 persons were 158, 115, 43, and 24 mL/m2. Distributions of these markers are provided both for the overall study population and for a subgroup of persons without kidney disease or any associated conditions. Multivariable adjusted regression analyses revealed that diabetes, male sex, and a higher estimated glomerular filtration rate (eGFR) are important predictors of higher total and cortical volumes. Each increase of eGFR by one unit (i.e., 1 mL/min per 1.73 m2 body surface area) was associated with a 0.98 mL/m2 increase in total kidney volume, and this association was significant. Volumes were lower in persons with eGFR-defined chronic kidney disease.

CONCLUSION

The extraction of image-based biomarkers through CNN-based renal sub-segmentation using data from a population-based study yields reliable results, forming a solid foundation for future investigations.

PSMA-PET improves deep learning-based automated CT kidney segmentation

For dosimetry of radiopharmaceutical therapies, it is essential to determine the volume of relevant structures exposed to therapeutic radiation. For many radiopharmaceuticals, the kidneys represent an important organ-at-risk. To reduce the time required for kidney segmentation, which is often still performed manually, numerous approaches have been presented in recent years to apply deep learning-based methods for CT-based automated segmentation. While the automatic segmentation methods presented so far have been based solely on CT information, the aim of this work is to examine the added value of incorporating PSMA-PET data in the automatic kidney segmentation.

METHODS

A total of 108 PET/CT examinations (53 [68Ga]Ga-PSMA-I&T and 55 [18F]F-PSMA-1007 examinations) were grouped to create a reference data set of manual segmentations of the kidney. These segmentations were performed by a human examiner. For each subject, two segmentations were carried out: one CT-based (detailed) segmentation and one PET-based (coarser) segmentation. Five different u-net based approaches were applied to the data set to perform an automated segmentation of the kidney: CT images only, PET images only (coarse segmentation), a combination of CT and PET images, a combination of CT images and a PET-based coarse mask, and a CT image, which had been pre-segmented using a PET-based coarse mask. A quantitative assessment of these approaches was performed based on a test data set of 20 patients, including Dice score, volume deviation and average Hausdorff distance between automated and manual segmentations. Additionally, a visual evaluation of automated segmentations for 100 additional (i.e., exclusively automatically segmented) patients was performed by a nuclear physician.

RESULTS

Out of all approaches, the best results were achieved by using CT images which had been pre-segmented using a PET-based coarse mask as input. In addition, this method performed significantly better than the segmentation based solely on CT, which was supported by the visual examination of the additional segmentations. In 80% of the cases, the segmentations created by exploiting the PET-based pre-segmentation were preferred by the nuclear physician.

CONCLUSION

This study shows that deep-learning based kidney segmentation can be significantly improved through the addition of a PET-based pre-segmentation. The presented method was shown to be especially beneficial for kidneys with cysts or kidneys that are closely adjacent to other organs such as the spleen, liver or pancreas. In the future, this could lead to a considerable reduction in the time required for dosimetry calculations as well as an improvement in the results.

Multi-Scale and Spatial Information Extraction for Kidney Tumor Segmentation: A Contextual Deformable Attention and Edge-Enhanced U-Net

Kidney tumor segmentation is a difficult task because of the complex spatial and volumetric information present in medical images. Recent advances in deep convolutional neural networks (DCNNs) have improved tumor segmentation accuracy. However, the practical usability of current CNN-based networks is constrained by their high computational complexity. Additionally, these techniques often struggle to make adaptive modifications based on the structure of the tumors, which can lead to blurred edges in segmentation results. A lightweight architecture called the contextual deformable attention and edge-enhanced U-Net (CDA2E-Net) for high-accuracy pixel-level kidney tumor segmentation is proposed to address these challenges. Rather than using complex deep encoders, the approach includes a lightweight depthwise dilated ShuffleNetV2 (LDS-Net) encoder integrated into the CDA2E-Net framework. The proposed method also contains a multiscale attention feature pyramid pooling (MAF2P) module that improves the ability of multiscale features to adapt to various tumor shapes. Finally, an edge-enhanced loss function is introduced to guide the CDA2E-Net to concentrate on tumor edge information. The CDA2E-Net is evaluated on the KiTS19 and KiTS21 datasets, and the results demonstrate its superiority over existing approaches in terms of Hausdorff distance (HD), intersection over union (IoU), and dice coefficient (DSC) metrics.

UNesT: Local spatial representation learning with hierarchical transformer for efficient medical segmentation

Transformer-based models, capable of learning better global dependencies, have recently demonstrated exceptional representation learning capabilities in computer vision and medical image analysis. Transformer reformats the image into separate patches and realizes global communication via the self-attention mechanism. However, positional information between patches is hard to preserve in such 1D sequences, and loss of it can lead to sub-optimal performance when dealing with large amounts of heterogeneous tissues of various sizes in 3D medical image segmentation. Additionally, current methods are not robust and efficient for heavy-duty medical segmentation tasks such as predicting a large number of tissue classes or modeling globally inter-connected tissue structures. To address such challenges and inspired by the nested hierarchical structures in vision transformer, we proposed a novel 3D medical image segmentation method (UNesT), employing a simplified and faster-converging transformer encoder design that achieves local communication among spatially adjacent patch sequences by aggregating them hierarchically. We extensively validate our method on multiple challenging datasets, consisting of multiple modalities, anatomies, and a wide range of tissue classes, including 133 structures in the brain, 14 organs in the abdomen, 4 hierarchical components in the kidneys, inter-connected kidney tumors and brain tumors. We show that UNesT consistently achieves state-of-the-art performance and evaluate its generalizability and data efficiency. Particularly, the model achieves whole brain segmentation task complete ROI with 133 tissue classes in a single network, outperforming prior state-of-the-art method SLANT27 ensembled with 27 networks. Our model performance increases the mean DSC score of the publicly available Colin and CANDI dataset from 0.7264 to 0.7444 and from 0.6968 to 0.7025, respectively. Code, pre-trained models, and use case pipeline are available at: https://github.com/MASILab/UNesT.

Robust and efficient abdominal CT segmentation using shape constrained multi-scale attention network

PURPOSE

Although many deep learning-based abdominal multi-organ segmentation networks have been proposed, the various intensity distributions and organ shapes of the CT images from multi-center, multi-phase with various diseases introduce new challenges for robust abdominal CT segmentation. To achieve robust and efficient abdominal multi-organ segmentation, a new two-stage method is presented in this study.

METHODS

A binary segmentation network is used for coarse localization, followed by a multi-scale attention network for the fine segmentation of liver, kidney, spleen, and pancreas. To constrain the organ shapes produced by the fine segmentation network, an additional network is pre-trained to learn the shape features of the organs with serious diseases and then employed to constrain the training of the fine segmentation network.

RESULTS

The performance of the presented segmentation method was extensively evaluated on the multi-center data set from the Fast and Low GPU Memory Abdominal oRgan sEgmentation (FLARE) challenge, which was held in conjunction with International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2021. Dice Similarity Coefficient (DSC) and Normalized Surface Dice (NSD) were calculated to quantitatively evaluate the segmentation accuracy and efficiency. An average DSC and NSD of 83.7% and 64.4% were achieved, and our method finally won the second place among more than 90 participating teams.

CONCLUSIONS

The evaluation results on the public challenge demonstrate that our method shows promising performance in robustness and efficiency, which may promote the clinical application of the automatic abdominal multi-organ segmentation.

Tumorous kidney segmentation in abdominal CT images using active contour and 3D-UNet

BACKGROUND AND PURPOSE

The precise segmentation of the kidneys in computed tomography (CT) images is vital in urology for diagnosis, treatment, and surgical planning. Medical experts can get assistance through segmentation, as it provides information about kidney malformations in terms of shape and size. Manual segmentation is slow, tedious, and not reproducible. An automatic computer-aided system is a solution to this problem. This paper presents an automated kidney segmentation technique based on active contour and deep learning.

MATERIALS AND METHODS

In this work, 210 CTs from the KiTS 19 repository were used. The used dataset was divided into a train set (168 CTs), test set (21 CTs), and validation set (21 CTs). The suggested technique has broadly four phases: (1) extraction of kidney regions using active contours, (2) preprocessing, (3) kidney segmentation using 3D U-Net, and (4) reconstruction of the segmented CT images.

RESULTS

The proposed segmentation method has received the Dice score of 97.62%, Jaccard index of 95.74%, average sensitivity of 98.28%, specificity of 99.95%, and accuracy of 99.93% over the validation dataset.

CONCLUSION

The proposed method can efficiently solve the problem of tumorous kidney segmentation in CT images by using active contour and deep learning. The active contour was used to select kidney regions and 3D-UNet was used for precisely segmenting the tumorous kidney.

A deep learning-based precision and automatic kidney segmentation system using efficient feature pyramid networks in computed tomography images

This paper proposes an encoder-decoder architecture for kidney segmentation. A hyperparameter optimization process is implemented, including the development of a model architecture, selecting a windowing method and a loss function, and data augmentation. The model consists of EfficientNet-B5 as the encoder and a feature pyramid network as the decoder that yields the best performance with a Dice score of 0.969 on the 2019 Kidney and Kidney Tumor Segmentation Challenge dataset. The proposed model is tested with different voxel spacing, anatomical planes, and kidney and tumor volumes. Moreover, case studies are conducted to analyze segmentation outliers. Finally, five-fold cross-validation and the 3D-IRCAD-01 dataset are used to evaluate the developed model in terms of the following evaluation metrics: the Dice score, recall, precision, and the Intersection over Union score. A new development and application of artificial intelligence algorithms to solve image analysis and interpretation will be demonstrated in this paper. Overall, our experiment results show that the proposed kidney segmentation solutions in CT images can be significantly applied to clinical needs to assist surgeons in surgical planning. It enables the calculation of the total kidney volume for kidney function estimation in ADPKD and supports radiologists or doctors in disease diagnoses and disease progression.

A critical review on diagnosis of diabetic retinopathy using machine learning and deep learning

Diabetic Retinopathy (DR) is a health condition caused due to Diabetes Mellitus (DM). It causes vision problems and blindness due to disfigurement of human retina. According to statistics, 80% of diabetes patients battling from long diabetic period of 15 to 20 years, suffer from DR. Hence, it has become a dangerous threat to the health and life of people. To overcome DR, manual diagnosis of the disease is feasible but overwhelming and cumbersome at the same time and hence requires a revolutionary method. Thus, such a health condition necessitates primary recognition and diagnosis to prevent DR from developing into severe stages and prevent blindness. Innumerable Machine Learning (ML) models are proposed by researchers across the globe, to achieve this purpose. Various feature extraction techniques are proposed for extraction of DR features for early detection. However, traditional ML models have shown either meagre generalization throughout feature extraction and classification for deploying smaller datasets or consumes more of training time causing inefficiency in prediction while using larger datasets. Hence Deep Learning (DL), a new domain of ML, is introduced. DL models can handle a smaller dataset with help of efficient data processing techniques. However, they generally incorporate larger datasets for their deep architectures to enhance performance in feature extraction and image classification. This paper gives a detailed review on DR, its features, causes, ML models, state-of-the-art DL models, challenges, comparisons and future directions, for early detection of DR.

Multiorgan segmentation from partially labeled datasets with conditional nnU-Net

Accurate and robust multiorgan abdominal CT segmentation plays a significant role in numerous clinical applications, such as therapy treatment planning and treatment delivery. Almost all existing segmentation networks rely on fully annotated data with strong supervision. However, annotating fully annotated multiorgan data in CT images is both laborious and time-consuming. In comparison, massive partially labeled datasets are usually easily accessible. In this paper, we propose conditional nnU-Net trained on the union of partially labeled datasets for multiorgan segmentation. The deep model employs the state-of-the-art nnU-Net as the backbone and introduces a conditioning strategy by feeding auxiliary information into the decoder architecture as an additional input layer. This model leverages the prior conditional information to identify the organ class at the pixel-wise level and encourages organs' spatial information recovery. Furthermore, we adopt a deep supervision mechanism to refine the outputs at different scales and apply the combination of Dice loss and Focal loss to optimize the training model. Our proposed method is evaluated on seven publicly available datasets of the liver, pancreas, spleen and kidney, in which promising segmentation performance has been achieved. The proposed conditional nnU-Net breaks down the barriers between nonoverlapping labeled datasets and further alleviates the problem of data hunger in multiorgan segmentation.

Meta grayscale adaptive network for 3D integrated renal structures segmentation

Three-dimensional (3D) integrated renal structures (IRS) segmentation targets segmenting the kidneys, renal tumors, arteries, and veins in one inference. Clinicians will benefit from the 3D IRS visual model for accurate preoperative planning and intraoperative guidance of laparoscopic partial nephrectomy (LPN). However, no success has been reported in 3D IRS segmentation due to the inherent challenges in grayscale distribution: low contrast caused by the narrow task-dependent distribution range of regions of interest (ROIs), and the networks representation preferences caused by the distribution variation inter-images. In this paper, we propose the Meta Greyscale Adaptive Network (MGANet), the first deep learning framework to simultaneously segment the kidney, renal tumors, arteries and veins on CTA images in one inference. It makes innovations in two collaborate aspects: 1) The Grayscale Interest Search (GIS) adaptively focuses segmentation networks on task-dependent grayscale distributions via scaling the window width and center with two cross-correlated coefficients for the first time, thus learning the fine-grained representation for fine segmentation. 2) The Meta Grayscale Adaptive (MGA) learning makes an image-level meta-learning strategy. It represents diverse robust features from multiple distributions, perceives the distribution characteristic, and generates the model parameters to fuse features dynamically according to image's distribution, thus adapting the grayscale distribution variation. This study enrolls 123 patients and the average Dice coefficients of the renal structures are up to 87.9%. Fine selection of the task-dependent grayscale distribution ranges and personalized fusion of multiple representations on different distributions will lead to better 3D IRS segmentation quality. Extensive experiments with promising results on renal structures reveal powerful segmentation accuracy and great clinical significance in renal cancer treatment.

Advances in Auto-Segmentation

Manual image segmentation is a time-consuming task routinely performed in radiotherapy to identify each patient's targets and anatomical structures. The efficacy and safety of the radiotherapy plan requires accurate segmentations as these regions of interest are generally used to optimize and assess the quality of the plan. However, reports have shown that this process can be subject to significant inter- and intraobserver variability. Furthermore, the quality of the radiotherapy treatment, and subsequent analyses (ie, radiomics, dosimetric), can be subject to the accuracy of these manual segmentations. Automatic segmentation (or auto-segmentation) of targets and normal tissues is, therefore, preferable as it would address these challenges. Previously, auto-segmentation techniques have been clustered into 3 generations of algorithms, with multiatlas based and hybrid techniques (third generation) being considered the state-of-the-art. More recently, however, the field of medical image segmentation has seen accelerated growth driven by advances in computer vision, particularly through the application of deep learning algorithms, suggesting we have entered the fourth generation of auto-segmentation algorithm development. In this paper, the authors review traditional (nondeep learning) algorithms particularly relevant for applications in radiotherapy. Concepts from deep learning are introduced focusing on convolutional neural networks and fully-convolutional networks which are generally used for segmentation tasks. Furthermore, the authors provide a summary of deep learning auto-segmentation radiotherapy applications reported in the literature. Lastly, considerations for clinical deployment (commissioning and QA) of auto-segmentation software are provided.

A convolutional neural network algorithm for automatic segmentation of head and neck organs at risk using deep lifelong learning

PURPOSE

This study suggests a lifelong learning-based convolutional neural network (LL-CNN) algorithm as a superior alternative to single-task learning approaches for automatic segmentation of head and neck (OARs) organs at risk.

METHODS AND MATERIALS

Lifelong learning-based convolutional neural network was trained on twelve head and neck OARs simultaneously using a multitask learning framework. Once the weights of the shared network were established, the final multitask convolutional layer was replaced by a single-task convolutional layer. The single-task transfer learning network was trained on each OAR separately with early stoppage. The accuracy of LL-CNN was assessed based on Dice score and root-mean-square error (RMSE) compared to manually delineated contours set as the gold standard. LL-CNN was compared with 2D-UNet, 3D-UNet, a single-task CNN (ST-CNN), and a pure multitask CNN (MT-CNN). Training, validation, and testing followed Kaggle competition rules, where 160 patients were used for training, 20 were used for internal validation, and 20 in a separate test set were used to report final prediction accuracies.

RESULTS

On average contours generated with LL-CNN had higher Dice coefficients and lower RMSE than 2D-UNet, 3D-Unet, ST- CNN, and MT-CNN. LL-CNN required ~72 hrs to train using a distributed learning framework on 2 Nvidia 1080Ti graphics processing units. LL-CNN required 20 s to predict all 12 OARs, which was approximately as fast as the fastest alternative methods with the exception of MT-CNN.

CONCLUSIONS

This study demonstrated that for head and neck organs at risk, LL-CNN achieves a prediction accuracy superior to all alternative algorithms.

Kidney segmentation in ultrasound, magnetic resonance and computed tomography images: A systematic review

BACKGROUND AND OBJECTIVE

Segmentation is an essential step in computer-aided diagnosis and treatment planning of kidney diseases. In recent years, several researchers proposed multiple techniques to segment the kidney in medical images from distinct imaging acquisition systems, namely ultrasound, magnetic resonance, and computed tomography. This article aims to present a systematic review of the different methodologies developed for kidney segmentation.

METHODS

With this work, it is intended to analyze and categorize the different kidney segmentation algorithms, establishing a comparison between them and discussing the most appropriate methods for each modality. For that, articles published between 2010 and 2016 were analyzed. The search was performed in Scopus and Web of Science using the expressions "kidney segmentation" and "renal segmentation".

RESULTS

A total of 1528 articles were retrieved from the databases, and 95 articles were selected for this review. After analysis of the selected articles, the reviewed segmentation techniques were categorized according to their theoretical approach.

CONCLUSIONS

Based on the performed analysis, it was possible to identify segmentation approaches based on distinct image processing classes that can be used to accurately segment the kidney in images of different imaging modalities. Nevertheless, further research on kidney segmentation must be conducted to overcome the current drawbacks of the state-of-the-art methods. Moreover, a standardization of the evaluation database and metrics is needed to allow a direct comparison between methods.

Automatic detection of microaneurysms in retinal fundus images

Diabetic retinopathy (DR) is one of the leading causes of new cases of blindness. Early and accurate detection of microaneurysms (MAs) is important for diagnosis and grading of diabetic retinopathy. In this paper, a new method for the automatic detection of MAs in eye fundus images is proposed. The proposed method consists of four main steps: preprocessing, candidate extraction, feature extraction and classification. A total of 27 characteristic features which contain local features and profile features are extracted for KNN classifier to distinguish true MAs from spurious candidates. The proposed method has been evaluated on two public database: ROC and e-optha. The experimental result demonstrates the efficiency and effectiveness of the proposed method, and it has the potential to be used to diagnose DR clinically.