Multi-view convolutional neural networks for automated ocular structure and tumor segmentation in retinoblastoma

Convolutional Neural Networks to Study Contrast-Enhanced Magnetic Resonance Imaging-Based Skeletal Calf Muscle Perfusion in Peripheral Artery Disease

Peripheral artery disease (PAD) is associated with impaired blood flow in the lower extremities and histopathologic changes of the skeletal calf muscles, resulting in abnormal microvascular perfusion. We studied the use of convolution neural networks (CNNs) to differentiate patients with PAD from matched controls using perfusion pattern features from contrast-enhanced magnetic resonance imaging (CE-MRI) of the skeletal calf muscles. We acquired CE-MRI based skeletal calf muscle perfusion in 56 patients (36 patients with PAD and 20 matched controls). Microvascular perfusion imaging was performed after reactive hyperemia at the midcalf level, with a temporal resolution of 409 ms. We analyzed perfusion scans up to 2 minutes indexed from the local precontrast arrival time frame. Skeletal calf muscles, including the anterior muscle, lateral muscle, deep posterior muscle group, and the soleus and gastrocnemius muscles, were segmented semiautomatically. Segmented muscles were represented as 3-dimensional Digital Imaging and Communications in Medicine stacks of CE-MRI perfusion scans for deep learning (DL) analysis. We tested several CNN models for the 3-dimensional CE-MRI perfusion stacks to classify patients with PAD from matched controls. A total of 2 of the best performing CNNs (resNet and divNet) were selected to develop the final classification model. A peak accuracy of 75% was obtained for resNet and divNet. Specificity was 80% and 94% for resNet and divNet, respectively. In conclusion, DL using CNNs and CE-MRI skeletal calf muscle perfusion can discriminate patients with PAD from matched controls. DL methods may be of interest for the study of PAD.

MR Imaging of Adverse Effects and Ocular Growth Decline after Selective Intra-Arterial Chemotherapy for Retinoblastoma

This retrospective multicenter study examines therapy-induced orbital and ocular MRI findings in retinoblastoma patients following selective intra-arterial chemotherapy (SIAC) and quantifies the impact of SIAC on ocular and optic nerve growth. Patients were selected based on medical chart review, with inclusion criteria requiring the availability of posttreatment MR imaging encompassing T2-weighted and T1-weighted images (pre- and post-intravenous gadolinium administration). Qualitative features and quantitative measurements were independently scored by experienced radiologists, with deep learning segmentation aiding total eye volume assessment. Eyes were categorized into three groups: eyes receiving SIAC (Rb-SIAC), eyes treated with other eye-saving methods (Rb-control), and healthy eyes. The most prevalent adverse effects post-SIAC were inflammatory and vascular features, with therapy-induced contrast enhancement observed in the intraorbital optic nerve segment in 6% of patients. Quantitative analysis revealed significant growth arrest in Rb-SIAC eyes, particularly when treatment commenced ≤ 12 months of age. Optic nerve atrophy was a significant complication in Rb-SIAC eyes. In conclusion, this study highlights the vascular and inflammatory adverse effects observed post-SIAC in retinoblastoma patients and demonstrates a negative impact on eye and optic nerve growth, particularly in children treated ≤ 12 months of age, providing crucial insights for clinical management and future research.

Correlation of gene expression with magnetic resonance imaging features of retinoblastoma: a multi-center radiogenomics validation study

OBJECTIVES

To validate associations between MRI features and gene expression profiles in retinoblastoma, thereby evaluating the repeatability of radiogenomics in retinoblastoma.

METHODS

In this retrospective multicenter cohort study, retinoblastoma patients with gene expression data and MRI were included. MRI features (scored blinded for clinical data) and matched genome-wide gene expression data were used to perform radiogenomic analysis. Expression data from each center were first separately processed and analyzed. The end product normalized expression values from different sites were subsequently merged by their Z-score to permit cross-sites validation analysis. The MRI features were non-parametrically correlated with expression of photoreceptorness (radiogenomic analysis), a gene expression signature informing on disease progression. Outcomes were compared to outcomes in a previous described cohort.

RESULTS

Thirty-six retinoblastoma patients were included, 15 were female (42%), and mean age was 24 (SD 18) months. Similar to the prior evaluation, this validation study showed that low photoreceptorness gene expression was associated with advanced stage imaging features. Validated imaging features associated with low photoreceptorness were multifocality, a tumor encompassing the entire retina or entire globe, and a diffuse growth pattern (all p < 0.05). There were a number of radiogenomic associations that were also not validated.

CONCLUSIONS

A part of the radiogenomic associations could not be validated, underlining the importance of validation studies. Nevertheless, cross-center validation of imaging features associated with photoreceptorness gene expression highlighted the capability radiogenomics to non-invasively inform on molecular subtypes in retinoblastoma.

CLINICAL RELEVANCE STATEMENT

Radiogenomics may serve as a surrogate for molecular subtyping based on histopathology material in an era of eye-sparing retinoblastoma treatment strategies.

KEY POINTS

• Since retinoblastoma is increasingly treated using eye-sparing methods, MRI features informing on molecular subtypes that do not rely on histopathology material are important. • A part of the associations between retinoblastoma MRI features and gene expression profiles (radiogenomics) were validated. • Radiogenomics could be a non-invasive technique providing information on the molecular make-up of retinoblastoma.

Fully Automated Segmentation of Human Eyeball Using Three-Dimensional U-Net in T2 Magnetic Resonance Imaging

Purpose

To develop and validate a fully automated deep-learning-based tool for segmentation of the human eyeball using a three-dimensional (3D) U-Net, compare its performance to semiautomatic segmentation ground truth and a two-dimensional (2D) U-Net, and analyze age and sex differences in eyeball volume, as well as gaze-dependent volume consistency in normal subjects.

Methods

We retrospectively collected 474 magnetic resonance imaging (MRI) scans, including different gazing scans, from 119 patients. A 10-fold cross-validation was applied to separate the dataset into training, test, and validation sets for both the 3D U-Net and 2D U-Net. Performance accuracy was measured using four quantitative metrics compared to the ground truth, and Bland-Altman plot analysis was conducted. Age and sex differences in eyeball volume and variability in eyeball volume differences across gazing directions were analyzed.

Results

The 3D U-Net outperformed the 2D U-Net with mean accuracy scores >0.95, showing acceptable agreement in the Bland-Altman plot analysis despite a tendency for slight overestimation (mean difference = -0.172 cm³). Significant sex differences and age effects on eyeball volume were observed for both methods (P < 0.05). No significant volume differences were found between the segmentation methods or within each method for the different gazing directions. Significant differences in performance accuracy were identified among the five gazing directions, with the upward direction showing a notably lower performance.

Conclusions

Our study demonstrated the effectiveness of 3D U-Net human eyeball volume segmentation using T2-weighted MRI. The robustness and reliability of 3D U-Net across diverse populations and gaze directions support enhanced ophthalmic diagnosis and treatment strategies.

Translational Relevance

Our findings demonstrate the feasibility of using the proposed 3D U-Net model for the automatic segmentation of the human eyeball, with potential applications in various ophthalmic research fields that require the analysis of 3D geometric eye globe shapes or eye movement detection.

Artificial intelligence for ocular oncology

PURPOSE OF REVIEW

The aim of this article is to provide an update on the latest applications of deep learning (DL) and classical machine learning (ML) techniques to the detection and prognostication of intraocular and ocular surface malignancies.

RECENT FINDINGS

Most recent studies focused on using DL and classical ML techniques for prognostication purposes in patients with uveal melanoma (UM).

SUMMARY

DL has emerged as the leading ML technique for prognostication in ocular oncological conditions, particularly in UM. However, the application of DL may be limited by the relatively rarity of these conditions.

Semi-supervised segmentation of retinoblastoma tumors in fundus images

Retinoblastoma is a rare form of cancer that predominantly affects young children as the primary intraocular malignancy. Studies conducted in developed and some developing countries have revealed that early detection can successfully cure over 90% of children with retinoblastoma. An unusual white reflection in the pupil is the most common presenting symptom. Depending on the tumor size, shape, and location, medical experts may opt for different approaches and treatments, with the results varying significantly due to the high reliance on prior knowledge and experience. This study aims to present a model based on semi-supervised machine learning that will yield segmentation results comparable to those achieved by medical experts. First, the Gaussian mixture model is utilized to detect abnormalities in approximately 4200 fundus images. Due to the high computational cost of this process, the results of this approach are then used to train a cost-effective model for the same purpose. The proposed model demonstrated promising results in extracting highly detailed boundaries in fundus images. Using the Sørensen-Dice coefficient as the comparison metric for segmentation tasks, an average accuracy of 93% on evaluation data was achieved.

Explainable AI for Retinoblastoma Diagnosis: Interpreting Deep Learning Models with LIME and SHAP

Retinoblastoma is a rare and aggressive form of childhood eye cancer that requires prompt diagnosis and treatment to prevent vision loss and even death. Deep learning models have shown promising results in detecting retinoblastoma from fundus images, but their decision-making process is often considered a "black box" that lacks transparency and interpretability. In this project, we explore the use of LIME and SHAP, two popular explainable AI techniques, to generate local and global explanations for a deep learning model based on InceptionV3 architecture trained on retinoblastoma and non-retinoblastoma fundus images. We collected and labeled a dataset of 400 retinoblastoma and 400 non-retinoblastoma images, split it into training, validation, and test sets, and trained the model using transfer learning from the pre-trained InceptionV3 model. We then applied LIME and SHAP to generate explanations for the model's predictions on the validation and test sets. Our results demonstrate that LIME and SHAP can effectively identify the regions and features in the input images that contribute the most to the model's predictions, providing valuable insights into the decision-making process of the deep learning model. In addition, the use of InceptionV3 architecture with spatial attention mechanism achieved high accuracy of 97% on the test set, indicating the potential of combining deep learning and explainable AI for improving retinoblastoma diagnosis and treatment.

A Multi-Thresholding-Based Discriminative Neural Classifier for Detection of Retinoblastoma Using CNN Models

Cancer is one of the vital diseases which lead to the uncontrollable growth of the cell, and it affects the body tissue. A type of cancer that affects the children below five years and adults in a rare case is called retinoblastoma. It affects the retina in the eye and the surrounding region of eye like the eyelid, and sometimes, it leads to vision loss if it is not diagnosed at the early stage. MRI and CT are widely used scanning procedures to identify the cancerous region in the eye. Current screening methods for cancer region identification needs the clinicians' support to spot the affected regions. Modern healthcare systems develop an easy way to diagnose the disease. Discriminative architectures in deep learning can be viewed as supervised deep learning algorithms which use classification/regression techniques to predict the output. A convolutional neural network (CNN) is a part of the discriminative architecture which helps to process both image and text data. This work suggests the CNN-based classifier which classifies the tumor and nontumor regions in retinoblastoma. The tumor-like region (TLR) in retinoblastoma is identified using the automated thresholding method. After that, ResNet and AlexNet algorithms are used to classify the cancerous region along with classifiers. In addition, the comparison of discriminative algorithm along with its variants is experimented to produce the better image analysis method without the intervention of clinicians. The experimental study reveals that ResNet50 and AlexNet yield better results compared to other learning modules.

Artificial intelligence applications in pediatric oncology diagnosis

Artificial intelligence (AI) algorithms have been applied in abundant medical tasks with high accuracy and efficiency. Physicians can improve their diagnostic efficiency with the assistance of AI techniques for improving the subsequent personalized treatment and surveillance. AI algorithms fundamentally capture data, identify underlying patterns, achieve preset endpoints, and provide decisions and predictions about real-world events with working principles of machine learning and deep learning. AI algorithms with sufficient graphic processing unit power have been demonstrated to provide timely diagnostic references based on preliminary training of large amounts of clinical and imaging data. The sample size issue is an inevitable challenge for pediatric oncology considering its low morbidity and individual heterogeneity. However, this problem may be solved in the near future considering the exponential advancements of AI algorithms technically to decrease the dependence of AI operation on the amount of data sets and the efficiency of computing power. For instance, it could be a feasible solution by shifting convolutional neural networks (CNNs) from adults and sharing CNN algorithms across multiple institutions besides original data. The present review provides important insights into emerging AI applications for the diagnosis of pediatric oncology by systematically overviewing of up-to-date literature.

[Imaging of retinoblastoma : Current state-of-the-art and future prospects]

BACKGROUND

Retinoblastoma is the most common malignant eye tumor in children and is associated with tumor predisposition syndrome (RB1 mutation) in up to 40% of cases. Imaging is an important part of the diagnostic workup of children with retinoblastoma both during the initial diagnosis and follow-up.

OBJECTIVES

The goal of this review is to present the current state-of-the-art regarding imaging of children with retinoblastoma, including technical background and diagnostic clues with a brief discussion of future prospects. In addition, we summarize the general clinical diagnostic workup and therapeutic options.

MATERIALS AND METHODS

Review of the literature and our own experience in the imaging of retinoblastoma.

CONCLUSION

High-resolution magnetic resonance imaging (MRI) is the imaging modality of choice in children with retinoblastoma for diagnosis (estimation of diagnosis/differential diagnosis, evaluation of local and intracranial tumor extension) and during follow-up. Despite the characteristic calcifications, computed tomography (CT) examinations are no longer indicated in these patients. Due to the high association with tumor predisposition syndrome, genetic counselling is recommended.

Deep Learning for Automated Elective Lymph Node Level Segmentation for Head and Neck Cancer Radiotherapy

Depending on the clinical situation, different combinations of lymph node (LN) levels define the elective LN target volume in head-and-neck cancer (HNC) radiotherapy. The accurate auto-contouring of individual LN levels could reduce the burden and variability of manual segmentation and be used regardless of the primary tumor location. We evaluated three deep learning approaches for the segmenting individual LN levels I−V, which were manually contoured on CT scans from 70 HNC patients. The networks were trained and evaluated using five-fold cross-validation and ensemble learning for 60 patients with (1) 3D patch-based UNets, (2) multi-view (MV) voxel classification networks and (3) sequential UNet+MV. The performances were evaluated using Dice similarity coefficients (DSC) for automated and manual segmentations for individual levels, and the planning target volumes were extrapolated from the combined levels I−V and II−IV, both for the cross-validation and for an independent test set of 10 patients. The median DSC were 0.80, 0.66 and 0.82 for UNet, MV and UNet+MV, respectively. Overall, UNet+MV significantly (p < 0.0001) outperformed other arrangements and yielded DSC = 0.87, 0.85, 0.86, 0.82, 0.77, 0.77 for the combined and individual level I−V structures, respectively. Both PTVs were also significantly (p < 0.0001) more accurate with UNet+MV, with DSC = 0.91 and 0.90, respectively. The accurate segmentation of individual LN levels I−V can be achieved using an ensemble of UNets. UNet+MV can further refine this result.

Assessment of data consistency through cascades of independently recurrent inference machines for fast and robust accelerated MRI reconstruction

Objective. Machine Learning methods can learn how to reconstruct magnetic resonance images (MRI) and thereby accelerate acquisition, which is of paramount importance to the clinical workflow. Physics-informed networks incorporate the forward model of accelerated MRI reconstruction in the learning process. With increasing network complexity, robustness is not ensured when reconstructing data unseen during training. We aim to embed data consistency (DC) in deep networks while balancing the degree of network complexity. While doing so, we will assess whether either explicit or implicit enforcement of DC in varying network architectures is preferred to optimize performance. Approach. We propose a scheme called Cascades of Independently Recurrent Inference Machines (CIRIM) to assess DC through unrolled optimization. Herein we assess DC both implicitly by gradient descent and explicitly by a designed term. Extensive comparison of the CIRIM to compressed sensing as well as other Machine Learning methods is performed: the End-to-End Variational Network (E2EVN), CascadeNet, KIKINet, LPDNet, RIM, IRIM, and UNet. Models were trained and evaluated on T1-weighted and FLAIR contrast brain data, and T2-weighted knee data. Both 1D and 2D undersampling patterns were evaluated. Robustness was tested by reconstructing 7.5× prospectively undersampled 3D FLAIR MRI data of multiple sclerosis (MS) patients with white matter lesions. Main results. The CIRIM performed best when implicitly enforcing DC, while the E2EVN required an explicit DC formulation. Through its cascades, the CIRIM was able to score higher on structural similarity and PSNR compared to other methods, in particular under heterogeneous imaging conditions. In reconstructing MS patient data, prospectively acquired with a sampling pattern unseen during model training, the CIRIM maintained lesion contrast while efficiently denoising the images. Significance. The CIRIM showed highly promising generalization capabilities maintaining a very fair trade-off between reconstructed image quality and fast reconstruction times, which is crucial in the clinical workflow.

MRI-based 3D retinal shape determination

Objective

To establish a good method to determine the retinal shape from MRI using three-dimensional (3D) ellipsoids as well as evaluate its reproducibility.

Methods and analysis

The left eyes of 31 volunteers were imaged using high-resolution ocular MRI. The 3D MR-images were segmented and ellipsoids were fitted to the resulting contours. The dependency of the resulting ellipsoid parameters on the evaluated fraction of the retinal contour was assessed by fitting ellipsoids to 41 different fractions. Furthermore, the reproducibility of the complete procedure was evaluated in four subjects. Finally, a comparison with conventional two-dimensional (2D) methods was made.

Results

The mean distance between the fitted ellipsoids and the segmented retinal contour was 0.03±0.01 mm (mean±SD) for the central retina and 0.13±0.03 mm for the peripheral retina. For the central retina, the resulting ellipsoid radii were 12.9±0.9, 13.7±1.5 and 12.2±1.2 mm along the horizontal, vertical and central axes. For the peripheral retina, these radii decreased to 11.9±0.6, 11.6±0.4 and 10.4±0.7 mm, which was accompanied by a mean 1.8 mm posterior shift of the ellipsoid centre. The reproducibility of the ellipsoid fitting was 0.3±1.2 mm for the central retina and 0.0±0.1 mm for the peripheral retina. When 2D methods were used to fit the peripheral retina, the fitted radii differed a mean 0.1±0.1 mm from the 3D method.

Conclusion

An accurate and reproducible determination of the 3D retinal shape based on MRI is provided together with 2D alternatives, enabling wider use of this method in the field of ophthalmology.