Clinical explainable differential diagnosis of polypoidal choroidal vasculopathy and age-related macular degeneration using deep learning

DRFNet: a deep radiomic fusion network for nAMD/PCV differentiation in OCT images

Objective.Neovascular age-related macular degeneration (nAMD) and polypoidal choroidal vasculopathy (PCV) present many similar clinical features. However, there are significant differences in the progression of nAMD and PCV. and it is crucial to make accurate diagnosis for treatment. In this paper, we propose a structure-radiomic fusion network (DRFNet) to differentiate PCV and nAMD in optical coherence tomography (OCT) images.Approach.The subnetwork (RIMNet) is designed to automatically segment the lesion of nAMD and PCV. Another subnetwork (StrEncoder) is designed to extract deep structural features of the segmented lesion. The subnetwork (RadEncoder) is designed to extract radiomic features from the segmented lesions based on radiomics. 305 eyes (155 with nAMD and 150 with PCV) are included and manually annotated CNV region in this study. The proposed method was trained and evaluated by 4-fold cross validation using the collected data and was compared with the advanced differentiation methods.Main results.The proposed method achieved high classification performace of nAMD/PCV differentiation in OCT images, which was an improvement of 4.68 compared with other best method.Significance. The presented structure-radiomic fusion network (DRFNet) has great performance of diagnosing nAMD and PCV and high clinical value by using OCT instead of indocyanine green angiography.

Differential diagnosis of frontotemporal dementia subtypes with explainable deep learning on structural MRI

Background

Frontotemporal dementia (FTD) represents a collection of neurobehavioral and neurocognitive syndromes that are associated with a significant degree of clinical, pathological, and genetic heterogeneity. Such heterogeneity hinders the identification of effective biomarkers, preventing effective targeted recruitment of participants in clinical trials for developing potential interventions and treatments. In the present study, we aim to automatically differentiate patients with three clinical phenotypes of FTD, behavioral-variant FTD (bvFTD), semantic variant PPA (svPPA), and nonfluent variant PPA (nfvPPA), based on their structural MRI by training a deep neural network (DNN).

Methods

Data from 277 FTD patients (173 bvFTD, 63 nfvPPA, and 41 svPPA) recruited from two multi-site neuroimaging datasets: the Frontotemporal Lobar Degeneration Neuroimaging Initiative and the ARTFL-LEFFTDS Longitudinal Frontotemporal Lobar Degeneration databases. Raw T1-weighted MRI data were preprocessed and parcellated into patch-based ROIs, with cortical thickness and volume features extracted and harmonized to control the confounding effects of sex, age, total intracranial volume, cohort, and scanner difference. A multi-type parallel feature embedding framework was trained to classify three FTD subtypes with a weighted cross-entropy loss function used to account for unbalanced sample sizes. Feature visualization was achieved through post-hoc analysis using an integrated gradient approach.

Results

The proposed differential diagnosis framework achieved a mean balanced accuracy of 0.80 for bvFTD, 0.82 for nfvPPA, 0.89 for svPPA, and an overall balanced accuracy of 0.84. Feature importance maps showed more localized differential patterns among different FTD subtypes compared to groupwise statistical mapping.

Conclusion

In this study, we demonstrated the efficiency and effectiveness of using explainable deep-learning-based parallel feature embedding and visualization framework on MRI-derived multi-type structural patterns to differentiate three clinically defined subphenotypes of FTD: bvFTD, nfvPPA, and svPPA, which could help with the identification of at-risk populations for early and precise diagnosis for intervention planning.

Early inner plexiform layer thinning and retinal nerve fiber layer thickening in excitotoxic retinal injury using deep learning-assisted optical coherence tomography

Excitotoxicity from the impairment of glutamate uptake constitutes an important mechanism in neurodegenerative diseases such as Alzheimer's, multiple sclerosis, and Parkinson's disease. Within the eye, excitotoxicity is thought to play a critical role in retinal ganglion cell death in glaucoma, diabetic retinopathy, retinal ischemia, and optic nerve injury, yet how excitotoxic injury impacts different retinal layers is not well understood. Here, we investigated the longitudinal effects of N-methyl-D-aspartate (NMDA)-induced excitotoxic retinal injury in a rat model using deep learning-assisted retinal layer thickness estimation. Before and after unilateral intravitreal NMDA injection in nine adult Long Evans rats, spectral-domain optical coherence tomography (OCT) was used to acquire volumetric retinal images in both eyes over 4 weeks. Ten retinal layers were automatically segmented from the OCT data using our deep learning-based algorithm. Retinal degeneration was evaluated using layer-specific retinal thickness changes at each time point (before, and at 3, 7, and 28 days after NMDA injection). Within the inner retina, our OCT results showed that retinal thinning occurred first in the inner plexiform layer at 3 days after NMDA injection, followed by the inner nuclear layer at 7 days post-injury. In contrast, the retinal nerve fiber layer exhibited an initial thickening 3 days after NMDA injection, followed by normalization and thinning up to 4 weeks post-injury. Our results demonstrated the pathological cascades of NMDA-induced neurotoxicity across different layers of the retina. The early inner plexiform layer thinning suggests early dendritic shrinkage, whereas the initial retinal nerve fiber layer thickening before subsequent normalization and thinning indicates early inflammation before axonal loss and cell death. These findings implicate the inner plexiform layer as an early imaging biomarker of excitotoxic retinal degeneration, whereas caution is warranted when interpreting the ganglion cell complex combining retinal nerve fiber layer, ganglion cell layer, and inner plexiform layer thicknesses in conventional OCT measures. Deep learning-assisted retinal layer segmentation and longitudinal OCT monitoring can help evaluate the different phases of retinal layer damage upon excitotoxicity.

Deep Learning Applications to Classification and Detection of Age-Related Macular Degeneration on Optical Coherence Tomography Imaging: A Review

Age-related macular degeneration (AMD) is one of the leading causes of blindness in the elderly, more commonly in developed countries. Optical coherence tomography (OCT) is a non-invasive imaging device widely used for the diagnosis and management of AMD. Deep learning (DL) uses multilayered artificial neural networks (NN) for feature extraction, and is the cutting-edge technique for medical image analysis for diagnostic and prognostication purposes. Application of DL models to OCT image analysis has garnered significant interest in recent years. In this review, we aimed to summarize studies focusing on DL models used in classification and detection of AMD. Additionally, we provide a brief introduction to other DL applications in AMD, such as segmentation, prediction/prognostication, and models trained on multimodal imaging.

EffViT-COVID: A dual-path network for COVID-19 percentage estimation

The first case of novel Coronavirus (COVID-19) was reported in December 2019 in Wuhan City, China and led to an international outbreak. This virus causes serious respiratory illness and affects several other organs of the body differently for different patient. Worldwide, several waves of this infection have been reported, and researchers/doctors are working hard to develop novel solutions for the COVID diagnosis. Imaging and vision-based techniques are widely explored for the prediction of COVID-19; however, COVID infection percentage estimation is under explored. In this work, we propose a novel framework for the estimation of COVID-19 infection percentage based on deep learning techniques. The proposed network utilizes the features from vision transformers and CNN (Convolutional Neural Networks), specifically EfficientNet-B7. The features of both are fused together for preparing an information-rich feature vector that contributes to a more precise estimation of infection percentage. We evaluate our model on the Per-COVID-19 dataset (Bougourzi et al., 2021b) which comprises labeled CT data of COVID-19 patients. For the evaluation of the model on this dataset, we employ the most widely-used slice-level metrics, i.e., Pearson correlation coefficient (PC), Mean absolute error (MAE), and Root mean square error (RMSE). The network outperforms the other state-of-the-art methods and achieves 0 . 9886 ± 0 . 009 , 1 . 23 ± 0 . 378 , and 3 . 12 ± 1 . 56 , PC, MAE, and RMSE, respectively, using a 5-fold cross-validation technique. In addition, the overall average difference in the actual and predicted infection percentage is observed to be < 2 % . In conclusion, the detailed experimental results reveal the robustness and efficiency of the proposed network.

Polypoidal Choroidal Vasculopathy: Updates on Risk Factors, Diagnosis, and Treatments

There have been recent advances in basic research and clinical studies in polypoidal choroidal vasculopathy (PCV). A recent, large-scale, population-based study found systemic factors, such as male gender and smoking, were associated with PCV, and a recent systematic review reported plasma C-reactive protein, a systemic biomarker, was associated with PCV. Growing evidence points to an association between pachydrusen, recently proposed extracellular deposits associated with the thick choroid, and the risk of development of PCV. Many recent studies on diagnosis of PCV have focused on applying criteria from noninvasive multimodal retinal imaging without requirement of indocyanine green angiography. There have been attempts to develop deep learning models, a recent subset of artificial intelligence, for detecting PCV from different types of retinal imaging modality. Some of these deep learning models were found to have high performance when they were trained and tested on color retinal images with corresponding images from optical coherence tomography. The treatment of PCV is either a combination therapy using verteporfin photodynamic therapy and anti-vascular endothelial growth factor (VEGF), or anti-VEGF monotherapy, often used with a treat-and-extend regimen. New anti-VEGF agents may provide more durable treatment with similar efficacy, compared with existing anti-VEGF agents. It is not known if they can induce greater closure of polypoidal lesions, in which case, combination therapy may still be a mainstay. Recent evidence supports long-term follow-up of patients with PCV after treatment for early detection of recurrence, particularly in patients with incomplete closure of polypoidal lesions.

Deep ensemble learning for automated non-advanced AMD classification using optimized retinal layer segmentation and SD-OCT scans

BACKGROUND

Accurate retinal layer segmentation in optical coherence tomography (OCT) images is crucial for quantitatively analyzing age-related macular degeneration (AMD) and monitoring its progression. However, previous retinal segmentation models depend on experienced experts and manually annotating retinal layers is time-consuming. On the other hand, accuracy of AMD diagnosis is directly related to the segmentation model's performance. To address these issues, we aimed to improve AMD detection using optimized retinal layer segmentation and deep ensemble learning.

METHOD

We integrated a graph-cut algorithm with a cubic spline to automatically annotate 11 retinal boundaries. The refined images were fed into a deep ensemble mechanism that combined a Bagged Tree and end-to-end deep learning classifiers. We tested the developed deep ensemble model on internal and external datasets.

RESULTS

The total error rates for our segmentation model using the boundary refinement approach was significantly lower than OCT Explorer segmentations (1.7% vs. 7.8%, p-value = 0.03). We utilized the refinement approach to quantify 169 imaging features using Zeiss SD-OCT volume scans. The presence of drusen and thickness of total retina, neurosensory retina, and ellipsoid zone to inner-outer segment (EZ-ISOS) thickness had higher contributions to AMD classification compared to other features. The developed ensemble learning model obtained a higher diagnostic accuracy in a shorter time compared with two human graders. The area under the curve (AUC) for normal vs. early AMD was 99.4%.

CONCLUSION

Testing results showed that the developed framework is repeatable and effective as a potentially valuable tool in retinal imaging research.

Segmentation-guided domain adaptation and data harmonization of multi-device retinal optical coherence tomography using cycle-consistent generative adversarial networks

BACKGROUND

Medical images such as Optical Coherence Tomography (OCT) images acquired from different devices may show significantly different intensity profiles. An automatic segmentation model trained on images from one device may perform poorly when applied to images acquired using another device, resulting in a lack of generalizability. This study addresses this issue using domain adaptation methods improved by Cycle-Consistent Generative Adversarial Networks (CycleGAN), especially when the ground-truth labels are only available in the source domain.

METHODS

A two-stage pipeline is proposed to generate segmentation in the target domain. The first stage involves the training of a state-of-the-art segmentation model in the source domain. The second stage aims to adapt the images from the target domain to the source domain. The adapted target domain images are segmented using the model in the first stage. Ablation tests were performed with integration of different loss functions, and the statistical significance of these models is reported. Both the segmentation performance and the adapted image quality metrics were evaluated.

RESULTS

Regarding the segmentation Dice score, the proposed model ssppg achieves a significant improvement of 46.24% compared to without adaptation and reaches 87.4% of the upper limit of the segmentation performance. Furthermore, image quality metrics, including FID and KID scores, indicate that adapted images with better segmentation also have better image qualities.

CONCLUSION

The proposed method demonstrates the effectiveness of segmentation-driven domain adaptation in retinal imaging processing. It reduces the labor cost of manual labeling, incorporates prior anatomic information to regulate and guide domain adaptation, and provides insights into improving segmentation qualities in image domains without labels.

Polypoidal Choroidal Vasculopathy: An Update on Diagnosis and Treatment

Polypoidal choroidal vasculopathy (PCV) is a vascular disease of the choroid that leads to hemorrhagic and exudative macular degeneration. It may cause significant vision loss and thus affect the quality-of-life and psychological well-being. Non-invasive, non-ICGA-based OCT criteria have shown reliable results to plan adjunct photodynamic therapy (PDT) treatment, with the complete and consistent coverage of polypoidal lesions (PL) and branching neovascular network (BNN). The safety and efficacy of anti-vascular endothelial growth factor (anti-VEGF) monotherapy and its combination with verteporfin PDT have been established. However, treatment is still challenging due to frequent follow-ups, non-availability of PDT, and need for multiple anti-VEGF injection visits that increase the treatment burden and lead to patients being lost to follow-up. Effective treatments that prolong intervals between injections while maintaining vision and anatomical gains remain a critical unmet need. Longer acting molecules, like brolucizumab, have shown non-inferiority in BCVA gains and superior anatomical outcomes compared to other anti-VEGF agents. Newer therapies in the pipeline to enhance the efficacy and longevity of treatment include Faricimab and a port delivery system (PDS). This review summarizes the most recent diagnostic and treatment approaches in PCV to offer better treatment avenues.

Reverse translation of artificial intelligence in glaucoma: Connecting basic science with clinical applications

Artificial intelligence (AI) has been approved for biomedical research in diverse areas from bedside clinical studies to benchtop basic scientific research. For ophthalmic research, in particular glaucoma, AI applications are rapidly growing for potential clinical translation given the vast data available and the introduction of federated learning. Conversely, AI for basic science remains limited despite its useful power in providing mechanistic insight. In this perspective, we discuss recent progress, opportunities, and challenges in the application of AI in glaucoma for scientific discoveries. Specifically, we focus on the research paradigm of reverse translation, in which clinical data are first used for patient-centered hypothesis generation followed by transitioning into basic science studies for hypothesis validation. We elaborate on several distinctive areas of research opportunities for reverse translation of AI in glaucoma including disease risk and progression prediction, pathology characterization, and sub-phenotype identification. We conclude with current challenges and future opportunities for AI research in basic science for glaucoma such as inter-species diversity, AI model generalizability and explainability, as well as AI applications using advanced ocular imaging and genomic data.

Multi-label classification of fundus images with graph convolutional network and LightGBM

Early detection and treatment of retinal disorders are critical for avoiding irreversible visual impairment. Given that patients in the clinical setting may have various types of retinal illness, the development of multi-label fundus disease detection models capable of screening for multiple diseases is more in line with clinical needs. This article presented a composite model based on hybrid graph convolution for patient-level multi-label fundus illness identification. The composite model comprised a backbone module, a hybrid graph convolution module, and a classifier module. This article established the relationship between labels via graph convolution and then employed a self-attention mechanism to design a hybrid graph convolution structure. The backbone module extracted features using EfficientNet-B4, whereas the classifier module output multi-label using LightGBM. Additionally, this work investigated the input pattern of binocular images and the influence of label correlation on the model's identification performance. The proposed model MCGL-Net outperformed all other state-of-the-art methods on the publicly available ODIR dataset, with F1 reaching 91.60% on the test set. Ablation experiments were also performed in this paper. Experiments showed that the idea of hybrid graph convolutional structure and composite model designed in this paper promotes the model performance under any backbone CNN. The adoption of hybrid graph convolution can increase the F1 by 2.39% in trials using EfficientNet-B4 as the backbone. The composite model had a higher F1 index by 5.42% than the single EfficientNet-B4 model.