Imaging Features of Vessels and Leakage Patterns Predict Extended Interval Aflibercept Dosing Using Ultra-Widefield Angiography in Retinal Vascular Disease: Findings From the PERMEATE Study

Novel artificial intelligence algorithms for diabetic retinopathy and diabetic macular edema

BACKGROUND

Diabetic retinopathy (DR) and diabetic macular edema (DME) are major causes of visual impairment that challenge global vision health. New strategies are needed to tackle these growing global health problems, and the integration of artificial intelligence (AI) into ophthalmology has the potential to revolutionize DR and DME management to meet these challenges.

MAIN TEXT

This review discusses the latest AI-driven methodologies in the context of DR and DME in terms of disease identification, patient-specific disease profiling, and short-term and long-term management. This includes current screening and diagnostic systems and their real-world implementation, lesion detection and analysis, disease progression prediction, and treatment response models. It also highlights the technical advancements that have been made in these areas. Despite these advancements, there are obstacles to the widespread adoption of these technologies in clinical settings, including regulatory and privacy concerns, the need for extensive validation, and integration with existing healthcare systems. We also explore the disparity between the potential of AI models and their actual effectiveness in real-world applications.

CONCLUSION

AI has the potential to revolutionize the management of DR and DME, offering more efficient and precise tools for healthcare professionals. However, overcoming challenges in deployment, regulatory compliance, and patient privacy is essential for these technologies to realize their full potential. Future research should aim to bridge the gap between technological innovation and clinical application, ensuring AI tools integrate seamlessly into healthcare workflows to enhance patient outcomes.

Multimodal deep transfer learning to predict retinal vein occlusion macular edema recurrence after anti-VEGF therapy

Purpose

To develop a multimodal deep transfer learning (DTL) fusion model using optical coherence tomography angiography (OCTA) images to predict the recurrence of retinal vein occlusion (RVO) and macular edema (ME) after three consecutive anti-VEGF therapies.

Methods

This retrospective cross-sectional study consisted of 2800 B-scan OCTA macular images collected from 140 patients with RVO-ME. The central macular thickness (CMT) > 250 μm was used as a criterion for recurrence in the three-month follow-up after three injections of anti-VEGF therapy. The qualified OCTA image preprocessing and the lesion area segmentation were performed by senior ophthalmologists. We developed and validated the clinical, DTL, and multimodal fusion models based on clinical and extracted OCTA imaging features. The performance of the models and experts predictions were evaluated using several performance metrics, including the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity.

Results

The DTL models exhibited higher prediction efficacy than the clinical models and experts' predictions. Among the DTL models, the Vgg19 performed better than that of the other models, with an AUC of 0.968 (95 % CI, 0.943-0.994), accuracy of 0.913, sensitivity of 0.922, and specificity of 0.902 in the validation cohort. Moreover, the fusion Vgg19 model showed the highest prediction efficacy among all the models, with an AUC of 0.972 (95 % CI, 0.946-0.997), accuracy of 0.935, sensitivity of 0.935, and specificity of 0.934 in the validation cohort.

Conclusions

Multimodal fusion DTL models showed robust performance in predicting RVO-ME recurrence and may be applied to assist clinicians in determining patients' follow-up time after anti-VEGF therapy.

Controllable editing via diffusion inversion on ultra-widefield fluorescein angiography for the comprehensive analysis of diabetic retinopathy

By incorporating multiple indicators that facilitate clinical decision making and effective management of diabetic retinopathy (DR), a comprehensive understanding of the progression of the disease can be achieved. However, the diversity of DR complications poses challenges to the automatic analysis of various information within images. This study aims to establish a deep learning system designed to examine various metrics linked to DR in ultra-widefield fluorescein angiography (UWFA) images. We have developed a unified model based on image generation that transforms input images into corresponding disease-free versions. By incorporating an image-level supervised training process, the model significantly reduces the need for extensive manual involvement in clinical applications. Furthermore, compared to other comparative methods, the quality of our generated images is significantly superior.

How can machine learning and multiscale modeling benefit ocular drug development?

The eyes possess sophisticated physiological structures, diverse disease targets, limited drug delivery space, distinctive barriers, and complicated biomechanical processes, requiring a more in-depth understanding of the interactions between drug delivery systems and biological systems for ocular formulation development. However, the tiny size of the eyes makes sampling difficult and invasive studies costly and ethically constrained. Developing ocular formulations following conventional trial-and-error formulation and manufacturing process screening procedures is inefficient. Along with the popularity of computational pharmaceutics, non-invasive in silico modeling & simulation offer new opportunities for the paradigm shift of ocular formulation development. The current work first systematically reviews the theoretical underpinnings, advanced applications, and unique advantages of data-driven machine learning and multiscale simulation approaches represented by molecular simulation, mathematical modeling, and pharmacokinetic (PK)/pharmacodynamic (PD) modeling for ocular drug development. Following this, a new computer-driven framework for rational pharmaceutical formulation design is proposed, inspired by the potential of in silico explorations in understanding drug delivery details and facilitating drug formulation design. Lastly, to promote the paradigm shift, integrated in silico methodologies were highlighted, and discussions on data challenges, model practicality, personalized modeling, regulatory science, interdisciplinary collaboration, and talent training were conducted in detail with a view to achieving more efficient objective-oriented pharmaceutical formulation design.

Longitudinal Quantitative Ultrawide-field Fluorescein Angiography Dynamics in the RUBY Diabetic Macular Edema Study

OBJECTIVE

The purpose of this study was to evaluate the longitudinal change in quantitative ultrawide-field angiographic (UWFA) parameters and correlate them with functional outcomes and spectral domain-OCT metrics.

DESIGN

This study is a post hoc analysis of the phase II RUBY study: a prospective, randomized trial of patients with diabetic macular edema (DME) treated with either intravitreal aflibercept injection (IAI) or combined IAI/nesvacumab (antiangiopoietin 2 mAb).

SUBJECTS

Subjects with DME that underwent UWFA across all treatment groups (n = 44).

METHODS

A machine learning-enabled feature extraction system generated panretinal quantitative UWFA metrics, including leakage, ischemia, and microaneurysm (MA) burden. Zonal assessments were performed corresponding to the macula, midperiphery, and far periphery.

MAIN OUTCOME MEASURES

Changes in ischemic area and index (proportion of nonperfusion in analyzable retina), leakage area and index (proportion of leakage in analyzable retina), and MA count at baseline, week 12, week 24, and week 36 were analyzed. Spectral-domain-OCT quantitative metrics, such as central subfield thickness, ellipsoid zone (EZ) integrity parameters, intraretinal fluid (IRF) volume, and subretinal fluid (SRF) volume were extracted via a machine learning-enhanced OCT feature extraction platform and analyzed. Additionally, the effect of these changes on best-corrected visual acuity (BCVA) was evaluated.

RESULTS

Mean panretinal leakage index, zonal leakage area, and panretinal MA count improved significantly between baseline and week 36. Panretinal ischemic index decreased between baseline and week 36, with some aspects showing significant improvement. Mean BCVA significantly improved from baseline to week 36. There was a significant inverse correlation between change in BCVA and change in macular leakage area. A direct correlation was observed between both baseline macular leakage area and panretinal leakage index with IRF volume, SRF volume, and EZ disruption on OCT.

CONCLUSIONS

Assessment of UWFA parameters demonstrates a significant improvement in panretinal leakage index, leakage area, and MA burden in eyes treated with IAI with or without nesvacumab. A numeric reduction in panretinal ischemic index and area was noted. The analysis also shows the critical association of leakage with visual and OCT features. This highlights the potential role of UWFA in disease burden assessment, with leakage parameters serving as a primary end point.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found after the references.

Deep learning based diagnostic quality assessment of choroidal OCT features with expert-evaluated explainability

Various vision-threatening eye diseases including age-related macular degeneration (AMD) and central serous chorioretinopathy (CSCR) are caused due to the dysfunctions manifested in the highly vascular choroid layer of the posterior segment of the eye. In the current clinical practice, screening choroidal structural changes is widely based on optical coherence tomography (OCT) images. Accordingly, to assist clinicians, several automated choroidal biomarker detection methods using OCT images are developed. However, the performance of these algorithms is largely constrained by the quality of the OCT scan. Consequently, determining the quality of choroidal features in OCT scans is significant in building standardized quantification tools and hence constitutes our main objective. This study includes a dataset of 1593 good and 2581 bad quality Spectralis OCT images graded by an expert. Noting the efficacy of deep-learning (DL) in medical image analysis, we propose to train three state-of-the-art DL models: ResNet18, EfficientNet-B0 and EfficientNet-B3 to detect the quality of OCT images. The choice of these models was inspired by their ability to preserve the salient features across all the layers without information loss. To evaluate the attention of DL models on the choroid, we introduced color transparency maps (CTMs) based on GradCAM explanations. Further, we proposed two subjective grading scores: overall choroid coverage (OCC) and choroid coverage in the visible region(CCVR) based on CTMs to objectively correlate visual explanations vis-à-vis DL model attentions. We observed that the average accuracy and F-scores for the three DL models are greater than 96%. Further, the OCC and CCVR scores achieved for the three DL models under consideration substantiate that they mostly focus on the choroid layer in making the decision. In particular, of the three DL models, EfficientNet-B3 is in close agreement with the clinician's inference. The proposed DL-based framework demonstrated high detection accuracy as well as attention on the choroid layer, where EfficientNet-B3 reported superior performance. Our work assumes significance in bench-marking the automated choroid biomarker detection tools and facilitating high-throughput screening. Further, the methods proposed in this work can be adopted for evaluating the attention of DL-based approaches developed for other region-specific quality assessment tasks.

Radiomics-Based Assessment of OCT Angiography Images for Diabetic Retinopathy Diagnosis

Purpose

To evaluate the diagnostic accuracy of machine learning (ML) techniques applied to radiomic features extracted from OCT and OCT angiography (OCTA) images for diabetes mellitus (DM), diabetic retinopathy (DR), and referable DR (R-DR) diagnosis.

Design

Cross-sectional analysis of a retinal image dataset from a previous prospective OCTA study (ClinicalTrials.govNCT03422965).

Participants

Patients with type 1 DM and controls included in the progenitor study.

Methods

Radiomic features were extracted from fundus retinographies, OCT, and OCTA images in each study eye. Logistic regression, linear discriminant analysis, support vector classifier (SVC)-linear, SVC-radial basis function, and random forest models were created to evaluate their diagnostic accuracy for DM, DR, and R-DR diagnosis in all image types.

Main Outcome Measures

Area under the receiver operating characteristic curve (AUC) mean and standard deviation for each ML model and each individual and combined image types.

Results

A dataset of 726 eyes (439 individuals) were included. For DM diagnosis, the greatest AUC was observed for OCT (0.82, 0.03). For DR detection, the greatest AUC was observed for OCTA (0.77, 0.03), especially in the 3 × 3 mm superficial capillary plexus OCTA scan (0.76, 0.04). For R-DR diagnosis, the greatest AUC was observed for OCTA (0.87, 0.12) and the deep capillary plexus OCTA scan (0.86, 0.08). The addition of clinical variables (age, sex, etc.) improved most models AUC for DM, DR and R-DR diagnosis. The performance of the models was similar in unilateral and bilateral eyes image datasets.

Conclusions

Radiomics extracted from OCT and OCTA images allow identification of patients with DM, DR, and R-DR using standard ML classifiers. OCT was the best test for DM diagnosis, OCTA for DR and R-DR diagnosis and the addition of clinical variables improved most models. This pioneer study demonstrates that radiomics-based ML techniques applied to OCT and OCTA images may be an option for DR screening in patients with type 1 DM.

Financial Disclosures

Proprietary or commercial disclosure may be found after the references.

Retinal Microvasculature: A Potential Window Into Heart Failure Prevention

Endothelial dysfunction and microvascular disease have been shown to play an important role in the development and progression of heart failure (HF). Retinal imaging provides a unique opportunity to noninvasively assess vascular structure and function, vessel features, and microcirculation within the retina. Accumulating evidence suggests that retinal vessel caliber, microvascular features, and vascular characteristics extracted from various imaging modalities are associated with alterations in left ventricular structure and function in stage B HF, as well as incident development of symptomatic HF in the general population. Moreover, dynamic retinal vessel analysis has been shown to differentiate HF patients based on their phenotypes. Given the increasing availability of rapid image acquisition devices (eg, nonmydriatic widefield systems and smartphone-based retinal cameras) and the integration of artificial intelligence-based interrogation/assessment techniques, retinal imaging is a promising noninvasive tool, in conjunction with cardiac imaging and biomarkers, to prevent HF and risk stratify those at risk of developing HF. This review focuses on the current evidence on retinal microvasculature changes, and potential clinical relevance and promising utility of retinal imaging in HF.

Deep learning-enabled ultra-widefield retinal vessel segmentation with an automated quality-optimized angiographic phase selection tool

OBJECTIVES

To demonstrate the feasibility of a deep learning-based vascular segmentation tool for UWFA and evaluate its ability to automatically identify quality-optimized phase-specific images.

METHODS

Cumulative retinal vessel areas (RVA) were extracted from all available UWFA frames. Cubic splines were fitted for serial vascular assessment throughout the angiographic phases of eyes with diabetic retinopathy (DR), sickle cell retinopathy (SCR), or normal retinal vasculature. The image with maximum RVA was selected as the optimum early phase. A late phase frame was selected at a minimum of 4 min that most closely mirrored the RVA from the selected early image. Trained image analysts evaluated the selected pairs.

RESULTS

A total of 13,980 UWFA sequences from 462 sessions were used to evaluate the performance and 1578 UWFA sequences from 66 sessions were used to create cubic splines. Maximum RVA was detected at a mean of 41 ± 15, 47 ± 27, 38 ± 8 s for DR, SCR, and normals respectively. In 85.2% of the sessions, appropriate images for both phases were successfully identified. The individual success rate was 90.7% for early and 94.6% for late frames.

CONCLUSIONS

Retinal vascular characteristics are highly phased and field-of-view sensitive. Vascular parameters extracted by deep learning algorithms can be used for quality assessment of angiographic images and quality optimized phase selection. Clinical applications of a deep learning-based vascular segmentation and phase selection system might significantly improve the speed, consistency, and objectivity of UWFA evaluation.

Computational Imaging Biomarker Correlation with Intraocular Cytokine Expression in Diabetic Macular Edema

Purpose

Design

Participants

Methods

Main Outcome Measures

Results

Conclusions

Prediction of microvascular invasion in hepatocellular carcinoma with expert-inspiration and skeleton sharing deep learning

BACKGROUND AND AIMS

Radiological prediction of microvascular invasion (MVI) of hepatocellular carcinoma (HCC) is essential but few models were clinically implemented because of limited interpretability and generalizability.

METHODS

Based on 2096 patients in three independent HCC cohorts, we established and validated an MVI predicting model. First, we used data from the primary cohort to train a 3D-ResNet network for MVI prediction and then optimised the model with "expert-inspired training" for model construction. Second, we implemented the model to the other two cohorts using three implementation strategies, the original model implementation, data sharing model implementation and skeleton sharing model implementation, the latter two of which used part of the cohorts' data for fine-tuning. The areas under the receiver operating characteristic curve (AUCs) were calculated to compare the performances of different models.

RESULTS

For the MVI predicting model, the AUC of the expert-inspired model was 0.83 (95% CI: 0.77-0.88) compared to 0.54 (95% CI: 0.46-0.62) of model before expert-inspiring. Taking this model as an original model, AUC on the second cohort was 0.76 (95% CI: 0.67-0.84). The AUC was improved to 0.83 (95% CI: 0.77-0.90) with the data-sharing model, and further improved to 0.85 (95% CI: 0.79-0.92) with the skeleton sharing model. The trend that the skeleton sharing model had an advantage in performance was similar in the third cohort.

CONCLUSIONS

We established an expert-inspired model with better predictive performance and interpretability than the traditional constructed model. Skeleton sharing process is superior to data sharing and direct model implementation in model implementation.

Evaluating the utility of deep learning for predicting therapeutic response in diabetic eye disease

Purpose

Deep learning (DL) is a technique explored within ophthalmology that requires large datasets to distinguish feature representations with high diagnostic performance. There is a need for developing DL approaches to predict therapeutic response, but completed clinical trial datasets are limited in size. Predicting treatment response is more complex than disease diagnosis, where hallmarks of treatment response are subtle. This study seeks to understand the utility of DL for clinical problems in ophthalmology such as predicting treatment response and where large sample sizes for model training are not available.

Materials and Methods

Four DL architectures were trained using cross-validated transfer learning to classify ultra-widefield angiograms (UWFA) and fluid-compartmentalized optical coherence tomography (OCT) images from a completed clinical trial (PERMEATE) dataset (n=29) as tolerating or requiring extended interval Anti-VEGF dosing. UWFA images (n=217) from the Anti-VEGF study were divided into five increasingly larger subsets to evaluate the influence of dataset size on performance. Class activation maps (CAMs) were generated to identify regions of model attention.

Results

The best performing DL model had a mean AUC of 0.507 ± 0.042 on UWFA images, and highest observed AUC of 0.503 for fluid-compartmentalized OCT images. DL had a best performing AUC of 0.634 when dataset size was incrementally increased. Resulting CAMs show inconsistent regions of interest.

Conclusions

This study demonstrated the limitations of DL for predicting therapeutic response when large datasets were not available for model training. Our findings suggest the need for hand-crafted approaches for complex and data scarce prediction problems in ophthalmology.

Quantitative Imaging Biomarkers in Age-Related Macular Degeneration and Diabetic Eye Disease: A Step Closer to Precision Medicine

The management of retinal diseases relies heavily on digital imaging data, including optical coherence tomography (OCT) and fluorescein angiography (FA). Targeted feature extraction and the objective quantification of features provide important opportunities in biomarker discovery, disease burden assessment, and predicting treatment response. Additional important advantages include increased objectivity in interpretation, longitudinal tracking, and ability to incorporate computational models to create automated diagnostic and clinical decision support systems. Advances in computational technology, including deep learning and radiomics, open new doors for developing an imaging phenotype that may provide in-depth personalized disease characterization and enhance opportunities in precision medicine. In this review, we summarize current quantitative and radiomic imaging biomarkers described in the literature for age-related macular degeneration and diabetic eye disease using imaging modalities such as OCT, FA, and OCT angiography (OCTA). Various approaches used to identify and extract these biomarkers that utilize artificial intelligence and deep learning are also summarized in this review. These quantifiable biomarkers and automated approaches have unleashed new frontiers of personalized medicine where treatments are tailored, based on patient-specific longitudinally trackable biomarkers, and response monitoring can be achieved with a high degree of accuracy.

Multi-Compartment Spatially-Derived Radiomics From Optical Coherence Tomography Predict Anti-VEGF Treatment Durability in Macular Edema Secondary to Retinal Vascular Disease: Preliminary Findings

OBJECTIVE

Diabetic macular edema (DME) and retinal vein occlusion (RVO) are the leading causes of visual impairments across the world. Vascular endothelial growth factor (VEGF) stimulates breakdown of blood-retinal barrier that causes accumulation of fluid within macula. Anti-VEGF therapy is the first-line treatment for both the diseases; however, the degree of response varies for individual patients. The main objective of this work was to identify the (i) texture-based radiomics features within individual fluid and retinal tissue compartments of baseline spectral-domain optical coherence tomography (SD-OCT) images and (ii) the specific spatial compartments that contribute most pertinent features for predicting therapeutic response.

METHODS

A total of 962 texture-based radiomics features were extracted from each of the fluid and retinal tissue compartments of OCT images, obtained from the PERMEATE study. Top-performing features selected from the consensus of different feature selection methods were evaluated in conjunction with four different machine learning classifiers: Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA), Random Forest (RF), and Support Vector Machine (SVM) in a cross-validated approach to distinguish eyes tolerating extended interval dosing (non-rebounders) and those requiring more frequent dosing (rebounders).

RESULTS

Combination of fluid and retinal tissue features yielded a cross-validated area under receiver operating characteristic curve (AUC) of 0.78±0.08 in distinguishing rebounders from non-rebounders.

CONCLUSIONS

This study revealed that the texture-based radiomics features pertaining to IRF subcompartment were most discriminating between rebounders and non-rebounders to anti-VEGF therapy. Clinical Impact: With further validation, OCT-based imaging biomarkers could be used for treatment management of DME patients.