Deep Learning for Prediction of AMD Progression: A Pilot Study

A deep learning approach to explore the association of age-related macular degeneration polygenic risk score with retinal optical coherence tomography: A preliminary study

PURPOSE

Age-related macular degeneration (AMD) is a complex eye disorder affecting millions worldwide. This article uses deep learning techniques to investigate the relationship between AMD, genetics and optical coherence tomography (OCT) scans.

METHODS

The cohort consisted of 332 patients, of which 235 were diagnosed with AMD and 97 were controls with no signs of AMD. The genome-wide association studies summary statistics utilized to establish the polygenic risk score (PRS) in relation to AMD were derived from the GERA European study. A PRS estimation based on OCT volumes for both eyes was performed using a proprietary convolutional neural network (CNN) model supported by machine learning models. The method's performance was assessed using numerical evaluation metrics, and the Grad-CAM technique was used to evaluate the results by visualizing the features learned by the model.

RESULTS

The best results were obtained with the CNN and the Extra Tree regressor (MAE = 0.55, MSE = 0.49, RMSE = 0.70, R2 = 0.34). Extending the feature vector with additional information on AMD diagnosis, age and smoking history improved the results slightly, with mainly AMD diagnosis used by the model (MAE = 0.54, MSE = 0.44, RMSE = 0.66, R2 = 0.42). Grad-CAM heatmap evaluation showed that the model decisions rely on retinal morphology factors relevant to AMD diagnosis.

CONCLUSION

The developed method allows an efficient PRS estimation from OCT images. A new technique for analysing the association of OCT images with PRS of AMD, using a deep learning approach, may provide an opportunity to discover new associations between genotype-based AMD risk and retinal morphology.

Human versus Artificial Intelligence: Validation of a Deep Learning Model for Retinal Layer and Fluid Segmentation in Optical Coherence Tomography Images from Patients with Age-Related Macular Degeneration

Artificial intelligence (AI) models have received considerable attention in recent years for their ability to identify optical coherence tomography (OCT) biomarkers with clinical diagnostic potential and predict disease progression. This study aims to externally validate a deep learning (DL) algorithm by comparing its segmentation of retinal layers and fluid with a gold-standard method for manually adjusting the automatic segmentation of the Heidelberg Spectralis HRA + OCT software Version 6.16.8.0. A total of sixty OCT images of healthy subjects and patients with intermediate and exudative age-related macular degeneration (AMD) were included. A quantitative analysis of the retinal thickness and fluid area was performed, and the discrepancy between these methods was investigated. The results showed a moderate-to-strong correlation between the metrics extracted by both software types, in all the groups, and an overall near-perfect area overlap was observed, except for in the inner segment ellipsoid (ISE) layer. The DL system detected a significant difference in the outer retinal thickness across disease stages and accurately identified fluid in exudative cases. In more diseased eyes, there was significantly more disagreement between these methods. This DL system appears to be a reliable method for accessing important OCT biomarkers in AMD. However, further accuracy testing should be conducted to confirm its validity in real-world settings to ultimately aid ophthalmologists in OCT imaging management and guide timely treatment approaches.

Understanding and interpreting CNN's decision in optical coherence tomography-based AMD detection

INTRODUCTION

Automated assessment of age-related macular degeneration (AMD) using optical coherence tomography (OCT) has gained significant research attention in recent years. Though a list of convolutional neural network (CNN)-based methods has been proposed recently, methods that uncover the decision-making process of CNNs or critically interpret CNNs' decisions in the context are scant. This study aims to bridge this research gap.

METHODS

We independently trained several state-of-the-art CNN models, namely, VGG16, VGG19, Xception, ResNet50, InceptionResNetV2 for AMD detection and applied CNN visualization techniques, namely, Grad-CAM, Grad-CAM++, Score CAM, Faster Score CAM to highlight the regions of interest utilized by the CNNs in the context. Retinal layer segmentation methods were also developed to explore how the CNN regions of interest related to the layers of the retinal structure. Extensive experiments involving 2130 SD-OCT scans collected from Duke University were performed.

RESULTS

Experimental analysis shows that Outer Nuclear Layer to Inner Segment Myeloid (ONL-ISM) influences the AMD detection decision heavily as evident from the normalized intersection (NI) scores. For AMD cases the obtained average NI scores were respectively 13.13%, 17.2%, 9.7%, 10.95%, and 11.31% for VGG16, VGG19, ResNet50, Xception, and Inception ResNet V2, whereas, for normal cases, these values were respectively 21.7%, 21.3%, 16.85%, 10.175% and 16%.

CONCLUSION

Critical analysis reveals that the ONL-ISM is the most contributing layer in determining AMD, followed by Nerve Fiber Layer to Inner Plexiform Layer (NFL-IPL).

Enhancing Vault Prediction and ICL Sizing Through Advanced Machine Learning Models

PURPOSE

To use artificial intelligence (AI) technology to accurately predict vault and Implantable Collamer Lens (ICL) size.

METHODS

The methodology focused on enhancing predictive capabilities through the fusion of machine-learning algorithms. Specifically, AdaBoost, Random Forest, Decision Tree, Support Vector Regression, LightGBM, and XGBoost were integrated into a majority-vote model. The performance of each model was evaluated using appropriate metrics such as accuracy, precision, F1-score, and area under the curve (AUC).

RESULTS

The majority-vote model exhibited the highest performance among the classification models, with an accuracy of 81.9% area under the curve (AUC) of 0.807. Notably, LightGBM (accuracy = 0.788, AUC = 0.803) and XGBoost (ACC = 0.790, AUC = 0.801) demonstrated competitive results. For the ICL size prediction, the Random Forest model achieved an impressive accuracy of 85.3% (AUC = 0.973), whereas XG-Boost (accuracy = 0.834, AUC = 0.961) and LightGBM (accuracy = 0.816, AUC = 0.961) maintained their compatibility.

CONCLUSIONS

This study highlights the potential of diverse machine learning algorithms to enhance postoperative vault and ICL size prediction, ultimately contributing to the safety of ICL implantation procedures. Furthermore, the introduction of the novel majority-vote model demonstrates its capability to combine the advantages of multiple models, yielding superior accuracy. Importantly, this study will empower ophthalmologists to use a precise tool for vault prediction, facilitating informed ICL size selection in clinical practice. [J Refract Surg. 2024;40(3):e126-e132.].

RobOCTNet: Robotics and Deep Learning for Referable Posterior Segment Pathology Detection in an Emergency Department Population

Purpose

To evaluate the diagnostic performance of a robotically aligned optical coherence tomography (RAOCT) system coupled with a deep learning model in detecting referable posterior segment pathology in OCT images of emergency department patients.

Methods

A deep learning model, RobOCTNet, was trained and internally tested to classify OCT images as referable versus non-referable for ophthalmology consultation. For external testing, emergency department patients with signs or symptoms warranting evaluation of the posterior segment were imaged with RAOCT. RobOCTNet was used to classify the images. Model performance was evaluated against a reference standard based on clinical diagnosis and retina specialist OCT review.

Results

We included 90,250 OCT images for training and 1489 images for internal testing. RobOCTNet achieved an area under the curve (AUC) of 1.00 (95% confidence interval [CI], 0.99-1.00) for detection of referable posterior segment pathology in the internal test set. For external testing, RAOCT was used to image 72 eyes of 38 emergency department patients. In this set, RobOCTNet had an AUC of 0.91 (95% CI, 0.82-0.97), a sensitivity of 95% (95% CI, 87%-100%), and a specificity of 76% (95% CI, 62%-91%). The model's performance was comparable to two human experts' performance.

Conclusions

A robotically aligned OCT coupled with a deep learning model demonstrated high diagnostic performance in detecting referable posterior segment pathology in a cohort of emergency department patients.

Translational Relevance

Robotically aligned OCT coupled with a deep learning model may have the potential to improve emergency department patient triage for ophthalmology referral.

Clinical decision support system based on deep learning for evaluating implantable collamer lens size and vault after implantable collamer lens surgery: a retrospective study

OBJECTIVES

To aid doctors in selecting the optimal preoperative implantable collamer lens (ICL) size and to enhance the safety and surgical outcomes of ICL procedures, a clinical decision support system (CDSS) is proposed in our study.

DESIGN

A retrospective study of patients after ICL surgery.

SETTING

China Tertiary Myopia Prevention and Control Center.

PARTICIPANTS

2772 eyes belonging to 1512 patients after ICL surgery. Data were collected between 2018 and 2022.

OUTCOME MEASURES

A CDSS is constructed and used to predict vault at 1 month postoperatively and preoperative ICL dimensions using various artificial intelligence methods. Accuracy metrics as well as area under curve (AUC) parameters are used to determine the CDSS prediction methods.

RESULTS

Among the ICL size prediction models, conventional neural networks (CNNs) achieve the best prediction accuracy at 91.37% and exhibit the highest AUC of 0.842. Regarding the prediction model for vault values 1 month after surgery, CNN surpasses the other methods with an accuracy of 85.27%, which has the uppermost AUC of 0.815. Thus, we select CNN as the prediction algorithm for the CDSS.

CONCLUSIONS

This study introduces a CDSS to assist doctors in selecting the optimal ICL size for patients while improving the safety and postoperative outcomes of ICL surgery.

OCT Prognostic Biomarkers for Progression to Late Age-related Macular Degeneration: A Systematic Review and Meta-analysis

TOPIC

To evaluate which OCT prognostic biomarkers best predict the risk of progression from early/intermediate to late age-related macular degeneration (AMD).

CLINICAL RELEVANCE

Among > 100 OCT prognostic biomarkers for AMD, it is unclear which are the most relevant for clinicians and researchers to focus on. This review evaluated which OCT biomarkers confer the greatest magnitude of prediction for progression to late AMD.

METHODS

Study protocol was registered on PROSPERO (CRD42023400166). PubMed and Embase were searched from inception to March 2, 2023, and eligible studies assessed following the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach. The primary outcome was any quantified risk of progression from treatment-naive early/intermediate AMD to late AMD, including hazard ratios (HRs), odds ratios (ORs), and standardized mean differences (at baseline, between eyes with versus without progression), subgrouped by each OCT biomarker. Further meta-analyses were subgrouped by progression to geographic atrophy or neovascularization.

RESULTS

A total of 114 quantified OCT prognostic biomarkers were identified. With high GRADE certainty of evidence, the greatest magnitudes of prediction to late AMD belonged to: external limiting membrane abnormality (OR, 15.42 [7.63, 31.17]), ellipsoid zone abnormality (OR, 10.8 [4.58, 25.46]), interdigitation zone abnormality (OR, 7.68 [2.57, 23]), concurrent large drusen and reticular pseudodrusen (HR, 6.73 [1.35, 33.65], hyporeflective drusen cores (HR, 2.48 [1.8, 3.4]; OR 1.85 [1.29, 2.66]), intraretinal hyperreflective foci (IHRF; HR, 2.16 [0.92, 5.07]; OR 5.08 [3.26, 7.92]), and large drusen (HR, 2.01 [1.35, 2.99]); OR, 1.98 [1.27, 3.08]). There was greater risk of geographic atrophy for IHRF and hyporeflective drusen cores (P < 0.05), and neovascularization for ellipsoid zone abnormality (P < 0.05). Other OCT biomarkers such as drusenoid pigment epithelium detachment, shallow irregular retinal pigment epithelium elevations, and nascent geographic atrophy exhibited large magnitudes of risk but required further studies for validation.

CONCLUSION

This review synthesizes the 6 most relevant OCT prognostic biomarkers for AMD with greater predictive ability than large drusen alone, for clinicians and researchers to focus on. Further study is required to validate other biomarkers with less than high certainty of evidence, and assess how the copresence of biomarkers may affect risks.

FINANCIAL DISCLOSURE(S)

The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Artificial intelligence in retinal disease: clinical application, challenges, and future directions

Retinal diseases are a leading cause of blindness in developed countries, accounting for the largest share of visually impaired children, working-age adults (inherited retinal disease), and elderly individuals (age-related macular degeneration). These conditions need specialised clinicians to interpret multimodal retinal imaging, with diagnosis and intervention potentially delayed. With an increasing and ageing population, this is becoming a global health priority. One solution is the development of artificial intelligence (AI) software to facilitate rapid data processing. Herein, we review research offering decision support for the diagnosis, classification, monitoring, and treatment of retinal disease using AI. We have prioritised diabetic retinopathy, age-related macular degeneration, inherited retinal disease, and retinopathy of prematurity. There is cautious optimism that these algorithms will be integrated into routine clinical practice to facilitate access to vision-saving treatments, improve efficiency of healthcare systems, and assist clinicians in processing the ever-increasing volume of multimodal data, thereby also liberating time for doctor-patient interaction and co-development of personalised management plans.

Predicting Vault and Size of Posterior Chamber Phakic Intraocular Lens Using Sulcus to Sulcus-Optimized Artificial Intelligence Technology

PURPOSE

To investigate the accuracy of posterior chamber phakic intraocular lens (PIOL) vault and size prediction models based on sulcus to sulcus (STS) optimized artificial intelligence and big data analysis technology.

DESIGN

Big data and artificial intelligence prediction model.

METHODS

We included 5873 eyes with posterior chamber PIOL implantation, and the postoperative vault was measured using an anterior segment analyzer (Pentacam AXL) 1 month postoperatively. A random forest regression model and classification model were used to predict the postoperative vault and PIOL size. The postoperative vault and PIOL size were set as output features; other vault-related eye parameters were set as input features. The influence of white to white (WTW), horizontal sulcus to sulcus (STS), and vertical STS on predicting postoperative vault and PIOL size was analyzed and compared.

RESULTS

The mean preoperative WTW diameter was 11.64 ± 0.37 mm, the mean horizontal STS diameter was 11.85 ± 0.47 mm, and the mean vertical STS diameter was 12.39 ± 0.52 mm. In the regression model for numerical prediction of the vault, the combination of WTW, horizontal STS, and vertical STS was the most optimal for vault prediction (R2 = 0.3091, root mean square error [RMSE] = 0.1705); solely relying on WTW was the least optimal (R2 = 0.2849, RMSE = 0.1735). Among the models for classification prediction of the vault, the combination of WTW, horizontal STS, and vertical STS was the most accurate (accuracy, 0.6302; mean area under the curve, 0.8008; and mean precision recall rate, 0.6940). Moreover, the combination of WTW, horizontal STS, and vertical STS exhibited the highest accuracy for classification prediction of PIOL size (accuracy, 0.8170; mean area under the curve, 0.9540; and mean precision recall rate, 0.8864). Whether in the regression prediction models of vault values or in the classification prediction models of vault and PIOL size, the accuracy of STS optimized model was significantly improved compared with the traditional WTW model (P < .001).

CONCLUSION

Artificial intelligence combined with STS optimization contributes to the accuracy of PIOL size and vault prediction models. The random forest machine-learning model optimized by STS is superior to the traditional WTW model.

Federated learning for diagnosis of age-related macular degeneration

This paper presents a federated learning (FL) approach to train deep learning models for classifying age-related macular degeneration (AMD) using optical coherence tomography image data. We employ the use of residual network and vision transformer encoders for the normal vs. AMD binary classification, integrating four unique domain adaptation techniques to address domain shift issues caused by heterogeneous data distribution in different institutions. Experimental results indicate that FL strategies can achieve competitive performance similar to centralized models even though each local model has access to a portion of the training data. Notably, the Adaptive Personalization FL strategy stood out in our FL evaluations, consistently delivering high performance across all tests due to its additional local model. Furthermore, the study provides valuable insights into the efficacy of simpler architectures in image classification tasks, particularly in scenarios where data privacy and decentralization are critical using both encoders. It suggests future exploration into deeper models and other FL strategies for a more nuanced understanding of these models' performance. Data and code are available at https://github.com/QIAIUNCC/FL_UNCC_QIAI.

An AI model to estimate visual acuity based solely on cross-sectional OCT imaging of various diseases

PURPOSE

To develop an artificial intelligence (AI) model for estimating best-corrected visual acuity (BCVA) using horizontal and vertical optical coherence tomography (OCT) scans of various retinal diseases and examine factors associated with its accuracy.

METHODS

OCT images and associated BCVA measurements from 2,700 OCT images (accrued from 2004 to 2018 with an Atlantis, Triton; Topcon, Tokyo, Japan) of 756 eyes of 469 patients and their BCVA were retrospectively analysed. For each eye, one horizontal and one vertical OCT scan in cross-line mode were used. The GoogLeNet architecture was implemented. The coefficient of determination (R2), root mean square error (RMSE) and mean absolute error (MAE) were computed to evaluate the performance of the trained network.

RESULTS

R2, RMSE, and MAE were 0.512, 0.350, and 0.321, respectively. R2 was higher in phakic eyes than in pseudophakic eyes. Multivariable regression analysis showed that a higher R2 was significantly associated with better BCVA (p < 0.001) and a higher standard deviation of BCVA (p < 0.001). However, the performance was worse in an external validation, with R2 of 0.19. R2 values for retinal vein occlusion and age-related macular degeneration were 0.961 and 0.373 in the internal validation but 0.20 and 0.22 in the external validation.

CONCLUSION

Although underspecification appears to be a fundamental problem to be addressed in AI models for predicting visual acuity, the present results suggest that AI models might have potential for estimating BCVA from OCT in AMD and RVO. Further research is needed to improve the utility of BCVA estimation for these diseases.

OCT-based deep-learning models for the identification of retinal key signs

A new system based on binary Deep Learning (DL) convolutional neural networks has been developed to recognize specific retinal abnormality signs on Optical Coherence Tomography (OCT) images useful for clinical practice. Images from the local hospital database were retrospectively selected from 2017 to 2022. Images were labeled by two retinal specialists and included central fovea cross-section OCTs. Nine models were developed using the Visual Geometry Group 16 architecture to distinguish healthy versus abnormal retinas and to identify eight different retinal abnormality signs. A total of 21,500 OCT images were screened, and 10,770 central fovea cross-section OCTs were included in the study. The system achieved high accuracy in identifying healthy retinas and specific pathological signs, ranging from 93 to 99%. Accurately detecting abnormal retinal signs from OCT images is crucial for patient care. This study aimed to identify specific signs related to retinal pathologies, aiding ophthalmologists in diagnosis. The high-accuracy system identified healthy retinas and pathological signs, making it a useful diagnostic aid. Labelled OCT images remain a challenge, but our approach reduces dataset creation time and shows DL models' potential to improve ocular pathology diagnosis and clinical decision-making.

The Predictive Capabilities of Artificial Intelligence-Based OCT Analysis for Age-Related Macular Degeneration Progression-A Systematic Review

The era of artificial intelligence (AI) has revolutionized our daily lives and AI has become a powerful force that is gradually transforming the field of medicine. Ophthalmology sits at the forefront of this transformation thanks to the effortless acquisition of an abundance of imaging modalities. There has been tremendous work in the field of AI for retinal diseases, with age-related macular degeneration being at the top of the most studied conditions. The purpose of the current systematic review was to identify and evaluate, in terms of strengths and limitations, the articles that apply AI to optical coherence tomography (OCT) images in order to predict the future evolution of age-related macular degeneration (AMD) during its natural history and after treatment in terms of OCT morphological structure and visual function. After a thorough search through seven databases up to 1 January 2022 using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, 1800 records were identified. After screening, 48 articles were selected for full-text retrieval and 19 articles were finally included. From these 19 articles, 4 articles concentrated on predicting the anti-VEGF requirement in neovascular AMD (nAMD), 4 articles focused on predicting anti-VEGF efficacy in nAMD patients, 3 articles predicted the conversion from early or intermediate AMD (iAMD) to nAMD, 1 article predicted the conversion from iAMD to geographic atrophy (GA), 1 article predicted the conversion from iAMD to both nAMD and GA, 3 articles predicted the future growth of GA and 3 articles predicted the future outcome for visual acuity (VA) after anti-VEGF treatment in nAMD patients. Since using AI methods to predict future changes in AMD is only in its initial phase, a systematic review provides the opportunity of setting the context of previous work in this area and can present a starting point for future research.

Deep survival modeling of longitudinal retinal OCT volumes for predicting the onset of atrophy in patients with intermediate AMD

In patients with age-related macular degeneration (AMD), the risk of progression to late stages is highly heterogeneous, and the prognostic imaging biomarkers remain unclear. We propose a deep survival model to predict the progression towards the late atrophic stage of AMD. The model combines the advantages of survival modelling, accounting for time-to-event and censoring, and the advantages of deep learning, generating prediction from raw 3D OCT scans, without the need for extracting a predefined set of quantitative biomarkers. We demonstrate, in an extensive set of evaluations, based on two large longitudinal datasets with 231 eyes from 121 patients for internal evaluation, and 280 eyes from 140 patients for the external evaluation, that this model improves the risk estimation performance over standard deep learning classification models.

A Deep Learning Model for Automated Segmentation of Geographic Atrophy Imaged Using Swept-Source OCT

PURPOSE

To present a deep learning algorithm for segmentation of geographic atrophy (GA) using en face swept-source OCT (SS-OCT) images that is accurate and reproducible for the assessment of GA growth over time.

DESIGN

Retrospective review of images obtained as part of a prospective natural history study.

SUBJECTS

Patients with GA (n = 90), patients with early or intermediate age-related macular degeneration (n = 32), and healthy controls (n = 16).

METHODS

An automated algorithm using scan volume data to generate 3 image inputs characterizing the main OCT features of GA-hypertransmission in subretinal pigment epithelium (sub-RPE) slab, regions of RPE loss, and loss of retinal thickness-was trained using 126 images (93 with GA and 33 without GA, from the same number of eyes) using a fivefold cross-validation method and data augmentation techniques. It was tested in an independent set of one hundred eighty 6 × 6-mm² macular SS-OCT scans consisting of 3 repeated scans of 30 eyes with GA at baseline and follow-up as well as 45 images obtained from 42 eyes without GA.

MAIN OUTCOME MEASURES

The GA area, enlargement rate of GA area, square root of GA area, and square root of the enlargement rate of GA area measurements were calculated using the automated algorithm and compared with ground truth calculations performed by 2 manual graders. The repeatability of these measurements was determined using intraclass coefficients (ICCs).

RESULTS

There were no significant differences in the GA areas, enlargement rates of GA area, square roots of GA area, and square roots of the enlargement rates of GA area between the graders and the automated algorithm. The algorithm showed high repeatability, with ICCs of 0.99 and 0.94 for the GA area measurements and the enlargement rates of GA area, respectively. The repeatability limit for the GA area measurements made by grader 1, grader 2, and the automated algorithm was 0.28, 0.33, and 0.92 mm², respectively.

CONCLUSIONS

When compared with manual methods, this proposed deep learning-based automated algorithm for GA segmentation using en face SS-OCT images was able to accurately delineate GA and produce reproducible measurements of the enlargement rates of GA.

Big-data and artificial-intelligence-assisted vault prediction and EVO-ICL size selection for myopia correction

AIMS

To predict the vault and the EVO-implantable collamer lens (ICL) size by artificial intelligence (AI) and big data analytics.

METHODS

Six thousand two hundred and ninety-seven eyes implanted with an ICL from 3536 patients were included. The vault values were measured by the anterior segment analyzer (Pentacam HR). Permutation importance and Impurity-based feature importance are used to investigate the importance between the vault and input parameters. Regression models and classification models are applied to predict the vault. The ICL size is set as the target of the prediction, and the vault and the other input features are set as the new inputs for the ICL size prediction. Data were collected from 2015 to 2020. Random Forest, Gradient Boosting and XGBoost were demonstrated satisfying accuracy and mean area under the curve (AUC) scores in vault predicting and ICL sizing.

RESULTS

In the prediction of the vault, the Random Forest has the best results in the regression model (R²=0.315), then follows the Gradient Boosting (R²=0.291) and XGBoost (R²=0.285). The maximum classification accuracy is 0.828 in Random Forest, and the mean AUC is 0.765. The Random Forest predicts the ICL size with an accuracy of 82.2% and the Gradient Boosting and XGBoost, which are also compatible with 81.5% and 81.8% accuracy, respectively.

CONCLUSIONS

Random Forest, Gradient Boosting and XGBoost models are applicable for vault predicting and ICL sizing. AI may assist ophthalmologists in improving ICL surgery safety, designing surgical strategies, and predicting clinical outcomes.

A vision transformer architecture for the automated segmentation of retinal lesions in spectral domain optical coherence tomography images

Neovascular age-related macular degeneration (nAMD) is one of the major causes of irreversible blindness and is characterized by accumulations of different lesions inside the retina. AMD biomarkers enable experts to grade the AMD and could be used for therapy prognosis and individualized treatment decisions. In particular, intra-retinal fluid (IRF), sub-retinal fluid (SRF), and pigment epithelium detachment (PED) are prominent biomarkers for grading neovascular AMD. Spectral-domain optical coherence tomography (SD-OCT) revolutionized nAMD early diagnosis by providing cross-sectional images of the retina. Automatic segmentation and quantification of IRF, SRF, and PED in SD-OCT images can be extremely useful for clinical decision-making. Despite the excellent performance of convolutional neural network (CNN)-based methods, the task still presents some challenges due to relevant variations in the location, size, shape, and texture of the lesions. This work adopts a transformer-based method to automatically segment retinal lesion from SD-OCT images and qualitatively and quantitatively evaluate its performance against CNN-based methods. The method combines the efficient long-range feature extraction and aggregation capabilities of Vision Transformers with data-efficient training of CNNs. The proposed method was tested on a private dataset containing 3842 2-dimensional SD-OCT retina images, manually labeled by experts of the Franziskus Eye-Center, Muenster. While one of the competitors presents a better performance in terms of Dice score, the proposed method is significantly less computationally expensive. Thus, future research will focus on the proposed network's architecture to increase its segmentation performance while maintaining its computational efficiency.

Predicting treat-and-extend outcomes and treatment intervals in neovascular age-related macular degeneration from retinal optical coherence tomography using artificial intelligence

Purpose

To predict visual outcomes and treatment needs in a treat &amp; extend (T&amp;E) regimen in neovascular age-related macular degeneration (nAMD) using a machine learning model based on quantitative optical coherence tomography (OCT) imaging biomarkers.

Materials and methods

Study eyes of 270 treatment-naïve subjects, randomized to receiving ranibizumab therapy in the T&amp;E arm of a randomized clinical trial were considered. OCT volume scans were processed at baseline and at the first follow-up visit 4 weeks later. Automated image segmentation was performed, where intraretinal (IRF), subretinal (SRF) fluid, pigment epithelial detachment (PED), hyperreflective foci, and the photoreceptor layer were delineated using a convolutional neural network (CNN). A set of respective quantitative imaging biomarkers were computed across an Early Treatment Diabetic Retinopathy Study (ETDRS) grid to describe the retinal pathomorphology spatially and its change after the first injection. Lastly, using the computed set of OCT features and available clinical and demographic information, predictive models of outcomes and retreatment intervals were built using machine learning and their performance evaluated with a 10-fold cross-validation.

Results

Data of 228 evaluable patients were included, as some had missing scans or were lost to follow-up. Of those patients, 55% reached and maintained long (8, 10, 12 weeks) and another 45% stayed at short (4, 6 weeks) treatment intervals. This provides further evidence for a high disease activity in a major proportion of patients. The model predicted the extendable treatment interval group with an AUROC of 0.71, and the visual outcome with an AUROC of up to 0.87 when utilizing both, clinical and imaging features. The volume of SRF and the volume of IRF, remaining at the first follow-up visit, were found to be the most important predictive markers for treatment intervals and visual outcomes, respectively, supporting the important role of quantitative fluid parameters on OCT.

Conclusion

The proposed Artificial intelligence (AI) methodology was able to predict visual outcomes and retreatment intervals of a T&amp;E regimen from a single injection. The result of this study is an urgently needed step toward AI-supported management of patients with active and progressive nAMD.

Automatic Screening of the Eyes in a Deep-Learning–Based Ensemble Model Using Actual Eye Checkup Optical Coherence Tomography Images

Eye checkups have become increasingly important to maintain good vision and quality of life. As the population requiring eye checkups increases, so does the clinical work burden of clinicians. An automatic screening algorithm to reduce the clinicians’ workload is necessary. Machine learning (ML) has recently become one of the chief techniques for automated image recognition and is a helpful tool for identifying ocular diseases. However, the accuracy of ML models is lower in a clinical setting than in the laboratory. The performance of ML models depends on the training dataset. Eye checkups often prioritize speed and minimize image processing. Data distribution differs from the training dataset and, consequently, decreases prediction performance. The study aim was to investigate an ML model to screen for retinal diseases from low-quality optical coherence tomography (OCT) images captured during actual eye chechups to prevent a dataset shift. The ensemble model with convolutional neural networks (CNNs) and random forest models showed high screening performance in the single-shot OCT images captured during the actual eye checkups. Our study indicates the strong potential of the ensemble model combining the CNN and random forest models in accurately predicting abnormalities during eye checkups.

Learning Two-Stream CNN for Multi-Modal Age-related Macular Degeneration Categorization

This paper tackles automated categorization of Age-related Macular Degeneration (AMD), a common macular disease among people over 50. Previous research efforts mainly focus on AMD categorization with a single-modal input, let it be a color fundus photograph (CFP) or an OCT B-scan image. By contrast, we consider AMD categorization given a multi-modal input, a direction that is clinically meaningful yet mostly unexplored. Contrary to the prior art that takes a traditional approach of feature extraction plus classifier training that cannot be jointly optimized, we opt for end-to-end multi-modal Convolutional Neural Networks (MM-CNN). Our MM-CNN is instantiated by a two-stream CNN, with spatially-invariant fusion to combine information from the CFP and OCT streams. In order to visually interpret the contribution of the individual modalities to the final prediction, we extend the class activation mapping (CAM) technique to the multi-modal scenario. For effective training of MM-CNN, we develop two data augmentation methods. One is GAN-based CFP/OCT image synthesis, with our novel use of CAMs as conditional input of a high-resolution image-to-image translation GAN. The other method is Loose Pairing, which pairs a CFP image and an OCT image on the basis of their classes instead of eye identities. Experiments on a clinical dataset consisting of 1,094 CFP images and 1,289 OCT images acquired from 1,093 distinct eyes show that the proposed solution obtains better F1 and Accuracy than multiple baselines for multi-modal AMD categorization. Code and data are available at https://github.com/li-xirong/mmc-amd.

Novel technical and privacy-preserving technology for artificial intelligence in ophthalmology

PURPOSE OF REVIEW

The application of artificial intelligence (AI) in medicine and ophthalmology has experienced exponential breakthroughs in recent years in diagnosis, prognosis, and aiding clinical decision-making. The use of digital data has also heralded the need for privacy-preserving technology to protect patient confidentiality and to guard against threats such as adversarial attacks. Hence, this review aims to outline novel AI-based systems for ophthalmology use, privacy-preserving measures, potential challenges, and future directions of each.

RECENT FINDINGS

Several key AI algorithms used to improve disease detection and outcomes include: Data-driven, imagedriven, natural language processing (NLP)-driven, genomics-driven, and multimodality algorithms. However, deep learning systems are susceptible to adversarial attacks, and use of data for training models is associated with privacy concerns. Several data protection methods address these concerns in the form of blockchain technology, federated learning, and generative adversarial networks.

SUMMARY

AI-applications have vast potential to meet many eyecare needs, consequently reducing burden on scarce healthcare resources. A pertinent challenge would be to maintain data privacy and confidentiality while supporting AI endeavors, where data protection methods would need to rapidly evolve with AI technology needs. Ultimately, for AI to succeed in medicine and ophthalmology, a balance would need to be found between innovation and privacy.

SD-OCT Biomarkers and the Current Status of Artificial Intelligence in Predicting Progression from Intermediate to Advanced AMD

Age-related macular degeneration (AMD) is one of the leading causes of blindness in the Western World. Optical coherence tomography (OCT) has revolutionized the diagnosis and follow-up of AMD patients. This review focuses on SD-OCT imaging biomarkers which were identified as predictors for progression in intermediate AMD to late AMD, either geographic atrophy (GA) or choroidal neovascularization (CNV). Structural OCT remains the most compelling modality to study AMD features related to the progression such as drusen characteristics, hyperreflective foci (HRF), reticular pseudo-drusen (RPD), sub-RPE hyper-reflective columns and their impact on retinal layers. Further on, we reviewed articles that attempted to integrate biomarkers that have already proven their involvement in intermediate AMD progression, in their models of artificial intelligence (AI). By combining structural biomarkers with genetic risk and lifestyle the predictive ability becomes more accurate.

Risk Classification for Progression to Subfoveal Geographic Atrophy in Dry Age-Related Macular Degeneration Using Machine Learning-Enabled Outer Retinal Feature Extraction

BACKGROUND AND OBJECTIVE

To evaluate the utility of spectral-domain optical coherence tomography biomarkers to predict the development of subfoveal geographic atrophy (sfGA).

PATIENTS AND METHODS

This was a retrospective cohort analysis including 137 individuals with dry age-related macular degeneration without sfGA with 5 years of follow-up. Multiple spectral-domain optical coherence tomography quantitative metrics were generated, including ellipsoid zone (EZ) integrity and subretinal pigment epithelium (sub-RPE) compartment features.

RESULTS

Reduced mean EZ-RPE central subfield thickness and increased sub-RPE compartment thickness were significantly different between sfGA convertors and nonconvertors at baseline in both 2-year and 5-year sfGA risk assessment. Longitudinal change assessment showed a significantly higher degradation of EZ integrity in sfGA convertors. The predictive performance of a machine learning classification model based on 5-year and 2-year risk conversion to sfGA demonstrated an area under the receiver operating characteristic curve of 0.92 ± 0.06 and 0.96 ± 0.04, respectively.

CONCLUSIONS

Quantitative outer retinal and sub-RPE feature assessment using a machine learning-enabled retinal segmentation platform provides multiple parameters that are associated with progression to sfGA. [Ophthalmic Surg Lasers Imaging. 2022;53:31-39.].

Early Diagnosis of Multiple Sclerosis Using Swept-Source Optical Coherence Tomography and Convolutional Neural Networks Trained with Data Augmentation

BACKGROUND

The aim of this paper is to implement a system to facilitate the diagnosis of multiple sclerosis (MS) in its initial stages. It does so using a convolutional neural network (CNN) to classify images captured with swept-source optical coherence tomography (SS-OCT).

METHODS

SS-OCT images from 48 control subjects and 48 recently diagnosed MS patients have been used. These images show the thicknesses (45 × 60 points) of the following structures: complete retina, retinal nerve fiber layer, two ganglion cell layers (GCL+, GCL++) and choroid. The Cohen distance is used to identify the structures and the regions within them with greatest discriminant capacity. The original database of OCT images is augmented by a deep convolutional generative adversarial network to expand the CNN's training set.

RESULTS

The retinal structures with greatest discriminant capacity are the GCL++ (44.99% of image points), complete retina (26.71%) and GCL+ (22.93%). Thresholding these images and using them as inputs to a CNN comprising two convolution modules and one classification module obtains sensitivity = specificity = 1.0.

CONCLUSIONS

Feature pre-selection and the use of a convolutional neural network may be a promising, nonharmful, low-cost, easy-to-perform and effective means of assisting the early diagnosis of MS based on SS-OCT thickness data.

Artificial Intelligence to Detect Meibomian Gland Dysfunction From in-vivo Laser Confocal Microscopy

Background: In recent years, deep learning has been widely used in a variety of ophthalmic diseases. As a common ophthalmic disease, meibomian gland dysfunction (MGD) has a unique phenotype in in-vivo laser confocal microscope imaging (VLCMI). The purpose of our study was to investigate a deep learning algorithm to differentiate and classify obstructive MGD (OMGD), atrophic MGD (AMGD) and normal groups. Methods: In this study, a multi-layer deep convolution neural network (CNN) was trained using VLCMI from OMGD, AMGD and healthy subjects as verified by medical experts. The automatic differential diagnosis of OMGD, AMGD and healthy people was tested by comparing its image-based identification of each group with the medical expert diagnosis. The CNN was trained and validated with 4,985 and 1,663 VLCMI images, respectively. By using established enhancement techniques, 1,663 untrained VLCMI images were tested. Results: In this study, we included 2,766 healthy control VLCMIs, 2,744 from OMGD and 2,801 from AMGD. Of the three models, differential diagnostic accuracy of the DenseNet169 CNN was highest at over 97%. The sensitivity and specificity of the DenseNet169 model for OMGD were 88.8 and 95.4%, respectively; and for AMGD 89.4 and 98.4%, respectively. Conclusion: This study described a deep learning algorithm to automatically check and classify VLCMI images of MGD. By optimizing the algorithm, the classifier model displayed excellent accuracy. With further development, this model may become an effective tool for the differential diagnosis of MGD.

Artificial intelligence in OCT angiography

Optical coherence tomographic angiography (OCTA) is a non-invasive imaging modality that provides three-dimensional, information-rich vascular images. With numerous studies demonstrating unique capabilities in biomarker quantification, diagnosis, and monitoring, OCTA technology has seen rapid adoption in research and clinical settings. The value of OCTA imaging is significantly enhanced by image analysis tools that provide rapid and accurate quantification of vascular features and pathology. Today, the most powerful image analysis methods are based on artificial intelligence (AI). While AI encompasses a large variety of techniques, machine-learning-based, and especially deep-learning-based, image analysis provides accurate measurements in a variety of contexts, including different diseases and regions of the eye. Here, we discuss the principles of both OCTA and AI that make their combination capable of answering new questions. We also review contemporary applications of AI in OCTA, which include accurate detection of pathologies such as choroidal neovascularization, precise quantification of retinal perfusion, and reliable disease diagnosis.

Prediction of keratoconus progression using deep learning of anterior segment optical coherence tomography maps

Background

To predict keratoconus progression using deep learning of the color-coded maps measured with a swept-source anterior segment optical coherence tomography (As-OCT) device.

Methods

We enrolled 218 keratoconic eyes with and without disease progression. Using deep learning of the 6 color-coded maps (anterior elevation, anterior curvature, posterior elevation, posterior curvature, total refractive power, and pachymetry map) obtained by the As-OCT (CASIA, Tomey), we assessed the accuracy, sensitivity, and specificity of prediction of keratoconus progression in such eyes.

Results

Deep learning of the 6 color-coded maps exhibited an accuracy of 0.794 in discriminating keratoconus with and without progression. For a single map analysis, posterior elevation map (0.798) showed the highest accuracy, followed by anterior curvature map (0.775), posterior corneal curvature map (0.757), anterior elevation map (0.752), total refractive power map (0.729), and pachymetry map (0.720), in distinguishing between progressive and non-progressive keratoconus. The use of the adjusted algorithm by age subgroups improved to an accuracy of 0.849.

Conclusions

Deep learning of the As-OCT color-coded maps effectively discriminates progressive keratoconus from non-progressive keratoconus with an accuracy of approximately 85% using the adjusted age algorithm, indicating that it will become an aid for predicting the progression of the disease, which is clinically beneficial for decision-making of the surgical indication of corneal cross-linking (CXL).

Imaging and artificial intelligence for progression of age-related macular degeneration

Age-related macular degeneration (AMD) is a leading cause of severe vision loss. With our aging population, it may affect 288 million people globally by the year 2040. AMD progresses from an early and intermediate dry form to an advanced one, which manifests as choroidal neovascularization and geographic atrophy. Conversion to AMD-related exudation is known as progression to neovascular AMD, and presence of geographic atrophy is known as progression to advanced dry AMD. AMD progression predictions could enable timely monitoring, earlier detection and treatment, improving vision outcomes. Machine learning approaches, a subset of artificial intelligence applications, applied on imaging data are showing promising results in predicting progression. Extracted biomarkers, specifically from optical coherence tomography scans, are informative in predicting progression events. The purpose of this mini review is to provide an overview about current machine learning applications in artificial intelligence for predicting AMD progression, and describe the various methods, data-input types, and imaging modalities used to identify high-risk patients. With advances in computational capabilities, artificial intelligence applications are likely to transform patient care and management in AMD. External validation studies that improve generalizability to populations and devices, as well as evaluating systems in real-world clinical settings are needed to improve the clinical translations of artificial intelligence AMD applications.

An end-to-end network for segmenting the vasculature of three retinal capillary plexuses from OCT angiographic volumes

The segmentation of en face retinal capillary angiograms from volumetric optical coherence tomographic angiography (OCTA) usually relies on retinal layer segmentation, which is time-consuming and error-prone. In this study, we developed a deep-learning-based method to segment vessels in the superficial vascular plexus (SVP), intermediate capillary plexus (ICP), and deep capillary plexus (DCP) directly from volumetric OCTA data. The method contains a three-dimensional convolutional neural network (CNN) for extracting distinct retinal layers, a custom projection module to generate three vascular plexuses from OCTA data, and three parallel CNNs to segment vasculature. Experimental results on OCTA data from rat eyes demonstrated the feasibility of the proposed method. This end-to-end network has the potential to simplify OCTA data processing on retinal vasculature segmentation. The main contribution of this study is that we propose a custom projection module to connect retinal layer segmentation and vasculature segmentation modules and automatically convert data from three to two dimensions, thus establishing an end-to-end method to segment three retinal capillary plexuses from volumetric OCTA without any human intervention.

Assessing the Clinical Utility of Expanded Macular OCTs Using Machine Learning

Purpose

Optical coherence tomography (OCT) is widely used in the management of retinal pathologies, including age-related macular degeneration (AMD), diabetic macular edema (DME), and primary open-angle glaucoma (POAG). We used machine learning techniques to understand diagnostic performance gains from expanding macular OCT B-scans compared with foveal-only OCT B-scans for these conditions.

Methods

Electronic medical records were extracted to obtain 61 B-scans per eye from patients with AMD, diabetic retinopathy, or POAG. We constructed deep neural networks and random forest ensembles and generated area under the receiver operating characteristic (AUROC) and area under the precision recall (AUPR) curves.

Results

After extracting 630,000 OCT images, we achieved improved AUROC and AUPR curves when comparing the central image (one B-scan) to all images (61 B-scans). The AUROC and AUPR points of diminishing return for diagnostic accuracy for macular OCT coverage were found to be within 2.75 to 4.00 mm (14–19 B-scans), 4.25 to 4.50 mm (20–21 B-scans), and 4.50 to 6.25 mm (21–28 B-scans) for AMD, DME, and POAG, respectively. All models with >0.25 mm of coverage had statistically significantly improved AUROC/AUPR curves for all diseases (P < 0.05).

Conclusions

Systematically expanded macular coverage models demonstrated significant differences in total macular coverage required for improved diagnostic accuracy, with the largest macular area being relevant in POAG followed by DME and then AMD. These findings support our hypothesis that the extent of macular coverage by OCT imaging in the clinical setting, for any of the three major disorders, has a measurable impact on the functionality of artificial intelligence decision support.

Translational Relevance

We used machine learning techniques to improve OCT imaging standards for common retinal disease diagnoses.

Classifying Retinal Degeneration in Histological Sections Using Deep Learning

Purpose

Artificial intelligence (AI) techniques are increasingly being used to classify retinal diseases. In this study we investigated the ability of a convolutional neural network (CNN) in categorizing histological images into different classes of retinal degeneration.

Methods

Images were obtained from a chemically induced feline model of monocular retinal dystrophy and split into training and testing sets. The training set was graded for the level of retinal degeneration and used to train various CNN architectures. The testing set was evaluated through the best architecture and graded by six observers. Comparisons between model and observer classifications, and interobserver variability were measured. Finally, the effects of using less training images or images containing half the presentable context were investigated.

Results

The best model gave weighted-F1 scores in the range 85% to 90%. Cohen kappa scores reached up to 0.86, indicating high agreement between the model and observers. Interobserver variability was consistent with the model-observer variability in the model's ability to match predictions with the observers. Image context restriction resulted in model performance reduction by up to 6% and at least one training set size resulted in a model performance reduction of 10% compared to the original size.

Conclusions

Detecting the presence and severity of up to three classes of retinal degeneration in histological data can be reliably achieved with a deep learning classifier.

Translational Relevance

This work lays the foundations for future AI models which could aid in the evaluation of more intricate changes occurring in retinal degeneration, particularly in other types of clinically derived image data.

AI-based monitoring of retinal fluid in disease activity and under therapy

Retinal fluid as the major biomarker in exudative macular disease is accurately visualized by high-resolution three-dimensional optical coherence tomography (OCT), which is used world-wide as a diagnostic gold standard largely replacing clinical examination. Artificial intelligence (AI) with its capability to objectively identify, localize and quantify fluid introduces fully automated tools into OCT imaging for personalized disease management. Deep learning performance has already proven superior to human experts, including physicians and certified readers, in terms of accuracy and speed. Reproducible measurement of retinal fluid relies on precise AI-based segmentation methods that assign a label to each OCT voxel denoting its fluid type such as intraretinal fluid (IRF) and subretinal fluid (SRF) or pigment epithelial detachment (PED) and its location within the central 1-, 3- and 6-mm macular area. Such reliable analysis is most relevant to reflect differences in pathophysiological mechanisms and impacts on retinal function, and the dynamics of fluid resolution during therapy with different regimens and substances. Yet, an in-depth understanding of the mode of action of supervised and unsupervised learning, the functionality of a convolutional neural net (CNN) and various network architectures is needed. Greater insight regarding adequate methods for performance, validation assessment, and device- and scanning-pattern-dependent variations is necessary to empower ophthalmologists to become qualified AI users. Fluid/function correlation can lead to a better definition of valid fluid variables relevant for optimal outcomes on an individual and a population level. AI-based fluid analysis opens the way for precision medicine in real-world practice of the leading retinal diseases of modern times.

Diagnosis of central serous chorioretinopathy by deep learning analysis of en face images of choroidal vasculature: A pilot study

Purpose

To diagnose central serous chorioretinopathy (CSC) by deep learning (DL) analyses of en face images of the choroidal vasculature obtained by optical coherence tomography (OCT) and to analyze the regions of interest for the DL from heatmaps.

Methods

One-hundred eyes were studied; 53 eyes with CSC and 47 normal eyes. Volume scans of 12×12 mm square were obtained at the same time as the OCT angiographic (OCTA) scans (Plex Elite 9000 Swept-Source OCT^®, Zeiss). High-quality en face images of the choroidal vasculature of the segmentation slab of one-half of the subfoveal choroidal thickness were created for the analyses. The 100 en face images were divided into 80 for training and 20 for validation. Thus, we divided it into five groups of 20 eyes each, trained the remaining 80 eyes in each group, and then calculated the correct answer rate for each group by validation with 20 eyes. The Neural Network Console (NNC) developed by Sony and the Keras-Tensorflow backend developed by Google were used as the software for the classification with 16 layers of convolutional neural networks. The active region of the heatmap based on the feature quantity extracted by DL was also evaluated as the percentages with gradient-weighted class activation mapping implemented in Keras.

Results

The mean accuracy rate of the validation was 95% for NNC and 88% for Keras. This difference was not significant (P >0.1). The mean active region in the heatmap image was 12.5% in CSC eyes which was significantly lower than the 79.8% in normal eyes (P<0.01).

Conclusions

CSC can be automatically diagnosed by DL with high accuracy from en face images of the choroidal vasculature with different programs, convolutional layer structures, and small data sets. Heatmap analyses showed that the DL focused on the area occupied by the choroidal vessels and their uniformity. We conclude that DL can help in the diagnosis of CSC.

Detection of Fuchs' Uveitis Syndrome From Slit-Lamp Images Using Deep Convolutional Neural Networks in a Chinese Population

Fuchs' uveitis syndrome (FUS) is one of the most under- or misdiagnosed uveitis entities. Many undiagnosed FUS patients are unnecessarily overtreated with anti-inflammatory drugs, which may lead to serious complications. To offer assistance for ophthalmologists in the screening and diagnosis of FUS, we developed seven deep convolutional neural networks (DCNNs) to detect FUS using slit-lamp images. We also proposed a new optimized model with a mixed "attention" module to improve test accuracy. In the same independent set, we compared the performance between these DCNNs and ophthalmologists in detecting FUS. Seven different network models, including Xception, Resnet50, SE-Resnet50, ResNext50, SE-ResNext50, ST-ResNext50, and SET-ResNext50, were used to predict FUS automatically with the area under the receiver operating characteristic curves (AUCs) that ranged from 0.951 to 0.977. Our proposed SET-ResNext50 model (accuracy = 0.930; Precision = 0.918; Recall = 0.923; F1 measure = 0.920) with an AUC of 0.977 consistently outperformed the other networks and outperformed general ophthalmologists by a large margin. Heat-map visualizations of the SET-ResNext50 were provided to identify the target areas in the slit-lamp images. In conclusion, we confirmed that a trained classification method based on DCNNs achieved high effectiveness in distinguishing FUS from other forms of anterior uveitis. The performance of the DCNNs was better than that of general ophthalmologists and could be of value in the diagnosis of FUS.

Next-Generation Sequencing Applications for Inherited Retinal Diseases

Inherited retinal diseases (IRDs) represent a collection of phenotypically and genetically diverse conditions. IRDs phenotype(s) can be isolated to the eye or can involve multiple tissues. These conditions are associated with diverse forms of inheritance, and variants within the same gene often can be associated with multiple distinct phenotypes. Such aspects of the IRDs highlight the difficulty met when establishing a genetic diagnosis in patients. Here we provide an overview of cutting-edge next-generation sequencing techniques and strategies currently in use to maximise the effectivity of IRD gene screening. These techniques have helped researchers globally to find elusive causes of IRDs, including copy number variants, structural variants, new IRD genes and deep intronic variants, among others. Resolving a genetic diagnosis with thorough testing enables a more accurate diagnosis and more informed prognosis and should also provide information on inheritance patterns which may be of particular interest to patients of a child-bearing age. Given that IRDs are heritable conditions, genetic counselling may be offered to help inform family planning, carrier testing and prenatal screening. Additionally, a verified genetic diagnosis may enable access to appropriate clinical trials or approved medications that may be available for the condition.

Spatio-temporal alterations in retinal and choroidal layers in the progression of age-related macular degeneration (AMD) in optical coherence tomography

Age-related macular degeneration (AMD) is the predominant cause of vision loss in the elderly with a major impact on ageing societies and healthcare systems. A major challenge in AMD management is the difficulty to determine the disease stage, the highly variable progression speed and the risk of conversion to advanced AMD, where irreversible functional loss occurs. In this study we developed an optical coherence tomography (OCT) imaging based spatio-temporal reference frame to characterize the morphologic progression of intermediate age-related macular degeneration (AMD) and to identify distinctive patterns of conversion to the advanced stages macular neovascularization (MNV) and macular atrophy (MA). We included 10,040 OCT volumes of 518 eyes with intermediate AMD acquired according to a standardized protocol in monthly intervals over two years. Two independent masked retina specialists determined the time of conversion to MNV or MA. All scans were aligned to a common reference frame by intra-patient and inter-patient registration. Automated segmentations of retinal layers and the choroid were computed and en-face maps were transformed into the common reference frame. Population maps were constructed in the subgroups converting to MNV (n=135), MA (n=50) and in non-progressors (n=333). Topographically resolved maps of changes were computed and tested for statistical significant differences. The development over time was analysed by a joint model accounting for longitudinal and right-censoring aspect. Significantly enhanced thinning of the outer nuclear layer (ONL) and retinal pigment epithelium (RPE)–photoreceptorinner segment/outer segment (PR-IS/OS) layers within the central 3 mm and a faster thinning speed preceding conversion was documented for MA progressors. Converters to MNV presented an accelerated thinning of the choroid and appearance changes in the choroid prior to MNV onset. The large-scale automated image analysis allowed us to distinctly assess the progression of morphologic changes in intermediate AMD based on conventional OCT imaging. Distinct topographic and temporal patterns allow to prospectively determine eyes with risk of progression and thereby greatly improving early detection, prevention and development of novel therapeutic strategies.

Automated identification of clinical features from sparsely annotated 3-dimensional medical imaging

One of the core challenges in applying machine learning and artificial intelligence to medicine is the limited availability of annotated medical data. Unlike in other applications of machine learning, where an abundance of labeled data is available, the labeling and annotation of medical data and images require a major effort of manual work by expert clinicians who do not have the time to annotate manually. In this work, we propose a new deep learning technique (SLIVER-net), to predict clinical features from 3-dimensional volumes using a limited number of manually annotated examples. SLIVER-net is based on transfer learning, where we borrow information about the structure and parameters of the network from publicly available large datasets. Since public volume data are scarce, we use 2D images and account for the 3-dimensional structure using a novel deep learning method which tiles the volume scans, and then adds layers that leverage the 3D structure. In order to illustrate its utility, we apply SLIVER-net to predict risk factors for progression of age-related macular degeneration (AMD), a leading cause of blindness, from optical coherence tomography (OCT) volumes acquired from multiple sites. SLIVER-net successfully predicts these factors despite being trained with a relatively small number of annotated volumes (hundreds) and only dozens of positive training examples. Our empirical evaluation demonstrates that SLIVER-net significantly outperforms standard state-of-the-art deep learning techniques used for medical volumes, and its performance is generalizable as it was validated on an external testing set. In a direct comparison with a clinician panel, we find that SLIVER-net also outperforms junior specialists, and identifies AMD progression risk factors similarly to expert retina specialists.

Application of Deep Learning for Diagnosing, Classifying, and Treating Age-Related Macular Degeneration

Age-related macular degeneration (AMD) affects nearly 200 million people and is the third leading cause of irreversible vision loss worldwide. Deep learning, a branch of artificial intelligence that can learn image recognition based on pre-existing datasets, creates an opportunity for more accurate and efficient diagnosis, classification, and treatment of AMD on both individual and population levels. Current algorithms based on fundus photography and optical coherence tomography imaging have already achieved diagnostic accuracy levels comparable to human graders. This accuracy can be further increased when deep learning algorithms are simultaneously applied to multiple diagnostic imaging modalities. Combined with advances in telemedicine and imaging technology, deep learning can enable large populations of patients to be screened than would otherwise be possible and allow ophthalmologists to focus on seeing those patients who are in need of treatment, thus reducing the number of patients with significant visual impairment from AMD.

AMD Genetics: Methods and Analyses for Association, Progression, and Prediction

Age-related macular degeneration (AMD) is a multifactorial neurodegenerative disease, which is a leading cause of vision loss among the elderly in the developed countries. As one of the most successful examples of genome-wide association study (GWAS), a large number of genetic studies have been conducted to explore the genetic basis for AMD and its progression, of which over 30 loci were identified and confirmed. In this chapter, we review the recent development and findings of GWAS for AMD risk and progression. Then, we present emerging methods and models for predicting AMD development or its progression using large-scale genetic data. Finally, we discuss a set of novel statistical and analytical methods that were recently developed to tackle the challenges such as analyzing bilateral correlated eye-level outcomes that are subject to censoring with high-dimensional genetic data. Future directions for analytical studies of AMD genetics are also proposed.

Predicting wet age-related macular degeneration (AMD) using DARC (detecting apoptosing retinal cells) AI (artificial intelligence) technology

OBJECTIVES

To assess a recently described CNN (convolutional neural network) DARC (Detection of Apoptosing Retinal Cells) algorithm in predicting new Subretinal Fluid (SRF) formation in Age-related-Macular-Degeneration (AMD).

METHODS

Anonymized DARC, baseline and serial OCT images (n = 427) from 29 AMD eyes of Phase 2 clinical trial (ISRCTN10751859) were assessed with CNN algorithms, enabling the location of each DARC spot on corresponding OCT slices (n = 20,629). Assessment of DARC in a rabbit model of angiogenesis was performed in parallel.

RESULTS

A CNN DARC count >5 at baseline was significantly (p = 0.0156) related to development of new SRF throughout 36 months. Prediction rate of eyes using unique DARC spots overlying new SRF had positive predictive values, sensitivities and specificities >70%, with DARC count significantly (p < 0.005) related to the magnitude of SRF accumulation at all time points. DARC identified earliest stages of angiogenesis in-vivo.

CONCLUSIONS

DARC was able to predict new wet-AMD activity. Using only an OCT-CNN definition of new SRF, we demonstrate that DARC can identify early endothelial neovascular activity, as confirmed by rabbit studies. Although larger validation studies are required, this shows the potential of DARC as a biomarker of wet AMD, and potentially saving vision-loss.

End-to-End Deep Learning Model for Predicting Treatment Requirements in Neovascular AMD From Longitudinal Retinal OCT Imaging

Neovascular age-related macular degeneration (nAMD) is nowadays successfully treated with anti-VEGF substances, but inter-individual treatment requirements are vastly heterogeneous and currently poorly plannable resulting in suboptimal treatment frequency. Optical coherence tomography (OCT) with its 3D high-resolution imaging serves as a companion diagnostic to anti-VEGF therapy. This creates a need for building predictive models using automated image analysis of OCT scans acquired during the treatment initiation phase. We propose such a model based on deep learning (DL) architecture, comprised of a densely connected neural network (DenseNet) and a recurrent neural network (RNN), trainable end-to-end. The method starts by sampling several 2D-images from an OCT volume to obtain a lower-dimensional OCT representation. At the core of the predictive model, the DenseNet learns useful retinal spatial features while the RNN integrates information from different time points. The introduced model was evaluated on the prediction of anti-VEGF treatment requirements in nAMD patients treated under a pro-re-nata (PRN) regimen. The DL model was trained on 281 patients and evaluated on a hold-out test set of 69 patient. The predictive model achieved a concordance index of 0.7 in regressing the number of received treatments, while in a classification task it obtained an 0.85 (0.81) AUC in detecting the patients with low (high) treatment requirements. The proposed model outperformed previous machine learning strategies that relied on a set of spatio-temporal image features, showing that the proposed DL architecture successfully learned to extract the relevant spatio-temporal patterns directly from raw longitudinal OCT images.

Can the Onset of Neovascular Age-Related Macular Degeneration Be an Acceptable Endpoint for Prophylactic Clinical Trials?

Many studies over the past 20 years have pursued the goal of preventing or deferring progression from early and intermediate age-related macular degeneration (AMD) to advanced AMD. The onset of neovascular AMD has been used as a primary endpoint in some prophylactic clinical trials because it is easy to assess and relatively well-defined. Nevertheless, the use of this endpoint for assessing progression of AMD lacks validation. The aims of this paper are to review the current practice of clinical trials investigating the prevention of progression of early or intermediate AMD to neovascular AMD, so-called prophylactic trials, as well as identify ongoing efforts to standardize endpoints and select the ideal population for such studies.

An Intelligent Optical Coherence Tomography-based System for Pathological Retinal Cases Identification and Urgent Referrals

Purpose

This study aimed to develop an automated system with artificial intelligence algorithms to comprehensively identify pathologic retinal cases and make urgent referrals.

Methods

To build and test the intelligent system, this study obtained 28,664 optical coherence tomography (OCT) images from 2254 patients in the Eye and ENT Hospital of Fudan University (EENT Hospital) and Shanghai Tenth People's Hospital (TENTH Hospital). We applied a deep learning model with an adapted feature pyramid network to detect 15 categories of retinal pathologies from OCT images as common signs of various retinal diseases. Subsequently, the pathologies detected in the OCT images and thickness features extracted from retinal thickness measurements were combined for urgent referral using the random forest tool.

Results

The retinal pathologies detection model had a sensitivity of 96.39% and specificity of 98.91% from the EENT Hospital test dataset, whereas those from the TENTH Hospital test dataset were 94.89% and 98.76%, respectively. The urgent referral model achieved accuracies of 98.12% and 98.01% from the EENT Hospital and TENTH Hospital test datasets, respectively.

Conclusions

An intelligent system capable of automatically identifying pathologic retinal cases and offering urgent referrals was developed and demonstrated reliable performance with high sensitivity, specificity, and accuracy.

Translational Relevance

This intelligent system has great value and practicability in communities where exist increasing cases of retinal disease and a lack of ophthalmologists.

Artificial intelligence for diabetic retinopathy screening, prediction and management

PURPOSE OF REVIEW

Diabetic retinopathy is the most common specific complication of diabetes mellitus. Traditional care for patients with diabetes and diabetic retinopathy is fragmented, uncoordinated and delivered in a piecemeal nature, often in the most expensive and high-resource tertiary settings. Transformative new models incorporating digital technology are needed to address these gaps in clinical care.

RECENT FINDINGS

Artificial intelligence and telehealth may improve access, financial sustainability and coverage of diabetic retinopathy screening programs. They enable risk stratifying patients based on individual risk of vision-threatening diabetic retinopathy including diabetic macular edema (DME), and predicting which patients with DME best respond to antivascular endothelial growth factor therapy.

SUMMARY

Progress in artificial intelligence and tele-ophthalmology for diabetic retinopathy screening, including artificial intelligence applications in 'real-world settings' and cost-effectiveness studies are summarized. Furthermore, the initial research on the use of artificial intelligence models for diabetic retinopathy risk stratification and management of DME are outlined along with potential future directions. Finally, the need for artificial intelligence adoption within ophthalmology in response to coronavirus disease 2019 is discussed. Digital health solutions such as artificial intelligence and telehealth can facilitate the integration of community, primary and specialist eye care services, optimize the flow of patients within healthcare networks, and improve the efficiency of diabetic retinopathy management.

Diving Deep into Deep Learning: An Update on Artificial Intelligence in Retina

Purpose of Review

In the present article, we will provide an understanding and review of artificial intelligence in the subspecialty of retina and its potential applications within the specialty.

Recent Findings

Given the significant use of diagnostic imaging within retina, this subspecialty is a fitting area for the incorporation of artificial intelligence. Researchers have aimed at creating models to assist in the diagnosis and management of retinal disease as well as in the prediction of disease course and treatment response. Most of this work thus far has focused on diabetic retinopathy, age-related macular degeneration, and retinopathy of prematurity, although other retinal diseases have started to be explored as well.

Summary

Artificial intelligence is well-suited to transform the practice of ophthalmology. A basic understanding of the technology is important for its effective implementation and growth.

Predicting conversion to wet age-related macular degeneration using deep learning

Progression to exudative 'wet' age-related macular degeneration (exAMD) is a major cause of visual deterioration. In patients diagnosed with exAMD in one eye, we introduce an artificial intelligence (AI) system to predict progression to exAMD in the second eye. By combining models based on three-dimensional (3D) optical coherence tomography images and corresponding automatic tissue maps, our system predicts conversion to exAMD within a clinically actionable 6-month time window, achieving a per-volumetric-scan sensitivity of 80% at 55% specificity, and 34% sensitivity at 90% specificity. This level of performance corresponds to true positives in 78% and 41% of individual eyes, and false positives in 56% and 17% of individual eyes at the high sensitivity and high specificity points, respectively. Moreover, we show that automatic tissue segmentation can identify anatomical changes before conversion and high-risk subgroups. This AI system overcomes substantial interobserver variability in expert predictions, performing better than five out of six experts, and demonstrates the potential of using AI to predict disease progression.

Visual Pathway Degeneration in Chemotherapy-Related Neurotoxicity: A Review and Directions for Future Research

Chemotherapy-related neurotoxicity (CRNT) is an emerging public health concern. Visual pathway degeneration may be a symptom of CRNT. We surveyed the current literature for evidence of visual pathway degeneration in cancer patients receiving chemotherapy. A systematic review was conducted in PubMed. Six published articles met our inclusion criteria. The studies showed reduced retinal thickness, primarily in the retinal nerve fibre layer, and impaired inner retinal function in patients receiving chemotherapy. In summary, the current literature suggests chemotherapy may induce visual pathway degeneration. Future research may benefit from improving study design, exploring mechanisms of chemotherapy-related visual pathway degeneration, and incorporating these findings into biomarker development.