The possibility of the combination of OCT and fundus images for improving the diagnostic accuracy of deep learning for age-related macular degeneration: a preliminary experiment

Publicly available imaging datasets for age-related macular degeneration: Evaluation according to the Findable, Accessible, Interoperable, Reusable (FAIR) principles

Age-related macular degeneration (AMD), a leading cause of vision loss among older adults, affecting more than 200 million people worldwide. With no cure currently available and a rapidly increasing prevalence, emerging approaches such as artificial intelligence (AI) and machine learning (ML) hold promise for advancing the study of AMD. The effective utilization of AI and ML in AMD research is highly dependent on access to high-quality and reusable clinical data. The Findable, Accessible, Interoperable, Reusable (FAIR) principles, published in 2016, provide a framework for sharing data that is easily useable by both humans and machines. However, it is unclear how these principles are implemented with regards to ophthalmic imaging datasets for AMD research. We evaluated openly available AMD-related datasets containing optical coherence tomography (OCT) data against the FAIR principles. The assessment revealed that none of the datasets were fully compliant with FAIR principles. Specifically, compliance rates were 5 % for Findable, 82 % for Accessible, 73 % for Interoperable, and 0 % for Reusable. The low compliance rates can be attributed to the relatively recent emergence of these principles and the lack of established standards for data and metadata formatting in the AMD research community. This article presents our findings and offers guidelines for adopting FAIR practices to enhance data sharing in AMD research.

Enhancing pathological myopia diagnosis: a bimodal artificial intelligence approach integrating fundus and optical coherence tomography imaging for precise atrophy, traction and neovascularisation grading

BACKGROUND

Pathological myopia (PM) has emerged as a leading cause of global visual impairment, early detection and precise grading of PM are crucial for timely intervention. The atrophy, traction and neovascularisation (ATN) system is applied to define PM progression and stages with precision. This study focuses on constructing a comprehensive PM image dataset comprising both fundus and optical coherence tomography (OCT) images and developing a bimodal artificial intelligence (AI) classification model for ATN grading in PM.

METHODS

This single-centre retrospective cross-sectional study collected 2760 colour fundus photographs and matching OCT images of PM from January 2019 to November 2022 at Peking Union Medical College Hospital. Ophthalmology specialists labelled and inspected all paired images using the ATN grading system. The AI model used a ResNet-50 backbone and a multimodal multi-instance learning module to enhance interaction across instances from both modalities.

RESULTS

Performance comparisons among single-modality fundus, OCT and bimodal AI models were conducted for ATN grading in PM. The bimodality model, dual-deep learning (DL), demonstrated superior accuracy in both detailed multiclassification and biclassification of PM, which aligns well with our observation from instance attention-weight activation maps. The area under the curve for severe PM using dual-DL was 0.9635 (95% CI 0.9380 to 0.9890), compared with 0.9359 (95% CI 0.9027 to 0.9691) for the solely OCT model and 0.9268 (95% CI 0.8915 to 0.9621) for the fundus model.

CONCLUSIONS

Our novel bimodal AI multiclassification model for PM ATN staging proves accurate and beneficial for public health screening and prompt referral of PM patients.

Low-Rank Fine-Tuning Meets Cross-modal Analysis: A Robust Framework for Age-Related Macular Degeneration Categorization

Age-related macular degeneration (AMD) is a prevalent retinal degenerative disease among the elderly and is a major cause of irreversible vision loss worldwide. Although color fundus photography (CFP) and optical coherence tomography (OCT) are widely used for AMD diagnosis, information from a single modal is inadequate to fully capture the complex pathological features of AMD. To address this, this study proposes an innovative multi-modal deep learning framework that fine-tunes pre-trained single-modal retinal models for efficient application in multi-modal AMD categorization tasks. Specifically, two independent vision transformer models are used to extract features from CFP and OCT images, followed by deep canonical correlation analysis (DCCA) to perform nonlinear mapping and fusion of features from both modalities, maximizing cross-modal feature correlation. Moreover, to reduce the computational complexity of multi-modal integration, we introduce the low-rank adaptation (LoRA) technique, which uses low-rank decomposition of parameter matrices, achieving superior performance compared to full fine-tuning with only about 0.49% of the trainable parameters. Experimental results on the public dataset MMC-AMD validate the framework's effectiveness. The proposed model achieves an overall F1-score of 0.948, AUC-ROC of 0.991, and accuracy of 0.949, significantly outperforming existing single-modal and multi-modal baseline models, particularly excelling in recognizing complex pathological categories.

MultiEYE: Dataset and Benchmark for OCT-Enhanced Retinal Disease Recognition From Fundus Images

Existing multi-modal learning methods on fundus and OCT images mostly require both modalities to be available and strictly paired for training and testing, which appears less practical in clinical scenarios. To expand the scope of clinical applications, we formulate a novel setting, "OCT-enhanced disease recognition from fundus images", that allows for the use of unpaired multi-modal data during the training phase, and relies on the widespread fundus photographs for testing. To benchmark this setting, we present the first large multi-modal multi-class dataset for eye disease diagnosis, MultiEYE, and propose an OCT-assisted Conceptual Distillation Approach (OCT-CoDA), which employs semantically rich concepts to extract disease-related knowledge from OCT images and leverages them into the fundus model. Specifically, we regard the image-concept relation as a link to distill useful knowledge from OCT teacher model to fundus student model, which considerably improves the diagnostic performance based on fundus images and formulates the cross-modal knowledge transfer into an explainable process. Through extensive experiments on the multi-disease classification task, our proposed OCT-CoDA demonstrates remarkable results and interpretability, showing great potential for clinical application. Our dataset and code are available at https://github.com/xmed-lab/MultiEYE.

Oculomics approaches using retinal imaging to predict mental health disorders: a systematic review and meta-analysis

OBJECTIVE

This meta-analysis evaluated the diagnostic performance of oculomics approaches, including deep learning, machine learning, and logistic regression models, in detecting major mental disorders using retinal imaging.

METHODS

A systematic review identified 11 studies for inclusion. Study quality was assessed using the QUADAS-2 tool, revealing a high risk of bias, particularly in patient selection and index test design. Pooled sensitivity and specificity were estimated using random-effects models, and diagnostic performance was evaluated through a summary receiver operating characteristic curve.

RESULTS

The analysis included 13 diagnostic models across 11 studies, covering major depressive disorder, bipolar disorder, schizophrenia, obsessive-compulsive disorder, and autism spectrum disorder using color fundus photography, and optical coherence tomography (OCT), and OCT angiography. The pooled sensitivity was 0.89 (95% CI: 0.78-0.94), and specificity was 0.87 (95% CI: 0.74-0.95). The pooled area under the curve was 0.904, indicating high diagnostic accuracy. However, all studies exhibited a high risk of bias, primarily due to case-control study designs, lack of external validation, and selection bias in 77% of studies. Some models showed signs of overfitting, likely due to small sample sizes, insufficient validation, or dataset limitations. Additionally, no distinct retinal patterns specific to mental disorders were identified.

CONCLUSION

While oculomics demonstrates potential for detecting mental disorders through retinal imaging, significant methodological limitations, including high bias, overfitting risks, and the absence of disease-specific retinal biomarkers, limit its current clinical applicability. Future research should focus on large-scale, externally validated studies with prospective designs to establish reliable retinal markers for psychiatric diagnosis.

Predicting branch retinal vein occlusion development using multimodal deep learning and pre-onset fundus hemisection images

Branch retinal vein occlusion (BRVO) is a leading cause of visual impairment in working-age individuals, though predicting its occurrence from retinal vascular features alone remains challenging. We developed a deep learning model to predict BRVO based on pre-onset, metadata-matched fundus hemisection images. This retrospective cohort study included patients diagnosed with unilateral BRVO from two Korean tertiary centers (2005-2023), using hemisection fundus images from 27 BRVO-affected eyes paired with 81 unaffected hemisections (27 counter and 54 contralateral) for training. A U-net model segmented retinal optic discs and blood vessels (BVs), dividing them into upper and lower halves labeled for BRVO occurrence. Both unimodal models (using either fundus or BV images) and a BV-enhanced multimodal model were constructed to predict future BRVO. The multimodal model outperformed the unimodal models achieving an area under the receiver operating characteristic curve of 0.76 (95% confidence interval [CI], 0.66-0.83) and accuracy of 68.5% (95% CI 58.9-77.1%), with predictions focusing on arteriovenous crossing regions in the retinal vascular arcade. These findings demonstrate the potential of the BV-enhanced multimodal approach for BRVO prediction and highlight the need for larger, multicenter datasets to improve its clinical utility and predictive accuracy.

Enhanced AMD detection in OCT images using GLCM texture features with Machine Learning and CNN methods

Global blindness is substantially influenced by age-related macular degeneration (AMD). It significantly shortens people's lives and severely impairs their visual acuity. AMD is becoming more common, requiring improved diagnostic and prognostic methods. Treatment efficacy and patient survival rates stand to benefit greatly from these upgrades. To improve AMD diagnosis in preprocessed retinal images, this study uses Grey Level Co-occurrence Matrix (GLCM) features for texture analysis. The selected GLCM features include contrast and dissimilarity. Notably, grayscale pixel values were also integrated into the analysis. Key factors such as contrast, correlation, energy, and homogeneity were identified as the primary focuses of the study. Various supervised machine learning (ML) and CNN techniques were employed on Optical Coherence Tomography (OCT) image datasets. The impact of feature selection on model performance is evaluated by comparing all GLCM features, selected GLCM features, and grayscale pixel features. Models using GSF features showed low accuracy, with OCTID at 23% and Kermany at 54% for BC, and 23% and 53% for CNN. In contrast, GLCM features achieved 98% for OCTID and 73% for Kermany in RF, and 83% and 77% in CNN. SFGLCM features performed the best, achieving 98% for OCTID across both RF and CNN, and 77% for Kermany. Overall, SFGLCM and GLCM features outperformed GSF, improving accuracy, generalization, and reducing overfitting for AMD detection. The Python-based research demonstrates ML's potential in ophthalmology to enhance patient outcomes.

A Comprehensive CNN Model for Age-Related Macular Degeneration Classification Using OCT: Integrating Inception Modules, SE Blocks, and ConvMixer

Background/Objectives: Age-related macular degeneration (AMD) is a significant cause of vision loss in older adults, often progressing without early noticeable symptoms. Deep learning (DL) models, particularly convolutional neural networks (CNNs), demonstrate potential in accurately diagnosing and classifying AMD using medical imaging technologies like optical coherence to-mography (OCT) scans. This study introduces a novel CNN-based DL method for AMD diagnosis, aiming to enhance computational efficiency and classification accuracy. Methods: The proposed method (PM) combines modified Inception modules, Depthwise Squeeze-and-Excitation Blocks, and ConvMixer architecture. Its effectiveness was evaluated on two datasets: a private dataset with 2316 images and the public Noor dataset. Key performance metrics, including accuracy, precision, recall, and F1 score, were calculated to assess the method's diagnostic performance. Results: On the private dataset, the PM achieved outstanding performance: 97.98% accuracy, 97.95% precision, 97.77% recall, and 97.86% F1 score. When tested on the public Noor dataset, the method reached 100% across all evaluation metrics, outperforming existing DL approaches. Conclusions: These results highlight the promising role of AI-based systems in AMD diagnosis, of-fering advanced feature extraction capabilities that can potentially enable early detection and in-tervention, ultimately improving patient care and outcomes. While the proposed model demon-strates promising performance on the datasets tested, the study is limited by the size and diversity of the datasets. Future work will focus on external clinical validation to address these limita-tions.

A Deep Learning Network for Accurate Retinal Multidisease Diagnosis Using Multiview Fusion of En Face and B-Scan Images: A Multicenter Study

Purpose

Accurate diagnosis of retinal disease based on optical coherence tomography (OCT) requires scrutiny of both B-scan and en face images. The aim of this study was to investigate the effectiveness of fusing en face and B-scan images for better diagnostic performance of deep learning models.

Methods

A multiview fusion network (MVFN) with a decision fusion module to integrate fast-axis and slow-axis B-scans and en face information was proposed and compared with five state-of-the-art methods: a model using B-scans, a model using en face imaging, a model using three-dimensional volume, and two other relevant methods. They were evaluated using the OCTA-500 public dataset and a private multicenter dataset with 2330 cases; cases from the first center were used for training and cases from the second center were used for external validation. Performance was assessed by averaged area under the curve (AUC), accuracy, sensitivity, specificity, and precision.

Results

In the private external test set, our MVFN achieved the highest AUC of 0.994, significantly outperforming the other models (P < 0.01). Similarly, for the OCTA-500 public dataset, our proposed method also outperformed the other methods with the highest AUC of 0.976, further demonstrating its effectiveness. Typical cases were demonstrated using activation heatmaps to illustrate the synergy of combining en face and B-scan images.

Conclusions

The fusion of en face and B-scan information is an effective strategy for improving the diagnostic accuracy of deep learning models.

Translational Relevance

Multiview fusion models combining B-scan and en face images demonstrate great potential in improving AI performance for retina disease diagnosis.

Recent advances in the application of artificial intelligence in age-related macular degeneration

Recent advancements in ophthalmology have been driven by the incorporation of artificial intelligence (AI), especially in diagnosing, monitoring treatment and predicting outcomes for age-related macular degeneration (AMD). AMD is a leading cause of irreversible vision loss worldwide, and its increasing prevalence among the ageing population presents a significant challenge for managing the disease. AI holds considerable promise in tackling this issue. This paper provides an overview of the latest developments in AI applications for AMD. However, current limitations include insufficient and unbalanced data, lack of interpretability in models, dependence on data quality and limited generality.

Interpretable multimodal classification for age-related macular degeneration diagnosis

Explainable Artificial Intelligence (XAI) is an emerging machine learning field that has been successful in medical image analysis. Interpretable approaches are able to "unbox" the black-box decisions made by AI systems, aiding medical doctors to justify their diagnostics better. In this paper, we analyze the performance of three different XAI strategies for medical image analysis in ophthalmology. We consider a multimodal deep learning model that combines optical coherence tomography (OCT) and infrared reflectance (IR) imaging for the diagnosis of age-related macular degeneration (AMD). The classification model is able to achieve an accuracy of 0.94, performing better than other unimodal alternatives. We analyze the XAI methods in terms of their ability to identify retinal damage and ease of interpretation, concluding that grad-CAM and guided grad-CAM can be combined to have both a coarse visual justification and a fine-grained analysis of the retinal layers. We provide important insights and recommendations for practitioners on how to design automated and explainable screening tests based on the combination of two image sources.

Multi-modal representation learning in retinal imaging using self-supervised learning for enhanced clinical predictions

Self-supervised learning has become the cornerstone of building generalizable and transferable artificial intelligence systems in medical imaging. In particular, contrastive representation learning techniques trained on large multi-modal datasets have demonstrated impressive capabilities of producing highly transferable representations for different downstream tasks. In ophthalmology, large multi-modal datasets are abundantly available and conveniently accessible as modern retinal imaging scanners acquire both 2D fundus images and 3D optical coherence tomography (OCT) scans to assess the eye. In this context, we introduce a novel multi-modal contrastive learning-based pipeline to facilitate learning joint representations for the two retinal imaging modalities. After self-supervised pre-training on 153,306 scan pairs, we show that such a pre-training framework can provide both a retrieval system and encoders that produce comprehensive OCT and fundus image representations that generalize well for various downstream tasks on three independent external datasets, explicitly focusing on clinically pertinent prediction tasks. In addition, we show that interchanging OCT with lower-cost fundus imaging can preserve the predictive power of the trained models.

Multimodality Fusion Strategies in Eye Disease Diagnosis

Multimodality fusion has gained significance in medical applications, particularly in diagnosing challenging diseases like eye diseases, notably diabetic eye diseases that pose risks of vision loss and blindness. Mono-modality eye disease diagnosis proves difficult, often missing crucial disease indicators. In response, researchers advocate multimodality-based approaches to enhance diagnostics. This study is a unique exploration, evaluating three multimodality fusion strategies-early, joint, and late-in conjunction with state-of-the-art convolutional neural network models for automated eye disease binary detection across three datasets: fundus fluorescein angiography, macula, and combination of digital retinal images for vessel extraction, structured analysis of the retina, and high-resolution fundus. Findings reveal the efficacy of each fusion strategy: type 0 early fusion with DenseNet121 achieves an impressive 99.45% average accuracy. InceptionResNetV2 emerges as the top-performing joint fusion architecture with an average accuracy of 99.58%. Late fusion ResNet50V2 achieves a perfect score of 100% across all metrics, surpassing both early and joint fusion. Comparative analysis demonstrates that late fusion ResNet50V2 matches the accuracy of state-of-the-art feature-level fusion model for multiview learning. In conclusion, this study substantiates late fusion as the optimal strategy for eye disease diagnosis compared to early and joint fusion, showcasing its superiority in leveraging multimodal information.

Prediction of treatment outcome for branch retinal vein occlusion using convolutional neural network-based retinal fluorescein angiography

Deep learning techniques were used in ophthalmology to develop artificial intelligence (AI) models for predicting the short-term effectiveness of anti-VEGF therapy in patients with macular edema secondary to branch retinal vein occlusion (BRVO-ME). 180 BRVO-ME patients underwent pre-treatment FFA scans. After 3 months of ranibizumab injections, CMT measurements were taken at baseline and 1-month intervals. Patients were categorized into good and poor prognosis groups based on macular edema at the 4th month follow-up. FFA-Net, a VGG-based classification network, was trained using FFA images from both groups. Class activation heat maps highlighted important locations. Benchmark models (DesNet-201, MobileNet-V3, ResNet-152, MansNet-75) were compared for training results. Performance metrics included accuracy, sensitivity, specificity, F1 score, and ROC curves. FFA-Net predicted BRVO-ME treatment effect with an accuracy of 88.63% and an F1 score of 0.89, with a sensitivity and specificity of 79.40% and 71.34%, respectively.The AUC of the ROC curve for the FFA-Net model was 0.71. The use of FFA based on deep learning technology has feasibility in predicting the treatment effect of BRVO-ME. The FFA-Net model constructed with the VGG model as the main body has good results in predicting the treatment effect of BRVO-ME. The typing of BRVO in FFA may be an important factor affecting the prognosis.

Confidence-aware multi-modality learning for eye disease screening

Multi-modal ophthalmic image classification plays a key role in diagnosing eye diseases, as it integrates information from different sources to complement their respective performances. However, recent improvements have mainly focused on accuracy, often neglecting the importance of confidence and robustness in predictions for diverse modalities. In this study, we propose a novel multi-modality evidential fusion pipeline for eye disease screening. It provides a measure of confidence for each modality and elegantly integrates the multi-modality information using a multi-distribution fusion perspective. Specifically, our method first utilizes normal inverse gamma prior distributions over pre-trained models to learn both aleatoric and epistemic uncertainty for uni-modality. Then, the normal inverse gamma distribution is analyzed as the Student's t distribution. Furthermore, within a confidence-aware fusion framework, we propose a mixture of Student's t distributions to effectively integrate different modalities, imparting the model with heavy-tailed properties and enhancing its robustness and reliability. More importantly, the confidence-aware multi-modality ranking regularization term induces the model to more reasonably rank the noisy single-modal and fused-modal confidence, leading to improved reliability and accuracy. Experimental results on both public and internal datasets demonstrate that our model excels in robustness, particularly in challenging scenarios involving Gaussian noise and modality missing conditions. Moreover, our model exhibits strong generalization capabilities to out-of-distribution data, underscoring its potential as a promising solution for multimodal eye disease screening.

A hybrid model for the detection of retinal disorders using artificial intelligence techniques

The prevalence of vision impairment is increasing at an alarming rate. The goal of the study was to create an automated method that uses optical coherence tomography (OCT) to classify retinal disorders into four categories: choroidal neovascularization, diabetic macular edema, drusen, and normal cases. This study proposed a new framework that combines machine learning and deep learning-based techniques. The utilized classifiers were support vector machine (SVM), K-nearest neighbor (K-NN), decision tree (DT), and ensemble model (EM). A feature extractor, the InceptionV3 convolutional neural network, was also employed. The performance of the models was evaluated against nine criteria using a dataset of 18000 OCT images. For the SVM, K-NN, DT, and EM classifiers, the analysis exhibited state-of-the-art performance, with classification accuracies of 99.43%, 99.54%, 97.98%, and 99.31%, respectively. A promising methodology has been introduced for the automatic identification and classification of retinal disorders, leading to reduced human error and saved time.

Cross-modal attention network for retinal disease classification based on multi-modal images

Multi-modal eye disease screening improves diagnostic accuracy by providing lesion information from different sources. However, existing multi-modal automatic diagnosis methods tend to focus on the specificity of modalities and ignore the spatial correlation of images. This paper proposes a novel cross-modal retinal disease diagnosis network (CRD-Net) that digs out the relevant features from modal images aided for multiple retinal disease diagnosis. Specifically, our model introduces a cross-modal attention (CMA) module to query and adaptively pay attention to the relevant features of the lesion in the different modal images. In addition, we also propose multiple loss functions to fuse features with modality correlation and train a multi-modal retinal image classification network to achieve a more accurate diagnosis. Experimental evaluation on three publicly available datasets shows that our CRD-Net outperforms existing single-modal and multi-modal methods, demonstrating its superior performance.

Geometric Correspondence-Based Multimodal Learning for Ophthalmic Image Analysis

Color fundus photography (CFP) and Optical coherence tomography (OCT) images are two of the most widely used modalities in the clinical diagnosis and management of retinal diseases. Despite the widespread use of multimodal imaging in clinical practice, few methods for automated diagnosis of eye diseases utilize correlated and complementary information from multiple modalities effectively. This paper explores how to leverage the information from CFP and OCT images to improve the automated diagnosis of retinal diseases. We propose a novel multimodal learning method, named geometric correspondence-based multimodal learning network (GeCoM-Net), to achieve the fusion of CFP and OCT images. Specifically, inspired by clinical observations, we consider the geometric correspondence between the OCT slice and the CFP region to learn the correlated features of the two modalities for robust fusion. Furthermore, we design a new feature selection strategy to extract discriminative OCT representations by automatically selecting the important feature maps from OCT slices. Unlike the existing multimodal learning methods, GeCoM-Net is the first method that formulates the geometric relationships between the OCT slice and the corresponding region of the CFP image explicitly for CFP and OCT fusion. Experiments have been conducted on a large-scale private dataset and a publicly available dataset to evaluate the effectiveness of GeCoM-Net for diagnosing diabetic macular edema (DME), impaired visual acuity (VA) and glaucoma. The empirical results show that our method outperforms the current state-of-the-art multimodal learning methods by improving the AUROC score 0.4%, 1.9% and 2.9% for DME, VA and glaucoma detection, respectively.

Artificial intelligence in age-related macular degeneration: state of the art and recent updates

Age related macular degeneration (AMD) represents a leading cause of vision loss and it is expected to affect 288 million people by 2040. During the last decade, machine learning technologies have shown great potential to revolutionize clinical management of AMD and support research for a better understanding of the disease. The aim of this review is to provide a panoramic description of all the applications of AI to AMD management and screening that have been analyzed in recent past literature. Deep learning (DL) can be effectively used to diagnose AMD, to predict short term risk of exudation and need for injections within the next 2 years. Moreover, DL technology has the potential to customize anti-VEGF treatment choice with a higher accuracy than expert human experts. In addition, accurate prediction of VA response to treatment can be provided to the patients with the use of ML models, which could considerably increase patients' compliance to treatment in favorable cases. Lastly, AI, especially in the form of DL, can effectively predict conversion to GA in 12 months and also suggest new biomarkers of conversion with an innovative reverse engineering approach.

Relative yield of retinal imaging versus clinical exam in following neovascular exudative age related macular degeneration

PURPOSE

To compare therapeutic decisions between 3 diagnostic protocols and to assess the need for in-person physical doctor-patient encounter in follow up and treatment of neovascular exudative age-related macular degeneration (AMD).

METHODS

Analysis of 88 eyes of 88 unique patients with neovascular AMD who were routinely followed at our medical retina clinic. A retinal specialist reviewed all images in advance and wrote his decisions. He later attended an in-person encounters with all patients and documented his decisions. Masking was done by not exposing any identifying information to the specialist and by randomizing patient's images order before the in-person encounter. Therapeutic decisions regarding intravitreal injections intervals and agent selection were made based on three protocols: (1) optic coherence tomography (OCT); (2) OCT/Ultra-widefield (UWF) color image; (3) OCT/UWF/full clinical exam. Visual acuity (VA) was incorporated into all protocols.

RESULTS

We found an agreement of 93% between those protocols regarding the intervals of injections, and of 100% regarding injection agent selection. When comparing OCT, OCT/UWF and OCT/UWF/clinical exam guided decision making, there were no discrepancies between OCT and OCT/UWF. There were 6 out of 88 discrepancies (7%) between OCT/UWF and OCT/UWF/clinical exam. Of those 6 discrepancies, all were regarding intervals (Bland-Altman bias = - 0.2386). All discrepancies between OCT/UWF and OCT/UWF/Clinical exam were due to patients' preferences, socioeconomic issues and fellow eye considerations, addressed during the face-to-face encounter with patients. Physical examination itself did not affect decision making.

CONCLUSIONS

Neovascular exudative AMD follow up and treatment decisions can be guided by VA and OCT, with UWF adding important information regarding macula and peripheral retina, but rarely affecting decision making. However, decision making may also be driven by patients' preferences and other considerations that are being made only during the face-to-face visit and discussion. Thus, every approach supporting imaging only decision making, must take these factors into account.

Comparison of deep learning methods for the radiographic detection of patients with different periodontitis stages

OBJECTIVES

The objective of this study is to assess the accuracy of computer-assisted periodontal classification bone loss staging using deep learning (DL) methods on panoramic radiographs and to compare the performance of various models and layers.

METHODS

Panoramic radiographs were diagnosed and classified into 3 groups, namely "healthy," "Stage1/2," and "Stage3/4," and stored in separate folders. The feature extraction stage involved transferring and retraining the feature extraction layers and weights from 3 models, namely ResNet50, DenseNet121, and InceptionV3, which were proposed for classifying the ImageNet dataset, to 3 DL models designed for classifying periodontal bone loss. The features obtained from global average pooling (GAP), global max pooling (GMP), or flatten layers (FL) of convolutional neural network (CNN) models were used as input to the 8 different machine learning (ML) models. In addition, the features obtained from the GAP, GMP, or FL of the DL models were reduced using the minimum redundancy maximum relevance (mRMR) method and then classified again with 8 ML models.

RESULTS

A total of 2533 panoramic radiographs, including 721 in the healthy group, 842 in the Stage1/2 group, and 970 in the Stage3/4 group, were included in the dataset. The average performance values of DenseNet121 + GAP-based and DenseNet121 + GAP + mRMR-based ML techniques on 10 subdatasets and ML models developed using 2 feature selection techniques outperformed CNN models.

CONCLUSIONS

The new DenseNet121 + GAP + mRMR-based support vector machine model developed in this study achieved higher performance in periodontal bone loss classification compared to other models in the literature by detecting effective features from raw images without the need for manual selection.

Mstnet: method for glaucoma grading based on multimodal feature fusion of spatial relations

Objective.The objective of this study is to develop an efficient multimodal learning framework for the classification of glaucoma. Glaucoma is a group of eye diseases that can result in vision loss and blindness, often due to delayed detection and treatment. Fundus images and optical coherence tomography (OCT) images have proven valuable for the diagnosis and management of glaucoma. However, current models that combine features from both modalities often lack efficient spatial relationship modeling.Approach.In this study, we propose an innovative approach to address the classification of glaucoma. We focus on leveraging the features of OCT volumes and harness the capabilities of transformer models to capture long-range spatial relationships. To achieve this, we introduce a 3D transformer model to extract features from OCT volumes, enhancing the model's effectiveness. Additionally, we employ downsampling techniques to enhance model efficiency. We then utilize the spatial feature relationships between OCT volumes and fundus images to fuse the features extracted from both sources.Main results.Our proposed framework has yielded remarkable results, particularly in terms of glaucoma grading performance. We conducted our experiments using the GAMMA dataset, and our approach outperformed traditional feature fusion methods. By effectively modeling spatial relationships and combining OCT volume and fundus map features, our framework achieved outstanding classification results.Significance.This research is of significant importance in the field of glaucoma diagnosis and management. Efficient and accurate glaucoma classification is essential for timely intervention and prevention of vision loss. Our proposed approach, which integrates 3D transformer models, offers a novel way to extract and fuse features from OCT volumes and fundus images, ultimately enhancing the effectiveness of glaucoma classification. This work has the potential to contribute to improved patient care, particularly in the early detection and treatment of glaucoma, thereby reducing the risk of vision impairment and blindness.

Classification of Color Fundus Photographs Using Fusion Extracted Features and Customized CNN Models

This study focuses on overcoming challenges in classifying eye diseases using color fundus photographs by leveraging deep learning techniques, aiming to enhance early detection and diagnosis accuracy. We utilized a dataset of 6392 color fundus photographs across eight disease categories, which was later augmented to 17,766 images. Five well-known convolutional neural networks (CNNs)-efficientnetb0, mobilenetv2, shufflenet, resnet50, and resnet101-and a custom-built CNN were integrated and trained on this dataset. Image sizes were standardized, and model performance was evaluated via accuracy, Kappa coefficient, and precision metrics. Shufflenet and efficientnetb0demonstrated strong performances, while our custom 17-layer CNN outperformed all with an accuracy of 0.930 and a Kappa coefficient of 0.920. Furthermore, we found that the fusion of image features with classical machine learning classifiers increased the performance, with Logistic Regression showcasing the best results. Our study highlights the potential of AI and deep learning models in accurately classifying eye diseases and demonstrates the efficacy of custom-built models and the fusion of deep learning and classical methods. Future work should focus on validating these methods across larger datasets and assessing their real-world applicability.

Optical Coherence Tomography Image Classification Using Hybrid Deep Learning and Ant Colony Optimization

Optical coherence tomography (OCT) is widely used to detect and classify retinal diseases. However, OCT-image-based manual detection by ophthalmologists is prone to errors and subjectivity. Thus, various automation methods have been proposed; however, improvements in detection accuracy are required. Particularly, automated techniques using deep learning on OCT images are being developed to detect various retinal disorders at an early stage. Here, we propose a deep learning-based automatic method for detecting and classifying retinal diseases using OCT images. The diseases include age-related macular degeneration, branch retinal vein occlusion, central retinal vein occlusion, central serous chorioretinopathy, and diabetic macular edema. The proposed method comprises four main steps: three pretrained models, DenseNet-201, InceptionV3, and ResNet-50, are first modified according to the nature of the dataset, after which the features are extracted via transfer learning. The extracted features are improved, and the best features are selected using ant colony optimization. Finally, the best features are passed to the k-nearest neighbors and support vector machine algorithms for final classification. The proposed method, evaluated using OCT retinal images collected from Soonchunhyang University Bucheon Hospital, demonstrates an accuracy of 99.1% with the incorporation of ACO. Without ACO, the accuracy achieved is 97.4%. Furthermore, the proposed method exhibits state-of-the-art performance and outperforms existing techniques in terms of accuracy.

Towards population-independent, multi-disease detection in fundus photographs

Independent validation studies of automatic diabetic retinopathy screening systems have recently shown a drop of screening performance on external data. Beyond diabetic retinopathy, this study investigates the generalizability of deep learning (DL) algorithms for screening various ocular anomalies in fundus photographs, across heterogeneous populations and imaging protocols. The following datasets are considered: OPHDIAT (France, diabetic population), OphtaMaine (France, general population), RIADD (India, general population) and ODIR (China, general population). Two multi-disease DL algorithms were developed: a Single-Dataset (SD) network, trained on the largest dataset (OPHDIAT), and a Multiple-Dataset (MD) network, trained on multiple datasets simultaneously. To assess their generalizability, both algorithms were evaluated whenever training and test data originate from overlapping datasets or from disjoint datasets. The SD network achieved a mean per-disease area under the receiver operating characteristic curve (mAUC) of 0.9571 on OPHDIAT. However, it generalized poorly to the other three datasets (mAUC < 0.9). When all four datasets were involved in training, the MD network significantly outperformed the SD network (p = 0.0058), indicating improved generality. However, in leave-one-dataset-out experiments, performance of the MD network was significantly lower on populations unseen during training than on populations involved in training (p < 0.0001), indicating imperfect generalizability.

Scale-adaptive model for detection and grading of age-related macular degeneration from color retinal fundus images

Age-related Macular Degeneration (AMD), a retinal disease that affects the macula, can be caused by aging abnormalities in number of different cells and tissues in the retina, retinal pigment epithelium, and choroid, leading to vision loss. An advanced form of AMD, called exudative or wet AMD, is characterized by the ingrowth of abnormal blood vessels beneath or into the macula itself. The diagnosis is confirmed by either fundus auto-fluorescence imaging or optical coherence tomography (OCT) supplemented by fluorescein angiography or OCT angiography without dye. Fluorescein angiography, the gold standard diagnostic procedure for AMD, involves invasive injections of fluorescent dye to highlight retinal vasculature. Meanwhile, patients can be exposed to life-threatening allergic reactions and other risks. This study proposes a scale-adaptive auto-encoder-based model integrated with a deep learning model that can detect AMD early by automatically analyzing the texture patterns in color fundus imaging and correlating them to the vasculature activity in the retina. Moreover, the proposed model can automatically distinguish between AMD grades assisting in early diagnosis and thus allowing for earlier treatment of the patient's condition, slowing the disease and minimizing its severity. Our model features two main blocks, the first is an auto-encoder-based network for scale adaption, and the second is a convolutional neural network (CNN) classification network. Based on a conducted set of experiments, the proposed model achieves higher diagnostic accuracy compared to other models with accuracy, sensitivity, and specificity that reach 96.2%, 96.2%, and 99%, respectively.

Improved Detection of Dry Age-Related Macular Degeneration from Optical Coherence Tomography Images using Adaptive Window Based Feature Extraction and Weighted Ensemble Based Classification Approach

BACKGROUND

Dry Age-related macular degeneration (AMD), which affects the older population, can lead to blindness when left untreated. Preventing vision loss in elderly needs early identification. Dry-AMD diagnosis is still time-consuming and very subjective, depending on the ophthalmologist. Setting up a thorough eye-screening system to find Dry-AMD is a very difficult task.

METHODOLOGY

This study aims to develop a weighted majority voting (WMV) ensemble-based prediction model to diagnose Dry-AMD. The WMV approach combines the predictions from base-classifiers and chooses the class with greatest vote based on assigned weights to each classifier. A novel feature extraction method is used along the retinal pigment epithelium (RPE) layer, with the number of windows calculated for each picture playing an important part in identifying Dry-AMD/normal images using the WMV methodology. Pre-processing using hybrid-median filter followed by scale-invariant feature transform based segmentation of RPE layer and curvature flattening of retina is employed to measure exact thickness of RPE layer.

RESULT

The proposed model is trained on 70% of the OCT image database (OCTID) and evaluated on remaining OCTID and SD-OCT Noor dataset. Model has achieved accuracy of 96.15% and 96.94%, respectively. The suggested algorithm's effectiveness in Dry-AMD identification is demonstrated by comparison with alternative approaches. Even though the suggested model is only trained on the OCTID, it has performed well when tested on additional dataset.

CONCLUSION

The suggested architecture can be used for quick eye-screening for early identification of Dry-AMD. The recommended method may be applied in real-time since it requires fewer complexity and learning-variables.

Effectual accuracy of OCT image retinal segmentation with the aid of speckle noise reduction and boundary edge detection strategy

Optical coherence tomography (OCT) has shown to be a valuable imaging tool in the field of ophthalmology, and it is becoming increasingly relevant in the field of neurology. Several OCT image segmentation methods have been developed previously to segment retinal images, however sophisticated speckle noises with low-intensity restrictions, complex retinal tissues, and inaccurate retinal layer structure remain a challenge to perform effective retinal segmentation. Hence, in this research, complicated speckle noises are removed by using a novel far-flung ratio algorithm in which preprocessing has been done to treat the speckle noise thereby highly decreasing the speckle noise through new similarity and statistical measures. Additionally, a novel haphazard walk and inter-frame flattening algorithms have been presented to tackle the weak object boundaries in OCT images. These algorithms are effective at detecting edges and estimating minimal weighted paths to better diverge, which reduces the time complexity. In addition, the segmentation of OCT images is made simpler by using a novel N-ret layer segmentation approach that executes simultaneous segmentation of various surfaces, ensures unambiguous segmentation across neighbouring layers, and improves segmentation accuracy by using two grey scale values to construct data. Consequently, the novel work outperformed the OCT image segmentation with 98.5% of accuracy.

Development and validation of a deep learning system to classify aetiology and predict anatomical outcomes of macular hole

AIMS

To develop a deep learning (DL) model for automatic classification of macular hole (MH) aetiology (idiopathic or secondary), and a multimodal deep fusion network (MDFN) model for reliable prediction of MH status (closed or open) at 1 month after vitrectomy and internal limiting membrane peeling (VILMP).

METHODS

In this multicentre retrospective cohort study, a total of 330 MH eyes with 1082 optical coherence tomography (OCT) images and 3300 clinical data enrolled from four ophthalmic centres were used to train, validate and externally test the DL and MDFN models. 266 eyes from three centres were randomly split by eye-level into a training set (80%) and a validation set (20%). In the external testing dataset, 64 eyes were included from the remaining centre. All eyes underwent macular OCT scanning at baseline and 1 month after VILMP. The area under the receiver operated characteristic curve (AUC), accuracy, specificity and sensitivity were used to evaluate the performance of the models.

RESULTS

In the external testing set, the AUC, accuracy, specificity and sensitivity of the MH aetiology classification model were 0.965, 0.950, 0.870 and 0.938, respectively; the AUC, accuracy, specificity and sensitivity of the postoperative MH status prediction model were 0.904, 0.825, 0.977 and 0.766, respectively; the AUC, accuracy, specificity and sensitivity of the postoperative idiopathic MH status prediction model were 0.947, 0.875, 0.815 and 0.979, respectively.

CONCLUSION

Our DL-based models can accurately classify the MH aetiology and predict the MH status after VILMP. These models would help ophthalmologists in diagnosis and surgical planning of MH.

CLASSIFICATION OF AGE-RELATED MACULAR DEGENERATION USING DAG-CNN ARCHITECTURE

Age-related Macular Degeneration (AMD) is the prime reason for vision impairment observed in major countries worldwide. Hence an accurate early detection of the disease is vital for more research in this area. Also, having a thorough eye diagnosis to detect AMD is a complex job. This paper introduces a Directed Acyclic Graph (DAG) structure-based Convolutional Neural network (CNN) architecture to better classify Dry or Wet AMD. The DAG architecture can combine features from multiple layers to provide better results. The DAG model also has the capacity to learn multi-level visual properties to increase classification accuracy. Fine tuning of DAG-based CNN model helps in improving the performance of the network. The training and testing of the proposed model are carried out with the Mendeley data set and achieved an accuracy of 99.2% with an AUC value of 0.9999. The proposed model also obtains better results for other parameters such as precision, recall and F1-score. Performance of the proposed network is also compared to that of the related works performed on the same data set. This shows ability of the proposed method to grade AMD images to help early detection of the disease. The model also performs computationally efficient for real-time applications as it does the classification process with few learnable parameters and fewer Floating-Point Operations (FLOPs).

Asynchronous feature regularization and cross-modal distillation for OCT based glaucoma diagnosis

Glaucoma has become a major cause of vision loss. Early-stage diagnosis of glaucoma is critical for treatment planning to avoid irreversible vision damage. Meanwhile, interpreting the rapidly accumulated medical data from ophthalmic exams is cumbersome and resource-intensive. Therefore, automated methods are highly desired to assist ophthalmologists in achieving fast and accurate glaucoma diagnosis. Deep learning has achieved great successes in diagnosing glaucoma by analyzing data from different kinds of tests, such as peripapillary optical coherence tomography (OCT) and visual field (VF) testing. Nevertheless, applying these developed models to clinical practice is still challenging because of various limiting factors. OCT models present worse glaucoma diagnosis performances compared to those achieved by OCT&VF based models, whereas VF is time-consuming and highly variable, which can restrict the wide employment of OCT&VF models. To this end, we develop a novel deep learning framework that leverages the OCT&VF model to enhance the performance of the OCT model. To transfer the complementary knowledge from the structural and functional assessments to the OCT model, a cross-modal knowledge transfer method is designed by integrating a designed distillation loss and a proposed asynchronous feature regularization (AFR) module. We demonstrate the effectiveness of the proposed method for glaucoma diagnosis by utilizing a public OCT&VF dataset and evaluating it on an external OCT dataset. Our final model with only OCT inputs achieves the accuracy of 87.4% (3.1% absolute improvement) and AUC of 92.3%, which are on par with the OCT&VF joint model. Moreover, results on the external dataset sufficiently indicate the effectiveness and generalization capability of our model.

Generative adversarial network-based deep learning approach in classification of retinal conditions with optical coherence tomography images

PURPOSE

To determine whether a deep learning approach using generative adversarial networks (GANs) is beneficial for the classification of retinal conditions with Optical coherence tomography (OCT) images.

METHODS

Our study utilized 84,452 retinal OCT images obtained from a publicly available dataset (Kermany Dataset). Employing GAN, synthetic OCT images are produced to balance classes of retinal disorders. A deep learning classification model is constructed using pretrained deep neural networks (DNNs), and outcomes are evaluated using 2082 images collected from patients who visited the Department of Ophthalmology and the Department of Endocrinology and Metabolism at the Tri-service General Hospital in Taipei from January 2017 to December 2021.

RESULTS

The highest classification accuracies accomplished by deep learning machines trained on the unbalanced dataset for its training set, validation set, fivefold cross validation (CV), Kermany test set, and TSGH test set were 97.73%, 96.51%, 97.14%, 99.59%, and 81.03%, respectively. The highest classification accuracies accomplished by deep learning machines trained on the synthesis-balanced dataset for its training set, validation set, fivefold CV, Kermany test set, and TSGH test set were 98.60%, 98.41%, 98.52%, 99.38%, and 84.92%, respectively. In comparing the highest accuracies, deep learning machines trained on the synthesis-balanced dataset outperformed deep learning machines trained on the unbalanced dataset for the training set, validation set, fivefold CV, and TSGH test set.

CONCLUSIONS

Overall, deep learning machines on a synthesis-balanced dataset demonstrated to be advantageous over deep learning machines trained on an unbalanced dataset for the classification of retinal conditions.

Trustworthy AI: Closing the gap between development and integration of AI systems in ophthalmic practice

An increasing number of artificial intelligence (AI) systems are being proposed in ophthalmology, motivated by the variety and amount of clinical and imaging data, as well as their potential benefits at the different stages of patient care. Despite achieving close or even superior performance to that of experts, there is a critical gap between development and integration of AI systems in ophthalmic practice. This work focuses on the importance of trustworthy AI to close that gap. We identify the main aspects or challenges that need to be considered along the AI design pipeline so as to generate systems that meet the requirements to be deemed trustworthy, including those concerning accuracy, resiliency, reliability, safety, and accountability. We elaborate on mechanisms and considerations to address those aspects or challenges, and define the roles and responsibilities of the different stakeholders involved in AI for ophthalmic care, i.e., AI developers, reading centers, healthcare providers, healthcare institutions, ophthalmological societies and working groups or committees, patients, regulatory bodies, and payers. Generating trustworthy AI is not a responsibility of a sole stakeholder. There is an impending necessity for a collaborative approach where the different stakeholders are represented along the AI design pipeline, from the definition of the intended use to post-market surveillance after regulatory approval. This work contributes to establish such multi-stakeholder interaction and the main action points to be taken so that the potential benefits of AI reach real-world ophthalmic settings.

Deep learning: applications in retinal and optic nerve diseases

Deep learning (DL) represents a paradigm-shifting, burgeoning field of research with emerging clinical applications in optometry. Unlike traditional programming, which relies on human-set specific rules, DL works by exposing the algorithm to a large amount of annotated data and allowing the software to develop its own set of rules (i.e. learn) by adjusting the parameters inside the model (network) during a training process in order to complete the task on its own. One major limitation of traditional programming is that, with complex tasks, it may require an extensive set of rules to accurately complete the assignment. Additionally, traditional programming can be susceptible to human bias from programmer experience. With the dramatic increase in the amount and the complexity of clinical data, DL has been utilised to automate data analysis and thus to assist clinicians in patient management. This review will present the latest advances in DL, for managing posterior eye diseases as well as DL-based solutions for patients with vision loss.

HCTNet: A Hybrid ConvNet-Transformer Network for Retinal Optical Coherence Tomography Image Classification

Automatic and accurate optical coherence tomography (OCT) image classification is of great significance to computer-assisted diagnosis of retinal disease. In this study, we propose a hybrid ConvNet-Transformer network (HCTNet) and verify the feasibility of a Transformer-based method for retinal OCT image classification. The HCTNet first utilizes a low-level feature extraction module based on the residual dense block to generate low-level features for facilitating the network training. Then, two parallel branches of the Transformer and the ConvNet are designed to exploit the global and local context of the OCT images. Finally, a feature fusion module based on an adaptive re-weighting mechanism is employed to combine the extracted global and local features for predicting the category of OCT images in the testing datasets. The HCTNet combines the advantage of the convolutional neural network in extracting local features and the advantage of the vision Transformer in establishing long-range dependencies. A verification on two public retinal OCT datasets shows that our HCTNet method achieves an overall accuracy of 91.56% and 86.18%, respectively, outperforming the pure ViT and several ConvNet-based classification methods.

Automatic Screening of the Eyes in a Deep-Learning–Based Ensemble Model Using Actual Eye Checkup Optical Coherence Tomography Images

Eye checkups have become increasingly important to maintain good vision and quality of life. As the population requiring eye checkups increases, so does the clinical work burden of clinicians. An automatic screening algorithm to reduce the clinicians’ workload is necessary. Machine learning (ML) has recently become one of the chief techniques for automated image recognition and is a helpful tool for identifying ocular diseases. However, the accuracy of ML models is lower in a clinical setting than in the laboratory. The performance of ML models depends on the training dataset. Eye checkups often prioritize speed and minimize image processing. Data distribution differs from the training dataset and, consequently, decreases prediction performance. The study aim was to investigate an ML model to screen for retinal diseases from low-quality optical coherence tomography (OCT) images captured during actual eye chechups to prevent a dataset shift. The ensemble model with convolutional neural networks (CNNs) and random forest models showed high screening performance in the single-shot OCT images captured during the actual eye checkups. Our study indicates the strong potential of the ensemble model combining the CNN and random forest models in accurately predicting abnormalities during eye checkups.

Machine learning predicting myopic regression after corneal refractive surgery using preoperative data and fundus photography

PURPOSE

Myopic regression after surgery is the most common long-term complication of refractive surgery, but it is difficult to identify myopic regression without long-term observation. This study aimed to develop machine learning models to identify high-risk patients for refractive regression based on preoperative data and fundus photography.

METHODS

This retrospective study assigned subjects to the training (n = 1606 eyes) and validation (n = 403 eyes) datasets with chronological data splitting. Machine learning models with ResNet50 (for image analysis) and XGBoost (for integration of all variables and fundus photography) were developed based on subjects who underwent corneal refractive surgery. The primary outcome was the predictive performance for the presence of myopic regression at 4 years of follow-up examination postoperatively.

RESULTS

By integrating all factors and fundus photography, the final combined machine learning model showed good performance to predict myopic regression of more than 0.5 D (area under the receiver operating characteristic curve [ROC-AUC], 0.753; 95% confidence interval [CI], 0.710-0.793). The performance of the final model was better than the single ResNet50 model only using fundus photography (ROC-AUC, 0.673; 95% CI, 0.627-0.716). The top-five most important input features were fundus photography, preoperative anterior chamber depth, planned ablation thickness, age, and preoperative central corneal thickness.

CONCLUSION

Our machine learning algorithm provides an efficient strategy to identify high-risk patients with myopic regression without additional labor, cost, and time. Surgeons might benefit from preoperative risk assessment of myopic regression, patient counseling before surgery, and surgical option decisions.

FN-OCT: Disease Detection Algorithm for Retinal Optical Coherence Tomography Based on a Fusion Network

Optical coherence tomography (OCT) is a new type of tomography that has experienced rapid development and potential in recent years. It is playing an increasingly important role in retinopathy diagnoses. At present, due to the uneven distributions of medical resources in various regions, the uneven proficiency levels of doctors in grassroots and remote areas, and the development needs of rare disease diagnosis and precision medicine, artificial intelligence technology based on deep learning can provide fast, accurate, and effective solutions for the recognition and diagnosis of retinal OCT images. To prevent vision damage and blindness caused by the delayed discovery of retinopathy, a fusion network (FN)-based retinal OCT classification algorithm (FN-OCT) is proposed in this paper to improve upon the adaptability and accuracy of traditional classification algorithms. The InceptionV3, Inception-ResNet, and Xception deep learning algorithms are used as base classifiers, a convolutional block attention mechanism (CBAM) is added after each base classifier, and three different fusion strategies are used to merge the prediction results of the base classifiers to output the final prediction results (choroidal neovascularization (CNV), diabetic macular oedema (DME), drusen, normal). The results show that in a classification problem involving the UCSD common retinal OCT dataset (108,312 OCT images from 4,686 patients), compared with that of the InceptionV3 network model, the prediction accuracy of FN-OCT is improved by 5.3% (accuracy = 98.7%, area under the curve (AUC) = 99.1%). The predictive accuracy and AUC achieved on an external dataset for the classification of retinal OCT diseases are 92 and 94.5%, respectively, and gradient-weighted class activation mapping (Grad-CAM) is used as a visualization tool to verify the effectiveness of the proposed FNs. This finding indicates that the developed fusion algorithm can significantly improve the performance of classifiers while providing a powerful tool and theoretical support for assisting with the diagnosis of retinal OCT.

Learning Two-Stream CNN for Multi-Modal Age-related Macular Degeneration Categorization

This paper tackles automated categorization of Age-related Macular Degeneration (AMD), a common macular disease among people over 50. Previous research efforts mainly focus on AMD categorization with a single-modal input, let it be a color fundus photograph (CFP) or an OCT B-scan image. By contrast, we consider AMD categorization given a multi-modal input, a direction that is clinically meaningful yet mostly unexplored. Contrary to the prior art that takes a traditional approach of feature extraction plus classifier training that cannot be jointly optimized, we opt for end-to-end multi-modal Convolutional Neural Networks (MM-CNN). Our MM-CNN is instantiated by a two-stream CNN, with spatially-invariant fusion to combine information from the CFP and OCT streams. In order to visually interpret the contribution of the individual modalities to the final prediction, we extend the class activation mapping (CAM) technique to the multi-modal scenario. For effective training of MM-CNN, we develop two data augmentation methods. One is GAN-based CFP/OCT image synthesis, with our novel use of CAMs as conditional input of a high-resolution image-to-image translation GAN. The other method is Loose Pairing, which pairs a CFP image and an OCT image on the basis of their classes instead of eye identities. Experiments on a clinical dataset consisting of 1,094 CFP images and 1,289 OCT images acquired from 1,093 distinct eyes show that the proposed solution obtains better F1 and Accuracy than multiple baselines for multi-modal AMD categorization. Code and data are available at https://github.com/li-xirong/mmc-amd.

Diagnostic accuracy of current machine learning classifiers for age-related macular degeneration: a systematic review and meta-analysis

BACKGROUND AND OBJECTIVE

The objective of this study was to systematically review and meta-analyze the diagnostic accuracy of current machine learning classifiers for age-related macular degeneration (AMD). Artificial intelligence diagnostic algorithms can automatically detect and diagnose AMD through training data from large sets of fundus or OCT images. The use of AI algorithms is a powerful tool, and it is a method of obtaining a cost-effective, simple, and fast diagnosis of AMD.

METHODS

MEDLINE, EMBASE, CINAHL, and ProQuest Dissertations and Theses were searched systematically and thoroughly. Conferences held through Association for Research in Vision and Ophthalmology, American Academy of Ophthalmology, and Canadian Society of Ophthalmology were searched. Studies were screened using Covidence software and data on sensitivity, specificity and area under curve were extracted from the included studies. STATA 15.0 was used to conduct the meta-analysis.

RESULTS

Our search strategy identified 307 records from online databases and 174 records from gray literature. Total of 13 records, 64,798 subjects (and 612,429 images), were used for the quantitative analysis. The pooled estimate for sensitivity was 0.918 [95% CI: 0.678, 0.98] and specificity was 0.888 [95% CI: 0.578, 0.98] for AMD screening using machine learning classifiers. The relative odds of a positive screen test in AMD cases were 89.74 [95% CI: 3.05-2641.59] times more likely than a negative screen test in non-AMD cases. The positive likelihood ratio was 8.22 [95% CI: 1.52-44.48] and the negative likelihood ratio was 0.09 [95% CI: 0.02-0.52].

CONCLUSION

The included studies show promising results for the diagnostic accuracy of the machine learning classifiers for AMD and its implementation in clinical settings.

Clinical explainable differential diagnosis of polypoidal choroidal vasculopathy and age-related macular degeneration using deep learning

BACKGROUND

This study aims to achieve an automatic differential diagnosis between two types of retinal pathologies with similar pathological features - Polypoidal choroidal vasculopathy (PCV) and wet age-related macular degeneration (AMD) from volumetric optical coherence tomography (OCT) images, and identify clinically-relevant pathological features, using an explainable deep-learning-based framework.

METHODS

This is a retrospective study with data from a cross-sectional cohort. The OCT volume of 73 eyes from 59 patients was included in this study. Disease differentiation was achieved through single-B-scan-based classification followed by a volumetric probability prediction aggregation step. We compared different labeling strategies with and without identifying pathological B-scans within each OCT volume. Clinical interpretability was achieved through normalized aggregation of B-scan-based saliency maps followed by maximum-intensity-projection onto the en face plane. We derived the PCV score from the proposed differential diagnosis framework with different labeling strategies. The en face projection of saliency map was validated with the pathologies identified in Indocyanine green angiography (ICGA).

RESULTS

Model trained with both labeling strategies achieved similar level differentiation power (>90%), with good correspondence between pathological features detected from the projected en face saliency map and ICGA.

CONCLUSIONS

This study demonstrated the potential clinical application of non-invasive differential diagnosis using AI-driven OCT-based analysis, with minimal requirement of labeling efforts, along with clinical explainability achieved through automatically detected disease-related pathologies.

SD-OCT Biomarkers and the Current Status of Artificial Intelligence in Predicting Progression from Intermediate to Advanced AMD

Age-related macular degeneration (AMD) is one of the leading causes of blindness in the Western World. Optical coherence tomography (OCT) has revolutionized the diagnosis and follow-up of AMD patients. This review focuses on SD-OCT imaging biomarkers which were identified as predictors for progression in intermediate AMD to late AMD, either geographic atrophy (GA) or choroidal neovascularization (CNV). Structural OCT remains the most compelling modality to study AMD features related to the progression such as drusen characteristics, hyperreflective foci (HRF), reticular pseudo-drusen (RPD), sub-RPE hyper-reflective columns and their impact on retinal layers. Further on, we reviewed articles that attempted to integrate biomarkers that have already proven their involvement in intermediate AMD progression, in their models of artificial intelligence (AI). By combining structural biomarkers with genetic risk and lifestyle the predictive ability becomes more accurate.

Multimodal deep learning with feature level fusion for identification of choroidal neovascularization activity in age-related macular degeneration

PURPOSE

This study aimed to determine the efficacy of a multimodal deep learning (DL) model using optical coherence tomography (OCT) and optical coherence tomography angiography (OCTA) images for the assessment of choroidal neovascularization (CNV) in neovascular age-related macular degeneration (AMD).

METHODS

This retrospective and cross-sectional study was performed at a multicentre, and the inclusion criteria were age >50 years and a diagnosis of typical neovascular AMD. The OCT and OCTA data for an internal data set and two external data sets were collected. A DL model was developed with a novel feature-level fusion (FLF) method utilized to combine the multimodal data. The results were compared with identification performed by an ophthalmologist. The best model was tested on two external data sets to show its potential for clinical use.

RESULTS

Our best model achieved an accuracy of 95.5% and an area under the curve (AUC) of 0.9796 on multimodal data inputs for the internal data set, which is comparable to the performance of retinal specialists. The proposed model reached an accuracy of 100.00% and an AUC of 1.0 for the Ningbo data set, and these performance indicators were 90.48% and an AUC of 0.9727 for the Jinhua data set.

CONCLUSION

The FLF method is feasible and highly accurate, and could enhance the power of the existing computer-aided diagnosis systems. The bi-modal computer-aided diagnosis (CADx) system for the automated identification of CNV activity is an accurate and promising tool in the realm of public health.

Automatic Screening for Ocular Anomalies Using Fundus Photographs

SIGNIFICANCE

Screening for ocular anomalies using fundus photography is key to prevent vision impairment and blindness. With the growing and aging population, automated algorithms that can triage fundus photographs and provide instant referral decisions are relevant to scale-up screening and face the shortage of ophthalmic expertise.

PURPOSE

This study aimed to develop a deep learning algorithm that detects any ocular anomaly in fundus photographs and to evaluate this algorithm for "normal versus anomalous" eye examination classification in the diabetic and general populations.

METHODS

The deep learning algorithm was developed and evaluated in two populations: the diabetic and general populations. Our patient cohorts consist of 37,129 diabetic patients from the OPHDIAT diabetic retinopathy screening network in Paris, France, and 7356 general patients from the OphtaMaine private screening network, in Le Mans, France. Each data set was divided into a development subset and a test subset of more than 4000 examinations each. For ophthalmologist/algorithm comparison, a subset of 2014 examinations from the OphtaMaine test subset was labeled by a second ophthalmologist. First, the algorithm was trained on the OPHDIAT development subset. Then, it was fine-tuned on the OphtaMaine development subset.

RESULTS

On the OPHDIAT test subset, the area under the receiver operating characteristic curve for normal versus anomalous classification was 0.9592. On the OphtaMaine test subset, the area under the receiver operating characteristic curve was 0.8347 before fine-tuning and 0.9108 after fine-tuning. On the ophthalmologist/algorithm comparison subset, the second ophthalmologist achieved a specificity of 0.8648 and a sensitivity of 0.6682. For the same specificity, the fine-tuned algorithm achieved a sensitivity of 0.8248.

CONCLUSIONS

The proposed algorithm compares favorably with human performance for normal versus anomalous eye examination classification using fundus photography. Artificial intelligence, which previously targeted a few retinal pathologies, can be used to screen for ocular anomalies comprehensively.

Automatic Detection of Age-Related Macular Degeneration Based on Deep Learning and Local Outlier Factor Algorithm

Age-related macular degeneration (AMD) is a retinal disorder affecting the elderly, and society’s aging population means that the disease is becoming increasingly prevalent. The vision in patients with early AMD is usually unaffected or nearly normal but central vision may be weakened or even lost if timely treatment is not performed. Therefore, early diagnosis is particularly important to prevent the further exacerbation of AMD. This paper proposed a novel automatic detection method of AMD from optical coherence tomography (OCT) images based on deep learning and a local outlier factor (LOF) algorithm. A ResNet-50 model with L2-constrained softmax loss was retrained to extract features from OCT images and the LOF algorithm was used as the classifier. The proposed method was trained on the UCSD dataset and tested on both the UCSD dataset and Duke dataset, with an accuracy of 99.87% and 97.56%, respectively. Even though the model was only trained on the UCSD dataset, it obtained good detection accuracy when tested on another dataset. Comparison with other methods also indicates the efficiency of the proposed method in detecting AMD.