RETOUCH: The Retinal OCT Fluid Detection and Segmentation Benchmark and Challenge

Joint semi-supervised and contrastive learning enables domain generalization and multi-domain segmentation

Despite their effectiveness, current deep learning models face challenges with images coming from different domains with varying appearance and content. We introduce SegCLR, a versatile framework designed to segment images across different domains, employing supervised and contrastive learning simultaneously to effectively learn from both labeled and unlabeled data. We demonstrate the superior performance of SegCLR through a comprehensive evaluation involving three diverse clinical datasets of 3D retinal Optical Coherence Tomography (OCT) images, for the slice-wise segmentation of fluids with various network configurations and verification across 10 different network initializations. In an unsupervised domain adaptation context, SegCLR achieves results on par with a supervised upper-bound model trained on the intended target domain. Notably, we discover that the segmentation performance of SegCLR framework is marginally impacted by the abundance of unlabeled data from the target domain, thereby we also propose an effective domain generalization extension of SegCLR, known also as zero-shot domain adaptation, which eliminates the need for any target domain information. This shows that our proposed addition of contrastive loss in standard supervised training for segmentation leads to superior models, inherently more generalizable to both in- and out-of-domain test data. We additionally propose a pragmatic solution for SegCLR deployment in realistic scenarios with multiple domains containing labeled data. Accordingly, our framework pushes the boundaries of deep-learning based segmentation in multi-domain applications, regardless of data availability - labeled, unlabeled, or nonexistent.

Deep learning model for detecting cystoid fluid collections on optical coherence tomography in X-linked retinoschisis patients

PURPOSE

To validate a deep learning (DL) framework for detecting and quantifying cystoid fluid collections (CFC) on spectral-domain optical coherence tomography (SD-OCT) in X-linked retinoschisis (XLRS) patients.

METHODS

A no-new-U-Net model was trained using 112 OCT volumes from the RETOUCH challenge (70 for training and 42 for internal testing). External validation involved 37 SD-OCT scans from 20 XLRS patients, including 20 randomly sampled B-scans and 17 manually selected central B-scans. Three graders manually delineated the CFC on these B-scans in this external test set. The model's efficacy was evaluated using Dice and intraclass correlation coefficient (ICC) scores, assessed exclusively on the test set comprising B-scans from XLRS patients.

RESULTS

For the randomly sampled B-scans, the model achieved a mean Dice score of 0.886 (±0.010), compared to 0.912 (±0.014) for the observers. For the manually selected central B-scans, the Dice scores were 0.936 (±0.012) for the model and 0.946 (±0.012) for the graders. ICC scores between the model and reference were 0.945 (±0.014) for the randomly selected and 0.964 (±0.011) for the manually selected B-scans. Among the graders, ICC scores were 0.979 (±0.008) and 0.981 (±0.011), respectively.

CONCLUSIONS

Our validated DL model accurately segments and quantifies CFC on SD-OCT in XLRS, paving the way for reliable monitoring of structural changes. However, systematic overestimation by the DL model was observed, highlighting a key limitation for future refinement.

Retinal OCT image segmentation with deep learning: A review of advances, datasets, and evaluation metrics

Optical coherence tomography (OCT) is a widely used imaging technology in ophthalmic clinical practice, providing non-invasive access to high-resolution retinal images. Segmentation of anatomical structures and pathological lesions in retinal OCT images, directly impacts clinical decisions. While commercial OCT devices segment multiple retinal layers in healthy eyes, their performance degrades severely under pathological conditions. In recent years, the rapid advancements in deep learning have significantly driven research in OCT image segmentation. This review provides a comprehensive overview of the latest developments in deep learning-based segmentation methods for retinal OCT images. Additionally, it summarizes the medical significance, publicly available datasets, and commonly used evaluation metrics in this field. The review also discusses the current challenges faced by the research community and highlights potential future directions.

Joint segmentation of retinal layers and fluid lesions in optical coherence tomography with cross-dataset learning

BACKGROUND AND OBJECTIVES

Age-related macular degeneration (AMD) is the leading cause of irreversible vision loss among people over 50 years old, which manifests in the retina through various changes of retinal layers and pathological lesions. The accurate segmentation of optical coherence tomography (OCT) image features is crucial for the identification and tracking of AMD. Although the recent developments in deep neural network have brought profound progress in this area, accurately segmenting retinal layers and pathological lesions remains a challenging task because of the interaction between these two tasks.

METHODS

In this study, we propose a three-branch, hierarchical multi-task framework that enables joint segmentation of seven retinal layers and three types of pathological lesions. A regression guidance module is introduced to provide explicit shape guidance between sub-tasks. We also propose a cross-dataset learning strategy to leverage public datasets with partial labels. The proposed framework was evaluated on a clinical dataset consisting of 140 OCT B-scans with pixel-level annotations of seven retinal layers and three types of lesions. Additionally, we compared its performance with the state-of-the-art methods on two public datasets.

RESULTS

Comprehensive ablation showed that the proposed hierarchical architecture significantly improved performance for most retinal layers and pathological lesions, achieving the highest mean DSC of 76.88 %. The IRF also achieved the best performance with a DSC of 68.15 %. Comparative studies demonstrated that the hierarchical multi-task architecture could significantly enhance segmentation accuracy and outperform state-of-the-art methods.

CONCLUSION

The proposed framework could also be generalized to other medical image segmentation tasks with interdependent relationships.

SD-LayerNet: Robust and label-efficient retinal layer segmentation via anatomical priors

BACKGROUND AND OBJECTIVES

Automated, anatomically coherent retinal layer segmentation in optical coherence tomography (OCT) is one of the most important components of retinal disease management. However, current methods rely on large amounts of labeled data, which can be difficult and expensive to obtain. In addition, these systems tend often propose anatomically impossible results, which undermines their clinical reliability.

METHODS

This study introduces a semi-supervised approach to retinal layer segmentation that leverages large amounts of unlabeled data and anatomical prior knowledge related to the structure of the retina. During training, we use a novel topological engine that converts inferred retinal layer boundaries into pixel-wise structured segmentations. These compose a set of anatomically valid disentangled representations which, together with predicted style factors, are used to reconstruct the input image. At training time, the retinal layer boundaries and pixel-wise predictions are both guided by reference annotations, where available, but more importantly by innovatively exploiting anatomical priors that improve the performance, robustness and coherence of the method even if only a small amount of labeled data is available.

RESULTS

Exhaustive experiments with respect to label efficiency, contribution of unsupervised data and robustness to different acquisition settings were conducted. The proposed method showed state of-the-art performance on all the studied public and internal datasets, specially in low annotated data regimes. Additionally, the model was able to make use of unlabeled data from a different domain with only a small performance drop in comparison to a fully-supervised setting.

CONCLUSION

A novel, robust, label-efficient retinal layer segmentation method was proposed. The approach has shown state-of-the-art layer segmentation performance with a fraction of the training data available, while at the same time, its robustness against domain shift was also shown.

Anomaly Detection in Retinal OCT Images With Deep Learning-Based Knowledge Distillation

Purpose

The purpose of this study was to develop a robust and general purpose artificial intelligence (AI) system that allows the identification of retinal optical coherence tomography (OCT) volumes with pathomorphological manifestations not present in normal eyes in screening programs and large retrospective studies.

Methods

An unsupervised anomaly detection deep learning approach for the screening of retinal OCTs with any pathomorphological manifestations via Teacher-Student knowledge distillation is developed. The system is trained with only normal cases without any additional manual labeling. At test time, it scores how anomalous a sample is and produces localized anomaly maps with regions of interest in a B-scan. Fovea-centered OCT scans acquired with Spectralis (Heidelberg Engineering) were considered. A total of 3358 patients were used for development and testing. The detection performance was evaluated in a large data cohort with different pathologies including diabetic macular edema (DME) and the multiple stages of age-related macular degeneration (AMD) and on external public datasets with various disease biomarkers.

Results

The volume-wise anomaly detection receiver operating characteristic (ROC) area under the curve (AUC) was 0.94 ± 0.05 in the test set. Pathological B-scan detection on external datasets varied between 0.81 and 0.87 AUC. Qualitatively, the derived anomaly maps pointed toward diagnostically relevant regions. The behavior of the system across the datasets was similar and consistent.

Conclusions

Anomaly detection constitutes a valid complement to supervised systems aimed at improving the success of vision preservation and eye care, and is an important step toward more efficient and generalizable screening tools.

Translational Relevance

Deep learning approaches can enable an automated and objective screening of a wide range of pathological retinal conditions that deviate from normal appearance.

Enhancing AI reliability: A foundation model with uncertainty estimation for optical coherence tomography-based retinal disease diagnosis

Inability to express the confidence level and detect unseen disease classes limits the clinical implementation of artificial intelligence in the real world. We develop a foundation model with uncertainty estimation (FMUE) to detect 16 retinal conditions on optical coherence tomography (OCT). In the internal test set, FMUE achieves a higher F1 score of 95.74% than other state-of-the-art algorithms (92.03%-93.66%) and improves to 97.44% with threshold strategy. The model achieves similar excellent performance on two external test sets from the same and different OCT machines. In human-model comparison, FMUE achieves a higher F1 score of 96.30% than retinal experts (86.95%, p = 0.004), senior doctors (82.71%, p < 0.001), junior doctors (66.55%, p < 0.001), and generative pretrained transformer 4 with vision (GPT-4V) (32.39%, p < 0.001). Besides, FMUE predicts high uncertainty scores for >85% images of non-target-category diseases or with low quality to prompt manual checks and prevent misdiagnosis. Our FMUE provides a trustworthy method for automatic retinal anomaly detection in a clinical open-set environment.

An Efficient Retinal Fluid Segmentation Network Based on Large Receptive Field Context Capture for Optical Coherence Tomography Images

Optical Coherence Tomography (OCT) is a crucial imaging modality for diagnosing and monitoring retinal diseases. However, the accurate segmentation of fluid regions and lesions remains challenging due to noise, low contrast, and blurred edges in OCT images. Although feature modeling with wide or global receptive fields offers a feasible solution, it typically leads to significant computational overhead. To address these challenges, we propose LKMU-Lite, a lightweight U-shaped segmentation method tailored for retinal fluid segmentation. LKMU-Lite integrates a Decoupled Large Kernel Attention (DLKA) module that captures both local patterns and long-range dependencies, thereby enhancing feature representation. Additionally, it incorporates a Multi-scale Group Perception (MSGP) module that employs Dilated Convolutions with varying receptive field scales to effectively predict lesions of different shapes and sizes. Furthermore, a novel Aggregating-Shift decoder is proposed, reducing model complexity while preserving feature integrity. With only 1.02 million parameters and a computational complexity of 3.82 G FLOPs, LKMU-Lite achieves state-of-the-art performance across multiple metrics on the ICF and RETOUCH datasets, demonstrating both its efficiency and generalizability compared to existing methods.

Artificial intelligence-based analysis of retinal fluid volume dynamics in neovascular age-related macular degeneration and association with vision and atrophy

BACKGROUND/OBJECTIVES

To characterise morphological changes in neovascular age-related macular degeneration (nAMD) during anti-angiogenic therapy and explore relationships with best-corrected visual acuity (BCVA) and development of macular atrophy (MA).

SUBJECTS/METHODS

Post-hoc analysis of the phase III HARBOR trial. SD-OCT scans from 1097 treatment-naïve nAMD eyes were analysed. Volumes of intraretinal cystoid fluid (ICF), subretinal hyperreflective material (SHRM), subretinal fluid (SRF), pigment epithelial detachment (PED) and cyst-free retinal volume (CFRV) were measured by deep-learning model. Volumes were analysed by treatment regimen, macular neovascularisation (MNV) subtypes and topographic location. Associations of volumetric features with BCVA and MA development were quantified at month 12/24.

RESULTS

Differences in feature volume changes by treatment regimens and MNV subtypes were observed. Each additional 100 nanolitre unit (AHNU) of residual ICF, SHRM and CFRV at month 1 in the fovea was associated with deficits of 10.3, 7.3 and 12.2 letters at month 12. Baseline AHNUs of ICF, CFRV and PED were associated with increased odds of MA development at month 12 by 10%, 4% and 3%. While that of SRF was associated with a decrease in odds of 5%. Associations at month 24 were similar to those at month 12.

CONCLUSION

Eyes with different MNV subtypes showed distinct trajectories of feature volume response to treatment. Higher baseline volumes of ICF or PED and lower baseline volume of SRF were associated with higher likelihoods of MA development over 24 months. Residual intraretinal fluid, including ICF and CFRV, along with SHRM were predictors of poor visual outcomes.

Revolutionizing healthcare: a comparative insight into deep learning's role in medical imaging

Recently, Deep Learning (DL) models have shown promising accuracy in analysis of medical images. Alzeheimer Disease (AD), a prevalent form of dementia, uses Magnetic Resonance Imaging (MRI) scans, which is then analysed via DL models. To address the model computational constraints, Cloud Computing (CC) is integrated to operate with the DL models. Recent articles on DL-based MRI have not discussed datasets specific to different diseases, which makes it difficult to build the specific DL model. Thus, the article systematically explores a tutorial approach, where we first discuss a classification taxonomy of medical imaging datasets. Next, we present a case-study on AD MRI classification using the DL methods. We analyse three distinct models-Convolutional Neural Networks (CNN), Visual Geometry Group 16 (VGG-16), and an ensemble approach-for classification and predictive outcomes. In addition, we designed a novel framework that offers insight into how various layers interact with the dataset. Our architecture comprises an input layer, a cloud-based layer responsible for preprocessing and model execution, and a diagnostic layer that issues alerts after successful classification and prediction. According to our simulations, CNN outperformed other models with a test accuracy of 99.285%, followed by VGG-16 with 85.113%, while the ensemble model lagged with a disappointing test accuracy of 79.192%. Our cloud Computing framework serves as an efficient mechanism for medical image processing while safeguarding patient confidentiality and data privacy.

Fully automatic deep convolutional approaches for the screening of neurodegeneratives diseases using multi-view OCT images

The prevalence of neurodegenerative diseases (NDDs) such as Alzheimer's (AD), Parkinson's (PD), Essential tremor (ET), and Multiple Sclerosis (MS) is increasing alongside the aging population. Recent studies suggest that these disorders can be identified through retinal imaging, allowing for early detection and monitoring via Optical Coherence Tomography (OCT) scans. This study is at the forefront of research, pioneering the application of multi-view OCT and 3D information to the neurological diseases domain. Our methodology consists of two main steps. In the first one, we focus on the segmentation of the retinal nerve fiber layer (RNFL) and a class layer grouping between the ganglion cell layer and Bruch's membrane (GCL-BM) in both macular and optic disc OCT scans. These are the areas where changes in thickness serve as a potential indicator of NDDs. The second phase is to select patients based on information about the retinal layers. We explore how the integration of both views (macula and optic disc) improves each screening scenario: Healthy Controls (HC) vs. NDD, AD vs. NDD, ET vs. NDD, MS vs. NDD, PD vs. NDD, and a final multi-class approach considering all four NDDs. For the segmentation task, we obtained satisfactory results for both 2D and 3D approaches in macular segmentation, in which 3D performed better due to the inclusion of depth and cross-sectional information. As for the optic disc view, transfer learning did not improve the metrics over training from scratch, but it did provide a faster training. As for screening, 3D computational biomarkers provided better results than 2D ones, and multi-view methods were usually better than the single-view ones. Regarding separability among diseases, MS and PD were the ones that provided better results in their screening approaches, being also the most represented classes. In conclusion, our methodology has been successfully validated with an extensive experimentation of configurations, techniques and OCT views, becoming the first multi-view analysis that merges data from both macula-centered and optic disc-centered perspectives. Besides, it is also the first effort to examine key retinal layers across four major NDDs within the framework of pathological screening.

Harmonizing Unets: Attention Fusion module in cascaded-Unets for low-quality OCT image fluid segmentation

Optical coherence tomography (OCT) is widely used for its high resolution. Accurate OCT image segmentation can significantly improve the diagnosis and treatment of retinal diseases such as Diabetic Macular Edema (DME). However, in resource-limited regions, portable devices with low-quality output are more frequently used, severely affecting the performance of segmentation. To address this issue, we propose a novel methodology in this paper, including a dedicated pre-processing pipeline and an end-to-end double U-shaped cascaded architecture, H-Unets. In addition, an Adaptive Attention Fusion (AAF) module is elaborately designed to improve the segmentation performance of H-Unets. To demonstrate the effectiveness of our method, we conduct a bunch of ablation and comparative studies on three open-source datasets. The experimental results show the validity of the pre-processing pipeline and H-Unets, achieving the highest Dice score of 90.60%±0.87% among popular methods in a relatively small model size.

Semi-supervised 3D retinal fluid segmentation via correlation mutual learning with global reasoning attention

Accurate 3D segmentation of fluid lesions in optical coherence tomography (OCT) is crucial for the early diagnosis of diabetic macular edema (DME). However, higher-dimensional spatial complexity and limited annotated data present significant challenges for effective 3D lesion segmentation. To address these issues, we propose a novel semi-supervised strategy using a correlation mutual learning framework for segmenting 3D DME lesions from 3D OCT images. Our method integrates three key innovations: (1) a shared encoder with three parallel, slightly different decoders, exhibiting cognitive biases and calculating statistical discrepancies among the decoders to represent uncertainty in unlabeled challenging regions. (2) a global reasoning attention module integrated into the encoder's output to transfer label prior knowledge to unlabeled data; and (3) a correlation mutual learning scheme, enforcing mutual consistency between one decoder's probability map and the soft pseudo labels generated by the other decoders. Extensive experiments demonstrate that our approach outperforms state-of-the-art (SOTA) methods, highlighting the potential of our framework for tackling the complex task of 3D retinal lesion segmentation.

Quantifying Changes on OCT in Eyes Receiving Treatment for Neovascular Age-Related Macular Degeneration

Purpose

Application of artificial intelligence (AI) to macular OCT scans to segment and quantify volumetric change in anatomical and pathological features during intravitreal treatment for neovascular age-related macular degeneration (AMD).

Design

Retrospective analysis of OCT images from the Moorfields Eye Hospital AMD Database.

Participants

A total of 2115 eyes from 1801 patients starting anti-VEGF treatment between June 1, 2012, and June 30, 2017.

Methods

The Moorfields Eye Hospital neovascular AMD database was queried for first and second eyes receiving anti-VEGF treatment and had an OCT scan at baseline and 12 months. Follow-up scans were input into the AI system and volumes of OCT variables were studied at different time points and compared with baseline volume groups. Cross-sectional comparisons between time points were conducted using Mann-Whitney U test.

Main Outcome Measures

Volume outputs of the following variables were studied: intraretinal fluid, subretinal fluid, pigment epithelial detachment (PED), subretinal hyperreflective material (SHRM), hyperreflective foci, neurosensory retina, and retinal pigment epithelium.

Results

Mean volumes of analyzed features decreased significantly from baseline to both 4 and 12 months, in both first-treated and second-treated eyes. Pathological features that reflect exudation, including pure fluid components (intraretinal fluid and subretinal fluid) and those with fluid and fibrovascular tissue (PED and SHRM), displayed similar responses to treatment over 12 months. Mean PED and SHRM volumes showed less pronounced but also substantial decreases over the first 2 months, reaching a plateau postloading phase, and minimal change to 12 months. Both neurosensory retina and retinal pigment epithelium volumes showed gradual reductions over time, and were not as substantial as exudative features.

Conclusions

We report the results of a quantitative analysis of change in retinal segmented features over time, enabled by an AI segmentation system. Cross-sectional analysis at multiple time points demonstrated significant associations between baseline OCT-derived segmented features and the volume of biomarkers at follow-up. Demonstrating how certain OCT biomarkers progress with treatment and the impact of pretreatment retinal morphology on different structural volumes may provide novel insights into disease mechanisms and aid the personalization of care. Data will be made public for future studies.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Precision Segmentation of Subretinal Fluids in OCT Using Multiscale Attention-Based U-Net Architecture

This paper presents a deep-learning architecture for segmenting retinal fluids in patients with Diabetic Macular Oedema (DME) and Age-related Macular Degeneration (AMD). Accurate segmentation of multiple fluid types is critical for diagnosis and treatment planning, but existing techniques often struggle with precision. We propose an encoder-decoder network inspired by U-Net, processing enhanced OCT images and their edge maps. The encoder incorporates Residual and Inception modules with an autoencoder-based multiscale attention mechanism to extract detailed features. Our method shows superior performance across several datasets. On the RETOUCH dataset, the network achieved F1 Scores of 0.82 for intraretinal fluid (IRF), 0.93 for subretinal fluid (SRF), and 0.94 for pigment epithelial detachment (PED). The model also performed well on the OPTIMA and DUKE datasets, demonstrating high precision, recall, and F1 Scores. This architecture significantly enhances segmentation accuracy and edge precision, offering a valuable tool for diagnosing and managing retinal diseases. Its integration of dual-input processing, multiscale attention, and advanced encoder modules highlights its potential to improve clinical outcomes and advance retinal disease treatment.

The STOIC2021 COVID-19 AI challenge: Applying reusable training methodologies to private data

Challenges drive the state-of-the-art of automated medical image analysis. The quantity of public training data that they provide can limit the performance of their solutions. Public access to the training methodology for these solutions remains absent. This study implements the Type Three (T3) challenge format, which allows for training solutions on private data and guarantees reusable training methodologies. With T3, challenge organizers train a codebase provided by the participants on sequestered training data. T3 was implemented in the STOIC2021 challenge, with the goal of predicting from a computed tomography (CT) scan whether subjects had a severe COVID-19 infection, defined as intubation or death within one month. STOIC2021 consisted of a Qualification phase, where participants developed challenge solutions using 2000 publicly available CT scans, and a Final phase, where participants submitted their training methodologies with which solutions were trained on CT scans of 9724 subjects. The organizers successfully trained six of the eight Final phase submissions. The submitted codebases for training and running inference were released publicly. The winning solution obtained an area under the receiver operating characteristic curve for discerning between severe and non-severe COVID-19 of 0.815. The Final phase solutions of all finalists improved upon their Qualification phase solutions.

AMD-SD: An Optical Coherence Tomography Image Dataset for wet AMD Lesions Segmentation

Wet Age-related Macular Degeneration (wet AMD) is a common ophthalmic disease that significantly impacts patients' vision. Optical coherence tomography (OCT) examination has been widely utilized for diagnosing, treating, and monitoring wet AMD due to its cost-effectiveness, non-invasiveness, and repeatability, positioning it as the most valuable tool for diagnosis and tracking. OCT can provide clear visualization of retinal layers and precise segmentation of lesion areas, facilitating the identification and quantitative analysis of abnormalities. However, the lack of high-quality datasets for assessing wet AMD has impeded the advancement of related algorithms. To address this issue, we have curated a comprehensive wet AMD OCT Segmentation Dataset (AMD-SD), comprising 3049 B-scan images from 138 patients, each annotated with five segmentation labels: subretinal fluid, intraretinal fluid, ellipsoid zone continuity, subretinal hyperreflective material, and pigment epithelial detachment. This dataset presents a valuable opportunity to investigate the accuracy and reliability of various segmentation algorithms for wet AMD, offering essential data support for developing AI-assisted clinical applications targeting wet AMD.

Generalizable Deep Learning for the Detection of Incomplete and Complete Retinal Pigment Epithelium and Outer Retinal Atrophy: A MACUSTAR Report

Purpose

The purpose of this study was to develop a deep learning algorithm for detecting and quantifying incomplete retinal pigment epithelium and outer retinal atrophy (iRORA) and complete retinal pigment epithelium and outer retinal atrophy (cRORA) in optical coherence tomography (OCT) that generalizes well to data from different devices and to validate in an intermediate age-related macular degeneration (iAMD) cohort.

Methods

The algorithm comprised a domain adaptation (DA) model, promoting generalization across devices, and a segmentation model for detecting granular biomarkers defining iRORA/cRORA, which are combined into iRORA/cRORA segmentations. Manual annotations of iRORA/cRORA in OCTs from different devices in the MACUSTAR study (168 patients with iAMD) were compared to the algorithm's output. Eye level classification metrics included sensitivity, specificity, and quadratic weighted Cohen's κ score (κw). Segmentation performance was assessed quantitatively using Bland-Altman plots and qualitatively.

Results

For ZEISS OCTs, sensitivity and specificity for iRORA/cRORA classification were 38.5% and 93.1%, respectively, and 60.0% and 96.4% for cRORA. For Spectralis OCTs, these were 84.0% and 93.7% for iRORA/cRORA, and 62.5% and 97.4% for cRORA. The κw scores for 3-way classification (none, iRORA, and cRORA) were 0.37 and 0.73 for ZEISS and Spectralis, respectively. Removing DA reduced κw from 0.73 to 0.63 for Spectralis.

Conclusions

The DA-enabled iRORA/cRORA segmentation algorithm showed superior consistency compared to human annotations, and good generalization across OCT devices.

Translational Relevance

The application of this algorithm may help toward precise and automated tracking of iAMD-related lesion changes, which is crucial in clinical settings and multicenter longitudinal studies on iAMD.

Dual-Spatial Domain Generalization for Fundus Lesion Segmentation in Unseen Manufacturer's OCT Images

OBJECTIVE

Optical Coherence Tomography (OCT) images can provide non-invasive visualization of fundus lesions; however, scanners from different OCT manufacturers largely vary from each other, which often leads to model deterioration to unseen OCT scanners due to domain shift.

METHODS

To produce the T-styles of the potential target domain, an Orthogonal Style Space Reparameterization (OSSR) method is proposed to apply orthogonal constraints in the latent orthogonal style space to the sampled marginal styles. To leverage the high-level features of multi-source domains and potential T-styles in the graph semantic space, a Graph Adversarial Network (GAN) is constructed to align the generated samples with the source domain samples. To align features with the same label based on the semantic feature in the graph semantic space, Graph Semantic Alignment (GSA) is performed to focus on the shape and the morphological differences between the lesions and their surrounding regions.

RESULTS

Comprehensive experiments have been performed on two OCT image datasets. Compared to state-of-the-art methods, the proposed method can achieve better segmentation.

CONCLUSION

The proposed fundus lesion segmentation method can be trained with labeled OCT images from multiple manufacturers' scanners and be tested on an unseen manufacturer's scanner with better domain generalization.

SIGNIFICANCE

The proposed method can be used in routine clinical occasions when an unseen manufacturer's OCT image is available for a patient.

Exploring Publicly Accessible Optical Coherence Tomography Datasets: A Comprehensive Overview

Artificial intelligence has transformed medical diagnostic capabilities, particularly through medical image analysis. AI algorithms perform well in detecting abnormalities with a strong performance, enabling computer-aided diagnosis by analyzing the extensive amounts of patient data. The data serve as a foundation upon which algorithms learn and make predictions. Thus, the importance of data cannot be underestimated, and clinically corresponding datasets are required. Many researchers face a lack of medical data due to limited access, privacy concerns, or the absence of available annotations. One of the most widely used diagnostic tools in ophthalmology is Optical Coherence Tomography (OCT). Addressing the data availability issue is crucial for enhancing AI applications in the field of OCT diagnostics. This review aims to provide a comprehensive analysis of all publicly accessible retinal OCT datasets. Our main objective is to compile a list of OCT datasets and their properties, which can serve as an accessible reference, facilitating data curation for medical image analysis tasks. For this review, we searched through the Zenodo repository, Mendeley Data repository, MEDLINE database, and Google Dataset search engine. We systematically evaluated all the identified datasets and found 23 open-access datasets containing OCT images, which significantly vary in terms of size, scope, and ground-truth labels. Our findings indicate the need for improvement in data-sharing practices and standardized documentation. Enhancing the availability and quality of OCT datasets will support the development of AI algorithms and ultimately improve diagnostic capabilities in ophthalmology. By providing a comprehensive list of accessible OCT datasets, this review aims to facilitate better utilization and development of AI in medical image analysis.

A Comprehensive Review of AI Diagnosis Strategies for Age-Related Macular Degeneration (AMD)

The rapid advancement of computational infrastructure has led to unprecedented growth in machine learning, deep learning, and computer vision, fundamentally transforming the analysis of retinal images. By utilizing a wide array of visual cues extracted from retinal fundus images, sophisticated artificial intelligence models have been developed to diagnose various retinal disorders. This paper concentrates on the detection of Age-Related Macular Degeneration (AMD), a significant retinal condition, by offering an exhaustive examination of recent machine learning and deep learning methodologies. Additionally, it discusses potential obstacles and constraints associated with implementing this technology in the field of ophthalmology. Through a systematic review, this research aims to assess the efficacy of machine learning and deep learning techniques in discerning AMD from different modalities as they have shown promise in the field of AMD and retinal disorders diagnosis. Organized around prevalent datasets and imaging techniques, the paper initially outlines assessment criteria, image preprocessing methodologies, and learning frameworks before conducting a thorough investigation of diverse approaches for AMD detection. Drawing insights from the analysis of more than 30 selected studies, the conclusion underscores current research trajectories, major challenges, and future prospects in AMD diagnosis, providing a valuable resource for both scholars and practitioners in the domain.

An automated process for bulk downloading optical coherence tomography scans

OBJECTIVE

To develop an automated method for efficiently downloading a large number of optical coherence tomography (OCT) scans obtained using the Heidelberg Spectralis (Heidelberg Engineering, Heidelberg, Germany) platform.

METHODS

The electronic medical records and OCT scans were extracted for all patients with age-related macular degeneration treated at the Hadassah University Hospital Retina Clinic between 2010 and 2021. A macro was created using Visual Basic for Applications (VBA) and Microsoft Excel to automate the export process and anonymize the OCT scans in accordance with hospital policy. OCT scans were extracted as proprietary Heidelberg E2E files.

RESULTS

The VBA macro was used to export a total of 94,789 E2E files from 2807 patient records, with an average processing time of 4.32 min per volume scan (SD: 3.57 min). The entire export process took a total of approximately 202 h to complete over a period of 24 days. In a smaller sample, using the macro to download the scans was significantly faster than manually downloading the scans, averaging 3.88 vs. 11.08 min/file, respectively (t = 8.59, p < 0.001). Finally, we found that exporting the files during both off-clinic and working hours resulted in significantly faster processing times compared to exporting the files solely during working hours (t = 5.77, p < 0.001).

CONCLUSIONS

This study demonstrates the feasibility of using VBA and Excel to automate the process for bulk downloading data from a specific medical imaging platform. The specific steps and techniques will likely vary depending on the software used and hospital constraints and should be determined for each application.

Exploration on OCT biomarker candidate related to macular edema caused by diabetic retinopathy and retinal vein occlusion in SD-OCT images

To improve the understanding of potential pathological mechanisms of macular edema (ME), we try to discover biomarker candidates related to ME caused by diabetic retinopathy (DR) and retinal vein occlusion (RVO) in spectral-domain optical coherence tomography images by means of deep learning (DL). 32 eyes of 26 subjects with non-proliferative DR (NPDR), 77 eyes of 61 subjects with proliferative DR (PDR), 120 eyes of 116 subjects with branch RVO (BRVO), and 17 eyes of 15 subjects with central RVO (CRVO) were collected. A DL model was implemented to guide biomarker candidate discovery. The disorganization of the retinal outer layers (DROL), i.e., the gray value of the retinal tissues between the external limiting membrane (ELM) and retinal pigment epithelium (RPE), the disrupted and obscured rate of the ELM, ellipsoid zone (EZ), and RPE, was measured. In addition, the occurrence, number, volume, and projected area of hyperreflective foci (HRF) were recorded. ELM, EZ, and RPE are more likely to be obscured in RVO group and HRFs are observed more frequently in DR group (all P ≤ 0.001). In conclusion, the features of DROL and HRF can be possible biomarkers related to ME caused by DR and RVO in OCT modality.

All you need is data preparation: A systematic review of image harmonization techniques in Multi-center/device studies for medical support systems

BACKGROUND AND OBJECTIVES

Artificial intelligence (AI) models trained on multi-centric and multi-device studies can provide more robust insights and research findings compared to single-center studies. However, variability in acquisition protocols and equipment can introduce inconsistencies that hamper the effective pooling of multi-source datasets. This systematic review evaluates strategies for image harmonization, which standardizes appearances to enable reliable AI analysis of multi-source medical imaging.

METHODS

A literature search using PRISMA guidelines was conducted to identify relevant papers published between 2013 and 2023 analyzing multi-centric and multi-device medical imaging studies that utilized image harmonization approaches.

RESULTS

Common image harmonization techniques included grayscale normalization (improving classification accuracy by up to 24.42 %), resampling (increasing the percentage of robust radiomics features from 59.5 % to 89.25 %), and color normalization (enhancing AUC by up to 0.25 in external test sets). Initially, mathematical and statistical methods dominated, but machine and deep learning adoption has risen recently. Color imaging modalities like digital pathology and dermatology have remained prominent application areas, though harmonization efforts have expanded to diverse fields including radiology, nuclear medicine, and ultrasound imaging. In all the modalities covered by this review, image harmonization improved AI performance, with increasing of up to 24.42 % in classification accuracy and 47 % in segmentation Dice scores.

CONCLUSIONS

Continued progress in image harmonization represents a promising strategy for advancing healthcare by enabling large-scale, reliable analysis of integrated multi-source datasets using AI. Standardizing imaging data across clinical settings can help realize personalized, evidence-based care supported by data-driven technologies while mitigating biases associated with specific populations or acquisition protocols.

A Systematic Prospective Comparison of Fluid Volume Evaluation across OCT Devices Used in Clinical Practice

Objective

Treatment decisions in neovascular age-related macular degeneration (nAMD) are mainly based on subjective evaluation of OCT. The purpose of this cross-sectional study was to provide a comparison of qualitative and quantitative differences between OCT devices in a systematic manner.

Design

Prospective, cross-sectional study.

Subjects

One hundred sixty OCT volumes, 40 eyes of 40 patients with nAMD.

Methods

Patients from clinical practice were imaged with 4 different OCT devices during one visit: (1) Spectralis Heidelberg; (2) Cirrus; (3) Topcon Maestro2; and (4) Topcon Triton. Intraretinal fluid (IRF), subretinal fluid (SRF), and pigment epithelial detachment (PED) were manually annotated in all cubes by trained human experts to establish fluid measurements based on expert-reader annotations. Intraretinal fluid, SRF, and PED volume were quantified in nanoliters (nL). Bland-Altman plots were created to analyze the agreement of measurements in the central 1 and 6 mm. The Friedman test was performed to test for significant differences in the central 1, 3, and 6 mm.

Main Outcome Measures

Intraretinal fluid, SRF, and PED volume.

Results

In the central 6 mm, there was a trend toward higher IRF and PED volumes in Spectralis images compared with the other devices and no differences in SRF volume. In the central 1 mm, the standard deviation of the differences ranged from ± 3 nL to ± 6 nL for IRF, from ± 3 nL to ± 4 nL for SRF, and from ± 7 nL to ± 10 nL for PED in all pairwise comparisons. Manually annotated IRF and SRF volumes showed no significant differences in the central 1 mm.

Conclusions

Fluid volume quantification achieved excellent reliability in all 3 retinal compartments on images obtained from 4 OCT devices, particularly for clinically relevant IRF and SRF values. Although fluid volume quantification is reliable in all 4 OCT devices, switching OCT devices might lead to deviating fluid volume measurements with higher agreement in the central 1 mm compared with the central 6 mm, with highest agreement for SRF volume in the central 1 mm. Understanding device-dependent differences is essential for expanding the interpretation and implementation of pixel-wise fluid volume measurements in clinical practice and in clinical trials.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Semantic uncertainty Guided Cross-Transformer for enhanced macular edema segmentation in OCT images

Macular edema, a prevalent ocular complication observed in various retinal diseases, can lead to significant vision loss or blindness, necessitating accurate and timely diagnosis. Despite the potential of deep learning for segmentation of macular edema, challenges persist in accurately identifying lesion boundaries, especially in low-contrast and noisy regions, and in distinguishing between Inner Retinal Fluid (IRF), Sub-Retinal Fluid (SRF), and Pigment Epithelial Detachment (PED) lesions. To address these challenges, we present a novel approach, termed Semantic Uncertainty Guided Cross-Transformer Network (SuGCTNet), for the simultaneous segmentation of multi-class macular edema. Our proposed method comprises two key components, the semantic uncertainty guided attention module (SuGAM) and the Cross-Transformer module (CTM). The SuGAM module utilizes semantic uncertainty to allocate additional attention to regions with semantic ambiguity, improves the segmentation performance of these challenging areas. On the other hand, the CTM module capitalizes on both uncertainty information and multi-scale image features to enhance the overall continuity of the segmentation process, effectively minimizing feature confusion among different lesion types. Rigorous evaluation on public datasets and various OCT imaging device data demonstrates the superior performance of our proposed method compared to state-of-the-art approaches, highlighting its potential as a valuable tool for improving the accuracy and reproducibility of macular edema segmentation in clinical settings, and ultimately aiding in the early detection and diagnosis of macular edema-related diseases and associated retinal conditions.

DeepPyramid+: medical image segmentation using Pyramid View Fusion and Deformable Pyramid Reception

PURPOSE

Semantic segmentation plays a pivotal role in many applications related to medical image and video analysis. However, designing a neural network architecture for medical image and surgical video segmentation is challenging due to the diverse features of relevant classes, including heterogeneity, deformability, transparency, blunt boundaries, and various distortions. We propose a network architecture, DeepPyramid+, which addresses diverse challenges encountered in medical image and surgical video segmentation.

METHODS

The proposed DeepPyramid+ incorporates two major modules, namely "Pyramid View Fusion" (PVF) and "Deformable Pyramid Reception" (DPR), to address the outlined challenges. PVF replicates a deduction process within the neural network, aligning with the human visual system, thereby enhancing the representation of relative information at each pixel position. Complementarily, DPR introduces shape- and scale-adaptive feature extraction techniques using dilated deformable convolutions, enhancing accuracy and robustness in handling heterogeneous classes and deformable shapes.

RESULTS

Extensive experiments conducted on diverse datasets, including endometriosis videos, MRI images, OCT scans, and cataract and laparoscopy videos, demonstrate the effectiveness of DeepPyramid+ in handling various challenges such as shape and scale variation, reflection, and blur degradation. DeepPyramid+ demonstrates significant improvements in segmentation performance, achieving up to a 3.65% increase in Dice coefficient for intra-domain segmentation and up to a 17% increase in Dice coefficient for cross-domain segmentation.

CONCLUSIONS

DeepPyramid+ consistently outperforms state-of-the-art networks across diverse modalities considering different backbone networks, showcasing its versatility. Accordingly, DeepPyramid+ emerges as a robust and effective solution, successfully overcoming the intricate challenges associated with relevant content segmentation in medical images and surgical videos. Its consistent performance and adaptability indicate its potential to enhance precision in computerized medical image and surgical video analysis applications.

OCTDL: Optical Coherence Tomography Dataset for Image-Based Deep Learning Methods

Optical coherence tomography (OCT) is a non-invasive imaging technique with extensive clinical applications in ophthalmology. OCT enables the visualization of the retinal layers, playing a vital role in the early detection and monitoring of retinal diseases. OCT uses the principle of light wave interference to create detailed images of the retinal microstructures, making it a valuable tool for diagnosing ocular conditions. This work presents an open-access OCT dataset (OCTDL) comprising over 2000 OCT images labeled according to disease group and retinal pathology. The dataset consists of OCT records of patients with Age-related Macular Degeneration (AMD), Diabetic Macular Edema (DME), Epiretinal Membrane (ERM), Retinal Artery Occlusion (RAO), Retinal Vein Occlusion (RVO), and Vitreomacular Interface Disease (VID). The images were acquired with an Optovue Avanti RTVue XR using raster scanning protocols with dynamic scan length and image resolution. Each retinal b-scan was acquired by centering on the fovea and interpreted and cataloged by an experienced retinal specialist. In this work, we applied Deep Learning classification techniques to this new open-access dataset.

FNeXter: A Multi-Scale Feature Fusion Network Based on ConvNeXt and Transformer for Retinal OCT Fluid Segmentation

The accurate segmentation and quantification of retinal fluid in Optical Coherence Tomography (OCT) images are crucial for the diagnosis and treatment of ophthalmic diseases such as age-related macular degeneration. However, the accurate segmentation of retinal fluid is challenging due to significant variations in the size, position, and shape of fluid, as well as their complex, curved boundaries. To address these challenges, we propose a novel multi-scale feature fusion attention network (FNeXter), based on ConvNeXt and Transformer, for OCT fluid segmentation. In FNeXter, we introduce a novel global multi-scale hybrid encoder module that integrates ConvNeXt, Transformer, and region-aware spatial attention. This module can capture long-range dependencies and non-local similarities while also focusing on local features. Moreover, this module possesses the spatial region-aware capabilities, enabling it to adaptively focus on the lesions regions. Additionally, we propose a novel self-adaptive multi-scale feature fusion attention module to enhance the skip connections between the encoder and the decoder. The inclusion of this module elevates the model's capacity to learn global features and multi-scale contextual information effectively. Finally, we conduct comprehensive experiments to evaluate the performance of the proposed FNeXter. Experimental results demonstrate that our proposed approach outperforms other state-of-the-art methods in the task of fluid segmentation.

Anomaly guided segmentation: Introducing semantic context for lesion segmentation in retinal OCT using weak context supervision from anomaly detection

Automated lesion detection in retinal optical coherence tomography (OCT) scans has shown promise for several clinical applications, including diagnosis, monitoring and guidance of treatment decisions. However, segmentation models still struggle to achieve the desired results for some complex lesions or datasets that commonly occur in real-world, e.g. due to variability of lesion phenotypes, image quality or disease appearance. While several techniques have been proposed to improve them, one line of research that has not yet been investigated is the incorporation of additional semantic context through the application of anomaly detection models. In this study we experimentally show that incorporating weak anomaly labels to standard segmentation models consistently improves lesion segmentation results. This can be done relatively easy by detecting anomalies with a separate model and then adding these output masks as an extra class for training the segmentation model. This provides additional semantic context without requiring extra manual labels. We empirically validated this strategy using two in-house and two publicly available retinal OCT datasets for multiple lesion targets, demonstrating the potential of this generic anomaly guided segmentation approach to be used as an extra tool for improving lesion detection models.

OCTA-500: A retinal dataset for optical coherence tomography angiography study

Optical coherence tomography angiography (OCTA) is a novel imaging modality that has been widely utilized in ophthalmology and neuroscience studies to observe retinal vessels and microvascular systems. However, publicly available OCTA datasets remain scarce. In this paper, we introduce the largest and most comprehensive OCTA dataset dubbed OCTA-500, which contains OCTA imaging under two fields of view (FOVs) from 500 subjects. The dataset provides rich images and annotations including two modalities (OCT/OCTA volumes), six types of projections, four types of text labels (age/gender/eye/disease) and seven types of segmentation labels (large vessel/capillary/artery/vein/2D FAZ/3D FAZ/retinal layers). Then, we propose a multi-object segmentation task called CAVF, which integrates capillary segmentation, artery segmentation, vein segmentation, and FAZ segmentation under a unified framework. In addition, we optimize the 3D-to-2D image projection network (IPN) to IPN-V2 to serve as one of the segmentation baselines. Experimental results demonstrate that IPN-V2 achieves an about 10% mIoU improvement over IPN on CAVF task. Finally, we further study the impact of several dataset characteristics: the training set size, the model input (OCT/OCTA, 3D volume/2D projection), the baseline networks, and the diseases. The dataset and code are publicly available at: https://ieee-dataport.org/open-access/octa-500.

Analysis of OCT Scanning Parameters in AMD and RVO

Optical coherence tomography (OCT) is an extensively used imaging tool for disease monitoring in both age-related macular degeneration (AMD) and retinal vein occlusion (RVO). However, there is limited literature on minimum requirements of OCT settings for reliable biomarker detection. This study systematically investigates both the influence of scan size and interscan distance (ISD) on disease activity detection. We analyzed 80 OCT volumes of AMD patients and 12 OCT volumes of RVO patients for the presence of subretinal fluid (SRF), intraretinal fluid (IRF), and pigment epithelium detachment (PED). All volume scans had a scan size of 6 × 6 mm and an ISD of 125 µm. We analyzed both general fluid distribution and how biomarker detection sensitivity decreases when reducing scan size or density. We found that in AMD patients, all fluids were nearly normally distributed, with most occurrences in the foveal center and concentric decrease towards the periphery. When reducing the scan size to 3 × 3 and 2 × 2 mm, disease activity detection was still high (0.98 and 0.96). Increasing ISD only slightly can already compromise biomarker detection sensitivity (0.9 for 250 µm ISD against 125 µm ISD).

Cross-attention learning enables real-time nonuniform rotational distortion correction in OCT

Nonuniform rotational distortion (NURD) correction is vital for endoscopic optical coherence tomography (OCT) imaging and its functional extensions, such as angiography and elastography. Current NURD correction methods require time-consuming feature tracking/registration or cross-correlation calculations and thus sacrifice temporal resolution. Here we propose a cross-attention learning method for the NURD correction in OCT. Our method is inspired by the recent success of the self-attention mechanism in natural language processing and computer vision. By leveraging its ability to model long-range dependencies, we can directly obtain the spatial correlation between OCT A-lines at any distance, thus accelerating the NURD correction. We develop an end-to-end stacked cross-attention network and design three types of optimization constraints. We compare our method with two traditional feature-based methods and a CNN-based method on two publicly-available endoscopic OCT datasets. We further verify the NURD correction performance of our method on 3D stent reconstruction using a home-built endoscopic OCT system. Our method achieves a ∼3 × speedup to real time (26 ± 3 fps), and superior correction performance.

Deep learning in optical coherence tomography: Where are the gaps?

Optical coherence tomography (OCT) is a non-invasive optical imaging modality, which provides rapid, high-resolution and cross-sectional morphology of macular area and optic nerve head for diagnosis and managing of different eye diseases. However, interpreting OCT images requires experts in both OCT images and eye diseases since many factors such as artefacts and concomitant diseases can affect the accuracy of quantitative measurements made by post-processing algorithms. Currently, there is a growing interest in applying deep learning (DL) methods to analyse OCT images automatically. This review summarises the trends in DL-based OCT image analysis in ophthalmology, discusses the current gaps, and provides potential research directions. DL in OCT analysis shows promising performance in several tasks: (1) layers and features segmentation and quantification; (2) disease classification; (3) disease progression and prognosis; and (4) referral triage level prediction. Different studies and trends in the development of DL-based OCT image analysis are described and the following challenges are identified and described: (1) public OCT data are scarce and scattered; (2) models show performance discrepancies in real-world settings; (3) models lack of transparency; (4) there is a lack of societal acceptance and regulatory standards; and (5) OCT is still not widely available in underprivileged areas. More work is needed to tackle the challenges and gaps, before DL is further applied in OCT image analysis for clinical use.

Uncertainty-inspired open set learning for retinal anomaly identification

Failure to recognize samples from the classes unseen during training is a major limitation of artificial intelligence in the real-world implementation for recognition and classification of retinal anomalies. We establish an uncertainty-inspired open set (UIOS) model, which is trained with fundus images of 9 retinal conditions. Besides assessing the probability of each category, UIOS also calculates an uncertainty score to express its confidence. Our UIOS model with thresholding strategy achieves an F1 score of 99.55%, 97.01% and 91.91% for the internal testing set, external target categories (TC)-JSIEC dataset and TC-unseen testing set, respectively, compared to the F1 score of 92.20%, 80.69% and 64.74% by the standard AI model. Furthermore, UIOS correctly predicts high uncertainty scores, which would prompt the need for a manual check in the datasets of non-target categories retinal diseases, low-quality fundus images, and non-fundus images. UIOS provides a robust method for real-world screening of retinal anomalies.

Loss-balanced parallel decoding network for retinal fluid segmentation in OCT

As a leading cause of blindness worldwide, macular edema (ME) is mainly determined by sub-retinal fluid (SRF), intraretinal fluid (IRF), and pigment epithelial detachment (PED) accumulation, and therefore, the characterization of SRF, IRF, and PED, which is also known as ME segmentation, has become a crucial issue in ophthalmology. Due to the subjective and time-consuming nature of ME segmentation in retinal optical coherence tomography (OCT) images, automatic computer-aided systems are highly desired in clinical practice. This paper proposes a novel loss-balanced parallel decoding network, namely PadNet, for ME segmentation. Specifically, PadNet mainly consists of an encoder and three parallel decoder modules, which serve as segmentation, contour, and diffusion branches, and they are employed to extract the ME's characteristics, the contour area features, and to expand the ME area from the center to edge, respectively. A new loss-balanced joint-loss function with three components corresponding to each of the three parallel decoding branches is also devised for training. Experiments are conducted with three public datasets to verify the effectiveness of PadNet, and the performances of PadNet are compared with those of five state-of-the-art methods. Results show that PadNet improves ME segmentation accuracy by 8.1%, 11.1%, 0.6%, 1.4% and 8.3%, as compared with UNet, sASPP, MsTGANet, YNet, RetiFluidNet, respectively, which convincingly demonstrates that the proposed PadNet is robust and effective in ME segmentation in different cases.

Deep Learning with Automatic Data Augmentation for Segmenting Schisis Cavities in the Optical Coherence Tomography Images of X-Linked Juvenile Retinoschisis Patients

X-linked juvenile retinoschisis (XLRS) is an inherited disorder characterized by retinal schisis cavities, which can be observed in optical coherence tomography (OCT) images. Monitoring disease progression necessitates the accurate segmentation and quantification of these cavities; yet, current manual methods are time consuming and result in subjective interpretations, highlighting the need for automated and precise solutions. We employed five state-of-the-art deep learning models-U-Net, U-Net++, Attention U-Net, Residual U-Net, and TransUNet-for the task, leveraging a dataset of 1500 OCT images from 30 patients. To enhance the models' performance, we utilized data augmentation strategies that were optimized via deep reinforcement learning. The deep learning models achieved a human-equivalent accuracy level in the segmentation of schisis cavities, with U-Net++ surpassing others by attaining an accuracy of 0.9927 and a Dice coefficient of 0.8568. By utilizing reinforcement-learning-based automatic data augmentation, deep learning segmentation models demonstrate a robust and precise method for the automated segmentation of schisis cavities in OCT images. These findings are a promising step toward enhancing clinical evaluation and treatment planning for XLRS.

Retinal OCT Layer Segmentation via Joint Motion Correction and Graph-Assisted 3D Neural Network

Optical Coherence Tomography (OCT) is a widely used 3D imaging technology in ophthalmology. Segmentation of retinal layers in OCT is important for diagnosis and evaluation of various retinal and systemic diseases. While 2D segmentation algorithms have been developed, they do not fully utilize contextual information and suffer from inconsistency in 3D. We propose neural networks to combine motion correction and segmentation in 3D. The proposed segmentation network utilizes 3D convolution and a novel graph pyramid structure with graph-inspired building blocks. We also collected one of the largest OCT segmentation dataset with manually corrected segmentation covering both normal examples and various diseases. The experimental results on three datasets with multiple instruments and various diseases show the proposed method can achieve improved segmentation accuracy compared with commercial softwares and conventional or deep learning methods in literature. Specifically, the proposed method reduced the average error from 38.47% to 11.43% compared to clinically available commercial software for severe deformations caused by diseases. The diagnosis and evaluation of diseases with large deformation such as DME, wet AMD and CRVO would greatly benefit from the improved accuracy, which impacts tens of millions of patients.

Unraveling the complexity of Optical Coherence Tomography image segmentation using machine and deep learning techniques: A review

Optical Coherence Tomography (OCT) is an emerging technology that provides three-dimensional images of the microanatomy of biological tissue in-vivo and at micrometer-scale resolution. OCT imaging has been widely used to diagnose and manage various medical diseases, such as macular degeneration, glaucoma, and coronary artery disease. Despite its wide range of applications, the segmentation of OCT images remains difficult due to the complexity of tissue structures and the presence of artifacts. In recent years, different approaches have been used for OCT image segmentation, such as intensity-based, region-based, and deep learning-based methods. This paper reviews the major advances in state-of-the-art OCT image segmentation techniques. It provides an overview of the advantages and limitations of each method and presents the most relevant research works related to OCT image segmentation. It also provides an overview of existing datasets and discusses potential clinical applications. Additionally, this review gives an in-depth analysis of machine learning and deep learning approaches for OCT image segmentation. It outlines challenges and opportunities for further research in this field.

Elastic Deformation of Optical Coherence Tomography Images of Diabetic Macular Edema for Deep-Learning Models Training: How Far to Go?

- Objective: To explore the clinical validity of elastic deformation of optical coherence tomography (OCT) images for data augmentation in the development of deep-learning model for detection of diabetic macular edema (DME).

METHODS

Prospective evaluation of OCT images of DME (n = 320) subject to elastic transformation, with the deformation intensity represented by ([Formula: see text]). Three sets of images, each comprising 100 pairs of scans (100 original & 100 modified), were grouped according to the range of ([Formula: see text]), including low-, medium- and high-degree of augmentation; ([Formula: see text] = 1-6), ([Formula: see text] = 7-12), and ([Formula: see text] = 13-18), respectively. Three retina specialists evaluated all datasets in a blinded manner and designated each image as 'original' versus 'modified'. The rate of assignment of 'original' value to modified images (false-negative) was determined for each grader in each dataset.

RESULTS

The false-negative rates ranged between 71-77% for the low-, 63-76% for the medium-, and 50-75% for the high-augmentation categories. The corresponding rates of correct identification of original images ranged between 75-85% ([Formula: see text]0.05) in the low-, 73-85% ([Formula: see text]0.05 for graders 1 & 2, p = 0.01 for grader 3) in the medium-, and 81-91% ([Formula: see text]) in the high-augmentation categories. In the subcategory ([Formula: see text] = 7-9) the false-negative rates were 93-83%, whereas the rates of correctly identifying original images ranged between 89-99% ([Formula: see text]0.05 for all graders).

CONCLUSIONS

Deformation of low-medium intensity ([Formula: see text] = 1-9) may be applied without compromising OCT image representativeness in DME. Clinical and Translational Impact Statement-Elastic deformation may efficiently augment the size, robustness, and diversity of training datasets without altering their clinical value, enhancing the development of high-accuracy algorithms for automated interpretation of OCT images.

Annotation-efficient learning for OCT segmentation

Deep learning has been successfully applied to OCT segmentation. However, for data from different manufacturers and imaging protocols, and for different regions of interest (ROIs), it requires laborious and time-consuming data annotation and training, which is undesirable in many scenarios, such as surgical navigation and multi-center clinical trials. Here we propose an annotation-efficient learning method for OCT segmentation that could significantly reduce annotation costs. Leveraging self-supervised generative learning, we train a Transformer-based model to learn the OCT imagery. Then we connect the trained Transformer-based encoder to a CNN-based decoder, to learn the dense pixel-wise prediction in OCT segmentation. These training phases use open-access data and thus incur no annotation costs, and the pre-trained model can be adapted to different data and ROIs without re-training. Based on the greedy approximation for the k-center problem, we also introduce an algorithm for the selective annotation of the target data. We verified our method on publicly-available and private OCT datasets. Compared to the widely-used U-Net model with 100% training data, our method only requires ∼10% of the data for achieving the same segmentation accuracy, and it speeds the training up to ∼3.5 times. Furthermore, our proposed method outperforms other potential strategies that could improve annotation efficiency. We think this emphasis on learning efficiency may help improve the intelligence and application penetration of OCT-based technologies.

FedMix: Mixed Supervised Federated Learning for Medical Image Segmentation

The purpose of federated learning is to enable multiple clients to jointly train a machine learning model without sharing data. However, the existing methods for training an image segmentation model have been based on an unrealistic assumption that the training set for each local client is annotated in a similar fashion and thus follows the same image supervision level. To relax this assumption, in this work, we propose a label-agnostic unified federated learning framework, named FedMix, for medical image segmentation based on mixed image labels. In FedMix, each client updates the federated model by integrating and effectively making use of all available labeled data ranging from strong pixel-level labels, weak bounding box labels, to weakest image-level class labels. Based on these local models, we further propose an adaptive weight assignment procedure across local clients, where each client learns an aggregation weight during the global model update. Compared to the existing methods, FedMix not only breaks through the constraint of a single level of image supervision but also can dynamically adjust the aggregation weight of each local client, achieving rich yet discriminative feature representations. Experimental results on multiple publicly-available datasets validate that the proposed FedMix outperforms the state-of-the-art methods by a large margin. In addition, we demonstrate through experiments that FedMix is extendable to multi-class medical image segmentation and much more feasible in clinical scenarios. The code is available at: https://github.com/Jwicaksana/FedMix.

Directional Connectivity-based Segmentation of Medical Images

Anatomical consistency in biomarker segmentation is crucial for many medical image analysis tasks. A promising paradigm for achieving anatomically consistent segmentation via deep networks is incorporating pixel connectivity, a basic concept in digital topology, to model inter-pixel relationships. However, previous works on connectivity modeling have ignored the rich channel-wise directional information in the latent space. In this work, we demonstrate that effective disentanglement of directional sub-space from the shared latent space can significantly enhance the feature representation in the connectivity-based network. To this end, we propose a directional connectivity modeling scheme for segmentation that decouples, tracks, and utilizes the directional information across the network. Experiments on various public medical image segmentation benchmarks show the effectiveness of our model as compared to the state-of-the-art methods. Code is available at https://github.com/Zyun-Y/DconnNet.

A bibliometric analysis of artificial intelligence applications in macular edema: exploring research hotspots and Frontiers

Background: Artificial intelligence (AI) is used in ophthalmological disease screening and diagnostics, medical image diagnostics, and predicting late-disease progression rates. We reviewed all AI publications associated with macular edema (ME) research Between 2011 and 2022 and performed modeling, quantitative, and qualitative investigations. Methods: On 1st February 2023, we screened the Web of Science Core Collection for AI applications related to ME, from which 297 studies were identified and analyzed (2011-2022). We collected information on: publications, institutions, country/region, keywords, journal name, references, and research hotspots. Literature clustering networks and Frontier knowledge bases were investigated using bibliometrix-BiblioShiny, VOSviewer, and CiteSpace bibliometric platforms. We used the R "bibliometrix" package to synopsize our observations, enumerate keywords, visualize collaboration networks between countries/regions, and generate a topic trends plot. VOSviewer was used to examine cooperation between institutions and identify citation relationships between journals. We used CiteSpace to identify clustering keywords over the timeline and identify keywords with the strongest citation bursts. Results: In total, 47 countries published AI studies related to ME; the United States had the highest H-index, thus the greatest influence. China and the United States cooperated most closely between all countries. Also, 613 institutions generated publications - the Medical University of Vienna had the highest number of studies. This publication record and H-index meant the university was the most influential in the ME field. Reference clusters were also categorized into 10 headings: retinal Optical Coherence Tomography (OCT) fluid detection, convolutional network models, deep learning (DL)-based single-shot predictions, retinal vascular disease, diabetic retinopathy (DR), convolutional neural networks (CNNs), automated macular pathology diagnosis, dry age-related macular degeneration (DARMD), class weight, and advanced DL architecture systems. Frontier keywords were represented by diabetic macular edema (DME) (2021-2022). Conclusion: Our review of the AI-related ME literature was comprehensive, systematic, and objective, and identified future trends and current hotspots. With increased DL outputs, the ME research focus has gradually shifted from manual ME examinations to automatic ME detection and associated symptoms. In this review, we present a comprehensive and dynamic overview of AI in ME and identify future research areas.

Analysis of the Asymmetry between Both Eyes in Early Diagnosis of Glaucoma Combining Features Extracted from Retinal Images and OCTs into Classification Models

This study aims to analyze the asymmetry between both eyes of the same patient for the early diagnosis of glaucoma. Two imaging modalities, retinal fundus images and optical coherence tomographies (OCTs), have been considered in order to compare their different capabilities for glaucoma detection. From retinal fundus images, the difference between cup/disc ratio and the width of the optic rim has been extracted. Analogously, the thickness of the retinal nerve fiber layer has been measured in spectral-domain optical coherence tomographies. These measurements have been considered as asymmetry characteristics between eyes in the modeling of decision trees and support vector machines for the classification of healthy and glaucoma patients. The main contribution of this work is indeed the use of different classification models with both imaging modalities to jointly exploit the strengths of each of these modalities for the same diagnostic purpose based on the asymmetry characteristics between the eyes of the patient. The results show that the optimized classification models provide better performance with OCT asymmetry features between both eyes (sensitivity 80.9%, specificity 88.2%, precision 66.7%, accuracy 86.5%) than with those extracted from retinographies, although a linear relationship has been found between certain asymmetry features extracted from both imaging modalities. Therefore, the resulting performance of the models based on asymmetry features proves their ability to differentiate healthy from glaucoma patients using those metrics. Models trained from fundus characteristics are a useful option as a glaucoma screening method in the healthy population, although with lower performance than those trained from the thickness of the peripapillary retinal nerve fiber layer. In both imaging modalities, the asymmetry of morphological characteristics can be used as a glaucoma indicator, as detailed in this work.

RetiFluidNet: A Self-Adaptive and Multi-Attention Deep Convolutional Network for Retinal OCT Fluid Segmentation

Optical coherence tomography (OCT) helps ophthalmologists assess macular edema, accumulation of fluids, and lesions at microscopic resolution. Quantification of retinal fluids is necessary for OCT-guided treatment management, which relies on a precise image segmentation step. As manual analysis of retinal fluids is a time-consuming, subjective, and error-prone task, there is increasing demand for fast and robust automatic solutions. In this study, a new convolutional neural architecture named RetiFluidNet is proposed for multi-class retinal fluid segmentation. The model benefits from hierarchical representation learning of textural, contextual, and edge features using a new self-adaptive dual-attention (SDA) module, multiple self-adaptive attention-based skip connections (SASC), and a novel multi-scale deep self-supervision learning (DSL) scheme. The attention mechanism in the proposed SDA module enables the model to automatically extract deformation-aware representations at different levels, and the introduced SASC paths further consider spatial-channel interdependencies for concatenation of counterpart encoder and decoder units, which improve representational capability. RetiFluidNet is also optimized using a joint loss function comprising a weighted version of dice overlap and edge-preserved connectivity-based losses, where several hierarchical stages of multi-scale local losses are integrated into the optimization process. The model is validated based on three publicly available datasets: RETOUCH, OPTIMA, and DUKE, with comparisons against several baselines. Experimental results on the datasets prove the effectiveness of the proposed model in retinal OCT fluid segmentation and reveal that the suggested method is more effective than existing state-of-the-art fluid segmentation algorithms in adapting to retinal OCT scans recorded by various image scanning instruments.

On Machine Learning in Clinical Interpretation of Retinal Diseases Using OCT Images

Optical coherence tomography (OCT) is a noninvasive imaging technique that provides high-resolution cross-sectional retina images, enabling ophthalmologists to gather crucial information for diagnosing various retinal diseases. Despite its benefits, manual analysis of OCT images is time-consuming and heavily dependent on the personal experience of the analyst. This paper focuses on using machine learning to analyse OCT images in the clinical interpretation of retinal diseases. The complexity of understanding the biomarkers present in OCT images has been a challenge for many researchers, particularly those from nonclinical disciplines. This paper aims to provide an overview of the current state-of-the-art OCT image processing techniques, including image denoising and layer segmentation. It also highlights the potential of machine learning algorithms to automate the analysis of OCT images, reducing time consumption and improving diagnostic accuracy. Using machine learning in OCT image analysis can mitigate the limitations of manual analysis methods and provide a more reliable and objective approach to diagnosing retinal diseases. This paper will be of interest to ophthalmologists, researchers, and data scientists working in the field of retinal disease diagnosis and machine learning. By presenting the latest advancements in OCT image analysis using machine learning, this paper will contribute to the ongoing efforts to improve the diagnostic accuracy of retinal diseases.

A Review of Machine Learning Algorithms for Retinal Cyst Segmentation on Optical Coherence Tomography

Optical coherence tomography (OCT) is an emerging imaging technique for diagnosing ophthalmic diseases and the visual analysis of retinal structure changes, such as exudates, cysts, and fluid. In recent years, researchers have increasingly focused on applying machine learning algorithms, including classical machine learning and deep learning methods, to automate retinal cysts/fluid segmentation. These automated techniques can provide ophthalmologists with valuable tools for improved interpretation and quantification of retinal features, leading to more accurate diagnosis and informed treatment decisions for retinal diseases. This review summarized the state-of-the-art algorithms for the three essential steps of cyst/fluid segmentation: image denoising, layer segmentation, and cyst/fluid segmentation, while emphasizing the significance of machine learning techniques. Additionally, we provided a summary of the publicly available OCT datasets for cyst/fluid segmentation. Furthermore, the challenges, opportunities, and future directions of artificial intelligence (AI) in OCT cyst segmentation are discussed. This review is intended to summarize the key parameters for the development of a cyst/fluid segmentation system and the design of novel segmentation algorithms and has the potential to serve as a valuable resource for imaging researchers in the development of assessment systems related to ocular diseases exhibiting cyst/fluid in OCT imaging.

Self-training adversarial learning for cross-domain retinal OCT fluid segmentation

Accurate measurements of the size, shape and volume of macular edema can provide important biomarkers to jointly assess disease progression and treatment outcome. Although many deep learning-based segmentation algorithms have achieved remarkable success in semantic segmentation, these methods have difficulty obtaining satisfactory segmentation results in retinal optical coherence tomography (OCT) fluid segmentation tasks due to low contrast, blurred boundaries, and varied distribution. Moreover, directly applying a well-trained model on one device to test the images from other devices may cause the performance degradation in the joint analysis of multi-domain OCT images. In this paper, we propose a self-training adversarial learning framework for unsupervised domain adaptation in retinal OCT fluid segmentation tasks. Specifically, we develop an image style transfer module and a fine-grained feature transfer module to reduce discrepancies in the appearance and high-level features of images from different devices. Importantly, we transfer the target images to the appearance of source images to ensure that no image information of the source domain for supervised training is lost. To capture specific features of the target domain, we design a self-training module based on a discrepancy and similarity strategy to select the images with better segmentation results from the target domain and then introduce them into the source domain for the iterative training segmentation model. Extensive experiments on two challenging datasets demonstrate the effectiveness of our proposed method. In Particular, our proposed method achieves comparable results on cross-domain retinal OCT fluid segmentation compared with the state-of-the-art methods.