Automated 3-D intraretinal layer segmentation of macular spectral-domain optical coherence tomography images

Longitudinal deep network for consistent OCT layer segmentation

Retinal layer thickness is an important bio-marker for people with multiple sclerosis (PwMS). In clinical practice, retinal layer thickness changes in optical coherence tomography (OCT) are widely used for monitoring multiple sclerosis (MS) progression. Recent developments in automated retinal layer segmentation algorithms allow cohort-level retina thinning to be observed in a large study of PwMS. However, variability in these results make it difficult to identify patient-level trends; this prevents patient specific disease monitoring and treatment planning using OCT. Deep learning based retinal layer segmentation algorithms have achieved state-of-the-art accuracy, but the segmentation is performed on each individual scan without utilizing longitudinal information, which can be important in reducing segmentation error and reveal subtle changes in retinal layers. In this paper, we propose a longitudinal OCT segmentation network which achieves more accurate and consistent layer thickness measurements for PwMS.

Retinal photoreceptor layer thickness has disease specificity and distinguishes predicted FTLD-Tau from biomarker-determined Alzheimer's disease

While Alzheimer's disease (AD) is associated with inner retina thinning (retinal nerve fiber layer and ganglion cell layer), we have observed photoreceptor outer nuclear layer (ONL) thinning in patients with frontotemporal lobar degeneration tauopathy (FTLD-Tau) compared to normal controls. We hypothesized that ONL thinning may distinguish FTLD-Tau from patients with biomarker evidence of AD neuropathologic change (ADNC) and will correlate with FTLD-Tau disease severity. Predicted FTLD-Tau (pFTLD-Tau; n = 21; 33 eyes) and predicted ADNC (pADNC; n = 24; 46 eyes) patients were consecutively enrolled, underwent optical coherence tomography macula imaging, and disease was categorized (pFTLD-Tau vs. pADNC) with cerebrospinal fluid biomarkers, genetic testing, and autopsy data when available. Adjusting for age, sex, and race, pFTLD-Tau patients had a thinner ONL compared to pADNC, while retinal nerve fiber layer and ganglion cell layer were not significantly different. Reduced ONL thickness correlated with worse performance on Folstein Mini-Mental State Examination and clinical dementia rating plus frontotemporal dementia sum of boxes for pFTLD-Tau but not pADNC. Photoreceptor ONL thickness may serve as an important noninvasive diagnostic marker that distinguishes FTLD-Tau from AD neuropathologic change.

Modeling rod and cone photoreceptor cell survival in vivo using optical coherence tomography

Many retinal diseases involve the loss of light-sensing photoreceptor cells (rods and cones) over time. The severity and distribution of photoreceptor loss varies widely across diseases and affected individuals, so characterizing the degree and pattern of photoreceptor loss can clarify pathophysiology and prognosis. Currently, in vivo visualization of individual photoreceptors requires technology such as adaptive optics, which has numerous limitations and is not widely used. By contrast, optical coherence tomography (OCT) is nearly ubiquitous in daily clinical practice given its ease of image acquisition and detailed visualization of retinal structure. However, OCT cannot resolve individual photoreceptors, and no OCT-based method exists to distinguish between the loss of rods versus cones. Here, we present a computational model that quantitatively estimates rod versus cone photoreceptor loss from OCT. Using histologic data of human photoreceptor topography, we constructed an OCT-based reference model to simulate outer nuclear layer thinning caused by differential loss of rods and cones. The model was able to estimate rod and cone loss using in vivo OCT data from patients with Stargardt disease and healthy controls. Our model provides a powerful new tool to quantify photoreceptor loss using OCT data alone, with potentially broad applications for research and clinical care.

TranSegNet: Hybrid CNN-Vision Transformers Encoder for Retina Segmentation of Optical Coherence Tomography

Optical coherence tomography (OCT) provides unique advantages in ophthalmic examinations owing to its noncontact, high-resolution, and noninvasive features, which have evolved into one of the most crucial modalities for identifying and evaluating retinal abnormalities. Segmentation of laminar structures and lesion tissues in retinal OCT images can provide quantitative information on retinal morphology and reliable guidance for clinical diagnosis and treatment. Convolutional neural networks (CNNs) have achieved success in various medical image segmentation tasks. However, the receptive field of convolution has inherent locality constraints, resulting in limitations of mainstream frameworks based on CNNs, which is still evident in recognizing the morphological changes of retina OCT. In this study, we proposed an end-to-end network, TranSegNet, which incorporates a hybrid encoder that combines the advantages of a lightweight vision transformer (ViT) and the U-shaped network. The CNN features under multiscale resolution are extracted based on the improved U-net backbone, and a ViT with the multi-head convolutional attention is introduced to capture the feature information in a global view, realizing accurate localization and segmentation of retinal layers and lesion tissues. The experimental results illustrate that hybrid CNN-ViT is a strong encoder for retinal OCT image segmentation tasks and the lightweight design reduces its parameter size and computational complexity while maintaining its outstanding performance. By applying TranSegNet to healthy and diseased retinal OCT datasets separately, TranSegNet demonstrated superior efficiency, accuracy, and robustness in the segmentation results of retinal layers and accumulated fluid than the four advanced segmentation methods, such as FCN, SegNet, Unet and TransUnet.

Therapeutic response in the HAWK and HARRIER trials using deep learning in retinal fluid volume and compartment analysis

OBJECTIVES

To assess the therapeutic response to brolucizumab and aflibercept by deep learning/OCT-based analysis of macular fluid volumes in neovascular age-related macular degeneration.

METHODS

In this post-hoc analysis of two phase III, randomised, multi-centre studies (HAWK/HARRIER), 1078 and 739 treatment-naive eyes receiving brolucizumab or aflibercept according to protocol-specified criteria in HAWK and HARRIER, respectively, were included. Macular fluid on 41,840 OCT scans was localised and quantified using a validated deep learning-based algorithm. Volumes of intraretinal fluid (IRF), subretinal fluid (SRF), pigment epithelial detachment (PED) for all central macular areas (1, 3 and 6 mm) in nanolitres (nL) and best corrected visual acuity (BCVA) change in ETDRS letters were associated using mixed models for repeated measures.

RESULTS

Baseline IRF volumes decreased by >92% following the first intravitreal injection and consistently remained low during follow-up. Baseline SRF volumes decreased by >74% following the first injection, while PED volume resolved by 68-79% of its baseline volume. Resolution of SRF and PED was dependent on the substance and regimen used. Larger residual post-loading IRF, SRF and PED volumes were all independently associated with progressive vision loss during maintenance, where the differences in mean BCVA change between high and low fluid volume subgroups for IRF, SRF and PED were 3.4 letters (p < 0.0001), 1.7 letters (p < 0.001) and 2.5 letters (p < 0.0001), respectively.

CONCLUSIONS

Deep-learning methods allow an accurate assessment of substance and regimen efficacy. Irrespectively, all fluid compartments were found to be important markers of disease activity and were relevant for visual outcomes.

A Review of Machine Learning Algorithms for Retinal Cyst Segmentation on Optical Coherence Tomography

Optical coherence tomography (OCT) is an emerging imaging technique for diagnosing ophthalmic diseases and the visual analysis of retinal structure changes, such as exudates, cysts, and fluid. In recent years, researchers have increasingly focused on applying machine learning algorithms, including classical machine learning and deep learning methods, to automate retinal cysts/fluid segmentation. These automated techniques can provide ophthalmologists with valuable tools for improved interpretation and quantification of retinal features, leading to more accurate diagnosis and informed treatment decisions for retinal diseases. This review summarized the state-of-the-art algorithms for the three essential steps of cyst/fluid segmentation: image denoising, layer segmentation, and cyst/fluid segmentation, while emphasizing the significance of machine learning techniques. Additionally, we provided a summary of the publicly available OCT datasets for cyst/fluid segmentation. Furthermore, the challenges, opportunities, and future directions of artificial intelligence (AI) in OCT cyst segmentation are discussed. This review is intended to summarize the key parameters for the development of a cyst/fluid segmentation system and the design of novel segmentation algorithms and has the potential to serve as a valuable resource for imaging researchers in the development of assessment systems related to ocular diseases exhibiting cyst/fluid in OCT imaging.

Exploring healthy retinal aging with deep learning

Purpose

To study the individual course of retinal changes caused by healthy aging using deep learning.

Design

Retrospective analysis of a large data set of retinal OCT images.

Participants

A total of 85 709 adults between the age of 40 and 75 years of whom OCT images were acquired in the scope of the UK Biobank population study.

Methods

We created a counterfactual generative adversarial network (GAN), a type of neural network that learns from cross-sectional, retrospective data. It then synthesizes high-resolution counterfactual OCT images and longitudinal time series. These counterfactuals allow visualization and analysis of hypothetical scenarios in which certain characteristics of the imaged subject, such as age or sex, are altered, whereas other attributes, crucially the subject's identity and image acquisition settings, remain fixed.

Main Outcome Measures

Using our counterfactual GAN, we investigated subject-specific changes in the retinal layer structure as a function of age and sex. In particular, we measured changes in the retinal nerve fiber layer (RNFL), combined ganglion cell layer plus inner plexiform layer (GCIPL), inner nuclear layer to the inner boundary of the retinal pigment epithelium (INL-RPE), and retinal pigment epithelium (RPE).

Results

Our counterfactual GAN is able to smoothly visualize the individual course of retinal aging. Across all counterfactual images, the RNFL, GCIPL, INL-RPE, and RPE changed by -0.1 μm ± 0.1 μm, -0.5 μm ± 0.2 μm, -0.2 μm ± 0.1 μm, and 0.1 μm ± 0.1 μm, respectively, per decade of age. These results agree well with previous studies based on the same cohort from the UK Biobank population study. Beyond population-wide average measures, our counterfactual GAN allows us to explore whether the retinal layers of a given eye will increase in thickness, decrease in thickness, or stagnate as a subject ages.

Conclusion

This study demonstrates how counterfactual GANs can aid research into retinal aging by generating high-resolution, high-fidelity OCT images, and longitudinal time series. Ultimately, we envision that they will enable clinical experts to derive and explore hypotheses for potential imaging biomarkers for healthy and pathologic aging that can be refined and tested in prospective clinical trials.

Financial Disclosures

Proprietary or commercial disclosure may be found after the references.

Multiscale joint segmentation method for retinal optical coherence tomography images using a bidirectional wave algorithm and improved graph theory

Morphology and functional metrics of retinal layers are important biomarkers for many human ophthalmic diseases. Automatic and accurate segmentation of retinal layers is crucial for disease diagnosis and research. To improve the performance of retinal layer segmentation, a multiscale joint segmentation framework for retinal optical coherence tomography (OCT) images based on bidirectional wave algorithm and improved graph theory is proposed. In this framework, the bidirectional wave algorithm was used to segment edge information in multiscale images, and the improved graph theory was used to modify edge information globally, to realize automatic and accurate segmentation of eight retinal layer boundaries. This framework was tested on two public datasets and two OCT imaging systems. The test results show that, compared with other state-of-the-art methods, this framework does not need data pre-training and parameter pre-adjustment on different datasets, and can achieve sub-pixel retinal layer segmentation on a low-configuration computer.

Protocol for Multi-modality MEdical imaging sTudy bAsed on KaiLuan Study (META-KLS): rationale, design and database building

INTRODUCTION

Multi-modality medical imaging study, especially brain MRI, greatly facilitates the research on subclinical brain disease. However, there is still a lack of such studies with a wider age span of participants. The Multi-modality MEdical imaging sTudy bAsed on KaiLuan Study (META-KLS) was designed to address this issue with a large sample size population.

METHODS AND ANALYSIS

We aim to enrol at least 1000 subjects in META-KLS. All the participants without contraindications will perform multi-modality medical imaging, including brain MRI, retinal fundus photograph, fundus optical coherence tomography (OCT) and ultrasonography of the internal carotid artery (ICA) every 2-4 years. The acquired medical imaging will be further processed with a standardised and validated workflow. The clinical data at baseline and follow-up will be collected from the KaiLuan Study. The associations between multiple risk factors and subclinical brain disease are able to be fully investigated. Researches based on META-KLS will provide a series of state-of-the-art evidence for the prevention of neurological diseases and common chronic diseases.

ETHICS AND DISSEMINATION

The Kailuan Study and META-KLS have been approved by the Medical Ethics Committee of Kailuan General Hospital (IRB number: 2008 No. 1 and 2021002, respectively). Written informed consent will be acquired from each participant. Results are expected to be published in professional peer-reviewed journals beginning in 2023.

TRIAL REGISTRATION NUMBER

NCT05453877.

Optical Coherence Tomography: Imaging Visual System Structures in Mice

Optical coherence tomography (OCT) enables micron-scale resolution of structural anatomy, thereby making OCT a valuable tool for addressing ophthalmologic and neurologic inquiries. Although the murine eye and its structures are very small and offers challenges for OCT imaging, OCT can be used to monitor retinal layer thickness in healthy and diseased retinas in murine lines in vivo longitudinally. Thus, OCT can provide insights into disease severity and treatment efficacy. This chapter describes the use of OCT as a powerful non-invasive imaging technology for high-resolution retinal imaging and retinal thickness quantification in rodents.

Quantitative assessment of depolarization by the retinal pigment epithelium in healthy and glaucoma subjects measured over a large field of view

We present measurements of depolarization introduced by the retinal pigment epithelium (RPE) over a 45° field of view using polarization sensitive optical coherence tomography. A detailed spatial distribution analysis of depolarization caused by the RPE is presented in a total of 153 subjects including both healthy and diseased eyes. Age and sex related differences in the depolarizing character of the RPE are investigated.

Influence of signal‐to‐noise ratio, glaucoma stage and segmentation algorithm on OCT usability for quantifying layer thicknesses in the peripapillary retina

PURPOSE

OCT can be used for glaucoma assessment, but its usefulness may depend on image quality, disease stage and segmentation algorithm. We aimed to determine how layer thicknesses as assessed with different algorithms depend on image quality and disease stage, how reproducible the algorithms are, and if the algorithms (dis)agree.

METHODS

Optic disc OCT data (Canon OCT-HS100) from 20 healthy subjects and 28 early, 29 moderate, and 23 severe glaucoma patients were assessed with four different algorithms (CANON, IOWA, FWHM, DOCTRAP). We measured retinal nerve fibre layer thickness (RNFLT) and total retinal thickness (TRT) along the 1.7-mm-radius OCT measurement circle centred at the optic disc. In healthy subjects, image quality was degraded with neutral density filters (0.3-0.9 optical density [OD]); three scans were made to assess repeatability. Results were analysed with ANOVA with Bonferroni corrected t-tests for post hoc analysis and with intraclass correlation coefficient (ICC) analysis.

RESULTS

For all algorithms, RNFLT was more sensitive to image quality than TRT. Both RNFLT and TRT showed differences between healthy and glaucoma (all algorithms p < 0.001 for both RNFLT and TRT) and between early and moderate glaucoma (RNFLT: p = 0.001 to p = 0.09; TRT: p < 0.001 to p = 0.03); neither was able to discriminate between moderate and severe glaucoma (p = 0.08 to p = 1.0). Generally, repeatability was excellent (ICC >0.75); agreement between algorithms varied from moderate to excellent.

CONCLUSIONS

OCT becomes less informative with glaucoma progression, irrespective of the algorithm. For good-quality scans, RNFLT and TRT perform similarly; TRT may be advantageous with poor image quality.

Extending field-of-view of retinal imaging by optical coherence tomography using convolutional Lissajous and slow scan patterns

Optical coherence tomography (OCT) is a high-speed non-invasive cross-sectional imaging technique. Although its imaging speed is high, three-dimensional high-spatial-sampling-density imaging of in vivo tissues with a wide field-of-view (FOV) is challenging. We employed convolved Lissajous and slow circular scanning patterns to extend the FOV of retinal OCT imaging with a 1-µm, 100-kHz-sweep-rate swept-source OCT prototype system. Displacements of sampling points due to eye movements are corrected by post-processing based on a Lissajous scan. Wide FOV three-dimensional retinal imaging with high sampling density and motion correction is achieved. Three-dimensional structures obtained using repeated imaging sessions of a healthy volunteer and two patients showed good agreement. The demonstrated technique will extend the FOV of simple point-scanning OCT, such as commercial ophthalmic OCT devices, without sacrificing sampling density.

JOINT MOTION CORRECTION AND 3D SEGMENTATION WITH GRAPH-ASSISTED NEURAL NETWORKS FOR RETINAL OCT

Optical Coherence Tomography (OCT) is a widely used non-invasive high resolution 3D imaging technique for biological tissues and plays an important role in ophthalmology. OCT retinal layer segmentation is a fundamental image processing step for OCT-Angiography projection, and disease analysis. A major problem in retinal imaging is the motion artifacts introduced by involuntary eye movements. In this paper, we propose neural networks that jointly correct eye motion and retinal layer segmentation utilizing 3D OCT information, so that the segmentation among neighboring B-scans would be consistent. The experimental results show both visual and quantitative improvements by combining motion correction and 3D OCT layer segmentation comparing to conventional and deep-learning based 2D OCT layer segmentation.

Predicting treat-and-extend outcomes and treatment intervals in neovascular age-related macular degeneration from retinal optical coherence tomography using artificial intelligence

Purpose

To predict visual outcomes and treatment needs in a treat &amp; extend (T&amp;E) regimen in neovascular age-related macular degeneration (nAMD) using a machine learning model based on quantitative optical coherence tomography (OCT) imaging biomarkers.

Materials and methods

Study eyes of 270 treatment-naïve subjects, randomized to receiving ranibizumab therapy in the T&amp;E arm of a randomized clinical trial were considered. OCT volume scans were processed at baseline and at the first follow-up visit 4 weeks later. Automated image segmentation was performed, where intraretinal (IRF), subretinal (SRF) fluid, pigment epithelial detachment (PED), hyperreflective foci, and the photoreceptor layer were delineated using a convolutional neural network (CNN). A set of respective quantitative imaging biomarkers were computed across an Early Treatment Diabetic Retinopathy Study (ETDRS) grid to describe the retinal pathomorphology spatially and its change after the first injection. Lastly, using the computed set of OCT features and available clinical and demographic information, predictive models of outcomes and retreatment intervals were built using machine learning and their performance evaluated with a 10-fold cross-validation.

Results

Data of 228 evaluable patients were included, as some had missing scans or were lost to follow-up. Of those patients, 55% reached and maintained long (8, 10, 12 weeks) and another 45% stayed at short (4, 6 weeks) treatment intervals. This provides further evidence for a high disease activity in a major proportion of patients. The model predicted the extendable treatment interval group with an AUROC of 0.71, and the visual outcome with an AUROC of up to 0.87 when utilizing both, clinical and imaging features. The volume of SRF and the volume of IRF, remaining at the first follow-up visit, were found to be the most important predictive markers for treatment intervals and visual outcomes, respectively, supporting the important role of quantitative fluid parameters on OCT.

Conclusion

The proposed Artificial intelligence (AI) methodology was able to predict visual outcomes and retreatment intervals of a T&amp;E regimen from a single injection. The result of this study is an urgently needed step toward AI-supported management of patients with active and progressive nAMD.

Functional optical coherence tomography at altitude: retinal microvascular perfusion and retinal thickness at 3,800 meters

Cerebral hypoxia is a serious consequence of several cardiorespiratory illnesses. Measuring the retinal microvasculature at high altitude provides a surrogate for cerebral microvasculature, offering potential insight into cerebral hypoxia in critical illness. Additionally, while sex-specific differences in cardiovascular diseases are strongly supported, few have focused on differences in ocular blood flow. We evaluated the retinal microvasculature in males (n=11) and females (n=7) using functional optical coherence tomography at baseline (1,130m) (Day 0), following rapid ascent (Day 2) and prolonged exposure (Day 9) to high altitude (3,800m). Retinal vascular perfusion density (rVPD; an index of total blood supply), retinal thickness (RT; reflecting vascular and neural tissue volume) and arterial blood were acquired. As a group, rVPD increased on Day 2 vs. Day 0 (p<0.001) and was inversely related to PaO₂ (R²=0.45; p=0.006). By Day 9, rVPD recovered to baseline, but was significantly lower in males vs. females (p=0.007). RT was not different on Day 2 vs. Day 0 (p>0.99) but was reduced by Day 9 relative to Day 0 and Day 2 (p<0.001). RT changes relative to Day 0 were inversely related to changes in PaO₂ on Day 2 (R²=0.6; p=0.001) and Day 9 (R²=0.4; p=0.02). RT did not differ between sexes. These data suggest differential time course and regulation of the retina during rapid ascent and prolonged exposure to high altitude and are the first to demonstrate sex-specific differences in rVPD at high altitude. The ability to assess intact microvasculature contiguous with the brain has widespread research and clinical applications.

Predicting Visual Acuity in Patients Treated for AMD

The leading diagnostic tool in modern ophthalmology, Optical Coherence Tomography (OCT), is not yet able to establish the evolution of retinal diseases. Our task is to forecast the progression of retinal diseases by means of machine learning technologies. The aim is to help the ophthalmologist to determine when early treatment is needed in order to prevent severe vision impairment or even blindness. The acquired data are made up of sequences of visits from multiple patients with age-related macular degeneration (AMD), which, if not treated at the appropriate time, may result in irreversible blindness. The dataset contains 94 patients with AMD and there are 161 eyes included with more than one medical examination. We used various techniques from machine learning (linear regression, gradient boosting, random forest and extremely randomised trees, bidirectional recurrent neural network, LSTM network, GRU network) to handle technical challenges such as how to learn from small-sized time series, how to handle different time intervals between visits, and how to learn from different numbers of visits for each patient (1–5 visits). For predicting the visual acuity, we performed several experiments with different features. First, by considering only previous measured visual acuity, the best accuracy of 0.96 was obtained based on a linear regression. Second, by considering numerical OCT features such as previous thickness and volume values in all retinal zones, the LSTM network reached the highest score (R2=0.99). Third, by considering the fundus scan images represented as embeddings obtained from the convolutional autoencoder, the accuracy was increased for all algorithms. The best forecasting results for visual acuity depend on the number of visits and features used for predictions, i.e., 0.99 for LSTM based on three visits (monthly resampled series) based on numerical OCT values, fundus images, and previous visual acuities.

Neuroimaging and cognitive correlates of retinal Optical Coherence Tomography (OCT) measures at late middle age in a twin sample

Sharing in embryology and function between the eye and brain has led to interest in whether assessments of the eye reflect brain changes seen in neurodegeneration. We aimed to examine the associations between measures of retinal layer thickness using optical coherence tomography (OCT) and multimodal measures of brain structure and function. Using a convenient sample of twins discordant for type 2 diabetes, we performed cognitive testing, structural brain MRI (tissue volumetry), diffusion tensor imaging (white matter microstructure), and arterial spin labelling (cerebral blood flow). OCT images were recorded and retinal thickness maps generated. We used mixed level modelling to examine the relationship between retinal layer thicknesses and brain measures. We enrolled 35 people (18 pairs, mean age 63.8 years, 63% female). Ganglion cell layer thickness was positively associated with memory, speed, gray matter volume, and altered mean diffusivity. Ganglion cell layer thickness was strongly positively associated with regional cerebral blood flow. We found only a limited number of associations between other retinal layer thickness and measures of brain structure or function. Ganglion cell layer thickness showed consistent associations with a range of brain measures suggesting it may have utility as a marker for future dementia risk.

Automatic Segmentation of the Optic Nerve Head Region in Optical Coherence Tomography: A Methodological Review

The optic nerve head (ONH) represents the intraocular section of the optic nerve, which is prone to damage by intraocular pressure (IOP). The advent of optical coherence tomography (OCT) has enabled the evaluation of novel ONH parameters, namely the depth and curvature of the lamina cribrosa (LC). Together with the Bruch's membrane minimum-rim-width (BMO-MRW), these seem to be promising ONH parameters for diagnosis and monitoring of retinal diseases such as glaucoma. Nonetheless, these OCT derived biomarkers are mostly extracted through manual segmentation, which is time-consuming and prone to bias, thus limiting their usability in clinical practice. The automatic segmentation of ONH in OCT scans could further improve the current clinical management of glaucoma and other diseases. This review summarizes the current state-of-the-art in automatic segmentation of the ONH in OCT. PubMed and Scopus were used to perform a systematic review. Additional works from other databases (IEEE, Google Scholar and ARVO IOVS) were also included, resulting in a total of 29 reviewed studies. For each algorithm, the methods, the size and type of dataset used for validation, and the respective results were carefully analysed. The results show a lack of consensus regarding the definition of segmented regions, extracted parameters and validation approaches, highlighting the importance and need of standardized methodologies for ONH segmentation. Only with a concrete set of guidelines, these automatic segmentation algorithms will build trust in data-driven segmentation models and be able to enter clinical practice.

Retinal layer segmentation in optical coherence tomography (OCT) using a 3D deep-convolutional regression network for patients with age-related macular degeneration

Introduction - Retinal layer segmentation in optical coherence tomography (OCT) images is an important approach for detecting and prognosing disease. Automating segmentation using robust machine learning techniques lead to computationally efficient solutions and significantly reduces the cost of labor-intensive labeling, which is traditionally performed by trained graders at a reading center, sometimes aided by semi-automated algorithms. Although several algorithms have been proposed since the revival of deep learning, eyes with severe pathological conditions continue to challenge fully automated segmentation approaches. There remains an opportunity to leverage the underlying spatial correlations between the retinal surfaces in the segmentation approach. Methods - Some of these proposed traditional methods can be expanded to utilize the three-dimensional spatial context governing the retinal image volumes by replacing the use of 2D filters with 3D filters. Towards this purpose, we propose a spatial-context, continuity and anatomical relationship preserving semantic segmentation algorithm, which utilizes the 3D spatial context from the image volumes with the use of 3D filters. We propose a 3D deep neural network capable of learning the surface positions of the layers in the retinal volumes. Results - We utilize a dataset of OCT images from patients with Age-related Macular Degeneration (AMD) to assess performance of our model and provide both qualitative (including segmentation maps and thickness maps) and quantitative (including error metric comparisons and volumetric comparisons) results, which demonstrate that our proposed method performs favorably even for eyes with pathological changes caused by severe retinal diseases. The Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) for patients with a wide range of AMD severity scores (0-11) were within 0.84±0.41 and 1.33±0.73 pixels, respectively, which are significantly better than some of the other state-of-the-art algorithms. Conclusion - The results demonstrate the utility of extracting features from the entire OCT volume by treating the volume as a correlated entity and show the benefit of utilizing 3D autoencoder based regression networks for smoothing the approximated retinal layers by inducing shape based regularization constraints.

The macular inner plexiform layer thickness as an early diagnostic indicator for Parkinson's disease

Whether structural alterations of intraretinal layers are indicators for the early diagnosis of Parkinson’s disease (PD) remains unclear. We assessed the retinal layer thickness in different stages of PD and explored whether it can be an early diagnostic indicator for PD. In total, 397 [131, 146, and 120 with Hoehn-Yahr I (H-Y I), H-Y II, and H-Y III stages, respectively] patients with PD and 427 healthy controls (HCs) were enrolled. The peripapillary retinal nerve fiber layer (pRNFL), total macular retinal thickness (MRT), and macular volume (TMV) were measured by high-definition optical coherence tomography, and the macular intraretinal thickness was analyzed by the Iowa Reference Algorithms. As a result, the PD group had a significantly lower average, temporal quadrant pRNFL, MRT, and TMV than the HCs group (all p < 0.001). Moreover, the ganglion cell layer (GCL), inner plexiform layer (IPL), and outer nuclear layer were thinner in patients with PD with H-Y I, and significantly decreased as the H-Y stage increased. In addition, we observed that GCL and IPL thicknesses were both correlated with Movement Disorder Society-Unified Parkinson’s Disease Rating Scale III (MDS-UPDRS III) scores and non-motor symptoms assessment scores. Furthermore, macular IPL thickness in the superior inner (SI) quadrant (IPL-SI) had the best diagnostic performance in patients with PD with H-Y I versus HCs, with a sensitivity and specificity of 75.06% and 81.67%, respectively. In conclusion, we confirmed the retinal structure was significantly altered in patients with PD in different clinical stages, and that GCL and IPL changes occurred during early PD disease and were correlated with MDS-UPDRS III scores and non-motor symptoms assessment scores. Furthermore, macular IPL-SI thickness might be performed as an early diagnostic indicator for PD.

Therapeutic Validation of GEF-H1 Using a De Novo Designed Inhibitor in Models of Retinal Disease

Inflammation and fibrosis are important components of diseases that contribute to the malfunction of epithelia and endothelia. The Rho guanine nucleotide exchange factor (GEF) GEF-H1/ARHGEF-2 is induced in disease and stimulates inflammatory and fibrotic processes, cell migration, and metastasis. Here, we have generated peptide inhibitors to block the function of GEF-H1. Inhibitors were designed using a structural in silico approach or by isolating an inhibitory sequence from the autoregulatory C-terminal domain. Candidate inhibitors were tested for their ability to block RhoA/GEF-H1 binding in vitro, and their potency and specificity in cell-based assays. Successful inhibitors were then evaluated in models of TGFβ-induced fibrosis, LPS-stimulated endothelial cell-cell junction disruption, and cell migration. Finally, the most potent inhibitor was successfully tested in an experimental retinal disease mouse model, in which it inhibited blood vessel leakage and ameliorated retinal inflammation when treatment was initiated after disease diagnosis. Thus, an antagonist that blocks GEF-H1 signaling effectively inhibits disease features in in vitro and in vivo disease models, demonstrating that GEF-H1 is an effective therapeutic target and establishing a new therapeutic approach.

A multi-regression approach to improve optical coherence tomography diagnostic accuracy in multiple sclerosis patients without previous optic neuritis

•

OCT diagnostics for MS improved after combining macular data with compensated peripapillary RNFL.

Background

Optical coherence tomography (OCT) is a retinal imaging system that may improve the diagnosis of multiple sclerosis (MS) persons, but the evidence is currently equivocal. To assess whether compensating the peripapillary retinal nerve fiber layer (pRNFL) thickness for ocular anatomical features as well as the combination with macular layers can improve the capability of OCT in differentiating non-optic neuritis eyes of relapsing-remitting MS patients from healthy controls.

Methods

74 MS participants (n = 129 eyes) and 84 age- and sex-matched healthy controls (n = 149 eyes) were enrolled. Macular ganglion cell complex (mGCC) thickness was extracted and pRNFL measurement was compensated for ocular anatomical factors. Thickness measurements and their corresponding areas under the receiver operating characteristic curves (AUCs) were compared between groups.

Results

Participants with MS showed significantly thinner mGCC, measured and compensated pRNFL (p ≤ 0.026). Compensated pRNFL achieved better performance than measured pRNFL for MS differentiation (AUC, 0.75 vs 0.80; p = 0.020). Combining macular and compensated pRNFL parameters provided the best discrimination of MS (AUC = 0.85 vs 0.75; p < 0.001), translating to an average improvement in sensitivity of 24 percent for differentiation of MS individuals.

Conclusion

The capability of OCT in MS differentiation is made more robust by accounting OCT scans for individual anatomical differences and incorporating information from both optic disc and macular regions, representing markers of axonal damage and neuronal injury, respectively.

Intraretinal Layer Segmentation Using Cascaded Compressed U-Nets

Reliable biomarkers quantifying neurodegeneration and neuroinflammation in central nervous system disorders such as Multiple Sclerosis, Alzheimer's dementia or Parkinson's disease are an unmet clinical need. Intraretinal layer thicknesses on macular optical coherence tomography (OCT) images are promising noninvasive biomarkers querying neuroretinal structures with near cellular resolution. However, changes are typically subtle, while tissue gradients can be weak, making intraretinal segmentation a challenging task. A robust and efficient method that requires no or minimal manual correction is an unmet need to foster reliable and reproducible research as well as clinical application. Here, we propose and validate a cascaded two-stage network for intraretinal layer segmentation, with both networks being compressed versions of U-Net (CCU-INSEG). The first network is responsible for retinal tissue segmentation from OCT B-scans. The second network segments eight intraretinal layers with high fidelity. At the post-processing stage, we introduce Laplacian-based outlier detection with layer surface hole filling by adaptive non-linear interpolation. Additionally, we propose a weighted version of focal loss to minimize the foreground-background pixel imbalance in the training data. We train our method using 17,458 B-scans from patients with autoimmune optic neuropathies, i.e., multiple sclerosis, and healthy controls. Voxel-wise comparison against manual segmentation produces a mean absolute error of 2.3 μm, outperforming current state-of-the-art methods on the same data set. Voxel-wise comparison against external glaucoma data leads to a mean absolute error of 2.6 μm when using the same gold standard segmentation approach, and 3.7 μm mean absolute error in an externally segmented data set. In scans from patients with severe optic atrophy, 3.5% of B-scan segmentation results were rejected by an experienced grader, whereas this was the case in 41.4% of B-scans segmented with a graph-based reference method. The validation results suggest that the proposed method can robustly segment macular scans from eyes with even severe neuroretinal changes.

Automated retinal boundary segmentation of optical coherence tomography images using an improved Canny operator

Retinal segmentation is a prerequisite for quantifying retinal structural features and diagnosing related ophthalmic diseases. Canny operator is recognized as the best boundary detection operator so far, and is often used to obtain the initial boundary of the retina in retinal segmentation. However, the traditional Canny operator is susceptible to vascular shadows, vitreous artifacts, or noise interference in retinal segmentation, causing serious misdetection or missed detection. This paper proposed an improved Canny operator for automatic segmentation of retinal boundaries. The improved algorithm solves the problems of the traditional Canny operator by adding a multi-point boundary search step on the basis of the original method, and adjusts the convolution kernel. The algorithm was used to segment the retinal images of healthy subjects and age-related macular degeneration (AMD) patients; eleven retinal boundaries were identified and compared with the results of manual segmentation by the ophthalmologists. The average difference between the automatic and manual methods is: 2–6 microns (1–2 pixels) for healthy subjects and 3–10 microns (1–3 pixels) for AMD patients. Qualitative method is also used to verify the accuracy and stability of the algorithm. The percentage of “perfect segmentation” and “good segmentation” is 98% in healthy subjects and 94% in AMD patients. This algorithm can be used alone or in combination with other methods as an initial boundary detection algorithm. It is easy to understand and improve, and may become a useful tool for analyzing and diagnosing eye diseases.

Globally optimal OCT surface segmentation using a constrained IPM optimization

Segmentation of multiple surfaces in optical coherence tomography (OCT) images is a challenging problem, further complicated by the frequent presence of weak boundaries, varying layer thicknesses, and mutual influence between adjacent surfaces. The traditional graph-based optimal surface segmentation method has proven its effectiveness with its ability to capture various surface priors in a uniform graph model. However, its efficacy heavily relies on handcrafted features that are used to define the surface cost for the "goodness" of a surface. Recently, deep learning (DL) is emerging as a powerful tool for medical image segmentation thanks to its superior feature learning capability. Unfortunately, due to the scarcity of training data in medical imaging, it is nontrivial for DL networks to implicitly learn the global structure of the target surfaces, including surface interactions. This study proposes to parameterize the surface cost functions in the graph model and leverage DL to learn those parameters. The multiple optimal surfaces are then simultaneously detected by minimizing the total surface cost while explicitly enforcing the mutual surface interaction constraints. The optimization problem is solved by the primal-dual interior-point method (IPM), which can be implemented by a layer of neural networks, enabling efficient end-to-end training of the whole network. Experiments on spectral-domain optical coherence tomography (SD-OCT) retinal layer segmentation demonstrated promising segmentation results with sub-pixel accuracy.

DW-Net: Dynamic Multi-Hierarchical Weighting Segmentation Network for Joint Segmentation of Retina Layers With Choroid Neovascularization

Choroid neovascularization (CNV) is one of the blinding factors. The early detection and quantitative measurement of CNV are crucial for the establishment of subsequent treatment. Recently, many deep learning-based methods have been proposed for CNV segmentation. However, CNV is difficult to be segmented due to the complex structure of the surrounding retina. In this paper, we propose a novel dynamic multi-hierarchical weighting segmentation network (DW-Net) for the simultaneous segmentation of retinal layers and CNV. Specifically, the proposed network is composed of a residual aggregation encoder path for the selection of informative feature, a multi-hierarchical weighting connection for the fusion of detailed information and abstract information, and a dynamic decoder path. Comprehensive experimental results show that our proposed DW-Net achieves better performance than other state-of-the-art methods.

Quantitative assessment of retinal fluid in neovascular age-related macular degeneration under anti-VEGF therapy

The retinal world has been revolutionized by optical coherence tomography (OCT) and anti-vascular endothelial growth factor (VEGF) therapy. The numbers of intravitreal injections are on a constant rise and management in neovascular age-related macular degeneration (nAMD) is mainly driven by the qualitative assessment of macular fluid as detected on OCT scans. The presence of macular fluid, particularly subretinal fluid (SRF) and intraretinal fluid (IRF), has been used to trigger re-treatments in clinical trials and the real world. However, large discrepancies can be found between the evaluations of different readers or experts and especially small amounts of macular fluid might be missed during this process. Pixel-wise detection of macular fluid uses an entire OCT volume to calculate exact volumes of retinal fluid. While manual annotations of such pixel-wise fluid detection are unfeasible in a clinical setting, artificial intelligence (AI) is able to overcome this hurdle by providing real-time results of macular fluid in different retinal compartments. Quantitative fluid assessments have been used for various post hoc analyses of randomized controlled trials, providing novel insights into anti-VEGF treatment regimens. Nonetheless, the application of AI-algorithms in a prospective patient care setting is still limited. In this review, we discuss the use of quantitative fluid assessment in nAMD during anti-VEGF therapy and provide an outlook to novel forms of patient care with the support of AI quantifications.

Impact of intra- and subretinal fluid on vision based on volume quantification in the HARBOR trial

PURPOSE

To investigate the functional associations of intra- and subretinal fluid volumes at baseline, after the loading dose as well as fluid change after the first injection with best corrected visual acuity (BCVA) in patients with neovascular AMD (nAMD) who received an anti-VEGF treatment over 24 months.

DESIGN

Post-hoc analysis of a phase III, randomized, multicenter trial, in which ranibizumab was administered monthly or in a PRN regimen (HARBOR).

PARTICIPANTS

Study eyes of 1094 treatment-naïve nAMD patients.

METHODS

Intraretinal (IRF) and subretinal (SRF) fluid volumes were segmented automatically on monthly SD-OCT images. Fluid volumes and changes thereof were included as covariates into longitudinal mixed effects models, modeling BCVA trajectories.

MAIN OUTCOME MEASURES

BCVA estimates corresponding to baseline, follow-up and persistent IRF/SRF volumes following the loading dose; BCVA estimates of change in fluid volumes following the first injection; marginal and conditional R2.

RESULTS

Analysis of 22,494 volumetric scans revealed that foveal IRF consistently shows a negative correlation with BCVA at baseline and subsequent visits (-3.23 and -4.32 letters/100nl). After the first injection, BCVA increased by +2.13 letters/100nl decrease of foveal IRF. Persistent IRF was associated with lower baseline BCVA and less improvement. Foveal SRF correlated with better BCVA at baseline and subsequent visits (+6.52 and +1.42 letters/100nl). After the first injection, SRF decrease was associated with significant vision gain (+5.88 letters/100nl). Foveal fluid correlated more with BCVA than parafoveal IRF/SRF.

CONCLUSION

While IRF consistently correlates with decreased function and recovery throughout therapy, SRF is associated with a more pronounced functional improvement. Moreover, SRF resolution provides increased benefit. Fluid-function correlation represents an essential base for the development of personalized treatment regimens, optimizing functional outcomes and reducing treatment burden.

Automatic Segmentation of the Retinal Nerve Fiber Layer by Means of Mathematical Morphology and Deformable Models in 2D Optical Coherence Tomography Imaging

Glaucoma is a neurodegenerative disease process that leads to progressive damage of the optic nerve to produce visual impairment and blindness. Spectral-domain OCT technology enables peripapillary circular scans of the retina and the measurement of the thickness of the retinal nerve fiber layer (RNFL) for the assessment of the disease status or progression in glaucoma patients. This paper describes a new approach to segment and measure the retinal nerve fiber layer in peripapillary OCT images. The proposed method consists of two stages. In the first one, morphological operators robustly detect the coarse location of the layer boundaries, despite the speckle noise and diverse artifacts in the OCT image. In the second stage, deformable models are initialized with the results of the previous stage to perform a fine segmentation of the boundaries, providing an accurate measurement of the entire RNFL. The results of the RNFL segmentation were qualitatively assessed by ophthalmologists, and the measurements of the thickness of the RNFL were quantitatively compared with those provided by the OCT inbuilt software as well as the state-of-the-art methods.

Utility of Spectral-Domain Optical Coherence Tomography in Differentiating Papilledema From Pseudopapilledema: A Prospective Longitudinal Study

BACKGROUND

Prospective and longitudinal studies assessing the utility of spectral-domain optical coherence tomography (SD-OCT) to differentiate papilledema from pseudopapilledema are lacking. We studied the sensitivity and specificity of baseline and longitudinal changes in SD-OCT parameters with 3D segmentation software to distinguish between papilledema and pseudopapilledema in a cohort of patients referred for evaluation of undiagnosed optic disc elevation.

METHODS

Fifty-two adult patients with optic disc elevation were enrolled in a prospective longitudinal study. A diagnosis of papilledema was made when there was a change in the appearance of the optic disc elevation on fundus photographs as noted by an independent observer at or before 6 months. The degree of optic disc elevation was graded using the Frisen scale and patients with mild optic disc elevation (Frisen grades 1 and 2) were separately analyzed. SD-OCT parameters including peripapillary retinal nerve fiber layer (pRNFL), total retinal thickness (TRT), paracentral ganglion cell layer-inner plexiform layer (GCL-IPL) thickness, and optic nerve head volume (ONHV) at baseline and within 6 months of follow-up were measured.

RESULTS

Twenty-seven (52%) patients were diagnosed with papilledema and 25 (48%) with pseudopapilledema. Among patients with mild optic disc elevation (Frisen grades 1 and 2), baseline pRNFL (110.1 µm vs 151.3 µm) and change in pRNFL (ΔpRNFL) (7.3 µm vs 52.3 µm) were greater among those with papilledema. Baseline and absolute changes in TRT and ONHV were also significantly higher among patients with papilledema. The mean GCL-IPL thickness was similar at baseline, but there was a small reduction in GCL-IPL thickness among patients with papilledema. Receiver operator curves (ROCs) were generated; ΔpRNFL (0.93), ΔTRT (0.94), and ΔONHV (0.95) had the highest area under the curve (AUC).

CONCLUSIONS

The mean baseline and absolute changes in SD-OCT measurements (pRFNL, TRT, and ONHV) were significantly greater among patients with papilledema, and remained significantly greater when patients with mild optic disc elevation were separately analyzed. ROCs demonstrated that ΔpRNFL, ΔTRT, and ΔONHV have the highest AUC and are best able to differentiate between papilledema and pseudopapilledema.

Retina tissue validation of optical coherence tomography determined outer nuclear layer loss in FTLD-tau

Alzheimer's disease (AD) is associated with inner retina (nerve fiber and ganglion cell layers) thinning. In contrast, we have seen outer retina thinning driven by photoreceptor outer nuclear layer (ONL) thinning with antemortem optical coherence tomography (OCT) among patients considered to have a frontotemporal degeneration tauopathy (FTLD-Tau). Our objective was to determine if postmortem retinal tissue from FTLD-Tau patients demonstrates ONL loss observed antemortem on OCT. Two probable FTLD-Tau patients that were deeply phenotyped by clinical and genetic testing were imaged with OCT and followed to autopsy. Postmortem brain and retinal tissue were evaluated by a neuropathologist and ocular pathologist, respectively, masked to diagnosis. OCT findings were correlated with retinal histology. The two patients had autopsy-confirmed FTLD-Tau neuropathology and had antemortem OCT measurements showing ONL thinning (66.9 μm, patient #1; 74.9 μm, patient #2) below the 95% confidence interval of normal limits (75.1-120.7 μm) in our healthy control cohort. Postmortem, retinal tissue from both patients demonstrated loss of nuclei in the ONL, matching ONL loss visualized on antemortem OCT. Nuclei counts from each area of ONL loss (2 - 3 nuclei per column) seen in patient eyes were below the 95% confidence interval (4 - 8 nuclei per column for ONL) of 3 normal control retinas analyzed at the same location. Our evaluation of retinal tissue from FTLD-Tau patients confirms ONL loss seen antemortem by OCT. Continued investigation of ONL thinning as a biomarker that may distinguish FTLD-Tau from other dementias is warranted.

Assessing the validity of a cross-platform retinal image segmentation tool in normal and diseased retina

Comparing automated retinal layer segmentation using proprietary software (Heidelberg Spectralis HRA + OCT) and cross-platform Optical Coherence Tomography (OCT) segmentation software (Orion). Image segmentations of normal and diseased (iAMD, DME) eyes were performed using both softwares and then compared to the 'gold standard' of manual segmentation. A qualitative assessment and quantitative (layer volume) comparison of segmentations were performed. Segmented images from the two softwares were graded by two masked graders and in cases with difference, a senior retina specialist made a final independent decisive grading. Cross-platform software was significantly better than the proprietary software in the segmentation of NFL and INL layers in Normal eyes. It generated significantly better segmentation only for NFL in iAMD and for INL and OPL layers in DME eyes. In normal eyes, all retinal layer volumes calculated by the two softwares were moderate-strongly correlated except OUTLY. In iAMD eyes, GCIPL, INL, ONL, INLY, TRV layer volumes were moderate-strongly correlated between softwares. In eyes with DME, all layer volume values were moderate-strongly correlated between softwares. Cross-platform software can be used reliably in research settings to study the retinal layers as it compares well against manual segmentation and the commonly used proprietary software for both normal and diseased eyes.

A Hybrid Model Composed of Two Convolutional Neural Networks (CNNs) for Automatic Retinal Layer Segmentation of OCT Images in Retinitis Pigmentosa (RP)

Purpose

We propose and evaluate a hybrid model composed of two convolutional neural networks (CNNs) with different architectures for automatic segmentation of retina layers in spectral domain optical coherence tomography (SD-OCT) B-scans of retinitis pigmentosa (RP).

Methods

The hybrid model consisted of a U-Net for initial semantic segmentation and a sliding-window (SW) CNN for refinement by correcting the segmentation errors of U-Net. The U-Net construction followed Ronneberger et al. (2015) with an input image size of 256 × 32. The SW model was similar to our previously reported approach. Training image patches were generated from 480 horizontal midline B-scans obtained from 220 patients with RP and 20 normal participants. Testing images were 160 midline B-scans from a separate group of 80 patients with RP. The Spectralis segmentation of B-scans was manually corrected for the boundaries of the inner limiting membrane, inner nuclear layer, ellipsoid zone (EZ), retinal pigment epithelium, and Bruch's membrane by one grader for the training set and two for the testing set. The trained U-Net and SW, as well as the hybrid model, were used to classify all pixels in the testing B-scans. Bland–Altman and correlation analyses were conducted to compare layer boundary lines, EZ width, and photoreceptor outer segment (OS) length and area determined by the models to those by human graders.

Results

The mean times to classify a B-scan image were 0.3, 65.7, and 2.4 seconds for U-Net, SW, and the hybrid model, respectively. The mean ± SD accuracies to segment retinal layers were 90.8% ± 4.8% and 90.7% ± 4.0% for U-Net and SW, respectively. The hybrid model improved mean ± SD accuracy to 91.5% ± 4.8% (P < 0.039 vs. U-Net), resulting in an improvement in layer boundary segmentation as revealed by Bland–Altman analyses. EZ width, OS length, and OS area measured by the models were highly correlated with those measured by the human graders (r > 0.95 for EZ width; r > 0.83 for OS length; r > 0.97 for OS area; P < 0.05). The hybrid model further improved the performance of measuring retinal layer thickness by correcting misclassification of retinal layers from U-Net.

Conclusions

While the performances of U-Net and the SW model were comparable in delineating various retinal layers, U-Net was much faster than the SW model to segment B-scan images. The hybrid model that combines the two improves automatic retinal layer segmentation from OCT images in RP.

Translational Relevance

A hybrid deep machine learning model composed of CNNs with different architectures can be more effective than either model separately for automatic analysis of SD-OCT scan images, which is becoming increasingly necessary with current high-resolution, high-density volume scans.

Assignment Flow for Order-Constrained OCT Segmentation

At the present time optical coherence tomography (OCT) is among the most commonly used non-invasive imaging methods for the acquisition of large volumetric scans of human retinal tissues and vasculature. The substantial increase of accessible highly resolved 3D samples at the optic nerve head and the macula is directly linked to medical advancements in early detection of eye diseases. To resolve decisive information from extracted OCT volumes and to make it applicable for further diagnostic analysis, the exact measurement of retinal layer thicknesses serves as an essential task be done for each patient separately. However, manual examination of OCT scans is a demanding and time consuming task, which is typically made difficult by the presence of tissue-dependent speckle noise. Therefore, the elaboration of automated segmentation models has become an important task in the field of medical image processing. We propose a novel, purely data driven geometric approach to order-constrained 3D OCT retinal cell layer segmentation which takes as input data in any metric space and can be implemented using only simple, highly parallelizable operations. As opposed to many established retinal layer segmentation methods, we use only locally extracted features as input and do not employ any global shape prior. The physiological order of retinal cell layers and membranes is achieved through the introduction of a smoothed energy term. This is combined with additional regularization of local smoothness to yield highly accurate 3D segmentations. The approach thereby systematically avoid bias pertaining to global shape and is hence suited for the detection of anatomical changes of retinal tissue structure. To demonstrate its robustness, we compare two different choices of features on a data set of manually annotated 3D OCT volumes of healthy human retina. The quality of computed segmentations is compared to the state of the art in automatic retinal layer segmention as well as to manually annotated ground truth data in terms of mean absolute error and Dice similarity coefficient. Visualizations of segmented volumes are also provided.

TOPOGRAPHIC ANALYSIS OF PHOTORECEPTOR LOSS CORRELATED WITH DISEASE MORPHOLOGY IN NEOVASCULAR AGE-RELATED MACULAR DEGENERATION

PURPOSE

To quantify morphologic photoreceptor integrity during anti-vascular endothelial growth factor (anti-VEGF) therapy of neovascular age-related macular degeneration and correlate these findings with disease morphology and function.

METHODS

This presents a post hoc analysis on spectral-domain optical coherence tomography data of 185 patients, acquired at baseline, Month 3, and Month 12 in a multicenter, prospective trial. Loss of the ellipsoid zone (EZ) was manually quantified in all optical coherence tomography volumes. Intraretinal cystoid fluid, subretinal fluid (SRF), and pigment epithelial detachments were automatically segmented in the full volumes using validated deep learning methods. Spatiotemporal correlation of fluid markers with EZ integrity as well as bivariate analysis between EZ integrity and best-corrected visual acuity was performed.

RESULTS

At baseline, EZ integrity was predominantly impaired in the fovea, showing progressive recovery during anti-vascular endothelial growth factor therapy. Topographic analysis at baseline revealed EZ integrity to be more likely intact in areas with SRF and vice versa. Moreover, we observed a correlation between EZ integrity and resolution of SRF. Foveal EZ integrity correlated with best-corrected visual acuity at all timepoints.

CONCLUSION

Improvement of EZ integrity during anti-VEGF therapy of neovascular age-related macular degeneration occurred predominantly in the fovea. Photoreceptor integrity correlated with best-corrected visual acuity. Ellipsoid zone integrity was preserved in areas of SRF and showed deterioration upon SRF resolution.

Multiscale dual attention mechanism for fluid segmentation of optical coherence tomography images

Optical coherence tomography (OCT) technology can obtain a clear retinal structure map, which is greatly beneficial for the diagnosis of retinopathy. Ophthalmologists can use OCT technology to analyze information about the retina's internal structure and changes in retinal thickness. Therefore, segmentation of retinal layers in images and screening for retinal diseases have become important goals in OCT scanning. In this paper, we propose the multiscale dual attention (MSDA)-UNet network, an MSDA mechanism network for OCT lesion area segmentation. The MSDA-UNet network introduces position and multiscale channel attention modules to calculate a global reference for each pixel prediction. The network can extract the lesion area information of OCT images of different scales and perform end-to-end segmentation of the OCT retinopathy area. The network framework was trained and tested on the same OCT dataset and compared with other OCT fluid segmentation methods to assess its effectiveness.

Deep Learning-Based Retinal Nerve Fiber Layer Thickness Measurement of Murine Eyes

Purpose

To design a robust and automated estimation method for measuring the retinal nerve fiber layer (RNFL) thickness using spectral domain optical coherence tomography (SD-OCT).

Methods

We developed a deep learning-based image segmentation network for automated segmentation of the RNFL in SD-OCT B-scans of mouse eyes. In total, 5500 SD-OCT B-scans (5200 B-scans were used as training data with the remaining 300 B-scans used as testing data) were used to develop this segmentation network. Postprocessing operations were then applied on the segmentation results to fill any discontinuities or remove any speckles in the RNFL. Subsequently, a three-dimensional retina thickness map was generated by z-stacking 100 segmentation processed thickness B-scan images together. Finally, the average absolute difference between algorithm predicted RNFL thickness compared to the ground truth manual human segmentation was calculated.

Results

The proposed method achieves an average dice similarity coefficient of 0.929 in the SD-OCT segmentation task and an average absolute difference of 0.0009 mm in thickness estimation task on the basis of the testing dataset. We also evaluated our segmentation algorithm on another biological dataset with SD-OCT volumes for RNFL thickness after the optic nerve crush injury. Results were shown to be comparable between the predicted and manually measured retina thickness values.

Conclusions

Experimental results demonstrate that our automated segmentation algorithm reliably predicts the RNFL thickness in SD-OCT volumes of mouse eyes compared to laborious and more subjective manual SD-OCT RNFL segmentation.

Translational Relevance

Automated segmentation using a deep learning-based algorithm for murine eye OCT effectively and rapidly produced nerve fiber layer thicknesses comparable to manual segmentation.

QUANTITATIVE ANALYSIS OF LONGITUDINAL CHANGES IN MULTIMODAL IMAGING OF LATE-ONSET RETINAL DEGENERATION

PURPOSE

To quantitatively analyze clinically relevant features on longitudinal multimodal imaging of late-onset retinal degeneration to characterize disease progression.

METHODS

Fundus autofluorescence (FAF), infrared reflectance, and optical coherence tomography imaging of 4 patients with late-onset retinal degeneration were acquired over 3 to 15 years (20 visits total). Corresponding regions of interest were analyzed on FAF (reticular pseudodrusen [RPD], "speckled FAF," and chorioretinal atrophy) and infrared reflectance (hyporeflective RPD and target RPD) using quantitative measurements, including contour area, distance to fovea, contour overlap, retinal thickness, and texture features.

RESULTS

Cross-sectional analysis revealed a moderate correlation (RPD FAF ∩ RPD infrared reflectance = 63%) between contour area across modalities. Quantification of retinal thickness and texture analysis of areas contoured on FAF objectively differentiated the contour types. A longitudinal analysis of aligned images demonstrates that the contoured region of atrophy both encroaches toward the fovea and grows monotonically with a rate of 0.531 mm/year to 1.969 mm/year (square root of area, n = 5 eyes). A retrospective analysis of precursor lesions of atrophy reveals quantifiable progression from RPD to speckled FAF to atrophy.

CONCLUSION

Image analysis of time points before the development of atrophy reveals consistent patterns over time and space in late-onset retinal degeneration that may provide useful outcomes for this and other degenerative retinal diseases.

Weakly supervised anomaly segmentation in retinal OCT images using an adversarial learning approach

Lesion detection is a critical component of disease diagnosis, but the manual segmentation of lesions in medical images is time-consuming and experience-demanding. These issues have recently been addressed through deep learning models. However, most of the existing algorithms were developed using supervised training, which requires time-intensive manual labeling and prevents the model from detecting unaware lesions. As such, this study proposes a weakly supervised learning network based on CycleGAN for lesions segmentation in full-width optical coherence tomography (OCT) images. The model was trained to reconstruct underlying normal anatomic structures from abnormal input images, then the lesions can be detected by calculating the difference between the input and output images. A customized network architecture and a multi-scale similarity perceptual reconstruction loss were used to extend the CycleGAN model to transfer between objects exhibiting shape deformations. The proposed technique was validated using an open-source retinal OCT image dataset. Image-level anomaly detection and pixel-level lesion detection results were assessed using area-under-curve (AUC) and the Dice similarity coefficient, producing results of 96.94% and 0.8239, respectively, higher than all comparative methods. The average test time required to generate a single full-width image was 0.039 s, which is shorter than that reported in recent studies. These results indicate that our model can accurately detect and segment retinopathy lesions in real-time, without the need for supervised labeling. And we hope this method will be helpful to accelerate the clinical diagnosis process and reduce the misdiagnosis rate.

Livelayer: a semi-automatic software program for segmentation of layers and diabetic macular edema in optical coherence tomography images

Given the capacity of Optical Coherence Tomography (OCT) imaging to display structural changes in a wide variety of eye diseases and neurological disorders, the need for OCT image segmentation and the corresponding data interpretation is latterly felt more than ever before. In this paper, we wish to address this need by designing a semi-automatic software program for applying reliable segmentation of 8 different macular layers as well as outlining retinal pathologies such as diabetic macular edema. The software accommodates a novel graph-based semi-automatic method, called "Livelayer" which is designed for straightforward segmentation of retinal layers and fluids. This method is chiefly based on Dijkstra's Shortest Path First (SPF) algorithm and the Live-wire function together with some preprocessing operations on the to-be-segmented images. The software is indeed suitable for obtaining detailed segmentation of layers, exact localization of clear or unclear fluid objects and the ground truth, demanding far less endeavor in comparison to a common manual segmentation method. It is also valuable as a tool for calculating the irregularity index in deformed OCT images. The amount of time (seconds) that Livelayer required for segmentation of Inner Limiting Membrane, Inner Plexiform Layer-Inner Nuclear Layer, Outer Plexiform Layer-Outer Nuclear Layer was much less than that for the manual segmentation, 5 s for the ILM (minimum) and 15.57 s for the OPL-ONL (maximum). The unsigned errors (pixels) between the semi-automatically labeled and gold standard data was on average 2.7, 1.9, 2.1 for ILM, IPL-INL, OPL-ONL, respectively. The Bland-Altman plots indicated perfect concordance between the Livelayer and the manual algorithm and that they could be used interchangeably. The repeatability error was around one pixel for the OPL-ONL and < 1 for the other two. The unsigned errors between the Livelayer and the manual algorithm was 1.33 for ILM and 1.53 for Nerve Fiber Layer-Ganglion Cell Layer in peripapillary B-Scans. The Dice scores for comparing the two algorithms and for obtaining the repeatability on segmentation of fluid objects were at acceptable levels.

Identification and quantification of fibrotic areas in the human retina using polarization-sensitive OCT

Subretinal fibrosis is one of the most prevalent causes of blindness in the elderly population, but a true gold standard to objectively diagnose fibrosis is still lacking. Since fibrotic tissue is birefringent, it can be detected by polarization-sensitive optical coherence tomography (PS-OCT). We present a new algorithm to automatically detect, segment, and quantify fibrotic lesions within 3D data sets recorded by PS-OCT. The algorithm first compensates for the birefringence of anterior ocular tissues and then uses the uniformity of the birefringent optic axis as an indicator to identify fibrotic tissue, which is then segmented and quantified. The algorithm was applied to 3D volumes recorded in 57 eyes of 57 patients with neovascular age-related macular degeneration using a spectral domain PS-OCT system. The results of fibrosis detection were compared to the clinical diagnosis based on color fundus photography (CFP), and the precision of fibrotic area measurement was assessed by three repeated measurements in a sub-set of 15 eyes. The average standard deviation of the fibrotic area obtained in eyes with a lesion area > 0.7 mm2 was 15%. Fibrosis detection by CFP and PS-OCT agreed in 48 cases, discrepancies were only observed in cases of lesion area < 0.7 mm2. These remaining discrepancies are discussed, and a new method to treat ambiguous cases is presented.

Quantitative Assessment of Experimental Ocular Inflammatory Disease

Ocular inflammation imposes a high medical burden on patients and substantial costs on the health-care systems that mange these often chronic and debilitating diseases. Many clinical phenotypes are recognized and classifying the severity of inflammation in an eye with uveitis is an ongoing challenge. With the widespread application of optical coherence tomography in the clinic has come the impetus for more robust methods to compare disease between different patients and different treatment centers. Models can recapitulate many of the features seen in the clinic, but until recently the quality of imaging available has lagged that applied in humans. In the model experimental autoimmune uveitis (EAU), we highlight three linked clinical states that produce retinal vulnerability to inflammation, all different from healthy tissue, but distinct from each other. Deploying longitudinal, multimodal imaging approaches can be coupled to analysis in the tissue of changes in architecture, cell content and function. This can enrich our understanding of pathology, increase the sensitivity with which the impacts of therapeutic interventions are assessed and address questions of tissue regeneration and repair. Modern image processing, including the application of artificial intelligence, in the context of such models of disease can lay a foundation for new approaches to monitoring tissue health.

Temporal Relationship Between Visual Field, Retinal and Microvascular Pathology Following 125I-Plaque Brachytherapy for Uveal Melanoma

Purpose

To define the temporal relationship of vascular versus neuronal abnormalities in radiation retinopathy.

Methods

Twenty-five patients with uveal melanoma treated with brachytherapy and sixteen controls were tested. Functional outcome measures included visual acuity and threshold perimetry (HVF 10-2), while structural outcomes included retinal thickness by OCT and vascular measures by OCT angiography and digital fundus photography. The degree of structural abnormality was determined by intereye asymmetry compared with normal subject asymmetry. Diagnostic sensitivity and specificity of each measure were determined using receiver operating characteristic curves. The relationships between the outcome measures were quantified by Spearman correlation. The effect of time from brachytherapy on visual function, retinal layer thickness, and capillary density was also determined.

Results

Within the first 2 years of brachytherapy, outcome measures revealed visual field loss and microvascular abnormalities in 38% and 31% of subjects, respectively. After 2 years, they became more prevalent, increasing to 67% and 67%, respectively, as did retinal thinning (50%). Visual field loss, loss of capillary density, and inner retinal thickness were highly correlated with one another. Diagnostic sensitivity and specificity were highest for abnormalities in digital fundus photography, visual field loss within the central 10°, and decrease in vessel density.

Conclusions

Using quantitative approaches, radiation microvasculopathy and visual field defects were detected earlier than loss of inner retinal structure after brachytherapy. Strong correlations eventually developed between vascular pathology, change in retinal thickness, neuronal dysfunction, and radiation dose. Radiation-induced ischemia seems to be a primary early manifestation of radiation retinopathy preceding visual loss.

Roughness of retinal layers in Alzheimer's disease

There is growing evidence that thinned retinal regions are interspersed with thickened regions in all retinal layers of patients with Alzheimer's disease (AD), causing roughness to appear on layer thickness maps. The hypothesis is that roughness of retinal layers, assessed by the fractal dimension (FD) of their thickness maps, is an early biomarker of AD. Ten retinal layers have been studied in macular volumes of optical coherence tomography from 24 healthy volunteers and 19 patients with mild AD (Mini-Mental State Examination 23.42 ± 3.11). Results show that FD of retinal layers is greater in the AD group, the differences being statistically significant (p < 0.05). Correlation of layer FD with cognitive score, visual acuity and age reach statistical significance at 7 layers. Nearly all (44 out of 45) FD correlations among layers are positive and half of them reached statistical significance (p < 0.05). Factor analysis unveiled two independent factors identified as the dysregulation of the choroidal vascular network and the retinal inflammatory process. Conclusions: surface roughness is a holistic feature of retinal layers that can be assessed by the FD of their thickness maps and it is an early biomarker of AD.

CNN-based CP-OCT sensor integrated with a subretinal injector for retinal boundary tracking and injection guidance

SIGNIFICANCE

Subretinal injection is an effective way of delivering transplant genes and cells to treat many degenerative retinal diseases. However, the technique requires high-dexterity and microscale precision of experienced surgeons, who have to overcome the physiological hand tremor and limited visualization of the subretinal space.

AIM

To automatically guide the axial motion of microsurgical tools (i.e., a subretinal injector) with microscale precision in real time using a fiber-optic common-path swept-source optical coherence tomography distal sensor.

APPROACH

We propose, implement, and study real-time retinal boundary tracking of A-scan optical coherence tomography (OCT) images using a convolutional neural network (CNN) for automatic depth targeting of a selected retinal boundary for accurate subretinal injection guidance. A simplified 1D U-net is used for the retinal layer segmentation on A-scan OCT images. A Kalman filter, combining retinal boundary position measurement by CNN and velocity measurement by cross correlation between consecutive A-scan images, is applied to optimally estimate the retinal boundary position. Unwanted axial motions of the surgical tools are compensated by a piezoelectric linear motor based on the retinal boundary tracking.

RESULTS

CNN-based segmentation on A-scan OCT images achieves the mean unsigned error (MUE) of ∼3  pixels (8.1  μm) using an ex vivo bovine retina model. GPU parallel computing allows real-time inference (∼2  ms) and thus real-time retinal boundary tracking. Involuntary tremors, which include low-frequency draft in hundreds of micrometers and physiological tremors in tens of micrometers, are compensated effectively. The standard deviations of photoreceptor (PR) and choroid (CH) boundary positions get as low as 10.8  μm when the depth targeting is activated.

CONCLUSIONS

A CNN-based common-path OCT distal sensor successfully tracks retinal boundaries, especially the PR/CH boundary for subretinal injection, and automatically guides the tooltip's axial position in real time. The microscale depth targeting accuracy of our system shows its promising possibility for clinical application.

Capsule Network-based architectures for the segmentation of sub-retinal serous fluid in optical coherence tomography images of central serous chorioretinopathy

Central serous chorioretinopathy (CSCR) is a chorioretinal disorder of the eye characterized by serous detachment of the neurosensory retina at the posterior pole of the eye. CSCR results from the accumulation of subretinal fluid (SRF) due to idiopathic defects at the level of the retinal pigment epithelial (RPE) that allows serous fluid from the choriocapillaris to diffuse into the subretinal space between RPE and neurosensory retinal layers. This condition is presently investigated by clinicians using invasive angiography or non-invasive optical coherence tomography (OCT) imaging. OCT images provide a representation of the fluid underlying the retina, and in the absence of automated segmentation tools, currently only a qualitative assessment of the same is used to follow the progression of the disease. Automated segmentation of the SRF can prove to be extremely useful for the assessment of progression and for the timely management of CSCR. In this paper, we adopt an existing architecture called SegCaps, which is based on the recently introduced Capsule Networks concept, for the segmentation of SRF from CSCR OCT images. Furthermore, we propose an enhancement to SegCaps, which we have termed as DRIP-Caps, that utilizes the concepts of Dilation, Residual Connections, Inception Blocks, and Capsule Pooling to address the defined problem. The proposed model outperforms the benchmark UNet architecture while reducing the number of trainable parameters by 54.21%. Moreover, it reduces the computation complexity of SegCaps by reducing the number of trainable parameters by 37.85%, with competitive performance. The experiments demonstrate the generalizability of the proposed model, as evidenced by its remarkable performance even with a limited number of training samples. Graphical abstract is mandatory please provide.