Automatic Subretinal Fluid Segmentation of Retinal SD-OCT Images With Neurosensory Retinal Detachment Guided by Enface Fundus Imaging

FNeXter: A Multi-Scale Feature Fusion Network Based on ConvNeXt and Transformer for Retinal OCT Fluid Segmentation

The accurate segmentation and quantification of retinal fluid in Optical Coherence Tomography (OCT) images are crucial for the diagnosis and treatment of ophthalmic diseases such as age-related macular degeneration. However, the accurate segmentation of retinal fluid is challenging due to significant variations in the size, position, and shape of fluid, as well as their complex, curved boundaries. To address these challenges, we propose a novel multi-scale feature fusion attention network (FNeXter), based on ConvNeXt and Transformer, for OCT fluid segmentation. In FNeXter, we introduce a novel global multi-scale hybrid encoder module that integrates ConvNeXt, Transformer, and region-aware spatial attention. This module can capture long-range dependencies and non-local similarities while also focusing on local features. Moreover, this module possesses the spatial region-aware capabilities, enabling it to adaptively focus on the lesions regions. Additionally, we propose a novel self-adaptive multi-scale feature fusion attention module to enhance the skip connections between the encoder and the decoder. The inclusion of this module elevates the model's capacity to learn global features and multi-scale contextual information effectively. Finally, we conduct comprehensive experiments to evaluate the performance of the proposed FNeXter. Experimental results demonstrate that our proposed approach outperforms other state-of-the-art methods in the task of fluid segmentation.

Advancing Ocular Imaging: A Hybrid Attention Mechanism-Based U-Net Model for Precise Segmentation of Sub-Retinal Layers in OCT Images

This paper presents a novel U-Net model incorporating a hybrid attention mechanism for automating the segmentation of sub-retinal layers in Optical Coherence Tomography (OCT) images. OCT is an ophthalmology tool that provides detailed insights into retinal structures. Manual segmentation of these layers is time-consuming and subjective, calling for automated solutions. Our proposed model combines edge and spatial attention mechanisms with the U-Net architecture to improve segmentation accuracy. By leveraging attention mechanisms, the U-Net focuses selectively on image features. Extensive evaluations using datasets demonstrate that our model outperforms existing approaches, making it a valuable tool for medical professionals. The study also highlights the model's robustness through performance metrics such as an average Dice score of 94.99%, Adjusted Rand Index (ARI) of 97.00%, and Strength of Agreement (SOA) classifications like "Almost Perfect", "Excellent", and "Very Strong". This advanced predictive model shows promise in expediting processes and enhancing the precision of ocular imaging in real-world applications.

On Machine Learning in Clinical Interpretation of Retinal Diseases Using OCT Images

Optical coherence tomography (OCT) is a noninvasive imaging technique that provides high-resolution cross-sectional retina images, enabling ophthalmologists to gather crucial information for diagnosing various retinal diseases. Despite its benefits, manual analysis of OCT images is time-consuming and heavily dependent on the personal experience of the analyst. This paper focuses on using machine learning to analyse OCT images in the clinical interpretation of retinal diseases. The complexity of understanding the biomarkers present in OCT images has been a challenge for many researchers, particularly those from nonclinical disciplines. This paper aims to provide an overview of the current state-of-the-art OCT image processing techniques, including image denoising and layer segmentation. It also highlights the potential of machine learning algorithms to automate the analysis of OCT images, reducing time consumption and improving diagnostic accuracy. Using machine learning in OCT image analysis can mitigate the limitations of manual analysis methods and provide a more reliable and objective approach to diagnosing retinal diseases. This paper will be of interest to ophthalmologists, researchers, and data scientists working in the field of retinal disease diagnosis and machine learning. By presenting the latest advancements in OCT image analysis using machine learning, this paper will contribute to the ongoing efforts to improve the diagnostic accuracy of retinal diseases.

An Automatic Image Processing Method Based on Artificial Intelligence for Locating the Key Boundary Points in the Central Serous Chorioretinopathy Lesion Area

Accurately and rapidly measuring the diameter of central serous chorioretinopathy (CSCR) lesion area is the key to judge the severity of CSCR and evaluate the efficacy of the corresponding treatments. Currently, the manual measurement scheme based on a single or a small number of optical coherence tomography (OCT) B-scan images encounters the dilemma of incredibility. Although manually measuring the diameters of all OCT B-scan images of a single patient can alleviate the previous issue, the situation of inefficiency will thus arise. Additionally, manual operation is subject to subjective factors of ophthalmologists, resulting in unrepeatable measurement results. Therefore, an automatic image processing method (i.e., a joint framework) based on artificial intelligence (AI) is innovatively proposed for locating the key boundary points of CSCR lesion area to assist the diameter measurement. Firstly, the initial location module (ILM) benefiting from multitask learning is properly adjusted and tentatively achieves the preliminary location of key boundary points. Secondly, the location task is formulated as a Markov decision process, aiming at further improving the location accuracy by utilizing the single agent reinforcement learning module (SARLM). Finally, the joint framework based on the ILM and SARLM is skillfully established, in which ILM provides an initial starting point for SARLM to narrow the active region of agent, and SARLM makes up for the defect of low generalization of ILM by virtue of the independent exploration ability of agent. Experiments reveal the AI-based method which joins the multitask learning, and single agent reinforcement learning paradigms enable agents to work in local region, alleviating the time-consuming problem of SARLM, performing location task in a global scope, and improving the location accuracy of ILM, thus reflecting its effectiveness and clinical application value in the task of rapidly and accurately measuring the diameter of CSCR lesions.

Automatic segmentation of hyperreflective dots via focal priors and visual saliency

PURPOSE

Hyperreflective dots (HRDs) can be observed in spectral domain optical coherence tomography (SD-OCT), which can provide a sensitive marker in the treatment decision process. Quantitative analyses of HRDs are the key to make appropriate decisions on observation, treatment, and retreatment. The purpose of this study is to automatically and accurately segment HRDs in SD-OCT B-scans with diabetic retinopathy (DR).

METHODS

The authors propose an automatic segmentation algorithm of HRDs via focal priors and visual saliency. The algorithm is divided into three stages: segmentation of retinal layers, calculation of the multiscale local contrast saliency map, and adaptive threshold segmentation. First, a method based on improved graph search is used to segment retinal layers to obtain the region of interest (ROI) and the reflectivity estimation of the retinal pigment epithelium (RPE) layer; then, the multiscale local contrast saliency map is obtained by using a local contrast measure, which measures the dissimilarity between the current pixels and corresponding neighborhoods; finally, an adaptive threshold is applied to segment HRDs.

RESULTS

Experimental results on 20 SD-OCT B-scans demonstrate that our method is effective for HRDs segmentation. The average dice similarity coefficient (DSC) and detection accuracy are 71.12% and 85.07%, respectively.

CONCLUSIONS

The proposed method can accurately segment HRDs in SD-OCT B-scans with DR and outperforms current state-of-the-art methods. Our method can provide reliable HRDs segmentation to assist ophthalmologists in clinical diagnosis, treatment, disease monitoring, and progression.

Multi-Scale Pathological Fluid Segmentation in OCT With a Novel Curvature Loss in Convolutional Neural Network

The segmentation of pathological fluid lesions in optical coherence tomography (OCT), including intraretinal fluid, subretinal fluid, and pigment epithelial detachment, is of great importance for the diagnosis and treatment of various eye diseases such as neovascular age-related macular degeneration and diabetic macular edema. Although significant progress has been achieved with the rapid development of fully convolutional neural networks (FCN) in recent years, some important issues remain unsolved. First, pathological fluid lesions in OCT show large variations in location, size, and shape, imposing challenges on the design of FCN architecture. Second, fluid lesions should be continuous regions without holes inside. But the current architectures lack the capability to preserve the shape prior information. In this study, we introduce an FCN architecture for the simultaneous segmentation of three types of pathological fluid lesions in OCT. First, attention gate and spatial pyramid pooling modules are employed to improve the ability of the network to extract multi-scale objects. Then, we introduce a novel curvature regularization term in the loss function to incorporate shape prior information. The proposed method was extensively evaluated on public and clinical datasets with significantly improved performance compared with the state-of-the-art methods.

Livelayer: a semi-automatic software program for segmentation of layers and diabetic macular edema in optical coherence tomography images

Given the capacity of Optical Coherence Tomography (OCT) imaging to display structural changes in a wide variety of eye diseases and neurological disorders, the need for OCT image segmentation and the corresponding data interpretation is latterly felt more than ever before. In this paper, we wish to address this need by designing a semi-automatic software program for applying reliable segmentation of 8 different macular layers as well as outlining retinal pathologies such as diabetic macular edema. The software accommodates a novel graph-based semi-automatic method, called "Livelayer" which is designed for straightforward segmentation of retinal layers and fluids. This method is chiefly based on Dijkstra's Shortest Path First (SPF) algorithm and the Live-wire function together with some preprocessing operations on the to-be-segmented images. The software is indeed suitable for obtaining detailed segmentation of layers, exact localization of clear or unclear fluid objects and the ground truth, demanding far less endeavor in comparison to a common manual segmentation method. It is also valuable as a tool for calculating the irregularity index in deformed OCT images. The amount of time (seconds) that Livelayer required for segmentation of Inner Limiting Membrane, Inner Plexiform Layer-Inner Nuclear Layer, Outer Plexiform Layer-Outer Nuclear Layer was much less than that for the manual segmentation, 5 s for the ILM (minimum) and 15.57 s for the OPL-ONL (maximum). The unsigned errors (pixels) between the semi-automatically labeled and gold standard data was on average 2.7, 1.9, 2.1 for ILM, IPL-INL, OPL-ONL, respectively. The Bland-Altman plots indicated perfect concordance between the Livelayer and the manual algorithm and that they could be used interchangeably. The repeatability error was around one pixel for the OPL-ONL and < 1 for the other two. The unsigned errors between the Livelayer and the manual algorithm was 1.33 for ILM and 1.53 for Nerve Fiber Layer-Ganglion Cell Layer in peripapillary B-Scans. The Dice scores for comparing the two algorithms and for obtaining the repeatability on segmentation of fluid objects were at acceptable levels.

RetFluidNet: Retinal Fluid Segmentation for SD-OCT Images Using Convolutional Neural Network

Age-related macular degeneration (AMD) is one of the leading causes of irreversible blindness and is characterized by fluid-related accumulations such as intra-retinal fluid (IRF), subretinal fluid (SRF), and pigment epithelial detachment (PED). Spectral-domain optical coherence tomography (SD-OCT) is the primary modality used to diagnose AMD, yet it does not have algorithms that directly detect and quantify the fluid. This work presents an improved convolutional neural network (CNN)-based architecture called RetFluidNet to segment three types of fluid abnormalities from SD-OCT images. The model assimilates different skip-connect operations and atrous spatial pyramid pooling (ASPP) to integrate multi-scale contextual information; thus, achieving the best performance. This work also investigates between consequential and comparatively inconsequential hyperparameters and skip-connect techniques for fluid segmentation from the SD-OCT image to indicate the starting choice for future related researches. RetFluidNet was trained and tested on SD-OCT images from 124 patients and achieved an accuracy of 80.05%, 92.74%, and 95.53% for IRF, PED, and SRF, respectively. RetFluidNet showed significant improvement over competitive works to be clinically applicable in reasonable accuracy and time efficiency. RetFluidNet is a fully automated method that can support early detection and follow-up of AMD.

Weakly supervised serous retinal detachment segmentation in SD-OCT images by two-stage learning

Automated lesion segmentation is one of the important tasks for the quantitative assessment of retinal diseases in SD-OCT images. Recently, deep convolutional neural networks (CNN) have shown promising advancements in the field of automated image segmentation, whereas they always benefit from large-scale datasets with high-quality pixel-wise annotations. Unfortunately, obtaining accurate annotations is expensive in both human effort and finance. In this paper, we propose a weakly supervised two-stage learning architecture to detect and further segment central serous chorioretinopathy (CSC) retinal detachment with only image-level annotations. Specifically, in the first stage, a Located-CNN is designed to detect the location of lesion regions in the whole SD-OCT retinal images, and highlight the distinguishing regions. To generate available a pseudo pixel-level label, the conventional level set method is employed to refine the distinguishing regions. In the second stage, we customize the active-contour loss function in deep networks to achieve the effective segmentation of the lesion area. A challenging dataset is used to evaluate our proposed method, and the results demonstrate that the proposed method consistently outperforms some current models trained with a different level of supervision, and is even as competitive as those relying on stronger supervision. To our best knowledge, we are the first to achieve CSC segmentation in SD-OCT images using weakly supervised learning, which can greatly reduce the labeling efforts.

Lightweight Learning-Based Automatic Segmentation of Subretinal Blebs on Microscope-Integrated Optical Coherence Tomography Images

PURPOSE

Subretinal injections of therapeutics are commonly used to treat ocular diseases. Accurate dosing of therapeutics at target locations is crucial but difficult to achieve using subretinal injections due to leakage, and there is no method available to measure the volume of therapeutics successfully administered to the subretinal location during surgery. Here, we introduce the first automatic method for quantifying the volume of subretinal blebs, using porcine eyes injected with Ringer's lactate solution as samples.

DESIGN

Ex vivo animal study.

METHODS

Microscope-integrated optical coherence tomography was used to obtain 3D visualization of subretinal blebs in porcine eyes at Duke Eye Center. Two different injection phases were imaged and analyzed in 15 eyes (30 volumes), selected from a total of 37 eyes. The inclusion/exclusion criteria were set independently from the algorithm-development and testing team. A novel lightweight, deep learning-based algorithm was designed to segment subretinal bleb boundaries. A cross-validation method was used to avoid selection bias. An ensemble-classifier strategy was applied to generate final results for the test dataset.

RESULTS

The algorithm performs notably better than 4 other state-of-the-art deep learning-based segmentation methods, achieving an F1 score of 93.86 ± 1.17% and 96.90 ± 0.59% on the independent test data for entry and full blebs, respectively.

CONCLUSION

The proposed algorithm accurately segmented the volumetric boundaries of Ringer's lactate solution delivered into the subretinal space of porcine eyes with robust performance and real-time speed. This is the first step for future applications in computer-guided delivery of therapeutics into the subretinal space in human subjects.

Deriving external forces via convolutional neural networks for biomedical image segmentation

Active contours, or snakes, are widely applied on biomedical image segmentation. They are curves defined within an image domain that can move to object boundaries under the influence of internal forces and external forces, in which the internal forces are generally computed from curves themselves and external forces from image data. Designing external forces properly is a key point with active contour algorithms since the external forces play a leading role in the evolution of active contours. One of most popular external forces for active contour models is gradient vector flow (GVF). However, GVF is sensitive to noise and false edges, which limits its application area. To handle this problem, in this paper, we propose using GVF as reference to train a convolutional neural network to derive an external force. The derived external force is then integrated into the active contour models for curve evolution. Three clinical applications, segmentation of optic disk in fundus images, fluid in retinal optical coherence tomography images and fetal head in ultrasound images, are employed to evaluate the proposed method. The results show that the proposed method is very promising since it achieves competitive performance for different tasks compared to the state-of-the-art algorithms.

RETOUCH: The Retinal OCT Fluid Detection and Segmentation Benchmark and Challenge

Retinal swelling due to the accumulation of fluid is associated with the most vision-threatening retinal diseases. Optical coherence tomography (OCT) is the current standard of care in assessing the presence and quantity of retinal fluid and image-guided treatment management. Deep learning methods have made their impact across medical imaging, and many retinal OCT analysis methods have been proposed. However, it is currently not clear how successful they are in interpreting the retinal fluid on OCT, which is due to the lack of standardized benchmarks. To address this, we organized a challenge RETOUCH in conjunction with MICCAI 2017, with eight teams participating. The challenge consisted of two tasks: fluid detection and fluid segmentation. It featured for the first time: all three retinal fluid types, with annotated images provided by two clinical centers, which were acquired with the three most common OCT device vendors from patients with two different retinal diseases. The analysis revealed that in the detection task, the performance on the automated fluid detection was within the inter-grader variability. However, in the segmentation task, fusing the automated methods produced segmentations that were superior to all individual methods, indicating the need for further improvements in the segmentation performance.

Double-branched and area-constraint fully convolutional networks for automated serous retinal detachment segmentation in SD-OCT images

BACKGROUND AND OBJECTIVE

Quantitative assessment of subretinal fluid in spectral domain optical coherence tomography (SD-OCT) images is crucial for the diagnosis of central serous chorioretinopathy. For the subretinal fluid segmentation, the traditional methods need to segment retinal layers and then segment subretinal fluid. The layer segmentation has a high influence on subretinal fluid segmentation, so we aim to develop a deep learning model to segment subretinal fluid automatically without layer segmentation.

METHODS

In this paper, we propose a novel image-to-image double-branched and area-constraint fully convolutional networks (DA-FCN) for segmenting subretinal fluid in SD-OCT images. Firstly, the dataset is extended by mirroring image, which helps to overcome the over-fitting problem in the training stage. Then, double-branched structures are designed to learn the shallow coarse and deep representations from the SD-OCT images. DA-FCN model is directly trained using the image and corresponding pixel-based ground truth. Finally, we introduce a novel supervision mechanism by jointing the area loss L_A with the softmax loss L_S to learn more representative features.

RESULTS

The testing dataset with 52 SD-OCT volumes from 35 eyes of 35 patients is used for the evaluation of the proposed algorithm based on the cross-validation method. For the three criterions, including the true positive volume fraction, dice similarity coefficient, and positive predicative value, our method can obtain the results of (1) 94.3, 95.3, and 96.4 for dataset 1; (2) 97.3, 95.3, and 93.4 for dataset 2; (3) 93.0, 92.8, and 92.8 for dataset 3; (4) 89.7, 90.1, and 92.6 for dataset 4.

CONCLUSION

In this work, we propose a novel fully convolutional network for the automatic segmentation of the subretinal fluid. By constructing the double branched structures and area constraint term, our method shows higher segmentation accuracy without layer segmentation compared with other methods.

Intraretinal fluid identification via enhanced maps using optical coherence tomography images

Nowadays, among the main causes of blindness in developed countries are age-related macular degeneration (AMD) and the diabetic macular edema (DME). Both diseases present, as a common symptom, the appearance of cystoid fluid regions inside the retinal layers. Optical coherence tomography (OCT) image modality was one of the main medical imaging techniques for the early diagnosis and monitoring of AMD and DME via this intraretinal fluid detection and characterization. We present a novel methodology to identify these fluid accumulations by means of generating binary maps (offering a direct representation of these areas) and heat maps (containing the region confidence). To achieve this, a set of 312 intensity and texture-based features were studied. The most relevant features were selected using the sequential forward selection (SFS) strategy and tested with three archetypal classifiers: LDC, SVM and Parzen window. Finally, the most proficient classifier is used to create the proposed maps. All of the tested classifiers returned satisfactory results, the best classifier achieving a mean test accuracy higher than 94% in all of the experiments. The suitability of the maps was evaluated in a context of a screening issue with three different datasets obtained with two different devices, testing the capabilities of the system to work independently of the used OCT device. The experiments with the map creation were performed using 323 OCT images. Using only the binary maps, more than 91.33% of the images were correctly classified. With only the heat maps, the proposed methodology correctly separated 93.50% of the images.

Robust layer segmentation of esophageal OCT images based on graph search using edge-enhanced weights

Automatic segmentation of esophageal layers in OCT images is crucial for studying esophageal diseases and computer-assisted diagnosis. This work aims to improve the current techniques to increase the accuracy and robustness for esophageal OCT image segmentation. A two-step edge-enhanced graph search (EEGS) framework is proposed in this study. Firstly, a preprocessing scheme is applied to suppress speckle noise and remove the disturbance in the esophageal structure. Secondly, the image is formulated into a graph and layer boundaries are located by graph search. In this process, we propose an edge-enhanced weight matrix for the graph by combining the vertical gradients with a Canny edge map. Experiments on esophageal OCT images from guinea pigs demonstrate that the EEGS framework is more robust and more accurate than the current segmentation method. It can be potentially useful for the early detection of esophageal diseases.

Automatic segmentation of abnormal capillary nonperfusion regions in optical coherence tomography angiography images using marker-controlled watershed algorithm

Diabetic retinopathy (DR) is one of the most complications of diabetes. It is a progressive disease leading to significant vision loss in the patients. Abnormal capillary nonperfusion (CNP) regions are one of the important characteristics of DR increasing with its progression. Therefore, automatic segmentation and quantification of abnormal CNP regions can be helpful to monitor the patient's treatment process. We propose an automatic method for segmentation of abnormal CNP regions on the superficial and deep capillary plexuses of optical coherence tomography angiography (OCTA) images using the marker-controlled watershed algorithm. The proposed method has three main steps. In the first step, original images are enhanced using the vesselness filter and then foreground and background marker images are computed. In the second step, abnormal CNP region candidates are segmented using the marker-controlled watershed algorithm, and in the third step, the candidates are modeled using an undirected weighted graph and finally, by applying merging and removing procedures correct abnormal CNP regions are identified. The proposed method was evaluated on a dataset with 36 normal and diabetic subjects using the ground truth obtained by two observers. The results show the proposed method outperformed some of the state-of-the-art methods on the superficial and deep capillary plexuses according to the most important metrics.

Deep learning approach for the detection and quantification of intraretinal cystoid fluid in multivendor optical coherence tomography

We developed a deep learning algorithm for the automatic segmentation and quantification of intraretinal cystoid fluid (IRC) in spectral domain optical coherence tomography (SD-OCT) volumes independent of the device used for acquisition. A cascade of neural networks was introduced to include prior information on the retinal anatomy, boosting performance significantly. The proposed algorithm approached human performance reaching an overall Dice coefficient of 0.754 ± 0.136 and an intraclass correlation coefficient of 0.936, for the task of IRC segmentation and quantification, respectively. The proposed method allows for fast quantitative IRC volume measurements that can be used to improve patient care, reduce costs, and allow fast and reliable analysis in large population studies.