LF-UNet - A novel anatomical-aware dual-branch cascaded deep neural network for segmentation of retinal layers and fluid from optical coherence tomography images

Joint segmentation of retinal layers and fluid lesions in optical coherence tomography with cross-dataset learning

BACKGROUND AND OBJECTIVES

Age-related macular degeneration (AMD) is the leading cause of irreversible vision loss among people over 50 years old, which manifests in the retina through various changes of retinal layers and pathological lesions. The accurate segmentation of optical coherence tomography (OCT) image features is crucial for the identification and tracking of AMD. Although the recent developments in deep neural network have brought profound progress in this area, accurately segmenting retinal layers and pathological lesions remains a challenging task because of the interaction between these two tasks.

METHODS

In this study, we propose a three-branch, hierarchical multi-task framework that enables joint segmentation of seven retinal layers and three types of pathological lesions. A regression guidance module is introduced to provide explicit shape guidance between sub-tasks. We also propose a cross-dataset learning strategy to leverage public datasets with partial labels. The proposed framework was evaluated on a clinical dataset consisting of 140 OCT B-scans with pixel-level annotations of seven retinal layers and three types of lesions. Additionally, we compared its performance with the state-of-the-art methods on two public datasets.

RESULTS

Comprehensive ablation showed that the proposed hierarchical architecture significantly improved performance for most retinal layers and pathological lesions, achieving the highest mean DSC of 76.88 %. The IRF also achieved the best performance with a DSC of 68.15 %. Comparative studies demonstrated that the hierarchical multi-task architecture could significantly enhance segmentation accuracy and outperform state-of-the-art methods.

CONCLUSION

The proposed framework could also be generalized to other medical image segmentation tasks with interdependent relationships.

Boundary-Repairing Dual-Path Network for Retinal Layer Segmentation in OCT Image with Pigment Epithelial Detachment

Automatic retinal layer segmentation in optical coherence tomography (OCT) images is crucial for the diagnosis of ocular diseases. Currently, automatic retinal layer segmentation works well with normal OCT images. However, pigment epithelial detachment (PED) dramatically alters the retinal structure, causing blurred boundaries and partial disappearance of the Bruch's Membrane (BM), thus posing challenges to the segmentation. To tackle these problems, we propose a novel dual-path U-shaped network for simultaneous layer segmentation and boundary regression. This network first designs a feature interaction fusion (FIF) module to strengthen the boundary shape constraints in the layer path. To address the challenge posed by partial BM disappearance and boundary-blurring, we propose a layer boundary repair (LBR) module. This module aims to use contrastive loss to enhance the confidence of blurred boundary regions and refine the segmentation of layer boundaries through the re-prediction head. In addition, we introduce a novel bilateral threshold distance map (BTDM) designed for the boundary path. The BTDM serves to emphasize information within boundary regions. This map, combined with the updated probability map, culminates in topology-guaranteed segmentation results achieved through a topology correction (TC) module. We investigated the proposed network on two severely deformed datasets (i.e., OCTA-500 and Aier-PED) and one slightly deformed dataset (i.e., DUKE). The proposed method achieves an average Dice score of 94.26% on the OCTA-500 dataset, which was 1.5% higher than BAU-Net and outperformed other methods. In the DUKE and Aier-PED datasets, the proposed method achieved average Dice scores of 91.65% and 95.75%, respectively.

Fully Automated Detection of the Appendix Using U-Net Deep Learning Architecture in CT Scans

Background: The accurate segmentation of the appendix with well-defined boundaries is critical for diagnosing conditions such as acute appendicitis. The manual identification of the appendix is time-consuming and highly dependent on the expertise of the radiologist. Method: In this study, we propose a fully automated approach to the detection of the appendix using deep learning architecture based on the U-Net with specific training parameters in CT scans. The proposed U-Net architecture is trained on an annotated original dataset of abdominal CT scans to segment the appendix efficiently and with high performance. In addition, to extend the training set, data augmentation techniques are applied for the created dataset. Results: In experimental studies, the proposed U-Net model is implemented using hyperparameter optimization and the performance of the model is evaluated using key metrics to measure diagnostic reliability. The trained U-Net model achieved the segmentation performance for the detection of the appendix in CT slices with a Dice Similarity Coefficient (DSC), Volumetric Overlap Error (VOE), Average Symmetric Surface Distance (ASSD), Hausdorff Distance 95 (HD95), Precision (PRE) and Recall (REC) of 85.94%, 23.29%, 1.24 mm, 5.43 mm, 86.83% and 86.62%, respectively. Moreover, our model outperforms other methods by leveraging the U-Net's ability to capture spatial context through encoder-decoder structures and skip connections, providing a correct segmentation output. Conclusions: The proposed U-Net model showed reliable performance in segmenting the appendix region, with some limitations in cases where the appendix was close to other structures. These improvements highlight the potential of deep learning to significantly improve clinical outcomes in appendix detection.

Image segmentation of mouse eye in vivo with optical coherence tomography based on Bayesian classification

OBJECTIVES

Optical coherence tomography (OCT) is a new imaging technology that uses an optical analog of ultrasound imaging for biological tissues. Image segmentation plays an important role in dealing with quantitative analysis of medical images.

METHODS

We have proposed a novel framework to deal with the low intensity problem, based on the labeled patches and Bayesian classification (LPBC) model. The proposed method includes training and testing phases. During the training phase, firstly, we manually select the sub-images of background and Region of Interest (ROI) from the training image, and then extract features by patches. Finally, we train the Bayesian model with the features. The segmentation threshold of each patch is computed by the learned Bayesian model.

RESULTS

In addition, we have collected a new dataset of mouse eyes in vivo with OCT, named MEVOCT, which can be found at URL https://17861318579.github.io/LPBC. MEVOCT consists of 20 high-resolution images. The resolution of every image is 2048 × 2048 pixels.

CONCLUSIONS

The experimental results demonstrate the effectiveness of the LPBC method on the new MEVOCT dataset. The ROI segmentation is of great importance for the distortion correction.

AtPCa-Net: anatomical-aware prostate cancer detection network on multi-parametric MRI

Multi-parametric MRI (mpMRI) is widely used for prostate cancer (PCa) diagnosis. Deep learning models show good performance in detecting PCa on mpMRI, but domain-specific PCa-related anatomical information is sometimes overlooked and not fully explored even by state-of-the-art deep learning models, causing potential suboptimal performances in PCa detection. Symmetric-related anatomical information is commonly used when distinguishing PCa lesions from other visually similar but benign prostate tissue. In addition, different combinations of mpMRI findings are used for evaluating the aggressiveness of PCa for abnormal findings allocated in different prostate zones. In this study, we investigate these domain-specific anatomical properties in PCa diagnosis and how we can adopt them into the deep learning framework to improve the model's detection performance. We propose an anatomical-aware PCa detection Network (AtPCa-Net) for PCa detection on mpMRI. Experiments show that the AtPCa-Net can better utilize the anatomical-related information, and the proposed anatomical-aware designs help improve the overall model performance on both PCa detection and patient-level classification.

Early inner plexiform layer thinning and retinal nerve fiber layer thickening in excitotoxic retinal injury using deep learning-assisted optical coherence tomography

Excitotoxicity from the impairment of glutamate uptake constitutes an important mechanism in neurodegenerative diseases such as Alzheimer's, multiple sclerosis, and Parkinson's disease. Within the eye, excitotoxicity is thought to play a critical role in retinal ganglion cell death in glaucoma, diabetic retinopathy, retinal ischemia, and optic nerve injury, yet how excitotoxic injury impacts different retinal layers is not well understood. Here, we investigated the longitudinal effects of N-methyl-D-aspartate (NMDA)-induced excitotoxic retinal injury in a rat model using deep learning-assisted retinal layer thickness estimation. Before and after unilateral intravitreal NMDA injection in nine adult Long Evans rats, spectral-domain optical coherence tomography (OCT) was used to acquire volumetric retinal images in both eyes over 4 weeks. Ten retinal layers were automatically segmented from the OCT data using our deep learning-based algorithm. Retinal degeneration was evaluated using layer-specific retinal thickness changes at each time point (before, and at 3, 7, and 28 days after NMDA injection). Within the inner retina, our OCT results showed that retinal thinning occurred first in the inner plexiform layer at 3 days after NMDA injection, followed by the inner nuclear layer at 7 days post-injury. In contrast, the retinal nerve fiber layer exhibited an initial thickening 3 days after NMDA injection, followed by normalization and thinning up to 4 weeks post-injury. Our results demonstrated the pathological cascades of NMDA-induced neurotoxicity across different layers of the retina. The early inner plexiform layer thinning suggests early dendritic shrinkage, whereas the initial retinal nerve fiber layer thickening before subsequent normalization and thinning indicates early inflammation before axonal loss and cell death. These findings implicate the inner plexiform layer as an early imaging biomarker of excitotoxic retinal degeneration, whereas caution is warranted when interpreting the ganglion cell complex combining retinal nerve fiber layer, ganglion cell layer, and inner plexiform layer thicknesses in conventional OCT measures. Deep learning-assisted retinal layer segmentation and longitudinal OCT monitoring can help evaluate the different phases of retinal layer damage upon excitotoxicity.

Retinal vessel segmentation method based on RSP-SA Unet network

Segmenting retinal vessels plays a significant role in the diagnosis of fundus disorders. However, there are two problems in the retinal vessel segmentation methods. First, fine-grained features of fine blood vessels are difficult to be extracted. Second, it is easy to lose track of the details of blood vessel edges. To solve the problems above, the Residual SimAM Pyramid-Spatial Attention Unet (RSP-SA Unet) is proposed, in which the encoding, decoding, and upsampling layers of the Unet are mainly improved. Firstly, the RSP structure proposed in this paper approximates a residual structure combined with SimAM and Pyramid Segmentation Attention (PSA), which is applied to the encoding and decoding parts to extract multi-scale spatial information and important features across dimensions at a finer level. Secondly, the spatial attention (SA) is used in the upsampling layer to perform multi-attention mapping on the input feature map, which could enhance the segmentation effect of small blood vessels with low contrast. Finally, the RSP-SA Unet is verified on the CHASE_DB1, DRIVE, and STARE datasets, and the segmentation accuracy (ACC) of the RSP-SA Unet could reach 0.9763, 0.9704, and 0.9724, respectively. Area under the ROC curve (AUC) could reach 0.9896, 0.9858, and 0.9906, respectively. The RSP-SA Unet overall performance is better than the comparison methods.

A Review of Machine Learning Algorithms for Retinal Cyst Segmentation on Optical Coherence Tomography

Optical coherence tomography (OCT) is an emerging imaging technique for diagnosing ophthalmic diseases and the visual analysis of retinal structure changes, such as exudates, cysts, and fluid. In recent years, researchers have increasingly focused on applying machine learning algorithms, including classical machine learning and deep learning methods, to automate retinal cysts/fluid segmentation. These automated techniques can provide ophthalmologists with valuable tools for improved interpretation and quantification of retinal features, leading to more accurate diagnosis and informed treatment decisions for retinal diseases. This review summarized the state-of-the-art algorithms for the three essential steps of cyst/fluid segmentation: image denoising, layer segmentation, and cyst/fluid segmentation, while emphasizing the significance of machine learning techniques. Additionally, we provided a summary of the publicly available OCT datasets for cyst/fluid segmentation. Furthermore, the challenges, opportunities, and future directions of artificial intelligence (AI) in OCT cyst segmentation are discussed. This review is intended to summarize the key parameters for the development of a cyst/fluid segmentation system and the design of novel segmentation algorithms and has the potential to serve as a valuable resource for imaging researchers in the development of assessment systems related to ocular diseases exhibiting cyst/fluid in OCT imaging.

Segmentation-guided domain adaptation and data harmonization of multi-device retinal optical coherence tomography using cycle-consistent generative adversarial networks

BACKGROUND

Medical images such as Optical Coherence Tomography (OCT) images acquired from different devices may show significantly different intensity profiles. An automatic segmentation model trained on images from one device may perform poorly when applied to images acquired using another device, resulting in a lack of generalizability. This study addresses this issue using domain adaptation methods improved by Cycle-Consistent Generative Adversarial Networks (CycleGAN), especially when the ground-truth labels are only available in the source domain.

METHODS

A two-stage pipeline is proposed to generate segmentation in the target domain. The first stage involves the training of a state-of-the-art segmentation model in the source domain. The second stage aims to adapt the images from the target domain to the source domain. The adapted target domain images are segmented using the model in the first stage. Ablation tests were performed with integration of different loss functions, and the statistical significance of these models is reported. Both the segmentation performance and the adapted image quality metrics were evaluated.

RESULTS

Regarding the segmentation Dice score, the proposed model ssppg achieves a significant improvement of 46.24% compared to without adaptation and reaches 87.4% of the upper limit of the segmentation performance. Furthermore, image quality metrics, including FID and KID scores, indicate that adapted images with better segmentation also have better image qualities.

CONCLUSION

The proposed method demonstrates the effectiveness of segmentation-driven domain adaptation in retinal imaging processing. It reduces the labor cost of manual labeling, incorporates prior anatomic information to regulate and guide domain adaptation, and provides insights into improving segmentation qualities in image domains without labels.

A vision transformer architecture for the automated segmentation of retinal lesions in spectral domain optical coherence tomography images

Neovascular age-related macular degeneration (nAMD) is one of the major causes of irreversible blindness and is characterized by accumulations of different lesions inside the retina. AMD biomarkers enable experts to grade the AMD and could be used for therapy prognosis and individualized treatment decisions. In particular, intra-retinal fluid (IRF), sub-retinal fluid (SRF), and pigment epithelium detachment (PED) are prominent biomarkers for grading neovascular AMD. Spectral-domain optical coherence tomography (SD-OCT) revolutionized nAMD early diagnosis by providing cross-sectional images of the retina. Automatic segmentation and quantification of IRF, SRF, and PED in SD-OCT images can be extremely useful for clinical decision-making. Despite the excellent performance of convolutional neural network (CNN)-based methods, the task still presents some challenges due to relevant variations in the location, size, shape, and texture of the lesions. This work adopts a transformer-based method to automatically segment retinal lesion from SD-OCT images and qualitatively and quantitatively evaluate its performance against CNN-based methods. The method combines the efficient long-range feature extraction and aggregation capabilities of Vision Transformers with data-efficient training of CNNs. The proposed method was tested on a private dataset containing 3842 2-dimensional SD-OCT retina images, manually labeled by experts of the Franziskus Eye-Center, Muenster. While one of the competitors presents a better performance in terms of Dice score, the proposed method is significantly less computationally expensive. Thus, future research will focus on the proposed network's architecture to increase its segmentation performance while maintaining its computational efficiency.

Reverse translation of artificial intelligence in glaucoma: Connecting basic science with clinical applications

Artificial intelligence (AI) has been approved for biomedical research in diverse areas from bedside clinical studies to benchtop basic scientific research. For ophthalmic research, in particular glaucoma, AI applications are rapidly growing for potential clinical translation given the vast data available and the introduction of federated learning. Conversely, AI for basic science remains limited despite its useful power in providing mechanistic insight. In this perspective, we discuss recent progress, opportunities, and challenges in the application of AI in glaucoma for scientific discoveries. Specifically, we focus on the research paradigm of reverse translation, in which clinical data are first used for patient-centered hypothesis generation followed by transitioning into basic science studies for hypothesis validation. We elaborate on several distinctive areas of research opportunities for reverse translation of AI in glaucoma including disease risk and progression prediction, pathology characterization, and sub-phenotype identification. We conclude with current challenges and future opportunities for AI research in basic science for glaucoma such as inter-species diversity, AI model generalizability and explainability, as well as AI applications using advanced ocular imaging and genomic data.

Joint segmentation of retinal layers and macular edema in optical coherence tomography scans based on RLMENet

PURPOSE

The segmentation of retinal layers and fluid lesions on retinal optical coherence tomography (OCT) images is an important component of screening and diagnosing retinopathy in clinical ophthalmic treatment. We designed a novel network for accurate segmentation of the seven tissue layers of the retina and lesion areas of diabetic macular edema (DME), which can assist doctors to quantitatively analyze the disease.

METHODS

In this article, we propose the Retinal Layer Macular Edema Network (RLMENet) model to achieve end-to-end joint segmentation of retinal layers and fluids. The network employs dense multi-scale attention to enhance the extraction of retinal layer and fluid detail information and achieve efficient long-range modeling, which improves the receptive field and obtains multi-scale features. As the more complex decoder part is designed, which integrates more low-level feature information on the decoder side, more features are extracted to gradually restore the resolution of the feature map and improve the segmentation accuracy.

RESULTS

We used part of the OCT2017 dataset to train and verify the model to divide the data into a training set, validation set, and test set and set it to a 7:2:1 ratio. We evaluated our method on the ISIC2017 dataset. Experimental results showed that the RLMENet model designed in this work can accurately segment seven retinal tissue layers and DME lesions on the retinal OCT dataset. Finally, the MIoU value in the test set reached 86.55%. The model can be extended to other medical image segmentation datasets to achieve better segmentation performance.

CONCLUSIONS

The proposed method was superior to the existing segmentation methods, achieved a more refined segmentation effect, and provided an auxiliary analysis tool for clinical diagnosis and treatment.

Image-based cross-calibration method for multiple spectrometer-based OCT

A fast and practical computational cross-calibration of multiple spectrometers is described. A signal correlation matrix (CM) can be constructed from paired B-scans in a multiple-spectrometer optical coherence tomography (OCT), where the wavelength-corresponding pixels are indicated by high cross correlation. The CM can be used to either guide the physical alignment of spectrometers or to numerically match the spectra in the post-process. The performance is comparable to the previously reported optimization approach, as demonstrated by the mirror tests, qualitative comparison of OCT and optical coherence tomography angiography (OCTA) images, and quantitative comparison of image metrics.

Recent Advanced Deep Learning Architectures for Retinal Fluid Segmentation on Optical Coherence Tomography Images

With non-invasive and high-resolution properties, optical coherence tomography (OCT) has been widely used as a retinal imaging modality for the effective diagnosis of ophthalmic diseases. The retinal fluid is often segmented by medical experts as a pivotal biomarker to assist in the clinical diagnosis of age-related macular diseases, diabetic macular edema, and retinal vein occlusion. In recent years, the advanced machine learning methods, such as deep learning paradigms, have attracted more and more attention from academia in the retinal fluid segmentation applications. The automatic retinal fluid segmentation based on deep learning can improve the semantic segmentation accuracy and efficiency of macular change analysis, which has potential clinical implications for ophthalmic pathology detection. This article summarizes several different deep learning paradigms reported in the up-to-date literature for the retinal fluid segmentation in OCT images. The deep learning architectures include the backbone of convolutional neural network (CNN), fully convolutional network (FCN), U-shape network (U-Net), and the other hybrid computational methods. The article also provides a survey on the prevailing OCT image datasets used in recent retinal segmentation investigations. The future perspectives and some potential retinal segmentation directions are discussed in the concluding context.

Clinical explainable differential diagnosis of polypoidal choroidal vasculopathy and age-related macular degeneration using deep learning

BACKGROUND

This study aims to achieve an automatic differential diagnosis between two types of retinal pathologies with similar pathological features - Polypoidal choroidal vasculopathy (PCV) and wet age-related macular degeneration (AMD) from volumetric optical coherence tomography (OCT) images, and identify clinically-relevant pathological features, using an explainable deep-learning-based framework.

METHODS

This is a retrospective study with data from a cross-sectional cohort. The OCT volume of 73 eyes from 59 patients was included in this study. Disease differentiation was achieved through single-B-scan-based classification followed by a volumetric probability prediction aggregation step. We compared different labeling strategies with and without identifying pathological B-scans within each OCT volume. Clinical interpretability was achieved through normalized aggregation of B-scan-based saliency maps followed by maximum-intensity-projection onto the en face plane. We derived the PCV score from the proposed differential diagnosis framework with different labeling strategies. The en face projection of saliency map was validated with the pathologies identified in Indocyanine green angiography (ICGA).

RESULTS

Model trained with both labeling strategies achieved similar level differentiation power (>90%), with good correspondence between pathological features detected from the projected en face saliency map and ICGA.

CONCLUSIONS

This study demonstrated the potential clinical application of non-invasive differential diagnosis using AI-driven OCT-based analysis, with minimal requirement of labeling efforts, along with clinical explainability achieved through automatically detected disease-related pathologies.