Joint segmentation of multi-class hyper-reflective foci in retinal optical coherence tomography images

Retinal OCT image segmentation with deep learning: A review of advances, datasets, and evaluation metrics

Optical coherence tomography (OCT) is a widely used imaging technology in ophthalmic clinical practice, providing non-invasive access to high-resolution retinal images. Segmentation of anatomical structures and pathological lesions in retinal OCT images, directly impacts clinical decisions. While commercial OCT devices segment multiple retinal layers in healthy eyes, their performance degrades severely under pathological conditions. In recent years, the rapid advancements in deep learning have significantly driven research in OCT image segmentation. This review provides a comprehensive overview of the latest developments in deep learning-based segmentation methods for retinal OCT images. Additionally, it summarizes the medical significance, publicly available datasets, and commonly used evaluation metrics in this field. The review also discusses the current challenges faced by the research community and highlights potential future directions.

Strip and boundary detection multi-task learning network for segmentation of meibomian glands

BACKGROUND

Automatic segmentation of meibomian glands in near-infrared meibography images is basis of morphological parameter analysis, which plays a crucial role in facilitating the diagnosis of meibomian gland dysfunction (MGD). The special strip shape and the adhesion between glands make the automatic segmentation of meibomian glands very challenging.

PURPOSE

A strip and boundary detection multi-task learning network (SBD-MTLNet) based on encoder-decoder structure is proposed to realize the automatic segmentation of meibomian glands.

METHODS

A strip mixed attention module (SMAM) is proposed to enhance the network's ability to recognize the strip shape of glands. To alleviate the problem of adhesion between glands, a boundary detection auxiliary network (BDA-Net) is proposed, which introduces boundary features to assist gland segmentation. A self-adaptive interactive information fusion module (SIIFM) based on reverse attention mechanism is proposed to realize information complementation between meibomian gland segmentation and boundary detection tasks. The proposed SBD-MTLNet has been evaluated on an in-house dataset (453 images) and a public dataset MGD-1K (1000 images). Due to the limited number of images, a five-fold cross validation strategy is adopted.

RESULTS

Average dice coefficient of the proposed SBD-MTLNet reaches 81.08% and 84.32% on the in-house dataset and the public one, respectively. Comprehensive experimental results demonstrate the effectiveness the proposed SBD-MTLNet, outperforming other state-of-the-art methods.

CONCLUSIONS

The proposed SBD-MTLNet can focus more on the shape characteristics of the meibomian glands and the boundary contour information between the adjacent glands via multi-task learning strategy. The segmentation results of the proposed method can be used for the quantitative morphological characteristics analysis of meibomian glands, which has potential for the auxiliary diagnosis of MGD in clinic.

Enhancing diabetic retinopathy diagnosis: automatic segmentation of hyperreflective foci in OCT via deep learning

OBJECTIVE

Hyperreflective foci (HRF) are small, punctate lesions ranging from 20 to 50 μ m and exhibiting high reflective intensity in optical coherence tomography (OCT) images of patients with diabetic retinopathy (DR). The purpose of the model proposed in this paper is to precisely identify and segment the HRF in OCT images of patients with DR. This method is essential for assisting ophthalmologists in the early diagnosis and assessing the effectiveness of treatment and prognosis. In this study, we introduce an HRF segmentation algorithm based on KiU-Net, the algorithm that comprises the Kite-Net branch using up-sampling coding to collect more detailed information and a three-layer U-Net branch to extract high-level semantic information. To enhance the capacity of a single-branch network, we also design a cross-attention block (CAB) which combines the information extracted from two branches. The experimental results demonstrate that the number of parameters of our model is significantly reduced, and the sensitivity (SE) and the dice similarity coefficient (DSC) are respectively improved to 72.90 % and 66.84 % . Considering the SE and precision(P) of the segmentation, as well as the recall ratio and recall P of HRF, we believe that this model outperforms most advanced medical image segmentation algorithms and significantly relieves the strain on ophthalmologists.

PURPOSE

Hyperreflective foci (HRF) are small, punctate lesions ranging from 20 to 50 μm with high reflective intensity in optical coherence tomography (OCT) images of patients with diabetic retinopathy (DR). This study aims to develop a model that precisely identifies and segments HRF in OCT images of DR patients. Accurate segmentation of HRF is essential for assisting ophthalmologists in early diagnosis and in assessing the effectiveness of treatment and prognosis.

METHODS

We introduce an HRF segmentation algorithm based on the KiU-Net architecture. The model comprises two branches: a Kite-Net branch that uses up-sampling coding to capture detailed information, and a three-layer U-Net branch that extracts high-level semantic information. To enhance the capacity of the network, we designed a cross-attention block (CAB) that combines the information extracted from both branches, effectively integrating detail and semantic features.

RESULTS

Experimental results demonstrate that our model significantly reduces the number of parameters while improving performance. The sensitivity (SE) and Dice Similarity Coefficient (DSC) of our model are improved to 72.90% and 66.84%, respectively. Considering the SE and precision (P) of the segmentation, as well as the recall ratio and precision of HRF detection, our model outperforms most advanced medical image segmentation algorithms CONCLUSION: The proposed HRF segmentation algorithm effectively identifies and segments HRF in OCT images of DR patients, outperforming existing methods. This advancement can significantly alleviate the burden on ophthalmologists by aiding in early diagnosis and treatment evaluation, ultimately improving patient outcomes.

Automatic exudate and aneurysm segmentation in OCT images using UNET++ and hyperreflective-foci feature based bagged tree ensemble

Diabetic retinopathy's signs, such as exudates (EXs) and aneurysms (ANs), initially develop from under the retinal surface detectable from optical coherence tomography (OCT) images. Detecting these signs helps ophthalmologists diagnose DR sooner. Detecting and segmenting exudates (EXs) and aneurysms (ANs) in medical images is challenging due to their small size, similarity to other hyperreflective regions, noise presence, and low background contrast. Furthermore, the scarcity of public OCT images featuring these abnormalities has limited the number of studies related to the automatic segmentation of EXs and ANs, and the reported performance of such studies has not been satisfactory. This work proposes an efficient algorithm that can automatically segment these anomalies by improving key steps in the process. The potential area where these hyper-reflective EXs and ANs occur was scoped by our method using a deep-learning U-Net++ program. From this area, the candidates for EX-AN were segmented using the adaptive thresholding method. Nine features based on appearances, locations, and shadow markers were extracted from these candidates. They were trained and tested using bagged tree ensemble classifiers to obtain only EX-AN blobs. The proposed method was tested on a collection of a public dataset comprising 80 images with hand-drawn ground truths. The experimental results showed that our method could segment EX-AN blobs with average recall, precision, and F1-measure as 87.9%, 86.1%, and 87.0%, respectively. Its F1-measure drastically outperformed two comparative methods, binary thresholding and watershed (BT-WS) and adaptive thresholding with shadow tracking (AT-ST), by 78.0% and 82.1%, respectively.

Segmentation of paracentral acute middle maculopathy lesions in spectral-domain optical coherence tomography images through weakly supervised deep convolutional networks

BACKGROUND AND OBJECTIVES

Spectral-domain optical coherence tomography (SD-OCT) is a valuable tool for non-invasive imaging of the retina, allowing the discovery and visualization of localized lesions, the presence of which is associated with eye diseases. The present study introduces X-Net, a weakly supervised deep-learning framework for automated segmentation of paracentral acute middle maculopathy (PAMM) lesions in retinal SD-OCT images. Despite recent advances in the development of automatic methods for clinical analysis of OCT scans, there remains a scarcity of studies focusing on the automated detection of small retinal focal lesions. Additionally, most existing solutions depend on supervised learning, which can be time-consuming and require extensive image labeling, whereas X-Net offers a solution to these challenges. As far as we can determine, no prior study has addressed the segmentation of PAMM lesions in SD-OCT images.

METHODS

This study leverages 133 SD-OCT retinal images, each containing instances of paracentral acute middle maculopathy lesions. A team of eye experts annotated the PAMM lesions in these images using bounding boxes. Then, labeled data were used to train a U-Net that performs pre-segmentation, producing region labels of pixel-level accuracy. To attain a highly-accurate final segmentation, we introduced X-Net, a novel neural network made up of a master and a slave U-Net. During training, it takes the expert annotated, and pixel-level pre-segment annotated images and employs sophisticated strategies to ensure the highest segmentation accuracy.

RESULTS

The proposed method was rigorously evaluated on clinical retinal images excluded from training and achieved an accuracy of 99% with a high level of similarity between the automatic segmentation and expert annotation, as demonstrated by a mean Intersection-over-Union of 0.8. Alternative methods were tested on the same data. Single-stage neural networks proved insufficient for achieving satisfactory results, confirming that more advanced solutions, such as the proposed method, are necessary. We also found that X-Net using Attention U-net for both the pre-segmentation and X-Net arms for the final segmentation shows comparable performance to the proposed method, suggesting that the proposed approach remains a viable solution even when implemented with variants of the classic U-Net.

CONCLUSIONS

The proposed method exhibits reasonably high performance, validated through quantitative and qualitative evaluations. Medical eye specialists have also verified its validity and accuracy. Thus, it could be a viable tool in the clinical assessment of the retina. Additionally, the demonstrated approach for annotating the training set has proven to be effective in reducing the expert workload.

Automatic Segmentation of Hyperreflective Foci in OCT Images Based on Lightweight DBR Network

Hyperreflective foci (HF) reflects inflammatory responses for fundus diseases such as diabetic macular edema (DME), retina vein occlusion (RVO), and central serous chorioretinopathy (CSC). Shown as high contrast and reflectivity in optical coherence tomography (OCT) images, automatic segmentation of HF in OCT images is helpful for the prognosis of fundus diseases. Previous traditional methods were time-consuming and required high computing power. Hence, we proposed a lightweight network to segment HF (with a speed of 57 ms per OCT image, at least 150 ms faster than other methods). Our framework consists of two stages: an NLM filter and patch-based split to preprocess images and a lightweight DBR neural network to segment HF automatically. Experimental results from 3000 OCT images of 300 patients (100 DME,100 RVO, and 100 CSC) revealed that our method achieved HF segmentation successfully. The DBR network had the area under curves dice similarity coefficient (DSC) of 83.65%, 76.43%, and 82.20% in segmenting HF in DME, RVO, and CSC on the test cohort respectively. Our DBR network achieves at least 5% higher DSC than previous methods. HF in DME was more easily segmented compared with the other two types. In addition, our DBR network is universally applicable to clinical practice with the ability to segment HF in a wide range of fundus diseases.

OCT2Former: A retinal OCT-angiography vessel segmentation transformer

BACKGROUND AND OBJECTIVE

Retinal vessel segmentation plays an important role in the automatic retinal disease screening and diagnosis. How to segment thin vessels and maintain the connectivity of vessels are the key challenges of the retinal vessel segmentation task. Optical coherence tomography angiography (OCTA) is a noninvasive imaging technique that can reveal high-resolution retinal vessels. Aiming at make full use of its characteristic of high resolution, a new end-to-end transformer based network named as OCT²Former (OCT-a Transformer) is proposed to segment retinal vessel accurately in OCTA images.

METHODS

The proposed OCT²Former is based on encoder-decoder structure, which mainly includes dynamic transformer encoder and lightweight decoder. Dynamic transformer encoder consists of dynamic token aggregation transformer and auxiliary convolution branch, in which the multi-head dynamic token aggregation attention based dynamic token aggregation transformer is designed to capture the global retinal vessel context information from the first layer throughout the network and the auxiliary convolution branch is proposed to compensate for the lack of inductive bias of the transformer and assist in the efficient feature extraction. A convolution based lightweight decoder is proposed to decode features efficiently and reduce the complexity of the proposed OCT²Former.

RESULTS

The proposed OCT²Former is validated on three publicly available datasets i.e. OCTA-SS, ROSE-1, OCTA-500 (subset OCTA-6M and OCTA-3M). The Jaccard indexes of the proposed OCT²Former on these datasets are 0.8344, 0.7855, 0.8099 and 0.8513, respectively, outperforming the best convolution based network 1.43, 1.32, 0.75 and 1.46%, respectively.

CONCLUSION

The experimental results have demonstrated that the proposed OCT²Former can achieve competitive performance on retinal OCTA vessel segmentation tasks.

A deep network embedded with rough fuzzy discretization for OCT fundus image segmentation

The noise and redundant information are the main reasons for the performance bottleneck of medical image segmentation algorithms based on the deep learning. To this end, we propose a deep network embedded with rough fuzzy discretization (RFDDN) for OCT fundus image segmentation. Firstly, we establish the information decision table of OCT fundus image segmentation, and regard each category of segmentation region as a fuzzy set. Then, we use the fuzzy c-means clustering to get the membership degrees of pixels to each segmentation region. According to membership functions and the equivalence relation generated by the brightness attribute, we design the individual fitness function based on the rough fuzzy set, and use a genetic algorithm to search for the best breakpoints to discretize the features of OCT fundus images. Finally, we take the feature discretization based on the rough fuzzy set as the pre-module of the deep neural network, and introduce the deep supervised attention mechanism to obtain the important multi-scale information. We compare RFDDN with U-Net, ReLayNet, CE-Net, MultiResUNet, and ISCLNet on the two groups of 3D retinal OCT data. RFDDN is superior to the other five methods on all evaluation indicators. The results obtained by ISCLNet are the second only inferior to those obtained by RFDDN. DSC, sensitivity, and specificity of RFDDN are evenly 3.3%, 2.6%, and 7.1% higher than those of ISCLNet, respectively. HD95 and ASD of RFDDN are evenly 6.6% and 19.7% lower than those of ISCLNet, respectively. The experimental results show that our method can effectively eliminate the noise and redundant information in Oct fundus images, and greatly improve the accuracy of OCT fundus image segmentation while taking into account the interpretability and computational efficiency.

Automated Region of Interest Selection Improves Deep Learning-Based Segmentation of Hyper-Reflective Foci in Optical Coherence Tomography Images

Hyperreflective foci (HRF) have been associated with retinal disease progression and demonstrated as a negative prognostic biomarker for visual function. Automated segmentation of HRF in retinal optical coherence tomography (OCT) scans can be beneficial to identify the formation and movement of the HRF biomarker as a retinal disease progresses and can serve as the first step in understanding the nature and severity of the disease. In this paper, we propose a fully automated deep neural network based HRF segmentation model in OCT images. We enhance the model's performance by using a patch-based strategy that increases the model's compute on the HRF pixels. The patch-based strategy is evaluated against state of the art HRF segmentation pipelines on clinical retinal image data. Our results shows that the patch-based approach demonstrates a high precision score and intersection over union (IOU) using a ResNet34 segmentation model with Binary Cross Entropy loss function. The HRF segmentation pipeline can be used for analyzing HRF biomarkers for different retinopathies.

Local and Context-Attention Adaptive LCA-Net for Thyroid Nodule Segmentation in Ultrasound Images

The thyroid nodule segmentation of ultrasound images is a critical step for the early diagnosis of thyroid cancers in clinics. Due to the weak edge of ultrasound images and the complexity of thyroid tissue structure, it is still challenging to accurately segment the delicate contour of thyroid nodules. A local and context-attention adaptive network (LCA-Net) for thyroid nodule segmentation is proposed to address these shortcomings, which leverages both local feature information from convolution neural networks and global context information from transformers. Firstly, since most existing thyroid nodule segmentation models are skilled at local detail features and lose some context information, we propose a transformers-based context-attention module to capture more global associative information for the network and perceive the edge information of the nodule contour. Secondly, a backbone module with 7×1, 1×7 convolutions and the activation function Mish is designed, which enlarges the receptive field and extracts more feature details. Furthermore, a nodule adaptive convolution (NAC) module is introduced to adaptively deal with thyroid nodules of different sizes and positions, thereby improving the generalization performance of the model. Simultaneously, an optimized loss function is proposed to solve the pixels class imbalance problem in segmentation. The proposed LCA-Net, validated on the public TN-SCUI2020 and TN3K datasets, achieves Dice scores of 90.26% and 82.08% and PA scores of 98.87% and 96.97%, respectively, which outperforms other state-of-the-art thyroid nodule segmentation models. This paper demonstrates the superiority of the proposed LCA-Net for thyroid nodule segmentation, which possesses strong generalization performance and promising segmentation accuracy. Consequently, the proposed model has wide application prospects for thyroid nodule diagnosis in clinics.