Retinal Artery-Vein Classification via Topology Estimation

Robust semi-automatic vessel tracing in the human retinal image by an instance segmentation neural network

Vasculature morphology and hierarchy are essential for blood perfusion. Human retinal circulation is an intricate vascular system emerging and remerging at the optic nerve head (ONH). Tracing retinal vascular branching from ONH can allow detailed morphological quantification, and yet remains a challenging task. We presented a robust semi-automatic vessel tracing algorithm on human fundus images by an instance segmentation neural network (InSegNN). InSegNN separates and labels individual vascular trees and enables tracing each tree throughout its branching. We have three strategies to improve robustness and accuracy: pseudotemporal learning, spatial multisampling, and dynamic probability map. We achieved 83% specificity, 50% improvement in symmetric best dice (SBD) compared to literature, and outperformed baseline U-net, and achieved 91% precision with 71% sensitivity. We have demonstrated tracing individual vessel trees from fundus images, and simultaneously retain vessel hierarchy information. InSegNN paves a way for subsequent analysis of vascular morphology in relation to retinal diseases.

Automatic artery/vein classification methods for retinal blood vessel: A review

Automatic retinal arteriovenous classification can assist ophthalmologists in disease early diagnosis. Deep learning-based methods and topological graph-based methods have become the main solutions for retinal arteriovenous classification in recent years. This paper reviews the automatic retinal arteriovenous classification methods from 2003 to 2022. Firstly, we compare different methods and provide comparison tables of the summary results. Secondly, we complete the classification of the public arteriovenous classification datasets and provide the annotation development tables of different datasets. Finally, we sort out the challenges of evaluation methods and provide a comprehensive evaluation system. Quantitative and qualitative analysis shows the changes in research hotspots over time, Quantitative and qualitative analyses reveal the evolution of research hotspots over time, highlighting the significance of exploring the integration of deep learning with topological information in future research.

CF-Loss: Clinically-relevant feature optimised loss function for retinal multi-class vessel segmentation and vascular feature measurement

Characterising clinically-relevant vascular features, such as vessel density and fractal dimension, can benefit biomarker discovery and disease diagnosis for both ophthalmic and systemic diseases. In this work, we explicitly encode vascular features into an end-to-end loss function for multi-class vessel segmentation, categorising pixels into artery, vein, uncertain pixels, and background. This clinically-relevant feature optimised loss function (CF-Loss) regulates networks to segment accurate multi-class vessel maps that produce precise vascular features. Our experiments first verify that CF-Loss significantly improves both multi-class vessel segmentation and vascular feature estimation, with two standard segmentation networks, on three publicly available datasets. We reveal that pixel-based segmentation performance is not always positively correlated with accuracy of vascular features, thus highlighting the importance of optimising vascular features directly via CF-Loss. Finally, we show that improved vascular features from CF-Loss, as biomarkers, can yield quantitative improvements in the prediction of ischaemic stroke, a real-world clinical downstream task. The code is available at https://github.com/rmaphoh/feature-loss.

One-shot Retinal Artery and Vein Segmentation via Cross-modality Pretraining

Purpose

To perform one-shot retinal artery and vein segmentation with cross-modality artery-vein (AV) soft-label pretraining.

Design

Cross-sectional study.

Subjects

The study included 6479 color fundus photography (CFP) and arterial-venous fundus fluorescein angiography (FFA) pairs from 1964 participants for pretraining and 6 AV segmentation data sets with various image sources, including RITE, HRF, LES-AV, AV-WIDE, PortableAV, and DRSplusAV for one-shot finetuning and testing.

Methods

We structurally matched the arterial and venous phase of FFA with CFP, the AV soft labels were automatically generated by utilizing the fluorescein intensity difference of the arterial and venous-phase FFA images, and the soft labels were then used to train a generative adversarial network to learn to generate AV soft segmentations using CFP images as input. We then finetuned the pretrained model to perform AV segmentation using only one image from each of the AV segmentation data sets and test on the remainder. To investigate the effect and reliability of one-shot finetuning, we conducted experiments without finetuning and by finetuning the pretrained model on an iteratively different single image for each data set under the same experimental setting and tested the models on the remaining images.

Main Outcome Measures

The AV segmentation was assessed by area under the receiver operating characteristic curve (AUC), accuracy, Dice score, sensitivity, and specificity.

Results

After the FFA-AV soft label pretraining, our method required only one exemplar image from each camera or modality and achieved similar performance with full-data training, with AUC ranging from 0.901 to 0.971, accuracy from 0.959 to 0.980, Dice score from 0.585 to 0.773, sensitivity from 0.574 to 0.763, and specificity from 0.981 to 0.991. Compared with no finetuning, the segmentation performance improved after one-shot finetuning. When finetuned on different images in each data set, the standard deviation of the segmentation results across models ranged from 0.001 to 0.10.

Conclusions

This study presents the first one-shot approach to retinal artery and vein segmentation. The proposed labeling method is time-saving and efficient, demonstrating a promising direction for retinal-vessel segmentation and enabling the potential for widespread application.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Tree Recovery by Dynamic Programming

Tree-like structures are common, naturally occurring objects that are of interest to many fields of study, such as plant science and biomedicine. Analysis of these structures is typically based on skeletons extracted from captured data, which often contain spurious cycles that need to be removed. We propose a dynamic programming algorithm for solving the NP-hard tree recovery problem formulated by (Estrada et al. 2015), which seeks a least-cost partitioning of the graph nodes that yields a directed tree. Our algorithm finds the optimal solution by iteratively contracting the graph via node-merging until the problem can be trivially solved. By carefully designing the merging sequence, our algorithm can efficiently recover optimal trees for many real-world data where (Estrada et al. 2015) only produces sub-optimal solutions. We also propose an approximate variant of dynamic programming using beam search, which can process graphs containing thousands of cycles with significantly improved optimality and efficiency compared with (Estrada et al. 2015).

Computer-aided diagnostic system for hypertensive retinopathy: A review

Hypertensive Retinopathy (HR) is a retinal disease caused by elevated blood pressure for a prolonged period. There are no obvious signs in the early stages of high blood pressure, but it affects various body parts over time, including the eyes. HR is a biomarker for several illnesses, including retinal diseases, atherosclerosis, strokes, kidney disease, and cardiovascular risks. Early microcirculation abnormalities in chronic diseases can be diagnosed through retinal examination prior to the onset of major clinical consequences. Computer-aided diagnosis (CAD) plays a vital role in the early identification of HR with improved diagnostic accuracy, which is time-efficient and demands fewer resources. Recently, numerous studies have been reported on the automatic identification of HR. This paper provides a comprehensive review of the automated tasks of Artery-Vein (A/V) classification, Arteriovenous ratio (AVR) computation, HR detection (Binary classification), and HR severity grading. The review is conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol. The paper discusses the clinical features of HR, the availability of datasets, existing methods used for A/V classification, AVR computation, HR detection, and severity grading, and performance evaluation metrics. The reviewed articles are summarized with classifiers details, adoption of different kinds of methodologies, performance comparisons, datasets details, their pros and cons, and computational platform. For each task, a summary and critical in-depth analysis are provided, as well as common research issues and challenges in the existing studies. Finally, the paper proposes future research directions to overcome challenges associated with data set availability, HR detection, and severity grading.

Multi-Task Segmentation and Classification Network for Artery/Vein Classification in Retina Fundus

Automatic classification of arteries and veins (A/V) in fundus images has gained considerable attention from researchers due to its potential to detect vascular abnormalities and facilitate the diagnosis of some systemic diseases. However, the variability in vessel structures and the marginal distinction between arteries and veins poses challenges to accurate A/V classification. This paper proposes a novel Multi-task Segmentation and Classification Network (MSC-Net) that utilizes the vessel features extracted by a specific module to improve A/V classification and alleviate the aforementioned limitations. The proposed method introduces three modules to enhance the performance of A/V classification: a Multi-scale Vessel Extraction (MVE) module, which distinguishes between vessel pixels and background using semantics of vessels, a Multi-structure A/V Extraction (MAE) module that classifies arteries and veins by combining the original image with the vessel features produced by the MVE module, and a Multi-source Feature Integration (MFI) module that merges the outputs from the former two modules to obtain the final A/V classification results. Extensive empirical experiments verify the high performance of the proposed MSC-Net for retinal A/V classification over state-of-the-art methods on several public datasets.

A Global Review of Publicly Available Datasets Containing Fundus Images: Characteristics, Barriers to Access, Usability, and Generalizability

This article provides a comprehensive and up-to-date overview of the repositories that contain color fundus images. We analyzed them regarding availability and legality, presented the datasets' characteristics, and identified labeled and unlabeled image sets. This study aimed to complete all publicly available color fundus image datasets to create a central catalog of available color fundus image datasets.

End-to-End Automatic Classification of Retinal Vessel Based on Generative Adversarial Networks with Improved U-Net

The retinal vessels in the human body are the only ones that can be observed directly by non-invasive imaging techniques. Retinal vessel morphology and structure are the important objects of concern for physicians in the early diagnosis and treatment of related diseases. The classification of retinal vessels has important guiding significance in the basic stage of diagnostic treatment. This paper proposes a novel method based on generative adversarial networks with improved U-Net, which can achieve synchronous automatic segmentation and classification of blood vessels by an end-to-end network. The proposed method avoids the dependency of the segmentation results in the multiple classification tasks. Moreover, the proposed method builds on an accurate classification of arteries and veins while also classifying arteriovenous crossings. The validity of the proposed method is evaluated on the RITE dataset: the accuracy of image comprehensive classification reaches 96.87%. The sensitivity and specificity of arteriovenous classification reach 91.78% and 97.25%. The results verify the effectiveness of the proposed method and show the competitive classification performance.

AV-casNet: Fully Automatic Arteriole-Venule Segmentation and Differentiation in OCT Angiography

Automatic segmentation and differentiation of retinal arteriole and venule (AV), defined as small blood vessels directly before and after the capillary plexus, are of great importance for the diagnosis of various eye diseases and systemic diseases, such as diabetic retinopathy, hypertension, and cardiovascular diseases. Optical coherence tomography angiography (OCTA) is a recent imaging modality that provides capillary-level blood flow information. However, OCTA does not have the colorimetric and geometric differences between AV as the fundus photography does. Various methods have been proposed to differentiate AV in OCTA, which typically needs the guidance of other imaging modalities. In this study, we propose a cascaded neural network to automatically segment and differentiate AV solely based on OCTA. A convolutional neural network (CNN) module is first applied to generate an initial segmentation, followed by a graph neural network (GNN) to improve the connectivity of the initial segmentation. Various CNN and GNN architectures are employed and compared. The proposed method is evaluated on multi-center clinical datasets, including 3 ×3 mm2 and 6 ×6 mm2 OCTA. The proposed method holds the potential to enrich OCTA image information for the diagnosis of various diseases.

Recent trends and advances in fundus image analysis: A review

Automated retinal image analysis holds prime significance in the accurate diagnosis of various critical eye diseases that include diabetic retinopathy (DR), age-related macular degeneration (AMD), atherosclerosis, and glaucoma. Manual diagnosis of retinal diseases by ophthalmologists takes time, effort, and financial resources, and is prone to error, in comparison to computer-aided diagnosis systems. In this context, robust classification and segmentation of retinal images are primary operations that aid clinicians in the early screening of patients to ensure the prevention and/or treatment of these diseases. This paper conducts an extensive review of the state-of-the-art methods for the detection and segmentation of retinal image features. Existing notable techniques for the detection of retinal features are categorized into essential groups and compared in depth. Additionally, a summary of quantifiable performance measures for various important stages of retinal image analysis, such as image acquisition and preprocessing, is provided. Finally, the widely used in the literature datasets for analyzing retinal images are described and their significance is emphasized.

Automated Systems for Calculating Arteriovenous Ratio in Retinographies: A Scoping Review

There is evidence of an association between hypertension and retinal arteriolar narrowing. Manual measurement of retinal vessels comes with additional variability, which can be eliminated using automated software. This scoping review aims to summarize research on automated retinal vessel analysis systems. Searches were performed on Medline, Scopus, and Cochrane to find studies examining automated systems for the diagnosis of retinal vascular alterations caused by hypertension using the following keywords: diagnosis; diagnostic screening programs; image processing, computer-assisted; artificial intelligence; electronic data processing; hypertensive retinopathy; hypertension; retinal vessels; arteriovenous ratio and retinal image analysis. The searches generated 433 articles. Of these, 25 articles published from 2010 to 2022 were included in the review. The retinographies analyzed were extracted from international databases and real scenarios. Automated systems to detect alterations in the retinal vasculature are being introduced into clinical practice for diagnosis in ophthalmology and other medical specialties due to the association of such changes with various diseases. These systems make the classification of hypertensive retinopathy and cardiovascular risk more reliable. They also make it possible for diagnosis to be performed in primary care, thus optimizing ophthalmological visits.

MSGANet-RAV: A multiscale guided attention network for artery-vein segmentation and classification from optic disc and retinal images

BACKGROUND

Retinal and optic disc images are used to assess changes in the retinal vasculature. These can be changes associated with diseases such as diabetic retinopathy and glaucoma or induced using ophthalmodynamometry to measure arterial and venous pressure. Key steps toward automating the assessment of these changes are the segmentation and classification of the veins and arteries. However, such segmentation and classification are still required to be manually labelled by experts. Such automated labelling is challenging because of the complex morphology, anatomical variations, alterations due to disease and scarcity of labelled data for algorithm development. We present a deep machine learning solution called the multiscale guided attention network for retinal artery and vein segmentation and classification (MSGANet-RAV).

METHODS

MSGANet-RAV was developed and tested on 383 colour clinical optic disc images from LEI-CENTRAL, constructed in-house and 40 colour fundus images from the AV-DRIVE public dataset. The datasets have a mean optic disc occupancy per image of 60.6% and 2.18%, respectively. MSGANet-RAV is a U-shaped encoder-decoder network, where the encoder extracts multiscale features, and the decoder includes a sequence of self-attention modules. The self-attention modules explore, guide and incorporate vessel-specific structural and contextual feature information to segment and classify central optic disc and retinal vessel pixels.

RESULTS

MSGANet-RAV achieved a pixel classification accuracy of 93.15%, sensitivity of 92.19%, and specificity of 94.13% on LEI-CENTRAL, outperforming several reference models. It similarly performed highly on AV-DRIVE with an accuracy, sensitivity and specificity of 95.48%, 93.59% and 97.27%, respectively.

CONCLUSION

The results show the efficacy of MSGANet-RAV for identifying central optic disc and retinal arteries and veins. The method can be used in automated systems designed to assess vascular changes in retinal and optic disc images quantitatively.

AutoMorph: Automated Retinal Vascular Morphology Quantification Via a Deep Learning Pipeline

Purpose

To externally validate a deep learning pipeline (AutoMorph) for automated analysis of retinal vascular morphology on fundus photographs. AutoMorph has been made publicly available, facilitating widespread research in ophthalmic and systemic diseases.

Methods

AutoMorph consists of four functional modules: image preprocessing, image quality grading, anatomical segmentation (including binary vessel, artery/vein, and optic disc/cup segmentation), and vascular morphology feature measurement. Image quality grading and anatomical segmentation use the most recent deep learning techniques. We employ a model ensemble strategy to achieve robust results and analyze the prediction confidence to rectify false gradable cases in image quality grading. We externally validate the performance of each module on several independent publicly available datasets.

Results

The EfficientNet-b4 architecture used in the image grading module achieves performance comparable to that of the state of the art for EyePACS-Q, with an F1-score of 0.86. The confidence analysis reduces the number of images incorrectly assessed as gradable by 76%. Binary vessel segmentation achieves an F1-score of 0.73 on AV-WIDE and 0.78 on DR HAGIS. Artery/vein scores are 0.66 on IOSTAR-AV, and disc segmentation achieves 0.94 in IDRID. Vascular morphology features measured from the AutoMorph segmentation map and expert annotation show good to excellent agreement.

Conclusions

AutoMorph modules perform well even when external validation data show domain differences from training data (e.g., with different imaging devices). This fully automated pipeline can thus allow detailed, efficient, and comprehensive analysis of retinal vascular morphology on color fundus photographs.

Translational Relevance

By making AutoMorph publicly available and open source, we hope to facilitate ophthalmic and systemic disease research, particularly in the emerging field of oculomics.

Automated Retinal Vascular Topological Information Extraction From OCTA

The retinal vascular system adapts and reacts rapidly to ocular diseases such as glaucoma, diabetic retinopathy and age-related macular degeneration. Here we present a combination of methods to further extract vascular information from [Formula: see text] wide-field optical coherence tomography angiography (OCTA). An integrated U-Net for the segmentation and classification of arteries and veins reached a segmentation IoU of 0.7095±0.0224, and classification IoU of 0.8793±0.1049 and 0.8928±0.0929 respectively. A correcting algorithm which uses topological information was created to correct the misclassification and connectivity of the vessels, which showed an average increase of 8.29% in IoU. Finally, the vessel morphometry of branch orders was extracted, where this allows the direct comparison of artery/vein, arterioles/venules and capillaries.

Stacked Ensemble Network to Assess the Structural Variations in Retina: A Bio-marker for Early Disease Diagnosis

The retina is a unique tissue that extends the human brain in transmitting the incoming light into neural spikes. Researchers collaborating with domain experts proposed numerous deep networks to extract vessels from the retina; however, these techniques have the least response for micro-vessels. The proposed method has developed a stacked ensemble network approach with deep neural architectures for precise vessel extraction. Our method has used bi-directional LSTM for filling gaps in dis-joint vessels and applied W-Net for boundary refinement and emphasizing local regions to achieve better results for micro-vessels extraction. The platform has combined the strength of various networks to improve the automated screening process and has shown promising results on benchmark datasets.

State-of-the-art retinal vessel segmentation with minimalistic models

The segmentation of retinal vasculature from eye fundus images is a fundamental task in retinal image analysis. Over recent years, increasingly complex approaches based on sophisticated Convolutional Neural Network architectures have been pushing performance on well-established benchmark datasets. In this paper, we take a step back and analyze the real need of such complexity. We first compile and review the performance of 20 different techniques on some popular databases, and we demonstrate that a minimalistic version of a standard U-Net with several orders of magnitude less parameters, carefully trained and rigorously evaluated, closely approximates the performance of current best techniques. We then show that a cascaded extension (W-Net) reaches outstanding performance on several popular datasets, still using orders of magnitude less learnable weights than any previously published work. Furthermore, we provide the most comprehensive cross-dataset performance analysis to date, involving up to 10 different databases. Our analysis demonstrates that the retinal vessel segmentation is far from solved when considering test images that differ substantially from the training data, and that this task represents an ideal scenario for the exploration of domain adaptation techniques. In this context, we experiment with a simple self-labeling strategy that enables moderate enhancement of cross-dataset performance, indicating that there is still much room for improvement in this area. Finally, we test our approach on Artery/Vein and vessel segmentation from OCTA imaging problems, where we again achieve results well-aligned with the state-of-the-art, at a fraction of the model complexity available in recent literature. Code to reproduce the results in this paper is released.

Multi-scale Interactive Network with Artery/Vein Discriminator for Retinal Vessel Classification

Automatic classification of retinal arteries and veins plays an important role in assisting clinicians to diagnosis cardiovascular and eye-related diseases. However, due to the high degree of anatomical variation across the population, and the presence of inconsistent labels by the subjective judgment of annotators in available training data, most of existing methods generally suffer from blood vessel discontinuity and arteriovenous confusion, the artery/vein (A/V) classification task still faces great challenges. In this work, we propose a multi-scale interactive network with A/V discriminator for retinal artery and vein recognition, which can reduce the arteriovenous confusion and alleviate the disturbance of noisy label. A multi-scale interaction (MI) module is designed in encoder for realizing the cross-space multi-scale features interaction of fundus images, effectively integrate high-level and low-level context information. In particular, we design an ingenious A/V discriminator (AVD) that utilizes the independent and shared information between arteries and veins, and combine with topology loss, to further strengthen the learning ability of model to resolve the arteriovenous confusion. In addition, we adopt a sample re-weighting (SW) strategy to effectively alleviate the disturbance from data labeling errors. The proposed model is verified on three publicly available fundus image datasets (AV-DRIVE, HRF, LES-AV) and a private dataset. We achieve the accuracy of 97.47%, 96.91%, 97.79%, and 98.18% respectively on these four datasets. Extensive experimental results demonstrate that our method achieves competitive performance compared with state-of-the-art methods for A/V classification. To address the problem of training data scarcity, we publicly release 100 fundus images with A/V annotations to promote relevant research in the community.

RAVIR: A Dataset and Methodology for the Semantic Segmentation and Quantitative Analysis of Retinal Arteries and Veins in Infrared Reflectance Imaging

The retinal vasculature provides important clues in the diagnosis and monitoring of systemic diseases including hypertension and diabetes. The microvascular system is of primary involvement in such conditions, and the retina is the only anatomical site where the microvasculature can be directly observed. The objective assessment of retinal vessels has long been considered a surrogate biomarker for systemic vascular diseases, and with recent advancements in retinal imaging and computer vision technologies, this topic has become the subject of renewed attention. In this paper, we present a novel dataset, dubbed RAVIR, for the semantic segmentation of Retinal Arteries and Veins in Infrared Reflectance (IR) imaging. It enables the creation of deep learning-based models that distinguish extracted vessel type without extensive post-processing. We propose a novel deep learning-based methodology, denoted as SegRAVIR, for the semantic segmentation of retinal arteries and veins and the quantitative measurement of the widths of segmented vessels. Our extensive experiments validate the effectiveness of SegRAVIR and demonstrate its superior performance in comparison to state-of-the-art models. Additionally, we propose a knowledge distillation framework for the domain adaptation of RAVIR pretrained networks on color images. We demonstrate that our pretraining procedure yields new state-of-the-art benchmarks on the DRIVE, STARE, and CHASE\_DB1 datasets. Dataset link: https://ravirdataset.github.io/data.

A Deep Learning System for Fully Automated Retinal Vessel Measurement in High Throughput Image Analysis

Motivation

Retinal microvasculature is a unique window for predicting and monitoring major cardiovascular diseases, but high throughput tools based on deep learning for in-detail retinal vessel analysis are lacking. As such, we aim to develop and validate an artificial intelligence system (Retina-based Microvascular Health Assessment System, RMHAS) for fully automated vessel segmentation and quantification of the retinal microvasculature.

Results

RMHAS achieved good segmentation accuracy across datasets with diverse eye conditions and image resolutions, having AUCs of 0.91, 0.88, 0.95, 0.93, 0.97, 0.95, 0.94 for artery segmentation and 0.92, 0.90, 0.96, 0.95, 0.97, 0.95, 0.96 for vein segmentation on the AV-WIDE, AVRDB, HRF, IOSTAR, LES-AV, RITE, and our internal datasets. Agreement and repeatability analysis supported the robustness of the algorithm. For vessel analysis in quantity, less than 2 s were needed to complete all required analysis.

VTG-Net: A CNN Based Vessel Topology Graph Network for Retinal Artery/Vein Classification

From diagnosing cardiovascular diseases to analyzing the progression of diabetic retinopathy, accurate retinal artery/vein (A/V) classification is critical. Promising approaches for A/V classification, ranging from conventional graph based methods to recent convolutional neural network (CNN) based models, have been known. However, the inability of traditional graph based methods to utilize deep hierarchical features extracted by CNNs and the limitations of current CNN based methods to incorporate vessel topology information hinder their effectiveness. In this paper, we propose a new CNN based framework, VTG-Net (vessel topology graph network), for retinal A/V classification by incorporating vessel topology information. VTG-Net exploits retinal vessel topology along with CNN features to improve A/V classification accuracy. Specifically, we transform vessel features extracted by CNN in the image domain into a graph representation preserving the vessel topology. Then by exploiting a graph convolutional network (GCN), we enable our model to learn both CNN features and vessel topological features simultaneously. The final predication is attained by fusing the CNN and GCN outputs. Using a publicly available AV-DRIVE dataset and an in-house dataset, we verify the high performance of our VTG-Net for retinal A/V classification over state-of-the-art methods (with ~2% improvement in accuracy on the AV-DRIVE dataset).

A Hybrid Method to Enhance Thick and Thin Vessels for Blood Vessel Segmentation

Retinal blood vessels have been presented to contribute confirmation with regard to tortuosity, branching angles, or change in diameter as a result of ophthalmic disease. Although many enhancement filters are extensively utilized, the Jerman filter responds quite effectively at vessels, edges, and bifurcations and improves the visualization of structures. In contrast, curvelet transform is specifically designed to associate scale with orientation and can be used to recover from noisy data by curvelet shrinkage. This paper describes a method to improve the performance of curvelet transform further. A distinctive fusion of curvelet transform and the Jerman filter is presented for retinal blood vessel segmentation. Mean-C thresholding is employed for the segmentation purpose. The suggested method achieves average accuracies of 0.9600 and 0.9559 for DRIVE and CHASE_DB1, respectively. Simulation results establish a better performance and faster implementation of the suggested scheme in comparison with similar approaches seen in the literature.

Simultaneous segmentation and classification of the retinal arteries and veins from color fundus images

BACKGROUND AND OBJECTIVES

The study of the retinal vasculature represents a fundamental stage in the screening and diagnosis of many high-incidence diseases, both systemic and ophthalmic. A complete retinal vascular analysis requires the segmentation of the vascular tree along with the classification of the blood vessels into arteries and veins. Early automatic methods approach these complementary segmentation and classification tasks in two sequential stages. However, currently, these two tasks are approached as a joint semantic segmentation, because the classification results highly depend on the effectiveness of the vessel segmentation. In that regard, we propose a novel approach for the simultaneous segmentation and classification of the retinal arteries and veins from eye fundus images.

METHODS

We propose a novel method that, unlike previous approaches, and thanks to the proposal of a novel loss, decomposes the joint task into three segmentation problems targeting arteries, veins and the whole vascular tree. This configuration allows to handle vessel crossings intuitively and directly provides accurate segmentation masks of the different target vascular trees.

RESULTS

The provided ablation study on the public Retinal Images vessel Tree Extraction (RITE) dataset demonstrates that the proposed method provides a satisfactory performance, particularly in the segmentation of the different structures. Furthermore, the comparison with the state of the art shows that our method achieves highly competitive results in the artery/vein classification, while significantly improving the vascular segmentation.

CONCLUSIONS

The proposed multi-segmentation method allows to detect more vessels and better segment the different structures, while achieving a competitive classification performance. Also, in these terms, our approach outperforms the approaches of various reference works. Moreover, in contrast with previous approaches, the proposed method allows to directly detect the vessel crossings, as well as preserving the continuity of both arteries and veins at these complex locations.

Automatic Artery/Vein Classification Using a Vessel-Constraint Network for Multicenter Fundus Images

Retinal blood vessel morphological abnormalities are generally associated with cardiovascular, cerebrovascular, and systemic diseases, automatic artery/vein (A/V) classification is particularly important for medical image analysis and clinical decision making. However, the current method still has some limitations in A/V classification, especially the blood vessel edge and end error problems caused by the single scale and the blurred boundary of the A/V. To alleviate these problems, in this work, we propose a vessel-constraint network (VC-Net) that utilizes the information of vessel distribution and edge to enhance A/V classification, which is a high-precision A/V classification model based on data fusion. Particularly, the VC-Net introduces a vessel-constraint (VC) module that combines local and global vessel information to generate a weight map to constrain the A/V features, which suppresses the background-prone features and enhances the edge and end features of blood vessels. In addition, the VC-Net employs a multiscale feature (MSF) module to extract blood vessel information with different scales to improve the feature extraction capability and robustness of the model. And the VC-Net can get vessel segmentation results simultaneously. The proposed method is tested on publicly available fundus image datasets with different scales, namely, DRIVE, LES, and HRF, and validated on two newly created multicenter datasets: Tongren and Kailuan. We achieve a balance accuracy of 0.9554 and F1 scores of 0.7616 and 0.7971 for the arteries and veins, respectively, on the DRIVE dataset. The experimental results prove that the proposed model achieves competitive performance in A/V classification and vessel segmentation tasks compared with state-of-the-art methods. Finally, we test the Kailuan dataset with other trained fusion datasets, the results also show good robustness. To promote research in this area, the Tongren dataset and source code will be made publicly available. The dataset and code will be made available at https://github.com/huawang123/VC-Net.

A Deep Learning Architecture for Vascular Area Measurement in Fundus Images

Purpose

To develop a novel evaluation system for retinal vessel alterations caused by hypertension using a deep learning algorithm.

Design

Retrospective study.

Participants

Fundus photographs (n = 10 571) of health-check participants (n = 5598).

Methods

The participants were analyzed using a fully automatic architecture assisted by a deep learning system, and the total area of retinal arterioles and venules was assessed separately. The retinal vessels were extracted automatically from each photograph and categorized as arterioles or venules. Subsequently, the total arteriolar area (AA) and total venular area (VA) were measured. The correlations among AA, VA, age, systolic blood pressure (SBP), and diastolic blood pressure were analyzed. Six ophthalmologists manually evaluated the arteriovenous ratio (AVR) in fundus images (n = 102), and the correlation between the SBP and AVR was evaluated manually.

Main Outcome Measures

Total arteriolar area and VA.

Results

The deep learning algorithm demonstrated favorable properties of vessel segmentation and arteriovenous classification, comparable with pre-existing techniques. Using the algorithm, a significant positive correlation was found between AA and VA. Both AA and VA demonstrated negative correlations with age and blood pressure. Furthermore, the SBP showed a higher negative correlation with AA measured by the algorithm than with AVR.

Conclusions

The current data demonstrated that the retinal vascular area measured with the deep learning system could be a novel index of hypertension-related vascular changes.

Differential artery-vein analysis in quantitative retinal imaging: a review

Quantitative retinal imaging is essential for eye disease detection, staging classification, and treatment assessment. It is known that different eye diseases or severity stages can affect the artery and vein systems in different ways. Therefore, differential artery-vein (AV) analysis can improve the performance of quantitative retinal imaging. In this article, we provide a brief summary of technical rationales and clinical applications of differential AV analysis in fundus photography, optical coherence tomography (OCT), and OCT angiography (OCTA).

Unsupervised sorting of retinal vessels using locally consistent Gaussian mixtures

BACKGROUND AND OBJECTIVES

Retinal blood vessels classification into arterioles and venules is a major task for biomarker identification. Especially, clustering of retinal blood vessels is a challenging task due to factors affecting the images such as contrast variability, non-uniform illumination etc. Hence, a high performance automatic retinal vessel classification system is highly prized. In this paper, we propose a novel unsupervised methodology to classify retinal vessels extracted from fundus camera images into arterioles and venules.

METHODS

The proposed method utilises the homomorphic filtering (HF) to preprocess the input image for non-uniform illumination and denoising. In the next step, an unsupervised multiscale line operator segmentation technique is used to segment the retinal vasculature before extracting the discriminating features. Finally, the Locally Consistent Gaussian Mixture Model (LCGMM) is utilised for unsupervised sorting of retinal vessels.

RESULTS

The performance of the proposed unsupervised method was assessed using three publicly accessible databases: INSPIRE-AVR, VICAVR, and MESSIDOR. The proposed framework achieved 90.14%,90.3% and 93.8% classification rate in zone B for the three datasets respectively.

CONCLUSIONS

The proposed clustering framework provided high classification rate as compared to conventional Gaussian mixture model using Expectation-Maximisation (GMM-EM) approach, thus have a great capability to enhance computer assisted diagnosis and research in field of biomarker discovery.

Vascular morphology and blood flow signatures for differential artery-vein analysis in optical coherence tomography of the retina

Differential artery-vein (AV) analysis is essential for retinal study, disease detection, and treatment assessment. This study is to characterize vascular reflectance profiles and blood flow patterns of retinal artery and vein systems in optical coherence tomography (OCT) and OCT angiography (OCTA), and establish them as robust signatures for objective AV classification. A custom designed OCT was employed for three-dimensional (3D) imaging of mouse retina, and corresponding OCTA was reconstructed. Radially resliced OCT B-scans revealed two, i.e. top and bottom, hyperreflective wall boundaries in retinal arteries, while these wall boundaries were absent in OCT of retinal veins. Additional OCTA analysis consistently displayed a layered speckle distribution in the vein, which may indicate the venous laminar flow. These OCT and OCTA differences offer unique signatures for objective AV classification in OCT and OCTA.

Topology-Aware Retinal Artery–Vein Classification via Deep Vascular Connectivity Prediction

Retinal artery–vein (AV) classification is a prerequisite for quantitative analysis of retinal vessels, which provides a biomarker for neurologic, cardiac, and systemic diseases, as well as ocular diseases. Although convolutional neural networks have presented remarkable performance on AV classification, it often comes with a topological error, like an abrupt class flipping on the same vessel segment or a weakness for thin vessels due to their indistinct appearances. In this paper, we present a new method for AV classification where the underlying vessel topology is estimated to give consistent prediction along the actual vessel structure. We cast the vessel topology estimation as iterative vascular connectivity prediction, which is implemented as deep-learning-based pairwise classification. In consequence, a whole vessel graph is separated into sub-trees, and each of them is classified as an artery or vein in whole via a voting scheme. The effectiveness and efficiency of the proposed method is validated by conducting experiments on two retinal image datasets acquired using different imaging techniques called DRIVE and IOSTAR.

A review of machine learning methods for retinal blood vessel segmentation and artery/vein classification

The eye affords a unique opportunity to inspect a rich part of the human microvasculature non-invasively via retinal imaging. Retinal blood vessel segmentation and classification are prime steps for the diagnosis and risk assessment of microvascular and systemic diseases. A high volume of techniques based on deep learning have been published in recent years. In this context, we review 158 papers published between 2012 and 2020, focussing on methods based on machine and deep learning (DL) for automatic vessel segmentation and classification for fundus camera images. We divide the methods into various classes by task (segmentation or artery-vein classification), technique (supervised or unsupervised, deep and non-deep learning, hand-crafted methods) and more specific algorithms (e.g. multiscale, morphology). We discuss advantages and limitations, and include tables summarising results at-a-glance. Finally, we attempt to assess the quantitative merit of DL methods in terms of accuracy improvement compared to other methods. The results allow us to offer our views on the outlook for vessel segmentation and classification for fundus camera images.

AVNet: A retinal artery/vein classification network with category-attention weighted fusion

BACKGROUND AND OBJECTIVE

Automatic artery/vein (A/V) classification in retinal images is of great importance in detecting vascular abnormalities, which may provide biomarkers for early diagnosis of many systemic diseases. It is intuitive to apply popular deep semantic segmentation network for A/V classification. However, the model is required to provide powerful representation ability since vessel is much more complex than general objects. Moreover, deep network may lead to inconsistent classification results for the same vessel due to the lack of structured optimization objective.

METHODS

In this paper, we propose a novel segmentation network named AVNet, which effectively enhances the classification ability of the model by integrating category-attention weighted fusion (CWF) module, significantly improving the pixel-level A/V classification results. Then, a graph based vascular structure reconstruction (VSR) algorithm is employed to reduce the segment-wise inconsistency, verifying the effect of the graph model on noisy vessel segmentation results.

RESULTS

The proposed method has been verified on three datasets, i.e. DRIVE, LES-AV and WIDE. AVNet achieves pixel-level accuracies of 90.62%, 90.34%, and 93.16%, respectively, and VSR further improves the performance by 0.19%, 1.85% and 0.64%, achieving the state-of-the-art results on these three datasets.

CONCLUSION

The proposed method achieves competitive performance in A/V classification task.

Multi-Task Siamese Network for Retinal Artery/Vein Separation via Deep Convolution Along Vessel

Vascular tree disentanglement and vessel type classification are two crucial steps of the graph-based method for retinal artery-vein (A/V) separation. Existing approaches treat them as two independent tasks and mostly rely on ad hoc rules (e.g. change of vessel directions) and hand-crafted features (e.g. color, thickness) to handle them respectively. However, we argue that the two tasks are highly correlated and should be handled jointly since knowing the A/V type can unravel those highly entangled vascular trees, which in turn helps to infer the types of connected vessels that are hard to classify based on only appearance. Therefore, designing features and models isolatedly for the two tasks often leads to a suboptimal solution of A/V separation. In view of this, this paper proposes a multi-task siamese network which aims to learn the two tasks jointly and thus yields more robust deep features for accurate A/V separation. Specifically, we first introduce Convolution Along Vessel (CAV) to extract the visual features by convolving a fundus image along vessel segments, and the geometric features by tracking the directions of blood flow in vessels. The siamese network is then trained to learn multiple tasks: i) classifying A/V types of vessel segments using visual features only, and ii) estimating the similarity of every two connected segments by comparing their visual and geometric features in order to disentangle the vasculature into individual vessel trees. Finally, the results of two tasks mutually correct each other to accomplish final A/V separation. Experimental results demonstrate that our method can achieve accuracy values of 94.7%, 96.9%, and 94.5% on three major databases (DRIVE, INSPIRE, WIDE) respectively, which outperforms recent state-of-the-arts.

From Local to Global: A Graph Framework for Retinal Artery/Vein Classification

Fundus photography has been widely used for inspecting eye disorders by ophthalmologists or computer algorithms. Biomarkers related to retinal vessels plays an essential role to detect early diabetes. To quantify vascular biomarkers or the corresponding changes, an accurate artery and vein classification is necessary. In this work, we propose a new framework to boost local vessel classification with a global vascular network model using graph convolution. We compare our proposed method with two traditional state-of-the-art methods on a testing dataset of 750 images from the Maastricht Study. After incorporating global information, our model achieves the best accuracy of 86.45% compared to 85.5% from convolutional neural networks (CNN) and 82.9% from handcrafted pixel feature classification (HPFC). Our model also obtains the best area under receiver operating characteristic curve (AUC) of 0.95, compared to 0.93 from CNN and 0.90 from HPFC. The new classification framework has the advantage of easy deployment on top of local classification features. It corrects the local classification error by minimizing global classification error and it brings free additional classification performance.

Artery-venous classification in fluorescein angiograms based on region growing with sequential and structural features

BACKGROUND AND OBJECTIVES

Fluorescein angiography (FA) is widely used in ophthalmology for examining retinal hemodynamics and vascular morphology. Artery-venous classification is an important step in FA image processing for measurement of feature parameters, such as arterio-venous passage time (AVP) and arterio-venous width ratio (AVR) that are proven useful in clinical assessment of circulation disturbance and vessel abnormalities. However, manual artery-venous classification needs expertise and is rather time consuming, and little effort has been devoted to develop automatic classification methods. In order to solve this problem, we propose a novel artery-venous classification method using region growing strategy with sequential and structural features (RGSS).

METHODS

The main procedures of our proposed RGSS method include: (i) registration of FA image sequence by mutual-information method; (ii) extraction of sequential features of the dye perfusion process from the registrated FA images; (iii) extraction of vessel structural features from vascular centerline map; (iv) based on the obtained features, seeds of arteries and veins within initial growing region (here optic disk) are generated and then propagated in the entire vessel network using region growing strategy. The RGSS method was tested on our own dataset and public Duke dataset, and its performance was evaluated quantitatively.

RESULTS

Tests show that RGSS method is able to classify arteries and veins from the complicated vessel network in FA images, with high classification accuracy of 0.91 ± 0.04 on Duke dataset and 0.92 ± 0.03 on our dataset. The employed sequential and structural features are demonstrated to be effective in classifying thin arteries and veins at vessel crossings.

CONCLUSIONS

Automatic artery-venous classification can be accomplished using our proposed RGSS method with high accuracy. The RGSS method not only emancipates ophthalmologists from hard work of manual marking of arteries and veins, but also helps in measuring important parameters (such as AVP and AVR) for clinical assessment of circulation disturbance and vessel abnormalities.

Retinal Vascular Network Topology Reconstruction and Artery/Vein Classification via Dominant Set Clustering

The estimation of vascular network topology in complex networks is important in understanding the relationship between vascular changes and a wide spectrum of diseases. Automatic classification of the retinal vascular trees into arteries and veins is of direct assistance to the ophthalmologist in terms of diagnosis and treatment of eye disease. However, it is challenging due to their projective ambiguity and subtle changes in appearance, contrast, and geometry in the imaging process. In this paper, we propose a novel method that is capable of making the artery/vein (A/V) distinction in retinal color fundus images based on vascular network topological properties. To this end, we adapt the concept of dominant set clustering and formalize the retinal blood vessel topology estimation and the A/V classification as a pairwise clustering problem. The graph is constructed through image segmentation, skeletonization, and identification of significant nodes. The edge weight is defined as the inverse Euclidean distance between its two end points in the feature space of intensity, orientation, curvature, diameter, and entropy. The reconstructed vascular network is classified into arteries and veins based on their intensity and morphology. The proposed approach has been applied to five public databases, namely INSPIRE, IOSTAR, VICAVR, DRIVE, and WIDE, and achieved high accuracies of 95.1%, 94.2%, 93.8%, 91.1%, and 91.0%, respectively. Furthermore, we have made manual annotations of the blood vessel topologies for INSPIRE, IOSTAR, VICAVR, and DRIVE datasets, and these annotations are released for public access so as to facilitate researchers in the community.

Artery/vein classification of retinal vessels using classifiers fusion

The morphological changes in retinal blood vessels indicate cardiovascular diseases and consequently those diseases lead to ocular complications such as Hypertensive Retinopathy. One of the significant clinical findings related to this ocular abnormality is alteration of width of vessel. The classification of retinal vessels into arteries and veins in eye fundus images is a relevant task for the automatic assessment of vascular changes. This paper presents an important approach to solve this problem by means of feature ranking strategies and multiple classifiers decision-combination scheme that is specifically adapted for artery/vein classification. For this, three databases are used with a local dataset of 44 images and two publically available databases, INSPIRE-AVR containing 40 images and VICAVR containing 58 images. The local database also contains images with pathologically diseased structures. The performance of the proposed system is assessed by comparing the experimental results with the gold standard estimations as well as with the results of previous methodologies, achieving promising classification performance, with an over all accuracy of 90.45%, 93.90% and 87.82%, in retinal blood vessel separation for Local, INSPIRE-AVR and VICAVR dataset, respectively.

Multiloss Function Based Deep Convolutional Neural Network for Segmentation of Retinal Vasculature into Arterioles and Venules

The arterioles and venules (AV) classification of retinal vasculature is considered as the first step in the development of an automated system for analysing the vasculature biomarker association with disease prognosis. Most of the existing AV classification methods depend on the accurate segmentation of retinal blood vessels. Moreover, the unavailability of large-scale annotated data is a major hindrance in the application of deep learning techniques for AV classification. This paper presents an encoder-decoder based fully convolutional neural network for classification of retinal vasculature into arterioles and venules, without requiring the preliminary step of vessel segmentation. An optimized multiloss function is used to learn the pixel-wise and segment-wise retinal vessel labels. The proposed method is trained and evaluated on DRIVE, AVRDB, and a newly created AV classification dataset; and it attains 96%, 98%, and 97% accuracy, respectively. The new AV classification dataset is comprised of 700 annotated retinal images, which will offer the researchers a benchmark to compare their AV classification results.

Retinal Artery and Vein Classification for Automatic Vessel Caliber Grading

Automated retinal artery and vein identification is a necessity to measure their caliber automatically and to achieve high efficiency and repeatability for a large number of images. In this paper, a novel framework for retinal artery and vein classification is provided. The proposed method utilizes the vessel crossover and color intensity profile which are the most significant features for artery and vein classification. The method first extracts retinal vascular network and then identify individual blood vessels for further classification as artery or vein. We apply deep learning algorithm based segmentation method to extract the retinal vascular network. We then identify each blood vessels to measure caliber that will be used for computing the Central Retinal Artery Equivalent (CRAE) and Central Retinal Vein Equivalent (CRVE). We map the vessel network and use the individual vessel crossover information, vessel color and intensity profile to identify individual vessel segment as artery and vein. We compared automatically classified artery and vein results with a human grader which showed an accuracy of 95%. We compare our results of caliber grading against an established semi-automated caliber grading system and protocol which showed a very high correlation of 0.85 and 0.92, for CRAE and CRVE respectively.

Network-based features for retinal fundus vessel structure analysis

Retinal fundus imaging is a non-invasive method that allows visualizing the structure of the blood vessels in the retina whose features may indicate the presence of diseases such as diabetic retinopathy (DR) and glaucoma. Here we present a novel method to analyze and quantify changes in the retinal blood vessel structure in patients diagnosed with glaucoma or with DR. First, we use an automatic unsupervised segmentation algorithm to extract a tree-like graph from the retina blood vessel structure. The nodes of the graph represent branching (bifurcation) points and endpoints, while the links represent vessel segments that connect the nodes. Then, we quantify structural differences between the graphs extracted from the groups of healthy and non-healthy patients. We also use fractal analysis to characterize the extracted graphs. Applying these techniques to three retina fundus image databases we find significant differences between the healthy and non-healthy groups (p-values lower than 0.005 or 0.001 depending on the method and on the database). The results are sensitive to the segmentation method (manual or automatic) and to the resolution of the images.

Artery-vein segmentation in fundus images using a fully convolutional network

Epidemiological studies demonstrate that dimensions of retinal vessels change with ocular diseases, coronary heart disease and stroke. Different metrics have been described to quantify these changes in fundus images, with arteriolar and venular calibers among the most widely used. The analysis often includes a manual procedure during which a trained grader differentiates between arterioles and venules. This step can be time-consuming and can introduce variability, especially when large volumes of images need to be analyzed. In light of the recent successes of fully convolutional networks (FCNs) applied to biomedical image segmentation, we assess its potential in the context of retinal artery-vein (A/V) discrimination. To the best of our knowledge, a deep learning (DL) architecture for simultaneous vessel extraction and A/V discrimination has not been previously employed. With the aim of improving the automation of vessel analysis, a novel application of the U-Net semantic segmentation architecture (based on FCNs) on the discrimination of arteries and veins in fundus images is presented. By utilizing DL, results are obtained that exceed accuracies reported in the literature. Our model was trained and tested on the public DRIVE and HRF datasets. For DRIVE, measuring performance on vessels wider than two pixels, the FCN achieved accuracies of 94.42% and 94.11% on arteries and veins, respectively. This represents a decrease in error of 25% over the previous state of the art reported by Xu et al. (2017). Additionally, we introduce the HRF A/V ground truth, on which our model achieves 96.98% accuracy on all discovered centerline pixels. HRF A/V ground truth validated by an ophthalmologist, predicted A/V annotations and evaluation code are available at https://github.com/rubenhx/av-segmentation.

Automated Method for Retinal Artery/Vein Separation via Graph Search Metaheuristic Approach

Separation of the vascular tree into arteries and veins is a fundamental prerequisite in the automatic diagnosis of retinal biomarkers associated with systemic and neurodegenerative diseases. In this paper, we present a novel graph search metaheuristic approach for automatic separation of arteries/veins (A/V) from color fundus images. Our method exploits local information to disentangle the complex vascular tree into multiple subtrees, and global information to label these vessel subtrees into arteries and veins. Given a binary vessel map, a graph representation of the vascular network is constructed representing the topological and spatial connectivity of the vascular structures. Based on the anatomical uniqueness at vessel crossing and branching points, the vascular tree is split into multiple subtrees containing arteries and veins. Finally, the identified vessel subtrees are labeled with A/V based on a set of handcrafted features trained with random forest classifier. The proposed method has been tested on four different publicly available retinal datasets with an average accuracy of 94.7%, 93.2%, 96.8% and 90.2% across AV-DRIVE, CT-DRIVE. INSPIRE-AVR and WIDE datasets, respectively. These results demonstrate the superiority of our proposed approach in outperforming state-ofthe- art methods for A/V separation.