A Graph Cut Approach to Artery/Vein Classification in Ultra-Widefield Scanning Laser Ophthalmoscopy

Arteries and veins in awake mice using two-photon microscopy

Distinguishing arteries from veins in the cerebral cortex is critical for studying hemodynamics under pathophysiological conditions, which plays an important role in the diagnosis and treatment of various vessel-related diseases. However, due to the complexity of the cerebral vascular network, it is challenging to identify arteries and veins in vivo. Here, we demonstrate an artery-vein separation method that employs a combination of multiple scanning modes of two-photon microscopy and a custom-designed stereoscopic fixation device for mice. In this process, we propose a novel method for determining the line scanning direction, which allows us to determine the blood flow directions. The vasculature branches have been identified using an optimized z-stack scanning mode, followed by the separation of blood vessel types according to the directions of blood flow and branching patterns. Using this strategy, the penetrating arterioles and penetrating venules in awake mice could be accurately identified and the type of cerebral thrombus has been also successfully isolated without any empirical knowledge or algorithms. Our research presents a new, more accurate, and efficient method for cortical artery-vein separation in awake mice, providing a useful strategy for the application of two-photon microscopy in the study of cerebrovascular pathophysiology.

Automatic artery/vein classification methods for retinal blood vessel: A review

Automatic retinal arteriovenous classification can assist ophthalmologists in disease early diagnosis. Deep learning-based methods and topological graph-based methods have become the main solutions for retinal arteriovenous classification in recent years. This paper reviews the automatic retinal arteriovenous classification methods from 2003 to 2022. Firstly, we compare different methods and provide comparison tables of the summary results. Secondly, we complete the classification of the public arteriovenous classification datasets and provide the annotation development tables of different datasets. Finally, we sort out the challenges of evaluation methods and provide a comprehensive evaluation system. Quantitative and qualitative analysis shows the changes in research hotspots over time, Quantitative and qualitative analyses reveal the evolution of research hotspots over time, highlighting the significance of exploring the integration of deep learning with topological information in future research.

Ultra-Widefield Imaging of the Retinal Macrovasculature in Parkinson Disease Versus Controls With Normal Cognition Using Alpha-Shapes Analysis

Purpose

To investigate retinal vascular characteristics using ultra-widefield (UWF) scanning laser ophthalmoscopy in Parkinson disease (PD).

Methods

Individuals with an expert-confirmed clinical diagnosis of PD and controls with normal cognition without PD underwent Optos California UWF imaging. Patients with diabetes, uncontrolled hypertension, glaucoma, dementia, other movement disorders, or known retinal or optic nerve pathology were excluded. Images were analyzed using Vasculature Assessment and Measurement Platform for Images of the Retina (VAMPIRE-UWF) software, which describes retinal vessel width gradient and tortuosity, provides vascular network fractal dimensions, and conducts alpha-shape analysis to further characterize vascular morphology (complexity, Opαmin; spread, OpA).

Results

In the PD cohort, 53 eyes of 38 subjects were assessed; in the control cohort, 51 eyes of 33 subjects were assessed. Eyes with PD had more tortuous retinal arteries in the superotemporal quadrant (P = 0.043). In eyes with PD, alpha-shape analysis revealed decreased OpA, indicating less retinal vasculature spread compared to controls (P = 0.032). Opαmin was decreased in PD (P = 0.044), suggesting increased vascular network complexity. No differences were observed in fractal dimension in any region of interest.

Conclusions

This pilot study suggests that retinal vasculature assessment on UWF images using alpha-shape analysis reveals differences in retinal vascular network spread and complexity in PD and may be a more sensitive metric compared to fractal dimension.

Translational Relevance

Retinal vasculature assessment using these novel methods may be useful in understanding ocular manifestations of PD and the development of retinal biomarkers.

Computer-aided diagnostic system for hypertensive retinopathy: A review

Hypertensive Retinopathy (HR) is a retinal disease caused by elevated blood pressure for a prolonged period. There are no obvious signs in the early stages of high blood pressure, but it affects various body parts over time, including the eyes. HR is a biomarker for several illnesses, including retinal diseases, atherosclerosis, strokes, kidney disease, and cardiovascular risks. Early microcirculation abnormalities in chronic diseases can be diagnosed through retinal examination prior to the onset of major clinical consequences. Computer-aided diagnosis (CAD) plays a vital role in the early identification of HR with improved diagnostic accuracy, which is time-efficient and demands fewer resources. Recently, numerous studies have been reported on the automatic identification of HR. This paper provides a comprehensive review of the automated tasks of Artery-Vein (A/V) classification, Arteriovenous ratio (AVR) computation, HR detection (Binary classification), and HR severity grading. The review is conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol. The paper discusses the clinical features of HR, the availability of datasets, existing methods used for A/V classification, AVR computation, HR detection, and severity grading, and performance evaluation metrics. The reviewed articles are summarized with classifiers details, adoption of different kinds of methodologies, performance comparisons, datasets details, their pros and cons, and computational platform. For each task, a summary and critical in-depth analysis are provided, as well as common research issues and challenges in the existing studies. Finally, the paper proposes future research directions to overcome challenges associated with data set availability, HR detection, and severity grading.

Segmentation and Classification Approaches of Clinically Relevant Curvilinear Structures: A Review

Detection of curvilinear structures from microscopic images, which help the clinicians to make an unambiguous diagnosis is assuming paramount importance in recent clinical practice. Appearance and size of dermatophytic hyphae, keratitic fungi, corneal and retinal vessels vary widely making their automated detection cumbersome. Automated deep learning methods, endowed with superior self-learning capacity, have superseded the traditional machine learning methods, especially in complex images with challenging background. Automatic feature learning ability using large input data with better generalization and recognition capability, but devoid of human interference and excessive pre-processing, is highly beneficial in the above context. Varied attempts have been made by researchers to overcome challenges such as thin vessels, bifurcations and obstructive lesions in retinal vessel detection as revealed through several publications reviewed here. Revelations of diabetic neuropathic complications such as tortuosity, changes in the density and angles of the corneal fibers have been successfully sorted in many publications reviewed here. Since artifacts complicate the images and affect the quality of analysis, methods addressing these challenges have been described. Traditional and deep learning methods, that have been adapted and published between 2015 and 2021 covering retinal vessels, corneal nerves and filamentous fungi have been summarized in this review. We find several novel and meritorious ideas and techniques being put to use in the case of retinal vessel segmentation and classification, which by way of cross-domain adaptation can be utilized in the case of corneal and filamentous fungi also, making suitable adaptations to the challenges to be addressed.

Retinal Vascular Changes in Alzheimer's Dementia and Mild Cognitive Impairment: A Pilot Study Using Ultra-Widefield Imaging

Purpose

Retinal microvascular abnormalities measured on retinal images are a potential source of prognostic biomarkers of vascular changes in the neurodegenerating brain. We assessed the presence of these abnormalities in Alzheimer's dementia and mild cognitive impairment (MCI) using ultra-widefield (UWF) retinal imaging.

Methods

UWF images from 103 participants (28 with Alzheimer's dementia, 30 with MCI, and 45 with normal cognition) underwent analysis to quantify measures of retinal vascular branching complexity, width, and tortuosity.

Results

Participants with Alzheimer's dementia displayed increased vessel branching in the midperipheral retina and increased arteriolar thinning. Participants with MCI displayed increased rates of arteriolar and venular thinning and a trend for decreased vessel branching.

Conclusions

Statistically significant differences in the retinal vasculature in peripheral regions of the retina were observed among the distinct cognitive stages. However, larger studies are required to establish the clinical importance of our findings. UWF imaging may be a promising modality to assess a larger view of the retinal vasculature to uncover retinal changes in Alzheimer's disease.

Translational Relevance

This pilot work reports an investigation into which retinal vasculature measurements may be useful surrogate measures of cognitive decline, as well as technical developments (e.g., measurement standardization), that are first required to establish their recommended use and translational potential.

Origami Inspired Laser Scanner

Diverse origami techniques and various selections of paper open new possibilities to create micromachines. By folding paper, this article proposes an original approach to build laser scanners, which manipulate optical beams precisely and realize valuable applications, including laser marking, cutting, engraving, and displaying. A prototype has been designed, implemented, actuated, and controlled. The experimental results demonstrate that the angular stroke, repeatability, full scale settling time, and resonant frequency are 20°, 0.849 m°, 330 ms, 68 Hz, respectively. Its durability, more than 35 million cycles, shows the potential to carry out serious tasks.

Retinal vascular measures from diabetes retinal screening photographs and risk of incident dementia in type 2 diabetes: A GoDARTS study

Objective

Patients with diabetes have an increased risk of dementia. Improved prediction of dementia is an important goal in developing future prevention strategies. Diabetic retinopathy screening (DRS) photographs may be a convenient source of imaging biomarkers of brain health. We therefore investigated the association of retinal vascular measures (RVMs) from DRS photographs in patients with type 2 diabetes with dementia risk.

Research Design and Methods

RVMs were obtained from 6,111 patients in the GoDARTS bioresource (635 incident cases) using VAMPIRE software. Their association, independent of Apo E4 genotype and clinical parameters, was determined for incident all cause dementia (ACD) and separately Alzheimer's disease (AD) and vascular dementia (VD). We used Cox's proportional hazards with competing risk of death without dementia. The potential value of RVMs to increase the accuracy of risk prediction was evaluated.

Results

Increased retinal arteriolar fractal dimension associated with increased risk of ACD (csHR 1.17; 1.08-1.26) and AD (HR 1.33; 1.16-1.52), whereas increased venular fractal dimension (FDV) was associated with reduced risk of AD (csHR 0.85; 0.74-0.96). Conversely, FDV was associated with increased risk of VD (csHR 1.22; 1.07-1.40). Wider arteriolar calibre was associated with a reduced risk of ACD (csHR 0.9; 0.83-0.98) and wider venular calibre was associated with a reduced risk of AD (csHR 0.87; 0.78-0.97). Accounting for competing risk did not substantially alter these findings. RVMs significantly increased the accuracy of prediction.

Conclusions

Conventional DRS photographs could enhance stratifying patients with diabetes at increased risk of dementia facilitating the development of future prevention strategies.

Improving U-Net architecture and graph cuts optimization to classify arterioles and venules in retina fundus images

The central retinal artery and its branches supply blood to the inner retina. Vascular manifestations in the retina indirectly reflect the vascular changes and damage in organs such as the heart, kidneys, and brain because of the similar vascular structure of these organs. The diabetic retinopathy and risk of stroke are caused by increased venular caliber. The degrees of these diseases depend on the changes of arterioles and venules. The ratio between the calibers of arterioles and venules (AVR) is various. AVR is considered as the useful diagnostic indicator of different associated health problems. However, the task is not easy because of the lack of information of the features being used to classify the retinal vessels as arterioles and venules. This paper proposed a method to classify the retinal vessels into the arterioles and venules based on improving U-Net architecture and graph cuts. The accuracy of the proposed method is about 97.6%. The results of the proposed method are better than the other methods in RITE dataset and AVRDB dataset.

Automatic Artery/Vein Classification Using a Vessel-Constraint Network for Multicenter Fundus Images

Retinal blood vessel morphological abnormalities are generally associated with cardiovascular, cerebrovascular, and systemic diseases, automatic artery/vein (A/V) classification is particularly important for medical image analysis and clinical decision making. However, the current method still has some limitations in A/V classification, especially the blood vessel edge and end error problems caused by the single scale and the blurred boundary of the A/V. To alleviate these problems, in this work, we propose a vessel-constraint network (VC-Net) that utilizes the information of vessel distribution and edge to enhance A/V classification, which is a high-precision A/V classification model based on data fusion. Particularly, the VC-Net introduces a vessel-constraint (VC) module that combines local and global vessel information to generate a weight map to constrain the A/V features, which suppresses the background-prone features and enhances the edge and end features of blood vessels. In addition, the VC-Net employs a multiscale feature (MSF) module to extract blood vessel information with different scales to improve the feature extraction capability and robustness of the model. And the VC-Net can get vessel segmentation results simultaneously. The proposed method is tested on publicly available fundus image datasets with different scales, namely, DRIVE, LES, and HRF, and validated on two newly created multicenter datasets: Tongren and Kailuan. We achieve a balance accuracy of 0.9554 and F1 scores of 0.7616 and 0.7971 for the arteries and veins, respectively, on the DRIVE dataset. The experimental results prove that the proposed model achieves competitive performance in A/V classification and vessel segmentation tasks compared with state-of-the-art methods. Finally, we test the Kailuan dataset with other trained fusion datasets, the results also show good robustness. To promote research in this area, the Tongren dataset and source code will be made publicly available. The dataset and code will be made available at https://github.com/huawang123/VC-Net.

Cross-attention multi-branch network for fundus diseases classification using SLO images

Fundus diseases classification is vital for the health of human beings. However, most of existing methods detect diseases by means of single angle fundus images, which lead to the lack of pathological information. To address this limitation, this paper proposes a novel deep learning method to complete different fundus diseases classification tasks using ultra-wide field scanning laser ophthalmoscopy (SLO) images, which have an ultra-wide field view of 180-200˚. The proposed deep model consists of multi-branch network, atrous spatial pyramid pooling module (ASPP), cross-attention and depth-wise attention module. Specifically, the multi-branch network employs the ResNet-34 model as the backbone to extract feature information, where the ResNet-34 model with two-branch is followed by the ASPP module to extract multi-scale spatial contextual features by setting different dilated rates. The depth-wise attention module can provide the global attention map from the multi-branch network, which enables the network to focus on the salient targets of interest. The cross-attention module adopts the cross-fusion mode to fuse the channel and spatial attention maps from the ResNet-34 model with two-branch, which can enhance the representation ability of the disease-specific features. The extensive experiments on our collected SLO images and two publicly available datasets demonstrate that the proposed method can outperform the state-of-the-art methods and achieve quite promising classification performance of the fundus diseases.

A review of machine learning methods for retinal blood vessel segmentation and artery/vein classification

The eye affords a unique opportunity to inspect a rich part of the human microvasculature non-invasively via retinal imaging. Retinal blood vessel segmentation and classification are prime steps for the diagnosis and risk assessment of microvascular and systemic diseases. A high volume of techniques based on deep learning have been published in recent years. In this context, we review 158 papers published between 2012 and 2020, focussing on methods based on machine and deep learning (DL) for automatic vessel segmentation and classification for fundus camera images. We divide the methods into various classes by task (segmentation or artery-vein classification), technique (supervised or unsupervised, deep and non-deep learning, hand-crafted methods) and more specific algorithms (e.g. multiscale, morphology). We discuss advantages and limitations, and include tables summarising results at-a-glance. Finally, we attempt to assess the quantitative merit of DL methods in terms of accuracy improvement compared to other methods. The results allow us to offer our views on the outlook for vessel segmentation and classification for fundus camera images.

AVNet: A retinal artery/vein classification network with category-attention weighted fusion

BACKGROUND AND OBJECTIVE

Automatic artery/vein (A/V) classification in retinal images is of great importance in detecting vascular abnormalities, which may provide biomarkers for early diagnosis of many systemic diseases. It is intuitive to apply popular deep semantic segmentation network for A/V classification. However, the model is required to provide powerful representation ability since vessel is much more complex than general objects. Moreover, deep network may lead to inconsistent classification results for the same vessel due to the lack of structured optimization objective.

METHODS

In this paper, we propose a novel segmentation network named AVNet, which effectively enhances the classification ability of the model by integrating category-attention weighted fusion (CWF) module, significantly improving the pixel-level A/V classification results. Then, a graph based vascular structure reconstruction (VSR) algorithm is employed to reduce the segment-wise inconsistency, verifying the effect of the graph model on noisy vessel segmentation results.

RESULTS

The proposed method has been verified on three datasets, i.e. DRIVE, LES-AV and WIDE. AVNet achieves pixel-level accuracies of 90.62%, 90.34%, and 93.16%, respectively, and VSR further improves the performance by 0.19%, 1.85% and 0.64%, achieving the state-of-the-art results on these three datasets.

CONCLUSION

The proposed method achieves competitive performance in A/V classification task.

AMD-GAN: Attention encoder and multi-branch structure based generative adversarial networks for fundus disease detection from scanning laser ophthalmoscopy images

The scanning laser ophthalmoscopy (SLO) has become an important tool for the determination of peripheral retinal pathology, in recent years. However, the collected SLO images are easily interfered by the eyelash and frame of the devices, which heavily affect the key feature extraction of the images. To address this, we propose a generative adversarial network called AMD-GAN based on the attention encoder (AE) and multi-branch (MB) structure for fundus disease detection from SLO images. Specifically, the designed generator consists of two parts: the AE and generation flow network, where the real SLO images are encoded by the AE module to extract features and the generation flow network to handle the random Gaussian noise by a series of residual block with up-sampling (RU) operations to generate fake images with the same size as the real ones, where the AE is also used to mine features for generator. For discriminator, a ResNet network using MB is devised by copying the stage 3 and stage 4 structures of the ResNet-34 model to extract deep features. Furthermore, the depth-wise asymmetric dilated convolution is leveraged to extract local high-level contextual features and accelerate the training process. Besides, the last layer of discriminator is modified to build the classifier to detect the diseased and normal SLO images. In addition, the prior knowledge of experts is utilized to improve the detection results. Experimental results on the two local SLO datasets demonstrate that our proposed method is promising in detecting the diseased and normal SLO images with the experts labeling.

Multi-Task Siamese Network for Retinal Artery/Vein Separation via Deep Convolution Along Vessel

Vascular tree disentanglement and vessel type classification are two crucial steps of the graph-based method for retinal artery-vein (A/V) separation. Existing approaches treat them as two independent tasks and mostly rely on ad hoc rules (e.g. change of vessel directions) and hand-crafted features (e.g. color, thickness) to handle them respectively. However, we argue that the two tasks are highly correlated and should be handled jointly since knowing the A/V type can unravel those highly entangled vascular trees, which in turn helps to infer the types of connected vessels that are hard to classify based on only appearance. Therefore, designing features and models isolatedly for the two tasks often leads to a suboptimal solution of A/V separation. In view of this, this paper proposes a multi-task siamese network which aims to learn the two tasks jointly and thus yields more robust deep features for accurate A/V separation. Specifically, we first introduce Convolution Along Vessel (CAV) to extract the visual features by convolving a fundus image along vessel segments, and the geometric features by tracking the directions of blood flow in vessels. The siamese network is then trained to learn multiple tasks: i) classifying A/V types of vessel segments using visual features only, and ii) estimating the similarity of every two connected segments by comparing their visual and geometric features in order to disentangle the vasculature into individual vessel trees. Finally, the results of two tasks mutually correct each other to accomplish final A/V separation. Experimental results demonstrate that our method can achieve accuracy values of 94.7%, 96.9%, and 94.5% on three major databases (DRIVE, INSPIRE, WIDE) respectively, which outperforms recent state-of-the-arts.

Retinal Vascular Network Topology Reconstruction and Artery/Vein Classification via Dominant Set Clustering

The estimation of vascular network topology in complex networks is important in understanding the relationship between vascular changes and a wide spectrum of diseases. Automatic classification of the retinal vascular trees into arteries and veins is of direct assistance to the ophthalmologist in terms of diagnosis and treatment of eye disease. However, it is challenging due to their projective ambiguity and subtle changes in appearance, contrast, and geometry in the imaging process. In this paper, we propose a novel method that is capable of making the artery/vein (A/V) distinction in retinal color fundus images based on vascular network topological properties. To this end, we adapt the concept of dominant set clustering and formalize the retinal blood vessel topology estimation and the A/V classification as a pairwise clustering problem. The graph is constructed through image segmentation, skeletonization, and identification of significant nodes. The edge weight is defined as the inverse Euclidean distance between its two end points in the feature space of intensity, orientation, curvature, diameter, and entropy. The reconstructed vascular network is classified into arteries and veins based on their intensity and morphology. The proposed approach has been applied to five public databases, namely INSPIRE, IOSTAR, VICAVR, DRIVE, and WIDE, and achieved high accuracies of 95.1%, 94.2%, 93.8%, 91.1%, and 91.0%, respectively. Furthermore, we have made manual annotations of the blood vessel topologies for INSPIRE, IOSTAR, VICAVR, and DRIVE datasets, and these annotations are released for public access so as to facilitate researchers in the community.

Automated Method for Retinal Artery/Vein Separation via Graph Search Metaheuristic Approach

Separation of the vascular tree into arteries and veins is a fundamental prerequisite in the automatic diagnosis of retinal biomarkers associated with systemic and neurodegenerative diseases. In this paper, we present a novel graph search metaheuristic approach for automatic separation of arteries/veins (A/V) from color fundus images. Our method exploits local information to disentangle the complex vascular tree into multiple subtrees, and global information to label these vessel subtrees into arteries and veins. Given a binary vessel map, a graph representation of the vascular network is constructed representing the topological and spatial connectivity of the vascular structures. Based on the anatomical uniqueness at vessel crossing and branching points, the vascular tree is split into multiple subtrees containing arteries and veins. Finally, the identified vessel subtrees are labeled with A/V based on a set of handcrafted features trained with random forest classifier. The proposed method has been tested on four different publicly available retinal datasets with an average accuracy of 94.7%, 93.2%, 96.8% and 90.2% across AV-DRIVE, CT-DRIVE. INSPIRE-AVR and WIDE datasets, respectively. These results demonstrate the superiority of our proposed approach in outperforming state-ofthe- art methods for A/V separation.