A review of machine learning methods for retinal blood vessel segmentation and artery/vein classification

Predicting systemic diseases in fundus images: systematic review of setting, reporting, bias, and models' clinical availability in deep learning studies

BACKGROUND

Analyzing fundus images with deep learning techniques is promising for screening systematic diseases. However, the quality of the rapidly increasing number of studies was variable and lacked systematic evaluation.

OBJECTIVE

To systematically review all the articles that aimed to predict systemic parameters and conditions using fundus image and deep learning, assessing their performance, and providing suggestions that would enable translation into clinical practice.

METHODS

Two major electronic databases (MEDLINE and EMBASE) were searched until August 22, 2023, with keywords 'deep learning' and 'fundus'. Studies using deep learning and fundus images to predict systematic parameters were included, and assessed in four aspects: study characteristics, transparent reporting, risk of bias, and clinical availability. Transparent reporting was assessed by the TRIPOD statement, while the risk of bias was assessed by PROBAST.

RESULTS

4969 articles were identified through systematic research. Thirty-one articles were included in the review. A variety of vascular and non-vascular diseases can be predicted by fundus images, including diabetes and related diseases (19%), sex (22%) and age (19%). Most of the studies focused on developed countries. The models' reporting was insufficient in determining sample size and missing data treatment according to the TRIPOD. Full access to datasets and code was also under-reported. 1/31(3.2%) study was classified as having a low risk of bias overall, whereas 30/31(96.8%) were classified as having a high risk of bias according to the PROBAST. 5/31(16.1%) of studies used prospective external validation cohorts. Only two (6.4%) described the study's calibration. The number of publications by year increased significantly from 2018 to 2023. However, only two models (6.5%) were applied to the device, and no model has been applied in clinical.

CONCLUSION

Deep learning fundus images have shown great potential in predicting systematic conditions in clinical situations. Further work needs to be done to improve the methodology and clinical application.

Linear semantic transformation for semi-supervised medical image segmentation

Medical image segmentation is a focus research and foundation in developing intelligent medical systems. Recently, deep learning for medical image segmentation has become a standard process and succeeded significantly, promoting the development of reconstruction, and surgical planning of disease diagnosis. However, semantic learning is often inefficient owing to the lack of supervision of feature maps, resulting in that high-quality segmentation models always rely on numerous and accurate data annotations. Learning robust semantic representation in latent spaces remains a challenge. In this paper, we propose a novel semi-supervised learning framework to learn vital attributes in medical images, which constructs generalized representation from diverse semantics to realize medical image segmentation. We first build a self-supervised learning part that achieves context recovery by reconstructing space and intensity of medical images, which conduct semantic representation for feature maps. Subsequently, we combine semantic-rich feature maps and utilize simple linear semantic transformation to convert them into image segmentation. The proposed framework was tested using five medical segmentation datasets. Quantitative assessments indicate the highest scores of our method on IXI (73.78%), ScaF (47.50%), COVID-19-Seg (50.72%), PC-Seg (65.06%), and Brain-MR (72.63%) datasets. Finally, we compared our method with the latest semi-supervised learning methods and obtained 77.15% and 75.22% DSC values, respectively, ranking first on two representative datasets. The experimental results not only proved that the proposed linear semantic transformation was effectively applied to medical image segmentation, but also presented its simplicity and ease-of-use to pursue robust segmentation in semi-supervised learning. Our code is now open at: https://github.com/QingYunA/Linear-Semantic-Transformation-for-Semi-Supervised-Medical-Image-Segmentation.

Cross-modality Labeling Enables Noninvasive Capillary Quantification as a Sensitive Biomarker for Assessing Cardiovascular Risk

Purpose

We aim to use fundus fluorescein angiography (FFA) to label the capillaries on color fundus (CF) photographs and train a deep learning model to quantify retinal capillaries noninvasively from CF and apply it to cardiovascular disease (CVD) risk assessment.

Design

Cross-sectional and longitudinal study.

Participants

A total of 90732 pairs of CF-FFA images from 3893 participants for segmentation model development, and 49229 participants in the UK Biobank for association analysis.

Methods

We matched the vessels extracted from FFA and CF, and used vessels from FFA as labels to train a deep learning model (RMHAS-FA) to segment retinal capillaries using CF. We tested the model's accuracy on a manually labeled internal test set (FundusCapi). For external validation, we tested the segmentation model on 7 vessel segmentation datasets, and investigated the clinical value of the segmented vessels in predicting CVD events in the UK Biobank.

Main Outcome Measures

Area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity for segmentation. Hazard ratio (HR; 95% confidence interval [CI]) for Cox regression analysis.

Results

On the FundusCapi dataset, the segmentation performance was AUC = 0.95, accuracy = 0.94, sensitivity = 0.90, and specificity = 0.93. Smaller vessel skeleton density had a stronger correlation with CVD risk factors and incidence (P < 0.01). Reduced density of small vessel skeletons was strongly associated with an increased risk of CVD incidence and mortality for women (HR [95% CI] = 0.91 [0.84-0.98] and 0.68 [0.54-0.86], respectively).

Conclusions

Using paired CF-FFA images, we automated the laborious manual labeling process and enabled noninvasive capillary quantification from CF, supporting its potential as a sensitive screening method for identifying individuals at high risk of future CVD events.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

SurgNet: Self-Supervised Pretraining With Semantic Consistency for Vessel and Instrument Segmentation in Surgical Images

Blood vessel and surgical instrument segmentation is a fundamental technique for robot-assisted surgical navigation. Despite the significant progress in natural image segmentation, surgical image-based vessel and instrument segmentation are rarely studied. In this work, we propose a novel self-supervised pretraining method (SurgNet) that can effectively learn representative vessel and instrument features from unlabeled surgical images. As a result, it allows for precise and efficient segmentation of vessels and instruments with only a small amount of labeled data. Specifically, we first construct a region adjacency graph (RAG) based on local semantic consistency in unlabeled surgical images and use it as a self-supervision signal for pseudo-mask segmentation. We then use the pseudo-mask to perform guided masked image modeling (GMIM) to learn representations that integrate structural information of intraoperative objectives more effectively. Our pretrained model, paired with various segmentation methods, can be applied to perform vessel and instrument segmentation accurately using limited labeled data for fine-tuning. We build an Intraoperative Vessel and Instrument Segmentation (IVIS) dataset, comprised of ~3 million unlabeled images and over 4,000 labeled images with manual vessel and instrument annotations to evaluate the effectiveness of our self-supervised pretraining method. We also evaluated the generalizability of our method to similar tasks using two public datasets. The results demonstrate that our approach outperforms the current state-of-the-art (SOTA) self-supervised representation learning methods in various surgical image segmentation tasks.

Automatic artery/vein classification methods for retinal blood vessel: A review

Automatic retinal arteriovenous classification can assist ophthalmologists in disease early diagnosis. Deep learning-based methods and topological graph-based methods have become the main solutions for retinal arteriovenous classification in recent years. This paper reviews the automatic retinal arteriovenous classification methods from 2003 to 2022. Firstly, we compare different methods and provide comparison tables of the summary results. Secondly, we complete the classification of the public arteriovenous classification datasets and provide the annotation development tables of different datasets. Finally, we sort out the challenges of evaluation methods and provide a comprehensive evaluation system. Quantitative and qualitative analysis shows the changes in research hotspots over time, Quantitative and qualitative analyses reveal the evolution of research hotspots over time, highlighting the significance of exploring the integration of deep learning with topological information in future research.

CF-Loss: Clinically-relevant feature optimised loss function for retinal multi-class vessel segmentation and vascular feature measurement

Characterising clinically-relevant vascular features, such as vessel density and fractal dimension, can benefit biomarker discovery and disease diagnosis for both ophthalmic and systemic diseases. In this work, we explicitly encode vascular features into an end-to-end loss function for multi-class vessel segmentation, categorising pixels into artery, vein, uncertain pixels, and background. This clinically-relevant feature optimised loss function (CF-Loss) regulates networks to segment accurate multi-class vessel maps that produce precise vascular features. Our experiments first verify that CF-Loss significantly improves both multi-class vessel segmentation and vascular feature estimation, with two standard segmentation networks, on three publicly available datasets. We reveal that pixel-based segmentation performance is not always positively correlated with accuracy of vascular features, thus highlighting the importance of optimising vascular features directly via CF-Loss. Finally, we show that improved vascular features from CF-Loss, as biomarkers, can yield quantitative improvements in the prediction of ischaemic stroke, a real-world clinical downstream task. The code is available at https://github.com/rmaphoh/feature-loss.

OCTA-500: A retinal dataset for optical coherence tomography angiography study

Optical coherence tomography angiography (OCTA) is a novel imaging modality that has been widely utilized in ophthalmology and neuroscience studies to observe retinal vessels and microvascular systems. However, publicly available OCTA datasets remain scarce. In this paper, we introduce the largest and most comprehensive OCTA dataset dubbed OCTA-500, which contains OCTA imaging under two fields of view (FOVs) from 500 subjects. The dataset provides rich images and annotations including two modalities (OCT/OCTA volumes), six types of projections, four types of text labels (age/gender/eye/disease) and seven types of segmentation labels (large vessel/capillary/artery/vein/2D FAZ/3D FAZ/retinal layers). Then, we propose a multi-object segmentation task called CAVF, which integrates capillary segmentation, artery segmentation, vein segmentation, and FAZ segmentation under a unified framework. In addition, we optimize the 3D-to-2D image projection network (IPN) to IPN-V2 to serve as one of the segmentation baselines. Experimental results demonstrate that IPN-V2 achieves an about 10% mIoU improvement over IPN on CAVF task. Finally, we further study the impact of several dataset characteristics: the training set size, the model input (OCT/OCTA, 3D volume/2D projection), the baseline networks, and the diseases. The dataset and code are publicly available at: https://ieee-dataport.org/open-access/octa-500.

Explainable artificial intelligence for the automated assessment of the retinal vascular tortuosity

Retinal vascular tortuosity is an excessive bending and twisting of the blood vessels in the retina that is associated with numerous health conditions. We propose a novel methodology for the automated assessment of the retinal vascular tortuosity from color fundus images. Our methodology takes into consideration several anatomical factors to weigh the importance of each individual blood vessel. First, we use deep neural networks to produce a robust extraction of the different anatomical structures. Then, the weighting coefficients that are required for the integration of the different anatomical factors are adjusted using evolutionary computation. Finally, the proposed methodology also provides visual representations that explain the contribution of each individual blood vessel to the predicted tortuosity, hence allowing us to understand the decisions of the model. We validate our proposal in a dataset of color fundus images providing a consensus ground truth as well as the annotations of five clinical experts. Our proposal outperforms previous automated methods and offers a performance that is comparable to that of the clinical experts. Therefore, our methodology demonstrates to be a viable alternative for the assessment of the retinal vascular tortuosity. This could facilitate the use of this biomarker in clinical practice and medical research.

One-shot Retinal Artery and Vein Segmentation via Cross-modality Pretraining

Purpose

To perform one-shot retinal artery and vein segmentation with cross-modality artery-vein (AV) soft-label pretraining.

Design

Cross-sectional study.

Subjects

The study included 6479 color fundus photography (CFP) and arterial-venous fundus fluorescein angiography (FFA) pairs from 1964 participants for pretraining and 6 AV segmentation data sets with various image sources, including RITE, HRF, LES-AV, AV-WIDE, PortableAV, and DRSplusAV for one-shot finetuning and testing.

Methods

We structurally matched the arterial and venous phase of FFA with CFP, the AV soft labels were automatically generated by utilizing the fluorescein intensity difference of the arterial and venous-phase FFA images, and the soft labels were then used to train a generative adversarial network to learn to generate AV soft segmentations using CFP images as input. We then finetuned the pretrained model to perform AV segmentation using only one image from each of the AV segmentation data sets and test on the remainder. To investigate the effect and reliability of one-shot finetuning, we conducted experiments without finetuning and by finetuning the pretrained model on an iteratively different single image for each data set under the same experimental setting and tested the models on the remaining images.

Main Outcome Measures

The AV segmentation was assessed by area under the receiver operating characteristic curve (AUC), accuracy, Dice score, sensitivity, and specificity.

Results

After the FFA-AV soft label pretraining, our method required only one exemplar image from each camera or modality and achieved similar performance with full-data training, with AUC ranging from 0.901 to 0.971, accuracy from 0.959 to 0.980, Dice score from 0.585 to 0.773, sensitivity from 0.574 to 0.763, and specificity from 0.981 to 0.991. Compared with no finetuning, the segmentation performance improved after one-shot finetuning. When finetuned on different images in each data set, the standard deviation of the segmentation results across models ranged from 0.001 to 0.10.

Conclusions

This study presents the first one-shot approach to retinal artery and vein segmentation. The proposed labeling method is time-saving and efficient, demonstrating a promising direction for retinal-vessel segmentation and enabling the potential for widespread application.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Leuven-Haifa High-Resolution Fundus Image Dataset for Retinal Blood Vessel Segmentation and Glaucoma Diagnosis

The Leuven-Haifa dataset contains 240 disc-centered fundus images of 224 unique patients (75 patients with normal tension glaucoma, 63 patients with high tension glaucoma, 30 patients with other eye diseases and 56 healthy controls) from the University Hospitals of Leuven. The arterioles and venules of these images were both annotated by master students in medicine and corrected by a senior annotator. All senior segmentation corrections are provided as well as the junior segmentations of the test set. An open-source toolbox for the parametrization of segmentations was developed. Diagnosis, age, sex, vascular parameters as well as a quality score are provided as metadata. Potential reuse is envisioned as the development or external validation of blood vessels segmentation algorithms or study of the vasculature in glaucoma and the development of glaucoma diagnosis algorithms. The dataset is available on the KU Leuven Research Data Repository (RDR).

EAMR-Net: A multiscale effective spatial and cross-channel attention network for retinal vessel segmentation

Delineation of retinal vessels in fundus images is essential for detecting a range of eye disorders. An automated technique for vessel segmentation can assist clinicians and enhance the efficiency of the diagnostic process. Traditional methods fail to extract multiscale information, discard unnecessary information, and delineate thin vessels. In this paper, a novel residual U-Net architecture that incorporates multi-scale feature learning and effective attention is proposed to delineate the retinal vessels precisely. Since drop block regularization performs better than drop out in preventing overfitting, drop block was used in this study. A multi-scale feature learning module was added instead of a skip connection to learn multi-scale features. A novel effective attention block was proposed and integrated with the decoder block to obtain precise spatial and channel information. Experimental findings indicated that the proposed model exhibited outstanding performance in retinal vessel delineation. The sensitivities achieved for DRIVE, STARE, and CHASE_DB datasets were 0.8293, 0.8151 and 0.8084, respectively.

Retinal vessel segmentation method based on RSP-SA Unet network

Segmenting retinal vessels plays a significant role in the diagnosis of fundus disorders. However, there are two problems in the retinal vessel segmentation methods. First, fine-grained features of fine blood vessels are difficult to be extracted. Second, it is easy to lose track of the details of blood vessel edges. To solve the problems above, the Residual SimAM Pyramid-Spatial Attention Unet (RSP-SA Unet) is proposed, in which the encoding, decoding, and upsampling layers of the Unet are mainly improved. Firstly, the RSP structure proposed in this paper approximates a residual structure combined with SimAM and Pyramid Segmentation Attention (PSA), which is applied to the encoding and decoding parts to extract multi-scale spatial information and important features across dimensions at a finer level. Secondly, the spatial attention (SA) is used in the upsampling layer to perform multi-attention mapping on the input feature map, which could enhance the segmentation effect of small blood vessels with low contrast. Finally, the RSP-SA Unet is verified on the CHASE_DB1, DRIVE, and STARE datasets, and the segmentation accuracy (ACC) of the RSP-SA Unet could reach 0.9763, 0.9704, and 0.9724, respectively. Area under the ROC curve (AUC) could reach 0.9896, 0.9858, and 0.9906, respectively. The RSP-SA Unet overall performance is better than the comparison methods.

A machine learning approach to differentiate wide QRS tachycardia: distinguishing ventricular tachycardia from supraventricular tachycardia

BACKGROUND

Differential diagnosis of wide QRS tachycardia (WQCT) has been a challenging issue. Published algorithms to distinguish ventricular tachycardia (VT) and supraventricular tachycardia (SVT) have limited diagnostic capabilities.

METHODS

A total of 278 patients with WQCT from January 2010 to March 2022 were enrolled. The electrophysiological study confirmed SVT in 154 patients and VT in 65 ones. Two hundred nineteen WQCT 12-lead ECGs were randomly divided into development cohort (n = 165) and testing cohort (n = 54) data sets. The development cohort was split into a training group (n = 115) and an internal validation group (n = 50). Forty ECG features extracted from the 219 WQCT ECGs are fed into 9 iteratively trained ML algorithms. This novel ML algorithm was also compared with four published algorithms.

RESULTS

In the development cohort, the Gradient Boosting Machine (GBM) model displayed the maximum area under curve (AUC) (0.91, 95% confidence interval (CI) 0.81-1.00). In the testing cohort, the GBM model had a higher AUC of 0.97 compared to 4 validated ECG algorithms, namely, Brugada (0.68), avR (0.62), RWPTII (0.72), and LLA algorithms (0.70). Accuracy, sensitivity, specificity, negative predictive value, and positive predictive value of the GBM model were 0.94, 0.97, 0.90, 0.94, and 0.95, respectively.

CONCLUSIONS

A GBM ML model contributes to distinguishing SVT from VT based on surface ECG features. In addition, we were able to identify important indicators for distinguishing WQCT.

MAG-Net : Multi-fusion network with grouped attention for retinal vessel segmentation

Retinal vessel segmentation plays a vital role in the clinical diagnosis of ophthalmic diseases. Despite convolutional neural networks (CNNs) excelling in this task, challenges persist, such as restricted receptive fields and information loss from downsampling. To address these issues, we propose a new multi-fusion network with grouped attention (MAG-Net). First, we introduce a hybrid convolutional fusion module instead of the original encoding block to learn more feature information by expanding the receptive field. Additionally, the grouped attention enhancement module uses high-level features to guide low-level features and facilitates detailed information transmission through skip connections. Finally, the multi-scale feature fusion module aggregates features at different scales, effectively reducing information loss during decoder upsampling. To evaluate the performance of the MAG-Net, we conducted experiments on three widely used retinal datasets: DRIVE, CHASE and STARE. The results demonstrate remarkable segmentation accuracy, specificity and Dice coefficients. Specifically, the MAG-Net achieved segmentation accuracy values of 0.9708, 0.9773 and 0.9743, specificity values of 0.9836, 0.9875 and 0.9906 and Dice coefficients of 0.8576, 0.8069 and 0.8228, respectively. The experimental results demonstrate that our method outperforms existing segmentation methods exhibiting superior performance and segmentation outcomes.

Cell classification with worse-case boosting for intelligent cervical cancer screening

Cell classification underpins intelligent cervical cancer screening, a cytology examination that effectively decreases both the morbidity and mortality of cervical cancer. This task, however, is rather challenging, mainly due to the difficulty of collecting a training dataset representative sufficiently of the unseen test data, as there are wide variations of cells' appearance and shape at different cancerous statuses. This difficulty makes the classifier, though trained properly, often classify wrongly for cells that are underrepresented by the training dataset, eventually leading to a wrong screening result. To address it, we propose a new learning algorithm, called worse-case boosting, for classifiers effectively learning from under-representative datasets in cervical cell classification. The key idea is to learn more from worse-case data for which the classifier has a larger gradient norm compared to other training data, so these data are more likely to correspond to underrepresented data, by dynamically assigning them more training iterations and larger loss weights for boosting the generalizability of the classifier on underrepresented data. We achieve this idea by sampling worse-case data per the gradient norm information and then enhancing their loss values to update the classifier. We demonstrate the effectiveness of this new learning algorithm on two publicly available cervical cell classification datasets (the two largest ones to the best of our knowledge), and positive results (4% accuracy improvement) yield in the extensive experiments. The source codes are available at: https://github.com/YouyiSong/Worse-Case-Boosting.

Selection of HBV key reactivation factors based on maximum information coefficient combined with cosine similarity

BACKGROUND

Hepatitis B Virus (HBV) reactivation is the most common complication for patients with primary liver cancer (PLC) after radiotherapy. How to reduce the reactivation of HBV has been a hot topic in the study of postoperative radiotherapy for liver cancer.

OBJECTIVE

To find out the inducement of HBV reactivation, a feature selection algorithm (MIC-CS) using maximum information coefficient (MIC) combined with cosine similarity (CS) was proposed to screen the risk factors that may affect HBV reactivation.

METHOD

Firstly, different factors were coded and MIC between patients was calculated to acquire the association between different factors and HBV reactivation. Secondly, a cosine similarity algorithm was constructed to calculate the similarity relationship between different factors, thus removing redundant information. Finally, combined with the weight of the two, the potential risk factors were sorted and the key factors leading to HBV reactivation were selected.

RESULTS

The results indicated that HBV baseline, external boundary, TNM, KPS score, VD, AFP, and Child-Pugh could lead to HBV reactivation after radiotherapy. The classification model was constructed for the above factors, with the highest classification accuracy of 84% and the AUC value of 0.71.

CONCLUSION

Comparing multiple feature selection methods, the results showed that the effect of the MIC-CS was significantly better than MIM, CMIM, and mRMR, so it has a very broad application prospect.

Micro-segmentation of retinal image lesions in diabetic retinopathy using energy-based fuzzy C-Means clustering (EFM-FCM)

Diabetic retinopathy (DR) is a prevalent cause of global visual impairment, contributing to approximately 4.8% of blindness cases worldwide as reported by the World Health Organization (WHO). The condition is characterized by pathological abnormalities in the retinal layer, including microaneurysms, vitreous hemorrhages, and exudates. Microscopic analysis of retinal images is crucial in diagnosing and treating DR. This article proposes a novel method for early DR screening using segmentation and unsupervised learning techniques. The approach integrates a neural network energy-based model into the Fuzzy C-Means (FCM) algorithm to enhance convergence criteria, aiming to improve the accuracy and efficiency of automated DR screening tools. The evaluation of results includes the primary dataset from the Shiva Netralaya Centre, IDRiD, and DIARETDB1. The performance of the proposed method is compared against FCM, EFCM, FLICM, and M-FLICM techniques, utilizing metrics such as accuracy in noiseless and noisy conditions and average execution time. The results showcase auspicious performance on both primary and secondary datasets, achieving accuracy rates of 99.03% in noiseless conditions and 93.13% in noisy images, with an average execution time of 16.1 s. The proposed method holds significant potential in medical image analysis and could pave the way for future advancements in automated DR diagnosis and management. RESEARCH HIGHLIGHTS: A novel approach is proposed in the article, integrating a neural network energy-based model into the FCM algorithm to enhance the convergence criteria and the accuracy of automated DR screening tools. By leveraging the microscopic characteristics of retinal images, the proposed method significantly improves the accuracy of lesion segmentation, facilitating early detection and monitoring of DR. The evaluation of the method's performance includes primary datasets from reputable sources such as the Shiva Netralaya Centre, IDRiD, and DIARETDB1, demonstrating its effectiveness in comparison to other techniques (FCM, EFCM, FLICM, and M-FLICM) in terms of accuracy in both noiseless and noisy conditions. It achieves impressive accuracy rates of 99.03% in noiseless conditions and 93.13% in noisy images, with an average execution time of 16.1 s.

Semi-supervised point consistency network for retinal artery/vein classification

Recent deep learning methods with convolutional neural networks (CNNs) have boosted advance prosperity of medical image analysis and expedited the automatic retinal artery/vein (A/V) classification. However, it is challenging for these CNN-based approaches in two aspects: (1) specific tubular structures and subtle variations in appearance, contrast, and geometry, which tend to be ignored in CNNs with network layer increasing; (2) limited well-labeled data for supervised segmentation of retinal vessels, which may hinder the effectiveness of deep learning methods. To address these issues, we propose a novel semi-supervised point consistency network (SPC-Net) for retinal A/V classification. SPC-Net consists of an A/V classification (AVC) module and a multi-class point consistency (MPC) module. The AVC module adopts an encoder-decoder segmentation network to generate the prediction probability map of A/V for supervised learning. The MPC module introduces point set representations to adaptively generate point set classification maps of the arteriovenous skeleton, which enjoys its prediction flexibility and consistency (i.e. point consistency) to effectively alleviate arteriovenous confusion. In addition, we propose a consistency regularization between the predicted A/V classification probability maps and point set representations maps for unlabeled data to explore the inherent segmentation perturbation of the point consistency, reducing the need for annotated data. We validate our method on two typical public datasets (DRIVE, HRF) and a private dataset (TR280) with different resolutions. Extensive qualitative and quantitative experimental results demonstrate the effectiveness of our proposed method for supervised and semi-supervised learning.

A Review on Retinal Blood Vessel Enhancement and Segmentation Techniques for Color Fundus Photography

The retinal image is a trusted modality in biomedical image-based diagnosis of many ophthalmologic and cardiovascular diseases. Periodic examination of the retina can help in spotting these abnormalities in the early stage. However, to deal with today's large population, computerized retinal image analysis is preferred over manual inspection. The precise extraction of the retinal vessel is the first and decisive step for clinical applications. Every year, many more articles are added to the literature that describe new algorithms for the problem at hand. The majority of the review article is restricted to a fairly small number of approaches, assessment indices, and databases. In this context, a comprehensive review of different vessel extraction methods is inevitable. It includes the development of a first-hand classification of these methods. A bibliometric analysis of these articles is also presented. The benefits and drawbacks of the most commonly used techniques are summarized. The primary challenges, as well as the scope of possible changes, are discussed. In order to make a fair comparison, numerous assessment indices are considered. The findings of this survey could provide a new path for researchers for further work in this domain.

Retinal microvascular complexity as a putative biomarker of biological age: a pilot study

Physiological changes associated with aging increase the risk for the development of age-related diseases. This increase is non-specific to the type of age-related disease, although each disease develops through a unique pathophysiologic mechanism. People who age at a faster rate develop age-related diseases earlier in their life. They have an older "biological age" compared to their "chronological age". Early detection of individuals with accelerated aging would allow timely intervention to postpone the onset of age-related diseases. This would increase their life expectancy and their length of good quality life. The goal of this study was to investigate whether retinal microvascular complexity could be used as a biomarker of biological age. Retinal images of 68 participants ages ranging from 19 to 82 years were collected in an observational cross-sectional study. Twenty of the old participants had age-related diseases such as hypertension, type 2 diabetes, and/or Alzheimer's dementia. The rest of the participants were healthy. Retinal images were captured by a hand-held, non-mydriatic fundus camera and quantification of the microvascular complexity was performed by using Sholl's, box-counting fractal, and lacunarity analysis. In the healthy subjects, increasing chronological age was associated with lower retinal microvascular complexity measured by Sholl's analysis. Decreased box-counting fractal dimension was present in old patients, and this decrease was 2.1 times faster in participants who had age-related diseases (p = 0.047). Retinal microvascular complexity could be a promising new biomarker of biological age. The data from this study is the first of this kind collected in Montenegro. It is freely available for use.

SegR-Net: A deep learning framework with multi-scale feature fusion for robust retinal vessel segmentation

Retinal vessel segmentation is an important task in medical image analysis and has a variety of applications in the diagnosis and treatment of retinal diseases. In this paper, we propose SegR-Net, a deep learning framework for robust retinal vessel segmentation. SegR-Net utilizes a combination of feature extraction and embedding, deep feature magnification, feature precision and interference, and dense multiscale feature fusion to generate accurate segmentation masks. The model consists of an encoder module that extracts high-level features from the input images and a decoder module that reconstructs the segmentation masks by combining features from the encoder module. The encoder module consists of a feature extraction and embedding block that enhances by dense multiscale feature fusion, followed by a deep feature magnification block that magnifies the retinal vessels. To further improve the quality of the extracted features, we use a group of two convolutional layers after each DFM block. In the decoder module, we utilize a feature precision and interference block and a dense multiscale feature fusion block (DMFF) to combine features from the encoder module and reconstruct the segmentation mask. We also incorporate data augmentation and pre-processing techniques to improve the generalization of the trained model. Experimental results on three fundus image publicly available datasets (CHASE_DB1, STARE, and DRIVE) demonstrate that SegR-Net outperforms state-of-the-art models in terms of accuracy, sensitivity, specificity, and F1 score. The proposed framework can provide more accurate and more efficient segmentation of retinal blood vessels in comparison to the state-of-the-art techniques, which is essential for clinical decision-making and diagnosis of various eye diseases.

Automated analysis of vessel morphometry in retinal images from a Danish high street optician setting

PURPOSE

To evaluate the test performance of the QUARTZ (QUantitative Analysis of Retinal vessel Topology and siZe) software in detecting retinal features from retinal images captured by health care professionals in a Danish high street optician chain, compared with test performance from other large population studies (i.e., UK Biobank) where retinal images were captured by non-experts.

METHOD

The dataset FOREVERP (Finding Ophthalmic Risk and Evaluating the Value of Eye exams and their predictive Reliability, Pilot) contains retinal images obtained from a Danish high street optician chain. The QUARTZ algorithm utilizes both image processing and machine learning methods to determine retinal image quality, vessel segmentation, vessel width, vessel classification (arterioles or venules), and optic disc localization. Outcomes were evaluated by metrics including sensitivity, specificity, and accuracy and compared to human expert ground truths.

RESULTS

QUARTZ's performance was evaluated on a subset of 3,682 images from the FOREVERP database. 80.55% of the FOREVERP images were labelled as being of adequate quality compared to 71.53% of UK Biobank images, with a vessel segmentation sensitivity of 74.64% and specificity of 98.41% (FOREVERP) compared with a sensitivity of 69.12% and specificity of 98.88% (UK Biobank). The mean (± standard deviation) vessel width of the ground truth was 16.21 (4.73) pixels compared to that predicted by QUARTZ of 17.01 (4.49) pixels, resulting in a difference of -0.8 (1.96) pixels. The differences were stable across a range of vessels. The detection rate for optic disc localisation was similar for the two datasets.

CONCLUSION

QUARTZ showed high performance when evaluated on the FOREVERP dataset, and demonstrated robustness across datasets, providing validity to direct comparisons and pooling of retinal feature measures across data sources.

Enhancing fine retinal vessel segmentation: Morphological reconstruction and double thresholds filtering strategy

Eye diseases such as diabetic retinopathy are progressive with various changes in the retinal vessels, and it is difficult to analyze the disease for future treatment. There are many computerized algorithms implemented for retinal vessel segmentation, but the tiny vessels drop off, impacting the performance of the overall algorithms. This research work contains the new image processing techniques such as enhancement filters, coherence filters and binary thresholding techniques to handle the different color retinal fundus image problems to achieve a vessel image that is well-segmented, and the proposed algorithm has improved performance over existing work. Our developed technique incorporates morphological techniques to address the center light reflex issue. Additionally, to effectively resolve the problem of insufficient and varying contrast, our developed technique employs homomorphic methods and Wiener filtering. Coherent filters are used to address the coherence issue of the retina vessels, and then a double thresholding technique is applied with image reconstruction to achieve a correctly segmented vessel image. The results of our developed technique were evaluated using the STARE and DRIVE datasets and it achieves an accuracy of about 0.96 and a sensitivity of 0.81. The performance obtained from our proposed method proved the capability of the method which can be used by ophthalmology experts to diagnose ocular abnormalities and recommended for further treatment.

Retinal vessel caliber and tortuosity and prediction of 5-year incidence of hypertension

PURPOSE

With arterial hypertension as a global risk factor for cerebrovascular and cardiovascular diseases, we examined whether retinal blood vessel caliber and tortuosity assessed by a vessel-constraint network model can predict the incidence of hypertension.

METHODS

The community-based prospective study included 9230 individuals who were followed for 5 years. Ocular fundus photographs taken at baseline were analyzed by a vessel-constraint network model.

RESULTS

Within the 5-year follow-up, 1279 (18.8%) and 474 (7.0%) participants out of 6813 individuals free of hypertension at baseline developed hypertension and severe hypertension, respectively. In multivariable analysis, a higher incidence of hypertension was related to a narrower retinal arteriolar diameter ( P  < 0.001), wider venular diameter ( P  = 0.005), and a smaller arteriole-to-venule diameter ratio ( P  < 0.001) at baseline. Individuals with the 5% narrowest arteriole or the 5% widest venule diameter had a 17.1-fold [95% confidence interval (CI):7.9, 37.2] or 2.3-fold (95% CI: 1.4, 3.7) increased risk for developing hypertension, as compared with those with the 5% widest arteriole or the 5% narrowest venule. The area under the receiver operator characteristic curve for predicting the 5-year incidence of hypertension and severe hypertension was 0.791 (95% CI: 0.778, 0.804) and 0.839 (95% CI: 0.821, 0.856), respectively. Although the venular tortuosity was positively associated with the presence of hypertension at baseline ( P  = 0.01), neither arteriolar tortuosity nor venular tortuosity was associated with incident hypertension (both P  ≥ 0.10).

CONCLUSION AND RELEVANCE

Narrower retinal arterioles and wider venules indicate an increased risk for incident hypertension within 5 years, while tortuous retinal venules are associated with the presence rather than the incidence of hypertension. The automatic assessment of retinal vessel features performed well in identifying individuals at risk of developing hypertension.

Segmentation and Classification Approaches of Clinically Relevant Curvilinear Structures: A Review

Detection of curvilinear structures from microscopic images, which help the clinicians to make an unambiguous diagnosis is assuming paramount importance in recent clinical practice. Appearance and size of dermatophytic hyphae, keratitic fungi, corneal and retinal vessels vary widely making their automated detection cumbersome. Automated deep learning methods, endowed with superior self-learning capacity, have superseded the traditional machine learning methods, especially in complex images with challenging background. Automatic feature learning ability using large input data with better generalization and recognition capability, but devoid of human interference and excessive pre-processing, is highly beneficial in the above context. Varied attempts have been made by researchers to overcome challenges such as thin vessels, bifurcations and obstructive lesions in retinal vessel detection as revealed through several publications reviewed here. Revelations of diabetic neuropathic complications such as tortuosity, changes in the density and angles of the corneal fibers have been successfully sorted in many publications reviewed here. Since artifacts complicate the images and affect the quality of analysis, methods addressing these challenges have been described. Traditional and deep learning methods, that have been adapted and published between 2015 and 2021 covering retinal vessels, corneal nerves and filamentous fungi have been summarized in this review. We find several novel and meritorious ideas and techniques being put to use in the case of retinal vessel segmentation and classification, which by way of cross-domain adaptation can be utilized in the case of corneal and filamentous fungi also, making suitable adaptations to the challenges to be addressed.

Automation of flow analysis in scleral vessels based on descriptive-associative algorithms

Blood flow reflects the eye's health and is disrupted in many diseases. Many pathological processes take place at the cellular level like as microcirculation of blood in vessels, and the processing of medical images is a difficult recognition task. Existing techniques for measuring blood flow are limited due to the complex assumptions, equipment and calculations requirements. In this paper, we propose a method for determining the blood flow characteristics in eye conjunctiva vessels, such as linear and volumetric blood speed and topological characteristics of the vascular net. The method preprocesses the video to improve the conditions of analysis and then builds an integral optical flow for definition of flow dynamical characteristic of eye vessels. These characteristics make it possible to determine changes in blood flow in eye vessels. We show the efficiency of our method in natural eye vessel scenes. The research provides valuable insights to novices with limited experience in the diagnosis and can serve as a valuable tool for experienced medical professionals.

End-to-End Automatic Classification of Retinal Vessel Based on Generative Adversarial Networks with Improved U-Net

The retinal vessels in the human body are the only ones that can be observed directly by non-invasive imaging techniques. Retinal vessel morphology and structure are the important objects of concern for physicians in the early diagnosis and treatment of related diseases. The classification of retinal vessels has important guiding significance in the basic stage of diagnostic treatment. This paper proposes a novel method based on generative adversarial networks with improved U-Net, which can achieve synchronous automatic segmentation and classification of blood vessels by an end-to-end network. The proposed method avoids the dependency of the segmentation results in the multiple classification tasks. Moreover, the proposed method builds on an accurate classification of arteries and veins while also classifying arteriovenous crossings. The validity of the proposed method is evaluated on the RITE dataset: the accuracy of image comprehensive classification reaches 96.87%. The sensitivity and specificity of arteriovenous classification reach 91.78% and 97.25%. The results verify the effectiveness of the proposed method and show the competitive classification performance.

Separable Paravector Orientation Tensors for Enhancing Retinal Vessels

Robust detection of retinal vessels remains an unsolved research problem, particularly in handling the intrinsic real-world challenges of highly imbalanced contrast between thick vessels and thin ones, inhomogeneous background regions, uneven illumination, and complex geometries of crossing/bifurcations. This paper presents a new separable paravector orientation tensor that addresses these difficulties by characterizing the enhancement of retinal vessels to be dependent on a nonlinear scale representation, invariant to changes in contrast and lighting, responsive for symmetric patterns, and fitted with elliptical cross-sections. The proposed method is built on projecting vessels as a 3D paravector valued function rotated in an alpha quarter domain, providing geometrical, structural, symmetric, and energetic features. We introduce an innovative symmetrical inhibitory scheme that incorporates paravector features for producing a set of directional contrast-independent elongated-like patterns reconstructing vessel tree in orientation tensors. By fitting constraint elliptical volumes via eigensystem analysis, the final vessel tree is produced with a strong and uniform response preserving various vessel features. The validation of proposed method on clinically relevant retinal images with high-quality results, shows its excellent performance compared to the state-of-the-art benchmarks and the second human observers.

AA-WGAN: Attention augmented Wasserstein generative adversarial network with application to fundus retinal vessel segmentation

In this paper, a novel attention augmented Wasserstein generative adversarial network (AA-WGAN) is proposed for fundus retinal vessel segmentation, where a U-shaped network with attention augmented convolution and squeeze-excitation module is designed to serve as the generator. In particular, the complex vascular structures make some tiny vessels hard to segment, while the proposed AA-WGAN can effectively handle such imperfect data property, which is competent in capturing the dependency among pixels in the whole image to highlight the regions of interests via the applied attention augmented convolution. By applying the squeeze-excitation module, the generator is able to pay attention to the important channels of the feature maps, and the useless information can be suppressed as well. In addition, gradient penalty method is adopted in the WGAN backbone to alleviate the phenomenon of generating large amounts of repeated images due to excessive concentration on accuracy. The proposed model is comprehensively evaluated on three datasets DRIVE, STARE, and CHASE_DB1, and the results show that the proposed AA-WGAN is a competitive vessel segmentation model as compared with several other advanced models, which obtains the accuracy of 96.51%, 97.19% and 96.94% on each dataset, respectively. The effectiveness of the applied important components is validated by ablation study, which also endows the proposed AA-WGAN with considerable generalization ability.

Retinal Blood-Vessel Extraction Using Weighted Kernel Fuzzy C-Means Clustering and Dilation-Based Functions

Automated blood-vessel extraction is essential in diagnosing Diabetic Retinopathy (DR) and other eye-related diseases. However, the traditional methods for extracting blood vessels tend to provide low accuracy when dealing with difficult situations, such as extracting both micro and large blood vessels simultaneously with low-intensity images and blood vessels with DR. This paper proposes a complete preprocessing method to enhance original retinal images before transferring the enhanced images to a novel blood-vessel extraction method by a combined three extraction stages. The first stage focuses on the fast extraction of retinal blood vessels using Weighted Kernel Fuzzy C-Means (WKFCM) Clustering to draw the vessel feature from the retinal background. The second stage focuses on the accuracy of full-size images to achieve regional vessel feature recognition of large and micro blood vessels and to minimize false extraction. This stage implements the mathematical dilation operator from a trained model called Dilation-Based Function (DBF). Finally, an optimal parameter threshold is empirically determined in the third stage to remove non-vessel features in the binary image and improve the overall vessel extraction results. According to evaluations of the method via the datasets DRIVE, STARE, and DiaretDB0, the proposed WKFCM-DBF method achieved sensitivities, specificities, and accuracy performances of 98.12%, 98.20%, and 98.16%, 98.42%, 98.80%, and 98.51%, and 98.89%, 98.10%, and 98.09%, respectively.

A multi-task convolutional neural network for classification and segmentation of chronic venous disorders

Chronic Venous Disorders (CVD) of the lower limbs are one of the most prevalent medical conditions, affecting 35% of adults in Europe and North America. Due to the exponential growth of the aging population and the worsening of CVD with age, it is expected that the healthcare costs and the resources needed for the treatment of CVD will increase in the coming years. The early diagnosis of CVD is fundamental in treatment planning, while the monitoring of its treatment is fundamental to assess a patient's condition and quantify the evolution of CVD. However, correct diagnosis relies on a qualitative approach through visual recognition of the various venous disorders, being time-consuming and highly dependent on the physician's expertise. In this paper, we propose a novel automatic strategy for the joint segmentation and classification of CVDs. The strategy relies on a multi-task deep learning network, denominated VENet, that simultaneously solves segmentation and classification tasks, exploiting the information of both tasks to increase learning efficiency, ultimately improving their performance. The proposed method was compared against state-of-the-art strategies in a dataset of 1376 CVD images. Experiments showed that the VENet achieved a classification performance of 96.4%, 96.4%, and 97.2% for accuracy, precision, and recall, respectively, and a segmentation performance of 75.4%, 76.7.0%, 76.7% for the Dice coefficient, precision, and recall, respectively. The joint formulation increased the robustness of both tasks when compared to the conventional classification or segmentation strategies, proving its added value, mainly for the segmentation of small lesions.

Identification of ocular refraction based on deep learning algorithm as a novel retinoscopy method

BACKGROUND

The evaluation of refraction is indispensable in ophthalmic clinics, generally requiring a refractor or retinoscopy under cycloplegia. Retinal fundus photographs (RFPs) supply a wealth of information related to the human eye and might provide a promising approach that is more convenient and objective. Here, we aimed to develop and validate a fusion model-based deep learning system (FMDLS) to identify ocular refraction via RFPs and compare with the cycloplegic refraction. In this population-based comparative study, we retrospectively collected 11,973 RFPs from May 1, 2020 to November 20, 2021. The performance of the regression models for sphere and cylinder was evaluated using mean absolute error (MAE). The accuracy, sensitivity, specificity, area under the receiver operating characteristic curve, and F1-score were used to evaluate the classification model of the cylinder axis.

RESULTS

Overall, 7873 RFPs were retained for analysis. For sphere and cylinder, the MAE values between the FMDLS and cycloplegic refraction were 0.50 D and 0.31 D, representing an increase of 29.41% and 26.67%, respectively, when compared with the single models. The correlation coefficients (r) were 0.949 and 0.807, respectively. For axis analysis, the accuracy, specificity, sensitivity, and area under the curve value of the classification model were 0.89, 0.941, 0.882, and 0.814, respectively, and the F1-score was 0.88.

CONCLUSIONS

The FMDLS successfully identified the ocular refraction in sphere, cylinder, and axis, and showed good agreement with the cycloplegic refraction. The RFPs can provide not only comprehensive fundus information but also the refractive state of the eye, highlighting their potential clinical value.

Human treelike tubular structure segmentation: A comprehensive review and future perspectives

Various structures in human physiology follow a treelike morphology, which often expresses complexity at very fine scales. Examples of such structures are intrathoracic airways, retinal blood vessels, and hepatic blood vessels. Large collections of 2D and 3D images have been made available by medical imaging modalities such as magnetic resonance imaging (MRI), computed tomography (CT), Optical coherence tomography (OCT) and ultrasound in which the spatial arrangement can be observed. Segmentation of these structures in medical imaging is of great importance since the analysis of the structure provides insights into disease diagnosis, treatment planning, and prognosis. Manually labelling extensive data by radiologists is often time-consuming and error-prone. As a result, automated or semi-automated computational models have become a popular research field of medical imaging in the past two decades, and many have been developed to date. In this survey, we aim to provide a comprehensive review of currently publicly available datasets, segmentation algorithms, and evaluation metrics. In addition, current challenges and future research directions are discussed.

Comparison of Retinal Imaging Techniques in Individuals with Pulmonary Artery Hypertension Using Vessel Generation Analysis

(1) Background: Retinal vascular imaging plays an essential role in diagnosing and managing chronic diseases such as diabetic retinopathy, sickle cell retinopathy, and systemic hypertension. Previously, we have shown that individuals with pulmonary arterial hypertension (PAH), a rare disorder, exhibit unique retinal vascular changes as seen using fluorescein angiography (FA) and that these changes correlate with PAH severity. This study aimed to determine if color fundus (CF) imaging could garner identical retinal information as previously seen using FA images in individuals with PAH. (2) Methods: VESGEN, computer software which provides detailed vascular patterns, was used to compare manual segmentations of FA to CF imaging in PAH subjects (n = 9) followed by deep learning (DL) processing of CF imaging to increase the speed of analysis and facilitate a noninvasive clinical translation. (3) Results: When manual segmentation of FA and CF images were compared using VESGEN analysis, both showed identical tortuosity and vessel area density measures. This remained true even when separating images based on arterial trees only. However, this was not observed with microvessels. DL segmentation when compared to manual segmentation of CF images showed similarities in vascular structure as defined by fractal dimension. Similarities were lost for tortuosity and vessel area density when comparing manual CF imaging to DL imaging. (4) Conclusions: Noninvasive imaging such as CF can be used with VESGEN to provide an accurate and safe assessment of retinal vascular changes in individuals with PAH. In addition to providing insight into possible future clinical translational use.

BCR-UNet: Bi-directional ConvLSTM residual U-Net for retinal blood vessel segmentation

Background

High precision segmentation of retinal blood vessels from retinal images is a significant step for doctors to diagnose many diseases such as glaucoma and cardiovascular diseases. However, at the peripheral region of vessels, previous U-Net-based segmentation methods failed to significantly preserve the low-contrast tiny vessels.

Methods

For solving this challenge, we propose a novel network model called Bi-directional ConvLSTM Residual U-Net (BCR-UNet), which takes full advantage of U-Net, Dropblock, Residual convolution and Bi-directional ConvLSTM (BConvLSTM). In this proposed BCR-UNet model, we propose a novel Structured Dropout Residual Block (SDRB) instead of using the original U-Net convolutional block, to construct our network skeleton for improving the robustness of the network. Furthermore, to improve the discriminative ability of the network and preserve more original semantic information of tiny vessels, we adopt BConvLSTM to integrate the feature maps captured from the first residual block and the last up-convolutional layer in a nonlinear manner.

Results and discussion

We conduct experiments on four public retinal blood vessel datasets, and the results show that the proposed BCR-UNet can preserve more tiny blood vessels at the low-contrast peripheral regions, even outperforming previous state-of-the-art methods.

Deep reinforcement learning and its applications in medical imaging and radiation therapy: a survey

Reinforcement learning takes sequential decision-making approaches by learning the policy through trial and error based on interaction with the environment. Combining deep learning and reinforcement learning can empower the agent to learn the interactions and the distribution of rewards from state-action pairs to achieve effective and efficient solutions in more complex and dynamic environments. Deep reinforcement learning (DRL) has demonstrated astonishing performance in surpassing the human-level performance in the game domain and many other simulated environments. This paper introduces the basics of reinforcement learning and reviews various categories of DRL algorithms and DRL models developed for medical image analysis and radiation treatment planning optimization. We will also discuss the current challenges of DRL and approaches proposed to make DRL more generalizable and robust in a real-world environment. DRL algorithms, by fostering the designs of the reward function, agents interactions and environment models, can resolve the challenges from scarce and heterogeneous annotated medical image data, which has been a major obstacle to implementing deep learning models in the clinic. DRL is an active research area with enormous potential to improve deep learning applications in medical imaging and radiation therapy planning.

Machine learning for distinguishing right from left premature ventricular contraction origin using surface electrocardiogram features

BACKGROUND

Precise localization of the site of origin of premature ventricular contractions (PVCs) before ablation can facilitate the planning and execution of the electrophysiological procedure.

OBJECTIVE

The purpose of this study was to develop a predictive model that can be used to differentiate PVCs between the left ventricular outflow tract and right ventricular outflow tract (RVOT) using surface electrocardiogram characteristics.

METHODS

A total of 851 patients undergoing radiofrequency ablation of premature ventricular beats from January 2015 to March 2022 were enrolled. Ninety-two patients were excluded. The other 759 patients were enrolled into the development (n = 605), external validation (n = 104), or prospective cohort (n = 50). The development cohort consisted of the training group (n = 423) and the internal validation group (n = 182). Machine learning algorithms were used to construct predictive models for the origin of PVCs using body surface electrocardiogram features.

RESULTS

In the development cohort, the Random Forest model showed a maximum receiver operating characteristic curve area of 0.96. In the external validation cohort, the Random Forest model surpasses 4 reported algorithms in predicting performance (accuracy 94.23%; sensitivity 97.10%; specificity 88.57%). In the prospective cohort, the Random Forest model showed good performance (accuracy 94.00%; sensitivity 85.71%; specificity 97.22%).

CONCLUSION

Random Forest algorithm has improved the accuracy of distinguishing the origin of PVCs, which surpasses 4 previous standards, and would be used to identify the origin of PVCs before the interventional procedure.

Retinal Vessel Segmentation, a Review of Classic and Deep Methods

Retinal illnesses such as diabetic retinopathy (DR) are the main causes of vision loss. In the early recognition of eye diseases, the segmentation of blood vessels in retina images plays an important role. Different symptoms of ocular diseases can be identified by the geometric features of ocular arteries. However, due to the complex construction of the blood vessels and their different thicknesses, segmenting the retina image is a challenging task. There are a number of algorithms that helped the detection of retinal diseases. This paper presents an overview of papers from 2016 to 2022 that discuss machine learning and deep learning methods for automatic vessel segmentation. The methods are divided into two groups: Deep learning-based, and classic methods. Algorithms, classifiers, pre-processing and specific techniques of each group is described, comprehensively. The performances of recent works are compared based on their achieved accuracy in different datasets in inclusive tables. A survey of most popular datasets like DRIVE, STARE, HRF and CHASE_DB1 is also given in this paper. Finally, a list of findings from this review is presented in the conclusion section.

FIVES: A Fundus Image Dataset for Artificial Intelligence based Vessel Segmentation

Retinal vasculature provides an opportunity for direct observation of vessel morphology, which is linked to multiple clinical conditions. However, objective and quantitative interpretation of the retinal vasculature relies on precise vessel segmentation, which is time consuming and labor intensive. Artificial intelligence (AI) has demonstrated great promise in retinal vessel segmentation. The development and evaluation of AI-based models require large numbers of annotated retinal images. However, the public datasets that are usable for this task are scarce. In this paper, we collected a color fundus image vessel segmentation (FIVES) dataset. The FIVES dataset consists of 800 high-resolution multi-disease color fundus photographs with pixelwise manual annotation. The annotation process was standardized through crowdsourcing among medical experts. The quality of each image was also evaluated. To the best of our knowledge, this is the largest retinal vessel segmentation dataset for which we believe this work will be beneficial to the further development of retinal vessel segmentation.

Fractal dimension of retinal vasculature as an image quality metric for automated fundus image analysis systems

Automated fundus screening is becoming a significant programme of telemedicine in ophthalmology. Instant quality evaluation of uploaded retinal images could decrease unreliable diagnosis. In this work, we propose fractal dimension of retinal vasculature as an easy, effective and explainable indicator of retinal image quality. The pipeline of our approach is as follows: utilize image pre-processing technique to standardize input retinal images from possibly different sources to a uniform style; then, an improved deep learning empowered vessel segmentation model is employed to extract retinal vessels from the pre-processed images; finally, a box counting module is used to measure the fractal dimension of segmented vessel images. A small fractal threshold (could be a value between 1.45 and 1.50) indicates insufficient image quality. Our approach has been validated on 30,644 images from four public database.

The RETA Benchmark for Retinal Vascular Tree Analysis

Topological and geometrical analysis of retinal blood vessels could be a cost-effective way to detect various common diseases. Automated vessel segmentation and vascular tree analysis models require powerful generalization capability in clinical applications. In this work, we constructed a novel benchmark RETA with 81 labelled vessel masks aiming to facilitate retinal vessel analysis. A semi-automated coarse-to-fine workflow was proposed for vessel annotation task. During database construction, we strived to control inter-annotator and intra-annotator variability by means of multi-stage annotation and label disambiguation on self-developed dedicated software. In addition to binary vessel masks, we obtained other types of annotations including artery/vein masks, vascular skeletons, bifurcations, trees and abnormalities. Subjective and objective quality validations of the annotated vessel masks demonstrated significantly improved quality over the existing open datasets. Our annotation software is also made publicly available serving the purpose of pixel-level vessel visualization. Researchers could develop vessel segmentation algorithms and evaluate segmentation performance using RETA. Moreover, it might promote the study of cross-modality tubular structure segmentation and analysis.

MCPANet: Multiscale Cross-Position Attention Network for Retinal Vessel Image Segmentation

Accurate medical imaging segmentation of the retinal fundus vasculature is essential to assist physicians in diagnosis and treatment. In recent years, convolutional neural networks (CNN) are widely used to classify retinal blood vessel pixels for retinal blood vessel segmentation tasks. However, the convolutional block receptive field is limited, simple multiple superpositions tend to cause information loss, and there are limitations in feature extraction as well as vessel segmentation. To address these problems, this paper proposes a new retinal vessel segmentation network based on U-Net, which is called multi-scale cross-position attention network (MCPANet). MCPANet uses multiple scales of input to compensate for image detail information and applies to skip connections between encoding blocks and decoding blocks to ensure information transfer while effectively reducing noise. We propose a cross-position attention module to link the positional relationships between pixels and obtain global contextual information, which enables the model to segment not only the fine capillaries but also clear vessel edges. At the same time, multiple scale pooling operations are used to expand the receptive field and enhance feature extraction. It further reduces pixel classification errors and eases the segmentation difficulty caused by the asymmetry of fundus blood vessel distribution. We trained and validated our proposed model on three publicly available datasets, DRIVE, CHASE, and STARE, which obtained segmentation accuracy of 97.05%, 97.58%, and 97.68%, and Dice of 83.15%, 81.48%, and 85.05%, respectively. The results demonstrate that the proposed method in this paper achieves better results in terms of performance and segmentation results when compared with existing methods.

MFI-Net: Multiscale Feature Interaction Network for Retinal Vessel Segmentation

Segmentation of retinal vessels on fundus images plays a critical role in the diagnosis of micro-vascular and ophthalmological diseases. Although being extensively studied, this task remains challenging due to many factors including the highly variable vessel width and poor vessel-background contrast. In this paper, we propose a multiscale feature interaction network (MFI-Net) for retinal vessel segmentation, which is a U-shaped convolutional neural network equipped with the pyramid squeeze-and-excitation (PSE) module, coarse-to-fine (C2F) module, deep supervision, and feature fusion. We extend the SE operator to multiscale features, resulting in the PSE module, which uses the channel attention learned at multiple scales to enhance multiscale features and enables the network to handle the vessels with variable width. We further design the C2F module to generate and re-process the residual feature maps, aiming to preserve more vessel details during the decoding process. The proposed MFI-Net has been evaluated against several public models on the DRIVE, STARE, CHASE_DB1, and HRF datasets. Our results suggest that both PSE and C2F modules are effective in improving the accuracy of MFI-Net, and also indicate that our model has superior segmentation performance and generalization ability over existing models on four public datasets.

Hyperspectral evaluation of vasculature in induced peritonitis mouse models

Imaging of blood vessel structure in combination with functional information about blood oxygenation can be important in characterizing many different health conditions in which the growth of new vessels contributes to the overall condition. In this paper, we present a method for extracting comprehensive maps of the vasculature from hyperspectral images that include tissue and vascular oxygenation. We also show results from a preclinical study of peritonitis in mice. First, we analyze hyperspectral images using Beer-Lambert exponential attenuation law to obtain maps of hemoglobin species throughout the sample. We then use an automatic segmentation algorithm to extract blood vessels from the hemoglobin map and combine them into a vascular structure-oxygenation map. We apply this methodology to a series of hyperspectral images of the abdominal wall of mice with and without induced peritonitis. Peritonitis is an inflammation of peritoneum that leads, if untreated, to complications such as peritoneal sclerosis and even death. Characteristic inflammatory response can also be accompanied by changes in vasculature, such as neoangiogenesis. We demonstrate a potential application of the proposed segmentation and processing method by introducing an abnormal tissue fraction metric that quantifies the amount of tissue that deviates from the average values of healthy controls. It is shown that the proposed metric successfully discriminates between healthy control subjects and model subjects with induced peritonitis and has a high statistical significance.

Automatic segmentation model of intercondylar fossa based on deep learning: a novel and effective assessment method for the notch volume

Background

Notch volume is associated with anterior cruciate ligament (ACL) injury. Manual tracking of intercondylar notch on MR images is time-consuming and laborious. Deep learning has become a powerful tool for processing medical images. This study aims to develop an MRI segmentation model of intercondylar fossa based on deep learning to automatically measure notch volume, and explore its correlation with ACL injury.

Methods

The MRI data of 363 subjects (311 males and 52 females) with ACL injuries incurred during non-contact sports and 232 subjects (147 males and 85 females) with intact ACL were retrospectively analyzed. Each layer of intercondylar fossa was manually traced by radiologists on axial MR images. Notch volume was then calculated. We constructed an automatic segmentation system based on the architecture of Res-UNet for intercondylar fossa and used dice similarity coefficient (DSC) to compare the performance of segmentation systems by different networks. Unpaired t-test was performed to determine differences in notch volume between ACL-injured and intact groups, and between males and females.

Results

The DSCs of intercondylar fossa based on different networks were all more than 0.90, and Res-UNet showed the best performance. The notch volume was significantly lower in the ACL-injured group than in the control group (6.12 ± 1.34 cm³ vs. 6.95 ± 1.75 cm³, P < 0.001). Females had lower notch volume than males (5.41 ± 1.30 cm³ vs. 6.76 ± 1.51 cm³, P < 0.001). Males and females who had ACL injuries had smaller notch than those with intact ACL (p < 0.001 and p < 0.005). Men had larger notches than women, regardless of the ACL injuries (p < 0.001).

Conclusion

Using a deep neural network to segment intercondylar fossa automatically provides a technical support for the clinical prediction and prevention of ACL injury and re-injury after surgery.

Improving foveal avascular zone segmentation in fluorescein angiograms by leveraging manual vessel labels from public color fundus pictures

In clinical routine, ophthalmologists frequently analyze the shape and size of the foveal avascular zone (FAZ) to detect and monitor retinal diseases. In order to extract those parameters, the contours of the FAZ need to be segmented, which is normally achieved by analyzing the retinal vasculature (RV) around the macula in fluorescein angiograms (FA). Computer-aided segmentation methods based on deep learning (DL) can automate this task. However, current approaches for segmenting the FAZ are often tailored to a specific dataset or require manual initialization. Furthermore, they do not take the variability and challenges of clinical FA into account, which are often of low quality and difficult to analyze. In this paper we propose a DL-based framework to automatically segment the FAZ in challenging FA scans from clinical routine. Our approach mimics the workflow of retinal experts by using additional RV labels as a guidance during training. Hence, our model is able to produce RV segmentations simultaneously. We minimize the annotation work by using a multi-modal approach that leverages already available public datasets of color fundus pictures (CFPs) and their respective manual RV labels. Our experimental evaluation on two datasets with FA from 1) clinical routine and 2) large multicenter clinical trials shows that the addition of weak RV labels as a guidance during training improves the FAZ segmentation significantly with respect to using only manual FAZ annotations.

State-of-the-art retinal vessel segmentation with minimalistic models

The segmentation of retinal vasculature from eye fundus images is a fundamental task in retinal image analysis. Over recent years, increasingly complex approaches based on sophisticated Convolutional Neural Network architectures have been pushing performance on well-established benchmark datasets. In this paper, we take a step back and analyze the real need of such complexity. We first compile and review the performance of 20 different techniques on some popular databases, and we demonstrate that a minimalistic version of a standard U-Net with several orders of magnitude less parameters, carefully trained and rigorously evaluated, closely approximates the performance of current best techniques. We then show that a cascaded extension (W-Net) reaches outstanding performance on several popular datasets, still using orders of magnitude less learnable weights than any previously published work. Furthermore, we provide the most comprehensive cross-dataset performance analysis to date, involving up to 10 different databases. Our analysis demonstrates that the retinal vessel segmentation is far from solved when considering test images that differ substantially from the training data, and that this task represents an ideal scenario for the exploration of domain adaptation techniques. In this context, we experiment with a simple self-labeling strategy that enables moderate enhancement of cross-dataset performance, indicating that there is still much room for improvement in this area. Finally, we test our approach on Artery/Vein and vessel segmentation from OCTA imaging problems, where we again achieve results well-aligned with the state-of-the-art, at a fraction of the model complexity available in recent literature. Code to reproduce the results in this paper is released.

A Few-Shot Learning-Based Retinal Vessel Segmentation Method for Assisting in the Central Serous Chorioretinopathy Laser Surgery

Background

The location of retinal vessels is an important prerequisite for Central Serous Chorioretinopathy (CSC) Laser Surgery, which does not only assist the ophthalmologist in marking the location of the leakage point (LP) on the fundus color image but also avoids the damage of the laser spot to the vessel tissue, as well as the low efficiency of the surgery caused by the absorption of laser energy by retinal vessels. In acquiring an excellent intra- and cross-domain adaptability, the existing deep learning (DL)-based vessel segmentation scheme must be driven by big data, which makes the densely annotated work tedious and costly.

Methods

This paper aims to explore a new vessel segmentation method with a few samples and annotations to alleviate the above problems. Firstly, a key solution is presented to transform the vessel segmentation scene into the few-shot learning task, which lays a foundation for the vessel segmentation task with a few samples and annotations. Then, we improve the existing few-shot learning framework as our baseline model to adapt to the vessel segmentation scenario. Next, the baseline model is upgraded from the following three aspects: (1) A multi-scale class prototype extraction technique is designed to obtain more sufficient vessel features for better utilizing the information from the support images; (2) The multi-scale vessel features of the query images, inferred by the support image class prototype information, are gradually fused to provide more effective guidance for the vessel extraction tasks; and (3) A multi-scale attention module is proposed to promote the consideration of the global information in the upgraded model to assist vessel localization. Concurrently, the integrated framework is further conceived to appropriately alleviate the low performance of a single model in the cross-domain vessel segmentation scene, enabling to boost the domain adaptabilities of both the baseline and the upgraded models.

Results

Extensive experiments showed that the upgraded operation could further improve the performance of vessel segmentation significantly. Compared with the listed methods, both the baseline and the upgraded models achieved competitive results on the three public retinal image datasets (i.e., CHASE_DB, DRIVE, and STARE). In the practical application of private CSC datasets, the integrated scheme partially enhanced the domain adaptabilities of the two proposed models.