A retinal blood vessel segmentation based on improved D-MNet and pulse-coupled neural network

Joint high-resolution feature learning and vessel-shape aware convolutions for efficient vessel segmentation

Clear imagery of retinal vessels is one of the critical shreds of evidence in specific disease diagnosis and evaluation, including sophisticated hierarchical topology and plentiful-and-intensive capillaries. In this work, we propose a new topology- and shape-aware model named Multi-branch Vessel-shaped Convolution Network (MVCN) to adaptively learn high-resolution representations from retinal vessel imagery and thereby capture high-quality topology and shape information thereon. Two steps are involved in our pipeline. The former step is proposed as Multiple High-resolution Ensemble Module (MHEM) to enhance high-resolution characteristics of retinal vessel imagery via fusing scale-invariant hierarchical topology thereof. The latter is a novel vessel-shaped convolution that captures the retinal vessel topology to emerge from unrelated fundus structures. Moreover, our MVCN of separating such topology from the fundus is a dynamical multiple sub-label generation via using epistemic uncertainty, instead of manually separating raw labels to distinguish definitive and uncertain vessels. Compared to other existing methods, our method achieves the most advanced AUC values of 98.31%, 98.80%, 98.83%, and 98.65%, and the most advanced ACC of 95.83%, 96.82%, 97.09%,and 96.66% in DRIVE, CHASE_DB1, STARE, and HRF datasets. We also employ correctness, completeness, and quality metrics to evaluate skeletal similarity. Our method's evaluation metrics have doubled compared to previous methods, thereby demonstrating the effectiveness thereof.

Intraoperative Augmented Reality for Vitreoretinal Surgery Using Edge Computing

Purpose: Augmented reality (AR) may allow vitreoretinal surgeons to leverage microscope-integrated digital imaging systems to analyze and highlight key retinal anatomic features in real time, possibly improving safety and precision during surgery. By employing convolutional neural networks (CNNs) for retina vessel segmentation, a retinal coordinate system can be created that allows pre-operative images of capillary non-perfusion or retinal breaks to be digitally aligned and overlayed upon the surgical field in real time. Such technology may be useful in assuring thorough laser treatment of capillary non-perfusion or in using pre-operative optical coherence tomography (OCT) to guide macular surgery when microscope-integrated OCT (MIOCT) is not available. Methods: This study is a retrospective analysis involving the development and testing of a novel image-registration algorithm for vitreoretinal surgery. Fifteen anonymized cases of pars plana vitrectomy with epiretinal membrane peeling, along with corresponding preoperative fundus photographs and optical coherence tomography (OCT) images, were retrospectively collected from the Mayo Clinic database. We developed a TPU (Tensor-Processing Unit)-accelerated CNN for semantic segmentation of retinal vessels from fundus photographs and subsequent real-time image registration in surgical video streams. An iterative patch-wise cross-correlation (IPCC) algorithm was developed for image registration, with a focus on optimizing processing speeds and maintaining high spatial accuracy. The primary outcomes measured were processing speed in frames per second (FPS) and the spatial accuracy of image registration, quantified by the Dice coefficient between registered and manually aligned images. Results: When deployed on an Edge TPU, the CNN model combined with our image-registration algorithm processed video streams at a rate of 14 FPS, which is superior to processing rates achieved on other standard hardware configurations. The IPCC algorithm efficiently aligned pre-operative and intraoperative images, showing high accuracy in comparison to manual registration. Conclusions: This study demonstrates the feasibility of using TPU-accelerated CNNs for enhanced AR in vitreoretinal surgery.

In-depth analysis of research hotspots and emerging trends in AI for retinal diseases over the past decade

Background

The application of Artificial Intelligence (AI) in diagnosing retinal diseases represents a significant advancement in ophthalmological research, with the potential to reshape future practices in the field. This study explores the extensive applications and emerging research frontiers of AI in retinal diseases.

Objective

This study aims to uncover the developments and predict future directions of AI research in retinal disease over the past decade.

Methods

This study analyzes AI utilization in retinal disease research through articles, using citation data sourced from the Web of Science (WOS) Core Collection database, covering the period from January 1, 2014, to December 31, 2023. A combination of WOS analyzer, CiteSpace 6.2 R4, and VOSviewer 1.6.19 was used for a bibliometric analysis focusing on citation frequency, collaborations, and keyword trends from an expert perspective.

Results

A total of 2,861 articles across 93 countries or regions were cataloged, with notable growth in article numbers since 2017. China leads with 926 articles, constituting 32% of the total. The United States has the highest h-index at 66, while England has the most significant network centrality at 0.24. Notably, the University of London is the leading institution with 99 articles and shares the highest h-index (25) with University College London. The National University of Singapore stands out for its central role with a score of 0.16. Research primarily spans ophthalmology and computer science, with "network," "transfer learning," and "convolutional neural networks" being prominent burst keywords from 2021 to 2023.

Conclusion

China leads globally in article counts, while the United States has a significant research impact. The University of London and University College London have made significant contributions to the literature. Diabetic retinopathy is the retinal disease with the highest volume of research. AI applications have focused on developing algorithms for diagnosing retinal diseases and investigating abnormal physiological features of the eye. Future research should pivot toward more advanced diagnostic systems for ophthalmic diseases.

TLTNet: A novel transscale cascade layered transformer network for enhanced retinal blood vessel segmentation

Extracting global and local feature information is still challenging due to the problems of retinal blood vessel medical images like fuzzy edge features, noise, difficulty in distinguishing between lesion regions and background information, and loss of low-level feature information, which leads to insufficient extraction of feature information. To better solve these problems and fully extract the global and local feature information of the image, we propose a novel transscale cascade layered transformer network for enhanced retinal blood vessel segmentation, which consists of an encoder and a decoder and is connected between the encoder and decoder by a transscale transformer cascade module. Among them, the encoder consists of a local-global transscale transformer module, a multi-head layered transscale adaptive embedding module, and a local context(LCNet) module. The transscale transformer cascade module learns local and global feature information from the first three layers of the encoder, and multi-scale dependent features, fuses the hierarchical feature information from the skip connection block and the channel-token interaction fusion block, respectively, and inputs it to the decoder. The decoder includes a decoding module for the local context network and a transscale position transformer module to input the local and global feature information extracted from the encoder with retained key position information into the decoding module and the position embedding transformer module for recovery and output of the prediction results that are consistent with the input feature information. In addition, we propose an improved cross-entropy loss function based on the difference between the deterministic observation samples and the prediction results with the deviation distance, which is validated on the DRIVE and STARE datasets combined with the proposed network model based on the dual transformer structure in this paper, and the segmentation accuracies are 97.26% and 97.87%, respectively. Compared with other state-of-the-art networks, the results show that the proposed network model has a significant competitive advantage in improving the segmentation performance of retinal blood vessel images.

CS-UNet: Cross-scale U-Net with Semantic-position dependencies for retinal vessel segmentation

Accurate retinal vessel segmentation is the prerequisite for early recognition and treatment of retina-related diseases. However, segmenting retinal vessels is still challenging due to the intricate vessel tree in fundus images, which has a significant number of tiny vessels, low contrast, and lesion interference. For this task, the u-shaped architecture (U-Net) has become the de-facto standard and has achieved considerable success. However, U-Net is a pure convolutional network, which usually shows limitations in global modelling. In this paper, we propose a novel Cross-scale U-Net with Semantic-position Dependencies (CS-UNet) for retinal vessel segmentation. In particular, we first designed a Semantic-position Dependencies Aggregator (SPDA) and incorporate it into each layer of the encoder to better focus on global contextual information by integrating the relationship of semantic and position. To endow the model with the capability of cross-scale interaction, the Cross-scale Relation Refine Module (CSRR) is designed to dynamically select the information associated with the vessels, which helps guide the up-sampling operation. Finally, we have evaluated CS-UNet on three public datasets: DRIVE, CHASE_DB1, and STARE. Compared to most existing state-of-the-art methods, CS-UNet demonstrated better performance.

HRD-Net: High resolution segmentation network with adaptive learning ability of retinal vessel features

Retinal segmentation is a crucial step in the early warning of human health conditions. However, retinal blood vessels possess complex curvature, irregular distribution, and contain multi-scale fine structures, which make the limited receptive field of regular convolution challenging to process their vascular details efficiently. Additionally, the encoder-decoder based network leads to irreversible spatial information loss because of multiple downsampling, resulting in over-segmentation and missed segmentation of the vessels. For this reason, we develop a high-resolution network based on Deformable Convolution v3, called HRD-Net. By constructing a high-resolution representation, the network allows special attention to be paid to the details of tiny blood vessels. The proposed feature enhancement cascade module based on Deformable Convolution v3 can flexibly adapt and capture the ever-changing morphology and intricate connections of retinal blood vessels, ensuring the continuity of vessel segmentation. In the output phase of the network, the proposed global aggregation module integrates full-resolution feature maps while suppressing redundant features, achieving an effective fusion of high-level semantic information and spatial detail information. In addition, we have re-examined the selection criteria for activation and normalization methods, and also refine the network architectures from a spatial domain perspective to release redundant computational loads. Testing on the DRIVE, STARE, and CHASE_DB1 datasets indicates that HRD-Net, with fewer parameters, outperforms existing segmentation methods on several evaluation metrics such as F1, ACC, SE, SP, AUC, and IOU.

FRD-Net: a full-resolution dilated convolution network for retinal vessel segmentation

Accurate and automated retinal vessel segmentation is essential for performing diagnosis and surgical planning of retinal diseases. However, conventional U-shaped networks often suffer from segmentation errors when dealing with fine and low-contrast blood vessels due to the loss of continuous resolution in the encoding stage and the inability to recover the lost information in the decoding stage. To address this issue, this paper introduces an effective full-resolution retinal vessel segmentation network, namely FRD-Net, which consists of two core components: the backbone network and the multi-scale feature fusion module (MFFM). The backbone network achieves horizontal and vertical expansion through the interaction mechanism of multi-resolution dilated convolutions while preserving the complete image resolution. In the backbone network, the effective application of dilated convolutions with varying dilation rates, coupled with the utilization of dilated residual modules for integrating multi-scale feature maps from adjacent stages, facilitates continuous learning of multi-scale features to enhance high-level contextual information. Moreover, MFFM further enhances segmentation by fusing deeper multi-scale features with the original image, facilitating edge detail recovery for accurate vessel segmentation. In tests on multiple classical datasets,compared to state-of-the-art segmentation algorithms, FRD-Net achieves superior performance and generalization with fewer model parameters.

Retinal Blood Vessel Tracking and Diameter Estimation via Gaussian Process With Rider Optimization Algorithm

Retinal blood vessels structure analysis is an important step in the detection of ocular diseases such as diabetic retinopathy and retinopathy of prematurity. Accurate tracking and estimation of retinal blood vessels in terms of their diameter remains a major challenge in retinal structure analysis. In this research, we develop a rider-based Gaussian approach for accurate tracking and diameter estimation of retinal blood vessels. The diameter and curvature of the blood vessel are assumed as the Gaussian processes. The features are determined for training the Gaussian process using Radon transform. The kernel hyperparameter of Gaussian processes is optimized using Rider Optimization Algorithm for evaluating the direction of the vessel. Multiple Gaussian processes are used for detecting the bifurcations and the difference in the prediction direction is quantified. The performance of the proposed Rider-based Gaussian process is evaluated with mean and standard deviation. Our method achieved high performance with the standard deviation of 0.2499 and mean average of 0.0147, which outperformed the state-of-the-art method by 6.32%. Although the proposed model outperformed the state-of-the-art method in normal blood vessels, in future research, one can include tortuous blood vessels of different retinopathy patients, which would be more challenging due to large angle variations. We used Rider-based Gaussian process for tracking blood vessels to obtain the diameter of retinal blood vessels, and the method performed well on the "STrutred Analysis of the REtina (STARE) Database" accessed on Oct. 2020 (https://cecas.clemson.edu/~ahoover/stare/). To the best of our knowledge, this experiment is one of the most recent analysis using this type of algorithm.

GAN-Based Approach for Diabetic Retinopathy Retinal Vasculature Segmentation

Most diabetes patients develop a condition known as diabetic retinopathy after having diabetes for a prolonged period. Due to this ailment, damaged blood vessels may occur behind the retina, which can even progress to a stage of losing vision. Hence, doctors advise diabetes patients to screen their retinas regularly. Examining the fundus for this requires a long time and there are few ophthalmologists available to check the ever-increasing number of diabetes patients. To address this issue, several computer-aided automated systems are being developed with the help of many techniques like deep learning. Extracting the retinal vasculature is a significant step that aids in developing such systems. This paper presents a GAN-based model to perform retinal vasculature segmentation. The model achieves good results on the ARIA, DRIVE, and HRF datasets.

Multi-level spatial-temporal and attentional information deep fusion network for retinal vessel segmentation

Objective.Automatic segmentation of fundus vessels has the potential to enhance the judgment ability of intelligent disease diagnosis systems. Even though various methods have been proposed, it is still a demanding task to accurately segment the fundus vessels. The purpose of our study is to develop a robust and effective method to segment the vessels in human color retinal fundus images.Approach.We present a novel multi-level spatial-temporal and attentional information deep fusion network for the segmentation of retinal vessels, called MSAFNet, which enhances segmentation performance and robustness. Our method utilizes the multi-level spatial-temporal encoding module to obtain spatial-temporal information and the Self-Attention module to capture feature correlations in different levels of our network. Based on the encoder and decoder structure, we combine these features to get the final segmentation results.Main results.Through abundant experiments on four public datasets, our method achieves preferable performance compared with other SOTA retinal vessel segmentation methods. Our Accuracy and Area Under Curve achieve the highest scores of 96.96%, 96.57%, 96.48% and 98.78%, 98.54%, 98.27% on DRIVE, CHASE_DB1, and HRF datasets. Our Specificity achieves the highest score of 98.58% and 99.08% on DRIVE and STARE datasets.Significance.The experimental results demonstrate that our method has strong learning and representation capabilities and can accurately detect retinal blood vessels, thereby serving as a potential tool for assisting in diagnosis.

A lightweight dual-path cascaded network for vessel segmentation in fundus image

Automatic and fast segmentation of retinal vessels in fundus images is a prerequisite in clinical ophthalmic diseases; however, the high model complexity and low segmentation accuracy still limit its application. This paper proposes a lightweight dual-path cascaded network (LDPC-Net) for automatic and fast vessel segmentation. We designed a dual-path cascaded network via two U-shaped structures. Firstly, we employed a structured discarding (SD) convolution module to alleviate the over-fitting problem in both codec parts. Secondly, we introduced the depthwise separable convolution (DSC) technique to reduce the parameter amount of the model. Thirdly, a residual atrous spatial pyramid pooling (ResASPP) model is constructed in the connection layer to aggregate multi-scale information effectively. Finally, we performed comparative experiments on three public datasets. Experimental results show that the proposed method achieved superior performance on the accuracy, connectivity, and parameter quantity, thus proving that it can be a promising lightweight assisted tool for ophthalmic diseases.

MIC-Net: multi-scale integrated context network for automatic retinal vessel segmentation in fundus image

PURPOSE

Accurate retinal vessel segmentation is of great value in the auxiliary screening of various diseases. However, due to the low contrast between the ends of the branches of the fundus blood vessels and the background, and the variable morphology of the optic disc and cup in the retinal image, the task of high-precision retinal blood vessel segmentation still faces difficulties.

METHOD

This paper proposes a multi-scale integrated context network, MIC-Net, which fully fuses the encoder-decoder features, and extracts multi-scale information. First, a hybrid stride sampling (HSS) block was designed in the encoder to minimize the loss of helpful information caused by the downsampling operation. Second, a dense hybrid dilated convolution (DHDC) was employed in the connection layer. On the premise of preserving feature resolution, it can perceive richer contextual information. Third, a squeeze-and-excitation with residual connections (SERC) was introduced in the decoder to adjust the channel attention adaptively. Finally, we utilized a multi-layer feature fusion mechanism in the skip connection part, which enables the network to consider both low-level details and high-level semantic information.

RESULTS

We evaluated the proposed method on three public datasets DRIVE, STARE and CHASE. In the experimental results, the Area under the receiver operating characteristic (ROC) and the accuracy rate (Acc) achieved high performances of 98.62%/97.02%, 98.60%/97.76% and 98.73%/97.38%, respectively.

CONCLUSIONS

Experimental results show that the proposed method can obtain comparable segmentation performance compared with the state-of-the-art (SOTA) methods. Specifically, the proposed method can effectively reduce the small blood vessel segmentation error, thus proving it a promising tool for auxiliary diagnosis of ophthalmic diseases.

Segmentation of retinal vessels based on MRANet

The segmentation of retinal vessel takes a crucial part in computer-aided diagnosis of diseases and eye disorders. However, the insufficient segmentation of the capillary vessels and weak anti-noise interference ability make such task more difficult. To solve this problem, we proposed a multi-scale residual attention network (MRANet) which is based on U-Net network. Firstly, to collect useful information about the blood vessels more effectively, we proposed a multi-level feature fusion block (MLF block). Then, different weights of each fused feature are learned by using attention blocks, which can retain more useful feature information while reducing the interference of redundant features. Thirdly, multi-scale residual connection block (MSR block) is constructed, which can better extract the image features. Finally, we use the DropBlock layer in the network to reduce the network parameters and alleviate network overfitting. Experiments show that based on DRIVE, the accuracy rate and the AUC performance value of our network are 0.9698 and 0.9899 respectively, and based on CHASE_DB1 dataset, they are 0.9755 and 0.9893 respectively. Our network has a better segmentation effect compared with other methods, which can ensure the continuity and completeness of blood vessel segmentation.

BCR-UNet: Bi-directional ConvLSTM residual U-Net for retinal blood vessel segmentation

Background

High precision segmentation of retinal blood vessels from retinal images is a significant step for doctors to diagnose many diseases such as glaucoma and cardiovascular diseases. However, at the peripheral region of vessels, previous U-Net-based segmentation methods failed to significantly preserve the low-contrast tiny vessels.

Methods

For solving this challenge, we propose a novel network model called Bi-directional ConvLSTM Residual U-Net (BCR-UNet), which takes full advantage of U-Net, Dropblock, Residual convolution and Bi-directional ConvLSTM (BConvLSTM). In this proposed BCR-UNet model, we propose a novel Structured Dropout Residual Block (SDRB) instead of using the original U-Net convolutional block, to construct our network skeleton for improving the robustness of the network. Furthermore, to improve the discriminative ability of the network and preserve more original semantic information of tiny vessels, we adopt BConvLSTM to integrate the feature maps captured from the first residual block and the last up-convolutional layer in a nonlinear manner.

Results and discussion

We conduct experiments on four public retinal blood vessel datasets, and the results show that the proposed BCR-UNet can preserve more tiny blood vessels at the low-contrast peripheral regions, even outperforming previous state-of-the-art methods.

Refractive Index of Hemoglobin Analysis: A Comparison of Alternating Conditional Expectations and Computational Intelligence Models

Hemoglobin is one of the most important blood elements, and its optical properties will determine all other optical properties of human blood. Since the refractive index (RI) of hemoglobin plays a vital role as a non-invasive indicator of some illnesses, accurate calculation of it would be of great importance. Moreover, measurement of the RI of hemoglobin in the laboratory is time-consuming and expensive; thus, developing a smart approach to estimate this parameter is necessary. In this research, four viable strategies were used to make a quantitative correlation between the RI of hemoglobin and its influencing parameters including the concentration, wavelength, and temperature. First, alternating conditional expectations (ACE), a statistical approach, was employed to generate a correlation to predict the RI of hemoglobin. Then, three different optimized intelligent techniques-optimized neural network (ONN), optimized fuzzy inference system (OFIS), and optimized support vector regression (OSVR)-were used to model the RI. A bat-inspired (BA) algorithm was embedded in the formulation of intelligent models to obtain the optimal values of weights and biases of an artificial neural network, membership functions of the fuzzy inference system, and free parameters of support vector regression. The coefficient of determination, root-mean-square error, average absolute relative error, and symmetric mean absolute percentage error for each of the ACE, ONN, OFIS, and OSVR were found as the measure of each model's accuracy. Results showed that ACE and optimized models (ONN, OFIS, and OSVR) have promising results in the estimation of hemoglobin's RI. Collectively, ACE outperformed ONN, OFIS, and OSVR, while sensitivity analysis indicated that the concentration, wavelength, and, lastly, temperature would have the highest impact on the RI.

Retinal Vessel Segmentation, a Review of Classic and Deep Methods

Retinal illnesses such as diabetic retinopathy (DR) are the main causes of vision loss. In the early recognition of eye diseases, the segmentation of blood vessels in retina images plays an important role. Different symptoms of ocular diseases can be identified by the geometric features of ocular arteries. However, due to the complex construction of the blood vessels and their different thicknesses, segmenting the retina image is a challenging task. There are a number of algorithms that helped the detection of retinal diseases. This paper presents an overview of papers from 2016 to 2022 that discuss machine learning and deep learning methods for automatic vessel segmentation. The methods are divided into two groups: Deep learning-based, and classic methods. Algorithms, classifiers, pre-processing and specific techniques of each group is described, comprehensively. The performances of recent works are compared based on their achieved accuracy in different datasets in inclusive tables. A survey of most popular datasets like DRIVE, STARE, HRF and CHASE_DB1 is also given in this paper. Finally, a list of findings from this review is presented in the conclusion section.

A Novel Image Encryption Algorithm Based on Improved Arnold Transform and Chaotic Pulse-Coupled Neural Network

Information security has become a focal topic in the information and digital age. How to realize secure transmission and the secure storage of image data is a major research focus of information security. Aiming at this hot topic, in order to improve the security of image data transmission, this paper proposes an image encryption algorithm based on improved Arnold transform and a chaotic pulse-coupled neural network. Firstly, the oscillatory reset voltage is introduced into the uncoupled impulse neural network, which makes the uncoupled impulse neural network exhibit chaotic characteristics. The chaotic sequence is generated by multiple iterations of the chaotic pulse-coupled neural network, and then the image is pre-encrypted by XOR operation with the generated chaotic sequence. Secondly, using the improved Arnold transform, the pre-encrypted image is scrambled to further improve the scrambling degree and encryption effect of the pre-encrypted image so as to obtain the final ciphertext image. Finally, the security analysis and experimental simulation of the encrypted image are carried out. The results of quantitative evaluation show that the proposed algorithm has a better encryption effect than the partial encryption algorithm. The algorithm is highly sensitive to keys and plaintexts, has a large key space, and can effectively resist differential attacks and attacks such as noise and clipping.