Segmenting Retinal Blood Vessels With Deep Neural Networks

Joint high-resolution feature learning and vessel-shape aware convolutions for efficient vessel segmentation

Clear imagery of retinal vessels is one of the critical shreds of evidence in specific disease diagnosis and evaluation, including sophisticated hierarchical topology and plentiful-and-intensive capillaries. In this work, we propose a new topology- and shape-aware model named Multi-branch Vessel-shaped Convolution Network (MVCN) to adaptively learn high-resolution representations from retinal vessel imagery and thereby capture high-quality topology and shape information thereon. Two steps are involved in our pipeline. The former step is proposed as Multiple High-resolution Ensemble Module (MHEM) to enhance high-resolution characteristics of retinal vessel imagery via fusing scale-invariant hierarchical topology thereof. The latter is a novel vessel-shaped convolution that captures the retinal vessel topology to emerge from unrelated fundus structures. Moreover, our MVCN of separating such topology from the fundus is a dynamical multiple sub-label generation via using epistemic uncertainty, instead of manually separating raw labels to distinguish definitive and uncertain vessels. Compared to other existing methods, our method achieves the most advanced AUC values of 98.31%, 98.80%, 98.83%, and 98.65%, and the most advanced ACC of 95.83%, 96.82%, 97.09%,and 96.66% in DRIVE, CHASE_DB1, STARE, and HRF datasets. We also employ correctness, completeness, and quality metrics to evaluate skeletal similarity. Our method's evaluation metrics have doubled compared to previous methods, thereby demonstrating the effectiveness thereof.

Role Exchange-Based Self-Training Semi-Supervision Framework for Complex Medical Image Segmentation

Segmentation of complex medical images such as vascular network and pulmonary tracheal network requires segmentation of many tiny targets on each tomographic section of the 3-D medical image volume. Although semantic segmentation of medical images based on deep learning has made great progress, fully supervised models require a great amount of annotations, making such complex medical image segmentation a difficult problem. In this article, we propose a semi-supervised model for complex medical image segmentation, which innovatively proposes a bidirectional self-training paradigm, through dynamically exchanging the roles of teacher and student by estimating the reliability at the model level. The direction of information and knowledge transfer between the two networks can be controlled, and the probability distribution of the roles of teacher and student in the next stage will be jointly determined by the model's uncertainty and instability in the training process. We also resolve the problem that loosely coupled networks are prone to collapse when training on small-scale annotated data by proposing asymmetric supervision (AS) strategy and hierarchical dual student (HDS) structure. In particular, a bidirectional distillation loss combined with the role exchange (RE) strategy and a global-local-aware consistency loss are introduced to obtain stable mutual promotion and achieve matching of global and local features, respectively. We conduct detailed experiments on two public datasets and one private dataset and lead existing semi-supervised methods by a large margin, while achieving fully supervised performance at a labeling cost of 5%.

Gabor-modulated depth separable convolution for retinal vessel segmentation in fundus images

BACKGROUND

In diabetic retinopathy, precise segmentation of retinal vessels is essential for accurate diagnosis and effective disease management. This task is particularly challenging due to the varying sizes of vessels, their bifurcations, and the presence of highly curved segments. While numerous automated segmentation techniques have demonstrated strong performance, deep neural networks have struggled to effectively model the geometric transformations of retinal vessels without extensive training datasets. Moreover, the inconsistent quality of fundus photographs often results in less than satisfactory accuracy in vessel structure detection.

METHOD

To tackle these challenges, we propose a Gabor-modulated depth separable convolution UNet model that offers flexibility in capturing visual properties such as vessels' spatial frequency and orientation. Gabor filters are highly sensitive to different orientations, allowing them to detect edges and lines at specific angles. Therefore, the proposed model can effectively recognize vessels with varying widths and orientations. We have reinforced the network's learning capability by integrating Gabor convolution with Depth separable convolution.

RESULTS

Extensive experiments conducted on the DRIVE, STARE, and CHASE_DB1 datasets demonstrate the proposed model's effectiveness, even with limited training data. The integration of Gabor filters within the UNet framework significantly improves the segmentation performance, particularly in capturing vessels with varying orientations and spatial dimensions, even on noisy and progressed DR images. Our approach achieves superior performance on all metrics compared to the other deep learning models, confirming the robustness and flexibility of the proposed architecture.

CONCLUSION

The Gabor-modulated depth separable convolution UNet model effectively addresses the challenges in retinal vessel segmentation by leveraging the orientation-sensitivity of Gabor filters and the efficiency of depth separable convolutions. The model exhibits excellent segmentation performance across multiple datasets and shows its potential to enhance diagnostic accuracy in diabetic retinopathy, even when data availability is limited. Furthermore, its lightweight architecture facilitates implementation in resource-constrained environments, making it a feasible option for various clinical applications.

Perspectives: Comparison of deep learning segmentation models on biophysical and biomedical data

Deep learning-based approaches are now widely used across biophysics to help automate a variety of tasks including image segmentation, feature selection, and deconvolution. However, the presence of multiple competing deep learning architectures, each with its own advantages and disadvantages, makes it challenging to select an architecture best suited for a specific application. As such, we present a comprehensive comparison of common models. Here, we focus on the task of segmentation assuming typical (often small) training dataset sizes available from biophysics experiments and compare the following four commonly used architectures: convolutional neural networks, U-Nets, vision transformers, and vision state space models. In doing so, we establish criteria for determining optimal conditions under which each model excels, thereby offering practical guidelines for researchers and practitioners in the field.

Multiple model visual feature embedding and selection method for an efficient oncular disease classification

Early detection of ocular diseases is vital to preventing severe complications, yet it remains challenging due to the need for skilled specialists, complex imaging processes, and limited resources. Automated solutions are essential to enhance diagnostic precision and support clinical workflows. This study presents a deep learning-based system for automated classification of ocular diseases using the Ocular Disease Intelligent Recognition (ODIR) dataset. The dataset includes 5,000 patient fundus images labeled into eight categories of ocular diseases. Initial experiments utilized transfer learning models such as DenseNet201, EfficientNetB3, and InceptionResNetV2. To optimize computational efficiency, a novel two-level feature selection framework combining Linear Discriminant Analysis (LDA) and advanced neural network classifiers-Deep Neural Networks (DNN), Long Short-Term Memory (LSTM), and Bidirectional LSTM (BiLSTM)-was introduced. Among the tested approaches, the "Combined Data" strategy utilizing features from all three models achieved the best results, with the BiLSTM classifier attaining 100% accuracy, precision, and recall on the training set, and over 98% performance on the validation set. The LDA-based framework significantly reduced computational complexity while enhancing classification accuracy. The proposed system demonstrates a scalable, efficient solution for ocular disease detection, offering robust support for clinical decision-making. By bridging the gap between clinical demands and technological capabilities, it has the potential to alleviate the workload of ophthalmologists, particularly in resource-constrained settings, and improve patient outcomes globally.

ConKeD: multiview contrastive descriptor learning for keypoint-based retinal image registration

Retinal image registration is of utmost importance due to its wide applications in medical practice. In this context, we propose ConKeD, a novel deep learning approach to learn descriptors for retinal image registration. In contrast to current registration methods, our approach employs a novel multi-positive multi-negative contrastive learning strategy that enables the utilization of additional information from the available training samples. This makes it possible to learn high-quality descriptors from limited training data. To train and evaluate ConKeD, we combine these descriptors with domain-specific keypoints, particularly blood vessel bifurcations and crossovers, that are detected using a deep neural network. Our experimental results demonstrate the benefits of the novel multi-positive multi-negative strategy, as it outperforms the widely used triplet loss technique (single-positive and single-negative) as well as the single-positive multi-negative alternative. Additionally, the combination of ConKeD with the domain-specific keypoints produces comparable results to the state-of-the-art methods for retinal image registration, while offering important advantages such as avoiding pre-processing, utilizing fewer training samples, and requiring fewer detected keypoints, among others. Therefore, ConKeD shows a promising potential towards facilitating the development and application of deep learning-based methods for retinal image registration.

VDMNet: A Deep Learning Framework with Vessel Dynamic Convolution and Multi-Scale Fusion for Retinal Vessel Segmentation

Retinal vessel segmentation is crucial for diagnosing and monitoring ophthalmic and systemic diseases. Optical Coherence Tomography Angiography (OCTA) enables detailed imaging of the retinal microvasculature, but existing methods for OCTA segmentation face significant limitations, such as susceptibility to noise, difficulty in handling class imbalance, and challenges in accurately segmenting complex vascular morphologies. In this study, we propose VDMNet, a novel segmentation network designed to overcome these challenges by integrating several advanced components. Firstly, we introduce the Fast Multi-Head Self-Attention (FastMHSA) module to effectively capture both global and local features, enhancing the network's robustness against complex backgrounds and pathological interference. Secondly, the Vessel Dynamic Convolution (VDConv) module is designed to dynamically adapt to curved and crossing vessels, thereby improving the segmentation of complex morphologies. Furthermore, we employ the Multi-Scale Fusion (MSF) mechanism to aggregate features across multiple scales, enhancing the detection of fine vessels while maintaining vascular continuity. Finally, we propose Weighted Asymmetric Focal Tversky Loss (WAFT Loss) to address class imbalance issues, focusing on the accurate segmentation of small and difficult-to-detect vessels. The proposed framework was evaluated on the publicly available ROSE-1 and OCTA-3M datasets. Experimental results demonstrated that our model effectively preserved the edge information of tiny vessels and achieved state-of-the-art performance in retinal vessel segmentation across several evaluation metrics. These improvements highlight VDMNet's superior ability to capture both fine vascular details and overall vessel connectivity, making it a robust solution for retinal vessel segmentation.

In-depth analysis of research hotspots and emerging trends in AI for retinal diseases over the past decade

Background

The application of Artificial Intelligence (AI) in diagnosing retinal diseases represents a significant advancement in ophthalmological research, with the potential to reshape future practices in the field. This study explores the extensive applications and emerging research frontiers of AI in retinal diseases.

Objective

This study aims to uncover the developments and predict future directions of AI research in retinal disease over the past decade.

Methods

This study analyzes AI utilization in retinal disease research through articles, using citation data sourced from the Web of Science (WOS) Core Collection database, covering the period from January 1, 2014, to December 31, 2023. A combination of WOS analyzer, CiteSpace 6.2 R4, and VOSviewer 1.6.19 was used for a bibliometric analysis focusing on citation frequency, collaborations, and keyword trends from an expert perspective.

Results

A total of 2,861 articles across 93 countries or regions were cataloged, with notable growth in article numbers since 2017. China leads with 926 articles, constituting 32% of the total. The United States has the highest h-index at 66, while England has the most significant network centrality at 0.24. Notably, the University of London is the leading institution with 99 articles and shares the highest h-index (25) with University College London. The National University of Singapore stands out for its central role with a score of 0.16. Research primarily spans ophthalmology and computer science, with "network," "transfer learning," and "convolutional neural networks" being prominent burst keywords from 2021 to 2023.

Conclusion

China leads globally in article counts, while the United States has a significant research impact. The University of London and University College London have made significant contributions to the literature. Diabetic retinopathy is the retinal disease with the highest volume of research. AI applications have focused on developing algorithms for diagnosing retinal diseases and investigating abnormal physiological features of the eye. Future research should pivot toward more advanced diagnostic systems for ophthalmic diseases.

Research trends and hotspots in fundus image segmentation from 2007 to 2023: A bibliometric analysis

Objective

To understand the current status, research hotspots, and trends of automatic segmentation of fundus lesion images worldwide, providing a reference for subsequent related studies.

Methods

The electronic database Web of Science Core Collection was searched for research in the field of automatic segmentation of fundus lesion images from 2007 to 2023. Visualization maps of countries, authors, institutions, journals, references, and keywords were generated and analyzed using the CiteSpace and VOSviewer software.

Results

After deduplication, 707 publications were sorted out, showing an overall increasing trend in publication volume. The countries with the highest publication counts were China, followed by India, the USA, the UK, Spain, Pakistan, and Singapore. A high degree of collaboration was observed among authors, and they cooperated widely. The keywords included "diabetic retinopathy," "deep learning," "vessel segmentation," "retinal images," "optic disc localization," and so forth, with keyword bursts starting in 2018 for "retinal images," "machine learning," "biomedical imaging," "deep learning," "convolutional neural networks," and "transfer learning." The most prolific author was U Rajendra Acharya from the University of Southern Queensland, and the journal with the most publications was Computer Methods and Programs in Biomedicine.

Conclusions

Compared with manual segmentation of fundus lesion images, the use of deep learning models for segmentation is more efficient and accurate, which is crucial for patients with eye diseases. Although the number of related publications globally is relatively small, a growing trend is still witnessed, with broad connections between countries and authors, mainly concentrated in East Asia and Europe. Research institutions in this field are limited, and hence, the research on diabetic retinopathy and retinal vessel segmentation should be strengthened to promote the development of this area.

Characterization of Retinal Arteries by Adaptive Optics Ophthalmoscopy Image Analysis

OBJECTIVE

This paper aims at quantifying biomarkers from the segmentation of retinal arteries in adaptive optics ophthalmoscopy images (AOO).

METHODS

The segmentation is based on the combination of deep learning and knowledge-driven deformable models to achieve a precise segmentation of the vessel walls, with a specific attention to bifurcations. Biomarkers (junction coefficient, branching coefficient, wall to lumen ratio ( wlr)) are derived from the resulting segmentation.

RESULTS

reliable and accurate segmentations ( mse = 1.75 ±1.24 pixel) and measurements are obtained, with high reproducibility with respect to images acquisition and users, and without bias.

SIGNIFICANCE

In a preliminary clinical study of patients with a genetic small vessel disease, some of them with vascular risk factors, an increased wlr was found in comparison to a control population.

CONCLUSION

The wlr estimated in AOO images with our method (AOV, Adaptive Optics Vessel analysis) seems to be a very robust biomarker as long as the wall is well contrasted.

Dynamic-Guided Spatiotemporal Attention for Echocardiography Video Segmentation

Left ventricle (LV) endocardium segmentation in echocardiography video has received much attention as an important step in quantifying LV ejection fraction. Most existing methods are dedicated to exploiting temporal information on top of 2D convolutional networks. In addition to single appearance semantic learning, some research attempted to introduce motion cues through the optical flow estimation (OFE) task to enhance temporal consistency modeling. However, OFE in these methods is tightly coupled to LV endocardium segmentation, resulting in noisy inter-frame flow prediction, and post-optimization based on these flows accumulates errors. To address these drawbacks, we propose dynamic-guided spatiotemporal attention (DSA) for semi-supervised echocardiography video segmentation. We first fine-tune the off-the-shelf OFE network RAFT on echocardiography data to provide dynamic information. Taking inter-frame flows as additional input, we use a dual-encoder structure to extract motion and appearance features separately. Based on the connection between dynamic continuity and semantic consistency, we propose a bilateral feature calibration module to enhance both features. For temporal consistency modeling, the DSA is proposed to aggregate neighboring frame context using deformable attention that is realized by offsets grid attention. Dynamic information is introduced into DSA through a bilateral offset estimation module to effectively combine with appearance semantics and predict attention offsets, thereby guiding semantic-based spatiotemporal attention. We evaluated our method on two popular echocardiography datasets, CAMUS and EchoNet-Dynamic, and achieved state-of-the-art.

AI in interventional cardiology: Innovations and challenges

Artificial Intelligence (AI) permeates all areas of our lives. Even now, we all use AI algorithms in our daily activities, and medicine is no exception. The potential of AI technology is hard to overestimate; AI has already proven its effectiveness in many fields of science and technology. A vast number of methods have been proposed and are being implemented in various areas of medicine, including interventional cardiology. A hallmark of this discipline is the extensive use of visualization techniques not only for diagnosis but also for the treatment of patients with coronary heart disease. The implementation of instrumental AI will reduce costs, in a broad sense. In this article, we provide an overview of AI research in interventional cardiology, practical applications, as well as the problems hindering the widespread use of neural network technologies in interventional cardiology.

Retinal Vessel Segmentation by a Transformer-U-Net Hybrid Model With Dual-Path Decoder

This paper introduces an effective and efficient framework for retinal vessel segmentation. First, we design a Transformer-CNN hybrid model in which a Transformer module is inserted inside the U-Net to capture long-range interactions. Second, we design a dual-path decoder in the U-Net framework, which contains two decoding paths for multi-task outputs. Specifically, we train the extra decoder to predict vessel skeletons as an auxiliary task which helps the model learn balanced features. The proposed framework, named as TSNet, not only achieves good performances in a fully supervised learning manner but also enables a rough skeleton annotation process. The annotators only need to roughly delineate vessel skeletons instead of giving precise pixel-wise vessel annotations. To learn with rough skeleton annotations plus a few precise vessel annotations, we propose a skeleton semi-supervised learning scheme. We adopt a mean teacher model to produce pseudo vessel annotations and conduct annotation correction for roughly labeled skeletons annotations. This learning scheme can achieve promising performance with fewer annotation efforts. We have evaluated TSNet through extensive experiments on five benchmarking datasets. Experimental results show that TSNet yields state-of-the-art performances on retinal vessel segmentation and provides an efficient training scheme in practice.

Systematic Review of Retinal Blood Vessels Segmentation Based on AI-driven Technique

Image segmentation is a crucial task in computer vision and image processing, with numerous segmentation algorithms being found in the literature. It has important applications in scene understanding, medical image analysis, robotic perception, video surveillance, augmented reality, image compression, among others. In light of this, the widespread popularity of deep learning (DL) and machine learning has inspired the creation of fresh methods for segmenting images using DL and ML models respectively. We offer a thorough analysis of this recent literature, encompassing the range of ground-breaking initiatives in semantic and instance segmentation, including convolutional pixel-labeling networks, encoder-decoder architectures, multi-scale and pyramid-based methods, recurrent networks, visual attention models, and generative models in adversarial settings. We study the connections, benefits, and importance of various DL- and ML-based segmentation models; look at the most popular datasets; and evaluate results in this Literature.

Partial class activation mapping guided graph convolution cascaded U-Net for retinal vessel segmentation

Accurate segmentation of retinal vessels in fundus images is of great importance for the diagnosis of numerous ocular diseases. However, due to the complex characteristics of fundus images, such as various lesions, image noise and complex background, the pixel features of some vessels have significant differences, which makes it easy for the segmentation networks to misjudge these vessels as noise, thus affecting the accuracy of the overall segmentation. Therefore, accurately segment retinal vessels in complex situations is still a great challenge. To address the problem, a partial class activation mapping guided graph convolution cascaded U-Net for retinal vessel segmentation is proposed. The core idea of the proposed network is first to use the partial class activation mapping guided graph convolutional network to eliminate the differences of local vessels and generate feature maps with global consistency, and subsequently these feature maps are further refined by segmentation network U-Net to achieve better segmentation results. Specifically, a new neural network block, called EdgeConv, is stacked multiple layers to form a graph convolutional network to realize the transfer an update of information from local to global, so as gradually enhance the feature consistency of graph nodes. Simultaneously, in an effort to suppress the noise information that may be transferred in graph convolution and thus reduce adverse effects of noise on segmentation results, the partial class activation mapping is introduced. The partial class activation mapping can guide the information transmission between graph nodes and effectively activate vessel feature through classification labels, thereby improving the accuracy of segmentation. The performance of the proposed method is validated on four different fundus image datasets. Compared with existing state-of-the-art methods, the proposed method can improve the integrity of vessel to a certain extent when the pixel features of local vessels are significantly different, caused by objective factors such as inappropriate illumination and exudates. Moreover, the proposed method shows robustness when segmenting complex retinal vessels.

Multispectral Imaging-Based System for Detecting Tissue Oxygen Saturation With Wound Segmentation for Monitoring Wound Healing

OBJECTIVE

Blood circulation is an important indicator of wound healing. In this study, a tissue oxygen saturation detecting (TOSD) system that is based on multispectral imaging (MSI) is proposed to quantify the degree of tissue oxygen saturation (StO2) in cutaneous tissues.

METHODS

A wound segmentation algorithm is used to segment automatically wound and skin areas, eliminating the need for manual labeling and applying adaptive tissue optics. Animal experiments were conducted on six mice in which they were observed seven times, once every two days. The TOSD system illuminated cutaneous tissues with two wavelengths of light - red ([Formula: see text] nm) and near-infrared ([Formula: see text] nm), and StO2 levels were calculated using images that were captured using a monochrome camera. The wound segmentation algorithm using ResNet34-based U-Net was integrated with computer vision techniques to improve its performance.

RESULTS

Animal experiments revealed that the wound segmentation algorithm achieved a Dice score of 93.49%. The StO2 levels that were determined using the TOSD system varied significantly among the phases of wound healing. Changes in StO2 levels were detected before laser speckle contrast imaging (LSCI) detected changes in blood flux. Moreover, statistical features that were extracted from the TOSD system and LSCI were utilized in principal component analysis (PCA) to visualize different wound healing phases. The average silhouette coefficients of the TOSD system with segmentation (ResNet34-based U-Net) and LSCI were 0.2890 and 0.0194, respectively.

CONCLUSION

By detecting the StO2 levels of cutaneous tissues using the TOSD system with segmentation, the phases of wound healing were accurately distinguished. This method can support medical personnel in conducting precise wound assessments. Clinical and Translational Impact Statement-This study supports efforts in monitoring StO2 levels, wound segmentation, and wound healing phase classification to improve the efficiency and accuracy of preclinical research in the field.

LUNet: deep learning for the segmentation of arterioles and venules in high resolution fundus images

Objective.This study aims to automate the segmentation of retinal arterioles and venules (A/V) from digital fundus images (DFI), as changes in the spatial distribution of retinal microvasculature are indicative of cardiovascular diseases, positioning the eyes as windows to cardiovascular health.Approach.We utilized active learning to create a new DFI dataset with 240 crowd-sourced manual A/V segmentations performed by 15 medical students and reviewed by an ophthalmologist. We then developed LUNet, a novel deep learning architecture optimized for high-resolution A/V segmentation. The LUNet model features a double dilated convolutional block to widen the receptive field and reduce parameter count, alongside a high-resolution tail to refine segmentation details. A custom loss function was designed to prioritize the continuity of blood vessel segmentation.Main Results.LUNet significantly outperformed three benchmark A/V segmentation algorithms both on a local test set and on four external test sets that simulated variations in ethnicity, comorbidities and annotators.Significance.The release of the new datasets and the LUNet model (www.aimlab-technion.com/lirot-ai) provides a valuable resource for the advancement of retinal microvasculature analysis. The improvements in A/V segmentation accuracy highlight LUNet's potential as a robust tool for diagnosing and understanding cardiovascular diseases through retinal imaging.

Artificial intelligence in chorioretinal pathology through fundoscopy: a comprehensive review

BACKGROUND

Applications for artificial intelligence (AI) in ophthalmology are continually evolving. Fundoscopy is one of the oldest ocular imaging techniques but remains a mainstay in posterior segment imaging due to its prevalence, ease of use, and ongoing technological advancement. AI has been leveraged for fundoscopy to accomplish core tasks including segmentation, classification, and prediction.

MAIN BODY

In this article we provide a review of AI in fundoscopy applied to representative chorioretinal pathologies, including diabetic retinopathy and age-related macular degeneration, among others. We conclude with a discussion of future directions and current limitations.

SHORT CONCLUSION

As AI evolves, it will become increasingly essential for the modern ophthalmologist to understand its applications and limitations to improve patient outcomes and continue to innovate.

Leuven-Haifa High-Resolution Fundus Image Dataset for Retinal Blood Vessel Segmentation and Glaucoma Diagnosis

The Leuven-Haifa dataset contains 240 disc-centered fundus images of 224 unique patients (75 patients with normal tension glaucoma, 63 patients with high tension glaucoma, 30 patients with other eye diseases and 56 healthy controls) from the University Hospitals of Leuven. The arterioles and venules of these images were both annotated by master students in medicine and corrected by a senior annotator. All senior segmentation corrections are provided as well as the junior segmentations of the test set. An open-source toolbox for the parametrization of segmentations was developed. Diagnosis, age, sex, vascular parameters as well as a quality score are provided as metadata. Potential reuse is envisioned as the development or external validation of blood vessels segmentation algorithms or study of the vasculature in glaucoma and the development of glaucoma diagnosis algorithms. The dataset is available on the KU Leuven Research Data Repository (RDR).

A cognitive deep learning approach for medical image processing

In ophthalmic diagnostics, achieving precise segmentation of retinal blood vessels is a critical yet challenging task, primarily due to the complex nature of retinal images. The intricacies of these images often hinder the accuracy and efficiency of segmentation processes. To overcome these challenges, we introduce the cognitive DL retinal blood vessel segmentation (CoDLRBVS), a novel hybrid model that synergistically combines the deep learning capabilities of the U-Net architecture with a suite of advanced image processing techniques. This model uniquely integrates a preprocessing phase using a matched filter (MF) for feature enhancement and a post-processing phase employing morphological techniques (MT) for refining the segmentation output. Also, the model incorporates multi-scale line detection and scale space methods to enhance its segmentation capabilities. Hence, CoDLRBVS leverages the strengths of these combined approaches within the cognitive computing framework, endowing the system with human-like adaptability and reasoning. This strategic integration enables the model to emphasize blood vessels, accurately segment effectively, and proficiently detect vessels of varying sizes. CoDLRBVS achieves a notable mean accuracy of 96.7%, precision of 96.9%, sensitivity of 99.3%, and specificity of 80.4% across all of the studied datasets, including DRIVE, STARE, HRF, retinal blood vessel and Chase-DB1. CoDLRBVS has been compared with different models, and the resulting metrics surpass the compared models and establish a new benchmark in retinal vessel segmentation. The success of CoDLRBVS underscores its significant potential in advancing medical image processing, particularly in the realm of retinal blood vessel segmentation.

RNGU-NET: a novel efficient approach in Segmenting Tuberculosis using chest X-Ray images

Tuberculosis affects various tissues, including the lungs, kidneys, and brain. According to the medical report published by the World Health Organization (WHO) in 2020, approximately ten million people have been infected with tuberculosis. U-NET, a preferred method for detecting tuberculosis-like cases, is a convolutional neural network developed for segmentation in biomedical image processing. The proposed RNGU-NET architecture is a new segmentation technique combining the ResNet, Non-Local Block, and Gate Attention Block architectures. In the RNGU-NET design, the encoder phase is strengthened with ResNet, and the decoder phase incorporates the Gate Attention Block. The key innovation lies in the proposed Local Non-Local Block architecture, overcoming the bottleneck issue in U-Net models. In this study, the effectiveness of the proposed model in tuberculosis segmentation is compared to the U-NET, U-NET+ResNet, and RNGU-NET algorithms using the Shenzhen dataset. According to the results, the RNGU-NET architecture achieves the highest accuracy rate of 98.56%, Dice coefficient of 97.21%, and Jaccard index of 96.87% in tuberculosis segmentation. Conversely, the U-NET model exhibits the lowest accuracy and Jaccard index scores, while U-NET+ResNet has the poorest Dice coefficient. These findings underscore the success of the proposed RNGU-NET method in tuberculosis segmentation.

Robust Vascular Segmentation for Raw Complex Images of Laser Speckle Contrast Based on Weakly Supervised Learning

Laser speckle contrast imaging (LSCI) is widely used for in vivo real-time detection and analysis of local blood flow microcirculation due to its non-invasive ability and excellent spatial and temporal resolution. However, vascular segmentation of LSCI images still faces a lot of difficulties due to numerous specific noises caused by the complexity of blood microcirculation's structure and irregular vascular aberrations in diseased regions. In addition, the difficulties of LSCI image data annotation have hindered the application of deep learning methods based on supervised learning in the field of LSCI image vascular segmentation. To tackle these difficulties, we propose a robust weakly supervised learning method, which selects the threshold combinations and processing flows instead of labor-intensive annotation work to construct the ground truth of the dataset, and design a deep neural network, FURNet, based on UNet++ and ResNeXt. The model obtained from training achieves high-quality vascular segmentation and captures multi-scene vascular features on both constructed and unknown datasets with good generalization. Furthermore, we intravital verified the availability of this method on a tumor before and after embolization treatment. This work provides a new approach for realizing LSCI vascular segmentation and also makes a new application-level advance in the field of artificial intelligence-assisted disease diagnosis.

An improved supervised and attention mechanism-based U-Net algorithm for retinal vessel segmentation

The segmentation results of retinal blood vessels are crucial for automatically diagnosing ophthalmic diseases such as diabetic retinopathy, hypertension, cardiovascular and cerebrovascular diseases. To improve the accuracy of vessel segmentation and better extract information about small vessels and edges, we introduce the U-Net algorithm with a supervised attention mechanism for retinal vessel segmentation. We achieve this by introducing a decoder fusion module (DFM) in the encoding part, effectively combining different convolutional blocks to extract features comprehensively. Additionally, in the decoding part of U-Net, we propose the context squeeze and excitation (CSE) decoding module to enhance important contextual feature information and the detection of tiny blood vessels. For the final output, we introduce the supervised fusion mechanism (SFM), which combines multiple branches from shallow to deep layers, effectively fusing multi-scale features and capturing information from different levels, fully integrating low-level and high-level features to improve segmentation performance. Our experimental results on the public datasets of DRIVE, STARE, and CHASED_B1 demonstrate the excellent performance of our proposed network.

Retina image segmentation using the three-path Unet model

Unsupervised image segmentation is a technique that divides an image into distinct regions or objects without prior labeling. This approach offers flexibility and adaptability to various types of image data. Particularly for large datasets, it eliminates the need for manual labeling, thereby it presents advantages in terms of time and labor costs. However, when applied to retinal image segmentation, challenges arise due to variations in data, presence of noise, and manual threshold adjustments, which can lead to over-segmentation or under-segmentation of small blood vessel boundaries and endpoints. In order to enhance the precision and accuracy of retinal image segmentation, we propose a novel image supervised segmentation network based on three-path Unet model.Firstly, the Haar wavelet transform is employed to extract high-frequency image information, which forms the foundation for the proposed HaarNet, a Unet-inspired architecture. Next, the HaarNet is integrated with the Unet and SegNet frameworks to develop a three-path Unet model, referred to as TP-Unet. Finally, the model is further refined into TP-Unet+AE+DSL by incorporating the advantages of auto-encoding (AE) and deep supervised learning (DSL) techniques, thereby enhancing the overall performance of the system. To evaluate the effectiveness of our proposed model, we conduct experiments using the DRIVE and CHASE public datasets. On the DRIVE dataset, our recommended model achieves a Dice coefficient of 0.8291 and a sensitivity index of 0.8184. These results significantly outperform the Unet model by [Formula: see text] and [Formula: see text], respectively. Furthermore, our model demonstrates excellent performance on the CHASE dataset, with a Dice coefficient of 0.8162, a sensitivity of 0.8242, and an accuracy of 0.9664. These metrics surpass the Unet model by [Formula: see text], [Formula: see text], and [Formula: see text], respectively. Our proposed model provides more accurate and reliable results for retinal vessel segmentation, which holds significant potential for assisting doctors in their diagnosis.

Investigation of an efficient multi-modal convolutional neural network for multiple sclerosis lesion detection

In this study, an automated 2D machine learning approach for fast and precise segmentation of MS lesions from multi-modal magnetic resonance images (mmMRI) is presented. The method is based on an U-Net like convolutional neural network (CNN) for automated 2D slice-based-segmentation of brain MRI volumes. The individual modalities are encoded in separate downsampling branches without weight sharing, to leverage the specific features. Skip connections input feature maps to multi-scale feature fusion (MSFF) blocks at every decoder stage of the network. Those are followed by multi-scale feature upsampling (MSFU) blocks which use the information about lesion shape and location. The CNN is evaluated on two publicly available datasets: The ISBI 2015 longitudinal MS lesion segmentation challenge dataset containing 19 subjects and the MICCAI 2016 MSSEG challenge dataset containing 15 subjects from various scanners. The proposed multi-input 2D architecture is among the top performing approaches in the ISBI challenge, to which open-access papers are available, is able to outperform state-of-the-art 3D approaches without additional post-processing, can be adapted to other scanners quickly, is robust against scanner variability and can be deployed for inference even on a standard laptop without a dedicated GPU.

Multi-level spatial-temporal and attentional information deep fusion network for retinal vessel segmentation

Objective.Automatic segmentation of fundus vessels has the potential to enhance the judgment ability of intelligent disease diagnosis systems. Even though various methods have been proposed, it is still a demanding task to accurately segment the fundus vessels. The purpose of our study is to develop a robust and effective method to segment the vessels in human color retinal fundus images.Approach.We present a novel multi-level spatial-temporal and attentional information deep fusion network for the segmentation of retinal vessels, called MSAFNet, which enhances segmentation performance and robustness. Our method utilizes the multi-level spatial-temporal encoding module to obtain spatial-temporal information and the Self-Attention module to capture feature correlations in different levels of our network. Based on the encoder and decoder structure, we combine these features to get the final segmentation results.Main results.Through abundant experiments on four public datasets, our method achieves preferable performance compared with other SOTA retinal vessel segmentation methods. Our Accuracy and Area Under Curve achieve the highest scores of 96.96%, 96.57%, 96.48% and 98.78%, 98.54%, 98.27% on DRIVE, CHASE_DB1, and HRF datasets. Our Specificity achieves the highest score of 98.58% and 99.08% on DRIVE and STARE datasets.Significance.The experimental results demonstrate that our method has strong learning and representation capabilities and can accurately detect retinal blood vessels, thereby serving as a potential tool for assisting in diagnosis.

CCS-UNet: a cross-channel spatial attention model for accurate retinal vessel segmentation

Precise segmentation of retinal vessels plays an important role in computer-assisted diagnosis. Deep learning models have been applied to retinal vessel segmentation, but the efficacy is limited by the significant scale variation of vascular structures and the intricate background of retinal images. This paper supposes a cross-channel spatial attention U-Net (CCS-UNet) for accurate retinal vessel segmentation. In comparison to other models based on U-Net, our model employes a ResNeSt block for the encoder-decoder architecture. The block has a multi-branch structure that enables the model to extract more diverse vascular features. It facilitates weight distribution across channels through the incorporation of soft attention, which effectively aggregates contextual information in vascular images. Furthermore, we suppose an attention mechanism within the skip connection. This mechanism serves to enhance feature integration across various layers, thereby mitigating the degradation of effective information. It helps acquire cross-channel information and enhance the localization of regions of interest, ultimately leading to improved recognition of vascular structures. In addition, the feature fusion module (FFM) module is used to provide semantic information for a more refined vascular segmentation map. We evaluated CCS-UNet based on five benchmark retinal image datasets, DRIVE, CHASEDB1, STARE, IOSTAR and HRF. Our proposed method exhibits superior segmentation efficacy compared to other state-of-the-art techniques with a global accuracy of 0.9617/0.9806/0.9766/0.9786/0.9834 and AUC of 0.9863/0.9894/0.9938/0.9902/0.9855 on DRIVE, CHASEDB1, STARE, IOSTAR and HRF respectively. Ablation studies are also performed to evaluate the the relative contributions of different architectural components. Our proposed model is potential for diagnostic aid of retinal diseases.

Dual-path multi-scale context dense aggregation network for retinal vessel segmentation

There has been steady progress in the field of deep learning-based blood vessel segmentation. However, several challenging issues still continue to limit its progress, including inadequate sample sizes, the neglect of contextual information, and the loss of microvascular details. To address these limitations, we propose a dual-path deep learning framework for blood vessel segmentation. In our framework, the fundus images are divided into concentric patches with different scales to alleviate the overfitting problem. Then, a Multi-scale Context Dense Aggregation Network (MCDAU-Net) is proposed to accurately extract the blood vessel boundaries from these patches. In MCDAU-Net, a Cascaded Dilated Spatial Pyramid Pooling (CDSPP) module is designed and incorporated into intermediate layers of the model, enhancing the receptive field and producing feature maps enriched with contextual information. To improve segmentation performance for low-contrast vessels, we propose an InceptionConv (IConv) module, which can explore deeper semantic features and suppress the propagation of non-vessel information. Furthermore, we design a Multi-scale Adaptive Feature Aggregation (MAFA) module to fuse the multi-scale feature by assigning adaptive weight coefficients to different feature maps through skip connections. Finally, to explore the complementary contextual information and enhance the continuity of microvascular structures, a fusion module is designed to combine the segmentation results obtained from patches of different sizes, achieving fine microvascular segmentation performance. In order to assess the effectiveness of our approach, we conducted evaluations on three widely-used public datasets: DRIVE, CHASE-DB1, and STARE. Our findings reveal a remarkable advancement over the current state-of-the-art (SOTA) techniques, with the mean values of Se and F1 scores being an increase of 7.9% and 4.7%, respectively. The code is available at https://github.com/bai101315/MCDAU-Net.

Feature-guided attention network for medical image segmentation

BACKGROUND

U-Net and its variations have achieved remarkable performances in medical image segmentation. However, they have two limitations. First, the shallow layer feature of the encoder always contains background noise. Second, semantic gaps exist between the features of the encoder and the decoder. Skip-connections directly connect the encoder to the decoder, which will lead to the fusion of semantically dissimilar feature maps.

PURPOSE

To overcome these two limitations, this paper proposes a novel medical image segmentation algorithm, called feature-guided attention network, which consists of U-Net, the cross-level attention filtering module (CAFM), and the attention-guided upsampling module (AUM).

METHODS

In the proposed method, the AUM and the CAFM were introduced into the U-Net, where the AUM learns to filter the background noise in the low-level feature map of the encoder and the CAFM tries to eliminate the semantic gap between the encoder and the decoder. Specifically, the AUM adopts a top-down pathway to use the high-level feature map so as to filter the background noise in the low-level feature map of the encoder. The AUM uses the encoder features to guide the upsampling of the corresponding decoder features, thus eliminating the semantic gap between them. Four medical image segmentation tasks, including coronary atherosclerotic plaque segmentation (Dataset A), retinal vessel segmentation (Dataset B), skin lesion segmentation (Dataset C), and multiclass retinal edema lesions segmentation (Dataset D), were used to validate the proposed method.

RESULTS

For Dataset A, the proposed method achieved higher Intersection over Union (IoU) (67.91 ± 3.82 % $67.91\pm 3.82\%$ ), dice (79.39 ± 3.37 % $79.39\pm 3.37\%$ ), accuracy (98.39 ± 0.34 % $98.39\pm 0.34\%$ ), and sensitivity (85.10 ± 3.74 % $85.10\pm 3.74\%$ ) than the previous best method: CA-Net. For Dataset B, the proposed method achieved higher sensitivity (83.50%) and accuracy (97.55%) than the previous best method: SCS-Net. For Dataset C, the proposed method had highest IoU (83.47 ± 0.41 % $83.47\pm 0.41\%$ ) and dice (90.81 ± 0.34 % $90.81\pm 0.34\%$ ) than those of all compared previous methods. For Dataset D, the proposed method had highest dice (average: 81.53%; retina edema area [REA]: 83.78%; pigment epithelial detachment [PED] 77.13%), sensitivity (REA: 89.01%; SRF: 85.50%), specificity (REA: 99.35%; PED: 100.00), and accuracy (98.73%) among all compared previous networks. In addition, the number of parameters of the proposed method was 2.43 M, which is less than CA-Net (3.21 M) and CPF-Net (3.07 M).

CONCLUSIONS

The proposed method demonstrated state-of-the-art performance, outperforming other top-notch medical image segmentation algorithms. The CAFM filtered the background noise in the low-level feature map of the encoder, while the AUM eliminated the semantic gap between the encoder and the decoder. Furthermore, the proposed method was of high computational efficiency.

3D-FVS: construction and application of three-dimensional fundus vascular structure model based on single image features

BACKGROUND

Fundus microvasculature may be visually observed by ophthalmoscope and has been widely used in clinical practice. Due to the limitations of available equipment and technology, most studies only utilized the two-dimensional planar features of the fundus microvasculature.

METHODS

This study proposed a novel method for establishing the three-dimensional fundus vascular structure model and generating hemodynamic characteristics based on a single image. Firstly, the fundus vascular are segmented through our proposed network framework. Then, the length and width of vascular segments and the relationship among the adjacent segments are collected to construct the three-dimensional vascular structure model. Finally, the hemodynamic model is generated based on the vascular structure model, and highly correlated hemodynamic features are selected to diagnose the ophthalmic diseases.

RESULTS

In fundus vascular segmentation, the proposed network framework obtained 98.63% and 97.52% on Area Under Curve (AUC) and accuracy respectively. In diagnosis, the high correlation features extracted based on the proposed method achieved 95% on accuracy.

CONCLUSIONS

This study demonstrated that hemodynamic features filtered by relevance were essential for diagnosing retinal diseases. Additionally, the method proposed also outperformed the existing models on the levels of retina vessel segmentation. In conclusion, the proposed method may represent a novel way to diagnose retinal related diseases, which can analysis two-dimensional fundus pictures by extracting heterogeneous three-dimensional features.

TBENet:A two-branch boundary enhancement Network for cerebrovascular segmentation

Cerebrovascular segmentation in digital subtraction angiography (DSA) images is the gold standard for clinical diagnosis. However, owing to the complexity of cerebrovascular, automatic cerebrovascular segmentation in DSA images is a challenging task. In this paper, we propose a CNN-based Two-branch Boundary Enhancement Network (TBENet) for automatic segmentation of cerebrovascular in DSA images. The TBENet is inspired by U-Net and designed as an encoder-decoder architecture. We propose an additional boundary branch to segment the boundary of cerebrovascular and a Main and Boundary branches Fusion Module (MBFM) to integrate the boundary branch outcome with the main branch outcome to achieve better segmentation performance. The TBENet was evaluated on HMCDSA (an in-house DSA cerebrovascular dataset), and reaches 0.9611, 0.7486, 0.7152, 0.9860 and 0.9556 in Accuracy, F1 score, Sensitivity, Specificity, and AUC, respectively. Meanwhile, we tested our TBENet on the public vessel segmentation benchmark DRIVE, and the results show that our TBENet can be extended to diverse vessel segmentation tasks.

Near-Infrared Blood Vessel Image Segmentation Using Background Subtraction and Improved Mathematical Morphology

The precise display of blood vessel information for doctors is crucial. This is not only true for facilitating intravenous injections, but also for the diagnosis and analysis of diseases. Currently, infrared cameras can be used to capture images of superficial blood vessels. However, their imaging quality always has the problems of noises, breaks, and uneven vascular information. In order to overcome these problems, this paper proposes an image segmentation algorithm based on the background subtraction and improved mathematical morphology. The algorithm regards the image as a superposition of blood vessels into the background, removes the noise by calculating the size of connected domains, achieves uniform blood vessel width, and smooths edges that reflect the actual blood vessel state. The algorithm is evaluated subjectively and objectively in this paper to provide a basis for vascular image quality assessment. Extensive experimental results demonstrate that the proposed method can effectively extract accurate and clear vascular information.

Optical Identification of Diabetic Retinopathy Using Hyperspectral Imaging

The severity of diabetic retinopathy (DR) is directly correlated to changes in both the oxygen utilization rate of retinal tissue as well as the blood oxygen saturation of both arteries and veins. Therefore, the current stage of DR in a patient can be identified by analyzing the oxygen content in blood vessels through fundus images. This enables medical professionals to make accurate and prompt judgments regarding the patient's condition. However, in order to use this method to implement supplementary medical treatment, blood vessels under fundus images need to be determined first, and arteries and veins then need to be differentiated from one another. Therefore, the entire study was split into three sections. After first removing the background from the fundus images using image processing, the blood vessels in the images were then separated from the background. Second, the method of hyperspectral imaging (HSI) was utilized in order to construct the spectral data. The HSI algorithm was utilized in order to perform analysis and simulations on the overall reflection spectrum of the retinal image. Thirdly, principal component analysis (PCA) was performed in order to both simplify the data and acquire the major principal components score plot for retinopathy in arteries and veins at all stages. In the final step, arteries and veins in the original fundus images were separated using the principal components score plots for each stage. As retinopathy progresses, the difference in reflectance between the arteries and veins gradually decreases. This results in a more difficult differentiation of PCA results in later stages, along with decreased precision and sensitivity. As a consequence of this, the precision and sensitivity of the HSI method in DR patients who are in the normal stage and those who are in the proliferative DR (PDR) stage are the highest and lowest, respectively. On the other hand, the indicator values are comparable between the background DR (BDR) and pre-proliferative DR (PPDR) stages due to the fact that both stages exhibit comparable clinical-pathological severity characteristics. The results indicate that the sensitivity values of arteries are 82.4%, 77.5%, 78.1%, and 72.9% in the normal, BDR, PPDR, and PDR, while for veins, these values are 88.5%, 85.4%, 81.4%, and 75.1% in the normal, BDR, PPDR, and PDR, respectively.

A lightweight network guided with differential matched filtering for retinal vessel segmentation

The geometric morphology of retinal vessels reflects the state of cardiovascular health, and fundus images are important reference materials for ophthalmologists. Great progress has been made in automated vessel segmentation, but few studies have focused on thin vessel breakage and false-positives in areas with lesions or low contrast. In this work, we propose a new network, differential matched filtering guided attention UNet (DMF-AU), to address these issues, incorporating a differential matched filtering layer, feature anisotropic attention, and a multiscale consistency constrained backbone to perform thin vessel segmentation. The differential matched filtering is used for the early identification of locally linear vessels, and the resulting rough vessel map guides the backbone to learn vascular details. Feature anisotropic attention reinforces the vessel features of spatial linearity at each stage of the model. Multiscale constraints reduce the loss of vessel information while pooling within large receptive fields. In tests on multiple classical datasets, the proposed model performed well compared with other algorithms on several specially designed criteria for vessel segmentation. DMF-AU is a high-performance, lightweight vessel segmentation model. The source code is at https://github.com/tyb311/DMF-AU.

TP-Net: Two-Path Network for Retinal Vessel Segmentation

Refined and automatic retinal vessel segmentation is crucial for computer-aided early diagnosis of retinopathy. However, existing methods often suffer from mis-segmentation when dealing with thin and low-contrast vessels. In this paper, a two-path retinal vessel segmentation network is proposed, namely TP-Net, which consists of three core parts, i.e., main-path, sub-path, and multi-scale feature aggregation module (MFAM). Main-path is to detect the trunk area of the retinal vessels, and the sub-path to effectively capture edge information of the retinal vessels. The prediction results of the two paths are combined by MFAM, obtaining refined segmentation of retinal vessels. In the main-path, a three-layer lightweight backbone network is elaborately designed according to the characteristics of retinal vessels, and then a global feature selection mechanism (GFSM) is proposed, which can autonomously select features that are more important for the segmentation task from the features at different layers of the network, thereby, enhancing the segmentation capability for low-contrast vessels. In the sub-path, an edge feature extraction method and an edge loss function are proposed, which can enhance the ability of the network to capture edge information and reduce the mis-segmentation of thin vessels. Finally, MFAM is proposed to fuse the prediction results of main-path and sub-path, which can remove background noises while preserving edge details, and thus, obtaining refined segmentation of retinal vessels. The proposed TP-Net has been evaluated on three public retinal vessel datasets, namely DRIVE, STARE, and CHASE DB1. The experimental results show that the TP-Net achieved a superior performance and generalization ability with fewer model parameters compared with the state-of-the-art methods.

Machine Learning in Diagnosing Middle Ear Disorders Using Tympanic Membrane Images: A Meta-Analysis

OBJECTIVE

To systematically evaluate the development of Machine Learning (ML) models and compare their diagnostic accuracy for the classification of Middle Ear Disorders (MED) using Tympanic Membrane (TM) images.

METHODS

PubMed, EMBASE, CINAHL, and CENTRAL were searched up until November 30, 2021. Studies on the development of ML approaches for diagnosing MED using TM images were selected according to the inclusion criteria. PRISMA guidelines were followed with study design, analysis method, and outcomes extracted. Sensitivity, specificity, and area under the curve (AUC) were used to summarize the performance metrics of the meta-analysis. Risk of Bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 tool in combination with the Prediction Model Risk of Bias Assessment Tool.

RESULTS

Sixteen studies were included, encompassing 20254 TM images (7025 normal TM and 13229 MED). The sample size ranged from 45 to 6066 per study. The accuracy of the 25 included ML approaches ranged from 76.00% to 98.26%. Eleven studies (68.8%) were rated as having a low risk of bias, with the reference standard as the major domain of high risk of bias (37.5%). Sensitivity and specificity were 93% (95% CI, 90%-95%) and 85% (95% CI, 82%-88%), respectively. The AUC of total TM images was 94% (95% CI, 91%-96%). The greater AUC was found using otoendoscopic images than otoscopic images.

CONCLUSIONS

ML approaches perform robustly in distinguishing between normal ears and MED, however, it is proposed that a standardized TM image acquisition and annotation protocol should be developed.

LEVEL OF EVIDENCE

NA Laryngoscope, 133:732-741, 2023.

Contrast Enhancement of RGB Retinal Fundus Images for Improved Segmentation of Blood Vessels Using Convolutional Neural Networks

Retinal fundus images are non-invasively acquired and faced with low contrast, noise, and uneven illumination. The low-contrast problem makes objects in the retinal fundus image indistinguishable and the segmentation of blood vessels very challenging. Retinal blood vessels are significant because of their diagnostic importance in ophthalmologic diseases. This paper proposes improved retinal fundus images for optimal segmentation of blood vessels using convolutional neural networks (CNNs). This study explores some robust contrast enhancement tools on the RGB and the green channel of the retinal fundus images. The improved images undergo quality evaluation using mean square error (MSE), peak signal to noise ratio (PSNR), Similar Structure Index Matrix (SSIM), histogram, correlation, and intersection distance measures for histogram comparison before segmentation in the CNN-based model. The simulation results analysis reveals that the improved RGB quality outperforms the improved green channel. This revelation implies that the choice of RGB to the green channel for contrast enhancement is adequate and effectively improves the quality of the fundus images. This improved contrast will, in turn, boost the predictive accuracy of the CNN-based model during the segmentation process. The evaluation of the proposed method on the DRIVE dataset achieves an accuracy of 94.47, sensitivity of 70.92, specificity of 98.20, and AUC (ROC) of 97.56.

A deep learning-based framework for retinal fundus image enhancement

PROBLEM

Low-quality fundus images with complex degredation can cause costly re-examinations of patients or inaccurate clinical diagnosis.

AIM

This study aims to create an automatic fundus macular image enhancement framework to improve low-quality fundus images and remove complex image degradation.

METHOD

We propose a new deep learning-based model that automatically enhances low-quality retinal fundus images that suffer from complex degradation. We collected a dataset, comprising 1068 pairs of high-quality (HQ) and low-quality (LQ) fundus images from the Kangbuk Samsung Hospital's health screening program and ophthalmology department from 2017 to 2019. Then, we used these dataset to develop data augmentation methods to simulate major aspects of retinal image degradation and to propose a customized convolutional neural network (CNN) architecture to enhance LQ images, depending on the nature of the degradation. Peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), r-value (linear index of fuzziness), and proportion of ungradable fundus photographs before and after the enhancement process are calculated to assess the performance of proposed model. A comparative evaluation is conducted on an external database and four different open-source databases.

RESULTS

The results of the evaluation on the external test dataset showed an significant increase in PSNR and SSIM compared with the original LQ images. Moreover, PSNR and SSIM increased by over 4 dB and 0.04, respectively compared with the previous state-of-the-art methods (P < 0.05). The proportion of ungradable fundus photographs decreased from 42.6% to 26.4% (P = 0.012).

CONCLUSION

Our enhancement process improves LQ fundus images that suffer from complex degradation significantly. Moreover our customized CNN achieved improved performance over the existing state-of-the-art methods. Overall, our framework can have a clinical impact on reducing re-examinations and improving the accuracy of diagnosis.

COVID-19 disease identification network based on weakly supervised feature selection

The coronavirus disease 2019 (COVID-19) outbreak has resulted in countless infections and deaths worldwide, posing increasing challenges for the health care system. The use of artificial intelligence to assist in diagnosis not only had a high accuracy rate but also saved time and effort in the sudden outbreak phase with the lack of doctors and medical equipment. This study aimed to propose a weakly supervised COVID-19 classification network (W-COVNet). This network was divided into three main modules: weakly supervised feature selection module (W-FS), deep learning bilinear feature fusion module (DBFF) and Grad-CAM++ based network visualization module (Grad-Ⅴ). The first module, W-FS, mainly removed redundant background features from computed tomography (CT) images, performed feature selection and retained core feature regions. The second module, DBFF, mainly used two symmetric networks to extract different features and thus obtain rich complementary features. The third module, Grad-Ⅴ, allowed the visualization of lesions in unlabeled images. A fivefold cross-validation experiment showed an average classification accuracy of 85.3%, and a comparison with seven advanced classification models showed that our proposed network had a better performance.

Segmentation of retinal vessels in fundus images based on U-Net with self-calibrated convolutions and spatial attention modules

Automated and accurate segmentation of retinal vessels in fundus images is an important step for screening and diagnosing various ophthalmologic diseases. However, many factors, including the variations of vessels in color, shape and size, make this task become an intricate challenge. One kind of the most popular methods for vessel segmentation is U-Net based methods. However, in the U-Net based methods, the size of the convolution kernels is generally fixed. As a result, the receptive field for an individual convolution operation is single, which is not conducive to the segmentation of retinal vessels with various thicknesses. To overcome this problem, in this paper, we employed self-calibrated convolutions to replace the traditional convolutions for the U-Net, which can make the U-Net learn discriminative representations from different receptive fields. Besides, we proposed an improved spatial attention module, instead of using traditional convolutions, to connect the encoding part and decoding part of the U-Net, which can improve the ability of the U-Net to detect thin vessels. The proposed method has been tested on Digital Retinal Images for Vessel Extraction (DRIVE) database and Child Heart and Health Study in England Database (CHASE DB1). The metrics used to evaluate the performance of the proposed method are accuracy (ACC), sensitivity (SE), specificity (SP), F1-score (F1) and the area under the receiver operating characteristic curve (AUC). The ACC, SE, SP, F1 and AUC obtained by the proposed method are 0.9680, 0.8036, 0.9840, 0.8138 and 0.9840 respectively on DRIVE database, and 0.9756, 0.8118, 0.9867, 0.8068 and 0.9888 respectively on CHASE DB1, which are better than those obtained by the traditional U-Net (the ACC, SE, SP, F1 and AUC obtained by U-Net are 0.9646, 0.7895, 0.9814, 0.7963 and 0.9791 respectively on DRIVE database, and 0.9733, 0.7817, 0.9862, 0.7870 and 0.9810 respectively on CHASE DB1). The experimental results indicate that the proposed modifications in the U-Net are effective for vessel segmentation. The structure of the proposed network.

Computational intelligence in eye disease diagnosis: a comparative study

In recent years, eye disorders are an important health issue among older people. Generally, individuals with eye diseases are unaware of the gradual growth of symptoms. Therefore, routine eye examinations are required for early diagnosis. Usually, eye disorders are identified by an ophthalmologist via a slit-lamp investigation. Slit-lamp interpretations are inadequate due to the differences in the analytical skills of the ophthalmologist, inconsistency in eye disorder analysis, and record maintenance issues. Therefore, digital images of an eye and computational intelligence (CI)-based approaches are preferred as assistive methods for eye disease diagnosis. A comparative study of CI-based decision support models for eye disorder diagnosis is presented in this paper. The CI-based decision support systems used for eye abnormalities diagnosis were grouped as anterior and retinal eye abnormalities diagnostic systems, and numerous algorithms used for diagnosing the eye abnormalities were also briefed. Various eye imaging modalities, pre-processing methods such as reflection removal, contrast enhancement, region of interest segmentation methods, and public eye image databases used for CI-based eye disease diagnosis system development were also discussed in this paper. In this comparative study, the reliability of various CI-based systems used for anterior eye and retinal disorder diagnosis was compared based on the precision, sensitivity, and specificity in eye disease diagnosis. The outcomes of the comparative analysis indicate that the CI-based anterior and retinal disease diagnosis systems attained significant prediction accuracy. Hence, these CI-based diagnosis systems can be used in clinics to reduce the burden on physicians, minimize fatigue-related misdetection, and take precise clinical decisions.

Automatic vessel crossing and bifurcation detection based on multi-attention network vessel segmentation and directed graph search

Analysis of the vascular tree is the basic premise to automatically diagnose retinal biomarkers associated with ophthalmic and systemic diseases, among which accurate identification of intersection and bifurcation points is quite challenging but important for disentangling complex vascular network and tracking vessel morphology. In this paper, we present a novel directed graph search-based multi-attentive neural network approach to automatically segment the vascular network and separate intersections and bifurcations from color fundus images. Our approach uses multi-dimensional attention to adaptively integrate local features and their global dependencies while learning to focus on target structures at different scales to generate binary vascular maps. A directed graphical representation of the vascular network is constructed to represent the topology and spatial connectivity of the vascular structures. Using local geometric information including color difference, diameter, and angle, the complex vascular tree is decomposed into multiple sub-trees to finally classify and label vascular feature points. The proposed method has been tested on the DRIVE dataset and the IOSTAR dataset containing 40 images and 30 images, respectively, with 0.863 and 0.764 F¹-score of detection points and average accuracy of 0.914 and 0.854 for classification points. These results demonstrate the superiority of our proposed method outperforming state-of-the-art methods in feature point detection and classification.

A population-based study to assess two convolutional neural networks for dental age estimation

BACKGROUND

Dental age (DA) estimation using two convolutional neural networks (CNNs), VGG16 and ResNet101, remains unexplored. In this study, we aimed to investigate the possibility of using artificial intelligence-based methods in an eastern Chinese population.

METHODS

A total of 9586 orthopantomograms (OPGs) (4054 boys and 5532 girls) of the Chinese Han population aged from 6 to 20 years were collected. DAs were automatically calculated using the two CNN model strategies. Accuracy, recall, precision, and F1 score of the models were used to evaluate VGG16 and ResNet101 for age estimation. An age threshold was also employed to evaluate the two CNN models.

RESULTS

The VGG16 network outperformed the ResNet101 network in terms of prediction performance. However, the model effect of VGG16 was less favorable than that in other age ranges in the 15-17 age group. The VGG16 network model prediction results for the younger age groups were acceptable. In the 6-to 8-year-old group, the accuracy of the VGG16 model can reach up to 93.63%, which was higher than the 88.73% accuracy of the ResNet101 network. The age threshold also implies that VGG16 has a smaller age-difference error.

CONCLUSIONS

This study demonstrated that VGG16 performed better when dealing with DA estimation via OPGs than the ResNet101 network on a wholescale. CNNs such as VGG16 hold great promise for future use in clinical practice and forensic sciences.

MIC-Net: multi-scale integrated context network for automatic retinal vessel segmentation in fundus image

PURPOSE

Accurate retinal vessel segmentation is of great value in the auxiliary screening of various diseases. However, due to the low contrast between the ends of the branches of the fundus blood vessels and the background, and the variable morphology of the optic disc and cup in the retinal image, the task of high-precision retinal blood vessel segmentation still faces difficulties.

METHOD

This paper proposes a multi-scale integrated context network, MIC-Net, which fully fuses the encoder-decoder features, and extracts multi-scale information. First, a hybrid stride sampling (HSS) block was designed in the encoder to minimize the loss of helpful information caused by the downsampling operation. Second, a dense hybrid dilated convolution (DHDC) was employed in the connection layer. On the premise of preserving feature resolution, it can perceive richer contextual information. Third, a squeeze-and-excitation with residual connections (SERC) was introduced in the decoder to adjust the channel attention adaptively. Finally, we utilized a multi-layer feature fusion mechanism in the skip connection part, which enables the network to consider both low-level details and high-level semantic information.

RESULTS

We evaluated the proposed method on three public datasets DRIVE, STARE and CHASE. In the experimental results, the Area under the receiver operating characteristic (ROC) and the accuracy rate (Acc) achieved high performances of 98.62%/97.02%, 98.60%/97.76% and 98.73%/97.38%, respectively.

CONCLUSIONS

Experimental results show that the proposed method can obtain comparable segmentation performance compared with the state-of-the-art (SOTA) methods. Specifically, the proposed method can effectively reduce the small blood vessel segmentation error, thus proving it a promising tool for auxiliary diagnosis of ophthalmic diseases.

Neural Networks Application for Accurate Retina Vessel Segmentation from OCT Fundus Reconstruction

The use of neural networks for retinal vessel segmentation has gained significant attention in recent years. Most of the research related to the segmentation of retinal blood vessels is based on fundus images. In this study, we examine five neural network architectures to accurately segment vessels in fundus images reconstructed from 3D OCT scan data. OCT-based fundus reconstructions are of much lower quality compared to color fundus photographs due to noise and lower and disproportionate resolutions. The fundus image reconstruction process was performed based on the segmentation of the retinal layers in B-scans. Three reconstruction variants were proposed, which were then used in the process of detecting blood vessels using neural networks. We evaluated performance using a custom dataset of 24 3D OCT scans (with manual annotations performed by an ophthalmologist) using 6-fold cross-validation and demonstrated segmentation accuracy up to 98%. Our results indicate that the use of neural networks is a promising approach to segmenting the retinal vessel from a properly reconstructed fundus.

Retinal blood vessel segmentation by using the MS-LSDNet network and geometric skeleton reconnection method

Automatic retinal blood vessel segmentation is a key link in the diagnosis of ophthalmic diseases. Recent deep learning methods have achieved high accuracy in vessel segmentation but still face challenges in maintaining vascular structural connectivity. Therefore, this paper proposes a novel retinal blood vessel segmentation strategy that includes three stages: vessel structure detection, vessel branch extraction and broken vessel segment reconnection. First, we propose a multiscale linear structure detection network (MS-LSDNet), which improves the detection ability of fine blood vessels by learning the types of rich hierarchical features. In addition, to maintain the connectivity of the vascular structure in the process of binarization of the vascular probability map, an adaptive hysteresis threshold method for vascular extraction is proposed. Finally, we propose a vascular tree structure reconstruction algorithm based on a geometric skeleton to connect the broken vessel segments. Experimental results on three publicly available datasets show that compared with current state-of-the-art algorithms, our strategy effectively maintains the connectivity of retinal vascular tree structure.

A deep ensemble learning method for single finger-vein identification

Finger-vein biometrics has been extensively investigated for personal verification. Single sample per person (SSPP) finger-vein recognition is one of the open issues in finger-vein recognition. Despite recent advances in deep neural networks for finger-vein recognition, current approaches depend on a large number of training data. However, they lack the robustness of extracting robust and discriminative finger-vein features from a single training image sample. A deep ensemble learning method is proposed to solve the SSPP finger-vein recognition in this article. In the proposed method, multiple feature maps were generated from an input finger-vein image, based on various independent deep learning-based classifiers. A shared learning scheme is investigated among classifiers to improve their feature representation captivity. The learning speed of weak classifiers is also adjusted to achieve the simultaneously best performance. A deep learning model is proposed by an ensemble of all these adjusted classifiers. The proposed method is tested with two public finger vein databases. The result shows that the proposed approach has a distinct advantage over all the other tested popular solutions for the SSPP problem.

Fundus image classification using Inception V3 and ResNet-50 for the early diagnostics of fundus diseases

Purpose: We aim to present effective and computer aided diagnostics in the field of ophthalmology and improve eye health. This study aims to create an automated deep learning based system for categorizing fundus images into three classes: normal, macular degeneration and tessellated fundus for the timely recognition and treatment of diabetic retinopathy and other diseases. Methods: A total of 1,032 fundus images were collected from 516 patients using fundus camera from Health Management Center, Shenzhen University General Hospital Shenzhen University, Shenzhen 518055, Guangdong, China. Then, Inception V3 and ResNet-50 deep learning models are used to classify fundus images into three classes, Normal, Macular degeneration and tessellated fundus for the timely recognition and treatment of fundus diseases. Results: The experimental results show that the effect of model recognition is the best when the Adam is used as optimizer method, the number of iterations is 150, and 0.00 as the learning rate. According to our proposed approach we, achieved the highest accuracy of 93.81% and 91.76% by using ResNet-50 and Inception V3 after fine-tuned and adjusted hyper parameters according to our classification problem. Conclusion: Our research provides a reference to the clinical diagnosis or screening for diabetic retinopathy and other eye diseases. Our suggested computer aided diagnostics framework will prevent incorrect diagnoses caused by the low image quality and individual experience, and other factors. In future implementations, the ophthalmologists can implement more advanced learning algorithms to improve the accuracy of diagnosis.