Red Lesion Detection Using Dynamic Shape Features for Diabetic Retinopathy Screening

Lesion classification and diabetic retinopathy grading by integrating softmax and pooling operators into vision transformer

Introduction

Diabetic retinopathy grading plays a vital role in the diagnosis and treatment of patients. In practice, this task mainly relies on manual inspection using human visual system. However, the human visual system-based screening process is labor-intensive, time-consuming, and error-prone. Therefore, plenty of automated screening technique have been developed to address this task.

Methods

Among these techniques, the deep learning models have demonstrated promising outcomes in various types of machine vision tasks. However, most of the medical image analysis-oriented deep learning approaches are built upon the convolutional operations, which might neglect the global dependencies between long-range pixels in the medical images. Therefore, the vision transformer models, which can unveil the associations between global pixels, have been gradually employed in medical image analysis. However, the quadratic computation complexity of attention mechanism has hindered the deployment of vision transformer in clinical practices. Bearing the analysis above in mind, this study introduces an integrated self-attention mechanism with both softmax and linear modules to guarantee efficiency and expressiveness, simultaneously. To be specific, a portion of query and key tokens, which are much less than the original query and key tokens, are adopted in the attention module by adding a set of proxy tokens. Note that the proxy tokens can fully utilize both the advantages of softmax and linear attention.

Results

To evaluate the performance of the presented approach, the comparison experiments between state-of-the-art algorithms and the proposed approach are conducted. Experimental results demonstrate that the proposed approach achieves superior outcome over the state-of-the-art algorithms on the publicly available datasets.

Discussion

Accordingly, the proposed approach can be taken as a potentially valuable instrument in clinical practices.

Enhancing diabetic retinopathy and macular edema detection through multi scale feature fusion using deep learning model

BACKGROUND

This work tackles the growing problem of early identification of diabetic retinopathy and diabetic macular edema. The deep neural network design utilizes multi-scale feature fusion to improve automated diagnostic accuracy. Methods This approach uses convolutional neural networks (CNN) and is designed to combine higher-level semantic inputs with low-level textural characteristics. The contextual and localized abstract representations that complement each other are combined via a unique fusion technique.

RESULTS

Use the MESSIDOR dataset, which comprises retinal images labeled with pathological annotations, for model training and validation to ensure robust algorithm development. The suggested model shows a 98% general precision and good performance in diabetic retinopathy. This model achieves an impressive nearly 100% exactness for diabetic macular edema, with particularly high accuracy (0.99).

CONCLUSION

Consistent performance increases the likelihood that the vision will be upheld through public screening and extensive clinical integration.

A User-friendly Approach for the Diagnosis of Diabetic Retinopathy Using ChatGPT and Automated Machine Learning

Purpose

To assess the capabilities of Chat Generative Pre-trained Transformer (ChatGPT) and Vertex AI in executing code-free preprocessing, training machine learning (ML) models, and analyzing the data.

Design

Evaluation of diagnostic test or technology.

Participants

ChatGPT and Vetrex AI as publicly available large language model and ML platform, respectively.

Methods

ChatGPT was employed to improve the resolution of fundus photography images from the Methods to Evaluate Segmentation and Indexing Techniques in the field of Retinal Ophthalmology (Messidor-2) open-source dataset using the Contrast Limited Adaptive Histogram Equalization (CLAHE) technique by Fiji software. Subsequently, Vertex AI, an automated ML (AutoML) platform, was utilized to develop 2 classification models. The first model served as a binary classifier for detecting the presence of diabetic retinopathy (DR), while the second determined its severity. Finally, ChatGPT was used to provide scripts for R and Python programming languages for data analysis and was also directly employed in analyzing the data in a code-free method.

Main Outcome Measures

Evaluating the utility of ChatGPT in generating scripts for preprocessing images using Fiji and analyzing data across Python and R and assessing its potential in analyzing data through a code-free method. Investigating the capabilities of Vertex AI to train image classification models for detection of DR and its severity.

Results

Two ML models were trained using 1740 images from the Messidor-2 database. The first model, designed to detect the severity of DR, achieved an area under the precision-recall curve (AUPRC) of 0.81, with a precision rate of 81.81% and recall of 72.83%. The second model, tailored for the detection of the presence of DR, recorded a precision and recall of 84.48% with an AUPRC of 0.90.

Conclusions

ChatGPT and Vertex AI have the potential to enable physicians without coding expertise to preprocess images, analyze data, and train ML models.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

An interpretable dual attention network for diabetic retinopathy grading: IDANet

Diabetic retinopathy (DR) is the most prevalent cause of visual impairment in adults worldwide. Typically, patients with DR do not show symptoms until later stages, by which time it may be too late to receive effective treatment. DR Grading is challenging because of the small size and variation in lesion patterns. The key to fine-grained DR grading is to discover more discriminating elements such as cotton wool, hard exudates, hemorrhages, microaneurysms etc. Although deep learning models like convolutional neural networks (CNN) seem ideal for the automated detection of abnormalities in advanced clinical imaging, small-size lesions are very hard to distinguish by using traditional networks. This work proposes a bi-directional spatial and channel-wise parallel attention based network to learn discriminative features for diabetic retinopathy grading. The proposed attention block plugged with a backbone network helps to extract features specific to fine-grained DR-grading. This scheme boosts classification performance along with the detection of small-sized lesion parts. Extensive experiments are performed on four widely used benchmark datasets for DR grading, and performance is evaluated on different quality metrics. Also, for model interpretability, activation maps are generated using the LIME method to visualize the predicted lesion parts. In comparison with state-of-the-art methods, the proposed IDANet exhibits better performance for DR grading and lesion detection.

Resilient back-propagation machine learning-based classification on fundus images for retinal microaneurysm detection

BACKGROUND

The timely diagnosis of medical conditions, particularly diabetic retinopathy, relies on the identification of retinal microaneurysms. However, the commonly used retinography method poses a challenge due to the diminutive dimensions and limited differentiation of microaneurysms in images.

PROBLEM STATEMENT

Automated identification of microaneurysms becomes crucial, necessitating the use of comprehensive ad-hoc processing techniques. Although fluorescein angiography enhances detectability, its invasiveness limits its suitability for routine preventative screening.

OBJECTIVE

This study proposes a novel approach for detecting retinal microaneurysms using a fundus scan, leveraging circular reference-based shape features (CR-SF) and radial gradient-based texture features (RG-TF).

METHODOLOGY

The proposed technique involves extracting CR-SF and RG-TF for each candidate microaneurysm, employing a robust back-propagation machine learning method for training. During testing, extracted features from test images are compared with training features to categorize microaneurysm presence.

RESULTS

The experimental assessment utilized four datasets (MESSIDOR, Diaretdb1, e-ophtha-MA, and ROC), employing various measures. The proposed approach demonstrated high accuracy (98.01%), sensitivity (98.74%), specificity (97.12%), and area under the curve (91.72%).

CONCLUSION

The presented approach showcases a successful method for detecting retinal microaneurysms using a fundus scan, providing promising accuracy and sensitivity. This non-invasive technique holds potential for effective screening in diabetic retinopathy and other related medical conditions.

MBT: Model-Based Transformer for retinal optical coherence tomography image and video multi-classification

BACKGROUND AND OBJECTIVE

The detection of retinal diseases using optical coherence tomography (OCT) images and videos is a concrete example of a data classification problem. In recent years, Transformer architectures have been successfully applied to solve a variety of real-world classification problems. Although they have shown impressive discriminative abilities compared to other state-of-the-art models, improving their performance is essential, especially in healthcare-related problems.

METHODS

This paper presents an effective technique named model-based transformer (MBT). It is based on popular pre-trained transformer models, particularly, vision transformer, swin transformer for OCT image classification, and multiscale vision transformer for OCT video classification. The proposed approach is designed to represent OCT data by taking advantage of an approximate sparse representation technique. Then, it estimates the optimal features, and performs data classification.

RESULTS

The experiments are carried out using three real-world retinal datasets. The experimental results on OCT image and OCT video datasets show that the proposed method outperforms existing state-of-the-art deep learning approaches in terms of classification accuracy, precision, recall, and f1-score, kappa, AUC-ROC, and AUC-PR. It can also boost the performance of existing transformer models, including Vision transformer and Swin transformer for OCT image classification, and Multiscale Vision Transformers for OCT video classification.

CONCLUSIONS

This work presents an approach for the automated detection of retinal diseases. Although deep neural networks have proven great potential in ophthalmology applications, our findings demonstrate for the first time a new way to identify retinal pathologies using OCT videos instead of images. Moreover, our proposal can help researchers enhance the discriminative capacity of a variety of powerful deep learning models presented in published papers. This can be valuable for future directions in medical research and clinical practice.

CCS-UNet: a cross-channel spatial attention model for accurate retinal vessel segmentation

Precise segmentation of retinal vessels plays an important role in computer-assisted diagnosis. Deep learning models have been applied to retinal vessel segmentation, but the efficacy is limited by the significant scale variation of vascular structures and the intricate background of retinal images. This paper supposes a cross-channel spatial attention U-Net (CCS-UNet) for accurate retinal vessel segmentation. In comparison to other models based on U-Net, our model employes a ResNeSt block for the encoder-decoder architecture. The block has a multi-branch structure that enables the model to extract more diverse vascular features. It facilitates weight distribution across channels through the incorporation of soft attention, which effectively aggregates contextual information in vascular images. Furthermore, we suppose an attention mechanism within the skip connection. This mechanism serves to enhance feature integration across various layers, thereby mitigating the degradation of effective information. It helps acquire cross-channel information and enhance the localization of regions of interest, ultimately leading to improved recognition of vascular structures. In addition, the feature fusion module (FFM) module is used to provide semantic information for a more refined vascular segmentation map. We evaluated CCS-UNet based on five benchmark retinal image datasets, DRIVE, CHASEDB1, STARE, IOSTAR and HRF. Our proposed method exhibits superior segmentation efficacy compared to other state-of-the-art techniques with a global accuracy of 0.9617/0.9806/0.9766/0.9786/0.9834 and AUC of 0.9863/0.9894/0.9938/0.9902/0.9855 on DRIVE, CHASEDB1, STARE, IOSTAR and HRF respectively. Ablation studies are also performed to evaluate the the relative contributions of different architectural components. Our proposed model is potential for diagnostic aid of retinal diseases.

Fine-grained attention & knowledge-based collaborative network for diabetic retinopathy grading

Accurate diabetic retinopathy (DR) grading is crucial for making the proper treatment plan to reduce the damage caused by vision loss. This task is challenging due to the fact that the DR related lesions are often small and subtle in visual differences and intra-class variations. Moreover, relationships between the lesions and the DR levels are complicated. Although many deep learning (DL) DR grading systems have been developed with some success, there are still rooms for grading accuracy improvement. A common issue is that not much medical knowledge was used in these DL DR grading systems. As a result, the grading results are not properly interpreted by ophthalmologists, thus hinder the potential for practical applications. This paper proposes a novel fine-grained attention & knowledge-based collaborative network (FA+KC-Net) to address this concern. The fine-grained attention network dynamically divides the extracted feature maps into smaller patches and effectively captures small image features that are meaningful in the sense of its training from large amount of retinopathy fundus images. The knowledge-based collaborative network extracts a-priori medical knowledge features, i.e., lesions such as the microaneurysms (MAs), soft exudates (SEs), hard exudates (EXs), and hemorrhages (HEs). Finally, decision rules are developed to fuse the DR grading results from the fine-grained network and the knowledge-based collaborative network to make the final grading. Extensive experiments are carried out on four widely-used datasets, the DDR, Messidor, APTOS, and EyePACS to evaluate the efficacy of our method and compare with other state-of-the-art (SOTA) DL models. Simulation results show that proposed FA+KC-Net is accurate and stable, achieves the best performances on the DDR, Messidor, and APTOS datasets.

Hybrid adaptive deep learning classifier for early detection of diabetic retinopathy using optimal feature extraction and classification

Objectives

Diabetic retinopathy (DR) is one of the leading causes of blindness. It is important to use a comprehensive learning method to identify the DR. However, comprehensive learning methods often rely heavily on encrypted data, which can be costly and time consuming. Also, the DR function is not displayed and is scattered in the high-definition image below.

Methods

Therefore, learning how to distribute such DR functions is a big challenge. In this work, we proposed a hybrid adaptive deep learning classifier for early detection of diabetic retinopathy (HADL-DR). First, we provide an improved multichannel-based generative adversarial network (MGAN) with semi-maintenance to detect blood vessels segmentation.

Results

By reducing the reliance on the encoded data, the following high-resolution images can be used to detect the indivisible features of some semi-observed MGAN references. Scale invariant feature transform (SIFT) function is then extracted and the best function is selected using the improved sequential approximation optimization (SAO) algorithm. After that, a hybrid recurrent neural network with long short-term memory (RNN-LSTM) is utilized for DR classification. The proposed RNN-LSTM classifier evaluated through standard benchmark Kaggle and Messidor datasets.

Conclusion

Finally, the simulation results are compared with the existing state-of-art classifiers in terms of accuracy, precision, recall, f-measure and area under cover (AUC), it is seen that more successful results are obtained.

Enhancing retinal images in low-light conditions using semidecoupled decomposition

Eye diseases that are common and many diseases that result in visual ailments, such as diabetes and vascular disease, can be diagnosed through retinal imaging. The enhancement of retinal images often helps in diagnosing diseases related to retinal organ failure. However, today's image enhancement methods may lead to artificial boundaries, sudden color gradation, and the loss of image details. Therefore, to prevent these side effects, a new method of retinal image enhancement is proposed. In this work, we propose a new method for enhancing the overall contrast of colored retinal images. That is, we propose low-light image enhancement using a new retinex method based on a powerful semidecoupled retinex method. In particular, illumination layer I gradually approximates the S input image according to the file. This leads to a complete Gaussian transformation model, while the R-layer reflectance is estimated jointly by S and intermediary by I to suppress image noise simultaneously during R estimation on the publicly available Messidor database. From our assessment measurements (PSNR and SSIM), we show that this proposed method is effective in comparison with the relevant and recently proposed retinal imaging methods; moreover, the color, which is determined by the data, does not change the image structure. Finally, a technique is presented to improve the pronounced color of a retinal image, which is useful for ophthalmologists to screen for retinal disease more effectively. Moreover, this technique can be used in the development of robotics for imaging tests to search for clinical markers.

Improved detection accuracy of red lesions in retinal fundus images with superlearning approach

BACKGROUND

Diabetic Retinopathy (DR) is a serious consequence of diabetes that can result to permanent vision loss for a person. Diabetes-related vision impairment can be significantly avoided with timely screening and treatment in its initial phase. The earliest and the most noticeable indications on the surface of the retina are micro-aneurysm and haemorrhage, which appear as dark patches. Therefore, the automatic detection of retinopathy begins with the identification of all these dark lesions.

METHOD

In our study, we have developed a clinical knowledge based segmentation built on Early Treatment DR Study (ETDRS). ETDRS is a gold standard for identifying all red lesions using adaptive-thresholding approach followed by different pre-processing steps. The lesions are classified using super-learning approach to improve multi-class detection accuracy. Ensemble based super-learning approach finds optimal weights of base learners by minimizing the cross validated risk-function and it pledges the improved performance compared to base-learners predictions. For multi-class classification, a well informative feature-set based on colour, intensity, shape, size and texture, is developed. In this work, we have handled the data imbalance problem and compared the final accuracy with different synthetic data creation ratios.

RESULT

The suggested approach uses publicly available resources to perform quantitative assessments at lesions-level. The overall accuracy of red lesion segregation is 93.5%, which has increased to 97.88% when data imbalance problem is taken care-off.

CONCLUSION

The results of our system have achieved competitive performance compared with other modern approaches and handling of data imbalance further increases the performance of it.

Detection and diabetic retinopathy grading using digital retinal images

Diabetic Retinopathy is an eye disorder that affects people suffering from diabetes. Higher sugar levels in blood leads to damage of blood vessels in eyes and may even cause blindness. Diabetic retinopathy is identified by red spots known as microanuerysms and bright yellow lesions called exudates. It has been observed that early detection of exudates and microaneurysms may save the patient’s vision and this paper proposes a simple and effective technique for diabetic retinopathy. Both publicly available and real time datasets of colored images captured by fundus camera have been used for the empirical analysis. In the proposed work, grading has been done to know the severity of diabetic retinopathy i.e. whether it is mild, moderate or severe using exudates and micro aneurysms in the fundus images. An automated approach that uses image processing, features extraction and machine learning models to predict accurately the presence of the exudates and micro aneurysms which can be used for grading has been proposed. The research is carried out in two segments; one for exudates and another for micro aneurysms. The grading via exudates is done based upon their distance from macula whereas grading via micro aneurysms is done by calculating their count. For grading using exudates, support vector machine and K-Nearest neighbor show the highest accuracy of 92.1% and for grading using micro aneurysms, decision tree shows the highest accuracy of 99.9% in prediction of severity levels of the disease.

Nested U-Net for Segmentation of Red Lesions in Retinal Fundus Images and Sub-image Classification for Removal of False Positives

Diabetic retinopathy is a pathological change of the retina that occurs for long-term diabetes. The patients become symptomatic in advanced stages of diabetic retinopathy resulting in severe non-proliferative diabetic retinopathy or proliferative diabetic retinopathy stages. There is a need of an automated screening tool for the early detection and treatment of patients with diabetic retinopathy. This paper focuses on the segmentation of red lesions using nested U-Net Zhou et al. (Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer, 2018) followed by removal of false positives based on the sub-image classification method. Different sizes of sub-images were studied for the reduction in false positives in the sub-image classification method. The network could capture semantic features and fine details due to dense convolutional blocks connected via skip connections in between down sampling and up sampling paths. False-negative candidates were very few and the sub-image classification network effectively reduced the falsely detected candidates. The proposed framework achieves a sensitivity of [Formula: see text], precision of [Formula: see text], and F1-Score of [Formula: see text] for the DIARETDB1 data set Kalviainen and Uusutalo (Medical Image Understanding and Analysis, Citeseer, 2007). It outperforms the state-of-the-art networks such as U-Net Ronneberger et al. (International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2015) and attention U-Net Oktay et al. (Attention u-net: Learning where to look for the pancreas, 2018).

SAFARI: shape analysis for AI-segmented images

BACKGROUND

Recent developments to segment and characterize the regions of interest (ROI) within medical images have led to promising shape analysis studies. However, the procedures to analyze the ROI are arbitrary and vary by study. A tool to translate the ROI to analyzable shape representations and features is greatly needed.

RESULTS

We developed SAFARI (shape analysis for AI-segmented images), an open-source R package with a user-friendly online tool kit for ROI labelling and shape feature extraction of segmented maps, provided by AI-algorithms or manual segmentation. We demonstrated that half of the shape features extracted by SAFARI were significantly associated with survival outcomes in a case study on 143 consecutive patients with stage I-IV lung cancer and another case study on 61 glioblastoma patients.

CONCLUSIONS

SAFARI is an efficient and easy-to-use toolkit for segmenting and analyzing ROI in medical images. It can be downloaded from the comprehensive R archive network (CRAN) and accessed at https://lce.biohpc.swmed.edu/safari/ .

Collaborative learning of weakly-supervised domain adaptation for diabetic retinopathy grading on retinal images

Early detection and treatment of diabetic retinopathy (DR) can significantly reduce the risk of vision loss in patients. In essence, we are faced with two challenges: (i) how to simultaneously achieve domain adaptation from the different domains and (ii) how to build an interpretable multi-instance learning (MIL) on the target domain in an end-to-end framework. In this paper, we address these issues and propose a unified weakly-supervised domain adaptation framework, which consists of three components: domain adaptation, instance progressive discriminator and multi-instance learning with attention. The method models the relationship between the patches and images in the target domain with a multi-instance learning scheme and an attention mechanism. Meanwhile, it incorporates all available information from both source and target domains for a jointly learning strategy. We validate the performance of the proposed framework for DR grading on the Messidor dataset and the large-scale Eyepacs dataset. The experimental results demonstrate that it achieves an average accuracy of 0.949 (95% CI 0.931-0.958)/0.764 (95% CI 0.755-0.772) and an average AUC value of 0.958 (95% CI 0.945-0.962)/0.749 (95% CI 0.732-0.761) for binary-class/multi-class classification tasks on the Messidor dataset. Moreover, the proposed method achieves an accuracy of 0.887 and a quadratic weighted kappa score value of 0.860 on the Eyepacs dataset, outperforming the state-of-the-art approaches. Comprehensive experiments confirm the effectiveness of the approach in terms of both grading performance and interpretability. The source code is available at https://github.com/HouQingshan/WAD-Net.

Hybrid multi-kernel SVM algorithm for detection of microaneurysm in color fundus images

Diabetic retinopathy (DR) is a chronic disease that may cause vision loss in diabetic patients. Microaneurysms which are characterized by small red spots on the retina due to fluid or blood leakage from the weak capillary wall often occur during the early stage of DR, making screening at this stage is essential. In this paper, an automatic screening system for early detection of DR in retinal images is developed using a combined shape and texture features. Due to minimum number of hand-crafted features, the computational burden is much reduced. The proposed hybrid multi-kernel support vector machine classifier is constructed by learning a kernel model formed as a combination of the base kernels. This approach outperforms the recent deep learning techniques in terms of the evaluation metrics. The efficiency of the proposed scheme is experimentally validated on three public datasets - Retinopathy Online Challenge, DIARETdB1, MESSIDOR, and AGAR300 (developed for this study). Studies reveal that the proposed model produced the best results of 0.503 in ROC dataset, 0.481 in DIARETdB1, and 0.464 in the MESSIDOR dataset in terms of FROC score. The AGAR300 database outperforms the existing MA detection algorithm in terms of FROC, AUC, F1 score, precision, sensitivity, and specificity which guarantees the robustness of this system.

Lesion-attention pyramid network for diabetic retinopathy grading

As one of the most common diabetic complications, diabetic retinopathy (DR) can cause retinal damage, vision loss and even blindness. Automated DR grading technology has important clinical significance, which can help ophthalmologists achieve rapid and early diagnosis. With the popularity of deep learning, DR grading based on the convolutional neural networks (CNNs) has become the mainstream method. Unfortunately, although the CNN-based method can achieve satisfactory diagnostic accuracy, it lacks significant clinical information. In this paper, a lesion-attention pyramid network (LAPN) is presented. The pyramid network integrates the subnetworks with different resolutions to get multi-scale features. In order to take the lesion regions in the high-resolution image as the diagnostic evidence, the low-resolution network calculates the lesion activation map (using the weakly-supervised localization method) and guides the high-resolution network to concentrate on the lesion regions. Furthermore, a lesion attention module (LAM) is designed to capture the complementary relationship between the high-resolution features and the low-resolution features, and to fuse the lesion activation map. Experiment results show that the proposed scheme outperforms other existing approaches, and the proposed method can provide lesion activation map with lesion consistency as an additional evidence for clinical diagnosis.

Red lesion in fundus image with hexagonal pattern feature and two-level segmentation

Red lesion identification at its early stage is very essential for the treatment of diabetic retinopathy to prevent loss of vision. This work proposes a red lesion detection algorithm that uses Hexagonal pattern-based features with two-level segmentation that can detect hemorrhage and microaneurysms in the fundus image. The proposed scheme initially pre-processes the fundus image followed by a two-level segmentation. The level 1 segmentation eliminates the background whereas the level 2 segmentation eliminates the blood vessels that introduce more false positives. A hexagonal pattern-based feature is extracted from the red lesion candidates which can highly differentiate the lesion from non-lesion regions. The hexagonal pattern features are then trained using the recurrent neural network and are classified to eliminate the false negatives. For the evaluation of the proposed red lesion algorithm, the datasets namely ROC challenge, e-ophtha, DiaretDB1, and Messidor are used with the metrics such as Accuracy, Recall, Precision, F1 score, Specificity, and AUC. The scheme provides an average Accuracy, Recall (Sensitivity), Precision, F1 score, Specificity, and AUC of 95.48%, 84.54%, 97.3%, 90.47%, 86.81% and 93.43% respectively.

A critical review on diagnosis of diabetic retinopathy using machine learning and deep learning

Diabetic Retinopathy (DR) is a health condition caused due to Diabetes Mellitus (DM). It causes vision problems and blindness due to disfigurement of human retina. According to statistics, 80% of diabetes patients battling from long diabetic period of 15 to 20 years, suffer from DR. Hence, it has become a dangerous threat to the health and life of people. To overcome DR, manual diagnosis of the disease is feasible but overwhelming and cumbersome at the same time and hence requires a revolutionary method. Thus, such a health condition necessitates primary recognition and diagnosis to prevent DR from developing into severe stages and prevent blindness. Innumerable Machine Learning (ML) models are proposed by researchers across the globe, to achieve this purpose. Various feature extraction techniques are proposed for extraction of DR features for early detection. However, traditional ML models have shown either meagre generalization throughout feature extraction and classification for deploying smaller datasets or consumes more of training time causing inefficiency in prediction while using larger datasets. Hence Deep Learning (DL), a new domain of ML, is introduced. DL models can handle a smaller dataset with help of efficient data processing techniques. However, they generally incorporate larger datasets for their deep architectures to enhance performance in feature extraction and image classification. This paper gives a detailed review on DR, its features, causes, ML models, state-of-the-art DL models, challenges, comparisons and future directions, for early detection of DR.

Deep Red Lesion Classification for Early Screening of Diabetic Retinopathy

Diabetic retinopathy (DR) is an asymptotic and vision-threatening complication among working-age adults. To prevent blindness, a deep convolutional neural network (CNN) based diagnosis can help to classify less-discriminative and small-sized red lesions in early screening of DR patients. However, training deep models with minimal data is a challenging task. Fine-tuning through transfer learning is a useful alternative, but performance degradation, overfitting, and domain adaptation issues further demand architectural amendments to effectively train deep models. Various pre-trained CNNs are fine-tuned on an augmented set of image patches. The best-performing ResNet50 model is modified by introducing reinforced skip connections, a global max-pooling layer, and the sum-of-squared-error loss function. The performance of the modified model (DR-ResNet50) on five public datasets is found to be better than state-of-the-art methods in terms of well-known metrics. The highest scores (0.9851, 0.991, 0.991, 0.991, 0.991, 0.9939, 0.0029, 0.9879, and 0.9879) for sensitivity, specificity, AUC, accuracy, precision, F1-score, false-positive rate, Matthews’s correlation coefficient, and kappa coefficient are obtained within a 95% confidence interval for unseen test instances from e-Ophtha_MA. This high sensitivity and low false-positive rate demonstrate the worth of a proposed framework. It is suitable for early screening due to its performance, simplicity, and robustness.

Optimal Feature Selection for Diagnosing Diabetic Retinopathy Using FireFly Migration Operator-Based Monarch Butterfly Optimization

In recent years, diabetic retinopathy (DR) needs to be focused with the intention of developing accurate and effective approaches by accomplishing the existing challenges in the traditional models. With this objective, this paper aims to introduce an effective diagnosis system by utilizing retinal fundus images. The implementation of this diagnosis model incorporates 4 stages like (i) preprocessing, (ii) blood vessel segmentation, (iii) feature extraction, as well as (iv) classification. Originally, the median filter as well as contrast limited adaptive histogram equalization (CLAHE) help to preprocess the image. Moreover, the Fuzzy C Mean (FCM) thresholding is applied for blood vessel segmentation, which generates stochastic clustering of pixels to obtain enhanced threshold values. Further, feature extraction is accomplished by utilizing gray-level run-length matrix (GLRM), local, and morphological transformation-based features. Furthermore, a deep learning (DL) model known as convolutional neural network (CNN) is employed for the diagnosis or classification purpose. As a main novelty, this paper introduces an optimal feature selection as well as classification model. Further, the feature selection is done optimally by FireFly Migration Operator-based Monarch Butterfly Optimization (FM-MBO) which hybridized of the monarch butterfly optimization (MBO) and fire fly (FF) algorithms as the entire adopted extracted features attain higher feature length. Moreover, the proposed FM-MBO algorithm helps for optimizing the count of CNN's convolutional neurons to further improve the performance accuracy. At the end, the enhanced outcomes of the adopted diagnostic scheme are validated via a valuable comparative examination in terms of significant performance measures.

Weakly Supervised Sensitive Heatmap framework to classify and localize diabetic retinopathy lesions

Vision loss happens due to diabetic retinopathy (DR) in severe stages. Thus, an automatic detection method applied to diagnose DR in an earlier phase may help medical doctors to make better decisions. DR is considered one of the main risks, leading to blindness. Computer-Aided Diagnosis systems play an essential role in detecting features in fundus images. Fundus images may include blood vessels, exudates, micro-aneurysm, hemorrhages, and neovascularization. In this paper, our model combines automatic detection for the diabetic retinopathy classification with localization methods depending on weakly-supervised learning. The model has four stages; in stage one, various preprocessing techniques are applied to smooth the data set. In stage two, the network had gotten deeply to the optic disk segment for eliminating any exudate's false prediction because the exudates had the same color pixel as the optic disk. In stage three, the network is fed through training data to classify each label. Finally, the layers of the convolution neural network are re-edited, and used to localize the impact of DR on the patient's eye. The framework tackles the matching technique between two essential concepts where the classification problem depends on the supervised learning method. While the localization problem was obtained by the weakly supervised method. An additional layer known as weakly supervised sensitive heat map (WSSH) was added to detect the ROI of the lesion at a test accuracy of 98.65%, while comparing with Class Activation Map that involved weakly supervised technology achieved 0.954. The main purpose is to learn a representation that collect the central localization of discriminative features in a retina image. CNN-WSSH model is able to highlight decisive features in a single forward pass for getting the best detection of lesions.

Telemedicine in diabetic retinopathy screening in India

With ever-growing prevalence of diabetes mellitus and its most common microvascular complication diabetic retinopathy (DR) in Indian population, screening for DR early for prevention of development of vision-threatening stages of the disease is becoming increasingly important. Most of the programs in India for DR screening are opportunistic and a universal screening program does not exist. Globally, telemedicine programs have demonstrated accuracy in classification of DR into referable disease, as well as into stages, with accuracies reaching that of human graders, in a cost-effective manner and with sufficient patient satisfaction. In this major review, we have summarized the global experience of telemedicine in DR screening and the way ahead toward planning a national integrated DR screening program based on telemedicine.

Fundus image segmentation via hierarchical feature learning

Fundus Image Segmentation (FIS) is an essential procedure for the automated diagnosis of ophthalmic diseases. Recently, deep fully convolutional networks have been widely used for FIS with state-of-the-art performance. The representative deep model is the U-Net, which follows an encoder-decoder architecture. I believe it is suboptimal for FIS because consecutive pooling operations in the encoder lead to low-resolution representation and loss of detailed spatial information, which is particularly important for the segmentation of tiny vessels and lesions. Motivated by this, a high-resolution hierarchical network (HHNet) is proposed to learn semantic-rich high-resolution representations and preserve spatial details simultaneously. Specifically, a High-resolution Feature Learning (HFL) module with increasing dilation rates was first designed to learn the high-level high-resolution representations. Then, the HHNet was constructed by incorporating three HFL modules and two feature aggregation modules. The HHNet runs in a coarse-to-fine manner, and fine segmentation maps are output at the last level. Extensive experiments were conducted on fundus lesion segmentation, vessel segmentation, and optic cup segmentation. The experimental results reveal that the proposed method shows highly competitive or even superior performance in terms of segmentation performance and computation cost, indicating its potential advantages in clinical application.

Red-lesion extraction in retinal fundus images by directional intensity changes' analysis

Diabetic retinopathy (DR) is an important retinal disease threatening people with the long diabetic history. Blood leakage in retina leads to the formation of red lesions in retina the analysis of which is helpful in the determination of severity of disease. In this paper, a novel red-lesion extraction method is proposed. The new method firstly determines the boundary pixels of blood vessel and red lesions. Then, it determines the distinguishing features of boundary pixels of red-lesions to discriminate them from other boundary pixels. The main point utilized here is that a red lesion can be observed as significant intensity changes in almost all directions in the fundus image. This can be feasible through considering special neighborhood windows around the extracted boundary pixels. The performance of the proposed method has been evaluated for three different datasets including Diaretdb0, Diaretdb1 and Kaggle datasets. It is shown that the method is capable of providing the values of 0.87 and 0.88 for sensitivity and specificity of Diaretdb1, 0.89 and 0.9 for sensitivity and specificity of Diaretdb0, 0.82 and 0.9 for sensitivity and specificity of Kaggle. Also, the proposed method has a time-efficient performance in the red-lesion extraction process.

Simple methods for the lesion detection and severity grading of diabetic retinopathy by image processing and transfer learning

Diabetic retinopathy (DR) has become one of the major causes of blindness. Due to the increased prevalence of diabetes worldwide, diabetic patients exhibit high probabilities of developing DR. There is a need to develop a labor-less computer-aided diagnosis system to support the clinical diagnosis. Here, we attempted to develop simple methods for severity grading and lesion detection from retinal fundus images. We developed a severity grading system for DR by transfer learning with a recent convolutional neural network called EfficientNet-B3 and the publicly available Kaggle Asia Pacific Tele-Ophthalmology Society (APTOS) 2019 training dataset, which includes artificial noise. After removing the blurred and duplicated images from the dataset using a numerical threshold, the trained model achieved specificity and sensitivity values ≳ 0.98 in the identification of DR retinas. For severity grading, the classification accuracy values of 0.84, 0.95, and 0.98 were recorded for the 1st, 2nd, and 3rd predicted labels, respectively. The utility of EfficientNets-B3 for the severity grading of DR as well as the detailed retinal areas referred were confirmed via visual explanation methods of convolutional neural networks. Lesion extraction was performed by applying an empirically defined threshold value to the enhanced retinal images. Although the extraction of blood vessels and detection of red lesions occurred simultaneously, the red and white lesions, including both soft and hard exudates, were clearly extracted. The detected lesion areas were further confirmed with ground truth using the DIARETDB1 database images with general accuracy. The simple and easily applicable methods proposed in this study will aid in the detection and severity grading of DR, which might help in the selection of appropriate treatment strategies for DR.

EAD-Net: A Novel Lesion Segmentation Method in Diabetic Retinopathy Using Neural Networks

Diabetic retinopathy (DR) is a common chronic fundus disease, which has four different kinds of microvessel structure and microvascular lesions: microaneurysms (MAs), hemorrhages (HEs), hard exudates, and soft exudates. Accurate detection and counting of them are a basic but important work. The manual annotation of these lesions is a labor-intensive task in clinical analysis. To solve the problem, we proposed a novel segmentation method for different lesions in DR. Our method is based on a convolutional neural network and can be divided into encoder module, attention module, and decoder module, so we refer it as EAD-Net. After normalization and augmentation, the fundus images were sent to the EAD-Net for automated feature extraction and pixel-wise label prediction. Given the evaluation metrics based on the matching degree between detected candidates and ground truth lesions, our method achieved sensitivity of 92.77%, specificity of 99.98%, and accuracy of 99.97% on the e_ophtha_EX dataset and comparable AUPR (Area under Precision-Recall curve) scores on IDRiD dataset. Moreover, the results on the local dataset also show that our EAD-Net has better performance than original U-net in most metrics, especially in the sensitivity and F1-score, with nearly ten percent improvement. The proposed EAD-Net is a novel method based on clinical DR diagnosis. It has satisfactory results on the segmentation of four different kinds of lesions. These effective segmentations have important clinical significance in the monitoring and diagnosis of DR.

Detection of Diabetic Eye Disease from Retinal Images Using a Deep Learning Based CenterNet Model

Diabetic retinopathy (DR) is an eye disease that alters the blood vessels of a person suffering from diabetes. Diabetic macular edema (DME) occurs when DR affects the macula, which causes fluid accumulation in the macula. Efficient screening systems require experts to manually analyze images to recognize diseases. However, due to the challenging nature of the screening method and lack of trained human resources, devising effective screening-oriented treatment is an expensive task. Automated systems are trying to cope with these challenges; however, these methods do not generalize well to multiple diseases and real-world scenarios. To solve the aforementioned issues, we propose a new method comprising two main steps. The first involves dataset preparation and feature extraction and the other relates to improving a custom deep learning based CenterNet model trained for eye disease classification. Initially, we generate annotations for suspected samples to locate the precise region of interest, while the other part of the proposed solution trains the Center Net model over annotated images. Specifically, we use DenseNet-100 as a feature extraction method on which the one-stage detector, CenterNet, is employed to localize and classify the disease lesions. We evaluated our method over challenging datasets, namely, APTOS-2019 and IDRiD, and attained average accuracy of 97.93% and 98.10%, respectively. We also performed cross-dataset validation with benchmark EYEPACS and Diaretdb1 datasets. Both qualitative and quantitative results demonstrate that our proposed approach outperforms state-of-the-art methods due to more effective localization power of CenterNet, as it can easily recognize small lesions and deal with over-fitted training data. Our proposed framework is proficient in correctly locating and classifying disease lesions. In comparison to existing DR and DME classification approaches, our method can extract representative key points from low-intensity and noisy images and accurately classify them. Hence our approach can play an important role in automated detection and recognition of DR and DME lesions.

An interpretable multiple-instance approach for the detection of referable diabetic retinopathy in fundus images

Diabetic retinopathy (DR) is one of the leading causes of vision loss across the world. Yet despite its wide prevalence, the majority of affected people lack access to the specialized ophthalmologists and equipment required for monitoring their condition. This can lead to delays in the start of treatment, thereby lowering their chances for a successful outcome. Machine learning systems that automatically detect the disease in eye fundus images have been proposed as a means of facilitating access to retinopathy severity estimates for patients in remote regions or even for complementing the human expert’s diagnosis. Here we propose a machine learning system for the detection of referable diabetic retinopathy in fundus images, which is based on the paradigm of multiple-instance learning. Our method extracts local information independently from multiple rectangular image patches and combines it efficiently through an attention mechanism that focuses on the abnormal regions of the eye (i.e. those that contain DR-induced lesions), thus resulting in a final image representation that is suitable for classification. Furthermore, by leveraging the attention mechanism our algorithm can seamlessly produce informative heatmaps that highlight the regions where the lesions are located. We evaluate our approach on the publicly available Kaggle, Messidor-2 and IDRiD retinal image datasets, in which it exhibits near state-of-the-art classification performance (AUC of 0.961 in Kaggle and 0.976 in Messidor-2), while also producing valid lesion heatmaps (AUPRC of 0.869 in the 81 images of IDRiD that contain pixel-level lesion annotations). Our results suggest that the proposed approach provides an efficient and interpretable solution against the problem of automated diabetic retinopathy grading.

Progress of artificial intelligence in diabetic retinopathy screening

Diabetic retinopathy (DR) is one of the leading causes of blindness worldwide, and the limited availability of qualified ophthalmologists restricts its early diagnosis. For the past few years, artificial intelligence technology has developed rapidly and has been applied in DR screening. The upcoming technology provides support on DR screening and improves the identification of DR lesions with a high sensitivity and specificity. This review aims to summarize the progress on automatic detection and classification models for the diagnosis of DR.

The Role of Telemedicine, In-Home Testing and Artificial Intelligence to Alleviate an Increasingly Burdened Healthcare System: Diabetic Retinopathy

In the presence of the ever-increasing incidence of diabetes mellitus (DM), the prevalence of diabetic eye disease (DED) is also growing. Despite many improvements in diabetic care, DM remains a leading cause of visual impairment in working-age patients. So far, prevention has been the best way to protect vision. The sooner we diagnose DED, the more effective the treatment is. Thus, diabetic retinopathy (DR) screening, especially with imaging techniques, is a method of choice for vision protection. To alleviate the burden of diabetic patients who need ophthalmic care, telemedicine and in-home testing are used, supported by artificial intelligence (AI) algorithms. This is why we decided to evaluate current image teleophthalmology methods used for DR screening. We searched the PubMed platform for papers published over the last 5 years (2015–2020) using the following key words: telemedicine in diabetic retinopathy screening, diabetic retinopathy screening, automated diabetic retinopathy screening, artificial intelligence in diabetic retinopathy screening, smartphone diabetic retinopathy testing. We have included 118 original articles meeting the above criteria, discussing imaging diabetic retinopathy screening methods. We have found that fundus cameras, stable or mobile, are most commonly used for retinal photography, with portable fundus cameras also relatively common. Other possibilities involve the use of ultra-wide-field (UWF) imaging and even optical coherence tomography (OCT) devices for DR screening. Also, the role of smartphones is increasingly recognized in the field. Retinal fundus images are assessed by humans instantly or remotely, while AI algorithms seem to be useful tools facilitating retinal image assessment. The common use of smartphones and availability of relatively cheap, easy-to-use adapters for retinal photographs augmented by AI algorithms make it possible for eye fundus photographs to be taken by non-specialists and in non-medical setting. This opens the way for in-home testing conducted on a much larger scale in the future. In conclusion, based on current DR screening techniques, we can suggest that the future practice of eye care specialists will be widely supported by AI algorithms, and this way will be more effective.

Review of Various Tasks Performed in the Preprocessing Phase of a Diabetic Retinopathy Diagnosis System

Diabetic Retinopathy (DR) is a major cause of blindness in diabetic patients. The increasing population of diabetic patients and difficulty to diagnose it at an early stage are limiting the screening capabilities of manual diagnosis by ophthalmologists. Color fundus images are widely used to detect DR lesions due to their comfortable, cost-effective and non-invasive acquisition procedure. Computer Aided Diagnosis (CAD) of DR based on these images can assist ophthalmologists and help in saving many sight years of diabetic patients. In a CAD system, preprocessing is a crucial phase, which significantly affects its performance. Commonly used preprocessing operations are the enhancement of poor contrast, balancing the illumination imbalance due to the spherical shape of a retina, noise reduction, image resizing to support multi-resolution, color normalization, extraction of a field of view (FOV), etc. Also, the presence of blood vessels and optic discs makes the lesion detection more challenging because these two artifacts exhibit specific attributes, which are similar to those of DR lesions. Preprocessing operations can be broadly divided into three categories: 1) fixing the native defects, 2) segmentation of blood vessels, and 3) localization and segmentation of optic discs. This paper presents a review of the state-of-the-art preprocessing techniques related to three categories of operations, highlighting their significant aspects and limitations. The survey is concluded with the most effective preprocessing methods, which have been shown to improve the accuracy and efficiency of the CAD systems.

Hemorrhage Detection Based on 3D CNN Deep Learning Framework and Feature Fusion for Evaluating Retinal Abnormality in Diabetic Patients

Diabetic retinopathy (DR) is the main cause of blindness in diabetic patients. Early and accurate diagnosis can improve the analysis and prognosis of the disease. One of the earliest symptoms of DR are the hemorrhages in the retina. Therefore, we propose a new method for accurate hemorrhage detection from the retinal fundus images. First, the proposed method uses the modified contrast enhancement method to improve the edge details from the input retinal fundus images. In the second stage, a new convolutional neural network (CNN) architecture is proposed to detect hemorrhages. A modified pre-trained CNN model is used to extract features from the detected hemorrhages. In the third stage, all extracted feature vectors are fused using the convolutional sparse image decomposition method, and finally, the best features are selected by using the multi-logistic regression controlled entropy variance approach. The proposed method is evaluated on 1509 images from HRF, DRIVE, STARE, MESSIDOR, DIARETDB0, and DIARETDB1 databases and achieves the average accuracy of 97.71%, which is superior to the previous works. Moreover, the proposed hemorrhage detection system attains better performance, in terms of visual quality and quantitative analysis with high accuracy, in comparison with the state-of-the-art methods.

Detection of microaneurysms and hemorrhages based on improved Hessian matrix

PURPOSE

Knowing the early lesion detection of fundus images is very important to prevent blindness, and accurate lesion segmentation can provide doctors with diagnostic evidence. This study proposes a method based on improved Hessian matrix eigenvalue analysis to detect microaneurysms and hemorrhages in the fundus images of diabetic patients.

METHODS

A two-step method including identification of lesion candidate regions and classification of candidate regions is adopted. In the first step, the method of eigenvalue analysis based on the improved hessian matrix was applied to enhance the image preprocessed. A dual-threshold method was used for segmentation. Then, blood vessels were gradually removed to obtain the lesion candidate regions. In the second step, all candidates were classified into three categories: microaneurysms, hemorrhages and the others.

RESULTS

The proposed method has achieved a better performance compared with the existing algorithms on accuracy rates. The classification accuracy rates of microaneurysms and hemorrhages obtained by using our method were 94.4% and 94.0%, respectively, while the classification accuracy rates obtained by using Frangi's filter based on the Hessian matrix to enhance the image were 90.9% and 92.1%.

CONCLUSION

This study demonstrated a methodology for enhancing images by using eigenvalue analysis based on the improved Hessian matrix and segmentation by using double thresholds. The proposed method is beneficial to improve the detection accuracy of microaneurysms and hemorrhages in fundus images.

Deep Learning-Based Diabetic Retinopathy Severity Grading System Employing Quadrant Ensemble Model

The diabetic retinopathy accounts in the deterioration of retinal blood vessels leading to a serious compilation affecting the eyes. The automated DR diagnosis frameworks are critically important for the early identification and detection of these eye-related problems, helping the ophthalmic experts in providing the second opinion for effectual treatment. The deep learning techniques have evolved as an improvement over the conventional approaches, which are dependent on the handcrafted feature extraction. To address the issue of proficient DR discrimination, the authors have proposed a quadrant ensemble automated DR grading approach by implementing InceptionResnet-V2 deep neural network framework. The presented model incorporates histogram equalization, optical disc localization, and quadrant cropping along with the data augmentation step for improving the network performance. A superior accuracy performance of 93.33% is observed for the proposed framework, and a significant reduction of 0.325 is noticed in the cross-entropy loss function for MESSIDOR benchmark dataset; however, its validation utilizing the latest IDRiD dataset establishes its generalization ability. The accuracy improvement of 13.58% is observed when the proposed QEIRV-2 model is compared with the classical Inception-V3 CNN model. To justify the viability of the proposed framework, its performance is compared with the existing state-of-the-art approaches and 25.23% of accuracy improvement is observed.

A Benchmark for Studying Diabetic Retinopathy: Segmentation, Grading, and Transferability

People with diabetes are at risk of developing an eye disease called diabetic retinopathy (DR). This disease occurs when high blood glucose levels cause damage to blood vessels in the retina. Computer-aided DR diagnosis has become a promising tool for the early detection and severity grading of DR, due to the great success of deep learning. However, most current DR diagnosis systems do not achieve satisfactory performance or interpretability for ophthalmologists, due to the lack of training data with consistent and fine-grained annotations. To address this problem, we construct a large fine-grained annotated DR dataset containing 2,842 images (FGADR). Specifically, this dataset has 1,842 images with pixel-level DR-related lesion annotations, and 1,000 images with image-level labels graded by six board-certified ophthalmologists with intra-rater consistency. The proposed dataset will enable extensive studies on DR diagnosis. Further, we establish three benchmark tasks for evaluation: 1. DR lesion segmentation; 2. DR grading by joint classification and segmentation; 3. Transfer learning for ocular multi-disease identification. Moreover, a novel inductive transfer learning method is introduced for the third task. Extensive experiments using different state-of-the-art methods are conducted on our FGADR dataset, which can serve as baselines for future research. Our dataset will be released in https://csyizhou.github.io/FGADR/.

CABNet: Category Attention Block for Imbalanced Diabetic Retinopathy Grading

Diabetic Retinopathy (DR) grading is challenging due to the presence of intra-class variations, small lesions and imbalanced data distributions. The key for solving fine-grained DR grading is to find more discriminative features corresponding to subtle visual differences, such as microaneurysms, hemorrhages and soft exudates. However, small lesions are quite difficult to identify using traditional convolutional neural networks (CNNs), and an imbalanced DR data distribution will cause the model to pay too much attention to DR grades with more samples, greatly affecting the final grading performance. In this article, we focus on developing an attention module to address these issues. Specifically, for imbalanced DR data distributions, we propose a novel Category Attention Block (CAB), which explores more discriminative region-wise features for each DR grade and treats each category equally. In order to capture more detailed small lesion information, we also propose the Global Attention Block (GAB), which can exploit detailed and class-agnostic global attention feature maps for fundus images. By aggregating the attention blocks with a backbone network, the CABNet is constructed for DR grading. The attention blocks can be applied to a wide range of backbone networks and trained efficiently in an end-to-end manner. Comprehensive experiments are conducted on three publicly available datasets, showing that CABNet produces significant performance improvements for existing state-of-the-art deep architectures with few additional parameters and achieves the state-of-the-art results for DR grading. Code and models will be available at https://github.com/he2016012996/CABnet.