Multi-retinal disease classification by reduced deep learning features

Automatic Detection of 30 Fundus Diseases Using Ultra-Widefield Fluorescein Angiography with Deep Experts Aggregation

INTRODUCTION

Inaccurate, untimely diagnoses of fundus diseases leads to vision-threatening complications and even blindness. We built a deep learning platform (DLP) for automatic detection of 30 fundus diseases using ultra-widefield fluorescein angiography (UWFFA) with deep experts aggregation.

METHODS

This retrospective and cross-sectional database study included a total of 61,609 UWFFA images dating from 2016 to 2021, involving more than 3364 subjects in multiple centers across China. All subjects were divided into 30 different groups. The state-of-the-art convolutional neural network architecture, ConvNeXt, was chosen as the backbone to train and test the receiver operating characteristic curve (ROC) of the proposed system on test data and external test date. We compared the classification performance of the proposed system with that of ophthalmologists, including two retinal specialists.

RESULTS

We built a DLP to analyze UWFFA, which can detect up to 30 fundus diseases, with a frequency-weighted average area under the receiver operating characteristic curve (AUC) of 0.940 in the primary test dataset and 0.954 in the external multi-hospital test dataset. The tool shows comparable accuracy with retina specialists in diagnosis and evaluation.

CONCLUSIONS

This is the first study on a large-scale UWFFA dataset for multi-retina disease classification. We believe that our UWFFA DLP advances the diagnosis by artificial intelligence (AI) in various retinal diseases and would contribute to labor-saving and precision medicine especially in remote areas.

A multidomain bio-inspired feature extraction and selection model for diabetic retinopathy severity classification: an ensemble learning approach

Diabetes retinopathy (DR) is one of the leading causes of blindness globally. Early detection of this condition is essential for preventing patients' loss of eyesight caused by diabetes mellitus being untreated for an extended period. This paper proposes the design of an augmented bioinspired multidomain feature extraction and selection model for diabetic retinopathy severity estimation using an ensemble learning process. The proposed approach initiates by identifying DR severity levels from retinal images that segment the optical disc, macula, blood vessels, exudates, and hemorrhages using an adaptive thresholding process. Once the images are segmented, multidomain features are extracted from the retinal images, including frequency, entropy, cosine, gabor, and wavelet components. These data were fed into a novel Modified Moth Flame Optimization-based feature selection method that assisted in optimal feature selection. Finally, an ensemble model using various ML (machine learning) algorithms, which included Naive Bayes, K-Nearest Neighbours, Support Vector Machine, Multilayer Perceptron, Random Forests, and Logistic Regression were used to identify the various severity complications of DR. The experiments on different openly accessible data sources have shown that the proposed method outperformed conventional methods and achieved an Accuracy of 96.5% in identifying DR severity levels.

Blockchain for deep learning: review and open challenges

Deep learning has gained huge traction in recent years because of its potential to make informed decisions. A large portion of today's deep learning systems are based on centralized servers and fall short in providing operational transparency, traceability, reliability, security, and trusted data provenance features. Also, training deep learning models by utilizing centralized data is vulnerable to the single point of failure problem. In this paper, we explore the importance of integrating blockchain technology with deep learning. We review the existing literature focused on the integration of blockchain with deep learning. We classify and categorize the literature by devising a thematic taxonomy based on seven parameters; namely, blockchain type, deep learning models, deep learning specific consensus protocols, application area, services, data types, and deployment goals. We provide insightful discussions on the state-of-the-art blockchain-based deep learning frameworks by highlighting their strengths and weaknesses. Furthermore, we compare the existing blockchain-based deep learning frameworks based on four parameters such as blockchain type, consensus protocol, deep learning method, and dataset. Finally, we present important research challenges which need to be addressed to develop highly trustworthy deep learning frameworks.

Deep Learning for Diabetic Retinopathy Analysis: A Review, Research Challenges, and Future Directions

Deep learning (DL) enables the creation of computational models comprising multiple processing layers that learn data representations at multiple levels of abstraction. In the recent past, the use of deep learning has been proliferating, yielding promising results in applications across a growing number of fields, most notably in image processing, medical image analysis, data analysis, and bioinformatics. DL algorithms have also had a significant positive impact through yielding improvements in screening, recognition, segmentation, prediction, and classification applications across different domains of healthcare, such as those concerning the abdomen, cardiac, pathology, and retina. Given the extensive body of recent scientific contributions in this discipline, a comprehensive review of deep learning developments in the domain of diabetic retinopathy (DR) analysis, viz., screening, segmentation, prediction, classification, and validation, is presented here. A critical analysis of the relevant reported techniques is carried out, and the associated advantages and limitations highlighted, culminating in the identification of research gaps and future challenges that help to inform the research community to develop more efficient, robust, and accurate DL models for the various challenges in the monitoring and diagnosis of DR.

Automated Diabetic Retinopathy Detection Using Horizontal and Vertical Patch Division-Based Pre-Trained DenseNET with Digital Fundus Images

Diabetic retinopathy (DR) is a common complication of diabetes that can lead to progressive vision loss. Regular surveillance with fundal photography, early diagnosis, and prompt intervention are paramount to reducing the incidence of DR-induced vision loss. However, manual interpretation of fundal photographs is subject to human error. In this study, a new method based on horizontal and vertical patch division was proposed for the automated classification of DR images on fundal photographs. The novel sides of this study are given as follows. We proposed a new non-fixed-size patch division model to obtain high classification results and collected a new fundus image dataset. Moreover, two datasets are used to test the model: a newly collected three-class (normal, non-proliferative DR, and proliferative DR) dataset comprising 2355 DR images and the established open-access five-class Asia Pacific Tele-Ophthalmology Society (APTOS) 2019 dataset comprising 3662 images. Two analysis scenarios, Case 1 and Case 2, with three (normal, non-proliferative DR, and proliferative DR) and five classes (normal, mild DR, moderate DR, severe DR, and proliferative DR), respectively, were derived from the APTOS 2019 dataset. These datasets and these cases have been used to demonstrate the general classification performance of our proposal. By applying transfer learning, the last fully connected and global average pooling layers of the DenseNet201 architecture were used to extract deep features from input DR images and each of the eight subdivided horizontal and vertical patches. The most discriminative features are then selected using neighborhood component analysis. These were fed as input to a standard shallow cubic support vector machine for classification. Our new DR dataset obtained 94.06% and 91.55% accuracy values for three-class classification with 80:20 hold-out validation and 10-fold cross-validation, respectively. As can be seen from steps of the proposed model, a new patch-based deep-feature engineering model has been proposed. The proposed deep-feature engineering model is a cognitive model, since it uses efficient methods in each phase. Similar excellent results were seen for three-class classification with the Case 1 dataset. In addition, the model attained 87.43% and 84.90% five-class classification accuracy rates using 80:20 hold-out validation and 10-fold cross-validation, respectively, on the Case 2 dataset, which outperformed prior DR classification studies based on the five-class APTOS 2019 dataset. Our model attained about >2% classification results compared to others. These findings demonstrate the accuracy and robustness of the proposed model for classification of DR images.

PADAr: physician-oriented artificial intelligence-facilitating diagnosis aid for retinal diseases

Purpose: Retinopathy screening via digital imaging is promising for early detection and timely treatment, and tracking retinopathic abnormality over time can help to reveal the risk of disease progression. We developed an innovative physician-oriented artificial intelligence-facilitating diagnosis aid system for retinal diseases for screening multiple retinopathies and monitoring the regions of potential abnormality over time. Approach: Our dataset contains 4908 fundus images from 304 eyes with image-level annotations, including diabetic retinopathy, age-related macular degeneration, cellophane maculopathy, pathological myopia, and healthy control (HC). The screening model utilized a VGG-based feature extractor and multiple-binary convolutional neural network-based classifiers. Images in time series were aligned via affine transforms estimated through speeded-up robust features. Heatmaps of retinopathy were generated from the feature extractor using gradient-weighted class activation mapping++, and individual candidate retinopathy sites were identified from the heatmaps using clustering algorithm. Nested cross-validation with a train-to-test split of 80% to 20% was used to evaluate the performance of the screening model. Results: Our screening model achieved 99% accuracy, 93% sensitivity, and 97% specificity in discriminating between patients with retinopathy and HCs. For discriminating between types of retinopathy, our model achieved an averaged performance of 80% accuracy, 78% sensitivity, 94% specificity, 79% F1-score, and Cohen's kappa coefficient of 0.70. Moreover, visualization results were also shown to provide reasonable candidate sites of retinopathy. Conclusions: Our results demonstrated the capability of the proposed model for extracting diagnostic information of the abnormality and lesion locations, which allows clinicians to focus on patient-centered treatment and untangles the pathological plausibility hidden in deep learning models.

RetFluidNet: Retinal Fluid Segmentation for SD-OCT Images Using Convolutional Neural Network

Age-related macular degeneration (AMD) is one of the leading causes of irreversible blindness and is characterized by fluid-related accumulations such as intra-retinal fluid (IRF), subretinal fluid (SRF), and pigment epithelial detachment (PED). Spectral-domain optical coherence tomography (SD-OCT) is the primary modality used to diagnose AMD, yet it does not have algorithms that directly detect and quantify the fluid. This work presents an improved convolutional neural network (CNN)-based architecture called RetFluidNet to segment three types of fluid abnormalities from SD-OCT images. The model assimilates different skip-connect operations and atrous spatial pyramid pooling (ASPP) to integrate multi-scale contextual information; thus, achieving the best performance. This work also investigates between consequential and comparatively inconsequential hyperparameters and skip-connect techniques for fluid segmentation from the SD-OCT image to indicate the starting choice for future related researches. RetFluidNet was trained and tested on SD-OCT images from 124 patients and achieved an accuracy of 80.05%, 92.74%, and 95.53% for IRF, PED, and SRF, respectively. RetFluidNet showed significant improvement over competitive works to be clinically applicable in reasonable accuracy and time efficiency. RetFluidNet is a fully automated method that can support early detection and follow-up of AMD.

Kim P, Kim J. Retinal Healthcare Diagnosis Approaches with Deep Learning Techniques

The retina is an important organ of the human body, with a crucial function in the vision mechanism. A minor disturbance in the retina can cause various abnormalities in the eye, as well as complex retinal diseases such as diabetic retinopathy. To diagnose such diseases in early stages, many researchers are incorporating machine learning (ML) technique. The combination of medical science with ML improves the healthcare diagnosis systems of hospitals, clinics, and other providers. Recently, AI-based healthcare diagnosis systems assist clinicians in handling more patients in less time and improves diagnosis accuracy. In this paper, we review cutting-edge AI-based retinal diagnosis technologies. This article also briefly describes the potential of the latest densely connected convolutional networks (DenseNets) to improve the performance of diagnosis systems. Moreover, this paper focuses on state-of-the-art results from comprehensive investigations in retinal diagnosis and the development of AI-based retinal healthcare diagnosis approaches with deep-learning models.

Feasibility study to improve deep learning in OCT diagnosis of rare retinal diseases with few-shot classification

Deep learning (DL) has been successfully applied to the diagnosis of ophthalmic diseases. However, rare diseases are commonly neglected due to insufficient data. Here, we demonstrate that few-shot learning (FSL) using a generative adversarial network (GAN) can improve the applicability of DL in the optical coherence tomography (OCT) diagnosis of rare diseases. Four major classes with a large number of datasets and five rare disease classes with a few-shot dataset are included in this study. Before training the classifier, we constructed GAN models to generate pathological OCT images of each rare disease from normal OCT images. The Inception-v3 architecture was trained using an augmented training dataset, and the final model was validated using an independent test dataset. The synthetic images helped in the extraction of the characteristic features of each rare disease. The proposed DL model demonstrated a significant improvement in the accuracy of the OCT diagnosis of rare retinal diseases and outperformed the traditional DL models, Siamese network, and prototypical network. By increasing the accuracy of diagnosing rare retinal diseases through FSL, clinicians can avoid neglecting rare diseases with DL assistance, thereby reducing diagnosis delay and patient burden.

Survey on recent developments in automatic detection of diabetic retinopathy

Diabetic retinopathy (DR) is a disease facilitated by the rapid spread of diabetes worldwide. DR can blind diabetic individuals. Early detection of DR is essential to restoring vision and providing timely treatment. DR can be detected manually by an ophthalmologist, examining the retinal and fundus images to analyze the macula, morphological changes in blood vessels, hemorrhage, exudates, and/or microaneurysms. This is a time consuming, costly, and challenging task. An automated system can easily perform this function by using artificial intelligence, especially in screening for early DR. Recently, much state-of-the-art research relevant to the identification of DR has been reported. This article describes the current methods of detecting non-proliferative diabetic retinopathy, exudates, hemorrhage, and microaneurysms. In addition, the authors point out future directions in overcoming current challenges in the field of DR research.

Automated detection of mild and multi-class diabetic eye diseases using deep learning

Diabetic eye disease is a collection of ocular problems that affect patients with diabetes. Thus, timely screening enhances the chances of timely treatment and prevents permanent vision impairment. Retinal fundus images are a useful resource to diagnose retinal complications for ophthalmologists. However, manual detection can be laborious and time-consuming. Therefore, developing an automated diagnose system reduces the time and workload for ophthalmologists. Recently, the image classification using Deep Learning (DL) in between healthy or diseased retinal fundus image classification already achieved a state of the art performance. While the classification of mild and multi-class diseases remains an open challenge, therefore, this research aimed to build an automated classification system considering two scenarios: (i) mild multi-class diabetic eye disease (DED), and (ii) multi-class DED. Our model tested on various datasets, annotated by an opthalmologist. The experiment conducted employing the top two pretrained convolutional neural network (CNN) models on ImageNet. Furthermore, various performance improvement techniques were employed, i.e., fine-tune, optimization, and contrast enhancement. Maximum accuracy of 88.3% obtained on the VGG16 model for multi-class classification and 85.95% for mild multi-class classification.

DCT-MIL: Deep CNN transferred multiple instance learning for COPD identification using CT images

While many pre-defined computed tomographic (CT) measures have been utilized to characterize chronic obstructive pulmonary disease (COPD), it is still challenging to represent pathological alternations of multiple dimensions and highly spatial heterogeneity. Deep CNN transferred multiple instance learning (DCT-MIL) is proposed to identify COPD via CT images. After the lung is divided into eight sections along the axial direction, one random axial CT image is taken out from each section as one instance. With one instance as the input, the activations of neural layers of AlexNet trained by natural images are extracted as features. After dimension reduction through principle component analysis, features of all instances are input into three MIL methods: Citation k-Nearest-Neighbor (Citation-KNN), multiple instance support vector machine, and expectation-maximization diverse density. Moreover, the performance dependence of the resulted models on the depth of the neural layer where activations are extracted and the number of features is investigated. The proposed DCT-MIL achieves an exceptional performance with an accuracy of 99.29% and area under curve of 0.9826 while using 100 principle components of features extracted from the fourth convolutional layer and Citation-KNN. It outperforms not only DCT-MIL models using other settings and the pre-trained AlexNet with fine-tuning by montages of eight lung CT images, but also other state-of-art methods. Deep CNN transferred multiple instance learning is suited for identification of COPD using CT images. It can help finding subgroups with high risk of COPD from large populations through CT scans ordered doing lung cancer screening.

Recent Development on Detection Methods for the Diagnosis of Diabetic Retinopathy

Diabetic retinopathy (DR) is a complication of diabetes that exists throughout the world. DR occurs due to a high ratio of glucose in the blood, which causes alterations in the retinal microvasculature. Without preemptive symptoms of DR, it leads to complete vision loss. However, early screening through computer-assisted diagnosis (CAD) tools and proper treatment have the ability to control the prevalence of DR. Manual inspection of morphological changes in retinal anatomic parts are tedious and challenging tasks. Therefore, many CAD systems were developed in the past to assist ophthalmologists for observing inter- and intra-variations. In this paper, a recent review of state-of-the-art CAD systems for diagnosis of DR is presented. We describe all those CAD systems that have been developed by various computational intelligence and image processing techniques. The limitations and future trends of current CAD systems are also described in detail to help researchers. Moreover, potential CAD systems are also compared in terms of statistical parameters to quantitatively evaluate them. The comparison results indicate that there is still a need for accurate development of CAD systems to assist in the clinical diagnosis of diabetic retinopathy.

A Novel Fundus Image Reading Tool for Efficient Generation of a Multi-dimensional Categorical Image Database for Machine Learning Algorithm Training

BACKGROUND

We described a novel multi-step retinal fundus image reading system for providing high-quality large data for machine learning algorithms, and assessed the grader variability in the large-scale dataset generated with this system.

METHODS

A 5-step retinal fundus image reading tool was developed that rates image quality, presence of abnormality, findings with location information, diagnoses, and clinical significance. Each image was evaluated by 3 different graders. Agreements among graders for each decision were evaluated.

RESULTS

The 234,242 readings of 79,458 images were collected from 55 licensed ophthalmologists during 6 months. The 34,364 images were graded as abnormal by at-least one rater. Of these, all three raters agreed in 46.6% in abnormality, while 69.9% of the images were rated as abnormal by two or more raters. Agreement rate of at-least two raters on a certain finding was 26.7%-65.2%, and complete agreement rate of all-three raters was 5.7%-43.3%. As for diagnoses, agreement of at-least two raters was 35.6%-65.6%, and complete agreement rate was 11.0%-40.0%. Agreement of findings and diagnoses were higher when restricted to images with prior complete agreement on abnormality. Retinal/glaucoma specialists showed higher agreements on findings and diagnoses of their corresponding subspecialties.

CONCLUSION

This novel reading tool for retinal fundus images generated a large-scale dataset with high level of information, which can be utilized in future development of machine learning-based algorithms for automated identification of abnormal conditions and clinical decision supporting system. These results emphasize the importance of addressing grader variability in algorithm developments.

Grid-Based Crime Prediction Using Geographical Features

Machine learning is useful for grid-based crime prediction. Many previous studies have examined factors including time, space, and type of crime, but the geographic characteristics of the grid are rarely discussed, leaving prediction models unable to predict crime displacement. This study incorporates the concept of a criminal environment in grid-based crime prediction modeling, and establishes a range of spatial-temporal features based on 84 types of geographic information by applying the Google Places API to theft data for Taoyuan City, Taiwan. The best model was found to be Deep Neural Networks, which outperforms the popular Random Decision Forest, Support Vector Machine, and K-Near Neighbor algorithms. After tuning, compared to our design’s baseline 11-month moving average, the F1 score improves about 7% on 100-by-100 grids. Experiments demonstrate the importance of the geographic feature design for improving performance and explanatory ability. In addition, testing for crime displacement also shows that our model design outperforms the baseline.

Deep image mining for diabetic retinopathy screening

Deep learning is quickly becoming the leading methodology for medical image analysis. Given a large medical archive, where each image is associated with a diagnosis, efficient pathology detectors or classifiers can be trained with virtually no expert knowledge about the target pathologies. However, deep learning algorithms, including the popular ConvNets, are black boxes: little is known about the local patterns analyzed by ConvNets to make a decision at the image level. A solution is proposed in this paper to create heatmaps showing which pixels in images play a role in the image-level predictions. In other words, a ConvNet trained for image-level classification can be used to detect lesions as well. A generalization of the backpropagation method is proposed in order to train ConvNets that produce high-quality heatmaps. The proposed solution is applied to diabetic retinopathy (DR) screening in a dataset of almost 90,000 fundus photographs from the 2015 Kaggle Diabetic Retinopathy competition and a private dataset of almost 110,000 photographs (e-ophtha). For the task of detecting referable DR, very good detection performance was achieved: A_z=0.954 in Kaggle's dataset and A_z=0.949 in e-ophtha. Performance was also evaluated at the image level and at the lesion level in the DiaretDB1 dataset, where four types of lesions are manually segmented: microaneurysms, hemorrhages, exudates and cotton-wool spots. For the task of detecting images containing these four lesion types, the proposed detector, which was trained to detect referable DR, outperforms recent algorithms trained to detect those lesions specifically, with pixel-level supervision. At the lesion level, the proposed detector outperforms heatmap generation algorithms for ConvNets. This detector is part of the Messidor® system for mobile eye pathology screening. Because it does not rely on expert knowledge or manual segmentation for detecting relevant patterns, the proposed solution is a promising image mining tool, which has the potential to discover new biomarkers in images.