Clinical Report Guided Retinal Microaneurysm Detection With Multi-Sieving Deep Learning

Visual impairment prevention by early detection of diabetic retinopathy based on stacked auto-encoder

Diabetic retinopathy (DR) presents a significant concern among diabetic patients, often leading to vision impairment or blindness if left untreated. Traditional diagnosis methods are prone to human error, necessitating accurate alternatives. While various computer-aided systems have been developed to assist in DR detection, there remains a need for accurate and efficient methods to classify its stages. In this study, we propose a novel approach utilizing enhanced stacked auto-encoders for the detection and classification of DR stages. The classification is performed across one healthy (normal) stage and four DR stages: mild, moderate, severe, and proliferative. Unlike traditional CNN approaches, our method offers improved reliability by reducing time complexity, minimizing errors, and enhancing noise reduction. Leveraging a comprehensive dataset from KAGGLE containing 35,126 retinal fundus images representing one healthy (normal) stage and four DR stages, our proposed model demonstrates superior accuracy compared to existing deep learning algorithms. Data augmentation techniques address class imbalance, while SAEs facilitate accurate classification through layer-wise unsupervised pre-training and supervised fine-tuning. We evaluate our model's performance using rigorous quantitative measures, including accuracy, recall, precision, and F1-score, highlighting its effectiveness in early disease diagnosis and prevention of blindness. Experimental results across different training/testing ratios (50:50, 60:40, 70:30, and 75:25) showcase the model's robustness. The highest accuracy achieved during training was 93%, while testing accuracy reached 88% on a training/testing ratio of 75:25. Comparative analysis underscores the model's superiority over existing methods, positioning it as a promising tool for early-stage DR detection and blindness prevention.

An Organ-Aware Diagnosis Framework for Radiology Report Generation

Radiology report generation (RRG) is crucial to save the valuable time of radiologists in drafting the report, therefore increasing their work efficiency. Compared to typical methods that directly transfer image captioning technologies to RRG, our approach incorporates organ-wise priors into the report generation. Specifically, in this paper, we propose Organ-aware Diagnosis (OaD) to generate diagnostic reports containing descriptions of each physiological organ. During training, we first develop a task distillation (TD) module to extract organ-level descriptions from reports. We then introduce an organ-aware report generation module that, for one thing, provides a specific description for each organ, and for another, simulates clinical situations to provide short descriptions for normal cases. Furthermore, we design an auto-balance mask loss to ensure balanced training for normal/abnormal descriptions and various organs simultaneously. Being intuitively reasonable and practically simple, our OaD outperforms SOTA alternatives by large margins on commonly used IU-Xray and MIMIC-CXR datasets, as evidenced by a 3.4% BLEU-1 improvement on MIMIC-CXR and 2.0% BLEU-2 improvement on IU-Xray.

An attentional mechanism model for segmenting multiple lesion regions in the diabetic retina

Diabetic retinopathy (DR), a leading cause of blindness in diabetic patients, necessitates the precise segmentation of lesions for the effective grading of lesions. DR multi-lesion segmentation faces the main concerns as follows. On the one hand, retinal lesions vary in location, shape, and size. On the other hand, the currently available multi-lesion region segmentation models are insufficient in their extraction of minute features and are prone to overlooking microaneurysms. To solve the above problems, we propose a novel deep learning method: the Multi-Scale Spatial Attention Gate (MSAG) mechanism network. The model inputs images of varying scales in order to extract a range of semantic information. Our innovative Spatial Attention Gate merges low-level spatial details with high-level semantic content, assigning hierarchical attention weights for accurate segmentation. The incorporation of the modified spatial attention gate in the inference stage enhances precision by combining prediction scales hierarchically, thereby improving segmentation accuracy without increasing the associated training costs. We conduct the experiments on the public datasets IDRiD and DDR, and the experimental results show that the proposed method achieves better performance than other methods.

Medical-informed machine learning: integrating prior knowledge into medical decision systems

BACKGROUND

Clinical medicine offers a promising arena for applying Machine Learning (ML) models. However, despite numerous studies employing ML in medical data analysis, only a fraction have impacted clinical care. This article underscores the importance of utilising ML in medical data analysis, recognising that ML alone may not adequately capture the full complexity of clinical data, thereby advocating for the integration of medical domain knowledge in ML.

METHODS

The study conducts a comprehensive review of prior efforts in integrating medical knowledge into ML and maps these integration strategies onto the phases of the ML pipeline, encompassing data pre-processing, feature engineering, model training, and output evaluation. The study further explores the significance and impact of such integration through a case study on diabetes prediction. Here, clinical knowledge, encompassing rules, causal networks, intervals, and formulas, is integrated at each stage of the ML pipeline, resulting in a spectrum of integrated models.

RESULTS

The findings highlight the benefits of integration in terms of accuracy, interpretability, data efficiency, and adherence to clinical guidelines. In several cases, integrated models outperformed purely data-driven approaches, underscoring the potential for domain knowledge to enhance ML models through improved generalisation. In other cases, the integration was instrumental in enhancing model interpretability and ensuring conformity with established clinical guidelines. Notably, knowledge integration also proved effective in maintaining performance under limited data scenarios.

CONCLUSIONS

By illustrating various integration strategies through a clinical case study, this work provides guidance to inspire and facilitate future integration efforts. Furthermore, the study identifies the need to refine domain knowledge representation and fine-tune its contribution to the ML model as the two main challenges to integration and aims to stimulate further research in this direction.

Synthesizing multi-frame high-resolution fluorescein angiography images from retinal fundus images using generative adversarial networks

BACKGROUND

Fundus fluorescein angiography (FA) can be used to diagnose fundus diseases by observing dynamic fluorescein changes that reflect vascular circulation in the fundus. As FA may pose a risk to patients, generative adversarial networks have been used to convert retinal fundus images into fluorescein angiography images. However, the available methods focus on generating FA images of a single phase, and the resolution of the generated FA images is low, being unsuitable for accurately diagnosing fundus diseases.

METHODS

We propose a network that generates multi-frame high-resolution FA images. This network consists of a low-resolution GAN (LrGAN) and a high-resolution GAN (HrGAN), where LrGAN generates low-resolution and full-size FA images with global intensity information, HrGAN takes the FA images generated by LrGAN as input to generate multi-frame high-resolution FA patches. Finally, the FA patches are merged into full-size FA images.

RESULTS

Our approach combines supervised and unsupervised learning methods and achieves better quantitative and qualitative results than using either method alone. Structural similarity (SSIM), normalized cross-correlation (NCC) and peak signal-to-noise ratio (PSNR) were used as quantitative metrics to evaluate the performance of the proposed method. The experimental results show that our method achieves better quantitative results with structural similarity of 0.7126, normalized cross-correlation of 0.6799, and peak signal-to-noise ratio of 15.77. In addition, ablation experiments also demonstrate that using a shared encoder and residual channel attention module in HrGAN is helpful for the generation of high-resolution images.

CONCLUSIONS

Overall, our method has higher performance for generating retinal vessel details and leaky structures in multiple critical phases, showing a promising clinical diagnostic value.

Defect Severity Identification for a Catenary System Based on Deep Semantic Learning

A variety of Chinese textual operational text data has been recorded during the operation and maintenance of the high-speed railway catenary system. Such defect text records can facilitate defect detection and defect severity analysis if mined efficiently and accurately. Therefore, in this context, this paper focuses on a specific problem in defect text mining, which is to efficiently extract defect-relevant information from catenary defect text records and automatically identify catenary defect severity. The specific task is transformed into a machine learning problem for defect text classification. First, we summarize the characteristics of catenary defect texts and construct a text dataset. Second, we use BERT to learn defect texts and generate word embedding vectors with contextual features, fed into the classification model. Third, we developed a deep text categorization network (DTCN) to distinguish the catenary defect level, considering the contextualized semantic features. Finally, the effectiveness of our proposed method (BERT-DTCN) is validated using a catenary defect textual dataset collected from 2016 to 2018 in the China Railway Administration in Chengdu, Lanzhou, and Hengshui. Moreover, BERT-DTCN outperforms several competitive methods in terms of accuracy, precision, recall, and F1-score value.

Nested U-Net for Segmentation of Red Lesions in Retinal Fundus Images and Sub-image Classification for Removal of False Positives

Diabetic retinopathy is a pathological change of the retina that occurs for long-term diabetes. The patients become symptomatic in advanced stages of diabetic retinopathy resulting in severe non-proliferative diabetic retinopathy or proliferative diabetic retinopathy stages. There is a need of an automated screening tool for the early detection and treatment of patients with diabetic retinopathy. This paper focuses on the segmentation of red lesions using nested U-Net Zhou et al. (Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer, 2018) followed by removal of false positives based on the sub-image classification method. Different sizes of sub-images were studied for the reduction in false positives in the sub-image classification method. The network could capture semantic features and fine details due to dense convolutional blocks connected via skip connections in between down sampling and up sampling paths. False-negative candidates were very few and the sub-image classification network effectively reduced the falsely detected candidates. The proposed framework achieves a sensitivity of [Formula: see text], precision of [Formula: see text], and F1-Score of [Formula: see text] for the DIARETDB1 data set Kalviainen and Uusutalo (Medical Image Understanding and Analysis, Citeseer, 2007). It outperforms the state-of-the-art networks such as U-Net Ronneberger et al. (International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2015) and attention U-Net Oktay et al. (Attention u-net: Learning where to look for the pancreas, 2018).

A critical review on diagnosis of diabetic retinopathy using machine learning and deep learning

Diabetic Retinopathy (DR) is a health condition caused due to Diabetes Mellitus (DM). It causes vision problems and blindness due to disfigurement of human retina. According to statistics, 80% of diabetes patients battling from long diabetic period of 15 to 20 years, suffer from DR. Hence, it has become a dangerous threat to the health and life of people. To overcome DR, manual diagnosis of the disease is feasible but overwhelming and cumbersome at the same time and hence requires a revolutionary method. Thus, such a health condition necessitates primary recognition and diagnosis to prevent DR from developing into severe stages and prevent blindness. Innumerable Machine Learning (ML) models are proposed by researchers across the globe, to achieve this purpose. Various feature extraction techniques are proposed for extraction of DR features for early detection. However, traditional ML models have shown either meagre generalization throughout feature extraction and classification for deploying smaller datasets or consumes more of training time causing inefficiency in prediction while using larger datasets. Hence Deep Learning (DL), a new domain of ML, is introduced. DL models can handle a smaller dataset with help of efficient data processing techniques. However, they generally incorporate larger datasets for their deep architectures to enhance performance in feature extraction and image classification. This paper gives a detailed review on DR, its features, causes, ML models, state-of-the-art DL models, challenges, comparisons and future directions, for early detection of DR.

A new detection model of microaneurysms based on improved FC-DenseNet

Diabetic retinopathy (DR) is a frequent vascular complication of diabetes mellitus and remains a leading cause of vision loss worldwide. Microaneurysm (MA) is usually the first symptom of DR that leads to blood leakage in the retina. Periodic detection of MAs will facilitate early detection of DR and reduction of vision injury. In this study, we proposed a novel model for the detection of MAs in fluorescein fundus angiography (FFA) images based on the improved FC-DenseNet, MAs-FC-DenseNet. FFA images were pre-processed by the Histogram Stretching and Gaussian Filtering algorithm to improve the quality of FFA images. Then, MA regions were detected by the improved FC-DenseNet. MAs-FC-DenseNet was compared against other FC-DenseNet models (FC-DenseNet56 and FC-DenseNet67) or the end-to-end models (DeeplabV3+ and PSPNet) to evaluate the detection performance of MAs. The result suggested that MAs-FC-DenseNet had higher values of evaluation metrics than other models, including pixel accuracy (PA), mean pixel accuracy (MPA), precision (Pre), recall (Re), F1-score (F1), and mean intersection over union (MIoU). Moreover, MA detection performance for MAs-FC-DenseNet was very close to the ground truth. Taken together, MAs-FC-DenseNet is a reliable model for rapid and accurate detection of MAs, which would be used for mass screening of DR patients.

MICaps: Multi-instance capsule network for machine inspection of Munro's microabscess

Munro's Microabscess (MM) is the diagnostic hallmark of psoriasis. Neutrophil detection in the Stratum Corneum (SC) of the skin epidermis is an integral part of MM detection in skin biopsy. The microscopic inspection of skin biopsy is a tedious task and staining variations in skin histopathology often hinder human performance to differentiate neutrophils from skin keratinocytes. Motivated from this, we propose a computational framework that can assist human experts and reduce potential errors in diagnosis. The framework first segments the SC layer, and multiple patches are sampled from the segmented regions which are classified to detect neutrophils. Both UNet and CapsNet are used for segmentation and classification. Experiments show that of the two choices, CapsNet, owing to its robustness towards better hierarchical object representation and localisation ability, appears as a better candidate for both segmentation and classification tasks and hence, we termed our framework as MICaps. The training algorithm explores both minimisation of Dice Loss and Focal Loss and makes a comparative study between the two. The proposed framework is validated with our in-house dataset consisting of 290 skin biopsy images. Two different experiments are considered. Under the first protocol, only 3-fold cross-validation is done to directly compare the current results with the state-of-the-art ones. Next, the performance of the system on a held-out data set is reported. The experimental results show that MICaps improves the state-of-the-art diagnosis performance by 3.27% (maximum) and reduces the number of model parameters by 50%.

Review on diabetic retinopathy with deep learning methods

Purpose: The purpose of our review paper is to examine many existing works of literature presenting the different methods utilized for diabetic retinopathy (DR) recognition employing deep learning (DL) and machine learning (ML) techniques, and also to address the difficulties faced in various datasets used by DR. Approach: DR is a progressive illness and may become a reason for vision loss. Early identification of DR lesions is, therefore, helpful and prevents damage to the retina. However, it is a complex job in view of the fact that it is symptomless earlier, and also ophthalmologists have been needed in traditional approaches. Recently, automated identification of DR-based studies has been stated based on image processing, ML, and DL. We analyze the recent literature and provide a comparative study that also includes the limitations of the literature and future work directions. Results: A relative analysis among the databases used, performance metrics employed, and ML and DL techniques adopted recently in DR detection based on various DR features is presented. Conclusion: Our review paper discusses the methods employed in DR detection along with the technical and clinical challenges that are encountered, which is missing in existing reviews, as well as future scopes to assist researchers in the field of retinal imaging.

Using Metabolic and Biochemical Indicators to Predict Diabetic Retinopathy by Back-Propagation Artificial Neural Network

PURPOSE

Timely diagnosis of diabetic retinopathy (DR) can significantly improve the prognosis of patients. In this study, we established a prediction model by analyzing the relationship between diabetic retinopathy and related metabolic and biochemical indicators.

METHODS

A total of 427 type 2 diabetes mellitus (T2DM) patients were selected from the datadryad website data. Logistic regression (MLR) was used to input layer variables of the model were screened. Then, Tan-Sigmoid was selected as the transfer function of the hidden layer node, and the linear function was used as the output layer function to establish the back propagation artificial neural network (BP-ANN) model. The model was applied to 183 patients with type 2 diabetes mellitus (T2DM) in our hospital to predict DR.

RESULTS

A total of 167 patients (39.2%) with DR were obtained from the Datadryad database. Input variables were screened by MLR model, and it was concluded that the age, sex, albumin and creatinine, diabetes course were independently associated with the occurrence of DR. The above variables were used to establish BP-ANN model. The area under receiver operating characteristic curve (AUC) was significantly higher than that of MLR model (0.88 vs 0.74, P<0.05), the probability threshold of the model was 0.3. Type 2 diabetes mellitus (T2DM) were selected in our hospital, including 92 patients with DR (50.2%). The above BP-ANN model was used to predict the incidence of DR, and the AUC area was significantly higher than that of the MLR model (0.77 vs 0.70, P<0.05), the probability threshold was 0.7.

CONCLUSION

We established the BP-ANN model and applied it to diagnose DR. Taking diabetic course, age, sex, albumin and creatinine as the inputs of BP-ANN, the existence of DR could be well predicted. Meanwhile, the generalization ability of the model could be improved by selecting different probability thresholds in different ROC curves.

EAD-Net: A Novel Lesion Segmentation Method in Diabetic Retinopathy Using Neural Networks

Diabetic retinopathy (DR) is a common chronic fundus disease, which has four different kinds of microvessel structure and microvascular lesions: microaneurysms (MAs), hemorrhages (HEs), hard exudates, and soft exudates. Accurate detection and counting of them are a basic but important work. The manual annotation of these lesions is a labor-intensive task in clinical analysis. To solve the problem, we proposed a novel segmentation method for different lesions in DR. Our method is based on a convolutional neural network and can be divided into encoder module, attention module, and decoder module, so we refer it as EAD-Net. After normalization and augmentation, the fundus images were sent to the EAD-Net for automated feature extraction and pixel-wise label prediction. Given the evaluation metrics based on the matching degree between detected candidates and ground truth lesions, our method achieved sensitivity of 92.77%, specificity of 99.98%, and accuracy of 99.97% on the e_ophtha_EX dataset and comparable AUPR (Area under Precision-Recall curve) scores on IDRiD dataset. Moreover, the results on the local dataset also show that our EAD-Net has better performance than original U-net in most metrics, especially in the sensitivity and F1-score, with nearly ten percent improvement. The proposed EAD-Net is a novel method based on clinical DR diagnosis. It has satisfactory results on the segmentation of four different kinds of lesions. These effective segmentations have important clinical significance in the monitoring and diagnosis of DR.

Automated Detection and Diagnosis of Diabetic Retinopathy: A Comprehensive Survey

Diabetic Retinopathy (DR) is a leading cause of vision loss in the world. In the past few years, artificial intelligence (AI) based approaches have been used to detect and grade DR. Early detection enables appropriate treatment and thus prevents vision loss. For this purpose, both fundus and optical coherence tomography (OCT) images are used to image the retina. Next, Deep-learning (DL)-/machine-learning (ML)-based approaches make it possible to extract features from the images and to detect the presence of DR, grade its severity and segment associated lesions. This review covers the literature dealing with AI approaches to DR such as ML and DL in classification and segmentation that have been published in the open literature within six years (2016-2021). In addition, a comprehensive list of available DR datasets is reported. This list was constructed using both the PICO (P-Patient, I-Intervention, C-Control, O-Outcome) and Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) 2009 search strategies. We summarize a total of 114 published articles which conformed to the scope of the review. In addition, a list of 43 major datasets is presented.

Retinal vascular geometry detection as a biomarker in diabetes mellitus

OBJECTIVE

To compare the vessel geometry characteristics of color fundus photographs in normal control and diabetes mellitus (DM) patients and to find potential biomarkers for early diabetic retinopathy (DR) based on a neural network vessel segmentation system and automated vascular geometry parameter analysis software.

METHODS

A total of 102 consecutive patients with type 2 DM (T2DM) and 132 healthy controls were recruited. All participants underwent general ophthalmic examinations, and retinal fundus photographs were taken with a digital fundus camera without mydriasis. Color fundus photographs were input into a dense-block generative adversarial network (D-GAN)-assisted retinal vascular segmentation system (http://www.gdcerc.cn:8081/#/login) to obtain binary images. These images were then analyzed by customized software (ocular microvascular analysis system V2.9.1) for automatic processing of vessel geometry parameters, including the monofractal dimension (D_box), multifractal dimension (D₀), vessel area ratio (R), max vessel diameter (d_max), average vessel diameter (d_ave), arc-chord ratio (A/C), and tortuosity (τ_n). Geometric differences between the healthy subjects and DM patients were analyzed. Then, regression analysis and receiver operating characteristic (ROC) curve analysis were performed to evaluate the diagnostic efficiency of the vascular geometry parameters.

RESULTS

No significant differences were observed between the baseline characteristics of each group. DM patients had lower D_box and D₀ values (1.330 ± 0.041; 1.347 ± 0.038) than healthy subjects (1.343 ± 0.048, p < 0.05; 1.362 ± 0.042, p < 0.05) and showed increasing values of d_max, d_ave, A/C, and τ_n compared with normal controls, although only the differences in d_ave and τ_n between the groups were statistically significant. In the regression analysis, d_ave and τ_n showed a good correlation with diabetes (d_ave, OR 1.765, 95% CI 1.319-2.362, p < 0.001; τ_n, OR 9.323, 95% CI 1.492-58.262, p < 0.05).

CONCLUSIONS

We demonstrated the relationship between retinal vascular geometry and the process in DM patients, showing that D_box, D₀, d_ave, and τ_n may be indicators of morphological changes in retinal vessels in DM patients and can be early biomarkers of DR.

Progress of artificial intelligence in diabetic retinopathy screening

Diabetic retinopathy (DR) is one of the leading causes of blindness worldwide, and the limited availability of qualified ophthalmologists restricts its early diagnosis. For the past few years, artificial intelligence technology has developed rapidly and has been applied in DR screening. The upcoming technology provides support on DR screening and improves the identification of DR lesions with a high sensitivity and specificity. This review aims to summarize the progress on automatic detection and classification models for the diagnosis of DR.

Deep learning-based detection and stage grading for optimising diagnosis of diabetic retinopathy

AIMS

To establish an automated method for identifying referable diabetic retinopathy (DR), defined as moderate nonproliferative DR and above, using deep learning-based lesion detection and stage grading.

MATERIALS AND METHODS

A set of 12,252 eligible fundus images of diabetic patients were manually annotated by 45 licenced ophthalmologists and were randomly split into training, validation, and internal test sets (ratio of 7:1:2). Another set of 565 eligible consecutive clinical fundus images was established as an external test set. For automated referable DR identification, four deep learning models were programmed based on whether two factors were included: DR-related lesions and DR stages. Sensitivity, specificity and the area under the receiver operating characteristic curve (AUC) were reported for referable DR identification, while precision and recall were reported for lesion detection.

RESULTS

Adding lesion information to the five-stage grading model improved the AUC (0.943 vs. 0.938), sensitivity (90.6% vs. 90.5%) and specificity (80.7% vs. 78.5%) of the model for identifying referable DR in the internal test set. Adding stage information to the lesion-based model increased the AUC (0.943 vs. 0.936) and sensitivity (90.6% vs. 76.7%) of the model for identifying referable DR in the internal test set. Similar trends were also seen in the external test set. DR lesion types with high precision results were preretinal haemorrhage, hard exudate, vitreous haemorrhage, neovascularisation, cotton wool spots and fibrous proliferation.

CONCLUSIONS

The herein described automated model employed DR lesions and stage information to identify referable DR and displayed better diagnostic value than models built without this information.

Intelligent Health Care: Applications of Deep Learning in Computational Medicine

With the progress of medical technology, biomedical field ushered in the era of big data, based on which and driven by artificial intelligence technology, computational medicine has emerged. People need to extract the effective information contained in these big biomedical data to promote the development of precision medicine. Traditionally, the machine learning methods are used to dig out biomedical data to find the features from data, which generally rely on feature engineering and domain knowledge of experts, requiring tremendous time and human resources. Different from traditional approaches, deep learning, as a cutting-edge machine learning branch, can automatically learn complex and robust feature from raw data without the need for feature engineering. The applications of deep learning in medical image, electronic health record, genomics, and drug development are studied, where the suggestion is that deep learning has obvious advantage in making full use of biomedical data and improving medical health level. Deep learning plays an increasingly important role in the field of medical health and has a broad prospect of application. However, the problems and challenges of deep learning in computational medical health still exist, including insufficient data, interpretability, data privacy, and heterogeneity. Analysis and discussion on these problems provide a reference to improve the application of deep learning in medical health.

Deep Learning-Based Diabetic Retinopathy Severity Grading System Employing Quadrant Ensemble Model

The diabetic retinopathy accounts in the deterioration of retinal blood vessels leading to a serious compilation affecting the eyes. The automated DR diagnosis frameworks are critically important for the early identification and detection of these eye-related problems, helping the ophthalmic experts in providing the second opinion for effectual treatment. The deep learning techniques have evolved as an improvement over the conventional approaches, which are dependent on the handcrafted feature extraction. To address the issue of proficient DR discrimination, the authors have proposed a quadrant ensemble automated DR grading approach by implementing InceptionResnet-V2 deep neural network framework. The presented model incorporates histogram equalization, optical disc localization, and quadrant cropping along with the data augmentation step for improving the network performance. A superior accuracy performance of 93.33% is observed for the proposed framework, and a significant reduction of 0.325 is noticed in the cross-entropy loss function for MESSIDOR benchmark dataset; however, its validation utilizing the latest IDRiD dataset establishes its generalization ability. The accuracy improvement of 13.58% is observed when the proposed QEIRV-2 model is compared with the classical Inception-V3 CNN model. To justify the viability of the proposed framework, its performance is compared with the existing state-of-the-art approaches and 25.23% of accuracy improvement is observed.

Kim P, Kim J. Retinal Healthcare Diagnosis Approaches with Deep Learning Techniques

The retina is an important organ of the human body, with a crucial function in the vision mechanism. A minor disturbance in the retina can cause various abnormalities in the eye, as well as complex retinal diseases such as diabetic retinopathy. To diagnose such diseases in early stages, many researchers are incorporating machine learning (ML) technique. The combination of medical science with ML improves the healthcare diagnosis systems of hospitals, clinics, and other providers. Recently, AI-based healthcare diagnosis systems assist clinicians in handling more patients in less time and improves diagnosis accuracy. In this paper, we review cutting-edge AI-based retinal diagnosis technologies. This article also briefly describes the potential of the latest densely connected convolutional networks (DenseNets) to improve the performance of diagnosis systems. Moreover, this paper focuses on state-of-the-art results from comprehensive investigations in retinal diagnosis and the development of AI-based retinal healthcare diagnosis approaches with deep-learning models.

Applications of deep learning in fundus images: A review

The use of fundus images for the early screening of eye diseases is of great clinical importance. Due to its powerful performance, deep learning is becoming more and more popular in related applications, such as lesion segmentation, biomarkers segmentation, disease diagnosis and image synthesis. Therefore, it is very necessary to summarize the recent developments in deep learning for fundus images with a review paper. In this review, we introduce 143 application papers with a carefully designed hierarchy. Moreover, 33 publicly available datasets are presented. Summaries and analyses are provided for each task. Finally, limitations common to all tasks are revealed and possible solutions are given. We will also release and regularly update the state-of-the-art results and newly-released datasets at https://github.com/nkicsl/Fundus_Review to adapt to the rapid development of this field.

SCNET: A Novel UGI Cancer Screening Framework Based on Semantic-Level Multimodal Data Fusion

Upper gastrointestinal (UGI) cancer has been identified as one of the ten most common causes of cancer deaths globally. UGI cancer screening is critical to improving the survival rate of UGI cancer patients. While many approaches to UGI cancer screening rely on single-modality data such as gastroscope imaging, limited studies have been dedicated to UGI cancer screening exploiting multisource and multimodal medical data, which could potentially lead to improved screening results. In this paper, we propose semantic-level cancer-screening network (SCNET), a framework for UGI cancer screening based on semantic-level multimodal upper gastrointestinal data fusion. Specifically, the proposed SCNET consists of a gastrointestinal image recognition flow and a textual medical record processing flow. High-level features of upper gastrointestinal data are extracted by identifying effective feature channels according to the correlation between the textual features and the spatial structure of the image features. The final screening results are obtained after the data fusion step. The experimental results show that the improvement of our approach over the state-of-the-art ones reached 4.01% in average. The source code of SCNET is available at https://github.com/netflymachine/SCNET.

Automatic microaneurysm detection in fundus image based on local cross-section transformation and multi-feature fusion

BACKGROUND AND OBJECTIVE

Retinal microaneurysm (MA) is one of the earliest clinical signs of diabetic retinopathy(DR). Its detection is essential for controlling DR and preventing vision loss. However, the spatial scale of MA is extremely small and the contrast to surrounding background is subtle, which make MA detection challenging. The purpose of this work is to automatically detect MAs from fundus images.

METHODS

Our MA detector involves two stages: MA candidate extraction and classification. In MA candidate extraction stage, local minimum region extraction and block filtering are used to exploit the regions where MA may exist. In this way, most of irrelavent background regions are discarded , which subsequently facilitates the training of MA classifier. In the second stage, multiple features are extracted to train the MA classifier. To distinguish MA from vascular regions, we propose a series of descriptors according to the cross-section profile of MA. Specially, as MAs are small and their contrast to surroundings is subtle, we propose local cross-section transformation (LCT) to amplify the difference between the MA and confusing structures. Finally, an under-sampling boosting-based classifier (RUSBoost) is trained to determine whether the candidate is an MA.

RESULTS

The proposed method is evaluated on three public available databases i.e. e-ophtha-MA, DiaretDB1 and ROC training set. It achieves high sensitivities for low false positive rates on the three databases. Using the FROC metric, the final scores are 0.516, 0.402 and 0.293 respectively, which are comparable to existing state-of-the-art methods.

CONCLUSIONS

The proposed local cross-section transformation enhances the discrimination of descriptors by amplifying difference between MAs and confusing structures, which facilitates the classification and improves the detection performances. With the powerful descriptors, our method achieves state-of-the-art performances on three public datasets consistently.

Microaneurysm detection in color eye fundus images for diabetic retinopathy screening

Diabetic retinopathy (DR) is a diabetes complication, which in extreme situations may lead to blindness. Since the first stages are often asymptomatic, regular eye examinations are required for an early diagnosis. As microaneurysms (MAs) are one of the first signs of DR, several automated methods have been proposed for their detection in order to reduce the ophthalmologists' workload. Although local convergence filters (LCFs) have already been applied for feature extraction, their potential as MA enhancement operators was not explored yet. In this work, we propose a sliding band filter for MA enhancement aiming at obtaining a set of initial MA candidates. Then, a combination of the filter responses with color, contrast and shape information is used by an ensemble of classifiers for final candidate classification. Finally, for each eye fundus image, a score is computed from the confidence values assigned to the MAs detected in the image. The performance of the proposed methodology was evaluated in four datasets. At the lesion level, sensitivities of 64% and 81% were achieved for an average of 8 false positives per image (FPIs) in e-ophtha MA and SCREEN-DR, respectively. In the last dataset, an AUC of 0.83 was also obtained for DR detection.

A survey on medical image analysis in diabetic retinopathy

Diabetic Retinopathy (DR) represents a highly-prevalent complication of diabetes in which individuals suffer from damage to the blood vessels in the retina. The disease manifests itself through lesion presence, starting with microaneurysms, at the nonproliferative stage before being characterized by neovascularization in the proliferative stage. Retinal specialists strive to detect DR early so that the disease can be treated before substantial, irreversible vision loss occurs. The level of DR severity indicates the extent of treatment necessary - vision loss may be preventable by effective diabetes management in mild (early) stages, rather than subjecting the patient to invasive laser surgery. Using artificial intelligence (AI), highly accurate and efficient systems can be developed to help assist medical professionals in screening and diagnosing DR earlier and without the full resources that are available in specialty clinics. In particular, deep learning facilitates diagnosis earlier and with higher sensitivity and specificity. Such systems make decisions based on minimally handcrafted features and pave the way for personalized therapies. Thus, this survey provides a comprehensive description of the current technology used in each step of DR diagnosis. First, it begins with an introduction to the disease and the current technologies and resources available in this space. It proceeds to discuss the frameworks that different teams have used to detect and classify DR. Ultimately, we conclude that deep learning systems offer revolutionary potential to DR identification and prevention of vision loss.

Deep and Densely Connected Networks for Classification of Diabetic Retinopathy

Diabetes has recently emerged as a worldwide problem, and diabetic retinopathy is an abnormal state associated with the human retina. Due to the increase in daily screen-related activities of modern human beings, diabetic retinopathy is more prevalent among adults, leading to minor and major blindness. Doctors and clinicians are unable to perform early diagnoses due to the large number of patients. To solve this problem, this study introduces a classification model for retinal images that distinguishes between the various stages of diabetic retinopathy. This work involves deploying deep and densely connected networks for retinal image analysis with training from scratch. Dense connections between the convolutional layers of the network are an essential factor to enhance accuracy owing to the deeper supervision between layers. Another factor is the growth rate that further assists our model in learning more sophisticated feature maps regarding retinal images from every stage of the network. We compute the area under the curve, sensitivity, and specificity, particularly for messidor-2 and EyePACS. Compared to existing approaches, our method achieved better results, with an approximate rise rate of 0.01, 0.03, and 0.01, respectively. Therefore, computer-aided programs can help in diagnostic centers as automated detection systems.

Retinal image assessment using bi-level adaptive morphological component analysis

The automated analysis of retinal images is a widely researched area which can help to diagnose several diseases like diabetic retinopathy in early stages of the disease. More specifically, separation of vessels and lesions is very critical as features of these structures are directly related to the diagnosis and treatment process of diabetic retinopathy. The complexity of the retinal image contents especially in images with severe diabetic retinopathy makes detection of vascular structure and lesions difficult. In this paper, a novel framework based on morphological component analysis (MCA) is presented which benefits from the adaptive representations obtained via dictionary learning. In the proposed Bi-level Adaptive MCA (BAMCA), MCA is extended to locally deal with sparse representation of the retinal images at patch level whereas the decomposition process occurs globally at the image level. BAMCA method with appropriately offline learnt dictionaries is adopted to work on retinal images with severe diabetic retinopathy in order to simultaneously separate vessels and exudate lesions as diagnostically useful morphological components. To obtain the appropriate dictionaries, K-SVD dictionary learning algorithm is modified to use a gated error which guides the process toward learning the main structures of the retinal images using vessel or lesion maps. Computational efficiency of the proposed framework is also increased significantly through some improvement leading to noticeable reduction in run time. We experimentally show how effective dictionaries can be learnt which help BAMCA to successfully separate exudate and vessel components from retinal images even in severe cases of diabetic retinopathy. In this paper, in addition to visual qualitative assessment, the performance of the proposed method is quantitatively measured in the framework of vessel and exudate segmentation. The reported experimental results on public datasets demonstrate that the obtained components can be used to achieve competitive results with regard to the state-of-the-art vessel and exudate segmentation methods.

IDRiD: Diabetic Retinopathy - Segmentation and Grading Challenge

Diabetic Retinopathy (DR) is the most common cause of avoidable vision loss, predominantly affecting the working-age population across the globe. Screening for DR, coupled with timely consultation and treatment, is a globally trusted policy to avoid vision loss. However, implementation of DR screening programs is challenging due to the scarcity of medical professionals able to screen a growing global diabetic population at risk for DR. Computer-aided disease diagnosis in retinal image analysis could provide a sustainable approach for such large-scale screening effort. The recent scientific advances in computing capacity and machine learning approaches provide an avenue for biomedical scientists to reach this goal. Aiming to advance the state-of-the-art in automatic DR diagnosis, a grand challenge on "Diabetic Retinopathy - Segmentation and Grading" was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI - 2018). In this paper, we report the set-up and results of this challenge that is primarily based on Indian Diabetic Retinopathy Image Dataset (IDRiD). There were three principal sub-challenges: lesion segmentation, disease severity grading, and localization of retinal landmarks and segmentation. These multiple tasks in this challenge allow to test the generalizability of algorithms, and this is what makes it different from existing ones. It received a positive response from the scientific community with 148 submissions from 495 registrations effectively entered in this challenge. This paper outlines the challenge, its organization, the dataset used, evaluation methods and results of top-performing participating solutions. The top-performing approaches utilized a blend of clinical information, data augmentation, and an ensemble of models. These findings have the potential to enable new developments in retinal image analysis and image-based DR screening in particular.

An Intelligent Segmentation and Diagnosis Method for Diabetic Retinopathy Based on Improved U-NET Network

Due to insufficient samples, the generalization performance of deep network is insufficient. In order to solve this problem, an improved U-net based image automatic segmentation and diagnosis algorithm was proposed, in which the max-pooling operation in original U-net model was replaced by the convolution operation to keep more feature information. Firstly, the regions of 128×128 were extracted from all slices of the patients as data samples. Secondly, the patient samples were divided into training sample set and testing sample set, and data augmentation was performed on the training samples. Finally, all the training samples were adopted to train the model. Compared with Fully Convolutional Network (FCN) model and max-pooling based U-net model, DSC and CR coefficients of the proposed method achieve the best results, while PM coefficient is 2.55 percentage lower than the maximum value in the two comparison models, and Average Symmetric Surface Distance is slightly higher than the minimum value of the two comparison models by 0.004. The experimental results show that the proposed model can achieve good segmentation and diagnosis results.

Microaneurysms segmentation with a U-Net based on recurrent residual convolutional neural network

Microaneurysms (MAs) play an important role in the diagnosis of clinical diabetic retinopathy at the early stage. Annotation of MAs manually by experts is laborious and so it is essential to develop automatic segmentation methods. Automatic MA segmentation remains a challenging task mainly due to the low local contrast of the image and the small size of MAs. A deep learning-based method called U-Net has become one of the most popular methods for the medical image segmentation task. We propose an architecture for U-Net, named deep recurrent U-Net (DRU-Net), obtained by combining the deep residual model and recurrent convolutional operations into U-Net. In the MA segmentation task, DRU-Net can accumulate effective features much better than the typical U-Net. The proposed method is evaluated on two publicly available datasets: E-Ophtha and IDRiD. Our results show that the proposed DRU-Net achieves the best performance with 0.9999 accuracy value and 0.9943 area under curve (AUC) value on the E-Ophtha dataset. And on the IDRiD dataset, it has achieved 0.987 AUC value (to our knowledge, this is the first result of segmenting MAs on this dataset). Compared with other methods, such as U-Net, FCNN, and ResU-Net, our architecture (DRU-Net) achieves state-of-the-art performance.

Fundus photograph-based deep learning algorithms in detecting diabetic retinopathy

Remarkable advances in biomedical research have led to the generation of large amounts of data. Using artificial intelligence, it has become possible to extract meaningful information from large volumes of data, in a shorter frame of time, with very less human interference. In effect, convolutional neural networks (a deep learning method) have been taught to recognize pathological lesions from images. Diabetes has high morbidity, with millions of people who need to be screened for diabetic retinopathy (DR). Deep neural networks offer a great advantage of screening for DR from retinal images, in improved identification of DR lesions and risk factors for diseases, with high accuracy and reliability. This review aims to compare the current evidences on various deep learning models for diagnosis of diabetic retinopathy (DR).

Deep learning in medical imaging and radiation therapy

The goals of this review paper on deep learning (DL) in medical imaging and radiation therapy are to (a) summarize what has been achieved to date; (b) identify common and unique challenges, and strategies that researchers have taken to address these challenges; and (c) identify some of the promising avenues for the future both in terms of applications as well as technical innovations. We introduce the general principles of DL and convolutional neural networks, survey five major areas of application of DL in medical imaging and radiation therapy, identify common themes, discuss methods for dataset expansion, and conclude by summarizing lessons learned, remaining challenges, and future directions.