Deep Learning-Based Detection of Early Renal Function Impairment Using Retinal Fundus Images: Model Development and Validation

Artificial intelligence in chronic kidney disease management: a scoping review

Rationale: Chronic kidney disease (CKD) is a major public health problem worldwide associated with cardiovascular disease, renal failure, and mortality. To effectively address this growing burden, innovative solutions to management are urgently required. We conducted a scoping review to identify key use cases in which artificial intelligence (AI) could be leveraged for improving management of CKD. Additionally, we examined the challenges faced by AI in CKD management, proposed potential solutions to overcome these barriers. Methods: We reviewed 41 articles published between 2014-2024 which examined various AI techniques including machine learning (ML) and deep learning (DL), unsupervised clustering, digital twin, natural language processing (NLP) and large language models (LLMs) in CKD management. We focused on four areas: early detection, risk stratification and prediction, treatment recommendations and patient care and communication. Results: We identified 41 articles published between 2014-2024 that assessed image-based DL models for early detection (n = 6), ML models for risk stratification and prediction (n = 14) and treatment recommendations (n = 4), and NLP and LLMs for patient care and communication (n = 17). Key challenges in integrating AI models into healthcare include technical issues such as data quality and access, model accuracy, and interpretability, alongside adoption barriers like workflow integration, user training, and regulatory approval. Conclusions: There is tremendous potential of integrating AI into clinical care of CKD patients to enable early detection, prediction, and improved patient outcomes. Collaboration among healthcare providers, researchers, regulators, and industries is crucial to developing robust protocols that ensure compliance with legal standards, while minimizing risks and maintaining patient safety.

Oculomics: Current concepts and evidence

The eye provides novel insights into general health, as well as pathogenesis and development of systemic diseases. In the past decade, growing evidence has demonstrated that the eye's structure and function mirror multiple systemic health conditions, especially in cardiovascular diseases, neurodegenerative disorders, and kidney impairments. This has given rise to the field of oculomics-the application of ophthalmic biomarkers to understand mechanisms, detect and predict disease. The development of this field has been accelerated by three major advances: 1) the availability and widespread clinical adoption of high-resolution and non-invasive ophthalmic imaging ("hardware"); 2) the availability of large studies to interrogate associations ("big data"); 3) the development of novel analytical methods, including artificial intelligence (AI) ("software"). Oculomics offers an opportunity to enhance our understanding of the interplay between the eye and the body, while supporting development of innovative diagnostic, prognostic, and therapeutic tools. These advances have been further accelerated by developments in AI, coupled with large-scale linkage datasets linking ocular imaging data with systemic health data. Oculomics also enables the detection, screening, diagnosis, and monitoring of many systemic health conditions. Furthermore, oculomics with AI allows prediction of the risk of systemic diseases, enabling risk stratification, opening up new avenues for prevention or individualized risk prediction and prevention, facilitating personalized medicine. In this review, we summarise current concepts and evidence in the field of oculomics, highlighting the progress that has been made, remaining challenges, and the opportunities for future research.

Predicting renal function using fundus photography: role of confounders

BACKGROUND/AIMS

The kidneys and retina are highly vascularized organs that frequently exhibit shared pathologies, with nephropathy often associated with retinopathy. Previous studies have successfully predicted estimated glomerular filtration rates (eGFRs) using fundus photographs. We evaluated the performance of the Modification of Diet in Renal Disease (MDRD) and Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) formulas in eGFR prediction.

METHODS

We enrolled patients with fundus photographs and corresponding creatinine measurements taken on the same date. One photograph per eye was randomly selected, resulting in a final dataset of 45,108 patients (88,260 photographs). Data including sex, age, and blood creatinine levels were collected for eGFR calculation using the MDRD and CKD-EPI formulas. EfficientNet B3 models were used to predict each parameter.

RESULTS

Deep neural network models accurately predicted age and sex using fundus photographs. Sex was identified as a confounding variable in creatinine prediction. The MDRD formula was more susceptible to this confounding effect than the CKD-EPI formula. Notably, the CKD-EPI formula demonstrated superior performance compared to the MDRD formula (area under the curve 0.864 vs. 0.802).

CONCLUSION

Fundus photographs are a valuable tool for screening renal function using deep neural network models, demonstrating the role of noninvasive imaging in medical diagnostics. However, these models are susceptible to the influence of sex, a potential confounding factor. The CKD-EPI formula, less susceptible to sex bias, is recommended to obtain more reliable results.

Use of artificial intelligence with retinal imaging in screening for diabetes-associated complications: systematic review

Background

Artificial Intelligence (AI) has been used to automate detection of retinal diseases from retinal images with great success, in particular for screening for diabetic retinopathy, a major complication of diabetes. Since persons with diabetes routinely receive retinal imaging to evaluate their diabetic retinopathy status, AI-based retinal imaging may have potential to be used as an opportunistic comprehensive screening for multiple systemic micro- and macro-vascular complications of diabetes.

Methods

We conducted a qualitative systematic review on published literature using AI on retina images to detect systemic diabetes complications. We searched three main databases: PubMed, Google Scholar, and Web of Science (January 1, 2000, to October 1, 2024). Research that used AI to evaluate the associations between retinal images and diabetes-associated complications, or research involving diabetes patients with retinal imaging and AI systems were included. Our primary focus was on articles related to AI, retinal images, and diabetes-associated complications. We evaluated each study for the robustness of the studies by development of the AI algorithm, size and quality of the training dataset, internal validation and external testing, and the performance. Quality assessments were employed to ensure the inclusion of high-quality studies, and data extraction was conducted systematically to gather pertinent information for analysis. This study has been registered on PROSPERO under the registration ID CRD42023493512.

Findings

From a total of 337 abstracts, 38 studies were included. These studies covered a range of topics related to prediction of diabetes from pre-diabetes or non-diabeticindividuals (n = 4), diabetes related systemic risk factors (n = 10), detection of microvascular complications (n = 8) and detection of macrovascular complications (n = 17). Most studies (n = 32) utilized color fundus photographs (CFP) as retinal image modality, while others employed optical coherence tomography (OCT) (n = 6). The performance of the AI systems varied, with an AUC ranging from 0.676 to 0.971 in prediction or identification of different complications. Study designs included cross-sectional and cohort studies with sample sizes ranging from 100 to over 100,000 participants. Risk of bias was evaluated by using the Newcastle-Ottawa Scale and AXIS, with most studies scoring as low to moderate risk.

Interpretation

Our review highlights the potential for the use of AI algorithms applied to retina images, particularly CFP, to screen, predict, or diagnose the various microvascular and macrovascular complications of diabetes. However, we identified few studies with longitudinal data and a paucity of randomized control trials, reflecting a gap between the development of AI algorithms and real-world implementation and translational studies.

Funding

Dr. Gavin Siew Wei TAN is supported by: 1. DYNAMO: Diabetes studY on Nephropathy And other Microvascular cOmplications II supported by National Medical Research Council (MOH-001327-03): data collection, analysis, trial design 2. Prognositc significance of novel multimodal imaging markers for diabetic retinopathy: towards improving the staging for diabetic retinopathy supported by NMRC Clinician Scientist Award (CSA)-Investigator (INV) (MOH-001047-00).

Artificial intelligence-enhanced retinal imaging as a biomarker for systemic diseases

Retinal images provide a non-invasive and accessible means to directly visualize human blood vessels and nerve fibers. Growing studies have investigated the intricate microvascular and neural circuitry within the retina, its interactions with other systemic vascular and nervous systems, and the link between retinal biomarkers and various systemic diseases. Using the eye to study systemic health, based on these connections, has been given a term as oculomics. Advancements in artificial intelligence (AI) technologies, particularly deep learning, have further increased the potential impact of this study. Leveraging these technologies, retinal analysis has demonstrated potentials in detecting numerous diseases, including cardiovascular diseases, central nervous system diseases, chronic kidney diseases, metabolic diseases, endocrine disorders, and hepatobiliary diseases. AI-based retinal imaging, which incorporates established modalities such as digital color fundus photographs, optical coherence tomography (OCT) and OCT angiography, as well as emerging technologies like ultra-wide field imaging, shows great promises in predicting systemic diseases. This provides a valuable opportunity for systemic diseases screening, early detection, prediction, risk stratification, and personalized prognostication. As the AI and big data research field grows, with the mission of transforming healthcare, they also face numerous challenges and limitations both in data and technology. The application of natural language processing framework, large language model, and other generative AI techniques presents both opportunities and concerns that require careful consideration. In this review, we not only summarize key studies on AI-enhanced retinal imaging for predicting systemic diseases but also underscore the significance of these advancements in transforming healthcare. By highlighting the remarkable progress made thus far, we provide a comprehensive overview of state-of-the-art techniques and explore the opportunities and challenges in this rapidly evolving field. This review aims to serve as a valuable resource for researchers and clinicians, guiding future studies and fostering the integration of AI in clinical practice.

Diagnosis of Chronic Kidney Disease Using Retinal Imaging and Urine Dipstick Data: Multimodal Deep Learning Approach

Background

Chronic kidney disease (CKD) is a prevalent condition with significant global health implications. Early detection and management are critical to prevent disease progression and complications. Deep learning (DL) models using retinal images have emerged as potential noninvasive screening tools for CKD, though their performance may be limited, especially in identifying individuals with proteinuria and in specific subgroups.

Objective

We aim to evaluate the efficacy of integrating retinal images and urine dipstick data into DL models for enhanced CKD diagnosis.

Methods

The 3 models were developed and validated: eGFR-RIDL (estimated glomerular filtration rate-retinal image deep learning), eGFR-UDLR (logistic regression using urine dipstick data), and eGFR-MMDL (multimodal deep learning combining retinal images and urine dipstick data). All models were trained to predict an eGFR<60 mL/min/1.73 m², a key indicator of CKD, calculated using the 2009 CKD-EPI (Chronic Kidney Disease Epidemiology Collaboration) equation. This study used a multicenter dataset of participants aged 20-79 years, including a development set (65,082 people) and an external validation set (58,284 people). Wide Residual Networks were used for DL, and saliency maps were used to visualize model attention. Sensitivity analyses assessed the impact of numerical variables.

Results

eGFR-MMDL outperformed eGFR-RIDL in both the test and external validation sets, with area under the curves of 0.94 versus 0.90 and 0.88 versus 0.77 (P<.001 for both, DeLong test). eGFR-UDLR outperformed eGFR-RIDL and was comparable to eGFR-MMDL, particularly in the external validation. However, in the subgroup analysis, eGFR-MMDL showed improvement across all subgroups, while eGFR-UDLR demonstrated no such gains. This suggested that the enhanced performance of eGFR-MMDL was not due to urine data alone, but rather from the synergistic integration of both retinal images and urine data. The eGFR-MMDL model demonstrated the best performance in individuals younger than 65 years or those with proteinuria. Age and proteinuria were identified as critical factors influencing model performance. Saliency maps indicated that urine data and retinal images provide complementary information, with urine offering insights into retinal abnormalities and retinal images, particularly the arcade vessels, being key for predicting kidney function.

Conclusions

The MMDL model integrating retinal images and urine dipstick data show significant promise for noninvasive CKD screening, outperforming the retinal image-only model. However, routine blood tests are still recommended for individuals aged 65 years and older due to the model's limited performance in this age group.

Evaluating the potential of retinal photography in chronic kidney disease detection: a review

Background

Chronic kidney disease (CKD) is a significant global health concern, emphasizing the necessity of early detection to facilitate prompt clinical intervention. Leveraging the unique ability of the retina to offer insights into systemic vascular health, it emerges as an interesting, non-invasive option for early CKD detection. Integrating this approach with existing invasive methods could provide a comprehensive understanding of patient health, enhancing diagnostic accuracy and treatment effectiveness.

Objectives

The purpose of this review is to critically assess the potential of retinal imaging to serve as a diagnostic tool for CKD detection based on retinal vascular changes. The review tracks the evolution from conventional manual evaluations to the latest state-of-the-art in deep learning.

Survey Methodology

A comprehensive examination of the literature was carried out, using targeted database searches and a three-step methodology for article evaluation: identification, screening, and inclusion based on Prisma guidelines. Priority was given to unique and new research concerning the detection of CKD with retinal imaging. A total of 70 publications from 457 that were initially discovered satisfied our inclusion criteria and were thus subjected to analysis. Out of the 70 studies included, 35 investigated the correlation between diabetic retinopathy and CKD, 23 centered on the detection of CKD via retinal imaging, and four attempted to automate the detection through the combination of artificial intelligence and retinal imaging.

Results

Significant retinal features such as arteriolar narrowing, venular widening, specific retinopathy markers (like microaneurysms, hemorrhages, and exudates), and changes in arteriovenous ratio (AVR) have shown strong correlations with CKD progression. We also found that the combination of deep learning with retinal imaging for CKD detection could provide a very promising pathway. Accordingly, leveraging retinal imaging through this technique is expected to enhance the precision and prognostic capacity of the CKD detection system, offering a non-invasive diagnostic alternative that could transform patient care practices.

Conclusion

In summary, retinal imaging holds high potential as a diagnostic tool for CKD because it is non-invasive, facilitates early detection through observable microvascular changes, offers predictive insights into renal health, and, when paired with deep learning algorithms, enhances the accuracy and effectiveness of CKD screening.

Progress and challenges of in vivo flow cytometry and its applications in circulating cells of eyes

Circulating inflammatory cells in eyes have emerged as early indicators of numerous major diseases, yet the monitoring of these cells remains an underdeveloped field. In vivo flow cytometry (IVFC), a noninvasive technique, offers the promise of real-time, dynamic quantification of circulating cells. However, IVFC has not seen extensive applications in the detection of circulating cells in eyes, possibly due to the eye's unique physiological structure and fundus imaging limitations. This study reviews the current research progress in retinal flow cytometry and other fundus examination techniques, such as adaptive optics, ultra-widefield retinal imaging, multispectral imaging, and optical coherence tomography, to propose novel ideas for circulating cell monitoring.

Performance of deep learning for detection of chronic kidney disease from retinal fundus photographs: A systematic review and meta-analysis

OBJECTIVE

Deep learning has been used to detect chronic kidney disease (CKD) from retinal fundus photographs. We aim to evaluate the performance of deep learning for CKD detection.

METHODS

The original studies in CKD patients detected by deep learning from retinal fundus photographs were eligible for inclusion. PubMed, Embase, the Cochrane Library, and Web of Science were searched up to October 31, 2022. The Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool was used to assess the risk of bias.

RESULTS

Four studies enrolled 114,860 subjects were included. The pooled sensitivity and specificity were 87.8% (95% confidence interval (CI): 61.6% to 98.3%), and 62.4% (95% CI: 44.9% to 78.7%). The area under the curve (AUC) was 0.864 (95%CI: 0.769, 0.986).

CONCLUSION

Deep learning based on retinal fundus photographs has the ability to detect CKD, but it currently has a lot of room for improvement. It is still a long way from clinical application.

Short-term duration of diabetic retinopathy as a predictor for development of diabetic kidney disease

Background

Diabetic retinopathy (DR) is a risk factor for diabetic kidney disease (DKD). Whether the duration, especially the short-term duration, of DR is associated with the development and progression of DKD remains unclear.

Materials and Methods

A retrospective study and two-sample Mendelian randomization (MR) analysis were conducted. Kidney disease was defined by the urinary albumin-to-creatinine ratio (ACR) and the estimated glomerular filtration rate (eGFR). DR was diagnosed by an expert ophthalmologist by using a digital fundus camera. Binary and ordinal logistic regression analyses were performed. A restricted cubic spline was utilized to detect nonlinear associations. Summary statistics for DR- and DKD-associated single-nuclear polymorphisms (SNPs) were extracted from the FinnGen and the UK Biobank consortia.

Results

A total of 2674 patients with type 2 diabetes mellitus (T2DM) and type 2 diabetic kidney disease (T2DKD) were included. The prevalence and mean duration of DR increased with elevation of ACR and decline in eGFR. Renal function was significantly reduced in patients with DR in the fifth year of life. Binary and ordinal logistic regression showed that each 1-year increase in DR duration was associated with a 19% risk increase in the development of DKD, 16% in the elevation of ACR, and 21% in the decline of renal function. MR estimates indicated that DR was causally associated with DKD development, with an odds ratio of 2.89.

Conclusions

DR and the duration of DR were independent risk factors for the development and progression of DKD. The short-term duration of DR may be associated with DKD development. DR had a statistically significant effect on DKD.

Examination of alternative eGFR definitions on the performance of deep learning models for detection of chronic kidney disease from fundus photographs

Deep learning (DL) models have shown promise in detecting chronic kidney disease (CKD) from fundus photographs. However, previous studies have utilized a serum creatinine-only estimated glomerular rate (eGFR) equation to measure kidney function despite the development of more up-to-date methods. In this study, we developed two sets of DL models using fundus images from the UK Biobank to ascertain the effects of using a creatinine and cystatin-C eGFR equation over the baseline creatinine-only eGFR equation on fundus image-based DL CKD predictors. Our results show that a creatinine and cystatin-C eGFR significantly improved classification performance over the baseline creatinine-only eGFR when the models were evaluated conventionally. However, these differences were no longer significant when the models were assessed on clinical labels based on ICD10. Furthermore, we also observed variations in model performance and systemic condition incidence between our study and the ones conducted previously. We hypothesize that limitations in existing eGFR equations and the paucity of retinal features uniquely indicative of CKD may contribute to these inconsistencies. These findings emphasize the need for developing more transparent models to facilitate a better understanding of the mechanisms underpinning the ability of DL models to detect CKD from fundus images.

Deep learning-based fundus image analysis for cardiovascular disease: a review

It is well established that the retina provides insights beyond the eye. Through observation of retinal microvascular changes, studies have shown that the retina contains information related to cardiovascular disease. Despite the tremendous efforts toward reducing the effects of cardiovascular diseases, they remain a global challenge and a significant public health concern. Conventionally, predicting the risk of cardiovascular disease involves the assessment of preclinical features, risk factors, or biomarkers. However, they are associated with cost implications, and tests to acquire predictive parameters are invasive. Artificial intelligence systems, particularly deep learning (DL) methods applied to fundus images have been generating significant interest as an adjunct assessment tool with the potential of enhancing efforts to prevent cardiovascular disease mortality. Risk factors such as age, gender, smoking status, hypertension, and diabetes can be predicted from fundus images using DL applications with comparable performance to human beings. A clinical change to incorporate DL systems for the analysis of fundus images as an equally good test over more expensive and invasive procedures may require conducting prospective clinical trials to mitigate all the possible ethical challenges and medicolegal implications. This review presents current evidence regarding the use of DL applications on fundus images to predict cardiovascular disease.

Deep learning algorithms to detect diabetic kidney disease from retinal photographs in multiethnic populations with diabetes

OBJECTIVE

To develop a deep learning algorithm (DLA) to detect diabetic kideny disease (DKD) from retinal photographs of patients with diabetes, and evaluate performance in multiethnic populations.

MATERIALS AND METHODS

We trained 3 models: (1) image-only; (2) risk factor (RF)-only multivariable logistic regression (LR) model adjusted for age, sex, ethnicity, diabetes duration, HbA1c, systolic blood pressure; (3) hybrid multivariable LR model combining RF data and standardized z-scores from image-only model. Data from Singapore Integrated Diabetic Retinopathy Program (SiDRP) were used to develop (6066 participants with diabetes, primary-care-based) and internally validate (5-fold cross-validation) the models. External testing on 2 independent datasets: (1) Singapore Epidemiology of Eye Diseases (SEED) study (1885 participants with diabetes, population-based); (2) Singapore Macroangiopathy and Microvascular Reactivity in Type 2 Diabetes (SMART2D) (439 participants with diabetes, cross-sectional) in Singapore. Supplementary external testing on 2 Caucasian cohorts: (3) Australian Eye and Heart Study (AHES) (460 participants with diabetes, cross-sectional) and (4) Northern Ireland Cohort for the Longitudinal Study of Ageing (NICOLA) (265 participants with diabetes, cross-sectional).

RESULTS

In SiDRP validation, area under the curve (AUC) was 0.826(95% CI 0.818-0.833) for image-only, 0.847(0.840-0.854) for RF-only, and 0.866(0.859-0.872) for hybrid. Estimates with SEED were 0.764(0.743-0.785) for image-only, 0.802(0.783-0.822) for RF-only, and 0.828(0.810-0.846) for hybrid. In SMART2D, AUC was 0.726(0.686-0.765) for image-only, 0.701(0.660-0.741) in RF-only, 0.761(0.724-0.797) for hybrid.

DISCUSSION AND CONCLUSION

There is potential for DLA using retinal images as a screening adjunct for DKD among individuals with diabetes. This can value-add to existing DLA systems which diagnose diabetic retinopathy from retinal images, facilitating primary screening for DKD.

Lesion-aware attention network for diabetic nephropathy diagnosis with optical coherence tomography images

Purpose

For early screening of diabetic nephropathy patients, we propose a deep learning algorithm to screen high-risk patients with diabetic nephropathy from retinal images of diabetic patients.

Methods

We propose the use of attentional mechanisms to improve the model's focus on lesion-prone regions of retinal OCT images. First, the data is trained using the base network and the Grad-CAM algorithm locates image regions that have a large impact on the model output and generates a rough mask localization map. The mask is used as a auxiliary region to realize the auxiliary attention module. We then inserted the region-guided attention module into the baseline model and trained the CNN model to guide the model to better focus on relevant lesion features. The proposed model improves the recognition of the lesion region.

Results

To evaluate the lesion-aware attention network, we trained and tested it using OCT volumetric data collected from 66 patients with diabetic retinal microangiopathy (89 eyes, male = 43, female = 23). There were 45 patients (60 eyes, male=27, female = 18) in DR group and 21 patients (29 eyes, male = 16, female = 5) in DN group. Our proposed model performs even better in disease classification, specifically, the accuracy of the proposed model was 91.68%, the sensitivity was 89.99%, and the specificity was 92.18%.

Conclusion

The proposed lesion-aware attention model can provide reliable screening of high-risk patients with diabetic nephropathy.

A Systematic Review and Meta-Analysis of Applying Deep Learning in the Prediction of the Risk of Cardiovascular Diseases From Retinal Images

Purpose

The purpose of this study was to perform a systematic review and meta-analysis to synthesize evidence from studies using deep learning (DL) to predict cardiovascular disease (CVD) risk from retinal images.

Methods

A systematic literature search was performed in MEDLINE, Scopus, and Web of Science up to June 2022. We extracted data pertaining to predicted outcomes, model development, and validation and model performance metrics. Included studies were graded using the Quality Assessment of Diagnostic Accuracies Studies 2 tool. Model performance was pooled across eligible studies using a random-effects meta-analysis model.

Results

A total of 26 studies were included in the analysis. There were 42 CVD risk-related outcomes predicted from retinal images were identified, including 33 CVD risk factors, 4 cardiac imaging biomarkers, 2 CVD risk scores, the presence of CVD, and incident CVD. Three studies that aimed to predict the development of future CVD events reported an area under the receiver operating curve (AUROC) between 0.68 and 0.81. Models that used retinal images as input data had a pooled mean absolute error of 3.19 years (95% confidence interval [CI] = 2.95-3.43) for age prediction; a pooled AUROC of 0.96 (95% CI = 0.95-0.97) for gender classification; a pooled AUROC of 0.80 (95% CI = 0.73-0.86) for diabetes detection; and a pooled AUROC of 0.86 (95% CI = 0.81-0.92) for the detection of chronic kidney disease. We observed a high level of heterogeneity and variation in study designs.

Conclusions

Although DL models appear to have reasonably good performance when it comes to predicting CVD risk, further work is necessary to evaluate the real-world applicability and predictive accuracy.

Translational Relevance

DL-based CVD risk assessment from retinal images holds great promise to be translated to clinical practice as a novel approach for CVD risk assessment, given its simple, quick, and noninvasive nature.

Ocular images-based artificial intelligence on systemic diseases

PURPOSE

To provide a summary of the research advances on ocular images-based artificial intelligence on systemic diseases.

METHODS

Narrative literature review.

RESULTS

Ocular images-based artificial intelligence has been used in a variety of systemic diseases, including endocrine, cardiovascular, neurological, renal, autoimmune, and hematological diseases, and many others. However, the studies are still at an early stage. The majority of studies have used AI only for diseases diagnosis, and the specific mechanisms linking systemic diseases to ocular images are still unclear. In addition, there are many limitations to the research, such as the number of images, the interpretability of artificial intelligence, rare diseases, and ethical and legal issues.

CONCLUSION

While ocular images-based artificial intelligence is widely used, the relationship between the eye and the whole body should be more clearly elucidated.

Deep Learning Algorithms for Screening and Diagnosis of Systemic Diseases Based on Ophthalmic Manifestations: A Systematic Review

Deep learning (DL) is the new high-profile technology in medical artificial intelligence (AI) for building screening and diagnosing algorithms for various diseases. The eye provides a window for observing neurovascular pathophysiological changes. Previous studies have proposed that ocular manifestations indicate systemic conditions, revealing a new route in disease screening and management. There have been multiple DL models developed for identifying systemic diseases based on ocular data. However, the methods and results varied immensely across studies. This systematic review aims to summarize the existing studies and provide an overview of the present and future aspects of DL-based algorithms for screening systemic diseases based on ophthalmic examinations. We performed a thorough search in PubMed^®, Embase, and Web of Science for English-language articles published until August 2022. Among the 2873 articles collected, 62 were included for analysis and quality assessment. The selected studies mainly utilized eye appearance, retinal data, and eye movements as model input and covered a wide range of systemic diseases such as cardiovascular diseases, neurodegenerative diseases, and systemic health features. Despite the decent performance reported, most models lack disease specificity and public generalizability for real-world application. This review concludes the pros and cons and discusses the prospect of implementing AI based on ocular data in real-world clinical scenarios.

Use of multimodal dataset in AI for detecting glaucoma based on fundus photographs assessed with OCT: focus group study on high prevalence of myopia

BACKGROUND

Glaucoma is one of the major causes of blindness; it is estimated that over 110 million people will be affected by glaucoma worldwide by 2040. Research on glaucoma detection using deep learning technology has been increasing, but the diagnosis of glaucoma in a large population with high incidence of myopia remains a challenge. This study aimed to provide a decision support system for the automatic detection of glaucoma using fundus images, which can be applied for general screening, especially in areas of high incidence of myopia.

METHODS

A total of 1,155 fundus images were acquired from 667 individuals with a mean axial length of 25.60 ± 2.0 mm at the National Taiwan University Hospital, Hsinchu Br. These images were graded based on the findings of complete ophthalmology examinations, visual field test, and optical coherence tomography into three groups: normal (N, n = 596), pre-perimetric glaucoma (PPG, n = 66), and glaucoma (G, n = 493), and divided into a training-validation (N: 476, PPG: 55, G: 373) and test (N: 120, PPG: 11, G: 120) sets. A multimodal model with the Xception model as image feature extraction and machine learning algorithms [random forest (RF), support vector machine (SVM), dense neural network (DNN), and others] was applied.

RESULTS

The Xception model classified the N, PPG, and G groups with 93.9% of the micro-average area under the receiver operating characteristic curve (AUROC) with tenfold cross-validation. Although normal and glaucoma sensitivity can reach 93.51% and 86.13% respectively, the PPG sensitivity was only 30.27%. The AUROC increased to 96.4% in the N + PPG and G groups. The multimodal model with the N + PPG and G groups showed that the AUROCs of RF, SVM, and DNN were 99.56%, 99.59%, and 99.10%, respectively; The N and PPG + G groups had less than 1% difference. The test set showed an overall 3%-5% less AUROC than the validation results.

CONCLUSION

The multimodal model had good AUROC while detecting glaucoma in a population with high incidence of myopia. The model shows the potential for general automatic screening and telemedicine, especially in Asia.

TRIAL REGISTRATION

The study was approved by the Institutional Review Board of the National Taiwan University Hospital, Hsinchu Branch (no. NTUHHCB 108-025-E).

Artificial Intelligence in Predicting Systemic Parameters and Diseases From Ophthalmic Imaging

Artificial Intelligence (AI) analytics has been used to predict, classify, and aid clinical management of multiple eye diseases. Its robust performances have prompted researchers to expand the use of AI into predicting systemic, non-ocular diseases and parameters based on ocular images. Herein, we discuss the reasons why the eye is well-suited for systemic applications, and review the applications of deep learning on ophthalmic images in the prediction of demographic parameters, body composition factors, and diseases of the cardiovascular, hematological, neurodegenerative, metabolic, renal, and hepatobiliary systems. Three main imaging modalities are included—retinal fundus photographs, optical coherence tomographs and external ophthalmic images. We examine the range of systemic factors studied from ophthalmic imaging in current literature and discuss areas of future research, while acknowledging current limitations of AI systems based on ophthalmic images.

Deep Learning Model Based on 3D Optical Coherence Tomography Images for the Automated Detection of Pathologic Myopia

Pathologic myopia causes vision impairment and blindness, and therefore, necessitates a prompt diagnosis. However, there is no standardized definition of pathologic myopia, and its interpretation by 3D optical coherence tomography images is subjective, requiring considerable time and money. Therefore, there is a need for a diagnostic tool that can automatically and quickly diagnose pathologic myopia in patients. This study aimed to develop an algorithm that uses 3D optical coherence tomography volumetric images (C-scan) to automatically diagnose patients with pathologic myopia. The study was conducted using 367 eyes of patients who underwent optical coherence tomography tests at the Ophthalmology Department of Incheon St. Mary’s Hospital and Seoul St. Mary’s Hospital from January 2012 to May 2020. To automatically diagnose pathologic myopia, a deep learning model was developed using 3D optical coherence tomography images. The model was developed using transfer learning based on four pre-trained convolutional neural networks (ResNet18, ResNext50, EfficientNetB0, EfficientNetB4). Grad-CAM was used to visualize features affecting the detection of pathologic myopia. The performance of each model was evaluated and compared based on accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC). The model based on EfficientNetB4 showed the best performance (95% accuracy, 93% sensitivity, 96% specificity, and 98% AUROC) in identifying pathologic myopia.

Heart Rate Modeling and Prediction Using Autoregressive Models and Deep Learning

Physiological time series are affected by many factors, making them highly nonlinear and nonstationary. As a consequence, heart rate time series are often considered difficult to predict and handle. However, heart rate behavior can indicate underlying cardiovascular and respiratory diseases as well as mood disorders. Given the importance of accurate modeling and reliable predictions of heart rate fluctuations for the prevention and control of certain diseases, it is paramount to identify models with the best performance in such tasks. The objectives of this study were to compare the results of three different forecasting models (Autoregressive Model, Long Short-Term Memory Network, and Convolutional Long Short-Term Memory Network) trained and tested on heart rate beats per minute data obtained from twelve heterogeneous participants and to identify the architecture with the best performance in terms of modeling and forecasting heart rate behavior. Heart rate beats per minute data were collected using a wearable device over a period of 10 days from twelve different participants who were heterogeneous in age, sex, medical history, and lifestyle behaviors. The goodness of the results produced by the models was measured using both the mean absolute error and the root mean square error as error metrics. Despite the three models showing similar performance, the Autoregressive Model gave the best results in all settings examined. For example, considering one of the participants, the Autoregressive Model gave a mean absolute error of 2.069 (compared to 2.173 of the Long Short-Term Memory Network and 2.138 of the Convolutional Long Short-Term Memory Network), achieving an improvement of 5.027% and 3.335%, respectively. Similar results can be observed for the other participants. The findings of the study suggest that regardless of an individual's age, sex, and lifestyle behaviors, their heart rate largely depends on the pattern observed in the previous few minutes, suggesting that heart rate can be reasonably regarded as an autoregressive process. The findings also suggest that minute-by-minute heart rate prediction can be accurately performed using a linear model, at least in individuals without pathologies that cause heartbeat irregularities. The findings also suggest many possible applications for the Autoregressive Model, in principle in any context where minute-by-minute heart rate prediction is required (arrhythmia detection and analysis of the response to training, among others).

Systemic retinal biomarkers

PURPOSE OF REVIEW

Systemic retinal biomarkers are biomarkers identified in the retina and related to evaluation and management of systemic disease. This review summarizes the background, categories and key findings from this body of research as well as potential applications to clinical care.

RECENT FINDINGS

Potential systemic retinal biomarkers for cardiovascular disease, kidney disease and neurodegenerative disease were identified using regression analysis as well as more sophisticated image processing techniques. Deep learning techniques were used in a number of studies predicting diseases including anaemia and chronic kidney disease. A virtual coronary artery calcium score performed well against other competing traditional models of event prediction.

SUMMARY

Systemic retinal biomarker research has progressed rapidly using regression studies with clearly identified biomarkers such as retinal microvascular patterns, as well as using deep learning models. Future systemic retinal biomarker research may be able to boost performance using larger data sets, the addition of meta-data and higher resolution image inputs.

A Multimodal Imaging-Based Deep Learning Model for Detecting Treatment-Requiring Retinal Vascular Diseases: Model Development and Validation Study

BACKGROUND

Retinal vascular diseases, including diabetic macular edema (DME), neovascular age-related macular degeneration (nAMD), myopic choroidal neovascularization (mCNV), and branch and central retinal vein occlusion (BRVO/CRVO), are considered vision-threatening eye diseases. However, accurate diagnosis depends on multimodal imaging and the expertise of retinal ophthalmologists.

OBJECTIVE

The aim of this study was to develop a deep learning model to detect treatment-requiring retinal vascular diseases using multimodal imaging.

METHODS

This retrospective study enrolled participants with multimodal ophthalmic imaging data from 3 hospitals in Taiwan from 2013 to 2019. Eye-related images were used, including those obtained through retinal fundus photography, optical coherence tomography (OCT), and fluorescein angiography with or without indocyanine green angiography (FA/ICGA). A deep learning model was constructed for detecting DME, nAMD, mCNV, BRVO, and CRVO and identifying treatment-requiring diseases. Model performance was evaluated and is presented as the area under the curve (AUC) for each receiver operating characteristic curve.

RESULTS

A total of 2992 eyes of 2185 patients were studied, with 239, 1209, 1008, 211, 189, and 136 eyes in the control, DME, nAMD, mCNV, BRVO, and CRVO groups, respectively. Among them, 1898 eyes required treatment. The eyes were divided into training, validation, and testing groups in a 5:1:1 ratio. In total, 5117 retinal fundus photos, 9316 OCT images, and 20,922 FA/ICGA images were used. The AUCs for detecting mCNV, DME, nAMD, BRVO, and CRVO were 0.996, 0.995, 0.990, 0.959, and 0.988, respectively. The AUC for detecting treatment-requiring diseases was 0.969. From the heat maps, we observed that the model could identify retinal vascular diseases.

CONCLUSIONS

Our study developed a deep learning model to detect retinal diseases using multimodal ophthalmic imaging. Furthermore, the model demonstrated good performance in detecting treatment-requiring retinal diseases.