Deep learning versus human graders for classifying diabetic retinopathy severity in a nationwide screening program

Artificial intelligence for early detection of diabetes mellitus complications via retinal imaging

Background

Diabetes mellitus (DM) increases the risk of vascular complications, and retinal vasculature imaging serves as a valuable indicator of both microvascular and macrovascular health. Moreover, artificial intelligence (AI)-enabled systems developed for high-throughput detection of diabetic retinopathy (DR) using digitized retinal images have become clinically adopted. This study reviews AI applications using retinal images for DM-related complications, highlighting advancements beyond DR screening, diagnosis, and prognosis, and addresses implementation challenges, such as ethics, data privacy, equitable access, and explainability.

Methods

We conducted a thorough literature search across several databases, including PubMed, Scopus, and Web of Science, focusing on studies involving diabetes, the retina, and artificial intelligence. We reviewed the original research based on their methodology, AI algorithms, data processing techniques, and validation procedures to ensure a detailed analysis of AI applications in diabetic retinal imaging.

Results

Retinal images can be used to diagnose DM complications including DR, neuropathy, nephropathy, and atherosclerotic cardiovascular disease, as well as to predict the risk of cardiovascular events. Beyond DR screening, AI integration also offers significant potential to address the challenges in the comprehensive care of patients with DM.

Conclusion

With the ability to evaluate the patient's health status in relation to DM complications as well as risk prognostication of future cardiovascular complications, AI-assisted retinal image analysis has the potential to become a central tool for modern personalized medicine in patients with DM.

A Deep Learning Segmentation Model for Detection of Active Proliferative Diabetic Retinopathy

INTRODUCTION

Existing deep learning (DL) algorithms lack the capability to accurately identify patients in immediate need of treatment for proliferative diabetic retinopathy (PDR). We aimed to develop a DL segmentation model to detect active PDR in six-field retinal images by the annotation of new retinal vessels and preretinal hemorrhages.

METHODS

We identified six-field retinal images classified at level 4 of the International Clinical Diabetic Retinopathy Disease Severity Scale collected at the Island of Funen from 2009 to 2019 as part of the Danish screening program for diabetic retinopathy (DR). A certified grader (grader 1) manually dichotomized the images into active or inactive PDR, and the images were then reassessed by two independent certified graders. In cases of disagreement, the final classification decision was made in collaboration between grader 1 and one of the secondary graders. Overall, 637 images were classified as active PDR. We then applied our pre-established DL segmentation model to annotate nine lesion types before training the algorithm. The segmentations of new vessels and preretinal hemorrhages were corrected for any inaccuracies before training the DL algorithm. After the classification and pre-segmentation phases the images were divided into training (70%), validation (10%), and testing (20%) datasets. We added 301 images with inactive PDR to the testing dataset.

RESULTS

We included 637 images of active PDR and 301 images of inactive PDR from 199 individuals. The training dataset had 1381 new vessel and preretinal hemorrhage lesions, while the validation dataset had 123 lesions and the testing dataset 374 lesions. The DL system demonstrated a sensitivity of 90% and a specificity of 70% for annotation-assisted classification of active PDR. The negative predictive value was 94%, while the positive predictive value was 57%.

CONCLUSIONS

Our DL segmentation model achieved excellent sensitivity and acceptable specificity in distinguishing active from inactive PDR.

How Foundational Is the Retina Foundation Model? Estimating RETFound's Label Efficiency on Binary Classification of Normal versus Abnormal OCT Images

Objective

While the availability of public internet-scale datasets of images and language has catalyzed remarkable progress in machine learning, medical datasets are constrained by regulations protecting patient privacy and the time and cost required for curation and labeling. Self-supervised learning or pretraining has demonstrated great success in learning meaningful representations from large unlabeled datasets to enable efficient learning on downstream tasks. In ophthalmology, the RETFound model, a large vision transformer (ViT-L) model trained by masked autoencoding on 1.6 million color fundus photos and OCT B-scans, is the first model pretrained at such scale for ophthalmology, demonstrating strong performance on downstream tasks from diabetic retinopathy grading to stroke detection. Here, we measure the label efficiency of the RETFound model in learning to identify normal vs. abnormal OCT B-scans obtained as part of a pilot study for primary care-based diabetic retinopathy screening in North Carolina.

Design

The 1150 TopCon Maestro OCT central B-scans (981 normal and 169 abnormal) were randomly split 80/10/10 into training, validation, and test datasets. Model training and hyperparameter tuning were performed on the training set guided by validation set performance. The best performing models were then evaluated on the final test set.

Subjects

Six hundred forty-seven patients with diabetes in the Duke Health System participating in primary care diabetic retinopathy screening contributed 1150 TopCon Maestro OCT central B-scans.

Methods

Three models (ResNet-50, ViT-L, and RETFound) were fine-tuned on the full training dataset of 915 OCT B-scans and on smaller training data subsets of 500, 250, 100, and 50 OCT B-scans, respectively, across 3 random seeds.

Main Outcome Measures

Mean accuracy, area under the receiver operator curve (AUROC), area under the precision recall curve (AUPRC), F1 score, precision, and recall on the final held-out test set were reported for each model.

Results

Across 3 random seeds and all training dataset sizes, RETFound outperformed both ResNet-50 and ViT-L on all evaluation metrics on the final held-out test dataset. Large vision transformer and ResNet-50 performed comparably at the largest training dataset sizes of 915 and 500 OCT B-scans; however, ResNet-50 suffered more pronounced performance degradation at the smallest dataset sizes of 100 and 50 OCT B-scans.

Conclusions

Our findings validate the benefits of RETFound's additional retina-specific pretraining. Further research is needed to establish best practices for fine-tuning RETFound to downstream tasks.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

CT-based artificial intelligence system complementing deep learning model and radiologist for liver fibrosis staging

Noninvasive methods for liver fibrosis staging are urgently needed due to its significance in predicting significant morbidity and mortality. In this study, we developed an automated DL-based segmentation and classification model (Model-C). Test-time adaptation was used to address data distribution shifts. We then established a deep learning-radiologist complementarity decision system (DRCDS) via a decision model determining whether to adopt Model-C's diagnosis or defer to radiologists. Model-C (AUCs of 0.89-0.92) outperformed models based on liver (AUCs: 0.84-0.90) or spleen (AUCs: 0.69-0.70). With test-time adaptation, the Obuchowski index values of Model-C in three external sets improved from 0.81, 0.73, and 0.73 to 0.85, 0.85, and 0.81. DRCDS performed slightly better than Model-C or senior radiologists, with 73.7%-92.0% of cases adopting Model-C's diagnosis. In conclusion, DRCDS could diagnose liver fibrosis with high accuracy. Additionally, we provided solutions to model generalization and human-machine complementarity issues in multi-classification problems.

Performance and limitation of machine learning algorithms for diabetic retinopathy screening and its application in health management: a meta-analysis

BACKGROUND

In recent years, artificial intelligence and machine learning algorithms have been used more extensively to diagnose diabetic retinopathy and other diseases. Still, the effectiveness of these methods has not been thoroughly investigated. This study aimed to evaluate the performance and limitations of machine learning and deep learning algorithms in detecting diabetic retinopathy.

METHODS

This study was conducted based on the PRISMA checklist. We searched online databases, including PubMed, Scopus, and Google Scholar, for relevant articles up to September 30, 2023. After the title, abstract, and full-text screening, data extraction and quality assessment were done for the included studies. Finally, a meta-analysis was performed.

RESULTS

We included 76 studies with a total of 1,371,517 retinal images, of which 51 were used for meta-analysis. Our meta-analysis showed a significant sensitivity and specificity with a percentage of 90.54 (95%CI [90.42, 90.66], P < 0.001) and 78.33% (95%CI [78.21, 78.45], P < 0.001). However, the AUC (area under curvature) did not statistically differ across studies, but had a significant figure of 0.94 (95% CI [- 46.71, 48.60], P = 1).

CONCLUSIONS

Although machine learning and deep learning algorithms can properly diagnose diabetic retinopathy, their discriminating capacity is limited. However, they could simplify the diagnosing process. Further studies are required to improve algorithms.

Transforming Non-Digital, Clinical Workflows to Detect and Track Vision-Threatening Diabetic Retinopathy via a Digital Platform Integrating Artificial Intelligence: Implementation Research

INTRODUCTION

Screening diabetic retinopathy (DR) for timely management can reduce global blindness. Many existing DR screening programs worldwide are non-digital, standalone, and deployed with grading retinal photographs by trained personnel. To integrate the screening programs, with or without artificial intelligence (AI), into hospital information systems to improve their effectiveness, the non-digital workflow must be transformed into digital. We developed a cloud-based digital platform and implemented it in an existing DR screening program.

METHODS

We conducted the following processes in the platform for prospective DR screening at a community hospital: capturing patients' retinal photographs, uploading them for grading by AI or trained personnel on alternate weeks for 32 weeks, and referring vision-threatening DR to a referral center. At this center, the platform was applied for the assessment of potential missed referrals via remote over-reading by a retinal specialist and tracking referrals. Implementational outcomes, such as detecting positive cases, agreement between AI and over-reading, and referral adherence were assessed.

RESULTS

Of 645 patients screened by AI, 201 (31.2%) were referrals, 129 (64.2%) of which were true positives agreeable by over-reading; 115 of these true positives (89.1%) had referral adherence. False negatives judged by over-reading were 1.1% (5/444). Of 730 patients in manual screening, 175 (24.0%) were potential referrals, 11 (6.3%) of which were referred at the point-of-screening; eight of these (72.7%) adhered to referral. The remaining 164 cases were appointed for later examination by a visiting general ophthalmologist; 11 of these 116 examined (9.5%) were referred for non-DR-related eye conditions with 81.8% (9/11) referral adherence. No system failure or interruption was found.

CONCLUSIONS

The digital platform can be practically integrated into the existing non-digital DR screening programs to implement AI and monitor previously unknown but important indicators, such as referral adherence, to improve the effectiveness of the programs.

TRIAL REGISTRATION

ClinicalTrials.gov. (registration number: NCT05166122).

Multimodal imaging in diabetic retinopathy and macular edema: An update about biomarkers

Diabetic macular edema (DME), defined as retinal thickening near, or involving the fovea caused by fluid accumulation in the retina, can lead to vision impairment and blindness in patients with diabetes. Current knowledge of retina anatomy and function and DME pathophysiology has taken great advantage of the availability of several techniques for visualizing the retina. Combining these techniques in a multimodal imaging approach to DME is recommended to improve diagnosis and to guide treatment decisions. We review the recent literature about the following retinal imaging technologies: optical coherence tomography (OCT), OCT angiography (OCTA), wide-field and ultrawide-field techniques applied to fundus photography, fluorescein angiography, and OCTA. The emphasis will be on characteristic DME features identified by these imaging technologies and their potential or established role as diagnostic, prognostic, or predictive biomarkers. The role of artificial intelligence in the assessment and interpretation of retina images is also discussed.

Trends and hotspots in the field of diabetic retinopathy imaging research from 2000-2023

Background

Diabetic retinopathy (DR) poses a major threat to diabetic patients' vision and is a critical public health issue. Imaging applications for DR have grown since the 21st century, aiding diagnosis, grading, and screening. This study uses bibliometric analysis to assess the field's advancements and key areas of interest.

Methods

This study performed a bibliometric analysis of DR imaging articles collected from the Web of Science Core Collection database between January 1st, 2000, and December 31st, 2023. The literature information was then analyzed through CiteSpace.

Results

The United States and China led in the number of publications, with 719 and 609, respectively. The University of London topped the institution list with 139 papers. Tien Yin Wong was the most prolific researcher. Invest. Ophthalmol. Vis. Sci. published the most articles (105). Notable burst keywords were "deep learning," "artificial intelligence," et al.

Conclusion

The United States is at the forefront of DR research, with the University of London as the top institution and Invest. Ophthalmol. Vis. Sci. as the most published journal. Tien Yin Wong is the most influential researcher. Hotspots like "deep learning," and "artificial intelligence," have seen a significant rise, indicating artificial intelligence's growing role in DR imaging.

Discriminative, generative artificial intelligence, and foundation models in retina imaging

Recent advances of artificial intelligence (AI) in retinal imaging found its application in two major categories: discriminative and generative AI. For discriminative tasks, conventional convolutional neural networks (CNNs) are still major AI techniques. Vision transformers (ViT), inspired by the transformer architecture in natural language processing, has emerged as useful techniques for discriminating retinal images. ViT can attain excellent results when pretrained at sufficient scale and transferred to specific tasks with fewer images, compared to conventional CNN. Many studies found better performance of ViT, compared to CNN, for common tasks such as diabetic retinopathy screening on color fundus photographs (CFP) and segmentation of retinal fluid on optical coherence tomography (OCT) images. Generative Adversarial Network (GAN) is the main AI technique in generative AI in retinal imaging. Novel images generated by GAN can be applied for training AI models in imbalanced or inadequate datasets. Foundation models are also recent advances in retinal imaging. They are pretrained with huge datasets, such as millions of CFP and OCT images and fine-tuned for downstream tasks with much smaller datasets. A foundation model, RETFound, which was self-supervised and found to discriminate many eye and systemic diseases better than supervised models. Large language models are foundation models that may be applied for text-related tasks, like reports of retinal angiography. Whereas AI technology moves forward fast, real-world use of AI models moves slowly, making the gap between development and deployment even wider. Strong evidence showing AI models can prevent visual loss may be required to close this gap.

Effects of artificial intelligence implementation on efficiency in medical imaging-a systematic literature review and meta-analysis

In healthcare, integration of artificial intelligence (AI) holds strong promise for facilitating clinicians' work, especially in clinical imaging. We aimed to assess the impact of AI implementation for medical imaging on efficiency in real-world clinical workflows and conducted a systematic review searching six medical databases. Two reviewers double-screened all records. Eligible records were evaluated for methodological quality. The outcomes of interest were workflow adaptation due to AI implementation, changes in time for tasks, and clinician workload. After screening 13,756 records, we identified 48 original studies to be incuded in the review. Thirty-three studies measured time for tasks, with 67% reporting reductions. Yet, three separate meta-analyses of 12 studies did not show significant effects after AI implementation. We identified five different workflows adapting to AI use. Most commonly, AI served as a secondary reader for detection tasks. Alternatively, AI was used as the primary reader for identifying positive cases, resulting in reorganizing worklists or issuing alerts. Only three studies scrutinized workload calculations based on the time saved through AI use. This systematic review and meta-analysis represents an assessment of the efficiency improvements offered by AI applications in real-world clinical imaging, predominantly revealing enhancements across the studies. However, considerable heterogeneity in available studies renders robust inferences regarding overall effectiveness in imaging tasks. Further work is needed on standardized reporting, evaluation of system integration, and real-world data collection to better understand the technological advances of AI in real-world healthcare workflows. Systematic review registration: Prospero ID CRD42022303439, International Registered Report Identifier (IRRID): RR2-10.2196/40485.

Artificial Intelligence-Based Screening System for Diabetic Retinopathy in Primary Care

BACKGROUND

This study aimed to test an artificial intelligence-based reading system (AIRS) capable of reading retinographies of type 2 diabetic (T2DM) patients and a predictive algorithm (DRPA) that predicts the risk of each patient with T2DM of developing diabetic retinopathy (DR).

METHODS

We tested the ability of the AIRS to read and classify 15,297 retinal photographs from our database of diabetics and 1200 retinal images taken with Messidor-2 into the different DR categories. We tested the DRPA in a sample of 40,129 T2DM patients. The results obtained by the AIRS and the DRPA were then compared with those provided by four retina specialists regarding sensitivity (S), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), accuracy (ACC), and area under the curve (AUC).

RESULTS

The results of testing the AIRS for identifying referral DR (RDR) in our database were ACC = 98.6, S = 96.7, SP = 99.8, PPV = 99.0, NPV = 98.0, and AUC = 0.958, and in Messidor-2 were ACC = 96.78%, S = 94.64%, SP = 99.14%, PPV = 90.54%, NPV = 99.53%, and AUC = 0.918. The results of our DRPA when predicting the presence of any type of DR were ACC = 0.97, S = 0.89, SP = 0.98, PPV = 0.79, NPV = 0.98, and AUC = 0.92.

CONCLUSIONS

The AIRS performed well when reading and classifying the retinographies of T2DM patients with RDR. The DRPA performed well in predicting the absence of DR based on some clinical variables.

Accuracy of Autonomous Artificial Intelligence-Based Diabetic Retinopathy Screening in Real-Life Clinical Practice

Background: In diabetic retinopathy, early detection and intervention are crucial in preventing vision loss and improving patient outcomes. In the era of artificial intelligence (AI) and machine learning, new promising diagnostic tools have emerged. The IDX-DR machine (Digital Diagnostics, Coralville, IA, USA) represents a diagnostic tool that combines advanced imaging techniques, AI algorithms, and deep learning methodologies to identify and classify diabetic retinopathy. Methods: All patients that participated in our AI-based DR screening were considered for this study. For this study, all retinal images were additionally reviewed retrospectively by two experienced retinal specialists. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy were calculated for the IDX-DR machine compared to the graders' responses. Results: We included a total of 2282 images from 1141 patients who were screened between January 2021 and January 2023 at the Jules Gonin Eye Hospital in Lausanne, Switzerland. Sensitivity was calculated to be 100% for 'no DR', 'mild DR', and 'moderate DR'. Specificity for no DR', 'mild DR', 'moderate DR', and 'severe DR' was calculated to be, respectively, 78.4%, 81.2%, 93.4%, and 97.6%. PPV was calculated to be, respectively, 36.7%, 24.6%, 1.4%, and 0%. NPV was calculated to be 100% for each category. Accuracy was calculated to be higher than 80% for 'no DR', 'mild DR', and 'moderate DR'. Conclusions: In this study, based in Jules Gonin Eye Hospital in Lausanne, we compared the autonomous diagnostic AI system of the IDX-DR machine detecting diabetic retinopathy to human gradings established by two experienced retinal specialists. Our results showed that the ID-x DR machine constantly overestimates the DR stages, thus permitting the clinicians to fully trust negative results delivered by the screening software. Nevertheless, all fundus images classified as 'mild DR' or greater should always be controlled by a specialist in order to assert whether the predicted stage is truly present.

Hybrid deep learning models for the screening of Diabetic Macular Edema in optical coherence tomography volumes

Several studies published so far used highly selective image datasets from unclear sources to train computer vision models and that may lead to overestimated results, while those studies conducted in real-life remain scarce. To avoid image selection bias, we stacked convolutional and recurrent neural networks (CNN-RNN) to analyze complete optical coherence tomography (OCT) cubes in a row and predict diabetic macular edema (DME), in a real-world diabetic retinopathy screening program. A retrospective cohort study was carried out. Throughout 4-years, 5314 OCT cubes from 4408 subjects who attended to the diabetic retinopathy (DR) screening program were included. We arranged twenty-two (22) pre-trained CNNs in parallel with a bidirectional RNN layer stacked at the bottom, allowing the model to make a prediction for the whole OCT cube. The staff of retina experts built a ground truth of DME later used to train a set of these CNN-RNN models with different configurations. For each trained CNN-RNN model, we performed threshold tuning to find the optimal cut-off point for binary classification of DME. Finally, the best models were selected according to sensitivity, specificity, and area under the receiver operating characteristics curve (AUROC) with their 95% confidence intervals (95%CI). An ensemble of the best models was also explored. 5188 cubes were non-DME and 126 were DME. Three models achieved an AUROC of 0.94. Among these, sensitivity, and specificity (95%CI) ranged from 84.1-90.5 and 89.7-93.3, respectively, at threshold 1, from 89.7-92.1 and 80-83.1 at threshold 2, and from 80.2-81 and 93.8-97, at threshold 3. The ensemble model improved these results, and lower specificity was observed among subjects with sight-threatening DR. Analysis by age, gender, or grade of DME did not vary the performance of the models. CNN-RNN models showed high diagnostic accuracy for detecting DME in a real-world setting. This engine allowed us to detect extra-foveal DMEs commonly overlooked in other studies, and showed potential for application as the first filter of non-referable patients in an outpatient center within a population-based DR screening program, otherwise ended up in specialized care.

Big data to guide glaucoma treatment

Ophthalmology has been at the forefront of the medical application of big data. Often harnessed with a machine learning approach, big data has demonstrated potential to transform ophthalmic care, as evidenced by prior success on clinical tasks such as the screening of ophthalmic diseases and lesions via retinal images. With the recent establishment of various large ophthalmic datasets, there has been greater interest in determining whether the benefits of big data may extend to the downstream process of ophthalmic disease management. An area of substantial investigation has been the use of big data to help guide or streamline management of glaucoma, which remains a leading cause of irreversible blindness worldwide. In this review, we summarize relevant studies utilizing big data and discuss the application of the findings in the risk assessment and treatment of glaucoma.

Effectiveness of artificial intelligence for diabetic retinopathy screening in community in Binh Dinh Province, Vietnam

PURPOSE

The objective of this study is to evaluate the sensitivity, specificity, and accuracy of artificial intelligence (AI) for diabetic retinopathy (DR) screening in community in Binh Dinh Province in Vietnam.

MATERIALS AND METHODS

This retrospective, descriptive, cross-sectional study assessed the DR screening efficacy of EyeArt system v2.0 by analyzing 2332 nonmydriatic digital fundus pictures of 583 diabetic patients from hospitals and health centers in Binh Dinh province. First, we selected thirty patients with 120 digital fundus pictures to perform the Kappa index by two eye doctors who would be responsible for the DR clinical feature evaluation and DR severity scale classification. Second, all digital fundus pictures were coded and then sent to the two above-mentioned eye doctors for the evaluation and classifications according to the International Committee of Ophthalmology's guidelines. Finally, DR severity scales with EyeArt were compared with those by eye doctors as a reference standard for EyeArt's effectiveness. All the data were analyzed using the SPSS software version 20.0. Values (with confidence interval 95%) of sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were calculated according to DR state, referable or not and vision-threatening DR state or not. P < 0.05 was considered statistically significant.

RESULTS

The sensitivity and specificity of EyeArt for DR screening were 94.1% and 87.2%. The sensitivity and specificity for referable DR and vision-threatening DR were 96.6%, 90.1%, and 100.0%, 92.2%. Accuracy for DR screening, referable DR, and vision-threatening DR were 88.9%, 91.4%, and 93.0%, respectively.

CONCLUSION

EyeArt AI was effective for DR screening in community.

Diabetes management in the era of artificial intelligence

Artificial intelligence is growing quickly, and its application in the global diabetes pandemic has the potential to completely change the way this chronic illness is identified and treated. Machine learning methods have been used to construct algorithms supporting predictive models for the risk of getting diabetes or its complications. Social media and Internet forums also increase patient participation in diabetes care. Diabetes resource usage optimisation has benefited from technological improvements. As a lifestyle therapy intervention, digital therapies have made a name for themselves in the treatment of diabetes. Artificial intelligence will cause a paradigm shift in diabetes care, moving away from current methods and toward the creation of focused, data-driven precision treatment.

Head to head comparison of diagnostic performance of three non-mydriatic cameras for diabetic retinopathy screening with artificial intelligence

BACKGROUND

Diabetic Retinopathy (DR) is a leading cause of blindness worldwide, affecting people with diabetes. The timely diagnosis and treatment of DR are essential in preventing vision loss. Non-mydriatic fundus cameras and artificial intelligence (AI) software have been shown to improve DR screening efficiency. However, few studies have compared the diagnostic performance of different non-mydriatic cameras and AI software.

METHODS

This clinical study was conducted at the endocrinology clinic of Akdeniz University with 900 volunteer patients that were previously diagnosed with diabetes but not with diabetic retinopathy. Fundus images of each patient were taken using three non-mydriatic fundus cameras and EyeCheckup AI software was used to diagnose more than mild diabetic retinopathy, vision-threatening diabetic retinopathy, and clinically significant diabetic macular oedema using images from all three cameras. Then patients underwent dilation and 4 wide-field fundus photography. Three retina specialists graded the 4 wide-field fundus images according to the diabetic retinopathy treatment preferred practice patterns of the American Academy of Ophthalmology. The study was pre-registered on clinicaltrials.gov with the ClinicalTrials.gov Identifier: NCT04805541.

RESULTS

The Canon CR2 AF AF camera had a sensitivity and specificity of 95.65% / 95.92% for diagnosing more than mild DR, the Topcon TRC-NW400 had 95.19% / 96.46%, and the Optomed Aurora had 90.48% / 97.21%. For vision threatening diabetic retinopathy, the Canon CR2 AF had a sensitivity and specificity of 96.00% / 96.34%, the Topcon TRC-NW400 had 98.52% / 95.93%, and the Optomed Aurora had 95.12% / 98.82%. For clinically significant diabetic macular oedema, the Canon CR2 AF had a sensitivity and specificity of 95.83% / 96.83%, the Topcon TRC-NW400 had 98.50% / 96.52%, and the Optomed Aurora had 94.93% / 98.95%.

CONCLUSION

The study demonstrates the potential of using non-mydriatic fundus cameras combined with artificial intelligence software in detecting diabetic retinopathy. Several cameras were tested and, notably, each camera exhibited varying but adequate levels of sensitivity and specificity. The Canon CR2 AF emerged with the highest accuracy in identifying both more than mild diabetic retinopathy and vision-threatening cases, while the Topcon TRC-NW400 excelled in detecting clinically significant diabetic macular oedema. The findings from this study emphasize the importance of considering a non mydriatic camera and artificial intelligence software for diabetic retinopathy screening. However, further research is imperative to explore additional factors influencing the efficiency of diabetic retinopathy screening using AI and non mydriatic cameras such as costs involved and effects of screening using and on an ethnically diverse population.

Using generative AI to investigate medical imagery models and datasets

BACKGROUND

AI models have shown promise in performing many medical imaging tasks. However, our ability to explain what signals these models have learned is severely lacking. Explanations are needed in order to increase the trust of doctors in AI-based models, especially in domains where AI prediction capabilities surpass those of humans. Moreover, such explanations could enable novel scientific discovery by uncovering signals in the data that aren't yet known to experts.

METHODS

In this paper, we present a workflow for generating hypotheses to understand which visual signals in images are correlated with a classification model's predictions for a given task. This approach leverages an automatic visual explanation algorithm followed by interdisciplinary expert review. We propose the following 4 steps: (i) Train a classifier to perform a given task to assess whether the imagery indeed contains signals relevant to the task; (ii) Train a StyleGAN-based image generator with an architecture that enables guidance by the classifier ("StylEx"); (iii) Automatically detect, extract, and visualize the top visual attributes that the classifier is sensitive towards. For visualization, we independently modify each of these attributes to generate counterfactual visualizations for a set of images (i.e., what the image would look like with the attribute increased or decreased); (iv) Formulate hypotheses for the underlying mechanisms, to stimulate future research. Specifically, present the discovered attributes and corresponding counterfactual visualizations to an interdisciplinary panel of experts so that hypotheses can account for social and structural determinants of health (e.g., whether the attributes correspond to known patho-physiological or socio-cultural phenomena, or could be novel discoveries).

FINDINGS

To demonstrate the broad applicability of our approach, we present results on eight prediction tasks across three medical imaging modalities-retinal fundus photographs, external eye photographs, and chest radiographs. We showcase examples where many of the automatically-learned attributes clearly capture clinically known features (e.g., types of cataract, enlarged heart), and demonstrate automatically-learned confounders that arise from factors beyond physiological mechanisms (e.g., chest X-ray underexposure is correlated with the classifier predicting abnormality, and eye makeup is correlated with the classifier predicting low hemoglobin levels). We further show that our method reveals a number of physiologically plausible, previously-unknown attributes based on the literature (e.g., differences in the fundus associated with self-reported sex, which were previously unknown).

INTERPRETATION

Our approach enables hypotheses generation via attribute visualizations and has the potential to enable researchers to better understand, improve their assessment, and extract new knowledge from AI-based models, as well as debug and design better datasets. Though not designed to infer causality, importantly, we highlight that attributes generated by our framework can capture phenomena beyond physiology or pathophysiology, reflecting the real world nature of healthcare delivery and socio-cultural factors, and hence interdisciplinary perspectives are critical in these investigations. Finally, we will release code to help researchers train their own StylEx models and analyze their predictive tasks of interest, and use the methodology presented in this paper for responsible interpretation of the revealed attributes.

FUNDING

Google.

A novel approach for intelligent diagnosis and grading of diabetic retinopathy

Diabetic retinopathy (DR) is a severe ocular complication of diabetes that can lead to vision damage and even blindness. Currently, traditional deep convolutional neural networks (CNNs) used for DR grading tasks face two primary challenges: (1) insensitivity to minority classes due to imbalanced data distribution, and (2) neglecting the relationship between the left and right eyes by utilizing the fundus image of only one eye for training without differentiating between them. To tackle these challenges, we proposed the DRGCNN (DR Grading CNN) model. To solve the problem caused by imbalanced data distribution, our model adopts a more balanced strategy by allocating an equal number of channels to feature maps representing various DR categories. Furthermore, we introduce a CAM-EfficientNetV2-M encoder dedicated to encoding input retinal fundus images for feature vector generation. The number of parameters of our encoder is 52.88 M, which is less than RegNet_y_16gf (80.57 M) and EfficientNetB7 (63.79 M), but the corresponding kappa value is higher. Additionally, in order to take advantage of the binocular relationship, we input fundus retinal images from both eyes of the patient into the network for features fusion during the training phase. We achieved a kappa value of 86.62% on the EyePACS dataset and 86.16% on the Messidor-2 dataset. Experimental results on these representative datasets for diabetic retinopathy (DR) demonstrate the exceptional performance of our DRGCNN model, establishing it as a highly competitive intelligent classification model in the field of DR. The code is available for use at https://github.com/Fat-Hai/DRGCNN.

Addressing Technical Failures in a Diabetic Retinopathy Screening Program

Purpose

Diabetic retinopathy (DR) is a preventable cause of blindness detectable through screening using retinal digital photography. The Irish National Diabetic Retina Screening (DRS) programme, Diabetic RetinaScreen, provides free screening services to patients with diabetes from aged 12 years and older. A technical failure (TF) occurs when digital retinal imaging is ungradable, resulting in delays in the diagnosis and treatment of sight-threatening disease. Despite their impact, the causes of TFs, and indeed the utility of interventions to prevent them, have not been extensively examined.

Aim

Primary analysis aimed to identify factors associated with TF. Secondary analysis examined a subset of cases, assessing patient data from five time points between 2019 and 2021 to identify photographer/patient factors associated with TF.

Methods

Patient data from the DRS database for one provider were extracted for analysis between 2018 and 2022. Information on patient demographics, screening results, and other factors previously associated with TF were analyzed. Primary analysis involved using mixed-effects logistic regression models with nested patient-eye random effects. Secondary analysis reviewed a subset of cases in detail, checking for causes of TF.

Results

The primary analysis included a total of 366,528 appointments from 104,407 patients over 5 years. Most patients had Type 2 diabetes (89.2%), and the overall TF rate was 4.9%. Diabetes type and duration, dilate pupil status, and the presence of lens artefacts on the camera were significantly associated with TF. The Secondary analysis identified the primary cause of TF was found to be optically dense cataracts, accounting for over half of the TFs.

Conclusion

This study provides insight into the causes of TF within the Irish DRS program, highlighting cataracts as the primary contributing factor. The identification of patient-level factors associated with TF facilitates appropriate interventions that can be put in place to improve patient outcomes and minimize delays in treatment and diagnosis.

How Can Artificial Intelligence Be Implemented Effectively in Diabetic Retinopathy Screening in Japan?

Diabetic retinopathy (DR) is a major microvascular complication of diabetes, affecting a substantial portion of diabetic patients worldwide. Timely intervention is pivotal in mitigating the risk of blindness associated with DR, yet early detection remains a challenge due to the absence of early symptoms. Screening programs have emerged as a strategy to address this burden, and this paper delves into the role of artificial intelligence (AI) in advancing DR screening in Japan. There are two pathways for DR screening in Japan: a health screening pathway and a clinical referral path from physicians to ophthalmologists. AI technologies that realize automated image classification by applying deep learning are emerging. These technologies have exhibited substantial promise, achieving sensitivity and specificity levels exceeding 90% in prospective studies. Moreover, we introduce the potential of Generative AI and large language models (LLMs) to transform healthcare delivery, particularly in patient engagement, medical records, and decision support. Considering the use of AI in DR screening in Japan, we propose to follow a seven-step framework for systematic screening and emphasize the importance of integrating AI into a well-designed screening program. Automated scoring systems with AI enhance screening quality, but their effectiveness depends on their integration into the broader screening ecosystem. LLMs emerge as an important tool to fill gaps in the screening process, from personalized invitations to reporting results, facilitating a seamless and efficient system. However, it is essential to address concerns surrounding technical accuracy and governance before full-scale integration into the healthcare system. In conclusion, this review highlights the challenges in the current screening pathway and the potential for AI, particularly LLM, to revolutionize DR screening in Japan. The future direction will depend on leadership from ophthalmologists and stakeholders to address long-standing challenges in DR screening so that all people have access to accessible and effective screening.

Validation of a deep learning system for the detection of diabetic retinopathy in Indigenous Australians

BACKGROUND/AIMS

Deep learning systems (DLSs) for diabetic retinopathy (DR) detection show promising results but can underperform in racial and ethnic minority groups, therefore external validation within these populations is critical for health equity. This study evaluates the performance of a DLS for DR detection among Indigenous Australians, an understudied ethnic group who suffer disproportionately from DR-related blindness.

METHODS

We performed a retrospective external validation study comparing the performance of a DLS against a retinal specialist for the detection of more-than-mild DR (mtmDR), vision-threatening DR (vtDR) and all-cause referable DR. The validation set consisted of 1682 consecutive, single-field, macula-centred retinal photographs from 864 patients with diabetes (mean age 54.9 years, 52.4% women) at an Indigenous primary care service in Perth, Australia. Three-person adjudication by a panel of specialists served as the reference standard.

RESULTS

For mtmDR detection, sensitivity of the DLS was superior to the retina specialist (98.0% (95% CI, 96.5 to 99.4) vs 87.1% (95% CI, 83.6 to 90.6), McNemar's test p<0.001) with a small reduction in specificity (95.1% (95% CI, 93.6 to 96.4) vs 97.0% (95% CI, 95.9 to 98.0), p=0.006). For vtDR, the DLS's sensitivity was again superior to the human grader (96.2% (95% CI, 93.4 to 98.6) vs 84.4% (95% CI, 79.7 to 89.2), p<0.001) with a slight drop in specificity (95.8% (95% CI, 94.6 to 96.9) vs 97.8% (95% CI, 96.9 to 98.6), p=0.002). For all-cause referable DR, there was a substantial increase in sensitivity (93.7% (95% CI, 91.8 to 95.5) vs 74.4% (95% CI, 71.1 to 77.5), p<0.001) and a smaller reduction in specificity (91.7% (95% CI, 90.0 to 93.3) vs 96.3% (95% CI, 95.2 to 97.4), p<0.001).

CONCLUSION

The DLS showed improved sensitivity and similar specificity compared with a retina specialist for DR detection. This demonstrates its potential to support DR screening among Indigenous Australians, an underserved population with a high burden of diabetic eye disease.

Artificial Intelligence and Diabetic Retinopathy: AI Framework, Prospective Studies, Head-to-head Validation, and Cost-effectiveness

Current guidelines recommend that individuals with diabetes receive yearly eye exams for detection of referable diabetic retinopathy (DR), one of the leading causes of new-onset blindness. For addressing the immense screening burden, artificial intelligence (AI) algorithms have been developed to autonomously screen for DR from fundus photography without human input. Over the last 10 years, many AI algorithms have achieved good sensitivity and specificity (>85%) for detection of referable DR compared with human graders; however, many questions still remain. In this narrative review on AI in DR screening, we discuss key concepts in AI algorithm development as a background for understanding the algorithms. We present the AI algorithms that have been prospectively validated against human graders and demonstrate the variability of reference standards and cohort demographics. We review the limited head-to-head validation studies where investigators attempt to directly compare the available algorithms. Next, we discuss the literature regarding cost-effectiveness, equity and bias, and medicolegal considerations, all of which play a role in the implementation of these AI algorithms in clinical practice. Lastly, we highlight ongoing efforts to bridge gaps in AI model data sets to pursue equitable development and delivery.

Leveraging physiology and artificial intelligence to deliver advancements in health care

Artificial intelligence in health care has experienced remarkable innovation and progress in the last decade. Significant advancements can be attributed to the utilization of artificial intelligence to transform physiology data to advance health care. In this review, we explore how past work has shaped the field and defined future challenges and directions. In particular, we focus on three areas of development. First, we give an overview of artificial intelligence, with special attention to the most relevant artificial intelligence models. We then detail how physiology data have been harnessed by artificial intelligence to advance the main areas of health care: automating existing health care tasks, increasing access to care, and augmenting health care capabilities. Finally, we discuss emerging concerns surrounding the use of individual physiology data and detail an increasingly important consideration for the field, namely the challenges of deploying artificial intelligence models to achieve meaningful clinical impact.

Test accuracy of artificial intelligence-based grading of fundus images in diabetic retinopathy screening: A systematic review

OBJECTIVES

To systematically review the accuracy of artificial intelligence (AI)-based systems for grading of fundus images in diabetic retinopathy (DR) screening.

METHODS

We searched MEDLINE, EMBASE, the Cochrane Library and the ClinicalTrials.gov from 1st January 2000 to 27th August 2021. Accuracy studies published in English were included if they met the pre-specified inclusion criteria. Selection of studies for inclusion, data extraction and quality assessment were conducted by one author with a second reviewer independently screening and checking 20% of titles. Results were analysed narratively.

RESULTS

Forty-three studies evaluating 15 deep learning (DL) and 4 machine learning (ML) systems were included. Nine systems were evaluated in a single study each. Most studies were judged to be at high or unclear risk of bias in at least one QUADAS-2 domain. Sensitivity for referable DR and higher grades was ≥85% while specificity varied and was <80% for all ML systems and in 6/31 studies evaluating DL systems. Studies reported high accuracy for detection of ungradable images, but the latter were analysed and reported inconsistently. Seven studies reported that AI was more sensitive but less specific than human graders.

CONCLUSIONS

AI-based systems are more sensitive than human graders and could be safe to use in clinical practice but have variable specificity. However, for many systems evidence is limited, at high risk of bias and may not generalise across settings. Therefore, pre-implementation assessment in the target clinical pathway is essential to obtain reliable and applicable accuracy estimates.

Cross-camera Performance of Deep Learning Algorithms to Diagnose Common Ophthalmic Diseases: A Comparative Study Highlighting Feasibility to Portable Fundus Camera Use

PURPOSE

To compare the inter-camera performance and consistency of various deep learning (DL) diagnostic algorithms applied to fundus images taken from desktop Topcon and portable Optain cameras.

METHODS

Participants over 18 years of age were enrolled between November 2021 and April 2022. Pair-wise fundus photographs from each patient were collected in a single visit; once by Topcon (used as the reference camera) and once by a portable Optain camera (the new target camera). These were analyzed by three previously validated DL models for the detection of diabetic retinopathy (DR), age-related macular degeneration (AMD), and glaucomatous optic neuropathy (GON). Ophthalmologists manually analyzed all fundus photos for the presence of DR and these were referred to as the ground truth. Sensitivity, specificity, the area under the curve (AUC) and agreement between cameras (estimated by Cohen's weighted kappa, K) were the primary outcomes of this study.

RESULTS

A total of 504 patients were recruited. After excluding 12 photographs with matching errors and 59 photographs with low quality, 906 pairs of Topcon-Optain fundus photos were available for algorithm assessment. Topcon and Optain cameras had excellent consistency (Κ=0.80) when applied to the referable DR algorithm, while AMD had moderate consistency (Κ=0.41) and GON had poor consistency (Κ=0.32). For the DR model, Topcon and Optain achieved a sensitivity of 97.70% and 97.67% and a specificity of 97.92% and 97.93%, respectively. There was no significant difference between the two camera models (McNemar's test: x2=0.08, p = .78).

CONCLUSION

Topcon and Optain cameras had excellent consistency for detecting referable DR, albeit performances for detecting AMD and GON models were unsatisfactory. This study highlights the methods of using pair-wise images to evaluate DL models between reference and new fundus cameras.

Artificial intelligence for diabetic retinopathy in low-income and middle-income countries: a scoping review

Diabetic retinopathy (DR) is a leading cause of blindness globally. There is growing evidence to support the use of artificial intelligence (AI) in diabetic eye care, particularly for screening populations at risk of sight loss from DR in low-income and middle-income countries (LMICs) where resources are most stretched. However, implementation into clinical practice remains limited. We conducted a scoping review to identify what AI tools have been used for DR in LMICs and to report their performance and relevant characteristics. 81 articles were included. The reported sensitivities and specificities were generally high providing evidence to support use in clinical practice. However, the majority of studies focused on sensitivity and specificity only and there was limited information on cost, regulatory approvals and whether the use of AI improved health outcomes. Further research that goes beyond reporting sensitivities and specificities is needed prior to wider implementation.

Towards population-independent, multi-disease detection in fundus photographs

Independent validation studies of automatic diabetic retinopathy screening systems have recently shown a drop of screening performance on external data. Beyond diabetic retinopathy, this study investigates the generalizability of deep learning (DL) algorithms for screening various ocular anomalies in fundus photographs, across heterogeneous populations and imaging protocols. The following datasets are considered: OPHDIAT (France, diabetic population), OphtaMaine (France, general population), RIADD (India, general population) and ODIR (China, general population). Two multi-disease DL algorithms were developed: a Single-Dataset (SD) network, trained on the largest dataset (OPHDIAT), and a Multiple-Dataset (MD) network, trained on multiple datasets simultaneously. To assess their generalizability, both algorithms were evaluated whenever training and test data originate from overlapping datasets or from disjoint datasets. The SD network achieved a mean per-disease area under the receiver operating characteristic curve (mAUC) of 0.9571 on OPHDIAT. However, it generalized poorly to the other three datasets (mAUC < 0.9). When all four datasets were involved in training, the MD network significantly outperformed the SD network (p = 0.0058), indicating improved generality. However, in leave-one-dataset-out experiments, performance of the MD network was significantly lower on populations unseen during training than on populations involved in training (p < 0.0001), indicating imperfect generalizability.

A Systematic Review and Meta-Analysis of Applying Deep Learning in the Prediction of the Risk of Cardiovascular Diseases From Retinal Images

Purpose

The purpose of this study was to perform a systematic review and meta-analysis to synthesize evidence from studies using deep learning (DL) to predict cardiovascular disease (CVD) risk from retinal images.

Methods

A systematic literature search was performed in MEDLINE, Scopus, and Web of Science up to June 2022. We extracted data pertaining to predicted outcomes, model development, and validation and model performance metrics. Included studies were graded using the Quality Assessment of Diagnostic Accuracies Studies 2 tool. Model performance was pooled across eligible studies using a random-effects meta-analysis model.

Results

A total of 26 studies were included in the analysis. There were 42 CVD risk-related outcomes predicted from retinal images were identified, including 33 CVD risk factors, 4 cardiac imaging biomarkers, 2 CVD risk scores, the presence of CVD, and incident CVD. Three studies that aimed to predict the development of future CVD events reported an area under the receiver operating curve (AUROC) between 0.68 and 0.81. Models that used retinal images as input data had a pooled mean absolute error of 3.19 years (95% confidence interval [CI] = 2.95-3.43) for age prediction; a pooled AUROC of 0.96 (95% CI = 0.95-0.97) for gender classification; a pooled AUROC of 0.80 (95% CI = 0.73-0.86) for diabetes detection; and a pooled AUROC of 0.86 (95% CI = 0.81-0.92) for the detection of chronic kidney disease. We observed a high level of heterogeneity and variation in study designs.

Conclusions

Although DL models appear to have reasonably good performance when it comes to predicting CVD risk, further work is necessary to evaluate the real-world applicability and predictive accuracy.

Translational Relevance

DL-based CVD risk assessment from retinal images holds great promise to be translated to clinical practice as a novel approach for CVD risk assessment, given its simple, quick, and noninvasive nature.

Enhancing the reliability and accuracy of AI-enabled diagnosis via complementarity-driven deferral to clinicians

Predictive artificial intelligence (AI) systems based on deep learning have been shown to achieve expert-level identification of diseases in multiple medical imaging settings, but can make errors in cases accurately diagnosed by clinicians and vice versa. We developed Complementarity-Driven Deferral to Clinical Workflow (CoDoC), a system that can learn to decide between the opinion of a predictive AI model and a clinical workflow. CoDoC enhances accuracy relative to clinician-only or AI-only baselines in clinical workflows that screen for breast cancer or tuberculosis (TB). For breast cancer screening, compared to double reading with arbitration in a screening program in the UK, CoDoC reduced false positives by 25% at the same false-negative rate, while achieving a 66% reduction in clinician workload. For TB triaging, compared to standalone AI and clinical workflows, CoDoC achieved a 5-15% reduction in false positives at the same false-negative rate for three of five commercially available predictive AI systems. To facilitate the deployment of CoDoC in novel futuristic clinical settings, we present results showing that CoDoC's performance gains are sustained across several axes of variation (imaging modality, clinical setting and predictive AI system) and discuss the limitations of our evaluation and where further validation would be needed. We provide an open-source implementation to encourage further research and application.

A bibliometric analysis of artificial intelligence applications in macular edema: exploring research hotspots and Frontiers

Background: Artificial intelligence (AI) is used in ophthalmological disease screening and diagnostics, medical image diagnostics, and predicting late-disease progression rates. We reviewed all AI publications associated with macular edema (ME) research Between 2011 and 2022 and performed modeling, quantitative, and qualitative investigations. Methods: On 1st February 2023, we screened the Web of Science Core Collection for AI applications related to ME, from which 297 studies were identified and analyzed (2011-2022). We collected information on: publications, institutions, country/region, keywords, journal name, references, and research hotspots. Literature clustering networks and Frontier knowledge bases were investigated using bibliometrix-BiblioShiny, VOSviewer, and CiteSpace bibliometric platforms. We used the R "bibliometrix" package to synopsize our observations, enumerate keywords, visualize collaboration networks between countries/regions, and generate a topic trends plot. VOSviewer was used to examine cooperation between institutions and identify citation relationships between journals. We used CiteSpace to identify clustering keywords over the timeline and identify keywords with the strongest citation bursts. Results: In total, 47 countries published AI studies related to ME; the United States had the highest H-index, thus the greatest influence. China and the United States cooperated most closely between all countries. Also, 613 institutions generated publications - the Medical University of Vienna had the highest number of studies. This publication record and H-index meant the university was the most influential in the ME field. Reference clusters were also categorized into 10 headings: retinal Optical Coherence Tomography (OCT) fluid detection, convolutional network models, deep learning (DL)-based single-shot predictions, retinal vascular disease, diabetic retinopathy (DR), convolutional neural networks (CNNs), automated macular pathology diagnosis, dry age-related macular degeneration (DARMD), class weight, and advanced DL architecture systems. Frontier keywords were represented by diabetic macular edema (DME) (2021-2022). Conclusion: Our review of the AI-related ME literature was comprehensive, systematic, and objective, and identified future trends and current hotspots. With increased DL outputs, the ME research focus has gradually shifted from manual ME examinations to automatic ME detection and associated symptoms. In this review, we present a comprehensive and dynamic overview of AI in ME and identify future research areas.

Evaluating the Efficacy of Teleophthalmology in Delivering Ophthalmic Care to Underserved Populations: A Literature Review

Technological advancement has brought commendable changes in medicine, advancing diagnosis, treatment, and interventions. Telemedicine has been adopted by various subspecialties including ophthalmology. Over the years, teleophthalmology has been implemented in various countries, and continuous progress is being made in this area. In underserved populations, due to socioeconomic factors, there is little or no access to healthcare facilities, and people are at higher risk of eye diseases and vision impairment. Transportation is the major hurdle for these people in obtaining access to eye care in the main hospitals. There is a dire need for accessible eye care for such populations, and teleophthalmology is the ray of hope for providing eye care facilities to underserved people. Numerous studies have reported the advantages of teleophthalmology for rural populations such as being cost-effective, timesaving, reliable, efficient, and satisfactory for patients. Although it is being practiced in urban populations, for rural populations, its benefits amplify. However, there are certain obstacles as well, such as the cost of equipment, lack of steady electricity and internet supply in rural areas, and the attitude of people in certain regions toward acceptance of teleophthalmology. In this review, we have discussed in detail eye health in rural populations, teleophthalmology, and its effectiveness in rural populations of different countries.

Cost-Utility Analysis of Deep Learning and Trained Human Graders for Diabetic Retinopathy Screening in a Nationwide Program

INTRODUCTION

Deep learning (DL) for screening diabetic retinopathy (DR) has the potential to address limited healthcare resources by enabling expanded access to healthcare. However, there is still limited health economic evaluation, particularly in low- and middle-income countries, on this subject to aid decision-making for DL adoption.

METHODS

In the context of a middle-income country (MIC), using Thailand as a model, we constructed a decision tree-Markov hybrid model to estimate lifetime costs and outcomes of Thailand's national DR screening program via DL and trained human graders (HG). We calculated the incremental cost-effectiveness ratio (ICER) between the two strategies. Sensitivity analyses were performed to probe the influence of modeling parameters.

RESULTS

From a societal perspective, screening with DL was associated with a reduction in costs of ~ US$ 2.70, similar quality-adjusted life-years (QALY) of + 0.0043, and an incremental net monetary benefit of ~ US$ 24.10 in the base case. In sensitivity analysis, DL remained cost-effective even with a price increase from US$ 1.00 to US$ 4.00 per patient at a Thai willingness-to-pay threshold of ~ US$ 4.997 per QALY gained. When further incorporating recent findings suggesting improved compliance to treatment referral with DL, our analysis models effectiveness benefits of ~ US$ 20 to US$ 50 depending on compliance.

CONCLUSION

DR screening using DL in an MIC using Thailand as a model may result in societal cost-savings and similar health outcomes compared with HG. This study may provide an economic rationale to expand DL-based DR screening in MICs as an alternative solution for limited availability of skilled human resources for primary screening, particularly in MICs with similar prevalence of diabetes and low compliance to referrals for treatment.

Artificial Intelligence for Diabetic Retinopathy Screening Using Color Retinal Photographs: From Development to Deployment

Diabetic retinopathy (DR), a leading cause of preventable blindness, is expected to remain a growing health burden worldwide. Screening to detect early sight-threatening lesions of DR can reduce the burden of vision loss; nevertheless, the process requires intensive manual labor and extensive resources to accommodate the increasing number of patients with diabetes. Artificial intelligence (AI) has been shown to be an effective tool which can potentially lower the burden of screening DR and vision loss. In this article, we review the use of AI for DR screening on color retinal photographs in different phases of application, ranging from development to deployment. Early studies of machine learning (ML)-based algorithms using feature extraction to detect DR achieved a high sensitivity but relatively lower specificity. Robust sensitivity and specificity were achieved with the application of deep learning (DL), although ML is still used in some tasks. Public datasets were utilized in retrospective validations of the developmental phases in most algorithms, which require a large number of photographs. Large prospective clinical validation studies led to the approval of DL for autonomous screening of DR although the semi-autonomous approach may be preferable in some real-world settings. There have been few reports on real-world implementations of DL for DR screening. It is possible that AI may improve some real-world indicators for eye care in DR, such as increased screening uptake and referral adherence, but this has not been proven. The challenges in deployment may include workflow issues, such as mydriasis to lower ungradable cases; technical issues, such as integration into electronic health record systems and integration into existing camera systems; ethical issues, such as data privacy and security; acceptance of personnel and patients; and health-economic issues, such as the need to conduct health economic evaluations of using AI in the context of the country. The deployment of AI for DR screening should follow the governance model for AI in healthcare which outlines four main components: fairness, transparency, trustworthiness, and accountability.

Artificial Intelligence Software for Diabetic Eye Screening: Diagnostic Performance and Impact of Stratification

AIM

To evaluate the MONA.health artificial intelligence screening software for detecting referable diabetic retinopathy (DR) and diabetic macular edema (DME), including subgroup analysis.

METHODS

The algorithm's threshold value was fixed at the 90% sensitivity operating point on the receiver operating curve to perform the disease classification. Diagnostic performance was appraised on a private test set and publicly available datasets. Stratification analysis was executed on the private test set considering age, ethnicity, sex, insulin dependency, year of examination, camera type, image quality, and dilatation status.

RESULTS

The software displayed an area under the curve (AUC) of 97.28% for DR and 98.08% for DME on the private test set. The specificity and sensitivity for combined DR and DME predictions were 94.24 and 90.91%, respectively. The AUC ranged from 96.91 to 97.99% on the publicly available datasets for DR. AUC values were above 95% in all subgroups, with lower predictive values found for individuals above the age of 65 (82.51% sensitivity) and Caucasians (84.03% sensitivity).

CONCLUSION

We report good overall performance of the MONA.health screening software for DR and DME. The software performance remains stable with no significant deterioration of the deep learning models in any studied strata.

Automated machine learning-based classification of proliferative and non-proliferative diabetic retinopathy using optical coherence tomography angiography vascular density maps

PURPOSE

The study aims to classify the eyes with proliferative diabetic retinopathy (PDR) and non-proliferative diabetic retinopathy (NPDR) based on the optical coherence tomography angiography (OCTA) vascular density maps using a supervised machine learning algorithm.

METHODS

OCTA vascular density maps (at superficial capillary plexus (SCP), deep capillary plexus (DCP), and total retina (R) levels) of 148 eyes from 78 patients with diabetic retinopathy (45 PDR and 103 NPDR) was used to classify the images to NPDR and PDR groups based on a supervised machine learning algorithm known as the support vector machine (SVM) classifier optimized by a genetic evolutionary algorithm.

RESULTS

The implemented algorithm in three different models reached up to 85% accuracy in classifying PDR and NPDR in all three levels of vascular density maps. The deep retinal layer vascular density map demonstrated the best performance with a 90% accuracy in discriminating between PDR and NPDR.

CONCLUSIONS

The current study on a limited number of patients with diabetic retinopathy demonstrated that a supervised machine learning-based method known as SVM can be used to differentiate PDR and NPDR patients using OCTA vascular density maps.

A Sociotechnical Systems Framework for the Application of Artificial Intelligence in Health Care Delivery

In the coming years, artificial intelligence (AI) will pervade almost every aspect of the health care delivery system. AI has the potential to improve patient safety (e.g. diagnostic accuracy) as well as reduce the burden on clinicians (e.g. documentation-related workload); however, these benefits are yet to be realized. AI is only one element of a larger sociotechnical system that needs to be considered for effective AI application. In this paper, we describe the current challenges of integrating AI into clinical care and propose a sociotechnical systems (STS) approach for AI design and implementation. We demonstrate the importance of an STS approach through a case study on the design and implementation of a clinical decision support (CDS). In order for AI to reach its potential, the entire work system as well as clinical workflow must be systematically considered throughout the design of AI technology.

Custom Hardware Architectures for Deep Learning on Portable Devices: A Review

The staggering innovations and emergence of numerous deep learning (DL) applications have forced researchers to reconsider hardware architecture to accommodate fast and efficient application-specific computations. Applications, such as object detection, image recognition, speech translation, as well as music synthesis and image generation, can be performed with high accuracy at the expense of substantial computational resources using DL. Furthermore, the desire to adopt Industry 4.0 and smart technologies within the Internet of Things infrastructure has initiated several studies to enable on-chip DL capabilities for resource-constrained devices. Specialized DL processors reduce dependence on cloud servers, improve privacy, lessen latency, and mitigate bandwidth congestion. As we reach the limits of shrinking transistors, researchers are exploring various application-specific hardware architectures to meet the performance and efficiency requirements for DL tasks. Over the past few years, several software optimizations and hardware innovations have been proposed to efficiently perform these computations. In this article, we review several DL accelerators, as well as technologies with emerging devices, to highlight their architectural features in application-specific integrated circuit (IC) and field-programmable gate array (FPGA) platforms. Finally, the design considerations for DL hardware in portable applications have been discussed, along with some deductions about the future trends and potential research directions to innovate DL accelerator architectures further. By compiling this review, we expect to help aspiring researchers widen their knowledge in custom hardware architectures for DL.

Influence of Different Types of Retinal Cameras on the Performance of Deep Learning Algorithms in Diabetic Retinopathy Screening

Background: The aim of this study was to assess the performance of regional graders and artificial intelligence algorithms across retinal cameras with different specifications in classifying an image as gradable and ungradable. Methods: Study subjects were included from a community-based nationwide diabetic retinopathy screening program in Thailand. Various non-mydriatic fundus cameras were used for image acquisition, including Kowa Nonmyd, Kowa Nonmyd α-DⅢ, Kowa Nonmyd 7, Kowa Nonmyd WX, Kowa VX 10 α, Kowa VX 20 and Nidek AFC 210. All retinal photographs were graded by deep learning algorithms and human graders and compared with a standard reference. Results: Images were divided into two categories as gradable and ungradable images. Four thousand eight hundred fifty-two participants with 19,408 fundus images were included, of which 15,351 (79.09%) were gradable images and the remaining 4057 (20.90%) were ungradable images. Conclusions: The deep learning (DL) algorithm demonstrated better sensitivity, specificity and kappa than the human graders for all eight types of non-mydriatic fundus cameras. The deep learning system showed, more consistent diagnostic performance than the human graders across images of varying quality and camera types.

Deep Learning for Diabetic Retinopathy Analysis: A Review, Research Challenges, and Future Directions

Deep learning (DL) enables the creation of computational models comprising multiple processing layers that learn data representations at multiple levels of abstraction. In the recent past, the use of deep learning has been proliferating, yielding promising results in applications across a growing number of fields, most notably in image processing, medical image analysis, data analysis, and bioinformatics. DL algorithms have also had a significant positive impact through yielding improvements in screening, recognition, segmentation, prediction, and classification applications across different domains of healthcare, such as those concerning the abdomen, cardiac, pathology, and retina. Given the extensive body of recent scientific contributions in this discipline, a comprehensive review of deep learning developments in the domain of diabetic retinopathy (DR) analysis, viz., screening, segmentation, prediction, classification, and validation, is presented here. A critical analysis of the relevant reported techniques is carried out, and the associated advantages and limitations highlighted, culminating in the identification of research gaps and future challenges that help to inform the research community to develop more efficient, robust, and accurate DL models for the various challenges in the monitoring and diagnosis of DR.

R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 9

In this latest update we highlight a recent International Society of Pharmacoeconomics and Outcomes Research Good Practice Report on machine learning (ML) for health economics and outcomes research. We specifically discuss use cases of ML that offer opportunities in the generation of evidence using real-world data, including improvements in the identification of study cohorts, confounder identification and adjustment and estimating treatment effect heterogeneity. Barriers to the wider adoption of ML methods are also discussed.

Automated image curation in diabetic retinopathy screening using deep learning

Diabetic retinopathy (DR) screening images are heterogeneous and contain undesirable non-retinal, incorrect field and ungradable samples which require curation, a laborious task to perform manually. We developed and validated single and multi-output laterality, retinal presence, retinal field and gradability classification deep learning (DL) models for automated curation. The internal dataset comprised of 7743 images from DR screening (UK) with 1479 external test images (Portugal and Paraguay). Internal vs external multi-output laterality AUROC were right (0.994 vs 0.905), left (0.994 vs 0.911) and unidentifiable (0.996 vs 0.680). Retinal presence AUROC were (1.000 vs 1.000). Retinal field AUROC were macula (0.994 vs 0.955), nasal (0.995 vs 0.962) and other retinal field (0.997 vs 0.944). Gradability AUROC were (0.985 vs 0.918). DL effectively detects laterality, retinal presence, retinal field and gradability of DR screening images with generalisation between centres and populations. DL models could be used for automated image curation within DR screening.

Determinants of the implementation of an artificial intelligence-supported device for the screening of diabetic retinopathy in primary care - a qualitative study

Diabetic retinopathy is a microvascular complication of diabetes mellitus that is usually asymptomatic in the early stages. Therefore, its timely detection and treatment are essential. First pilot projects exist to establish a smartphone-based and AI-supported screening of DR in primary care. This study explored health professionals' perceptions of potential barriers and enablers of using a screening such as this in primary care to understand the mechanisms that could influence implementation into routine clinical practice. Semi-structured telephone interviews were conducted and analysed with the help of qualitative analysis of Mayring. The following main influencing factors to implementation have been identified: personal attitude, organisation, time, financial factors, education, support, technical requirement, influence on profession and patient welfare. Most determinants could be relocated in the behaviour change wheel, a validated implementation model. Further research on the patients' perspective and a ranking of the determinants found is needed.

Systematic Bibliometric and Visualized Analysis of Research Hotspots and Trends on the Application of Artificial Intelligence in Ophthalmic Disease Diagnosis

Background: Artificial intelligence (AI) has been used in the research of ophthalmic disease diagnosis, and it may have an impact on medical and ophthalmic practice in the future. This study explores the general application and research frontier of artificial intelligence in ophthalmic disease detection. Methods: Citation data were downloaded from the Web of Science Core Collection database to evaluate the extent of the application of Artificial intelligence in ophthalmic disease diagnosis in publications from 1 January 2012, to 31 December 2021. This information was analyzed using CiteSpace.5.8. R3 and Vosviewer. Results: A total of 1,498 publications from 95 areas were examined, of which the United States was determined to be the most influential country in this research field. The largest cluster labeled "Brownian motion" was used prior to the application of AI for ophthalmic diagnosis from 2007 to 2017, and was an active topic during this period. The burst keywords in the period from 2020 to 2021 were system, disease, and model. Conclusion: The focus of artificial intelligence research in ophthalmic disease diagnosis has transitioned from the development of AI algorithms and the analysis of abnormal eye physiological structure to the investigation of more mature ophthalmic disease diagnosis systems. However, there is a need for further studies in ophthalmology and computer engineering.

Deep Learning to Detect OCT-derived Diabetic Macular Edema from Color Retinal Photographs: A Multicenter Validation Study

PURPOSE

To validate the generalizability of a deep learning system (DLS) that detects diabetic macular edema (DME) from 2-dimensional color fundus photographs (CFP), for which the reference standard for retinal thickness and fluid presence is derived from 3-dimensional OCT.

DESIGN

Retrospective validation of a DLS across international datasets.

PARTICIPANTS

Paired CFP and OCT of patients from diabetic retinopathy (DR) screening programs or retina clinics. The DLS was developed using data sets from Thailand, the United Kingdom, and the United States and validated using 3060 unique eyes from 1582 patients across screening populations in Australia, India, and Thailand. The DLS was separately validated in 698 eyes from 537 screened patients in the United Kingdom with mild DR and suspicion of DME based on CFP.

METHODS

The DLS was trained using DME labels from OCT. The presence of DME was based on retinal thickening or intraretinal fluid. The DLS's performance was compared with expert grades of maculopathy and to a previous proof-of-concept version of the DLS. We further simulated the integration of the current DLS into an algorithm trained to detect DR from CFP.

MAIN OUTCOME MEASURES

The superiority of specificity and noninferiority of sensitivity of the DLS for the detection of center-involving DME, using device-specific thresholds, compared with experts.

RESULTS

The primary analysis in a combined data set spanning Australia, India, and Thailand showed the DLS had 80% specificity and 81% sensitivity, compared with expert graders, who had 59% specificity and 70% sensitivity. Relative to human experts, the DLS had significantly higher specificity (P = 0.008) and noninferior sensitivity (P < 0.001). In the data set from the United Kingdom, the DLS had a specificity of 80% (P < 0.001 for specificity of >50%) and a sensitivity of 100% (P = 0.02 for sensitivity of > 90%).

CONCLUSIONS

The DLS can generalize to multiple international populations with an accuracy exceeding that of experts. The clinical value of this DLS to reduce false-positive referrals, thus decreasing the burden on specialist eye care, warrants a prospective evaluation.