An observational study to assess if automated diabetic retinopathy image assessment software can replace one or more steps of manual imaging grading and to determine their cost-effectiveness

Künstliche Intelligenz (KI) zur Früherkennung von Netzhauterkrankungen

Künstliche Intelligenz (KI) hat sich zu einem transformativen Werkzeug auf dem Gebiet der Augenheilkunde entwickelt und revolutioniert die Diagnose und Behandlung von Krankheiten. Diese Arbeit gibt einen umfassenden Überblick über KI-Anwendungen bei verschiedenen Netzhauterkrankungen und zeigt ihr Potenzial, die Effizienz von Vorsorgeuntersuchungen zu erhöhen, Frühdiagnosen zu erleichtern und die Patientenergebnisse zu verbessern. Wir erklären die grundlegenden Konzepte der KI, einschließlich des maschinellen Lernens (ML) und des Deep Learning (DL), und deren Anwendung in der Augenheilkunde und heben die Bedeutung von KI-basierten Lösungen bei der Bewältigung der Komplexität und Variabilität von Netzhauterkrankungen hervor. Wir gehen auch auf spezifische Anwendungen der KI im Zusammenhang mit Netzhauterkrankungen wie diabetischer Retinopathie (DR), altersbedingter Makuladegeneration (AMD), makulärer Neovaskularisation, Frühgeborenen-Retinopathie (ROP), retinalem Venenverschluss (RVO), hypertensiver Retinopathie (HR), Retinopathia pigmentosa, Morbus Stargardt, Morbus Best (Best’sche vitelliforme Makuladystrophie) und Sichelzellenretinopathie ein. Wir konzen­trieren uns auf die aktuelle Landschaft der KI-Technologien, einschließlich verschiedener KI-Modelle, ihrer Leistungsmetriken und klinischen Implikationen. Darüber hinaus befassen wir uns mit den Herausforderungen und Schwierigkeiten bei der Integration von KI in die klinische Praxis, einschließlich des «Black-Box-Phänomens», der Verzerrungen bei der Darstellung von Daten und der Einschränkungen im Zusammenhang mit der ganzheitlichen Bewertung von Patienten. Abschließend wird die kollaborative Rolle der KI an der Seite des medizinischen Fachpersonals hervorgehoben, wobei ein synergetischer Ansatz für die Erbringung von Gesundheitsdienstleistungen befürwortet wird. Es wird betont, wie wichtig es ist, KI als Ergänzung und nicht als Ersatz für menschliche Expertise einzusetzen, um ihr Potenzial zu maximieren, die Gesundheitsversorgung zu revolutionieren, Ungleichheiten in der Gesundheitsversorgung zu verringern und die Patientenergebnisse in der sich entwickelnden medizinischen Landschaft zu verbessern.

Generating evidence to support the role of AI in diabetic eye screening: considerations from the UK National Screening Committee

Screening for diabetic retinopathy has been shown to reduce the risk of sight loss in people with diabetes, because of early detection and treatment of sight-threatening disease. There is long-standing interest in the possibility of automating parts of this process through artificial intelligence, commonly known as automated retinal imaging analysis software (ARIAS). A number of such products are now on the market. In the UK, Scotland has used a rules-based autograder since 2011, but the diabetic eye screening programmes in the rest of the UK rely solely on human graders. With more sophisticated machine learning-based ARIAS now available and greater challenges in terms of human grader capacity, in 2019 the UK's National Screening Committee (NSC) was asked to consider the modification of diabetic eye screening in England with ARIAS. Following up on a review of ARIAS research highlighting the strengths and limitations of existing evidence, the NSC here sets out their considerations for evaluating evidence to support the introduction of ARIAS into the diabetic eye screening programme.

Patient and practitioner perceptions around use of artificial intelligence within the English NHS diabetic eye screening programme

AIMS

Automated retinal image analysis using Artificial Intelligence (AI) can detect diabetic retinopathy as accurately as human graders, but it is not yet licensed in the NHS Diabetic Eye Screening Programme (DESP) in England. This study aims to assess perceptions of People Living with Diabetes (PLD) and Healthcare Practitioners (HCP) towards AI's introduction in DESP.

METHODS

Two online surveys were co-developed with PLD and HCP from a diverse DESP in North East London. Surveys were validated through interviews across three centres and distributed via DESP centres, charities, and the British Association of Retinal Screeners. A coding framework was used to analyse free-text responses.

RESULTS

387 (24%) PLD and 98 (37%) HCP provided comments. Themes included trust, workforce impact, the patient-practitioner relationship, AI implementation challenges, and inequalities. Both groups agreed AI in DESP was inevitable, would improve efficiency, and save costs. Concerns included job losses, data security, and AI decision safety. A common misconception was that AI would directly affect patient interactions, though it only processes retinal images.

CONCLUSIONS

Limited understanding of AI was a barrier to acceptance. Educating diverse PLD groups and HCP about AI's accuracy and reliability is crucial to building trust and facilitating its integration into screening practices.

Harnessing the medical undergraduate human resource for screening of sight-threatening diabetic retinopathy

PURPOSE

To assess whether medical undergraduates can be trained to effectively screen diabetic retinopathy (DR) by statistical comparison with a retina specialist at a tertiary eye care center in India.

METHODS

Three final-year undergraduate medical students, having completed ophthalmology department rotation, received training from a retina specialist for grading DR, following which they were asked to grade a set of 50 fundus photographs centered on the macula with a view of 50° as sight-threatening DR (STDR), diabetic macular edema, and grade of DR. Agreement among the undergraduates and retina specialist was determined with the help of Cohen's Kappa coefficient.

RESULTS

Kappa coefficient between undergraduates for detection of STDR ranged from 0.695 to 0.817 and between each student and the retina specialist ranged from 0.663 to 0.712. The sensitivity and specificity for undergraduates' and retina specialist's detection of STDR were 93.93%-96.96% and 60%, respectively.

CONCLUSION

There was substantial agreement among the undergraduates as well as between the undergraduates and the retina specialist for the detection of STDR. Undergraduates also detected STDR with a high sensitivity. This study outlines the feasibility of training undergraduate students for screening DR.

Head to head comparison of diagnostic performance of three non-mydriatic cameras for diabetic retinopathy screening with artificial intelligence

BACKGROUND

Diabetic Retinopathy (DR) is a leading cause of blindness worldwide, affecting people with diabetes. The timely diagnosis and treatment of DR are essential in preventing vision loss. Non-mydriatic fundus cameras and artificial intelligence (AI) software have been shown to improve DR screening efficiency. However, few studies have compared the diagnostic performance of different non-mydriatic cameras and AI software.

METHODS

This clinical study was conducted at the endocrinology clinic of Akdeniz University with 900 volunteer patients that were previously diagnosed with diabetes but not with diabetic retinopathy. Fundus images of each patient were taken using three non-mydriatic fundus cameras and EyeCheckup AI software was used to diagnose more than mild diabetic retinopathy, vision-threatening diabetic retinopathy, and clinically significant diabetic macular oedema using images from all three cameras. Then patients underwent dilation and 4 wide-field fundus photography. Three retina specialists graded the 4 wide-field fundus images according to the diabetic retinopathy treatment preferred practice patterns of the American Academy of Ophthalmology. The study was pre-registered on clinicaltrials.gov with the ClinicalTrials.gov Identifier: NCT04805541.

RESULTS

The Canon CR2 AF AF camera had a sensitivity and specificity of 95.65% / 95.92% for diagnosing more than mild DR, the Topcon TRC-NW400 had 95.19% / 96.46%, and the Optomed Aurora had 90.48% / 97.21%. For vision threatening diabetic retinopathy, the Canon CR2 AF had a sensitivity and specificity of 96.00% / 96.34%, the Topcon TRC-NW400 had 98.52% / 95.93%, and the Optomed Aurora had 95.12% / 98.82%. For clinically significant diabetic macular oedema, the Canon CR2 AF had a sensitivity and specificity of 95.83% / 96.83%, the Topcon TRC-NW400 had 98.50% / 96.52%, and the Optomed Aurora had 94.93% / 98.95%.

CONCLUSION

The study demonstrates the potential of using non-mydriatic fundus cameras combined with artificial intelligence software in detecting diabetic retinopathy. Several cameras were tested and, notably, each camera exhibited varying but adequate levels of sensitivity and specificity. The Canon CR2 AF emerged with the highest accuracy in identifying both more than mild diabetic retinopathy and vision-threatening cases, while the Topcon TRC-NW400 excelled in detecting clinically significant diabetic macular oedema. The findings from this study emphasize the importance of considering a non mydriatic camera and artificial intelligence software for diabetic retinopathy screening. However, further research is imperative to explore additional factors influencing the efficiency of diabetic retinopathy screening using AI and non mydriatic cameras such as costs involved and effects of screening using and on an ethnically diverse population.

Rate and Predictors of Misclassification of Active Diabetic Macular Edema as Detected by an Automated Retinal Image Analysis System

INTRODUCTION

The aim of this work is to estimate the sensitivity, specificity, and misclassification rate of an automated retinal image analysis system (ARIAS) in diagnosing active diabetic macular edema (DME) and to identify factors associated with true and false positives.

METHODS

We conducted a cross-sectional study of prospectively enrolled patients with diabetes mellitus (DM) referred to a tertiary medical retina center for screening or management of DME. All patients underwent two-field fundus photography (macula- and disc-centered) with a true-color confocal camera; images were processed by EyeArt V.2.1.0 (Woodland Hills, CA, USA). Active DME was defined as the presence of intraretinal or subretinal fluid on spectral-domain optical coherence tomography (SD-OCT). Sensitivity and specificity and their 95% confidence intervals (CIs) were calculated. Variables associated with true (i.e., DME labeled as present by ARIAS + fluid on SD-OCT) and false positives (i.e., DME labeled as present by ARIAS + no fluid on SD-OCT) of active DME were explored.

RESULTS

A total of 298 eyes were included; 92 eyes (31%) had active DME. ARIAS sensitivity and specificity were 82.61% (95% CI 72.37-89.60) and 84.47% (95% CI 78.34-89.10). The misclassification rate was 16%. Factors associated with true positives included younger age (p = 0.01), shorter DM duration (p = 0.006), presence of hard exudates (p = 0.005), and microaneurysms (p = 0.002). Factors associated with false positives included longer DM duration (p = 0.01), worse diabetic retinopathy severity (p = 0.008), history of inactivated DME (p < 0.001), and presence of hard exudates (p < 0.001), microaneurysms (p < 0.001), or epiretinal membrane (p = 0.06).

CONCLUSIONS

The sensitivity of ARIAS was diminished in older patients and those without DME-related fundus lesions, while the specificity was reduced in cases with a history of inactivated DME. ARIAS performed well in screening for naïve DME but is not effective in surveillance inactivated DME.

A pilot cost-analysis study comparing AI-based EyeArt® and ophthalmologist assessment of diabetic retinopathy in minority women in Oslo, Norway

BACKGROUND

Diabetic retinopathy (DR) is the leading cause of adult blindness in the working age population worldwide, which can be prevented by early detection. Regular eye examinations are recommended and crucial for detecting sight-threatening DR. Use of artificial intelligence (AI) to lessen the burden on the healthcare system is needed.

PURPOSE

To perform a pilot cost-analysis study for detecting DR in a cohort of minority women with DM in Oslo, Norway, that have the highest prevalence of diabetes mellitus (DM) in the country, using both manual (ophthalmologist) and autonomous (AI) grading. This is the first study in Norway, as far as we know, that uses AI in DR- grading of retinal images.

METHODS

On Minority Women's Day, November 1, 2017, in Oslo, Norway, 33 patients (66 eyes) over 18 years of age diagnosed with DM (T1D and T2D) were screened. The Eidon - True Color Confocal Scanner (CenterVue, United States) was used for retinal imaging and graded for DR after screening had been completed, by an ophthalmologist and automatically, using EyeArt Automated DR Detection System, version 2.1.0 (EyeArt, EyeNuk, CA, USA). The gradings were based on the International Clinical Diabetic Retinopathy (ICDR) severity scale [1] detecting the presence or absence of referable DR. Cost-minimization analyses were performed for both grading methods.

RESULTS

33 women (64 eyes) were eligible for the analysis. A very good inter-rater agreement was found: 0.98 (P < 0.01), between the human and AI-based EyeArt grading system for detecting DR. The prevalence of DR was 18.6% (95% CI: 11.4-25.8%), and the sensitivity and specificity were 100% (95% CI: 100-100% and 95% CI: 100-100%), respectively. The cost difference for AI screening compared to human screening was $143 lower per patient (cost-saving) in favour of AI.

CONCLUSION

Our results indicate that The EyeArt AI system is both a reliable, cost-saving, and useful tool for DR grading in clinical practice.

Target Product Profile for a Machine Learning-Automated Retinal Imaging Analysis Software for Use in English Diabetic Eye Screening: Protocol for a Mixed Methods Study

BACKGROUND

Diabetic eye screening (DES) represents a significant opportunity for the application of machine learning (ML) technologies, which may improve clinical and service outcomes. However, successful integration of ML into DES requires careful product development, evaluation, and implementation. Target product profiles (TPPs) summarize the requirements necessary for successful implementation so these can guide product development and evaluation.

OBJECTIVE

This study aims to produce a TPP for an ML-automated retinal imaging analysis software (ML-ARIAS) system for use in DES in England.

METHODS

This work will consist of 3 phases. Phase 1 will establish the characteristics to be addressed in the TPP. A list of candidate characteristics will be generated from the following sources: an overview of systematic reviews of diagnostic test TPPs; a systematic review of digital health TPPs; and the National Institute for Health and Care Excellence's Evidence Standards Framework for Digital Health Technologies. The list of characteristics will be refined and validated by a study advisory group (SAG) made up of representatives from key stakeholders in DES. This includes people with diabetes; health care professionals; health care managers and leaders; and regulators and policy makers. In phase 2, specifications for these characteristics will be drafted following a series of semistructured interviews with participants from these stakeholder groups. Data collected from these interviews will be analyzed using the shortlist of characteristics as a framework, after which specifications will be drafted to create a draft TPP. Following approval by the SAG, in phase 3, the draft will enter an internet-based Delphi consensus study with participants sought from the groups previously identified, as well as ML-ARIAS developers, to ensure feasibility. Participants will be invited to score characteristic and specification pairs on a scale from "definitely exclude" to "definitely include," and suggest edits. The document will be iterated between rounds based on participants' feedback. Feedback on the draft document will be sought from a group of ML-ARIAS developers before its final contents are agreed upon in an in-person consensus meeting. At this meeting, representatives from the stakeholder groups previously identified (minus ML-ARIAS developers, to avoid bias) will be presented with the Delphi results and feedback of the user group and asked to agree on the final contents by vote.

RESULTS

Phase 1 was completed in November 2023. Phase 2 is underway and expected to finish in March 2024. Phase 3 is expected to be complete in July 2024.

CONCLUSIONS

The multistakeholder development of a TPP for an ML-ARIAS for use in DES in England will help developers produce tools that serve the needs of patients, health care providers, and their staff. The TPP development process will also provide methods and a template to produce similar documents in other disease areas.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/50568.

Artificial Intelligence (AI) for Early Diagnosis of Retinal Diseases

Artificial intelligence (AI) has emerged as a transformative tool in the field of ophthalmology, revolutionizing disease diagnosis and management. This paper provides a comprehensive overview of AI applications in various retinal diseases, highlighting its potential to enhance screening efficiency, facilitate early diagnosis, and improve patient outcomes. Herein, we elucidate the fundamental concepts of AI, including machine learning (ML) and deep learning (DL), and their application in ophthalmology, underscoring the significance of AI-driven solutions in addressing the complexity and variability of retinal diseases. Furthermore, we delve into the specific applications of AI in retinal diseases such as diabetic retinopathy (DR), age-related macular degeneration (AMD), Macular Neovascularization, retinopathy of prematurity (ROP), retinal vein occlusion (RVO), hypertensive retinopathy (HR), Retinitis Pigmentosa, Stargardt disease, best vitelliform macular dystrophy, and sickle cell retinopathy. We focus on the current landscape of AI technologies, including various AI models, their performance metrics, and clinical implications. Furthermore, we aim to address challenges and pitfalls associated with the integration of AI in clinical practice, including the "black box phenomenon", biases in data representation, and limitations in comprehensive patient assessment. In conclusion, this review emphasizes the collaborative role of AI alongside healthcare professionals, advocating for a synergistic approach to healthcare delivery. It highlights the importance of leveraging AI to augment, rather than replace, human expertise, thereby maximizing its potential to revolutionize healthcare delivery, mitigate healthcare disparities, and improve patient outcomes in the evolving landscape of medicine.

Assessing Diabetic Retinopathy Staging With AI: A Comparative Analysis Between Pseudocolor and LED Imaging

Purpose

To compare the diagnostic performance of artificial intelligence (AI)-based diabetic retinopathy (DR) staging system across pseudocolor, simulated white light (SWL), and light-emitting diode (LED) camera imaging modalities.

Methods

A cross-sectional investigation involved patients with diabetes undergoing imaging with an iCare DRSplus confocal LED camera and an Optos confocal, ultra-widefield pseudocolor camera, with and without SWL. Macula-centered and optic nerve-centered 45 × 45-degree photographs were processed using EyeArt v2.1. Human graders established the ground truth (GT) for DR severity on dilated fundus exams. Sensitivity and weighted Cohen's weighted kappa (wκ) were calculated. An ordinal generalized linear mixed model identified factors influencing accurate DR staging.

Results

The study included 362 eyes from 189 patients. The LED camera excelled in identifying sight-threatening DR stages (sensitivity = 0.83, specificity = 0.95 for proliferative DR) and had the highest agreement with the GT (wκ = 0.71). The addition of SWL to pseudocolor imaging resulted in decreased performance (sensitivity = 0.33, specificity = 0.98 for proliferative DR; wκ = 0.55). Peripheral lesions reduced the likelihood of being staged in the same or higher DR category by 80% (P < 0.001).

Conclusions

Pseudocolor and LED cameras, although proficient, demonstrated non-interchangeable performance, with the LED camera exhibiting superior accuracy in identifying advanced DR stages. These findings underscore the importance of implementing AI systems trained for ultra-widefield imaging, considering the impact of peripheral lesions on correct DR staging.

Translational Relevance

This study underscores the need for artificial intelligence-based systems specifically trained for ultra-widefield imaging in diabetic retinopathy assessment.

The impact of artificial intelligence on retinal disease management: Vision Academy retinal expert consensus

PURPOSE OF REVIEW

The aim of this review is to define the "state-of-the-art" in artificial intelligence (AI)-enabled devices that support the management of retinal conditions and to provide Vision Academy recommendations on the topic.

RECENT FINDINGS

Most of the AI models described in the literature have not been approved for disease management purposes by regulatory authorities. These new technologies are promising as they may be able to provide personalized treatments as well as a personalized risk score for various retinal diseases. However, several issues still need to be addressed, such as the lack of a common regulatory pathway and a lack of clarity regarding the applicability of AI-enabled medical devices in different populations.

SUMMARY

It is likely that current clinical practice will need to change following the application of AI-enabled medical devices. These devices are likely to have an impact on the management of retinal disease. However, a consensus needs to be reached to ensure they are safe and effective for the overall population.

Test accuracy of artificial intelligence-based grading of fundus images in diabetic retinopathy screening: A systematic review

OBJECTIVES

To systematically review the accuracy of artificial intelligence (AI)-based systems for grading of fundus images in diabetic retinopathy (DR) screening.

METHODS

We searched MEDLINE, EMBASE, the Cochrane Library and the ClinicalTrials.gov from 1st January 2000 to 27th August 2021. Accuracy studies published in English were included if they met the pre-specified inclusion criteria. Selection of studies for inclusion, data extraction and quality assessment were conducted by one author with a second reviewer independently screening and checking 20% of titles. Results were analysed narratively.

RESULTS

Forty-three studies evaluating 15 deep learning (DL) and 4 machine learning (ML) systems were included. Nine systems were evaluated in a single study each. Most studies were judged to be at high or unclear risk of bias in at least one QUADAS-2 domain. Sensitivity for referable DR and higher grades was ≥85% while specificity varied and was <80% for all ML systems and in 6/31 studies evaluating DL systems. Studies reported high accuracy for detection of ungradable images, but the latter were analysed and reported inconsistently. Seven studies reported that AI was more sensitive but less specific than human graders.

CONCLUSIONS

AI-based systems are more sensitive than human graders and could be safe to use in clinical practice but have variable specificity. However, for many systems evidence is limited, at high risk of bias and may not generalise across settings. Therefore, pre-implementation assessment in the target clinical pathway is essential to obtain reliable and applicable accuracy estimates.

Current status and practical considerations of artificial intelligence use in screening and diagnosing retinal diseases: Vision Academy retinal expert consensus

PURPOSE OF REVIEW

The application of artificial intelligence (AI) technologies in screening and diagnosing retinal diseases may play an important role in telemedicine and has potential to shape modern healthcare ecosystems, including within ophthalmology.

RECENT FINDINGS

In this article, we examine the latest publications relevant to AI in retinal disease and discuss the currently available algorithms. We summarize four key requirements underlining the successful application of AI algorithms in real-world practice: processing massive data; practicability of an AI model in ophthalmology; policy compliance and the regulatory environment; and balancing profit and cost when developing and maintaining AI models.

SUMMARY

The Vision Academy recognizes the advantages and disadvantages of AI-based technologies and gives insightful recommendations for future directions.

Performance of an artificial intelligence automated system for diabetic eye screening in a large English population

AIMS

A diabetic eye screening programme has huge value in reducing avoidable sight loss by identifying diabetic retinopathy at a stage when it can be treated. Artificial intelligence automated systems can be used for diabetic eye screening but are not employed in the national English Diabetic Eye Screening Programme. The aim was to report the performance of a commercially available deep-learning artificial intelligence software in a large English population.

METHODS

9817 anonymised image sets from 10,000 consecutive diabetic eye screening episodes were presented to an artificial intelligence software. The sensitivity and specificity of the artificial intelligence system for detecting diabetic retinopathy were determined using the diabetic eye screening programme manual grade according to national protocols as the reference standard.

RESULTS

For no diabetic retinopathy versus any diabetic retinopathy, the sensitivity of the artificial intelligence grading system was 69.7% and specificity 92.2%. The performance of the artificial intelligence system was superior for no or mild diabetic retinopathy versus significant or referrable diabetic retinopathy with a sensitivity of 95.4% and specificity of 92.0%. No cases were identified in which the artificial intelligence grade had missed significant diabetic retinopathy.

CONCLUSION

The performance of a commercially available deep-learning artificial intelligence system for identifying diabetic retinopathy in an English national Diabetic Eye Screening Programme is presented. Using the pre-defined settings artificial intelligence performance was highest when identifying diabetic retinopathy which requires an action by the screening programme.

Artificial Intelligence in Ophthalmology - Status Quo and Future Perspectives

Artificial intelligence (AI) is an emerging technology in healthcare and holds the potential to disrupt many arms in medical care. In particular, disciplines using medical imaging modalities, including e.g. radiology but ophthalmology as well, are already confronted with a wide variety of AI implications. In ophthalmologic research, AI has demonstrated promising results limited to specific diseases and imaging tools, respectively. Yet, implementation of AI in clinical routine is not widely spread due to availability, heterogeneity in imaging techniques and AI methods. In order to describe the status quo, this narrational review provides a brief introduction to AI ("what the ophthalmologist needs to know"), followed by an overview of different AI-based applications in ophthalmology and a discussion on future challenges.Abbreviations: Age-related macular degeneration, AMD; Artificial intelligence, AI; Anterior segment OCT, AS-OCT; Coronary artery calcium score, CACS; Convolutional neural network, CNN; Deep convolutional neural network, DCNN; Diabetic retinopathy, DR; Machine learning, ML; Optical coherence tomography, OCT; Retinopathy of prematurity, ROP; Support vector machine, SVM; Thyroid-associated ophthalmopathy, TAO.

Artificial Intelligence for Diabetic Retinopathy Screening Using Color Retinal Photographs: From Development to Deployment

Diabetic retinopathy (DR), a leading cause of preventable blindness, is expected to remain a growing health burden worldwide. Screening to detect early sight-threatening lesions of DR can reduce the burden of vision loss; nevertheless, the process requires intensive manual labor and extensive resources to accommodate the increasing number of patients with diabetes. Artificial intelligence (AI) has been shown to be an effective tool which can potentially lower the burden of screening DR and vision loss. In this article, we review the use of AI for DR screening on color retinal photographs in different phases of application, ranging from development to deployment. Early studies of machine learning (ML)-based algorithms using feature extraction to detect DR achieved a high sensitivity but relatively lower specificity. Robust sensitivity and specificity were achieved with the application of deep learning (DL), although ML is still used in some tasks. Public datasets were utilized in retrospective validations of the developmental phases in most algorithms, which require a large number of photographs. Large prospective clinical validation studies led to the approval of DL for autonomous screening of DR although the semi-autonomous approach may be preferable in some real-world settings. There have been few reports on real-world implementations of DL for DR screening. It is possible that AI may improve some real-world indicators for eye care in DR, such as increased screening uptake and referral adherence, but this has not been proven. The challenges in deployment may include workflow issues, such as mydriasis to lower ungradable cases; technical issues, such as integration into electronic health record systems and integration into existing camera systems; ethical issues, such as data privacy and security; acceptance of personnel and patients; and health-economic issues, such as the need to conduct health economic evaluations of using AI in the context of the country. The deployment of AI for DR screening should follow the governance model for AI in healthcare which outlines four main components: fairness, transparency, trustworthiness, and accountability.

Development and Application of a Standardized Testset for an Artificial Intelligence Medical Device Intended for the Computer-Aided Diagnosis of Diabetic Retinopathy

Objective

To explore a centralized approach to build test sets and assess the performance of an artificial intelligence medical device (AIMD) which is intended for computer-aided diagnosis of diabetic retinopathy (DR).

Method

A framework was proposed to conduct data collection, data curation, and annotation. Deidentified colour fundus photographs were collected from 11 partner hospitals with raw labels. Photographs with sensitive information or authenticity issues were excluded during vetting. A team of annotators was recruited through qualification examinations and trained. The annotation process included three steps: initial annotation, review, and arbitration. The annotated data then composed a standardized test set, which was further imported to algorithms under test (AUT) from different developers. The algorithm outputs were compared with the final annotation results (reference standard).

Result

The test set consists of 6327 digital colour fundus photographs. The final labels include 5 stages of DR and non-DR, as well as other ocular diseases and photographs with unacceptable quality. The Fleiss Kappa was 0.75 among the annotators. The Cohen's kappa between raw labels and final labels is 0.5. Using this test set, five AUTs were tested and compared quantitatively. The metrics include accuracy, sensitivity, and specificity. The AUTs showed inhomogeneous capabilities to classify different types of fundus photographs.

Conclusions

This article demonstrated a workflow to build standardized test sets and conduct algorithm testing of the AIMD for computer-aided diagnosis of diabetic retinopathy. It may provide a reference to develop technical standards that promote product verification and quality control, improving the comparability of products.

Evaluation of a New Neural Network Classifier for Diabetic Retinopathy

BACKGROUND

Medical image segmentation is a well-studied subject within the field of image processing. The goal of this research is to create an AI retinal screening grading system that is both accurate and fast. We introduce a new segmentation network which achieves state-of-the-art results on semantic segmentation of color fundus photographs. By applying the net-work to identify anatomical markers of diabetic retinopathy (DR) and diabetic macular edema (DME), we collect sufficient information to classify patients by grades R0 and R1 or above, M0 and M1.

METHODS

The AI grading system was trained on screening data to evaluate the presence of DR and DME. The core algorithm of the system is a deep learning network that segments relevant anatomical features in a retinal image. Patients were graded according to the standard NHS Diabetic Eye Screening Program feature-based grading protocol.

RESULTS

The algorithm performance was evaluated with a series of 6,981 patient retinal images from routine diabetic eye screenings. It correctly predicted 98.9% of retinopathy events and 95.5% of maculopathy events. Non-disease events prediction rate was 68.6% for retinopathy and 81.2% for maculopathy.

CONCLUSION

This novel deep learning model was trained and tested on patient data from annual diabetic retinopathy screenings can classify with high accuracy the DR and DME status of a person with diabetes. The system can be easily reconfigured according to any grading protocol, without running a long AI training procedure. The incorporation of the AI grading system can increase the graders' productivity and improve the final outcome accuracy of the screening process.

Medical Staff and Resident Preferences for Using Deep Learning in Eye Disease Screening: Discrete Choice Experiment

Background

Deep learning–assisted eye disease diagnosis technology is increasingly applied in eye disease screening. However, no research has suggested the prerequisites for health care service providers and residents willing to use it.

Objective

The aim of this paper is to reveal the preferences of health care service providers and residents for using artificial intelligence (AI) in community-based eye disease screening, particularly their preference for accuracy.

Methods

Discrete choice experiments for health care providers and residents were conducted in Shanghai, China. In total, 34 medical institutions with adequate AI-assisted screening experience participated. A total of 39 medical staff and 318 residents were asked to answer the questionnaire and make a trade-off among alternative screening strategies with different attributes, including missed diagnosis rate, overdiagnosis rate, screening result feedback efficiency, level of ophthalmologist involvement, organizational form, cost, and screening result feedback form. Conditional logit models with the stepwise selection method were used to estimate the preferences.

Results

Medical staff preferred high accuracy: The specificity of deep learning models should be more than 90% (odds ratio [OR]=0.61 for 10% overdiagnosis; P<.001), which was much higher than the Food and Drug Administration standards. However, accuracy was not the residents’ preference. Rather, they preferred to have the doctors involved in the screening process. In addition, when compared with a fully manual diagnosis, AI technology was more favored by the medical staff (OR=2.08 for semiautomated AI model and OR=2.39 for fully automated AI model; P<.001), while the residents were in disfavor of the AI technology without doctors’ supervision (OR=0.24; P<.001).

Conclusions

Deep learning model under doctors’ supervision is strongly recommended, and the specificity of the model should be more than 90%. In addition, digital transformation should help medical staff move away from heavy and repetitive work and spend more time on communicating with residents.

Trustworthy AI: Closing the gap between development and integration of AI systems in ophthalmic practice

An increasing number of artificial intelligence (AI) systems are being proposed in ophthalmology, motivated by the variety and amount of clinical and imaging data, as well as their potential benefits at the different stages of patient care. Despite achieving close or even superior performance to that of experts, there is a critical gap between development and integration of AI systems in ophthalmic practice. This work focuses on the importance of trustworthy AI to close that gap. We identify the main aspects or challenges that need to be considered along the AI design pipeline so as to generate systems that meet the requirements to be deemed trustworthy, including those concerning accuracy, resiliency, reliability, safety, and accountability. We elaborate on mechanisms and considerations to address those aspects or challenges, and define the roles and responsibilities of the different stakeholders involved in AI for ophthalmic care, i.e., AI developers, reading centers, healthcare providers, healthcare institutions, ophthalmological societies and working groups or committees, patients, regulatory bodies, and payers. Generating trustworthy AI is not a responsibility of a sole stakeholder. There is an impending necessity for a collaborative approach where the different stakeholders are represented along the AI design pipeline, from the definition of the intended use to post-market surveillance after regulatory approval. This work contributes to establish such multi-stakeholder interaction and the main action points to be taken so that the potential benefits of AI reach real-world ophthalmic settings.

Automatic Screening of the Eyes in a Deep-Learning–Based Ensemble Model Using Actual Eye Checkup Optical Coherence Tomography Images

Eye checkups have become increasingly important to maintain good vision and quality of life. As the population requiring eye checkups increases, so does the clinical work burden of clinicians. An automatic screening algorithm to reduce the clinicians’ workload is necessary. Machine learning (ML) has recently become one of the chief techniques for automated image recognition and is a helpful tool for identifying ocular diseases. However, the accuracy of ML models is lower in a clinical setting than in the laboratory. The performance of ML models depends on the training dataset. Eye checkups often prioritize speed and minimize image processing. Data distribution differs from the training dataset and, consequently, decreases prediction performance. The study aim was to investigate an ML model to screen for retinal diseases from low-quality optical coherence tomography (OCT) images captured during actual eye chechups to prevent a dataset shift. The ensemble model with convolutional neural networks (CNNs) and random forest models showed high screening performance in the single-shot OCT images captured during the actual eye checkups. Our study indicates the strong potential of the ensemble model combining the CNN and random forest models in accurately predicting abnormalities during eye checkups.

Artificial Intelligence Algorithms in Diabetic Retinopathy Screening

PURPOSE OF REVIEW

In this review, we focus on artificial intelligence (AI) algorithms for diabetic retinopathy (DR) screening and risk stratification and factors to consider when implementing AI algorithms in the clinic.

RECENT FINDINGS

AI algorithms have been adopted, and have received regulatory approval, for automated detection of referable DR with clinically acceptable diagnostic performance. While these metrics are an important first step, performance metrics that go beyond measures of technical accuracy are needed to fully evaluate the impact of AI algorithm on patient outcomes. Recent advances in AI present an exciting opportunity to improve patient care. Using DR as an example, we have reviewed factors to consider in the implementation of AI algorithms in real-world clinical practice. These include real-world evaluation of safety, efficacy, and equity (bias); impact on patient outcomes; ethical, logistical, and regulatory factors.

Evaluation of an Artificial Intelligence System for the Detection of Diabetic Retinopathy in Chinese Community Healthcare Centers

Objective

To evaluate the sensitivity and specificity of a Comprehensive Artificial Intelligence Retinal Expert (CARE) system for detecting diabetic retinopathy (DR) in a Chinese community population.

Methods

This was a cross-sectional, diagnostic study. Participants with a previous diagnosis of diabetes from three Chinese community healthcare centers were enrolled in the study. Single-field color fundus photography was obtained and analyzed by the AI system and two ophthalmologists. Primary outcome measures included the sensitivity, specificity, positive predictive value, and negative predictive value with their 95% confidence intervals (CIs) of the AI system in detecting DR and diabetic macular edema (DME).

Results

In this study, 443 subjects (848 eyes) were enrolled, and 283 (63.88%) were men. The mean age was 52.09 (11.51) years (range 18–82 years); 266 eyes were diagnosed with any DR, 233 with more-than-mild diabetic retinopathy (mtmDR), 112 with vision-threatening diabetic retinopathy (vtDR), and 57 with DME. The image ability of the AI system was as high as 99.06%, whereas its sensitivity and specificity varied significantly in detecting DR with different severities. The sensitivity/specificity to detect any DR was 75.19% (95%CI 69.47–80.17)/93.99% (95%CI 91.65–95.71), mtmDR 78.97% (95%CI 73.06–83.90)/92.52% (95%CI 90.07–94.41), vtDR 33.93% (95%CI 25.41–43.56)/97.69% (95%CI 96.25–98.61), and DME 47.37% (95%CI 34.18–60.91)/93.99% (95%CI 91.65–95.71).

Conclusions

This multicenter cross-sectional diagnostic study noted the safety and reliability of the CARE system for DR (especially mtmDR) detection in Chinese community healthcare centers. The system may effectively solve the dilemma faced by Chinese community healthcare centers: due to the lack of ophthalmic expertise of primary physicians, DR diagnosis and referral are not timely.

Systematic Review of Health Economic Evaluations Focused on Artificial Intelligence in Healthcare: The Tortoise and the Cheetah

OBJECTIVES

This study aimed to systematically review recent health economic evaluations (HEEs) of artificial intelligence (AI) applications in healthcare. The aim was to discuss pertinent methods, reporting quality and challenges for future implementation of AI in healthcare, and additionally advise future HEEs.

METHODS

A systematic literature review was conducted in 2 databases (PubMed and Scopus) for articles published in the last 5 years. Two reviewers performed independent screening, full-text inclusion, data extraction, and appraisal. The Consolidated Health Economic Evaluation Reporting Standards and Philips checklist were used for the quality assessment of included studies.

RESULTS

A total of 884 unique studies were identified; 20 were included for full-text review, covering a wide range of medical specialties and care pathway phases. The most commonly evaluated type of AI was automated medical image analysis models (n = 9, 45%). The prevailing health economic analysis was cost minimization (n = 8, 40%) with the costs saved per case as preferred outcome measure. A total of 9 studies (45%) reported model-based HEEs, 4 of which applied a time horizon >1 year. The evidence supporting the chosen analytical methods, assessment of uncertainty, and model structures was underreported. The reporting quality of the articles was moderate as on average studies reported on 66% of Consolidated Health Economic Evaluation Reporting Standards items.

CONCLUSIONS

HEEs of AI in healthcare are limited and often focus on costs rather than health impact. Surprisingly, model-based long-term evaluations are just as uncommon as model-based short-term evaluations. Consequently, insight into the actual benefits offered by AI is lagging behind current technological developments.

Novel biomarkers and therapeutic approaches for diabetic retinopathy and nephropathy: Recent progress and future perspectives

The global burden due to microvascular complications in patients with diabetes mellitus persists and even increases alarmingly, the intervention and management are now encountering many difficulties and challenges. This paper reviews the recent advancement and progress in novel biomarkers, artificial intelligence technology, therapeutic agents and approaches of diabetic retinopathy and nephropathy, providing more insights into the management of microvascular complications.

Clinical validation of an artificial intelligence-based diabetic retinopathy screening tool for a national health system

OBJECTIVE

To evaluate the accuracy and validity of an automated diabetic retinopathy (DR) screening tool (DART, TeleDx, Santiago, Chile) that uses artificial intelligence to analyze ocular fundus photographs for potential implementation in the national Chilean DR screening programme.

METHOD

This was an observational study of 1123 diabetic eye exams using a validation protocol designed by the commission of the Chilean Ministry of Health personnel and retina specialists.

RESULTS

Receiver operating characteristic (ROC) analysis indicated a sensitivity of 94.6% (95% CI: 90.9-96.9%), specificity of 74.3% (95% CI: 73.3-75%), and negative predictive value of 98.1% (95% CI: 96.8-98.9%) for the automated tool at the optimal operating point for DR screening. The area under the ROC curve was 0.915.

CONCLUSIONS

The results of this study suggest that DART is a valid tool that could be implemented in a heterogeneous health network such as the Chilean system.

Pediatric Diabetic Retinopathy: Updates in Prevalence, Risk Factors, Screening, and Management

PURPOSE OF REVIEW

Diabetic retinopathy (DR) is a microvascular complication of diabetes mellitus and a major cause of vision loss worldwide. The purpose of this review is to provide an update on the prevalence of diabetic retinopathy in youth, discuss risk factors, and review recent advances in diabetic retinopathy screening.

RECENT FINDINGS

While DR has long been considered a microvascular complication, recent data suggests that retinal neurodegeneration may precede the vascular changes associated with DR. The prevalence of DR has decreased in type 1 diabetes (T1D) patients following the results of the Diabetes Control and Complications Trial and implementation of intensive insulin therapy, with prevalence ranging from 14-20% before the year 2000 to 3.7-6% after 2000. In contrast, the prevalence of diabetic retinopathy in pediatric type 2 diabetes (T2D) is higher, ranging from 9.1-50%. Risk factors for diabetic retinopathy are well established and include glycemic control, diabetes duration, hypertension, and hyperlipidemia, whereas diabetes technology use including insulin pumps and continuous glucose monitors has been shown to have protective effects. Screening for DR is recommended for youth with T1D once they are aged ≥ 11 years or puberty has started and diabetes duration of 3-5 years. Pediatric T2D patients are advised to undergo screening at or soon after diagnosis, and annually thereafter, due to the insidious nature of T2D. Recent advances in DR screening methods including point of care and artificial intelligence technology have increased access to DR screening, while being cost-saving to patients and cost-effective to healthcare systems. While the prevalence of diabetic retinopathy in youth with T1D has been declining over the last few decades, there has been a significant increase in the prevalence of DR in youth with T2D. Improving access to diabetic retinopathy screening using novel screening methods may help improve detection and early treatment of diabetic retinopathy.

Using artificial intelligence for diabetic retinopathy screening: Policy implications

Artificial intelligence (AI) has evolved over the last few years; its use in DR screening has been demonstrated in multiple evidences across the globe. However, there are concerns right from the data acquisition, bias in data, difficulty in comparing between different algorithm, challenges in machine learning, its application in different group of population, and human barrier to AI adoption in health care. There are also legal and ethical concerns related to AI. The tension between risks and concerns on one hand versus potential and opportunity on the other have driven a need for authorities to implement policies for AI in DR screening to address these issues. The policy makers should support and facilitate research and development of AI in healthcare, but at the same time, it has to be ensured that the use of AI in healthcare aligns with recognized standards of safety, efficacy, and equity. It is essential to ensure that algorithms, datasets, and decisions are auditable and when applied to medical care (such as screening, diagnosis, or treatment) are clinically validated and explainable. Policy frameworks should require design of AI systems in health care that are informed by real-world workflow and human-centric design. Lastly, it should be ensured that healthcare AI solutions align with all relevant ethical obligations, from design to development to use and to be delivered properly in the real world.

Current status and future possibilities of retinal imaging in diabetic retinopathy care applicable to low- and medium-income countries

Diabetic retinopathy (DR) is a leading cause of blindness among adults and the numbers are projected to rise. There have been dramatic advances in the field of retinal imaging since the first fundus image was captured by Jackman and Webster in 1886. The currently available imaging modalities in the management of DR include fundus photography, fluorescein angiography, autofluorescence imaging, optical coherence tomography, optical coherence tomography angiography, and near-infrared reflectance imaging. These images are obtained using traditional fundus cameras, widefield fundus cameras, handheld fundus cameras, or smartphone-based fundus cameras. Fluorescence lifetime ophthalmoscopy, adaptive optics, multispectral and hyperspectral imaging, and multicolor imaging are the evolving technologies which are being researched for their potential applications in DR. Telemedicine has gained popularity in recent years as remote screening of DR has been made possible. Retinal imaging technologies integrated with artificial intelligence/deep-learning algorithms will likely be the way forward in the screening and grading of DR. We provide an overview of the current and upcoming imaging modalities which are relevant to the management of DR.

Facilitating diabetic retinopathy screening using automated retinal image analysis in underresourced settings

AIM

To evaluate an automated retinal image analysis (ARIA) of indigenous retinal fundus images against a human grading comparator for the classification of diabetic retinopathy (DR) status.

METHODS

Indigenous Australian adults with type 2 diabetes (n = 410) from three remote and very remote primary-care services in the Northern Territory, Australia, underwent teleretinal DR screening. A single, central retinal fundus photograph (opportunistic mydriasis) for each eye was later regraded using a single ARIA and a UK human grader and national DR classification system. The sensitivity and specificity of ARIA were assessed relative to the comparator. Proportionate agreement and a Kappa statistic were also computed.

RESULTS

Retinal images from 391 and 393 participants were gradable for 'Any DR' by the human grader and ARIA grader, respectively. 'Any DR' was detected by the human grader in 185 (47.3%) participants and by ARIA in 202 (48.6%) participants (agreement =88.0%, Kappa = 0.76,), whereas proliferative DR was detected in 31 (7.9%) and 37 (9.4%) participants (agreement = 98.2%, Kappa = 0.89,), respectively. The ARIA software had 91.4 (95% CI, 86.3-95.0) sensitivity and 85.0 (95% CI, 79.3-89.5) specificity for detecting 'Any DR' and 96.8 (95% CI, 83.3-99.9) sensitivity and 98.3 (95% CI, 96.4-99.4) specificity for detecting proliferative DR.

CONCLUSIONS

This ARIA software has high sensitivity for detecting 'Any DR', hence could be used as a triage tool for human graders. High sensitivity was also found for detection of proliferative DR by ARIA. Future versions of this ARIA should include maculopathy and referable DR (CSME and/or PDR). Such ARIA software may benefit diabetes care in less-resourced regions.

Teleophthalmology and Artificial Intelligence As Game Changers in Ophthalmic Care After the COVID-19 Pandemic

The current COVID-19 pandemic has boosted a sudden demand for telemedicine due to quarantine and travel restrictions. The exponential increase in the use of telemedicine is expected to affect ophthalmology drastically. The aim of this review is to discuss the utility, effectiveness and challenges of teleophthalmological new tools for eyecare delivery as well as its implementation and possible facilitation with artificial intelligence. We used the terms: “teleophthalmology,” “telemedicine and COVID-19,” “retinal diseases and telemedicine,” “virtual ophthalmology,” “cost effectiveness of teleophthalmology,” “pediatric teleophthalmology,” “Artificial intelligence and ophthalmology,” “Glaucoma and teleophthalmology” and “teleophthalmology limitations” in the database of PubMed and selected the articles being published in the course of 2015-2020. After the initial search, 321 articles returned as relevant. A meticulous screening followed and eventually 103 published manuscripts were included and used as our references. Emerging in the market, teleophthalmology is showing great potential for the future of ophthalmological care, benefiting both patients and ophthalmologists in times of pandemics. The spectrum of eye diseases that could benefit from teleophthalmology is wide, including mostly retinal diseases such as diabetic retinopathy, retinopathy of prematurity, age-related macular degeneration but also glaucoma and anterior segment conditions. Simultaneously, artificial intelligence provides ways of implementing teleophthalmology easier and with better outcomes, contributing as significant changing factors for ophthalmology practice after the COVID-19 pandemic.

Evaluation of focus and deep learning methods for automated image grading and factors influencing image quality in adaptive optics ophthalmoscopy

Adaptive optics flood illumination ophthalmoscopy (AO-FIO) is an established imaging tool in the investigation of retinal diseases. However, the clinical interpretation of AO-FIO images can be challenging due to varied image quality. Therefore, image quality assessment is essential before interpretation. An image assessment tool will also assist further work on improving the image quality, either during acquisition or post processing. In this paper, we describe, validate and compare two automated image quality assessment methods; the energy of Laplacian focus operator (LAPE; not commonly used but easily implemented) and convolutional neural network (CNN; effective but more complex approach). We also evaluate the effects of subject age, axial length, refractive error, fixation stability, disease status and retinal location on AO-FIO image quality. Based on analysis of 10,250 images of 50 × 50 μm size, at 41 retinal locations, from 50 subjects we demonstrate that CNN slightly outperforms LAPE in image quality assessment. CNN achieves accuracy of 89%, whereas LAPE metric achieves 73% and 80% (for a linear regression and random forest multiclass classifier methods, respectively) compared to ground truth. Furthermore, the retinal location, age and disease are factors that can influence the likelihood of poor image quality.

Testing a Deep Learning Algorithm for Detection of Diabetic Retinopathy in a Spanish Diabetic Population and with MESSIDOR Database

Background: The aim of the present study was to test our deep learning algorithm (DLA) by reading the retinographies. Methods: We tested our DLA built on convolutional neural networks in 14,186 retinographies from our population and 1200 images extracted from MESSIDOR. The retinal images were graded both by the DLA and independently by four retina specialists. Results of the DLA were compared according to accuracy (ACC), sensitivity (S), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUC), distinguishing between identification of any type of DR (any DR) and referable DR (RDR). Results: The results of testing the DLA for identifying any DR in our population were: ACC = 99.75, S = 97.92, SP = 99.91, PPV = 98.92, NPV = 99.82, and AUC = 0.983. When detecting RDR, the results were: ACC = 99.66, S = 96.7, SP = 99.92, PPV = 99.07, NPV = 99.71, and AUC = 0.988. The results of testing the DLA for identifying any DR with MESSIDOR were: ACC = 94.79, S = 97.32, SP = 94.57, PPV = 60.93, NPV = 99.75, and AUC = 0.959. When detecting RDR, the results were: ACC = 98.78, S = 94.64, SP = 99.14, PPV = 90.54, NPV = 99.53, and AUC = 0.968. Conclusions: Our DLA performed well, both in detecting any DR and in classifying those eyes with RDR in a sample of retinographies of type 2 DM patients in our population and the MESSIDOR database.

The Role of Telemedicine, In-Home Testing and Artificial Intelligence to Alleviate an Increasingly Burdened Healthcare System: Diabetic Retinopathy

In the presence of the ever-increasing incidence of diabetes mellitus (DM), the prevalence of diabetic eye disease (DED) is also growing. Despite many improvements in diabetic care, DM remains a leading cause of visual impairment in working-age patients. So far, prevention has been the best way to protect vision. The sooner we diagnose DED, the more effective the treatment is. Thus, diabetic retinopathy (DR) screening, especially with imaging techniques, is a method of choice for vision protection. To alleviate the burden of diabetic patients who need ophthalmic care, telemedicine and in-home testing are used, supported by artificial intelligence (AI) algorithms. This is why we decided to evaluate current image teleophthalmology methods used for DR screening. We searched the PubMed platform for papers published over the last 5 years (2015–2020) using the following key words: telemedicine in diabetic retinopathy screening, diabetic retinopathy screening, automated diabetic retinopathy screening, artificial intelligence in diabetic retinopathy screening, smartphone diabetic retinopathy testing. We have included 118 original articles meeting the above criteria, discussing imaging diabetic retinopathy screening methods. We have found that fundus cameras, stable or mobile, are most commonly used for retinal photography, with portable fundus cameras also relatively common. Other possibilities involve the use of ultra-wide-field (UWF) imaging and even optical coherence tomography (OCT) devices for DR screening. Also, the role of smartphones is increasingly recognized in the field. Retinal fundus images are assessed by humans instantly or remotely, while AI algorithms seem to be useful tools facilitating retinal image assessment. The common use of smartphones and availability of relatively cheap, easy-to-use adapters for retinal photographs augmented by AI algorithms make it possible for eye fundus photographs to be taken by non-specialists and in non-medical setting. This opens the way for in-home testing conducted on a much larger scale in the future. In conclusion, based on current DR screening techniques, we can suggest that the future practice of eye care specialists will be widely supported by AI algorithms, and this way will be more effective.

Telemedical Diabetic Retinopathy Screening in a Primary Care Setting: Quality of Retinal Photographs and Accuracy of Automated Image Analysis

Background: Screening for diabetic eye disease (DED) and general diabetes care is often separate, which leads to delays and low adherence to DED screening recommendations. Thus, we assessed the feasibility, achieved image quality, and possible barriers of telemedical DED screening in a point-of-care general practice setting and the accuracy of an automated algorithm for detection of DED.Methods: Patients with diabetes were recruited at general practices. Retinal images were acquired using a non-mydriatic camera (CenterVue, Italy) by medical assistants. Images were quality assessed and double graded by two graders. All images were also graded automatically using a commercially available artificial intelligence (AI) algorithm (EyeArt version 2.1.0, Eyenuk Inc.).Results: A total of 75 patients (147 eyes; mean age 69 years, 96% type 2 diabetes) were included. Most of the patients (51; 68%) preferred DED screening at the general practice, but only twenty-four (32%) were willing to pay for this service. Images of 63 patients (84%) were determined to be evaluable, and DED was diagnosed in 6 patients (8.0%). The algorithm's positive/negative predictive values (95% confidence interval) were 0.80 (0.28-0.99)/1.00 (0.92-1.00) and 0.75 (0.19-0.99)/0.98 (0.88-1.00) for detection of any DED and referral-warranted DED, respectively.Overall, the number of referrals was 18 (24%) for manual telemedical assessment and 31 (41%) for the artificial intelligence (AI) algorithm, resulting in a relative increase of referrals by 72% when using AI.Conclusions: Our study shows that achieved overall image quality in a telemedical GP-based DED screening was sufficient and that it would be accepted by medical assistants and patients in most cases. However, good image quality and integration into existing workflow remain challenging. Based on these findings, a larger-scale implementation study is warranted.

Evaluation der Implementierung eines zugelassenen Künstliche-Intelligenz-Systems zur Erkennung der diabetischen Retinopathie

Einleitung Ziel der Studie war die Evaluation der Genauigkeit einer auf einem Künstliche-Intelligenz-System (KI) basierenden Bewertung von Fundusfotografien im Vergleich zum Augenarzt in Bezug auf das diabetische Retinopathie-Screening in einer internistisch geführten Klinik. Zudem erfolgte die Erhebung der Gesamtuntersuchungsdauer wie auch der Patienten- und Untersucherzufriedenheit.

Methoden Im Rahmen der Studie erhielten 112 ambulante Patienten eine Fundusfotografie mit automatisierter Diagnose der diabetischen Retinopathie (DR) über das IDx-DR-System (Digital Diagnostics). Die Aufnahmen erfolgten mit der Kamera Topcon TRC-NW400 (Topcon Corp. Japan). Einschlusskriterium war die Diagnose eines Diabetes mellitus Typ 1, 2 oder 3. Bei Patienten, bei denen keine Aufnahme mit ausreichender Qualität in Miosis durchgeführt werden konnte, erfolgte die Aufnahme in Mydriasis.

Ergebnisse Von 112 Patienten konnte bei 107 Patienten (95,5 %) durch das Grading mittels IDx-DR, anhand der Fundusaufnahmen, eine Analyse durchgeführt werden – vs. bei 103 Patienten (91,9 %) durch das Grading derselben, hochauflösenden Fundusaufnahmen durch Augenärzte. Bei den verbleibenden Patienten war eine Beurteilung allein durch die Funduskopie in Mydriasis möglich. Es zeigte sich eine hochsignifikante Korrelation bezüglich der Einschätzung der Schwere der diabetischen Retinopathie zwischen Untersucher und dem IDx-DR-System (Correlation coefficient (r) = 0,8738; p < 0,0001). Die Patientenzufriedenheit lag bei 4,5 ± 0,6 [1–5], die Gesamtdauer der Untersuchung in Miosis lag im Mittel bei 3:04 ± 0:28 [min:sek].

Schlussfolgerung Das Retinopathiescreening mittels IDx-DR ermöglicht die automatisierte, zeitnahe und zuverlässige Beurteilung bzgl. des Vorliegens einer diabetischen Retinopathie mit einem robusten technischen und klinischen Arbeitsfluss, der mit einer hohen Patientenzufriedenheit einhergeht.

Analysis and Comparison of Two Artificial Intelligence Diabetic Retinopathy Screening Algorithms in a Pilot Study: IDx-DR and Retinalyze

BACKGROUND

The prevalence of diabetic retinopathy (DR) is expected to increase. This will put an increasing strain on health care resources. Recently, artificial intelligence-based, autonomous DR screening systems have been developed. A direct comparison between different systems is often difficult and only two such comparisons have been published so far. As different screening solutions are now available commercially, with more in the pipeline, choosing a system is not a simple matter. Based on the images gathered in a local DR screening program we performed a retrospective comparison of IDx-DR and Retinalyze.

METHODS

We chose a non-representative sample of all referable DR positive screening subjects (n = 60) and a random selection of DR negative patient images (n = 110). Only subjects with four good quality, 45-degree field of view images, a macula-centered and disc-centered image from both eyes were chosen for comparison. The images were captured by a Topcon NW-400 fundus camera, without mydriasis. The images were previously graded by a single ophthalmologist. For the purpose of this comparison, we assumed two screening strategies for Retinalyze-where either one or two out of the four images needed to be marked positive by the system for an overall positive result at the patient level.

RESULTS

Percentage agreement with a single reader in DR positive and DR negative cases respectively was: 93.3%, 95.5% for IDx-DR; 89.7% and 71.8% for Retinalyze strategy 1; 74.1% and 93.6% for Retinalyze under strategy 2.

CONCLUSIONS

Both systems were able to analyse the vast majority of images. Both systems were easy to set up and use. There were several limitations to the current pilot study, concerning sample choice and the reference grading that need to be addressed before attempting a more robust future study.

Digital technology, tele-medicine and artificial intelligence in ophthalmology: A global perspective

The simultaneous maturation of multiple digital and telecommunications technologies in 2020 has created an unprecedented opportunity for ophthalmology to adapt to new models of care using tele-health supported by digital innovations. These digital innovations include artificial intelligence (AI), 5th generation (5G) telecommunication networks and the Internet of Things (IoT), creating an inter-dependent ecosystem offering opportunities to develop new models of eye care addressing the challenges of COVID-19 and beyond. Ophthalmology has thrived in some of these areas partly due to its many image-based investigations. Tele-health and AI provide synchronous solutions to challenges facing ophthalmologists and healthcare providers worldwide. This article reviews how countries across the world have utilised these digital innovations to tackle diabetic retinopathy, retinopathy of prematurity, age-related macular degeneration, glaucoma, refractive error correction, cataract and other anterior segment disorders. The review summarises the digital strategies that countries are developing and discusses technologies that may increasingly enter the clinical workflow and processes of ophthalmologists. Furthermore as countries around the world have initiated a series of escalating containment and mitigation measures during the COVID-19 pandemic, the delivery of eye care services globally has been significantly impacted. As ophthalmic services adapt and form a "new normal", the rapid adoption of some of telehealth and digital innovation during the pandemic is also discussed. Finally, challenges for validation and clinical implementation are considered, as well as recommendations on future directions.

Multicenter, Head-to-Head, Real-World Validation Study of Seven Automated Artificial Intelligence Diabetic Retinopathy Screening Systems

OBJECTIVE

With rising global prevalence of diabetic retinopathy (DR), automated DR screening is needed for primary care settings. Two automated artificial intelligence (AI)-based DR screening algorithms have U.S. Food and Drug Administration (FDA) approval. Several others are under consideration while in clinical use in other countries, but their real-world performance has not been evaluated systematically. We compared the performance of seven automated AI-based DR screening algorithms (including one FDA-approved algorithm) against human graders when analyzing real-world retinal imaging data.

RESEARCH DESIGN AND METHODS

This was a multicenter, noninterventional device validation study evaluating a total of 311,604 retinal images from 23,724 veterans who presented for teleretinal DR screening at the Veterans Affairs (VA) Puget Sound Health Care System (HCS) or Atlanta VA HCS from 2006 to 2018. Five companies provided seven algorithms, including one with FDA approval, that independently analyzed all scans, regardless of image quality. The sensitivity/specificity of each algorithm when classifying images as referable DR or not were compared with original VA teleretinal grades and a regraded arbitrated data set. Value per encounter was estimated.

RESULTS

Although high negative predictive values (82.72-93.69%) were observed, sensitivities varied widely (50.98-85.90%). Most algorithms performed no better than humans against the arbitrated data set, but two achieved higher sensitivities, and one yielded comparable sensitivity (80.47%, P = 0.441) and specificity (81.28%, P = 0.195). Notably, one had lower sensitivity (74.42%) for proliferative DR (P = 9.77 × 10^-4) than the VA teleretinal graders. Value per encounter varied at $15.14-$18.06 for ophthalmologists and $7.74-$9.24 for optometrists.

CONCLUSIONS

The DR screening algorithms showed significant performance differences. These results argue for rigorous testing of all such algorithms on real-world data before clinical implementation.

ARTEFICIAL INTELLIGENCE IN DIABETIC RETINOPATHY SCREENING. A REVIEW

OBJECTIVE

The aim of this comprehensive paper is to acquaint the readers with evaluation of the retinal images using the arteficial intelligence (AI). Main focus of the paper is diabetic retinophaty (DR) screening. The basic principles of the artificial intelligence and algorithms that are already used in clinical practice or are shortly before approval will be described.

METHODOLOGY

Describing the basic characteristics and mechanisms of different approaches to the use of AI and subsequently literary minireview clarifying the current state of knowledge in the area.

RESULTS

Modern systems for screening diabetic retinopathy using deep neural networks achieve a sensitivity and specificity of over 80 % in most published studies. The results of specific studies vary depending on the definition of the gold standard, number of images tested and on the evaluated parameters.

CONCLUSION

Evaluation of images using AI will speed up and streamline the diagnosis of DR. The use of AI will allow to keep the quality of the eye care at least on the same level despite the raising number of the patients with diabetes.

Medios- An offline, smartphone-based artificial intelligence algorithm for the diagnosis of diabetic retinopathy

Purpose:

An observational study to assess the sensitivity and specificity of the Medios smartphone-based offline deep learning artificial intelligence (AI) software to detect diabetic retinopathy (DR) compared with the image diagnosis of ophthalmologists.

Methods:

Patients attending the outpatient services of a tertiary center for diabetes care underwent 3-field dilated retinal imaging using the Remidio NM FOP 10. Two fellowship-trained vitreoretinal specialists separately graded anonymized images and a patient-level diagnosis was reached based on grading of the worse eye. The images were subjected to offline grading using the Medios integrated AI-based software on the same smartphone used to acquire images. The sensitivity and specificity of the AI in detecting referable DR (moderate non-proliferative DR (NPDR) or worse disease) was compared to the gold standard diagnosis of the retina specialists.

Results:

Results include analysis of images from 297 patients of which 176 (59.2%) had no DR, 35 (11.7%) had mild NPDR, 41 (13.8%) had moderate NPDR, and 33 (11.1%) had severe NPDR. In addition, 12 (4%) patients had PDR and 36 (20.4%) had macular edema. Sensitivity and specificity of the AI in detecting referable DR was 98.84% (95% confidence interval [CI], 97.62–100%) and 86.73% (95% CI, 82.87–90.59%), respectively. The area under the curve was 0.92. The sensitivity for vision-threatening DR (VTDR) was 100%.

Conclusion:

The AI-based software had high sensitivity and specificity in detecting referable DR. Integration with the smartphone-based fundus camera with offline image grading has the potential for widespread applications in resource-poor settings.

Quantitative Assessment of the Severity of Diabetic Retinopathy

PURPOSE

To determine whether a quantitative approach to assessment of the severity of diabetic retinopathy (DR) lesions on ultrawide field (UWF) images can provide new parameters to predict progression to proliferative diabetic retinopathy (PDR).

METHODS

One hundred forty six eyes from 73 participants with DR and 4 years of follow-up data were included in this post hoc analysis, which was based on a cohort of 100 diabetic patients enrolled in a previously published prospective, comparative study of UWF imaging at the Joslin Diabetes Center. Diabetic Retinopathy Severity Score level was determined at baseline and 4-year follow-up visits using mydriatic 7-standard field Early Treatment Diabetic Retinopathy Study (ETDRS) photographs. All individual DR lesions (hemorrhage [H], microaneurysm [ma], cotton wool spot [CWS], intraretinal microvascular abnormality [IRMA]) were manually segmented on stereographic projected UWF. For each lesion type, the frequency/number, surface area, and distances from the optic nerve head (ONH) were computed. These quantitative parameters were compared between eyes that progressed to PDR in 4 years and eyes that did not progress. Univariable and multivariable logistic regression analyses were performed to identify parameters that were associated with an increased risk for progression to PDR.

RESULTS

A total of 146 eyes of 73 subjects were included in the final analysis. The mean age of the study cohort was 53.1 years, and 42 (56.8%) subjects were female. The number and surface area of H/ma's and CWSs were significantly (P ≤ .05) higher in eyes that progressed to PDR compared with eyes that did not progress by 4 years. Similarly, H/ma's and CWSs were located further away from the ONH (ie, more peripheral) in eyes that progressed (P < .05). DR lesion parameters that conferred a statistically significant increased risk for proliferative diabetic retinopathy in the multivariate model included hemorrhage area (odds ratio [OR], 2.63; 95% confidence interval [CI], 1.25-5.53), and greater distance of hemorrhages from the ONH (OR, 1.24; 95% CI, 0.97-1.59).

CONCLUSIONS

Quantitative analysis of DR lesions on UWF images identifies new risk parameters for progression to PDR including the surface area of hemorrhages and the distance of hemorrhages from the ONH. Although these risk factors will need to be confirmed in larger, prospective studies, they highlight the potential for quantitative lesion analysis to inform the design of a more precise and complete staging system for diabetic retinopathy severity in the future. NOTE: Publication of this article is sponsored by the American Ophthalmological Society.

The American Society of Retina Specialists Artificial Intelligence Task Force Report

Artificial intelligence (AI) is a growing area that relies on the heavy use of diagnostic imaging within the field of retina to offer exciting advancements in diagnostic capability to better understand and manage retinal conditions such as diabetic retinopathy, diabetic macular edema, age-related macular degeneration, and retinopathy of prematurity. However, there are discrepancies between the findings of these AI programs and their referral recommendations compared with evidence-based referral patterns, such as Preferred Practice Patterns by the American Academy of Ophthalmology. The overall focus of this task force report is to first describe the work in AI being completed in the management of retinal conditions. This report also discusses the guidelines of the Preferred Practice Pattern and how they can be used in the emerging field of AI.

Prospective evaluation of an artificial intelligence-enabled algorithm for automated diabetic retinopathy screening of 30 000 patients

Background/aims

Human grading of digital images from diabetic retinopathy (DR) screening programmes represents a significant challenge, due to the increasing prevalence of diabetes. We evaluate the performance of an automated artificial intelligence (AI) algorithm to triage retinal images from the English Diabetic Eye Screening Programme (DESP) into test-positive/technical failure versus test-negative, using human grading following a standard national protocol as the reference standard.

Methods

Retinal images from 30 405 consecutive screening episodes from three English DESPs were manually graded following a standard national protocol and by an automated process with machine learning enabled software, EyeArt v2.1. Screening performance (sensitivity, specificity) and diagnostic accuracy (95% CIs) were determined using human grades as the reference standard.

Results

Sensitivity (95% CIs) of EyeArt was 95.7% (94.8% to 96.5%) for referable retinopathy (human graded ungradable, referable maculopathy, moderate-to-severe non-proliferative or proliferative). This comprises sensitivities of 98.3% (97.3% to 98.9%) for mild-to-moderate non-proliferative retinopathy with referable maculopathy, 100% (98.7%,100%) for moderate-to-severe non-proliferative retinopathy and 100% (97.9%,100%) for proliferative disease. EyeArt agreed with the human grade of no retinopathy (specificity) in 68% (67% to 69%), with a specificity of 54.0% (53.4% to 54.5%) when combined with non-referable retinopathy.

Conclusion

The algorithm demonstrated safe levels of sensitivity for high-risk retinopathy in a real-world screening service, with specificity that could halve the workload for human graders. AI machine learning and deep learning algorithms such as this can provide clinically equivalent, rapid detection of retinopathy, particularly in settings where a trained workforce is unavailable or where large-scale and rapid results are needed.

A survey on medical image analysis in diabetic retinopathy

Diabetic Retinopathy (DR) represents a highly-prevalent complication of diabetes in which individuals suffer from damage to the blood vessels in the retina. The disease manifests itself through lesion presence, starting with microaneurysms, at the nonproliferative stage before being characterized by neovascularization in the proliferative stage. Retinal specialists strive to detect DR early so that the disease can be treated before substantial, irreversible vision loss occurs. The level of DR severity indicates the extent of treatment necessary - vision loss may be preventable by effective diabetes management in mild (early) stages, rather than subjecting the patient to invasive laser surgery. Using artificial intelligence (AI), highly accurate and efficient systems can be developed to help assist medical professionals in screening and diagnosing DR earlier and without the full resources that are available in specialty clinics. In particular, deep learning facilitates diagnosis earlier and with higher sensitivity and specificity. Such systems make decisions based on minimally handcrafted features and pave the way for personalized therapies. Thus, this survey provides a comprehensive description of the current technology used in each step of DR diagnosis. First, it begins with an introduction to the disease and the current technologies and resources available in this space. It proceeds to discuss the frameworks that different teams have used to detect and classify DR. Ultimately, we conclude that deep learning systems offer revolutionary potential to DR identification and prevention of vision loss.

Using Artificial Intelligence and Novel Polynomials to Predict Subjective Refraction

This work aimed to use artificial intelligence to predict subjective refraction from wavefront aberrometry data processed with a novel polynomial decomposition basis. Subjective refraction was converted to power vectors (M, J0, J45). Three gradient boosted trees (XGBoost) algorithms were trained to predict each power vector using data from 3729 eyes. The model was validated by predicting subjective refraction power vectors of 350 other eyes, unknown to the model. The machine learning models were significantly better than the paraxial matching method for producing a spectacle correction, resulting in a mean absolute error of 0.301 ± 0.252 Diopters (D) for the M vector, 0.120 ± 0.094 D for the J0 vector and 0.094 ± 0.084 D for the J45 vector. Our results suggest that subjective refraction can be accurately and precisely predicted from novel polynomial wavefront data using machine learning algorithms. We anticipate that the combination of machine learning and aberrometry based on this novel wavefront decomposition basis will aid the development of refined algorithms which could become a new gold standard to predict refraction objectively.