1
|
Joseph S, Selvaraj J, Mani I, Kumaragurupari T, Shang X, Mudgil P, Ravilla T, He M. Diagnostic Accuracy of Artificial Intelligence-Based Automated Diabetic Retinopathy Screening in Real-World Settings: A Systematic Review and Meta-Analysis. Am J Ophthalmol 2024; 263:214-230. [PMID: 38438095 DOI: 10.1016/j.ajo.2024.02.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2023] [Revised: 02/03/2024] [Accepted: 02/12/2024] [Indexed: 03/06/2024]
Abstract
PURPOSE To evaluate the diagnostic accuracy of artificial intelligence (AI)-based automated diabetic retinopathy (DR) screening in real-world settings. DESIGN Systematic review and meta-analysis METHODS: We conducted a systematic review of relevant literature from January 2012 to August 2022 using databases including PubMed, Scopus and Web of Science. The quality of studies was evaluated using Quality Assessment for Diagnostic Accuracy Studies 2 (QUADAS-2) checklist. We calculated pooled accuracy, sensitivity, specificity, and diagnostic odds ratio (DOR) as summary measures. The study protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO - CRD42022367034). RESULTS We included 34 studies which utilized AI algorithms for diagnosing DR based on real-world fundus images. Quality assessment of these studies indicated a low risk of bias and low applicability concern. Among gradable images, the overall pooled accuracy, sensitivity, specificity, and DOR were 81%, 94% (95% CI: 92.0-96.0), 89% (95% CI: 85.0-92.0) and 128 (95% CI: 80-204) respectively. Sub-group analysis showed that, when acceptable quality imaging could be obtained, non-mydriatic fundus images had a better DOR of 143 (95% CI: 82-251) and studies using 2 field images had a better DOR of 161 (95% CI 74-347). Our meta-regression analysis revealed a statistically significant association between DOR and variables such as the income status, and the type of fundus camera. CONCLUSION Our findings indicate that AI algorithms have acceptable performance in screening for DR using fundus images compared to human graders. Implementing a fundus camera with AI-based software has the potential to assist ophthalmologists in reducing their workload and improving the accuracy of DR diagnosis.
Collapse
Affiliation(s)
- Sanil Joseph
- From the Centre for Eye Research Australia (S.J, X.S, M.H), Royal Victorian Eye and Ear Hospital, East Melbourne, Australia; Department of Surgery (Ophthalmology) (S.J, X.S, M.H), The University of Melbourne, Melbourne, Australia; Lions Aravind Institute of Community Ophthalmology (S.J, J.S, T.R), Aravind Eye Care System. Madurai, India.
| | - Jerrome Selvaraj
- Lions Aravind Institute of Community Ophthalmology (S.J, J.S, T.R), Aravind Eye Care System. Madurai, India
| | - Iswarya Mani
- Aravind Eye Hospital and Postgraduate Institute of Ophthalmology (I.M, T.K), Madurai, India
| | | | - Xianwen Shang
- From the Centre for Eye Research Australia (S.J, X.S, M.H), Royal Victorian Eye and Ear Hospital, East Melbourne, Australia; Department of Surgery (Ophthalmology) (S.J, X.S, M.H), The University of Melbourne, Melbourne, Australia
| | - Poonam Mudgil
- School of Medicine (P.M), Western Sydney University, Campbell town, Australia; School of Rural Medicine (P.M), Charles Sturt University, Orange, NSW, Australia
| | - Thulasiraj Ravilla
- Lions Aravind Institute of Community Ophthalmology (S.J, J.S, T.R), Aravind Eye Care System. Madurai, India
| | - Mingguang He
- From the Centre for Eye Research Australia (S.J, X.S, M.H), Royal Victorian Eye and Ear Hospital, East Melbourne, Australia; Department of Surgery (Ophthalmology) (S.J, X.S, M.H), The University of Melbourne, Melbourne, Australia
| |
Collapse
|
2
|
Chen D, Geevarghese A, Lee S, Plovnick C, Elgin C, Zhou R, Oermann E, Aphinyonaphongs Y, Al-Aswad LA. Transparency in Artificial Intelligence Reporting in Ophthalmology-A Scoping Review. OPHTHALMOLOGY SCIENCE 2024; 4:100471. [PMID: 38591048 PMCID: PMC11000111 DOI: 10.1016/j.xops.2024.100471] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/29/2022] [Revised: 11/18/2023] [Accepted: 01/12/2024] [Indexed: 04/10/2024]
Abstract
Topic This scoping review summarizes artificial intelligence (AI) reporting in ophthalmology literature in respect to model development and validation. We characterize the state of transparency in reporting of studies prospectively validating models for disease classification. Clinical Relevance Understanding what elements authors currently describe regarding their AI models may aid in the future standardization of reporting. This review highlights the need for transparency to facilitate the critical appraisal of models prior to clinical implementation, to minimize bias and inappropriate use. Transparent reporting can improve effective and equitable use in clinical settings. Methods Eligible articles (as of January 2022) from PubMed, Embase, Web of Science, and CINAHL were independently screened by 2 reviewers. All observational and clinical trial studies evaluating the performance of an AI model for disease classification of ophthalmic conditions were included. Studies were evaluated for reporting of parameters derived from reporting guidelines (CONSORT-AI, MI-CLAIM) and our previously published editorial on model cards. The reporting of these factors, which included basic model and dataset details (source, demographics), and prospective validation outcomes, were summarized. Results Thirty-seven prospective validation studies were included in the scoping review. Eleven additional associated training and/or retrospective validation studies were included if this information could not be determined from the primary articles. These 37 studies validated 27 unique AI models; multiple studies evaluated the same algorithms (EyeArt, IDx-DR, and Medios AI). Details of model development were variably reported; 18 of 27 models described training dataset annotation and 10 of 27 studies reported training data distribution. Demographic information of training data was rarely reported; 7 of the 27 unique models reported age and gender and only 2 reported race and/or ethnicity. At the level of prospective clinical validation, age and gender of populations was more consistently reported (29 and 28 of 37 studies, respectively), but only 9 studies reported race and/or ethnicity data. Scope of use was difficult to discern for the majority of models. Fifteen studies did not state or imply primary users. Conclusion Our scoping review demonstrates variable reporting of information related to both model development and validation. The intention of our study was not to assess the quality of the factors we examined, but to characterize what information is, and is not, regularly reported. Our results suggest the need for greater transparency in the reporting of information necessary to determine the appropriateness and fairness of these tools prior to clinical use. Financial Disclosures Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
Collapse
Affiliation(s)
- Dinah Chen
- Department of Ophthalmology, NYU Langone Health, New York, New York
| | | | - Samuel Lee
- Department of Neurosurgery, NYU Grossman School of Medicine, New York, New York
| | | | - Cansu Elgin
- Department of Ophthalmology, Istanbul University-Cerrahpasa, Istanbul, Turkey
| | - Raymond Zhou
- Department of Neurosurgery, Vanderbilt School of Medicine, Nashville, Tennessee
| | - Eric Oermann
- Department of Neurosurgery, NYU Grossman School of Medicine, New York, New York
- Department of Neurosurgery, NYU Langone Health, New York, New York
| | - Yindalon Aphinyonaphongs
- Department of Medicine, NYU Langone Health, New York, New York
- Department of Population Health, NYU Grossman School of Medicine, New York, New York
| | - Lama A. Al-Aswad
- Department of Ophthalmology, NYU Langone Health, New York, New York
- Department of Population Health, NYU Grossman School of Medicine, New York, New York
| |
Collapse
|
3
|
Whitestone N, Nkurikiye J, Patnaik JL, Jaccard N, Lanouette G, Cherwek DH, Congdon N, Mathenge W. Feasibility and acceptance of artificial intelligence-based diabetic retinopathy screening in Rwanda. Br J Ophthalmol 2024; 108:840-845. [PMID: 37541766 DOI: 10.1136/bjo-2022-322683] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2022] [Accepted: 07/15/2023] [Indexed: 08/06/2023]
Abstract
BACKGROUND Evidence on the practical application of artificial intelligence (AI)-based diabetic retinopathy (DR) screening is needed. METHODS Consented participants were screened for DR using retinal imaging with AI interpretation from March 2021 to June 2021 at four diabetes clinics in Rwanda. Additionally, images were graded by a UK National Health System-certified retinal image grader. DR grades based on the International Classification of Diabetic Retinopathy with a grade of 2.0 or higher were considered referable. The AI system was designed to detect optic nerve and macular anomalies outside of DR. A vertical cup to disc ratio of 0.7 and higher and/or macular anomalies recognised at a cut-off of 60% and higher were also considered referable by AI. RESULTS Among 827 participants (59.6% women (n=493)) screened by AI, 33.2% (n=275) were referred for follow-up. Satisfaction with AI screening was high (99.5%, n=823), and 63.7% of participants (n=527) preferred AI over human grading. Compared with human grading, the sensitivity of the AI for referable DR was 92% (95% CI 0.863%, 0.968%), with a specificity of 85% (95% CI 0.751%, 0.882%). Of the participants referred by AI: 88 (32.0%) were for DR only, 109 (39.6%) for DR and an anomaly, 65 (23.6%) for an anomaly only and 13 (4.73%) for other reasons. Adherence to referrals was highest for those referred for DR at 53.4%. CONCLUSION DR screening using AI led to accurate referrals from diabetes clinics in Rwanda and high rates of participant satisfaction, suggesting AI screening for DR is practical and acceptable.
Collapse
Affiliation(s)
| | - John Nkurikiye
- RIIO iHospital, Rwanda International Institute of Ophthalmology, Kigali, Rwanda
- Department of Ophthalmology, Rwanda Military Hospital, Kigali, Rwanda
| | - Jennifer L Patnaik
- Clinical Services, Orbis International, New York, New York, USA
- Department of Ophthalmology, University of Colorado Denver School of Medicine, Aurora, Colorado, USA
| | - Nicolas Jaccard
- Clinical Services, Orbis International, New York, New York, USA
| | | | - David H Cherwek
- Clinical Services, Orbis International, New York, New York, USA
| | - Nathan Congdon
- Clinical Services, Orbis International, New York, New York, USA
- Centre for Public Health, Queen's University Belfast, Belfast, UK
| | - Wanjiku Mathenge
- Clinical Services, Orbis International, New York, New York, USA
- RIIO iHospital, Rwanda International Institute of Ophthalmology, Kigali, Rwanda
| |
Collapse
|
4
|
Hasan MM, Phu J, Sowmya A, Meijering E, Kalloniatis M. Artificial intelligence in the diagnosis of glaucoma and neurodegenerative diseases. Clin Exp Optom 2024; 107:130-146. [PMID: 37674264 DOI: 10.1080/08164622.2023.2235346] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2023] [Accepted: 07/07/2023] [Indexed: 09/08/2023] Open
Abstract
Artificial Intelligence is a rapidly expanding field within computer science that encompasses the emulation of human intelligence by machines. Machine learning and deep learning - two primary data-driven pattern analysis approaches under the umbrella of artificial intelligence - has created considerable interest in the last few decades. The evolution of technology has resulted in a substantial amount of artificial intelligence research on ophthalmic and neurodegenerative disease diagnosis using retinal images. Various artificial intelligence-based techniques have been used for diagnostic purposes, including traditional machine learning, deep learning, and their combinations. Presented here is a review of the literature covering the last 10 years on this topic, discussing the use of artificial intelligence in analysing data from different modalities and their combinations for the diagnosis of glaucoma and neurodegenerative diseases. The performance of published artificial intelligence methods varies due to several factors, yet the results suggest that such methods can potentially facilitate clinical diagnosis. Generally, the accuracy of artificial intelligence-assisted diagnosis ranges from 67-98%, and the area under the sensitivity-specificity curve (AUC) ranges from 0.71-0.98, which outperforms typical human performance of 71.5% accuracy and 0.86 area under the curve. This indicates that artificial intelligence-based tools can provide clinicians with useful information that would assist in providing improved diagnosis. The review suggests that there is room for improvement of existing artificial intelligence-based models using retinal imaging modalities before they are incorporated into clinical practice.
Collapse
Affiliation(s)
- Md Mahmudul Hasan
- School of Computer Science and Engineering, University of New South Wales, Kensington, New South Wales, Australia
| | - Jack Phu
- School of Optometry and Vision Science, University of New South Wales, Kensington, Australia
- Centre for Eye Health, University of New South Wales, Sydney, New South Wales, Australia
- School of Medicine (Optometry), Deakin University, Waurn Ponds, Victoria, Australia
| | - Arcot Sowmya
- School of Computer Science and Engineering, University of New South Wales, Kensington, New South Wales, Australia
| | - Erik Meijering
- School of Computer Science and Engineering, University of New South Wales, Kensington, New South Wales, Australia
| | - Michael Kalloniatis
- School of Optometry and Vision Science, University of New South Wales, Kensington, Australia
- School of Medicine (Optometry), Deakin University, Waurn Ponds, Victoria, Australia
| |
Collapse
|
5
|
Vilela MAP, Arrigo A, Parodi MB, da Silva Mengue C. Smartphone Eye Examination: Artificial Intelligence and Telemedicine. Telemed J E Health 2024; 30:341-353. [PMID: 37585566 DOI: 10.1089/tmj.2023.0041] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/18/2023] Open
Abstract
Background: The current medical scenario is closely linked to recent progress in telecommunications, photodocumentation, and artificial intelligence (AI). Smartphone eye examination may represent a promising tool in the technological spectrum, with special interest for primary health care services. Obtaining fundus imaging with this technique has improved and democratized the teaching of fundoscopy, but in particular, it contributes greatly to screening diseases with high rates of blindness. Eye examination using smartphones essentially represents a cheap and safe method, thus contributing to public policies on population screening. This review aims to provide an update on the use of this resource and its future prospects, especially as a screening and ophthalmic diagnostic tool. Methods: In this review, we surveyed major published advances in retinal and anterior segment analysis using AI. We performed an electronic search on the Medical Literature Analysis and Retrieval System Online (MEDLINE), EMBASE, and Cochrane Library for published literature without a deadline. We included studies that compared the diagnostic accuracy of smartphone ophthalmoscopy for detecting prevalent diseases with an accurate or commonly employed reference standard. Results: There are few databases with complete metadata, providing demographic data, and few databases with sufficient images involving current or new therapies. It should be taken into consideration that these are databases containing images captured using different systems and formats, with information often being excluded without essential detailing of the reasons for exclusion, which further distances them from real-life conditions. The safety, portability, low cost, and reproducibility of smartphone eye images are discussed in several studies, with encouraging results. Conclusions: The high level of agreement between conventional and a smartphone method shows a powerful arsenal for screening and early diagnosis of the main causes of blindness, such as cataract, glaucoma, diabetic retinopathy, and age-related macular degeneration. In addition to streamlining the medical workflow and bringing benefits for public health policies, smartphone eye examination can make safe and quality assessment available to the population.
Collapse
Affiliation(s)
| | - Alessandro Arrigo
- Department of Ophthalmology, Scientific Institute San Raffaele, Milan, Italy
- University Vita-Salute, Milan, Italy
| | - Maurizio Battaglia Parodi
- Department of Ophthalmology, Scientific Institute San Raffaele, Milan, Italy
- University Vita-Salute, Milan, Italy
| | - Carolina da Silva Mengue
- Post-Graduation Ophthalmological School, Ivo Corrêa-Meyer/Cardiology Institute, Porto Alegre, Brazil
| |
Collapse
|
6
|
Skevas C, de Olaguer NP, Lleó A, Thiwa D, Schroeter U, Lopes IV, Mautone L, Linke SJ, Spitzer MS, Yap D, Xiao D. Implementing and evaluating a fully functional AI-enabled model for chronic eye disease screening in a real clinical environment. BMC Ophthalmol 2024; 24:51. [PMID: 38302908 PMCID: PMC10832120 DOI: 10.1186/s12886-024-03306-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2023] [Accepted: 01/16/2024] [Indexed: 02/03/2024] Open
Abstract
BACKGROUND Artificial intelligence (AI) has the potential to increase the affordability and accessibility of eye disease screening, especially with the recent approval of AI-based diabetic retinopathy (DR) screening programs in several countries. METHODS This study investigated the performance, feasibility, and user experience of a seamless hardware and software solution for screening chronic eye diseases in a real-world clinical environment in Germany. The solution integrated AI grading for DR, age-related macular degeneration (AMD), and glaucoma, along with specialist auditing and patient referral decision. The study comprised several components: (1) evaluating the entire system solution from recruitment to eye image capture and AI grading for DR, AMD, and glaucoma; (2) comparing specialist's grading results with AI grading results; (3) gathering user feedback on the solution. RESULTS A total of 231 patients were recruited, and their consent forms were obtained. The sensitivity, specificity, and area under the curve for DR grading were 100.00%, 80.10%, and 90.00%, respectively. For AMD grading, the values were 90.91%, 78.79%, and 85.00%, and for glaucoma grading, the values were 93.26%, 76.76%, and 85.00%. The analysis of all false positive cases across the three diseases and their comparison with the final referral decisions revealed that only 17 patients were falsely referred among the 231 patients. The efficacy analysis of the system demonstrated the effectiveness of the AI grading process in the study's testing environment. Clinical staff involved in using the system provided positive feedback on the disease screening process, particularly praising the seamless workflow from patient registration to image transmission and obtaining the final result. Results from a questionnaire completed by 12 participants indicated that most found the system easy, quick, and highly satisfactory. The study also revealed room for improvement in the AMD model, suggesting the need to enhance its training data. Furthermore, the performance of the glaucoma model grading could be improved by incorporating additional measures such as intraocular pressure. CONCLUSIONS The implementation of the AI-based approach for screening three chronic eye diseases proved effective in real-world settings, earning positive feedback on the usability of the integrated platform from both the screening staff and auditors. The auditing function has proven valuable for obtaining efficient second opinions from experts, pointing to its potential for enhancing remote screening capabilities. TRIAL REGISTRATION Institutional Review Board of the Hamburg Medical Chamber (Ethik-Kommission der Ärztekammer Hamburg): 2021-10574-BO-ff.
Collapse
Affiliation(s)
- Christos Skevas
- Department of Ophthalmology, University Medical Center Hamburg - Eppendorf, Martinistr. 52, 20249, Hamburg, Germany
| | | | - Albert Lleó
- TeleMedC GmbH, Raboisen 32, 20095, Hamburg, Germany
| | - David Thiwa
- Department of Otorhinolaryngology, University Medical Center Hamburg - Eppendorf, Martinistr. 52, 20249, Hamburg, Germany
| | - Ulrike Schroeter
- Department of Ophthalmology, University Medical Center Hamburg - Eppendorf, Martinistr. 52, 20249, Hamburg, Germany
| | - Inês Valente Lopes
- Department of Ophthalmology, University Medical Center Hamburg - Eppendorf, Martinistr. 52, 20249, Hamburg, Germany.
| | - Luca Mautone
- Department of Ophthalmology, University Medical Center Hamburg - Eppendorf, Martinistr. 52, 20249, Hamburg, Germany
| | - Stephan J Linke
- Zentrum Sehestaerke, Martinistraße 64, 20251, Hamburg, Germany
| | - Martin Stephan Spitzer
- Department of Ophthalmology, University Medical Center Hamburg - Eppendorf, Martinistr. 52, 20249, Hamburg, Germany
| | - Daniel Yap
- TeleMedC Pty Ltd, 61 Ubi Avenue 1, #06-11 UBPoint, Singapore, 40894, Singapore
| | - Di Xiao
- TeleMedC Pty Ltd, Brisbane Technology Park, Level 2, 1 Westlink Court, Darra, QLD 4076, Australia
| |
Collapse
|
7
|
Chia MA, Hersch F, Sayres R, Bavishi P, Tiwari R, Keane PA, Turner AW. Validation of a deep learning system for the detection of diabetic retinopathy in Indigenous Australians. Br J Ophthalmol 2024; 108:268-273. [PMID: 36746615 PMCID: PMC10850716 DOI: 10.1136/bjo-2022-322237] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2022] [Accepted: 12/31/2022] [Indexed: 02/08/2023]
Abstract
BACKGROUND/AIMS Deep learning systems (DLSs) for diabetic retinopathy (DR) detection show promising results but can underperform in racial and ethnic minority groups, therefore external validation within these populations is critical for health equity. This study evaluates the performance of a DLS for DR detection among Indigenous Australians, an understudied ethnic group who suffer disproportionately from DR-related blindness. METHODS We performed a retrospective external validation study comparing the performance of a DLS against a retinal specialist for the detection of more-than-mild DR (mtmDR), vision-threatening DR (vtDR) and all-cause referable DR. The validation set consisted of 1682 consecutive, single-field, macula-centred retinal photographs from 864 patients with diabetes (mean age 54.9 years, 52.4% women) at an Indigenous primary care service in Perth, Australia. Three-person adjudication by a panel of specialists served as the reference standard. RESULTS For mtmDR detection, sensitivity of the DLS was superior to the retina specialist (98.0% (95% CI, 96.5 to 99.4) vs 87.1% (95% CI, 83.6 to 90.6), McNemar's test p<0.001) with a small reduction in specificity (95.1% (95% CI, 93.6 to 96.4) vs 97.0% (95% CI, 95.9 to 98.0), p=0.006). For vtDR, the DLS's sensitivity was again superior to the human grader (96.2% (95% CI, 93.4 to 98.6) vs 84.4% (95% CI, 79.7 to 89.2), p<0.001) with a slight drop in specificity (95.8% (95% CI, 94.6 to 96.9) vs 97.8% (95% CI, 96.9 to 98.6), p=0.002). For all-cause referable DR, there was a substantial increase in sensitivity (93.7% (95% CI, 91.8 to 95.5) vs 74.4% (95% CI, 71.1 to 77.5), p<0.001) and a smaller reduction in specificity (91.7% (95% CI, 90.0 to 93.3) vs 96.3% (95% CI, 95.2 to 97.4), p<0.001). CONCLUSION The DLS showed improved sensitivity and similar specificity compared with a retina specialist for DR detection. This demonstrates its potential to support DR screening among Indigenous Australians, an underserved population with a high burden of diabetic eye disease.
Collapse
Affiliation(s)
- Mark A Chia
- Institute of Ophthalmology, University College London, London, UK
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
- Lions Outback Vision, Lions Eye Institute, Nedlands, Western Australia, Australia
| | | | | | | | | | - Pearse A Keane
- Institute of Ophthalmology, University College London, London, UK
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Angus W Turner
- Lions Outback Vision, Lions Eye Institute, Nedlands, Western Australia, Australia
- Centre for Ophthalmology and Visual Science, The University of Western Australia, Nedlands, Western Australia, Australia
| |
Collapse
|
8
|
Hu W, Joseph S, Li R, Woods E, Sun J, Shen M, Jan CL, Zhu Z, He M, Zhang L. Population impact and cost-effectiveness of artificial intelligence-based diabetic retinopathy screening in people living with diabetes in Australia: a cost effectiveness analysis. EClinicalMedicine 2024; 67:102387. [PMID: 38314061 PMCID: PMC10837545 DOI: 10.1016/j.eclinm.2023.102387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Revised: 11/29/2023] [Accepted: 12/05/2023] [Indexed: 02/06/2024] Open
Abstract
Background We aimed to evaluate the cost-effectiveness of an artificial intelligence-(AI) based diabetic retinopathy (DR) screening system in the primary care setting for both non-Indigenous and Indigenous people living with diabetes in Australia. Methods We performed a cost-effectiveness analysis between January 01, 2022 and August 01, 2023. A decision-analytic Markov model was constructed to simulate DR progression in a population of 1,197,818 non-Indigenous and 65,160 Indigenous Australians living with diabetes aged ≥20 years over 40 years. From a healthcare provider's perspective, we compared current practice to three primary care AI-based screening scenarios-(A) substitution of current manual grading, (B) scaling up to patient acceptance level, and (C) achieving universal screening. Study results were presented as incremental cost-effectiveness ratio (ICER), benefit-cost ratio (BCR), and net monetary benefits (NMB). A Willingness-to-pay (WTP) threshold of AU$50,000 per quality-adjusted life year (QALY) and a discount rate of 3.5% were adopted in this study. Findings With the status quo, the non-Indigenous diabetic population was projected to develop 96,269 blindness cases, resulting in AU$13,039.6 m spending on DR screening and treatment during 2020-2060. In comparison, all three intervention scenarios were effective and cost-saving. In particular, if a universal screening program was to be implemented (Scenario C), it would prevent 38,347 blindness cases, gain 172,090 QALYs and save AU$595.8 m, leading to a BCR of 3.96 and NMB of AU$9,200 m. Similar findings were also reported in the Indigenous population. With the status quo, 3,396 Indigenous individuals would develop blindness, which would cost the health system AU$796.0 m during 2020-2060. All three intervention scenarios were cost-saving for the Indigenous population. Notably, universal AI-based DR screening (Scenario C) would prevent 1,211 blindness cases and gain 9,800 QALYs in the Indigenous population, leading to a saving of AU$19.2 m with a BCR of 1.62 and NMB of AU$509 m. Interpretation Our findings suggest that implementing AI-based DR screening in primary care is highly effective and cost-saving in both Indigenous and non-Indigenous populations. Funding This project received grant funding from the Australian Government: the National Critical Research Infrastructure Initiative, Medical Research Future Fund (MRFAI00035) and the NHMRC Investigator Grant (APP1175405). The contents of the published material are solely the responsibility of the Administering Institution, a participating institution or individual authors and do not reflect the views of the NHMRC. This work was supported by the Global STEM Professorship Scheme (P0046113), the Fundamental Research Funds of the State Key Laboratory of Ophthalmology, Project of Investigation on Health Status of Employees in Financial Industry in Guangzhou, China (Z012014075). The Centre for Eye Research Australia receives Operational Infrastructure Support from the Victorian State Government. W.H. is supported by the Melbourne Research Scholarship established by the University of Melbourne. The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Collapse
Affiliation(s)
- Wenyi Hu
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Sanil Joseph
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Rui Li
- Central Clinical School, Faculty of Medicine, Monash University, Melbourne, VIC, Australia
- Artificial Intelligence and Modelling in Epidemiology Program, Melbourne Sexual Health Centre, Alfred Health, Melbourne, VIC, Australia
- China-Australia Joint Research Center for Infectious Diseases, School of Public Health, Xi’an Jiaotong University Health Science Center, Xi’an, Shaanxi, 710061, PR China
| | - Ekaterina Woods
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Jason Sun
- Eyetelligence Pty Ltd., Melbourne, Australia
| | - Mingwang Shen
- China-Australia Joint Research Center for Infectious Diseases, School of Public Health, Xi’an Jiaotong University Health Science Center, Xi’an, Shaanxi, 710061, PR China
| | - Catherine Lingxue Jan
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Zhuoting Zhu
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
| | - Mingguang He
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Australia
- School of Optometry, The Hong Kong Polytechnic University, Hong Kong, China
- Research Centre for SHARP Vision, The Hong Kong Polytechnic University, Kowloon, Hong Kong SAR, China
| | - Lei Zhang
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Clinical Medical Research Center, Children's Hospital of Nanjing Medical University, Nanjing, Jiangsu Province 210008, China
- Central Clinical School, Faculty of Medicine, Monash University, Melbourne, VIC, Australia
- Artificial Intelligence and Modelling in Epidemiology Program, Melbourne Sexual Health Centre, Alfred Health, Melbourne, VIC, Australia
| |
Collapse
|
9
|
Glauberman G, Ito-Fujita A, Katz S, Callahan J. Artificial Intelligence in Nursing Education: Opportunities and Challenges. HAWAI'I JOURNAL OF HEALTH & SOCIAL WELFARE 2023; 82:302-305. [PMID: 38093763 PMCID: PMC10713739] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Affiliation(s)
- Gary Glauberman
- Nancy Atmospera-Walch School of Nursing, University of Hawai'i at Mānoa, Honolulu HI
| | - Avree Ito-Fujita
- Nancy Atmospera-Walch School of Nursing, University of Hawai'i at Mānoa, Honolulu HI
| | - Shayna Katz
- Nancy Atmospera-Walch School of Nursing, University of Hawai'i at Mānoa, Honolulu HI
| | - James Callahan
- Nancy Atmospera-Walch School of Nursing, University of Hawai'i at Mānoa, Honolulu HI
| |
Collapse
|
10
|
Vo V, Chen G, Aquino YSJ, Carter SM, Do QN, Woode ME. Multi-stakeholder preferences for the use of artificial intelligence in healthcare: A systematic review and thematic analysis. Soc Sci Med 2023; 338:116357. [PMID: 37949020 DOI: 10.1016/j.socscimed.2023.116357] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2023] [Revised: 09/04/2023] [Accepted: 10/24/2023] [Indexed: 11/12/2023]
Abstract
INTRODUCTION Despite the proliferation of Artificial Intelligence (AI) technology over the last decade, clinician, patient, and public perceptions of its use in healthcare raise a number of ethical, legal and social questions. We systematically review the literature on attitudes towards the use of AI in healthcare from patients, the general public and health professionals' perspectives to understand these issues from multiple perspectives. METHODOLOGY A search for original research articles using qualitative, quantitative, and mixed methods published between 1 Jan 2001 to 24 Aug 2021 was conducted on six bibliographic databases. Data were extracted and classified into different themes representing views on: (i) knowledge and familiarity of AI, (ii) AI benefits, risks, and challenges, (iii) AI acceptability, (iv) AI development, (v) AI implementation, (vi) AI regulations, and (vii) Human - AI relationship. RESULTS The final search identified 7,490 different records of which 105 publications were selected based on predefined inclusion/exclusion criteria. While the majority of patients, the general public and health professionals generally had a positive attitude towards the use of AI in healthcare, all groups indicated some perceived risks and challenges. Commonly perceived risks included data privacy; reduced professional autonomy; algorithmic bias; healthcare inequities; and greater burnout to acquire AI-related skills. While patients had mixed opinions on whether healthcare workers suffer from job loss due to the use of AI, health professionals strongly indicated that AI would not be able to completely replace them in their professions. Both groups shared similar doubts about AI's ability to deliver empathic care. The need for AI validation, transparency, explainability, and patient and clinical involvement in the development of AI was emphasised. To help successfully implement AI in health care, most participants envisioned that an investment in training and education campaigns was necessary, especially for health professionals. Lack of familiarity, lack of trust, and regulatory uncertainties were identified as factors hindering AI implementation. Regarding AI regulations, key themes included data access and data privacy. While the general public and patients exhibited a willingness to share anonymised data for AI development, there remained concerns about sharing data with insurance or technology companies. One key domain under this theme was the question of who should be held accountable in the case of adverse events arising from using AI. CONCLUSIONS While overall positivity persists in attitudes and preferences toward AI use in healthcare, some prevalent problems require more attention. There is a need to go beyond addressing algorithm-related issues to look at the translation of legislation and guidelines into practice to ensure fairness, accountability, transparency, and ethics in AI.
Collapse
Affiliation(s)
- Vinh Vo
- Centre for Health Economics, Monash University, Australia.
| | - Gang Chen
- Centre for Health Economics, Monash University, Australia
| | - Yves Saint James Aquino
- Australian Centre for Health Engagement, Evidence and Values, School of Health and Soceity, University of Wollongong, Australia
| | - Stacy M Carter
- Australian Centre for Health Engagement, Evidence and Values, School of Health and Soceity, University of Wollongong, Australia
| | - Quynh Nga Do
- Department of Economics, Monash University, Australia
| | - Maame Esi Woode
- Centre for Health Economics, Monash University, Australia; Monash Data Futures Research Institute, Australia
| |
Collapse
|
11
|
Curran K, Whitestone N, Zabeen B, Ahmed M, Husain L, Alauddin M, Hossain MA, Patnaik JL, Lanoutee G, Cherwek DH, Congdon N, Peto T, Jaccard N. CHILDSTAR: CHIldren Living With Diabetes See and Thrive with AI Review. Clin Med Insights Endocrinol Diabetes 2023; 16:11795514231203867. [PMID: 37822362 PMCID: PMC10563496 DOI: 10.1177/11795514231203867] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Accepted: 08/23/2023] [Indexed: 10/13/2023] Open
Abstract
Background Artificial intelligence (AI) appears capable of detecting diabetic retinopathy (DR) with a high degree of accuracy in adults; however, there are few studies in children and young adults. Methods Children and young adults (3-26 years) with type 1 diabetes mellitus (T1DM) or type 2 diabetes mellitus (T2DM) were screened at the Dhaka BIRDEM-2 hospital, Bangladesh. All gradable fundus images were uploaded to Cybersight AI for interpretation. Two main outcomes were considered at a patient level: 1) Any DR, defined as mild non-proliferative diabetic retinopathy (NPDR or more severe; and 2) Referable DR, defined as moderate NPDR or more severe. Diagnostic test performance comparing Orbis International's Cybersight AI with the reference standard, a fully qualified optometrist certified in DR grading, was assessed using the Matthews correlation coefficient (MCC), area under the receiver operating characteristic curve (AUC-ROC), area under the precision-recall curve (AUC-PR), sensitivity, specificity, positive and negative predictive values. Results Among 1274 participants (53.1% female, mean age 16.7 years), 19.4% (n = 247) had any DR according to AI. For referable DR, 2.35% (n = 30) were detected by AI. The sensitivity and specificity of AI for any DR were 75.5% (CI 69.7-81.3%) and 91.8% (CI 90.2-93.5%) respectively, and for referable DR, these values were 84.2% (CI 67.8-100%) and 98.9% (CI 98.3%-99.5%). The MCC, AUC-ROC and the AUC-PR for referable DR were 63.4, 91.2 and 76.2% respectively. AI was most successful in accurately classifying younger children with shorter duration of diabetes. Conclusions Cybersight AI accurately detected any DR and referable DR among children and young adults, despite its algorithms having been trained on adults. The observed high specificity is particularly important to avoid over-referral in low-resource settings. AI may be an effective tool to reduce demands on scarce physician resources for the care of children with diabetes in low-resource settings.
Collapse
Affiliation(s)
- Katie Curran
- Centre for Public Health, Queens University Belfast, Belfast, UK
| | | | - Bedowra Zabeen
- Department of Paediatrics, Life for a Child & Changing Diabetes in Children Programme, Bangladesh Institute of Research & Rehabilitation in Diabetes, Endocrine & Metabolic Disorders (BIRDEM), Diabetic Association of Bangladesh, Dhaka, Bangladesh
| | | | | | | | | | - Jennifer L Patnaik
- Orbis International, New York, NY, USA
- Department of Ophthalmology, University of Colorado School of Medicine, Aurora, CO, USA
| | | | | | - Nathan Congdon
- Centre for Public Health, Queens University Belfast, Belfast, UK
- Orbis International, New York, NY, USA
- Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Tunde Peto
- Centre for Public Health, Queens University Belfast, Belfast, UK
| | | |
Collapse
|
12
|
Rajesh AE, Davidson OQ, Lee CS, Lee AY. Artificial Intelligence and Diabetic Retinopathy: AI Framework, Prospective Studies, Head-to-head Validation, and Cost-effectiveness. Diabetes Care 2023; 46:1728-1739. [PMID: 37729502 PMCID: PMC10516248 DOI: 10.2337/dci23-0032] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/25/2023] [Accepted: 07/15/2023] [Indexed: 09/22/2023]
Abstract
Current guidelines recommend that individuals with diabetes receive yearly eye exams for detection of referable diabetic retinopathy (DR), one of the leading causes of new-onset blindness. For addressing the immense screening burden, artificial intelligence (AI) algorithms have been developed to autonomously screen for DR from fundus photography without human input. Over the last 10 years, many AI algorithms have achieved good sensitivity and specificity (>85%) for detection of referable DR compared with human graders; however, many questions still remain. In this narrative review on AI in DR screening, we discuss key concepts in AI algorithm development as a background for understanding the algorithms. We present the AI algorithms that have been prospectively validated against human graders and demonstrate the variability of reference standards and cohort demographics. We review the limited head-to-head validation studies where investigators attempt to directly compare the available algorithms. Next, we discuss the literature regarding cost-effectiveness, equity and bias, and medicolegal considerations, all of which play a role in the implementation of these AI algorithms in clinical practice. Lastly, we highlight ongoing efforts to bridge gaps in AI model data sets to pursue equitable development and delivery.
Collapse
Affiliation(s)
- Anand E. Rajesh
- Department of Ophthalmology, University of Washington, Seattle, WA
- Roger H. and Angie Karalis Johnson Retina Center, Seattle, WA
| | - Oliver Q. Davidson
- Department of Ophthalmology, University of Washington, Seattle, WA
- Roger H. and Angie Karalis Johnson Retina Center, Seattle, WA
| | - Cecilia S. Lee
- Department of Ophthalmology, University of Washington, Seattle, WA
- Roger H. and Angie Karalis Johnson Retina Center, Seattle, WA
| | - Aaron Y. Lee
- Department of Ophthalmology, University of Washington, Seattle, WA
- Roger H. and Angie Karalis Johnson Retina Center, Seattle, WA
| |
Collapse
|
13
|
Uy H, Fielding C, Hohlfeld A, Ochodo E, Opare A, Mukonda E, Minnies D, Engel ME. Diagnostic test accuracy of artificial intelligence in screening for referable diabetic retinopathy in real-world settings: A systematic review and meta-analysis. PLOS GLOBAL PUBLIC HEALTH 2023; 3:e0002160. [PMID: 37729122 PMCID: PMC10511145 DOI: 10.1371/journal.pgph.0002160] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Accepted: 08/23/2023] [Indexed: 09/22/2023]
Abstract
Retrospective studies on artificial intelligence (AI) in screening for diabetic retinopathy (DR) have shown promising results in addressing the mismatch between the capacity to implement DR screening and increasing DR incidence. This review sought to evaluate the diagnostic test accuracy (DTA) of AI in screening for referable diabetic retinopathy (RDR) in real-world settings. We searched CENTRAL, PubMed, CINAHL, Scopus, and Web of Science on 9 February 2023. We included prospective DTA studies assessing AI against trained human graders (HGs) in screening for RDR in patients with diabetes. Two reviewers independently extracted data and assessed methodological quality against QUADAS-2 criteria. We used the hierarchical summary receiver operating characteristics (HSROC) model to pool estimates of sensitivity and specificity and, forest plots and SROC plots to visually examine heterogeneity in accuracy estimates. From our initial search results of 3899 studies, we included 15 studies comprising 17 datasets. Meta-analyses revealed a sensitivity of 95.33% (95%CI: 90.60-100%) and specificity of 92.01% (95%CI: 87.61-96.42%) for patient-level analysis (10 datasets, N = 45,785) while, for the eye-level analysis, sensitivity was 91.24% (95%CI: 79.15-100%) and specificity, 93.90% (95%CI: 90.63-97.16%) (7 datasets, N = 15,390). Subgroup analyses did not provide variations in the diagnostic accuracy of country classification and DR classification criteria. However, a moderate increase was observed in diagnostic accuracy in the primary-level healthcare settings: sensitivity of 99.35% (95%CI: 96.85-100%), specificity of 93.72% (95%CI: 88.83-98.61%) and, a minimal decrease in the tertiary-level healthcare settings: sensitivity of 94.71% (95%CI: 89.00-100%), specificity of 90.88% (95%CI: 83.22-98.53%). Sensitivity analyses did not show any variations in studies that included diabetic macular edema in the RDR definition, nor studies with ≥3 HGs. This review provides evidence, for the first time from prospective studies, for the effectiveness of AI in screening for RDR in real-world settings. The results may serve to strengthen existing guidelines to improve current practices.
Collapse
Affiliation(s)
- Holijah Uy
- Community Eye Health Institute, Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa
| | - Christopher Fielding
- Division of Epidemiology and Biostatistics, School of Public Health, Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa
| | - Ameer Hohlfeld
- South African Medical Research Council, Cape Town, South Africa
| | - Eleanor Ochodo
- Centre for Global Health Research, Kenya Medical Research Institute, Nairobi, Kenya
- Centre for Evidence-Based Health Care, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Stellenbosch, South Africa
| | - Abraham Opare
- Community Eye Health Institute, Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa
| | - Elton Mukonda
- Division of Epidemiology and Biostatistics, School of Public Health, Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa
| | - Deon Minnies
- Community Eye Health Institute, Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa
| | - Mark E. Engel
- South African Medical Research Council, Cape Town, South Africa
- Department of Medicine, University of Cape Town, Cape Town, South Africa
| |
Collapse
|
14
|
Wang Z, Li Z, Li K, Mu S, Zhou X, Di Y. Performance of artificial intelligence in diabetic retinopathy screening: a systematic review and meta-analysis of prospective studies. Front Endocrinol (Lausanne) 2023; 14:1197783. [PMID: 37383397 PMCID: PMC10296189 DOI: 10.3389/fendo.2023.1197783] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 05/23/2023] [Indexed: 06/30/2023] Open
Abstract
Aims To systematically evaluate the diagnostic value of an artificial intelligence (AI) algorithm model for various types of diabetic retinopathy (DR) in prospective studies over the previous five years, and to explore the factors affecting its diagnostic effectiveness. Materials and methods A search was conducted in Cochrane Library, Embase, Web of Science, PubMed, and IEEE databases to collect prospective studies on AI models for the diagnosis of DR from January 2017 to December 2022. We used QUADAS-2 to evaluate the risk of bias in the included studies. Meta-analysis was performed using MetaDiSc and STATA 14.0 software to calculate the combined sensitivity, specificity, positive likelihood ratio, and negative likelihood ratio of various types of DR. Diagnostic odds ratios, summary receiver operating characteristic (SROC) plots, coupled forest plots, and subgroup analysis were performed according to the DR categories, patient source, region of study, and quality of literature, image, and algorithm. Results Finally, 21 studies were included. Meta-analysis showed that the pooled sensitivity, specificity, pooled positive likelihood ratio, pooled negative likelihood ratio, area under the curve, Cochrane Q index, and pooled diagnostic odds ratio of AI model for the diagnosis of DR were 0.880 (0.875-0.884), 0.912 (0.99-0.913), 13.021 (10.738-15.789), 0.083 (0.061-0.112), 0.9798, 0.9388, and 206.80 (124.82-342.63), respectively. The DR categories, patient source, region of study, sample size, quality of literature, image, and algorithm may affect the diagnostic efficiency of AI for DR. Conclusion AI model has a clear diagnostic value for DR, but it is influenced by many factors that deserve further study. Systematic review registration https://www.crd.york.ac.uk/prospero/, identifier CRD42023389687.
Collapse
|
15
|
Artificial Intelligence for Diabetic Retinopathy Screening Using Color Retinal Photographs: From Development to Deployment. Ophthalmol Ther 2023; 12:1419-1437. [PMID: 36862308 PMCID: PMC10164194 DOI: 10.1007/s40123-023-00691-3] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Accepted: 02/14/2023] [Indexed: 03/03/2023] Open
Abstract
Diabetic retinopathy (DR), a leading cause of preventable blindness, is expected to remain a growing health burden worldwide. Screening to detect early sight-threatening lesions of DR can reduce the burden of vision loss; nevertheless, the process requires intensive manual labor and extensive resources to accommodate the increasing number of patients with diabetes. Artificial intelligence (AI) has been shown to be an effective tool which can potentially lower the burden of screening DR and vision loss. In this article, we review the use of AI for DR screening on color retinal photographs in different phases of application, ranging from development to deployment. Early studies of machine learning (ML)-based algorithms using feature extraction to detect DR achieved a high sensitivity but relatively lower specificity. Robust sensitivity and specificity were achieved with the application of deep learning (DL), although ML is still used in some tasks. Public datasets were utilized in retrospective validations of the developmental phases in most algorithms, which require a large number of photographs. Large prospective clinical validation studies led to the approval of DL for autonomous screening of DR although the semi-autonomous approach may be preferable in some real-world settings. There have been few reports on real-world implementations of DL for DR screening. It is possible that AI may improve some real-world indicators for eye care in DR, such as increased screening uptake and referral adherence, but this has not been proven. The challenges in deployment may include workflow issues, such as mydriasis to lower ungradable cases; technical issues, such as integration into electronic health record systems and integration into existing camera systems; ethical issues, such as data privacy and security; acceptance of personnel and patients; and health-economic issues, such as the need to conduct health economic evaluations of using AI in the context of the country. The deployment of AI for DR screening should follow the governance model for AI in healthcare which outlines four main components: fairness, transparency, trustworthiness, and accountability.
Collapse
|
16
|
Zhang J, Zou H. Artificial intelligence technology for myopia challenges: A review. Front Cell Dev Biol 2023; 11:1124005. [PMID: 36733459 PMCID: PMC9887165 DOI: 10.3389/fcell.2023.1124005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2022] [Accepted: 01/10/2023] [Indexed: 01/19/2023] Open
Abstract
Myopia is a significant global health concern and affects human visual function, resulting in blurred vision at a distance. There are still many unsolved challenges in this field that require the help of new technologies. Currently, artificial intelligence (AI) technology is dominating medical image and data analysis and has been introduced to address challenges in the clinical practice of many ocular diseases. AI research in myopia is still in its early stages. Understanding the strengths and limitations of each AI method in specific tasks of myopia could be of great value and might help us to choose appropriate approaches for different tasks. This article reviews and elaborates on the technical details of AI methods applied for myopia risk prediction, screening and diagnosis, pathogenesis, and treatment.
Collapse
Affiliation(s)
- Juzhao Zhang
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Haidong Zou
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China,Shanghai Eye Diseases Prevention and Treatment Center, Shanghai Eye Hospital, Shanghai, China,National Clinical Research Center for Eye Diseases, Shanghai, China,Shanghai Engineering Center for Precise Diagnosis and Treatment of Eye Diseases, Shanghai, China,*Correspondence: Haidong Zou,
| |
Collapse
|
17
|
Nanegrungsunk O, Ruamviboonsuk P, Grzybowski A. Prospective studies on artificial intelligence (AI)-based diabetic retinopathy screening. ANNALS OF TRANSLATIONAL MEDICINE 2022; 10:1297. [PMID: 36660630 PMCID: PMC9843399 DOI: 10.21037/atm-2022-71] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Accepted: 12/06/2022] [Indexed: 12/23/2022]
Affiliation(s)
- Onnisa Nanegrungsunk
- Retina Division, Department of Ophthalmology, Faculty of Medicine, Chiang Mai University, Chiang Mai, Thailand
| | - Paisan Ruamviboonsuk
- Department of Ophthalmology, College of Medicine, Rangsit University, Rajavithi Hospital, Bangkok, Thailand
| | - Andrzej Grzybowski
- Department of Ophthalmology, University of Warmia and Mazury, Olsztyn, Poland;,Institute for Research in Ophthalmology, Foundation for Ophthalmology Development, Poznan, Poland
| |
Collapse
|
18
|
Mathenge W, Whitestone N, Nkurikiye J, Patnaik JL, Piyasena P, Uwaliraye P, Lanouette G, Kahook MY, Cherwek DH, Congdon N, Jaccard N. Impact of Artificial Intelligence Assessment of Diabetic Retinopathy on Referral Service Uptake in a Low-Resource Setting: The RAIDERS Randomized Trial. OPHTHALMOLOGY SCIENCE 2022; 2:100168. [PMID: 36531575 PMCID: PMC9754978 DOI: 10.1016/j.xops.2022.100168] [Citation(s) in RCA: 18] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/25/2021] [Revised: 04/21/2022] [Accepted: 04/25/2022] [Indexed: 06/02/2023]
Abstract
PURPOSE This trial was designed to determine if artificial intelligence (AI)-supported diabetic retinopathy (DR) screening improved referral uptake in Rwanda. DESIGN The Rwanda Artificial Intelligence for Diabetic Retinopathy Screening (RAIDERS) study was an investigator-masked, parallel-group randomized controlled trial. PARTICIPANTS Patients ≥ 18 years of age with known diabetes who required referral for DR based on AI interpretation. METHODS The RAIDERS study screened for DR using retinal imaging with AI interpretation implemented at 4 facilities from March 2021 through July 2021. Eligible participants were assigned randomly (1:1) to immediate feedback of AI grading (intervention) or communication of referral advice after human grading was completed 3 to 5 days after the initial screening (control). MAIN OUTCOME MEASURES Difference between study groups in the rate of presentation for referral services within 30 days of being informed of the need for a referral visit. RESULTS Of the 823 clinic patients who met inclusion criteria, 275 participants (33.4%) showed positive findings for referable DR based on AI screening and were randomized for inclusion in the trial. Study participants (mean age, 50.7 years; 58.2% women) were randomized to the intervention (n = 136 [49.5%]) or control (n = 139 [50.5%]) groups. No significant intergroup differences were found at baseline, and main outcome data were available for analyses for 100% of participants. Referral adherence was statistically significantly higher in the intervention group (70/136 [51.5%]) versus the control group (55/139 [39.6%]; P = 0.048), a 30.1% increase. Older age (odds ratio [OR], 1.04; 95% confidence interval [CI], 1.02-1.05; P < 0.0001), male sex (OR, 2.07; 95% CI, 1.22-3.51; P = 0.007), rural residence (OR, 1.79; 95% CI, 1.07-3.01; P = 0.027), and intervention group (OR, 1.74; 95% CI, 1.05-2.88; P = 0.031) were statistically significantly associated with acceptance of referral in multivariate analyses. CONCLUSIONS Immediate feedback on referral status based on AI-supported screening was associated with statistically significantly higher referral adherence compared with delayed communications of results from human graders. These results provide evidence for an important benefit of AI screening in promoting adherence to prescribed treatment for diabetic eye care in sub-Saharan Africa.
Collapse
Affiliation(s)
- Wanjiku Mathenge
- Rwanda International Institute of Ophthalmology, Kigali, Rwanda
- Orbis International, New York, New York
| | | | - John Nkurikiye
- Rwanda International Institute of Ophthalmology, Kigali, Rwanda
- Rwanda Military Hospital, Kigali, Rwanda
| | - Jennifer L. Patnaik
- Orbis International, New York, New York
- Department of Ophthalmology, University of Colorado School of Medicine, Aurora, Colorado
| | - Prabhath Piyasena
- Centre for Public Health, Queen’s University Belfast, Belfast, United Kingdom
| | | | | | - Malik Y. Kahook
- Department of Ophthalmology, University of Colorado School of Medicine, Aurora, Colorado
| | | | - Nathan Congdon
- Orbis International, New York, New York
- Centre for Public Health, Queen’s University Belfast, Belfast, United Kingdom
- Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | | |
Collapse
|
19
|
Kiburg KV, Turner A, He M. Telemedicine and delivery of ophthalmic care in rural and remote communities: Drawing from Australian experience. Clin Exp Ophthalmol 2022; 50:793-800. [PMID: 35975938 DOI: 10.1111/ceo.14147] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2022] [Revised: 08/07/2022] [Accepted: 08/12/2022] [Indexed: 01/07/2023]
Abstract
Rural and remote communities in Australia are characterised by small but widely dispersed populations. This has been proven to be a major hurdle in access to medical care services with screening and treatment goals repeatedly being missed. Telemedicine in ophthalmology provides the opportunity to increase the availability of high quality and timely access to healthcare within. Recent years has also seen the introduction of artificial intelligence (AI) in ophthalmology, particularly in the screening of diseases. AI will hopefully increase the number of appropriate referrals, reduce travel time for patients and ensure timely triage given the low number of qualified optometrists and ophthalmologists. Telemedicine and AI has been introduced in a number of countries and has led to tremendous benefits and advantages when compared to standard practices. This paper summarises current practices in telemedicine and AI and the future of this technology in improving patient care in the field of ophthalmology.
Collapse
Affiliation(s)
- Katerina V Kiburg
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Victoria, Australia.,Ophthalmology, Department of Surgery, University of Melbourne, Melbourne, Victoria, Australia
| | - Angus Turner
- Lions Outback Vision, Lions Eye Institute, Nedlands, Western Australia, Australia.,Centre for Ophthalmology and Visual Science, University of Western Australia, Nedlands, Western Australia, Australia
| | - Mingguang He
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Victoria, Australia.,Ophthalmology, Department of Surgery, University of Melbourne, Melbourne, Victoria, Australia
| |
Collapse
|
20
|
Chia MA, Turner AW. Benefits of Integrating Telemedicine and Artificial Intelligence Into Outreach Eye Care: Stepwise Approach and Future Directions. Front Med (Lausanne) 2022; 9:835804. [PMID: 35391876 PMCID: PMC8982071 DOI: 10.3389/fmed.2022.835804] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2021] [Accepted: 02/15/2022] [Indexed: 11/13/2022] Open
Abstract
Telemedicine has traditionally been applied within remote settings to overcome geographical barriers to healthcare access, providing an alternate means of connecting patients to specialist services. The coronavirus 2019 pandemic has rapidly expanded the use of telemedicine into metropolitan areas and enhanced global telemedicine capabilities. Through our experience of delivering real-time telemedicine over the past decade within a large outreach eye service, we have identified key themes for successful implementation which may be relevant to services facing common challenges. We present our journey toward establishing a comprehensive teleophthalmology model built on the principles of collaborative care, with a focus on delivering practical lessons for service design. Artificial intelligence is an emerging technology that has shown potential to further address resource limitations. We explore the applications of artificial intelligence and the need for targeted research within underserved settings in order to meet growing healthcare demands. Based on our rural telemedicine experience, we make the case that similar models may be adapted to urban settings with the aim of reducing surgical waitlists and improving efficiency.
Collapse
Affiliation(s)
- Mark A. Chia
- Lions Outback Vision, Lions Eye Institute, Nedlands, WA, Australia
- Institute of Ophthalmology, Faculty of Brain Sciences, University College London, London, United Kingdom
| | - Angus W. Turner
- Lions Outback Vision, Lions Eye Institute, Nedlands, WA, Australia
- Centre for Ophthalmology and Visual Science, University of Western Australia, Nedlands, WA, Australia
- *Correspondence: Angus W. Turner
| |
Collapse
|
21
|
Wang Y, Shi D, Tan Z, Niu Y, Jiang Y, Xiong R, Peng G, He M. Screening Referable Diabetic Retinopathy Using a Semi-automated Deep Learning Algorithm Assisted Approach. Front Med (Lausanne) 2021; 8:740987. [PMID: 34901058 PMCID: PMC8656222 DOI: 10.3389/fmed.2021.740987] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2021] [Accepted: 11/08/2021] [Indexed: 12/23/2022] Open
Abstract
Purpose: To assess the accuracy and efficacy of a semi-automated deep learning algorithm (DLA) assisted approach to detect vision-threatening diabetic retinopathy (DR). Methods: We developed a two-step semi-automated DLA-assisted approach to grade fundus photographs for vision-threatening referable DR. Study images were obtained from the Lingtou Cohort Study, and captured at participant enrollment in 2009–2010 (“baseline images”) and annual follow-up between 2011 and 2017. To begin, a validated DLA automatically graded baseline images for referable DR and classified them as positive, negative, or ungradable. Following, each positive image, all other available images from patients who had a positive image, and a 5% random sample of all negative images were selected and regraded by trained human graders. A reference standard diagnosis was assigned once all graders achieved consistent grading outcomes or with a senior ophthalmologist's final diagnosis. The semi-automated DLA assisted approach combined initial DLA screening and subsequent human grading for images identified as high-risk. This approach was further validated within the follow-up image datasets and its time and economic costs evaluated against fully human grading. Results: For evaluation of baseline images, a total of 33,115 images were included and automatically graded by the DLA. 2,604 images (480 positive results, 624 available other images from participants with a positive result, and 1500 random negative samples) were selected and regraded by graders. The DLA achieved an area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and accuracy of 0.953, 0.970, 0.879, and 88.6%, respectively. In further validation within the follow-up image datasets, a total of 88,363 images were graded using this semi-automated approach and human grading was performed on 8975 selected images. The DLA achieved an AUC, sensitivity, and specificity of 0.914, 0.852, 0.853, respectively. Compared against fully human grading, the semi-automated DLA-assisted approach achieved an estimated 75.6% time and 90.1% economic cost saving. Conclusions: The DLA described in this study was able to achieve high accuracy, sensitivity, and specificity in grading fundus images for referable DR. Validated against long-term follow-up datasets, a semi-automated DLA-assisted approach was able to accurately identify suspect cases, and minimize misdiagnosis whilst balancing safety, time, and economic cost.
Collapse
Affiliation(s)
- Yueye Wang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Danli Shi
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Zachary Tan
- Centre for Eye Research Australia, University of Melbourne, Melbourne, VIC, Australia
| | - Yong Niu
- Department of Ophthalmology, Guangzhou No. 11 People's Hospital, Guangzhou, China
| | - Yu Jiang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Ruilin Xiong
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Guankai Peng
- Guangzhou Vision Tech Medical Technology Co. Ltd., Guangzhou, China
| | - Mingguang He
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China.,Centre for Eye Research Australia, University of Melbourne, Melbourne, VIC, Australia.,Department of Ophthalmology, Guangdong Eye Institute, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China.,Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, Melbourne, VIC, Australia
| |
Collapse
|