1
|
Qian X, Xian S, Yifei S, Wei G, Liu H, Xiaoming X, Chu C, Yilong Y, Shuang Y, Kai M, Mei C, Yi Q. External validation of a deep learning detection system for glaucomatous optic neuropathy: a real-world multicentre study. Eye (Lond) 2023; 37:3813-3818. [PMID: 37322379 PMCID: PMC10698045 DOI: 10.1038/s41433-023-02622-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2022] [Revised: 05/17/2023] [Accepted: 06/02/2023] [Indexed: 06/17/2023] Open
Abstract
OBJECTIVES To conduct an external validation of an automated artificial intelligence (AI) diagnostic system using fundus photographs from a real-life multicentre cohort. METHODS We designed external validation in multiple scenarios, consisting of 3049 images from Qilu Hospital of Shandong University in China (QHSDU, validation dataset 1), 7495 images from three other hospitals in China (validation dataset 2), and 516 images from high myopia (HM) population of QHSDU (validation dataset 3). The corresponding sensitivity, specificity and accuracy of this AI diagnostic system to identify glaucomatous optic neuropathy (GON) were calculated. RESULTS In validation datasets 1 and 2, the algorithm yielded accuracy of 93.18% and 91.40%, area under the receiver operating curves (AUC) of 95.17% and 96.64%, and significantly higher sensitivity of 91.75% and 91.41%, respectively, compared to manual graders. On the subsets complicated with retinal comorbidities, such as diabetic retinopathy or age-related macular degeneration, in validation datasets 1 and 2, the algorithm achieved accuracy of 87.54% and 93.81%, and AUC of 97.02% and 97.46%, respectively. In validation dataset 3, the algorithm achieved comparable accuracy of 81.98% and AUC of 87.49%, with a sensitivity of 83.61% and specificity of 81.76% on GON recognition specifically in the HM population. CONCLUSIONS With acceptable generalization capability across varying levels of image quality, different clinical centres, or certain retinal comorbidities, such as HM, the automatic AI diagnostic system had the potential to provide expert-level glaucoma detection.
Collapse
Affiliation(s)
- Xu Qian
- Department of Geriatric Medicine, Qilu Hospital of Shandong University, No. 107, Wenhuaxi Road, Jinan, 250012, China
- Key Laboratory of Cardiovascular Proteomics of Shandong Province, Jinan, China
- Jinan Clinical Research Center for Geriatric Medicine (202132001), Jinan, China
| | - Song Xian
- Department of Geriatric Medicine, Qilu Hospital of Shandong University, No. 107, Wenhuaxi Road, Jinan, 250012, China
- Key Laboratory of Cardiovascular Proteomics of Shandong Province, Jinan, China
- Jinan Clinical Research Center for Geriatric Medicine (202132001), Jinan, China
| | - Su Yifei
- Global Health Research Center, Duke Kunshan University, No. 8 Duke Avenue, Kunshan, Jiangsu Province, 215316, China
| | - Guo Wei
- Lunan Eye Hospital, Linyi, 276000, China
| | - Hanruo Liu
- Beijing Tongren Hospital, Capital Medical University, Beijing, 100730, China
- School of Information and Electronics, Beijing Institute of Technology, Beijing, 100730, China
| | - Xi Xiaoming
- School of Computer Science and Technology, Shandong Jianzhu University, Jinan, 250101, China
| | | | - Yin Yilong
- School of Software, Shandong University, Jinan, 250101, China
| | - Yu Shuang
- Tencent Healthcare, Shenzhen, 51800, China
| | - Ma Kai
- Tencent Healthcare, Shenzhen, 51800, China
| | - Cheng Mei
- Department of Geriatric Medicine, Qilu Hospital of Shandong University, No. 107, Wenhuaxi Road, Jinan, 250012, China
- Key Laboratory of Cardiovascular Proteomics of Shandong Province, Jinan, China
- Jinan Clinical Research Center for Geriatric Medicine (202132001), Jinan, China
| | - Qu Yi
- Department of Geriatric Medicine, Qilu Hospital of Shandong University, No. 107, Wenhuaxi Road, Jinan, 250012, China.
- Key Laboratory of Cardiovascular Proteomics of Shandong Province, Jinan, China.
- Jinan Clinical Research Center for Geriatric Medicine (202132001), Jinan, China.
| |
Collapse
|