The Challenge of Diabetic Retinopathy Standardization in an Ophthalmological Dataset

Implementation of a Large-Scale Image Curation Workflow Using Deep Learning Framework

PURPOSE

The curation of images using human resources is time intensive but an essential step for developing artificial intelligence (AI) algorithms. Our goal was to develop and implement an AI algorithm for image curation in a high-volume setting. We also explored AI tools that will assist in deploying a tiered approach, in which the AI model labels images and flags potential mislabels for human review.

DESIGN

Implementation of an AI algorithm.

PARTICIPANTS

Seven-field stereoscopic images from multiple clinical trials.

METHODS

The 7-field stereoscopic image protocol includes 7 pairs of images from various parts of the central retina along with images of the anterior part of the eye. All images were labeled for field number by reading center graders. The model output included classification of the retinal images into 8 field numbers. Probability scores (0-1) were generated to identify misclassified images, with 1 indicating a high probability of a correct label.

MAIN OUTCOME MEASURES

Agreement of AI prediction with grader classification of field number and the use of probability scores to identify mislabeled images.

RESULTS

The AI model was trained and validated on 17 529 images and tested on 3004 images. The pooled agreement of field numbers between grader classification and the AI model was 88.3% (kappa, 0.87). The pooled mean probability score was 0.97 (standard deviation [SD], 0.08) for images for which the graders agreed with the AI-generated labels and 0.77 (SD, 0.19) for images for which the graders disagreed with the AI-generated labels (P < 0.0001). Using receiver operating characteristic curves, a probability score of 0.99 was identified as a cutoff for distinguishing mislabeled images. A tiered workflow using a probability score of < 0.99 as a cutoff would include 27.6% of the 3004 images for human review and reduce the error rate from 11.7% to 1.5%.

CONCLUSIONS

The implementation of AI algorithms requires measures in addition to model validation. Tools to flag potential errors in the labels generated by AI models will reduce inaccuracies, increase trust in the system, and provide data for continuous model development.

Diabetic retinopathy classification for supervised machine learning algorithms

BACKGROUND

Artificial intelligence and automated technology were first reported more than 70 years ago and nowadays provide unprecedented diagnostic accuracy, screening capacity, risk stratification, and workflow optimization. Diabetic retinopathy is an important cause of preventable blindness worldwide, and artificial intelligence technology provides precocious diagnosis, monitoring, and guide treatment. High-quality exams are fundamental in supervised artificial intelligence algorithms, but the lack of ground truth standards in retinal exams datasets is a problem.

MAIN BODY

In this article, ETDRS, NHS, ICDR, SDGS diabetic retinopathy grading, and manual annotation are described and compared in publicly available datasets. The various DR labeling systems generate a fundamental problem for AI datasets. Possible solutions are standardization of DR classification and direct retinal-finding identifications.

CONCLUSION

Reliable labeling methods also need to be considered in datasets with more trustworthy labeling.

[Ocular changes as a diagnostic tool for malaria]

BACKGROUND

According to the WHO Malaria Report 2019 a total of 229 million people fall ill with malaria each year and two thirds of deaths involve children under 5 years of age.

AIM

To review the fundus changes in the context of malaria and the importance of ophthalmoscopy in the diagnosis.

MATERIAL AND METHODS

Summary of changes in cerebral malaria visible on fundus examination, possible underlying pathomechanisms and the value of ophthalmoscopy in practice.

RESULTS

Retinal findings in malaria include white or gray staining of the retina (retinal whitening), color change of retinal vessels (orange or white staining), hemorrhages often with a white center, such as Roth's spot and papilledema.

DISCUSSION

The retinal changes in malaria are specific and may help to differentiate malaria from other causes of coma and fever. Smartphone-based fundus photography and artificial intelligence could support malaria diagnostics particularly in resource-poor regions.