Fairness and generalisability in deep learning of retinopathy of prematurity screening algorithms: a literature review

Image-free recognition of moderate ROP from mild with machine learning algorithm on plasma Raman spectrum

The retinopathy of prematurity (ROP) can cause serious clinical consequences and, fortunately, it is remediable while the time window for treatment is relatively narrow. Therefore, it is urgent to screen all premature infants and diagnose ROP degree timely, which has become a large workload for pediatric ophthalmologists. We developed a retinal image-free procedure using small amount of blood samples based on the plasma Raman spectrum with the machine learning model to automatically classify ROP cases before medical intervention was performed. Statistical differences in infrared Raman spectra of plasma samples were found among the control, mild (ZIIIS1), moderate (ZIIIS2 & ZIIS1), and advanced (ZIIS2) ROP groups. With the different wave points of Raman spectra as the inputs, the outputs of our support vector machine showed that the area under the curves in the receiver operating characteristic (AUC) were 0.763 for the pair comparisons of the control with the mild groups, 0.821 between moderate and advanced groups (ZIIS2), while more than 90% in comparisons of the other four pairs: control vs. moderate (0.981), control vs. advanced (0.963), mild vs. moderate (0.936), and mild vs. advanced (0.953), respectively. Our study could advance principally the ROP diagnosis in two dimensions: the moderate ROPs have been classified remarkably from the mild ones, which leaves more time for the medical treatments, and the procedure of Raman spectrum with a machine learning model based on blood samples can be conveniently promoted to those hospitals lacking of the pediatric ophthalmologists with experience in reading retinal images.

BRSET: A Brazilian Multilabel Ophthalmological Dataset of Retina Fundus Photos

Introduction

The Brazilian Multilabel Ophthalmological Dataset (BRSET) addresses the scarcity of publicly available ophthalmological datasets in Latin America. BRSET comprises 16,266 color fundus retinal photos from 8,524 Brazilian patients, aiming to enhance data representativeness, serving as a research and teaching tool. It contains sociodemographic information, enabling investigations into differential model performance across demographic groups.

Methods

Data from three São Paulo outpatient centers yielded demographic and medical information from electronic records, including nationality, age, sex, clinical history, insulin use, and duration of diabetes diagnosis. A retinal specialist labeled images for anatomical features (optic disc, blood vessels, macula), quality control (focus, illumination, image field, artifacts), and pathologies (e.g., diabetic retinopathy). Diabetic retinopathy was graded using International Clinic Diabetic Retinopathy and Scottish Diabetic Retinopathy Grading. Validation used Dino V2 Base for feature extraction, with 70% training and 30% testing subsets. Support Vector Machines (SVM) and Logistic Regression (LR) were employed with weighted training. Performance metrics included area under the receiver operating curve (AUC) and Macro F1-score.

Results

BRSET comprises 65.1% Canon CR2 and 34.9% Nikon NF5050 images. 61.8% of the patients are female, and the average age is 57.6 years. Diabetic retinopathy affected 15.8% of patients, across a spectrum of disease severity. Anatomically, 20.2% showed abnormal optic discs, 4.9% abnormal blood vessels, and 28.8% abnormal macula. Models were trained on BRSET in three prediction tasks: "diabetes diagnosis"; "sex classification"; and "diabetic retinopathy diagnosis".

Discussion

BRSET is the first multilabel ophthalmological dataset in Brazil and Latin America. It provides an opportunity for investigating model biases by evaluating performance across demographic groups. The model performance of three prediction tasks demonstrates the value of the dataset for external validation and for teaching medical computer vision to learners in Latin America using locally relevant data sources.