Machine Learning Classifiers in Glaucoma

Multimodal Deep Learning Classifier for Primary Open Angle Glaucoma Diagnosis Using Wide-Field Optic Nerve Head Cube Scans in Eyes With and Without High Myopia

PRCIS

An optical coherence tomography (OCT)-based multimodal deep learning (DL) classification model, including texture information, is introduced that outperforms single-modal models and multimodal models without texture information for glaucoma diagnosis in eyes with and without high myopia.

BACKGROUND/AIMS

To evaluate the diagnostic accuracy of a multimodal DL classifier using wide OCT optic nerve head cube scans in eyes with and without axial high myopia.

MATERIALS AND METHODS

Three hundred seventy-one primary open angle glaucoma (POAG) eyes and 86 healthy eyes, all without axial high myopia [axial length (AL) ≤ 26 mm] and 92 POAG eyes and 44 healthy eyes, all with axial high myopia (AL > 26 mm) were included. The multimodal DL classifier combined features of 3 individual VGG-16 models: (1) texture-based en face image, (2) retinal nerve fiber layer (RNFL) thickness map image, and (3) confocal scanning laser ophthalmoscope (cSLO) image. Age, AL, and disc area adjusted area under the receiver operating curves were used to compare model accuracy.

RESULTS

Adjusted area under the receiver operating curve for the multimodal DL model was 0.91 (95% CI = 0.87, 0.95). This value was significantly higher than the values of individual models [0.83 (0.79, 0.86) for texture-based en face image; 0.84 (0.81, 0.87) for RNFL thickness map; and 0.68 (0.61, 0.74) for cSLO image; all P ≤ 0.05]. Using only highly myopic eyes, the multimodal DL model showed significantly higher diagnostic accuracy [0.89 (0.86, 0.92)] compared with texture en face image [0.83 (0.78, 0.85)], RNFL [0.85 (0.81, 0.86)] and cSLO image models [0.69 (0.63, 0.76)] (all P ≤ 0.05).

CONCLUSIONS

Combining OCT-based RNFL thickness maps with texture-based en face images showed a better ability to discriminate between healthy and POAG than thickness maps alone, particularly in high axial myopic eyes.

Predictive Analytics for Glaucoma Using Data From the All of Us Research Program

PURPOSE

To (1) use All of Us (AoU) data to validate a previously published single-center model predicting the need for surgery among individuals with glaucoma, (2) train new models using AoU data, and (3) share insights regarding this novel data source for ophthalmic research.

DESIGN

Development and evaluation of machine learning models.

METHODS

Electronic health record data were extracted from AoU for 1,231 adults diagnosed with primary open-angle glaucoma. The single-center model was applied to AoU data for external validation. AoU data were then used to train new models for predicting the need for glaucoma surgery using multivariable logistic regression, artificial neural networks, and random forests. Five-fold cross-validation was performed. Model performance was evaluated based on area under the receiver operating characteristic curve (AUC), accuracy, precision, and recall.

RESULTS

The mean (standard deviation) age of the AoU cohort was 69.1 (10.5) years, with 57.3% women and 33.5% black, significantly exceeding representation in the single-center cohort (P = .04 and P < .001, respectively). Of 1,231 participants, 286 (23.2%) needed glaucoma surgery. When applying the single-center model to AoU data, accuracy was 0.69 and AUC was only 0.49. Using AoU data to train new models resulted in superior performance: AUCs ranged from 0.80 (logistic regression) to 0.99 (random forests).

CONCLUSIONS

Models trained with national AoU data achieved superior performance compared with using single-center data. Although AoU does not currently include ophthalmic imaging, it offers several strengths over similar big-data sources such as claims data. AoU is a promising new data source for ophthalmic research.

Glaucoma classification based on scanning laser ophthalmoscopic images using a deep learning ensemble method

This study aimed to assess the utility of optic nerve head (onh) en-face images, captured with scanning laser ophthalmoscopy (slo) during standard optical coherence tomography (oct) imaging of the posterior segment, and demonstrate the potential of deep learning (dl) ensemble method that operates in a low data regime to differentiate glaucoma patients from healthy controls. The two groups of subjects were initially categorized based on a range of clinical tests including measurements of intraocular pressure, visual fields, oct derived retinal nerve fiber layer (rnfl) thickness and dilated stereoscopic examination of onh. 227 slo images of 227 subjects (105 glaucoma patients and 122 controls) were used. A new task-specific convolutional neural network architecture was developed for slo image-based classification. To benchmark the results of the proposed method, a range of classifiers were tested including five machine learning methods to classify glaucoma based on rnfl thickness—a well-known biomarker in glaucoma diagnostics, ensemble classifier based on inception v3 architecture, and classifiers based on features extracted from the image. The study shows that cross-validation dl ensemble based on slo images achieved a good discrimination performance with up to 0.962 of balanced accuracy, outperforming all of the other tested classifiers.

Functional assessment of glaucoma: Uncovering progression

Clinicians who manage glaucoma patients carefully monitor the visual field to determine if treatments are effective or interventions are needed. Visual field tests may reflect disease progression or variability among examinations. We describe the approaches and perimetric tests used to evaluate glaucomatous visual field progression and factors that are important for identifying progression. These include stimulus size, which area of the visual field to assess (central versus peripheral), and the testing frequency, evaluating which is important to detect change early while minimizing patient testing burden. We also review the different statistical methods developed to identify change. These include trend- and event-based analyses, parametric and nonparametric tests, population-based versus individualized approaches, as well as pointwise and global analyses. We hope this information will prove useful and important to enhance the management of glaucoma patients. Overall, analysis procedures based on series of at least 5 to 6 examinations that require confirmation and persistence of changes, that are guided by the pattern and shape of the glaucomatous visual field deficits, and that are consistent with structural defects provide the best clinical performance.

Correlating Structural and Functional Damage in Glaucoma

Structural and functional tests are essential for detecting and monitoring glaucomatous damage. However, the correlations between structural and functional tests in glaucoma are complex and faulty, with the combination of both modalities being recommended for better assessment of glaucoma. The objective of this review is to explore investigations from the last 5 years in the field of structure-function correlation in glaucoma that contributed to increment in the understanding of this correlation and have the potential to improve the diagnosis and detection of glaucoma progression.

Unsupervised feature extraction of anterior chamber OCT images for ordering and classification

We propose an image processing method for ordering anterior chamber optical coherence tomography (OCT) images in a fully unsupervised manner. The method consists of three steps: Firstly we preprocess the images (filtering the noise, aligning and normalizing the resolution); secondly, a distance measure between images is computed for every pair of images; thirdly we apply a machine learning algorithm that exploits the distance measure to order the images in a two-dimensional plane. The method is applied to a large (~1000) database of anterior chamber OCT images of healthy subjects and patients with angle-closure and the resulting unsupervised ordering and classification is validated by two ophthalmologists.

Automated algorithms combining structure and function outperform general ophthalmologists in diagnosing glaucoma

PURPOSE

To test the ability of machine learning classifiers (MLCs) using optical coherence tomography (OCT) and standard automated perimetry (SAP) parameters to discriminate between healthy and glaucomatous individuals, and to compare it to the diagnostic ability of the combined structure-function index (CSFI), general ophthalmologists and glaucoma specialists.

DESIGN

Cross-sectional prospective study.

METHODS

Fifty eight eyes of 58 patients with early to moderate glaucoma (median value of the mean deviation = -3.44 dB; interquartile range, -6.0 to -2.4 dB) and 66 eyes of 66 healthy individuals underwent OCT and SAP tests. The diagnostic accuracy (area under the ROC curve-AUC) of 10 MLCs was compared to those obtained with the CSFI, 3 general ophthalmologists and 3 glaucoma specialists exposed to the same OCT and SAP data.

RESULTS

The AUCs obtained with MLCs ranged from 0.805 (Classification Tree) to 0.931 (Radial Basis Function Network, RBF). The sensitivity at 90% specificity ranged from 51.6% (Classification Tree) to 82.8% (Bagging, Multilayer Perceptron and Support Vector Machine Gaussian). The CSFI had a sensitivity of 79.3% at 90% specificity, and the highest AUC (0.948). General ophthalmologists and glaucoma specialists' grading had sensitivities of 66.2% and 83.8% at 90% specificity, and AUCs of 0.879 and 0.921, respectively. RBF (the best MLC), the CSFI, and glaucoma specialists showed significantly higher AUCs than that obtained by general ophthalmologists (P<0.05). However, there were no significant differences between the AUCs obtained by RBF, the CSFI, and glaucoma specialists (P>0.25).

CONCLUSION

Our findings suggest that both MLCs and the CSFI can be helpful in clinical practice and effectively improve glaucoma diagnosis in the primary eye care setting, when there is no glaucoma specialist available.

Prediction Accuracy of the Dynamic Structure-Function Model for Glaucoma Progression Using Contrast Sensitivity Perimetry and Confocal Scanning Laser Ophthalmoscopy

PURPOSE

The purpose of this study was to determine whether combining a structural measure with contrast sensitivity perimetry (CSP), which has lower test-retest variability than static automated perimetry (SAP), reduces prediction error with 2 models of glaucoma progression.

METHODS

In this retrospective analysis, eyes with 5 visits with rim area (RA), SAP, and CSP measures were selected from 2 datasets. Twenty-six eyes with open-angle glaucoma were included in the analyses. For CSP and SAP, mean sensitivity (MS) was obtained by converting the sensitivity values at each location from decibel (SAP) or log units (CSP) to linear units, and then averaging all values. MS and RA values were expressed as percent of mean normal based on independent normative data. Data from the first 3 and 4 visits were used to calculate errors in prediction for the fourth and fifth visits, respectively. Prediction errors were obtained for simple linear regression and the dynamic structure-function (DSF) model.

RESULTS

With linear regression, the median prediction errors ranged from 6% to 17% when SAP MS and RA were used and from 9% to 17% when CSP MS and RA were used. With the DSF model, the median prediction errors ranged from 6% to 11% when SAP MS and RA were used and from 7% to 16% when CSP MS and RA were used.

CONCLUSIONS

The DSF model had consistently lower prediction errors than simple linear regression. The lower test-retest variability of CSP in glaucomatous defects did not, however, result in lower prediction error.

The Future of Imaging in Detecting Glaucoma Progression

Ocular imaging has been heavily incorporated into glaucoma management and provides important information that aids in the detection of disease progression. Longitudinal studies have shown that the circumpapillary retinal nerve fiber layer is an important parameter for glaucoma progression detection, whereas other studies have demonstrated that macular parameters, such as the ganglion cell inner plexiform layer and optic nerve head parameters, also are useful for progression detection. The introduction of novel technologies with faster scan speeds, wider scanning fields, higher resolution, and improved tissue penetration has enabled the precise quantification of additional key ocular structures, such as the individual retinal layers, optic nerve head, choroid, and lamina cribrosa. Furthermore, extracting functional information from scans such as blood flow rate and oxygen consumption provides new perspectives on the disease and its progression. These novel methods promise improved detection of glaucoma progression and better insight into the mechanisms of progression that will lead to better targeted treatment options to prevent visual damage and blindness.

Machine Learning Techniques in Clinical Vision Sciences

This review presents and discusses the contribution of machine learning techniques for diagnosis and disease monitoring in the context of clinical vision science. Many ocular diseases leading to blindness can be halted or delayed when detected and treated at its earliest stages. With the recent developments in diagnostic devices, imaging and genomics, new sources of data for early disease detection and patients' management are now available. Machine learning techniques emerged in the biomedical sciences as clinical decision-support techniques to improve sensitivity and specificity of disease detection and monitoring, increasing objectively the clinical decision-making process. This manuscript presents a review in multimodal ocular disease diagnosis and monitoring based on machine learning approaches. In the first section, the technical issues related to the different machine learning approaches will be present. Machine learning techniques are used to automatically recognize complex patterns in a given dataset. These techniques allows creating homogeneous groups (unsupervised learning), or creating a classifier predicting group membership of new cases (supervised learning), when a group label is available for each case. To ensure a good performance of the machine learning techniques in a given dataset, all possible sources of bias should be removed or minimized. For that, the representativeness of the input dataset for the true population should be confirmed, the noise should be removed, the missing data should be treated and the data dimensionally (i.e., the number of parameters/features and the number of cases in the dataset) should be adjusted. The application of machine learning techniques in ocular disease diagnosis and monitoring will be presented and discussed in the second section of this manuscript. To show the clinical benefits of machine learning in clinical vision sciences, several examples will be presented in glaucoma, age-related macular degeneration, and diabetic retinopathy, these ocular pathologies being the major causes of irreversible visual impairment.

Prediction accuracy of a novel dynamic structure-function model for glaucoma progression

PURPOSE

To assess the prediction accuracy of a novel dynamic structure-function (DSF) model to monitor glaucoma progression.

METHODS

Longitudinal data of paired rim area (RA) and mean sensitivity (MS) from 220 eyes with ocular hypertension or primary open-angle glaucoma enrolled in the Diagnostic Innovations in Glaucoma Study or the African Descent and Glaucoma Evaluation Study were included. Rim area and MS were expressed as percent of mean normal based on an independent dataset of 91 healthy eyes. The DSF model uses centroids as estimates of the current state of the disease and velocity vectors as estimates of direction and rate of change over time. The first three visits were used to predict the fourth visit; the first four visits were used to predict the fifth visit, and so on up to the 11th visit. The prediction error (PE) was compared to that of ordinary least squares linear regression (OLSLR) using Wilcoxon signed-rank test.

RESULTS

For predictions at visit 4 to visit 7, the average PE for the DSF model was significantly lower than OLSLR by 1.19% to 3.42% of mean normal. No significant difference was observed for the predictions at visit 8 to visit 11. The DSF model had lower PE than OLSLR for 70% of eyes in predicting visit 4 and approximately 60% in predicting visits 5, 6, and 7.

CONCLUSIONS

The two models had similar prediction capabilities, and the DSF model performed better in shorter time series. The DSF model could be clinically useful when only limited follow-ups are available. (ClinicalTrials.gov numbers, NCT00221923, NCT00221897.).

Glaucoma progression detection using structural retinal nerve fiber layer measurements and functional visual field points

Machine learning classifiers were employed to detect glaucomatous progression using longitudinal series of structural data extracted from retinal nerve fiber layer thickness measurements and visual functional data recorded from standard automated perimetry tests. Using the collected data, a longitudinal feature vector was created for each patient's eye by computing the norm 1 difference vector of the data at the baseline and at each follow-up visit. The longitudinal features from each patient's eye were then fed to the machine learning classifier to classify each eye as stable or progressed over time. This study was performed using several machine learning classifiers including Bayesian, Lazy, Meta, and Tree, composing different families. Combinations of structural and functional features were selected and ranked to determine the relative effectiveness of each feature. Finally, the outcomes of the classifiers were assessed by several performance metrics and the effectiveness of structural and functional features were analyzed.

Improving glaucoma detection using spatially correspondent clusters of damage and by combining standard automated perimetry and optical coherence tomography

PURPOSE

To improve the detection of glaucoma, techniques for assessing local patterns of damage and for combining structure and function were developed.

METHODS

Standard automated perimetry (SAP) and frequency-domain optical coherence tomography (fdOCT) data, consisting of macular retinal ganglion cell plus inner plexiform layer (mRGCPL) as well as macular and optic disc retinal nerve fiber layer (mRNFL and dRNFL) thicknesses, were collected from 52 eyes of 52 healthy controls and 156 eyes of 96 glaucoma suspects and patients. In addition to generating simple global metrics, SAP and fdOCT data were searched for contiguous clusters of abnormal points and converted to a continuous metric (pcc). The pcc metric, along with simpler methods, was used to combine the information from the SAP and fdOCT. The performance of different methods was assessed using the area under receiver operator characteristic curves (AROC scores).

RESULTS

The pcc metric performed better than simple global measures for both the fdOCT and SAP. The best combined structure-function metric (mRGCPL&SAP pcc, AROC = 0.868 ± 0.032) was better (statistically significant) than the best metrics for independent measures of structure and function. When SAP was used as part of the inclusion and exclusion criteria, AROC scores increased for all metrics, including the best combined structure-function metric (AROC = 0.975 ± 0.014).

CONCLUSIONS

A combined structure-function metric improved the detection of glaucomatous eyes. Overall, the primary sources of value-added for glaucoma detection stem from the continuous cluster search (the pcc), the mRGCPL data, and the combination of structure and function.

Improving predictive models of glaucoma severity by incorporating quality indicators

OBJECTIVE

In this paper we present an evaluation of the role of reliability indicators in glaucoma severity prediction. In particular, we investigate whether it is possible to extract useful information from tests that would be normally discarded because they are considered unreliable.

METHODS

We set up a predictive modelling framework to predict glaucoma severity from visual field (VF) tests sensitivities in different reliability scenarios. Three quality indicators were considered in this study: false positives rate, false negatives rate and fixation losses. Glaucoma severity was evaluated by considering a 3-levels version of the Advanced Glaucoma Intervention Study scoring metric. A bootstrapping and class balancing technique was designed to overcome problems related to small sample size and unbalanced classes. As a classification model we selected Naïve Bayes. We also evaluated Bayesian networks to understand the relationships between the different anatomical sectors on the VF map.

RESULTS

The methods were tested on a data set of 28,778 VF tests collected at Moorfields Eye Hospital between 1986 and 2010. Applying Friedman test followed by the post hoc Tukey's honestly significant difference test, we observed that the classifiers trained on any kind of test, regardless of its reliability, showed comparable performance with respect to the classifier trained only considering totally reliable tests (p-value>0.01). Moreover, we showed that different quality indicators gave different effects on prediction results. Training classifiers using tests that exceeded the fixation losses threshold did not have a deteriorating impact on classification results (p-value>0.01). On the contrary, using only tests that fail to comply with the constraint on false negatives significantly decreased the accuracy of the results (p-value<0.01). Meaningful patterns related to glaucoma evolution were also extracted.

CONCLUSIONS

Results showed that classification modelling is not negatively affected by the inclusion of less reliable tests in the training process. This means that less reliable tests do not subtract useful information from a model trained using only completely reliable data. Future work will be devoted to exploring new quantitative thresholds to ensure high quality testing and low re-test rates. This could assist doctors in tuning patient follow-up and therapeutic plans, possibly slowing down disease progression.

Predicting glaucomatous progression in glaucoma suspect eyes using relevance vector machine classifiers for combined structural and functional measurements

PURPOSE

The goal of this study was to determine if glaucomatous progression in suspect eyes can be predicted from baseline confocal scanning laser ophthalmoscope (CSLO) and standard automated perimetry (SAP) measurements analyzed with relevance vector machine (RVM) classifiers.

METHODS

Two hundred sixty-four eyes of 193 participants were included. All eyes had normal SAP results at baseline with five or more SAP tests over time. Eyes were labeled progressed (n = 47) or stable (n = 217) during follow-up based on SAP Guided Progression Analysis or serial stereophotograph assessment. Baseline CSLO-measured topographic parameters (n = 117) and baseline total deviation values from the 24-2 SAP test-grid (n = 52) were selected from each eye. Ten-fold cross-validation was used to train and test RVMs using the CSLO and SAP features. Receiver operating characteristic (ROC) curve areas were calculated using full and optimized feature sets. ROC curve results from RVM analyses of CSLO, SAP, and CSLO and SAP combined were compared to CSLO and SAP global indices (Glaucoma Probability Score, mean deviation and pattern standard deviation).

RESULTS

The areas under the ROC curves (AUROCs) for RVMs trained on optimized feature sets of CSLO parameters, SAP parameters, and CSLO and SAP parameters combined were 0.640, 0.762, and 0.805, respectively. AUROCs for CSLO Glaucoma Probability Score, SAP mean deviation (MD), and SAP pattern standard deviation (PSD) were 0.517, 0.513, and 0.620, respectively. No CSLO or SAP global indices discriminated between baseline measurements from progressed and stable eyes better than chance.

CONCLUSIONS

In our sample, RVM analyses of baseline CSLO and SAP measurements could identify eyes that showed future glaucomatous progression with a higher accuracy than the CSLO and SAP global indices. (ClinicalTrials.gov numbers, NCT00221897, NCT00221923.).

Integration and fusion of standard automated perimetry and optical coherence tomography data for improved automated glaucoma diagnostics

Background

The performance of glaucoma diagnostic systems could be conceivably improved by the integration of functional and structural test measurements that provide relevant and complementary information for reaching a diagnosis. The purpose of this study was to investigate the performance of data fusion methods and techniques for simple combination of Standard Automated Perimetry (SAP) and Optical Coherence Tomography (OCT) data for the diagnosis of glaucoma using Artificial Neural Networks (ANNs).

Methods

Humphrey 24-2 SITA standard SAP and StratusOCT tests were prospectively collected from a randomly selected population of 125 healthy persons and 135 patients with glaucomatous optic nerve heads and used as input for the ANNs. We tested commercially available standard parameters as well as novel ones (fused OCT and SAP data) that exploit the spatial relationship between visual field areas and sectors of the OCT peripapillary scan circle. We evaluated the performance of these SAP and OCT derived parameters both separately and in combination.

Results

The diagnostic accuracy from a combination of fused SAP and OCT data (95.39%) was higher than that of the best conventional parameters of either instrument, i.e. SAP Glaucoma Hemifield Test (p < 0.001) and OCT Retinal Nerve Fiber Layer Thickness ≥ 1 quadrant (p = 0.031). Fused OCT and combined fused OCT and SAP data provided similar Area under the Receiver Operating Characteristic Curve (AROC) values of 0.978 that were significantly larger (p = 0.047) compared to ANNs using SAP parameters alone (AROC = 0.945). On the other hand, ANNs based on the OCT parameters (AROC = 0.970) did not perform significantly worse than the ANNs based on the fused or combined forms of input data. The use of fused input increased the number of tests that were correctly classified by both SAP and OCT based ANNs.

Conclusions

Compared to the use of SAP parameters, input from the combination of fused OCT and SAP parameters, and from fused OCT data, significantly increased the performance of ANNs. Integrating parameters by including a priori relevant information through data fusion may improve ANN classification accuracy compared to currently available methods.

Evaluation of machine learning classifiers in keratoconus detection from orbscan II examinations

PURPOSE

To evaluate the performance of support vector machine, multi-layer perceptron and radial basis function neural network as auxiliary tools to identify keratoconus from Orbscan II maps.

METHODS

A total of 318 maps were selected and classified into four categories: normal (n = 172), astigmatism (n = 89), keratoconus (n = 46) and photorefractive keratectomy (n = 11). For each map, 11 attributes were obtained or calculated from data provided by the Orbscan II. Ten-fold cross-validation was used to train and test the classifiers. Besides accuracy, sensitivity and specificity, receiver operating characteristic (ROC) curves for each classifier were generated, and the areas under the curves were calculated.

RESULTS

The three selected classifiers provided a good performance, and there were no differences between their performances. The area under the ROC curve of the support vector machine, multi-layer perceptron and radial basis function neural network were significantly larger than those for all individual Orbscan II attributes evaluated (p < 0.05).

CONCLUSION

Overall, the results suggest that using a support vector machine, multi-layer perceptron classifiers and radial basis function neural network, these classifiers, trained on Orbscan II data, could represent useful techniques for keratoconus detection.