Assessment of the Impact of an Endpoint Committee in the Ocular Hypertension Treatment Study

Deep Learning Identifies High-Quality Fundus Photographs and Increases Accuracy in Automated Primary Open Angle Glaucoma Detection

Purpose

To develop and evaluate a deep learning (DL) model to assess fundus photograph quality, and quantitatively measure its impact on automated POAG detection in independent study populations.

Methods

Image quality ground truth was determined by manual review of 2815 fundus photographs of healthy and POAG eyes from the Diagnostic Innovations in Glaucoma Study and African Descent and Glaucoma Evaluation Study (DIGS/ADAGES), as well as 11,350 from the Ocular Hypertension Treatment Study (OHTS). Human experts assessed a photograph as high quality if of sufficient quality to determine POAG status and poor quality if not. A DL quality model was trained on photographs from DIGS/ADAGES and tested on OHTS. The effect of DL quality assessment on DL POAG detection was measured using area under the receiver operating characteristic (AUROC).

Results

The DL quality model yielded an AUROC of 0.97 for differentiating between high- and low-quality photographs; qualitative human review affirmed high model performance. Diagnostic accuracy of the DL POAG model was significantly greater (P < 0.001) in good (AUROC, 0.87; 95% CI, 0.80-0.92) compared with poor quality photographs (AUROC, 0.77; 95% CI, 0.67-0.88).

Conclusions

The DL quality model was able to accurately assess fundus photograph quality. Using automated quality assessment to filter out low-quality photographs increased the accuracy of a DL POAG detection model.

Translational Relevance

Incorporating DL quality assessment into automated review of fundus photographs can help to decrease the burden of manual review and improve accuracy for automated DL POAG detection.

One-Vote Veto: Semi-Supervised Learning for Low-Shot Glaucoma Diagnosis

Convolutional neural networks (CNNs) are a promising technique for automated glaucoma diagnosis from images of the fundus, and these images are routinely acquired as part of an ophthalmic exam. Nevertheless, CNNs typically require a large amount of well-labeled data for training, which may not be available in many biomedical image classification applications, especially when diseases are rare and where labeling by experts is costly. This article makes two contributions to address this issue: 1) It extends the conventional Siamese network and introduces a training method for low-shot learning when labeled data are limited and imbalanced, and 2) it introduces a novel semi-supervised learning strategy that uses additional unlabeled training data to achieve greater accuracy. Our proposed multi-task Siamese network (MTSN) can employ any backbone CNN, and we demonstrate with four backbone CNNs that its accuracy with limited training data approaches the accuracy of backbone CNNs trained with a dataset that is 50 times larger. We also introduce One-Vote Veto (OVV) self-training, a semi-supervised learning strategy that is designed specifically for MTSNs. By taking both self-predictions and contrastive predictions of the unlabeled training data into account, OVV self-training provides additional pseudo labels for fine-tuning a pre-trained MTSN. Using a large (imbalanced) dataset with 66,715 fundus photographs acquired over 15 years, extensive experimental results demonstrate the effectiveness of low-shot learning with MTSN and semi-supervised learning with OVV self-training. Three additional, smaller clinical datasets of fundus images acquired under different conditions (cameras, instruments, locations, populations) are used to demonstrate the generalizability of the proposed methods.

Risk Calculation in the Medication Arm of the Ocular Hypertension Treatment Study

PURPOSE

Risk assessment is integral to the management of individuals with ocular hypertension (OHTN). This study aims to determine the predictive accuracy of the Ocular Hypertension Treatment Study 5-year risk calculator (OHTS calculator) among treated patients with OHTN by applying it to patients randomized to the Ocular Hypertension Treatment Study (OHTS) medication arm.

DESIGN

Post hoc secondary analysis of a randomized clinical trial.

SUBJECTS

Individuals participating in the OHTS who were randomized to the medication arm. Only participants with complete baseline data in both eyes were included (n = 726).

METHODS

The hazard ratios (HRs) of the medication group in OHTS were compared to the HR used for the OHTS calculator using the z-test statistic to establish the OHTS calculator's generalizability to the OHTS medication arm. The performance of the OHTS calculator among the OHTS medication group was evaluated twice, using both untreated baseline intraocular pressure (IOP) and average treated IOP during the first 24 months for the IOP variable.

MAIN OUTCOME MEASURES

The performance was determined based on the model's accuracy in estimating the risk of reaching an OHTS primary open-angle glaucoma (POAG) end point using calibration chi-square and discriminating between participants who did or did not develop POAG.

RESULTS

The HRs for the OHTS medication arm were not significantly different from those used in the OHTS calculator for untreated OHTN derived from observation arm data (P > 0.1). Based on the calibration chi-square test for the medication group, the OHTS calculator prediction model had good predictive accuracy when using the mean treated IOP and poorer predictive accuracy with the untreated baseline IOP (chi-square 10 and 29, respectively). The model had good discrimination with treated IOP (c-statistic = 0.77), comparable to what has been reported for the OHTS calculator in the OHTS observation group.

CONCLUSIONS

The OHTS calculator can be applied to treated patients with OHTN, and repeat risk calculation after initiating IOP reduction may provide useful information that can aid in disease management.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Glaucoma in rural China (the Rural Epidemiology for Glaucoma in China (REG-China)): a national cross-sectional study

OBJECTIVE

This study aimed to investigate the prevalence of glaucoma with associated factors in the rural populations of 10 provinces in China.

DESIGN

A population-based cross-sectional study.

METHODS

All participants aged 6 years or older from 10 provinces completed visual acuity testing, slit-lamp examination, ophthalmoscopy and non-contact tonometry. Glaucoma suspects underwent fundus photography, Goldmann applanation tonometry, visual field testing and gonioscopy. Glaucoma was determined according to the International Society of Geographical and Epidemiological Ophthalmology classification scheme. Associations of demographics and medical factors with glaucoma were assessed using multiple logistic regression models.

RESULTS

From June 2017 to October 2018, 48 398 of 52 041 participants were included in the final analyses. The age-standardised prevalence of glaucoma was 1.7% (95% CI 1.55% to 1.78%) among the participants older than 6 years, which was 2.1% (95% CI 1.93% to 2.23%) in participants aged over 40 years. The constituent ratios of glaucoma were: 44.4% primary angle-closure glaucoma (PACG), 34.7% primary open-angle glaucoma, 2.6% congenital glaucoma and 18.3% other types of glaucoma. Increasing age, smoking, cerebral stroke, type 2 diabetes, higher education (college or more) and higher personal income were significant risk factors for PACG. The unilateral and bilateral blindness rates in the entire study population were 4.692% and 1.068%, respectively. A family history of glaucoma was a significant risk factor for the prevalence of glaucoma and blindness in at least one eye.

CONCLUSIONS

Rural populations have a high prevalence of glaucoma, which should be included in chronic disease management programmes in China for long-term care.

A potential primary endpoint for clinical trials in glaucoma neuroprotection

The purpose of this retrospective, longitudinal study is to evaluate the relationship between MD slope from visual field tests collected over a short period of time (2 years) and the current United States' Food and Drug Administration (FDA) recommended endpoints for visual field outcomes. If this correlation is strong and highly predictive, clinical trials employing MD slopes as primary endpoints could be employed in neuroprotection clinical trials with shorter duration and help expedite the development of novel IOP-independent therapies. Visual field tests of patients with or suspected glaucoma were selected from an academic institution and evaluated based on two functional progression endpoints: (A) five or more locations worsening by at least 7 dB, and (B) at least five test locations based upon the GCP algorithm. A total of 271 (57.6%) and 278 (59.1%) eyes reached Endpoints A and B, respectively during the follow up period. The median (IQR) MD slope of eyes reaching vs. not reaching Endpoint A and B were -1.19 (-2.00 to -0.41) vs. 0.36 (0.00 to 1.00) dB/year and -1.16 (-1.98 to -0.40) vs. 0.41 (0.02 to 1.03) dB/year, respectively (P < 0.001). It was found that eyes experiencing rapid 24-2 visual field MD slopes over a 2-year period were on average tenfold more likely to reach one of the FDA accepted endpoints during or soon after that period.

Detecting Glaucoma from Fundus Photographs Using Deep Learning without Convolutions: Transformer for Improved Generalization

Purpose

To compare the diagnostic accuracy and explainability of a Vision Transformer deep learning technique, Data-efficient image Transformer (DeiT), and ResNet-50, trained on fundus photographs from the Ocular Hypertension Treatment Study (OHTS) to detect primary open-angle glaucoma (POAG) and identify the salient areas of the photographs most important for each model's decision-making process.

Design

Evaluation of a diagnostic technology.

Subjects Participants and Controls

Overall 66 715 photographs from 1636 OHTS participants and an additional 5 external datasets of 16 137 photographs of healthy and glaucoma eyes.

Methods

Data-efficient image Transformer models were trained to detect 5 ground-truth OHTS POAG classifications: OHTS end point committee POAG determinations because of disc changes (model 1), visual field (VF) changes (model 2), or either disc or VF changes (model 3) and Reading Center determinations based on disc (model 4) and VFs (model 5). The best-performing DeiT models were compared with ResNet-50 models on OHTS and 5 external datasets.

Main Outcome Measures

Diagnostic performance was compared using areas under the receiver operating characteristic curve (AUROC) and sensitivities at fixed specificities. The explainability of the DeiT and ResNet-50 models was compared by evaluating the attention maps derived directly from DeiT to 3 gradient-weighted class activation map strategies.

Results

Compared with our best-performing ResNet-50 models, the DeiT models demonstrated similar performance on the OHTS test sets for all 5 ground-truth POAG labels; AUROC ranged from 0.82 (model 5) to 0.91 (model 1). Data-efficient image Transformer AUROC was consistently higher than ResNet-50 on the 5 external datasets. For example, AUROC for the main OHTS end point (model 3) was between 0.08 and 0.20 higher in the DeiT than ResNet-50 models. The saliency maps from the DeiT highlight localized areas of the neuroretinal rim, suggesting important rim features for classification. The same maps in the ResNet-50 models show a more diffuse, generalized distribution around the optic disc.

Conclusions

Vision Transformers have the potential to improve generalizability and explainability in deep learning models, detecting eye disease and possibly other medical conditions that rely on imaging for clinical diagnosis and management.

The 24-2 Visual Field Guided Progression Analysis Can Miss the Progression of Glaucomatous Damage of the Macula Seen Using OCT

PURPOSE

To better understand the efficacy of the 24-2 guided progression analysis (GPA) in the detection of progression in eyes with early glaucoma (i.e., 24-2 mean deviation [MD] better than -6 dB) by comparing 24-2 GPA with a reference standard (RS) based on a combination of OCT and 24-2 and 10-2 visual field (VF) information.

DESIGN

Cross-sectional study.

PARTICIPANTS

Ninety-nine eyes from 99 individuals, including 70 suspected or early glaucomatous eyes (24-2 MD better than -6 dB) and 29 healthy controls (HCs).

METHODS

All the eyes had at least 4 OCT and VF test dates over a period that ranged from 12 to 59 months. The 24-2 VF tests included 2 baseline tests and at least 2 follow-up tests. The 2 baseline tests were performed within an average of 5.6 days (median, 7 days), and the last follow-up test was performed at least 1 year after the first baseline visit.

MAIN OUTCOME MEASURES

A commercial 24-2 GPA software, with default settings, characterized the eyes as having "likely progression" (LP) or "possible progression" (PP); both were considered "progressing" for this analysis. For RS, 3 authors graded progression using strict criteria and a combination of a custom OCT progression report and commercial 24-2 and 10-2 GPA reports for the same test dates as GPA.

RESULTS

The reference standard identified 10 (14%) of the 70 patient eyes and none of the HC eyes as having progression. The 24-2 guided progression analysis identified 13 of the 70 patient eyes as having progression (PP or LP). However, it correctly classified only 4 (40%) of the 10 RS progressors. All 6 of the RS progressors missed by the 24-2 GPA showed progression in the macula. In addition, the 24-2 GPA identified 2 of the 29 HC eyes as progressors and 9 patient eyes without progression based on the RS.

CONCLUSIONS

In eyes with early glaucoma (i.e., 24-2 MD, > -6 dB) in this study, the 24-2 GPA missed progression seen using OCT and exhibited a relatively high rate of false positives. Furthermore, the region progressing typically included the macula. The results suggest that including OCT and/or 10-2 VFs should improve the detection of progression.

Rationale and Development of an OCT-Based Method for Detection of Glaucomatous Optic Neuropathy

A specific, sensitive, and intersubjectively verifiable definition of disease for clinical care and research remains an important unmet need in the field of glaucoma. Using an iterative, consensus-building approach and employing pilot data, an optical coherence tomography (OCT)-based method to aid in the detection of glaucomatous optic neuropathy was sought to address this challenge. To maximize the chance of success, we utilized all available information from the OCT circle and cube scans, applied both quantitative and semiquantitative data analysis methods, and aimed to limit the use of perimetry to cases where it is absolutely necessary. The outcome of this approach was an OCT-based method for the diagnosis of glaucomatous optic neuropathy that did not require the use of perimetry for initial diagnosis. A decision tree was devised for testing and implementation in clinical practice and research that can be used by reading centers, researchers, and clinicians. While initial pilot data were encouraging, future testing and validation will be needed to establish its utility in clinical practice, as well as for research.

Detecting Glaucoma in the Ocular Hypertension Study Using Deep Learning

IMPORTANCE

Automated deep learning (DL) analyses of fundus photographs potentially can reduce the cost and improve the efficiency of reading center assessment of end points in clinical trials.

OBJECTIVE

To investigate the diagnostic accuracy of DL algorithms trained on fundus photographs from the Ocular Hypertension Treatment Study (OHTS) to detect primary open-angle glaucoma (POAG).

DESIGN, SETTING, AND PARTICIPANTS

In this diagnostic study, 1636 OHTS participants from 22 sites with a mean (range) follow-up of 10.7 (0-14.3) years. A total of 66 715 photographs from 3272 eyes were used to train and test a ResNet-50 model to detect the OHTS Endpoint Committee POAG determination based on optic disc (287 eyes, 3502 photographs) and/or visual field (198 eyes, 2300 visual fields) changes. Three independent test sets were used to evaluate the generalizability of the model.

MAIN OUTCOMES AND MEASURES

Areas under the receiver operating characteristic curve (AUROC) and sensitivities at fixed specificities were calculated to compare model performance. Evaluation of false-positive rates was used to determine whether the DL model detected POAG before the OHTS Endpoint Committee POAG determination.

RESULTS

A total of 1147 participants were included in the training set (661 [57.6%] female; mean age, 57.2 years; 95% CI, 56.6-57.8), 167 in the validation set (97 [58.1%] female; mean age, 57.1 years; 95% CI, 55.6-58.7), and 322 in the test set (173 [53.7%] female; mean age, 57.2 years; 95% CI, 56.1-58.2). The DL model achieved an AUROC of 0.88 (95% CI, 0.82-0.92) for the OHTS Endpoint Committee determination of optic disc or VF changes. For the OHTS end points based on optic disc changes or visual field changes, AUROCs were 0.91 (95% CI, 0.88-0.94) and 0.86 (95% CI, 0.76-0.93), respectively. False-positive rates (at 90% specificity) were higher in photographs of eyes that later developed POAG by disc or visual field (27.5% [56 of 204]) compared with eyes that did not develop POAG (11.4% [50 of 440]) during follow-up. The diagnostic accuracy of the DL model developed on the optic disc end point applied to 3 independent data sets was lower, with AUROCs ranging from 0.74 (95% CI, 0.70-0.77) to 0.79 (95% CI, 0.78-0.81).

CONCLUSIONS AND RELEVANCE

The model's high diagnostic accuracy using OHTS photographs suggests that DL has the potential to standardize and automate POAG determination for clinical trials and management. In addition, the higher false-positive rate in early photographs of eyes that later developed POAG suggests that DL models detected POAG in some eyes earlier than the OHTS Endpoint Committee, reflecting the OHTS design that emphasized a high specificity for POAG determination by requiring a clinically significant change from baseline.

Glaucoma Suspects: The Impact of Risk Factor-Driven Review Periods on Clinical Load, Diagnoses, and Healthcare Costs

Purpose

To model the healthcare impact (clinical attendance time and financial cost) and clinical outcomes (glaucoma diagnoses) of different risk factor–driven review frequencies for glaucoma suspect patients up until the point of discharge or diagnosis.

Methods

Medical records of 494 glaucoma suspects were examined to extract the clinical diagnosis. Two criteria for review periods were defined, based on contrasting stringency from established clinical guidelines: American Academy of Ophthalmology (AAO), more stringent/less frequent; and the Australian National Health and Medical Research Council (NHMRC), less stringent/more frequent. We used these data to model patient outcomes and healthcare costs using a Markov model.

Results

The less stringent/more frequent criterion resulted in more high-risk glaucoma suspects requiring more frequent review compared with the more stringent/less frequent criterion. Across the 15 Markov cycles (7.5 years), the less stringent/more frequent review criterion resulted in 6.6% more diagnoses and fewer overall clinical visits (14.7%) and reduced cost per diagnosis by 12% to 32% (P < 0.0001). The number of glaucoma diagnoses made using each criterion converged at 2.5 to 3 years.

Conclusions

The stringency of risk assessments for glaucoma suspects impacts review periods and therefore clinical load, healthcare costs, and diagnosis rates. Using current testing methods, more frequent review periods appear advantageous for diagnostic efficiency, with both lower clinic load and lower cost up until the point of discharge or glaucoma diagnosis.

Translational Relevance

A less stringent criterion for assessing the risk of developing glaucoma potentially offers a more cost-effective method for reviewing glaucoma suspects, especially within the first 2.5 years.

Assessment of Cumulative Incidence and Severity of Primary Open-Angle Glaucoma Among Participants in the Ocular Hypertension Treatment Study After 20 Years of Follow-up

Importance

Ocular hypertension is an important risk factor for the development of primary open-angle glaucoma (POAG). Data from long-term follow-up can be used to inform the management of patients with ocular hypertension.

Objective

To determine the cumulative incidence and severity of POAG after 20 years of follow-up among participants in the Ocular Hypertension Treatment Study.

Design, Setting, and Participants

Participants in the Ocular Hypertension Treatment Study were followed up from February 1994 to December 2008 in 22 clinics. Data were collected after 20 years of follow-up (from January 2016 to April 2019) or within 2 years of death. Analyses were performed from July 2019 to December 2020.

Interventions

From February 28, 1994, to June 2, 2002 (phase 1), participants were randomized to receive either topical ocular hypotensive medication (medication group) or close observation (observation group). From June 3, 2002, to December 30, 2008 (phase 2), both randomization groups received medication. Beginning in 2009, treatment was no longer determined by study protocol. From January 7, 2016, to April 15, 2019 (phase 3), participants received ophthalmic examinations and visual function assessments.

Main Outcomes and Measures

Twenty-year cumulative incidence and severity of POAG in 1 or both eyes after adjustment for exposure time.

Results

A total of 1636 individuals (mean [SD] age, 55.4 [9.6] years; 931 women [56.9%]; 1138 White participants [69.6%]; 407 Black/African American participants [24.9%]) were randomized in phase 1 of the clinical trial. Of those, 483 participants (29.5%) developed POAG in 1 or both eyes (unadjusted incidence). After adjusting for exposure time, the 20-year cumulative incidence of POAG in 1 or both eyes was 45.6% (95% CI, 42.3%-48.8%) among all participants, 49.3% (95% CI, 44.5%-53.8%) among participants in the observation group, and 41.9% (95% CI, 37.2%-46.3%) among participants in the medication group. The 20-year cumulative incidence of POAG was 55.2% (95% CI, 47.9%-61.5%) among Black/African American participants and 42.7% (95% CI, 38.9%-46.3%) among participants of other races. The 20-year cumulative incidence for visual field loss was 25.2% (95% CI, 22.5%-27.8%). Using a 5-factor baseline model, the cumulative incidence of POAG among participants in the low-, medium-, and high-risk tertiles was 31.7% (95% CI, 26.4%-36.6%), 47.6% (95% CI, 41.6%-53.0%), and 59.8% (95% CI, 53.1%-65.5%), respectively.

Conclusions and Relevance

In this study, only one-fourth of participants in the Ocular Hypertension Treatment Study developed visual field loss in either eye over long-term follow-up. This information, together with a prediction model, may help clinicians and patients make informed personalized decisions about the management of ocular hypertension.

Trial Registration

ClinicalTrials.gov Identifier: NCT00000125.

Evaluation of a Primary Open-Angle Glaucoma Prediction Model Using Long-term Intraocular Pressure Variability Data: A Secondary Analysis of 2 Randomized Clinical Trials

Importance

The contribution of long-term intraocular pressure (IOP) variability to the development of primary open-angle glaucoma is still controversial.

Objective

To assess whether long-term IOP variability data improve a prediction model for the development of primary open-angle glaucoma (POAG) in individuals with untreated ocular hypertension.

Design, Setting, and Participants

This post hoc secondary analysis of 2 randomized clinical trials included data from 709 of 819 participants in the observation group of the Ocular Hypertension Treatment Study (OHTS) followed up from February 28, 1994, to June 1, 2002, and 397 of 500 participants in the placebo group of the European Glaucoma Prevention Study (EGPS) followed up from January 1, 1997, to September 30, 2003. Data analyses were completed between January 1, 2019, and March 15, 2020.

Exposures

The original prediction model for the development of POAG included the following baseline factors: age, IOP, central corneal thickness, vertical cup-disc ratio, and pattern SD. This analysis tested whether substitution of baseline IOP with mean follow-up IOP, SD of IOP, maximum IOP, range of IOP, or coefficient of variation IOP would improve predictive accuracy.

Main Outcomes and Measures

The C statistic was used to compare the predictive accuracy of multivariable landmark Cox proportional hazards regression models for the development of POAG.

Results

Data from the OHTS consisted of 97 POAG end points from 709 of 819 participants (416 [58.7%] women; 177 [25.0%] African American and 490 [69.1%] white; mean [SD] age, 55.7 [9.59] years; median [range] follow-up, 6.9 [0.96-8.15] years). Data from the EGPS consisted of 44 POAG end points from 397 of 500 participants in the placebo group (201 [50.1%] women; 397 [100%] white; mean [SD] age, 57.8 [9.76] years; median [range] follow-up, 4.9 [1.45-5.76] years). The C statistic for the original prediction model was 0.741. When a measure of follow-up IOP was substituted for baseline IOP in this prediction model, the C statistics were as follows: mean follow-up IOP, 0.784; maximum IOP, 0.781; SD of IOP, 0.745; range of IOP, 0.741; and coefficient of variation IOP, 0.729. The C statistics in the EGPS were similarly ordered. No measure of IOP variability, when added to the prediction model that included mean follow-up IOP, age, central corneal thickness, vertical cup-disc ratio, and pattern SD, increased the C statistic by more than 0.007 in either cohort.

Conclusions and Relevance

Evidence from the OHTS and the EGPS suggests that long-term variability does not add substantial explanatory power to the prediction model as to which individuals with untreated ocular hypertension will develop POAG.