Joint multimodal deep learning-based automatic segmentation of ICGA and OCT images for assessment of PCV biomarkers

Purpose

To develop a fully-automatic hybrid algorithm to jointly segment and quantify biomarkers of polypoidal choroidal vasculopathy (PCV) on indocyanine green angiography (ICGA) and spectral domain-OCT (SD-OCT) images.

Design

Evaluation of diagnostic test or technology.

Participants

Seventy-two participants with PCV enrolled in clinical studies at Singapore National Eye Center.

Methods

The dataset consisted of 2-dimensional (2-D) ICGA and 3-dimensional (3-D) SD-OCT images which were spatially registered and manually segmented by clinicians. A deep learning-based hybrid algorithm called PCV-Net was developed for automatic joint segmentation of biomarkers. The PCV-Net consisted of a 2-D segmentation branch for ICGA and 3-D segmentation branch for SD-OCT. We developed fusion attention modules to connect the 2-D and 3-D branches for effective use of the spatial correspondence between the imaging modalities by sharing learned features. We also used self-supervised pretraining and ensembling to further enhance the performance of the algorithm without the need for additional datasets. We compared the proposed PCV-Net to several alternative model variants.

Main Outcome Measures

The PCV-Net was evaluated based on the Dice similarity coefficient (DSC) of the segmentations and the Pearson's correlation and absolute difference of the clinical measurements obtained from the segmentations. Manual grading was used as the gold standard.

Results

The PCV-Net showed good performance compared to manual grading and alternative model variants based on both quantitative and qualitative analyses. Compared to the baseline variant, PCV-Net improved the DSC by 0.04 to 0.43 across the different biomarkers, increased the correlations, and decreased the absolute differences of clinical measurements of interest. Specifically, the largest average (mean ± standard error) DSC improvement was for intraretinal fluid, from 0.02 ± 0.00 (baseline variant) to 0.45 ± 0.06 (PCV-Net). In general, improving trends were observed across the model variants as more technical specifications were added, demonstrating the importance of each aspect of the proposed method.

Conclusion

The PCV-Net has the potential to aid clinicians in disease assessment and research to improve clinical understanding and management of PCV.

Financial Disclosures

Proprietary or commercial disclosure may be found after the references.

Baseline Microperimetry and OCT in the RUSH2A Study: Structure-Function Association and Correlation With Disease Severity

PURPOSE

To investigate baseline mesopic microperimetry (MP) and spectral domain optical coherence tomography (OCT) in the Rate of Progression in USH2A-related Retinal Degeneration (RUSH2A) study.

DESIGN

Natural history study METHODS: Setting: 16 clinical sites in Europe and North AmericaStudy Population: Participants with Usher syndrome type 2 (USH2) (N = 80) or autosomal recessive nonsyndromic RP (ARRP) (N = 47) associated with biallelic disease-causing sequence variants in USH2AObservation Procedures: General linear models were used to assess characteristics including disease duration, MP mean sensitivity and OCT intact ellipsoid zone (EZ) area. The associations between mean sensitivity and EZ area with other measures, including best corrected visual acuity (BCVA) and central subfield thickness (CST) within the central 1 mm, were assessed using Spearman correlation coefficients.

MAIN OUTCOME MEASURES

Mean sensitivity on MP; EZ area and CST on OCT.

RESULTS

All participants (N = 127) had OCT, while MP was obtained at selected sites (N = 93). Participants with Usher syndrome type 2 (USH2, N = 80) and nonsyndromic autosomal recessive Retinitis Pigmentosa (ARRP, N = 47) had the following similar measurements: EZ area (median (interquartile range [IQR]): 1.4 (0.4, 3.1) mm2 vs 2.3 (0.7, 5.7) mm2) and CST (median (IQR): 247 (223, 280) µm vs 261 (246, 288), and mean sensitivity (median (IQR): 3.5 (2.1, 8.4) dB vs 5.1 (2.9, 9.0) dB). Longer disease duration was associated with smaller EZ area (P < 0.001) and lower mean sensitivity (P = 0.01). Better BCVA, larger EZ area, and larger CST were correlated with greater mean sensitivity (r > 0.3 and P < 0.01). Better BCVA and larger CST were associated with larger EZ area (r > 0.6 and P < 0.001).

CONCLUSIONS

Longer disease duration correlated with more severe retinal structure and function abnormalities, and there were associations between MP and OCT metrics. Monitoring changes in retinal structure-function relationships during disease progression will provide important insights into disease mechanism in USH2A-related retinal degeneration.

Impact of Baseline Quantitative OCT Features on Response to Risuteganib for the Treatment of Dry Age-Related Macular Degeneration: The Importance of Outer Retinal Integrity

OBJECTIVE

The purpose of the study was to perform a post hoc analysis to explore the effect of baseline anatomic characteristics identified on OCT on best-corrected visual acuity (BCVA) responses to risuteganib from the completed phase II study in subjects with dry age-related macular degeneration (AMD).

DESIGN

Post hoc analysis of a randomized, double-masked, placebo-controlled, phase II study.

SUBJECTS

Eyes with intermediate dry AMD with BCVA between 20/40 and 20/200. Patients with concurrent vision-influencing or macula-obscuring ocular pathologies were excluded.

METHODS

Patients were randomized to receive a 1-mg intravitreal risuteganib injection or a sham injection at baseline. A second 1-mg intravitreal injection of risuteganib was given at week 16 to those in the treatment arm. Two independent, masked reading centers evaluated the baseline anatomic characteristics on OCT to explore features associated with positive responses to risuteganib.

MAIN OUTCOME MEASURES

Treatment response was defined as a gain of ≥ 8 letters in BCVA from baseline to week 28 in the treatment arm, compared with baseline to week 12 in the sham group. Anatomic parameters, measured by retinal segmentation platforms, including measures of retinal thickness were compared between the responders and nonresponders to risuteganib.

RESULTS

Thirty-nine patients completed the study and underwent analysis. In the treatment arm, 48% of eyes demonstrated treatment responses, compared with 7% in the sham group. In the quantitative anatomic assessment, enhanced ellipsoid integrity, greater outer retinal thickness, and decreased geographic atrophy were associated with increased BCVA gains to risuteganib.

CONCLUSIONS

This post hoc analysis demonstrated that baseline OCT features may help determine the likelihood of a functional response to risuteganib. The characterization of higher-order OCT features may provide important information regarding biomarkers for treatment response and could facilitate optimized clinical trial enrollment and enrichment.

VALIDATION OF A DEEP LEARNING-BASED ALGORITHM FOR SEGMENTATION OF THE ELLIPSOID ZONE ON OPTICAL COHERENCE TOMOGRAPHY IMAGES OF AN USH2A-RELATED RETINAL DEGENERATION CLINICAL TRIAL

PURPOSE

To assess the generalizability of a deep learning-based algorithm to segment the ellipsoid zone (EZ).

METHODS

The dataset consisted of 127 spectral-domain optical coherence tomography volumes from eyes of participants with USH2A-related retinal degeneration enrolled in the RUSH2A clinical trial (NCT03146078). The EZ was segmented manually by trained readers and automatically by deep OCT atrophy detection, a deep learning-based algorithm originally developed for macular telangiectasia Type 2. Performance was evaluated using the Dice similarity coefficient between the segmentations, and the absolute difference and Pearson's correlation of measurements of interest obtained from the segmentations.

RESULTS

With deep OCT atrophy detection, the average (mean ± SD, median) Dice similarity coefficient was 0.79 ± 0.27, 0.90. The average absolute difference in total EZ area was 0.62 ± 1.41, 0.22 mm2 with a correlation of 0.97. The average absolute difference in the maximum EZ length was 222 ± 288, 126 µm with a correlation of 0.97.

CONCLUSION

Deep OCT atrophy detection segmented EZ in USH2A-related retinal degeneration with good performance. The algorithm is potentially generalizable to other diseases and other biomarkers of interest as well, which is an important aspect of clinical applicability.

Open-Source Automatic Biomarker Measurement on Slit-Lamp Photography to Estimate Visual Acuity in Microbial Keratitis

Purpose

To assess clinical applicability of automatic image analysis in microbial keratitis (MK) by evaluating the relationship between biomarker measurements on slit-lamp photography (SLP) and best-corrected visual acuity (BCVA).

Methods

Seventy-six patients with MK with SLP images and same-day logarithm of the minimum angle of resolution (logMAR) BCVA were evaluated. MK biomarkers (stromal infiltrate, white blood cell infiltration, corneal edema, hypopyon, epithelial defect) were segmented manually by ophthalmologists and automatically by a novel, open-source, deep learning–based segmentation algorithm. Five measurements (presence, maximum width, total area, proportion of the corneal limbus area affected, centrality) were calculated. Correlations between the measurements and BCVA were calculated. An automatic regression model estimated BCVA from the measurements. Differences in performance between using manual and automatic measurements were evaluated using William's test (for correlation) and the paired-sample t-test (for absolute error).

Results

Measurements had high correlations of 0.86 (manual) and 0.84 (automatic) with true BCVA. Estimated BCVA had average (mean ± SD) absolute errors of 0.39 ± 0.27 logMAR (manual, median: 0.30) and 0.35 ± 0.28 logMAR (automatic, median: 0.30) and high correlations of 0.76 (manual) and 0.80 (automatic) with true BCVA. Differences between using manual and automatic measurements were not statistically significant for correlations of measurements with true BCVA (P = .66), absolute errors of estimated BCVA (P = .15), or correlations of estimated BCVA with true BCVA (P = .60).

Conclusions

The proposed algorithm measured MK biomarkers as accurately as ophthalmologists. Measurements were highly correlated with and estimative of visual acuity.

Translational Relevance

This study demonstrates the potential of developing fully automatic objective and standardized strategies to aid ophthalmologists in the clinical assessment of MK.

COMPARISON OF SINGLE DRUSEN SIZE ON COLOR FUNDUS PHOTOGRAPHY AND SPECTRAL-DOMAIN OPTICAL COHERENCE TOMOGRAPHY

PURPOSE

To determine the relationship of drusen size as determined by spectral-domain optical coherence tomography (SD-OCT) with that measured on registered color fundus photography (CFP) images and to derive an OCT-based classification system that was comparable with that determined by CFP.

METHODS

Custom software was developed to register CFP images to the scanning laser ophthalmoscopy fundus images obtained simultaneously with the corresponding SD-OCT images, so that individual drusen observed on CFP could be matched with those seen on SD-OCT. Single druse size (diameter, area, volume, and height) on CFP and SD-OCT images from a phase two clinical trial was determined with the Duke OCT Retinal Analysis Program.

RESULTS

The sizes of 213 individual drusen were measured on CFP and SD-OCT. The drusen diameter measured on CFP was significantly correlated with those determined on SD-OCT (R: 0.879, P < 0.001). Based on the corresponding formula: drusen diameter on SD-OCT = 0.77 × (drusen diameter on CFP) + 50.67 µm, large drusen defined as ≥125 µm on CFP had a diameter ≥145 µm on OCT, medium drusen defined as 63 µm to 124 µm on CFP had diameters 100 µm to 144 µm on OCT, and small drusen defined as <63 µm on CFP had diameters <100 µm on OCT.

CONCLUSION

With our registration software and imaging processing algorithms, we were able to correlate individual druse sizes measured on CFP with those determined on SD-OCT. These data can be used to develop an SD-OCT-based grading scale, analogous to the CFP Age-Related Eye Disease Study drusen scale that may be useful in the clinic and in clinical trials.

Intraoperative Retinal Changes May Predict Surgical Outcomes After Epiretinal Membrane Peeling

Purpose

To investigate whether intraoperative retinal changes during epiretinal membrane (ERM) peeling affect anatomic or functional outcomes after surgery.

Methods

We measured retinal thickness using an intraoperative optical coherence tomography (iOCT) device in patients undergoing pars plana vitrectomy with membrane peeling for idiopathic ERM. Changes in intraoperative central macular thickness (iCMT) were compared with postoperative improvements in CMT and best-corrected visual acuity (VA).

Results

Twenty-seven eyes from 27 patients (mean age 68 years) underwent iOCT-assisted ERM peeling surgery. Before surgery, mean VA was logMAR 0.50 ± 0.36 (Snellen 20/63), and mean baseline CMT was 489 ± 82 µm. Mean iCMT before peeling was 477 ± 87 µm, which correlated well with preoperative CMT (P < 0.001). Mean change in iCMT was −39.6 ± 37 µm (range −116 to +77 µm). After surgery, VA improved to logMAR 0.40 ± 0.38 (Snellen 20/50) at month 1 and logMAR 0.27 ± 0.23 (Snellen 20/37) at month 3, whereas CMT decreased to 397 ± 44 µm and 396 ± 51 µm at months 1 and 3. Eyes that underwent greater amount of iCMT change (absolute value of iCMT change) were associated with greater CMT reduction at month 1 (P < 0.001) and month 3 (P = 0.010), whereas those with greater intraoperative thinning (actual iCMT change) showed a trend toward better VA outcomes at months 1 (P = 0.054) and 3 (P = 0.036).

Conclusions

Intraoperative changes in retinal thickness may predict anatomic and visual outcomes after idiopathic ERM peeling surgery.

Translational Relevance

Our study suggests that intraoperative retinal tissue response to ERM peeling surgery measured by iOCT may be a prognostic indicator for restoration of retinal architecture and for visual acuity outcomes.

Open-Source Automatic Segmentation of Ocular Structures and Biomarkers of Microbial Keratitis on Slit-Lamp Photography Images Using Deep Learning

We propose a fully-automatic deep learning-based algorithm for segmentation of ocular structures and microbial keratitis (MK) biomarkers on slit-lamp photography (SLP) images. The dataset consisted of SLP images from 133 eyes with manual annotations by a physician, P1. A modified region-based convolutional neural network, SLIT-Net, was developed and trained using P1's annotations to identify and segment four pathological regions of interest (ROIs) on diffuse white light images (stromal infiltrate (SI), hypopyon, white blood cell (WBC) border, corneal edema border), one pathological ROI on diffuse blue light images (epithelial defect (ED)), and two non-pathological ROIs on all images (corneal limbus, light reflexes). To assess inter-reader variability, 75 eyes were manually annotated for pathological ROIs by a second physician, P2. Performance was evaluated using the Dice similarity coefficient (DSC) and Hausdorff distance (HD). Using seven-fold cross-validation, the DSC of the algorithm (as compared to P1) for all ROIs was good (range: 0.62-0.95) on all 133 eyes. For the subset of 75 eyes with manual annotations by P2, the DSC for pathological ROIs ranged from 0.69-0.85 (SLIT-Net) vs. 0.37-0.92 (P2). DSCs for SLIT-Net were not significantly different than P2 for segmenting hypopyons (p > 0.05) and higher than P2 for WBCs (p < 0.001) and edema (p < 0.001). DSCs were higher for P2 for segmenting SIs (p < 0.001) and EDs (p < 0.001). HDs were lower for P2 for segmenting SIs (p = 0.005) and EDs (p < 0.001) and not significantly different for hypopyons (p > 0.05), WBCs (p > 0.05), and edema (p > 0.05). This prototype fully-automatic algorithm to segment MK biomarkers on SLP images performed to expectations on an exploratory dataset and holds promise for quantification of corneal physiology and pathology.

Deep learning-based classification and segmentation of retinal cavitations on optical coherence tomography images of macular telangiectasia type 2

AIM

To develop a fully automatic algorithm to segment retinal cavitations on optical coherence tomography (OCT) images of macular telangiectasia type 2 (MacTel2).

METHODS

The dataset consisted of 99 eyes from 67 participants enrolled in an international, multicentre, phase 2 MacTel2 clinical trial (NCT01949324). Each eye was imaged with spectral-domain OCT at three time points over 2 years. Retinal cavitations were manually segmented by a trained Reader and the retinal cavitation volume was calculated. Two convolutional neural networks (CNNs) were developed that operated in sequential stages. In the first stage, CNN1 classified whether a B-scan contained any retinal cavitations. In the second stage, CNN2 segmented the retinal cavitations in a B-scan. We evaluated the performance of the proposed method against alternative methods using several performance metrics and manual segmentations as the gold standard.

RESULTS

The proposed method was computationally efficient and accurately classified and segmented retinal cavitations on OCT images, with a sensitivity of 0.94, specificity of 0.80 and average Dice similarity coefficient of 0.94±0.07 across all time points. The proposed method produced measurements that were highly correlated with the manual measurements of retinal cavitation volume and change in retinal cavitation volume over time.

CONCLUSION

The proposed method will be useful to help clinicians quantify retinal cavitations, assess changes over time and further investigate the clinical significance of these early structural changes observed in MacTel2.

Computational modeling of retinal hypoxia and photoreceptor degeneration in patients with age-related macular degeneration

Although drusen have long been acknowledged as a primary hallmark of dry age-related macular degeneration (AMD) their role in the disease remains unclear. We hypothesize that drusen accumulation increases the barrier to metabolite transport ultimately resulting in photoreceptor cell death. To investigate this hypothesis, a computational model was developed to evaluate steady-state oxygen distribution in the retina. Optical coherence tomography images from fifteen AMD patients and six control subjects were segmented and translated into 3D in silico representations of retinal morphology. A finite element model was then used to determine the steady-state oxygen distribution throughout the retina for both generic and patient-specific retinal morphology. Oxygen levels were compared to the change in retinal thickness at a later time point to observe possible correlations. The generic finite element model of oxygen concentration in the retina agreed closely with both experimental measurements from literature and clinical observations, including the minimal pathological drusen size identified by AREDS (64 μm). Modeling oxygen distribution in the outer retina of AMD patients showed a substantially stronger correlation between hypoxia and future retinal thinning (Pearson correlation coefficient, r = 0.2162) than between drusen height and retinal thinning (r = 0.0303) indicating the potential value of this physiology-based approach. This study presents proof-of-concept for the potential utility of finite element modeling in evaluating retinal health and also suggests a potential link between transport and AMD pathogenesis. This strategy may prove useful as a prognostic tool for predicting the clinical risk of AMD progression.

Deep longitudinal transfer learning-based automatic segmentation of photoreceptor ellipsoid zone defects on optical coherence tomography images of macular telangiectasia type 2

Photoreceptor ellipsoid zone (EZ) defects visible on optical coherence tomography (OCT) are important imaging biomarkers for the onset and progression of macular diseases. As such, accurate quantification of EZ defects is paramount to monitor disease progression and treatment efficacy over time. We developed and trained a novel deep learning-based method called Deep OCT Atrophy Detection (DOCTAD) to automatically segment EZ defect areas by classifying 3-dimensional A-scan clusters as normal or defective. Furthermore, we introduce a longitudinal transfer learning paradigm in which the algorithm learns from segmentation errors on images obtained at one time point to segment subsequent images with higher accuracy. We evaluated the performance of this method on 134 eyes of 67 subjects enrolled in a clinical trial of a novel macular telangiectasia type 2 (MacTel2) therapeutic agent. Our method compared favorably to other deep learning-based and non-deep learning-based methods in matching expert manual segmentations. To the best of our knowledge, this is the first automatic segmentation method developed for EZ defects on OCT images of MacTel2.

Sample-to-answer on molecular diagnosis of bacterial infection using integrated lab--on--a--disc

Sepsis by bacterial infection causes high mortality in patients in intensive care unit (ICU). Rapid identification of bacterial infection is essential to ensure early appropriate administration of antibiotics to save lives of patients, yet the present benchtop molecular diagnosis is time-consuming and labor-intensive, which limits the treatment efficiency especially when the number of samples to be tested is extensive. Therefore, we hereby report a microfluidic platform lab-on-a-disc (LOAD) to provide a sample-to-answer solution. Our LOAD customized design of microfluidic channels allows automation to mimic sequential analytical steps in benchtop environment. It relies on a simple but controllable centrifugation force for the actuation of samples and reagents. Our LOAD system performs three major functions, namely DNA extraction, isothermal DNA amplification and real-time signal detection, in a predefined sequence. The disc is self-contained for conducting sample heating with chemical lysis buffer and silica microbeads are employed for DNA extraction from clinical specimens. Molecular diagnosis of specific target bacteria DNA sequences is then performed using a real-time loop-mediated isothermal amplification (RT-LAMP) with SYTO-9 as the signal reporter. Our LOAD system capable of bacterial identification of Mycobacterium tuberculosis (TB) and Acinetobacter baumanii (Ab) with the detection limits 103cfu/mL TB in sputum and 102cfu/mL Ab in blood within 2h after sample loading. The reported LOAD based on an integrated approach should address the growing needs for rapid point-of-care medical diagnosis in ICU.

Loss mitigation in plasmonic solar cells: aluminium nanoparticles for broadband photocurrent enhancements in GaAs photodiodes

We illustrate the important trade-off between far-field scattering effects, which have the potential to provide increased optical path length over broad bands, and parasitic absorption due to the excitation of localized surface plasmon resonances in metal nanoparticle arrays. Via detailed comparison of photocurrent enhancements given by Au, Ag and Al nanostructures on thin-film GaAs devices we reveal that parasitic losses can be mitigated through a careful choice of scattering medium. Absorption at the plasmon resonance in Au and Ag structures occurs in the visible spectrum, impairing device performance. In contrast, exploiting Al nanoparticle arrays results in a blue shift of the resonance, enabling the first demonstration of truly broadband plasmon enhanced photocurrent and a 22% integrated efficiency enhancement.

Peptide-functionalized microfabricated structures for improved on-chip neuronal adhesion

Extracellular, high signal-to-noise ratio recordings from electrogenic cells require a tight coupling between the cellular membrane and the recording electrode. Self assembled monolayers (SAMs) of alkanethiols functionalized with peptides were used in combination with micro- and nano-structured features on the sensor surface. This combination of surface chemistry and topography triggers a phagocytosis-like engulfment and ensures tight coupling. In this paper we report the results concerning usage of different SAMs and the influence of the peptide concentration towards cell adhesion and outgrowth. Later on, the optimized peptide functionalized SAMs were applied on micro- and nano-structured sensor surfaces. As a result, phagocytosis-like events could be shown using focused ion beam SEM and confocal fluorescence imaging.

Disposition of the antipsychotic agent CI-1007 in rats, monkeys, dogs, and human cytochrome P450 2D6 extensive metabolizers. Species comparison and allometric scaling

The disposition of CI-1007 (I), an antipsychotic dopamine agonist, was studied after iv or po administration to rats, monkeys, and dogs and po administration to human cytochrome P450 2D6 extensive metabolizers (EMs). I was extensively metabolized after po administration, with high hepatic clearance (CL) values and negligible urinary excretion. Values for systemic plasma CL (28-40 ml/min/kg) suggested hepatic plasma flow-limited CL. The oral CL of I was similar among the species. Strong correlation was achieved in interspecies scaling for CL. After oral administration of [14C]I, the major route of 14C elimination in rats was in the bile (64%), followed by feces (29%) and urine (3.2%). Fecal excretion (64%) was the major route of 14C elimination in monkeys, followed by urine (14%). Three hydroxy metabolites, i.e. PD 147693 (II), PD 149394 (III), and PD 155144 (IV), and two sulfates, i.e. PD 163637 (VI) and PD 163639 (VIII), were identified in monkey plasma, urine, or feces. VIII was the major metabolite excreted in monkey urine, and VI was the major component in feces. Trace amounts of II, VI, and VIII were detected in the plasma and urine of human EMs but not in rats or dogs. II is an active metabolite that was present in all species. After oral administration, observed maximal plasma concentration and AUC values for II were higher than the corresponding values for I in dog plasma, approximately 20-40% of the values for I in monkeys and human EMs, and <5% of the values for I in rat plasma. Although the metabolic profiles differ among species, strong correlation was achieved in allometric scaling because the elimination of I from the body is mainly limited by hepatic blood flow.

2,2'-Dithiobisbenzamides and 2-benzisothiazolones, two new classes of antiretroviral agents: SAR and mechanistic considerations

Substituted 2,2'-dithiobisbenzamides and 2-benzisothiazolones were prepared and shown to possess low microM activity with high therapeutic indices against HIV-1, HIV-2 and SIV in cell culture. The mechanism of antiviral action was determined to be directed toward the nucleocapsid protein (NCp7), which contains two zinc fingers and plays vital roles in the viral life cycle. The "active sulfides" of this study cause the extrusion of zinc from these zinc fingers. Structure-activity relationships of the 2,2'-dithiobisbenzamides reveal that the disulfide bond and the ortho benzamide functional groups are essential for activity, with the best compounds having a carboxylic acid, carboxamide, or sulfonamide substituent. The 2-benzisothiazolones are formed from the disulfides both chemically and in vivo and their SAR mimics that of the 2,2'-dithiobisbenzamides. The antiviral activity of the disulfides may require cyclization to the isothiazolones. Two agents, PD 159206 and PD 161374, which showed good antiviral activity, physical properties, and excellent pharmacokinetics in mice, were selected for advanced studies.