Deep Learning-Based Automated Classification of Multi-Categorical Abnormalities From Optical Coherence Tomography Images

The Role of Artificial Intelligence in Epiretinal Membrane Care: A Scoping Review

Topic

In ophthalmology, artificial intelligence (AI) demonstrates potential in using ophthalmic imaging across diverse diseases, often matching ophthalmologists' performance. However, the range of machine learning models for epiretinal membrane (ERM) management, which differ in methodology, application, and performance, remains largely unsynthesized.

Clinical Relevance

Epiretinal membrane management relies on clinical evaluation and imaging, with surgical intervention considered in cases of significant impairment. AI analysis of ophthalmic images and clinical features could enhance ERM detection, characterization, and prognostication, potentially improving clinical decision-making. This scoping review aims to evaluate the methodologies, applications, and reported performance of AI models in ERM diagnosis, characterization, and prognostication.

Methods

A comprehensive literature search was conducted across 5 electronic databases including Ovid MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, and Web of Science Core Collection from inception to November 14, 2024. Studies pertaining to AI algorithms in the context of ERM were included. The primary outcomes measured will be the reported design, application in ERM management, and performance of each AI model.

Results

Three hundred ninety articles were retrieved, with 33 studies meeting inclusion criteria. There were 30 studies (91%) reporting their training and validation methods. Altogether, 61 distinct AI models were included. OCT scans and fundus photographs were used in 26 (79%) and 7 (21%) papers, respectively. Supervised learning and both supervised and unsupervised learning were used in 32 (97%) and 1 (3%) studies, respectively. Twenty-seven studies (82%) developed or adapted AI models using images, whereas 5 (15%) had models using both images and clinical features, and 1 (3%) used preoperative and postoperative clinical features without ophthalmic images. Study objectives were categorized into 3 stages of ERM care. Twenty-three studies (70%) implemented AI for diagnosis (stage 1), 1 (3%) identified ERM characteristics (stage 2), and 6 (18%) predicted vision impairment after diagnosis or postoperative vision outcomes (stage 3). No articles studied treatment planning. Three studies (9%) used AI in stages 1 and 2. Of the 16 studies comparing AI performance to human graders (i.e., retinal specialists, general ophthalmologists, and trainees), 10 (63%) reported equivalent or higher performance.

Conclusion

Artificial intelligence-driven assessments of ophthalmic images and clinical features demonstrated high performance in detecting ERM, identifying its morphological properties, and predicting visual outcomes following ERM surgery. Future research might consider the validation of algorithms for clinical applications in personal treatment plan development, ideally to identify patients who might benefit most from surgery.

Financial Disclosures

The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Bilateral deformable attention transformer for screening of high myopia using optical coherence tomography

Myopia is a visual impairment caused by excessive refractive power of the cornea or lens or elongation of the eyeball. Due to the various classification criteria associated with high myopia, such as spherical equivalent (SE) and axial length (AL), existing methods primarily rely on individual classification criteria for model design. In this paper, to comprehensively utilize multiple indicators, we design a multi-label classification model for high myopia. Moreover, image data play a pivotal role in studying high myopia and pathological myopia. Notable features of high myopia, including increased retinal curvature, choroidal thinning, and scleral shadowing, are observable in Optical Coherence Tomography (OCT) images of the retina. We propose a model named Bilateral Deformable Attention Transformer (BDA-Tran) for multi-label screening of high myopia in OCT data. Based on the vision transformer, we introduce a bilateral deformable attention mechanism (BDA) where the queries in self-attention are composed of both the global queries and the data-dependent queries from the left and right sides. This flexible approach allows attention to focus on relevant regions and capture more myopia-related information features, thereby concentrating attention primarily on regions related to the choroid and sclera, among other areas associated with high myopia. BDA-Tran is trained and tested on OCT images of 243 patients, achieving the accuracies of 83.1 % and 87.7 % for SE and AL, respectively. Furthermore, we visualize attention maps to provide transparent and interpretable judgments. Experimental results demonstrate that BDA-Tran outperforms existing methods in terms of effectiveness and reliability under the same experimental conditions.

Transfer Learning and Multi-Feature Fusion-Based Deep Learning Model for Idiopathic Macular Hole Diagnosis and Grading from Optical Coherence Tomography Images

Background

Idiopathic macular hole is an ophthalmic disease that seriously affects vision, and its early diagnosis and treatment have important clinical significance to reduce the occurrence of blindness. At present, OCT is the gold standard for diagnosing this disease, but its application is limited due to the need for professional ophthalmologist to diagnose it. The introduction of artificial intelligence will break this situation and make its diagnosis efficient, and how to build an effective predictive model is the key to the problem, and more clinical trials are still needed to verify it.

Objective

This study aims to evaluate the role of deep learning systems in Idiopathic Macular Hole diagnosis, grading, and prediction.

Methods

A single-center, retrospective study used binocular OCT images from IMH patients at the First Affiliated Hospital of Nanchang University (November 2019 - January 2023). A deep learning algorithm, including traditional omics, Resnet101, and fusion models incorporating multi-feature fusion and transfer learning, was developed. Model performance was evaluated using accuracy and AUC. Logistic regression analyzed clinical factors, and a nomogram predicted surgical risk. Analysis was conducted with SPSS 22.0 and R 3.6.3. P < 0.05 was statistically significant.

Results

Among 229 OCT images, the traditional omics, Resnet101, and fusion models achieved accuracies of 93%, 94%, and 95%, respectively, in the training set. In the test set, the fusion model and Resnet101 correctly identified 39 images, while the traditional omics model identified 35. The nomogram had a C-index of 0.996, with macular hole diameter most strongly associated with surgical risk.

Conclusion

The deep learning system with transfer learning and multi-feature fusion effectively diagnoses and grades IMH from OCT images.

Optical Coherence Tomography Characteristics Between Idiopathic Epiretinal Membranes and Secondary Epiretinal Membranes due to Peripheral Retinal Hole

Purpose: In clinical practice, some eyes preoperatively diagnosed with "idiopathic epiretinal membranes (iERM)" will be amended to "secondary epiretinal membranes (sERM)" once peripheral retinal hole is detected. This study utilized optical coherence tomography (OCT) images to compare the characteristics between the iERM and sERM due to peripheral retinal hole (PRH). Methods: In this retrospective, cross-sectional study, 635 eyes that had undergone pars plana vitrectomy with membrane peeling were enrolled. A total of 115 eyes (18.1%) detected with peripheral retinal holes were allocated to the sERM-PRH group while the other 520 eyes were to the iERM group. The demographic data and OCT characteristics were compared between the two groups. Besides, all the eyes were evaluated by a double-grading scheme: severity grading of ERM progression into four stages plus anatomical classification into three kinds of part-thickness macular holes associated with ERMs. Results: No significant difference was found in age, gender, symptom duration, axial length, or best-corrected visual acuity between the two groups. There was also no difference concerning the features based on OCT, ranging from central macular thickness, the ratios of the photoreceptor inner/outer segment junction line defect, intraretinal fluid, cotton ball sign, to epiretinal proliferation. However, the native difference in parafoveal thickness between the temporal and nasal quadrants was observed in the iERM group, yet disappeared in the sERM-PRH group. Moreover, eyes between the two groups were distributionally similar in both grading scales. Conclusion: Our results demonstrated that even OCT images could hardly provide effective clues for early differentiating sERM from iERM, which highlighted the necessity of a thorough pre- and intro-operative fundus examination of the peripheral retina for clinicians.

A joint learning approach for automated diagnosis of keratinocyte carcinoma using optical attenuation coefficients

Keratinocyte carcinoma, such as Actinic Keratosis (AK) and Basal Cell Carcinoma (BCC), share similar clinical presentations but differ significantly in prognosis and treatment, highlighting the importance of effective screening. Optical coherence tomography (OCT) shows promise for diagnosing AK and BCC using signal intensity and skin layer thickness, but variability due to skin characteristics and system settings underscores the need for a standardized diagnostic method. Here, we propose an automated diagnostic method using the optical attenuation coefficient (OAC) and a joint learning strategy to classify AK, BCC, and normal skin. OAC images extracted from OCT data revealed notable disparities between normal and cancerous tissues. By incorporating probability distribution function (PDF) information alongside OAC images, the model achieved an accuracy of over 80% and approaching 100% by utilizing 3D OAC data to enhance robustness. This approach highlights the potential of OAC-based analysis for automated, intelligent diagnosis of early-stage non-melanoma skin cancers.

A method for quantifying and automatic grading of musculoskeletal ultrasound superb microvascular imaging based on dynamic analysis of optical flow model

Superb microvascular flow signals in joints are important indicators for evaluating inflammation in arthritis diagnosis. Super Microvascular Imaging (SMI), a musculoskeletal ultrasound technique, captures microvascular signals with enhanced resolution, enabling improved quantitative analysis of joint superb microvascular flow. However, existing musculoskeletal ultrasound imaging predominantly relies on static observations for analyzing these signals, which are heavily influenced by subjective factors, thereby limiting diagnostic accuracy for arthritis. This study introduces a novel quantitative and automated grading method utilizing dynamic analysis through an optical flow model. Real-time dynamic quantification of superb microvascular flow signals is achieved via motion estimation and skeleton extraction based on the optical flow model. The Kappa consistency test evaluates the agreement between the automated grading system and physician assessments, with differences between the two methods analyzed. A total of 47 patient samples were included, comprising 20 males and 27 females (p = 0.307 > 0.05, χ2=1.042). The agreement between the automated grading system and physician assessments reached 70.2%, with a Kappa value of 0.627 (p < 0.001), indicating good consistency. Nonetheless, the system displayed a tendency to high-grade cases of moderate inflammation. The proposed quantitative and automated grading method for superb microvascular flow, based on dynamic analysis through an optical flow model, improves the objectivity and consistency of superb microvascular flow grading and demonstrates significant clinical potential. The method shows strong anti-interference performance in noisy signal environments, representing a promising advancement for non-invasive arthritis diagnosis.

Automated Detection of Central Retinal Artery Occlusion Using OCT Imaging via Explainable Deep Learning

Objective

To demonstrate the capability of a deep learning model to detect central retinal artery occlusion (CRAO), a retinal pathology with significant clinical urgency, using OCT data.

Design

Retrospective, external validation study analyzing OCT and clinical baseline data of 2 institutions via deep learning classification analysis.

Subjects

Patients presenting to the University Medical Center Tübingen and the University Medical Center Hamburg-Eppendorf in Germany.

Methods

OCT data of patients suffering from CRAO, differential diagnosis with (sub) acute visual loss (central retinal vein occlusion, diabetic macular edema, nonarteritic ischemic optic neuropathy), and from controls were expertly graded and distinguished into 3 groups. Our methodological approach involved a nested multiclass five fold cross-validation classification scheme.

Main Outcome Measures

Area under the curve (AUC).

Results

The optimal performance of our algorithm was observed using 30 epochs, complemented by an early stopping mechanism to prevent overfitting. Our model followed a multiclass approach, distinguishing among the 3 different classes: control, CRAO, and differential diagnoses. The evaluation was conducted by the "one vs. all" area under the receiver operating characteristics curve (AUC) method. The results demonstrated AUC of 0.96 (95% confidence interval [CI], ± 0.01); 0.99 (95% CI, ± 0.00); and 0.90 (95% CI, ± 0.03) for each class, respectively.

Conclusions

Our machine learning algorithm (MLA) exhibited a high AUC, as well as sensitivity and specificity in detecting CRAO and the differential classes, respectively. These findings underscore the potential for deploying MLAs in the identification of less common etiologies within an acute emergency clinical setting.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Reinforcement-based leveraging transfer learning for multiclass optical coherence tomography images classification

The accurate diagnosis of retinal diseases, such as Diabetic Macular Edema (DME) and Age-related Macular Degeneration (AMD), is essential for preventing vision loss. Optical Coherence Tomography (OCT) imaging plays a crucial role in identifying these conditions, especially given the increasing prevalence of AMD. This study introduces a novel Reinforcement-Based Leveraging Transfer Learning (RBLTL) framework, which integrates reinforcement Q-learning with transfer learning using pre-trained models, including InceptionV3, DenseNet201, and InceptionResNetV2. The RBLTL framework dynamically optimizes hyperparameters, improving classification accuracy and generalization while mitigating overfitting. Experimental evaluations demonstrate remarkable performance, achieving testing accuracies of 98.75%, 98.90%, and 99.20% across three scenarios for multiclass OCT image classification. These results highlight the effectiveness of the RBLTL framework in categorizing OCT images for conditions like DME and AMD, establishing it as a reliable and versatile approach for automated medical image classification with significant implications for clinical diagnostics.

The role of artificial intelligence in macular hole management: A scoping review

We focus on the utility of artificial intelligence (AI) in the management of macular hole (MH). We synthesize 25 studies, comprehensively reporting on each AI model's development strategy, validation, tasks, performance, strengths, and limitations. All models analyzed ophthalmic images, and 5 (20 %) also analyzed clinical features. Study objectives were categorized based on 3 stages of MH care: diagnosis, identification of MH characteristics, and postoperative predictions of hole closure and vision recovery. Twenty-two (88 %) AI models underwent supervised learning, and the models were most often deployed to determine a MH diagnosis. None of the articles applied AI to guiding treatment plans. AI model performance was compared to other algorithms and to human graders. Of the 10 studies comparing AI to human graders (i.e., retinal specialists, general ophthalmologists, and ophthalmology trainees), 5 (50 %) reported equivalent or higher performance. Overall, AI analysis of images and clinical characteristics in MH demonstrated high diagnostic and predictive accuracy. Convolutional neural networks comprised the majority of included AI models, including those which were high performing. Future research may consider validating algorithms to propose personalized treatment plans and explore clinical use of the aforementioned algorithms.

Inter-rater reliability in labeling quality and pathological features of retinal OCT scans: A customized annotation software approach

OBJECTIVES

Various imaging features on optical coherence tomography (OCT) are crucial for identifying and defining disease progression. Establishing a consensus on these imaging features is essential, particularly for training deep learning models for disease classification. This study aims to analyze the inter-rater reliability in labeling the quality and common imaging signatures of retinal OCT scans.

METHODS

500 OCT scans obtained from CIRRUS HD-OCT 5000 devices were displayed at 512x1024x128 resolution on a customizable, in-house annotation software. Each patient's eye was represented by 16 random scans. Two masked reviewers independently labeled the quality and specific pathological features of each scan. Evaluated features included overall image quality, presence of fovea, and disease signatures including subretinal fluid (SRF), intraretinal fluid (IRF), drusen, pigment epithelial detachment (PED), and hyperreflective material. The raw percentage agreement and Cohen's kappa (κ) coefficient were used to evaluate concurrence between the two sets of labels.

RESULTS

Our analysis revealed κ = 0.60 for the inter-rater reliability of overall scan quality, indicating substantial agreement. In contrast, there was slight agreement in determining the cause of poor image quality (κ = 0.18). The binary determination of presence and absence of retinal disease signatures showed almost complete agreement between reviewers (κ = 0.85). Specific retinal pathologies, such as the foveal location of the scan (0.78), IRF (0.63), drusen (0.73), and PED (0.87), exhibited substantial concordance. However, less agreement was found in identifying SRF (0.52), hyperreflective dots (0.41), and hyperreflective foci (0.33).

CONCLUSIONS

Our study demonstrates significant inter-rater reliability in labeling the quality and retinal pathologies on OCT scans. While some features show stronger agreement than others, these standardized labels can be utilized to create automated machine learning tools for diagnosing retinal diseases and capturing valuable pathological features in each scan. This standardization will aid in the consistency of medical diagnoses and enhance the accessibility of OCT diagnostic tools.

Multiscale attention-over-attention network for retinal disease recognition in OCT radiology images

Retinal disease recognition using Optical Coherence Tomography (OCT) images plays a pivotal role in the early diagnosis and treatment of conditions. However, the previous attempts relied on extracting single-scale features often refined by stacked layered attentions. This paper presents a novel deep learning-based Multiscale Feature Enhancement via a Dual Attention Network specifically designed for retinal disease recognition in OCT images. Our approach leverages the EfficientNetB7 backbone to extract multiscale features from OCT images, ensuring a comprehensive representation of global and local retinal structures. To further refine feature extraction, we propose a Pyramidal Attention mechanism that integrates Multi-Head Self-Attention (MHSA) with Dense Atrous Spatial Pyramid Pooling (DASPP), effectively capturing long-range dependencies and contextual information at multiple scales. Additionally, Efficient Channel Attention (ECA) and Spatial Refinement modules are introduced to enhance channel-wise and spatial feature representations, enabling precise localization of retinal abnormalities. A comprehensive ablation study confirms the progressive impact of integrated blocks and attention mechanisms that enhance overall performance. Our findings underscore the potential of advanced attention mechanisms and multiscale processing, highlighting the effectiveness of the network. Extensive experiments on two benchmark datasets demonstrate the superiority of the proposed network over existing state-of-the-art methods.

Applications of artificial intelligence in diagnosis of uncommon cystoid macular edema using optical coherence tomography imaging: A systematic review

Cystoid macular edema (CME) is a sight-threatening condition often associated with inflammatory and diabetic diseases. Early detection is crucial to prevent irreversible vision loss. Artificial intelligence (AI) has shown promise in automating CME diagnosis through optical coherence tomography (OCT) imaging, but its utility needs critical evaluation. This systematic review assesses the application of AI to diagnosis CME, specifically focusing on disorders like postoperative CME (Irvine Gass syndrome) and retinitis pigmentosa without obvious vasculopathy, using OCT imaging. A comprehensive search was conducted across 6 databases (PubMed, Scopus, Web of Science, Wiley, ScienceDirect, and IEEE) from 2018 to November, 2023. Twenty-three articles met the inclusion criteria and were selected for in-depth analysis. We evaluate AI's role in CME diagnosis and its performance in "detection", "classification", and "segmentation" of OCT retinal images. We found that convolutional neural network (CNN)-based methods consistently outperformed other machine learning techniques, achieving an average accuracy of over 96 % in detecting and identifying CME from OCT images. Despite certain limitations such as dataset size and ethical concerns, the synergy between AI and OCT, particularly through CNNs, holds promise for significantly advancing CME diagnostics.

Validation of Inter-Reader Agreement/Consistency for Quantification of Ellipsoid Zone Integrity and Sub-RPE Compartmental Features Across Retinal Diseases

BACKGROUND

An unmet need exists when clinically assessing retinal and layer-based features of retinal diseases. Therefore, quantification of retinal-layer-thicknesses/fluid volumes using deep-learning-augmented platforms to reproduce human-obtained clinical measurements is needed.

METHODS

In this analysis, 210 spectral-domain optical coherence tomography (SD-OCT) scans (30 without pathology, 60 dry age-related macular degeneration [AMD], 60 wet AMD, and 60 diabetic macular edema [total 23,625 B-scans]) were included. A fully automated segmentation platform segmented four retinal layers for compartmental assessment (internal limiting membrane, ellipsoid zone [EZ], retinal pigment epithelium [RPE], and Bruch's membrane). Two certified OCT readers independently completed manual segmentation and B-scan level validation of automated segmentation, with segmentation correction when needed (semi-automated). Certified reader metrics were compared to gold standard metrics using intraclass correlation coefficients (ICCs) to assess overall agreement. Across different diseases, several metrics generated from automated segmentations approached or matched human readers performance.

RESULTS

Absolute ICCs for retinal mean thickness measurements showed excellent agreement (range 0.980-0.999) across four cohorts. EZ-RPE thickness values and sub-RPE compartment ICCs demonstrated excellent agreement (ranges of 0.953-0.987 and 0.944-0.997, respectively) for full dataset, dry-AMD, and wet-AMD cohorts.

CONCLUSIONS

Analyses demonstrated high reliability and consistency of segmentation of outer retinal compartmental features using a completely human/manual approach or a semi-automated approach to segmentation. These results support the critical role that measuring features, such as photoreceptor preservation through EZ integrity, in future clinical trials may optimize clinical care.

Artificial intelligence in retinal screening using OCT images: A review of the last decade (2013-2023)

BACKGROUND AND OBJECTIVES

Optical coherence tomography (OCT) has ushered in a transformative era in the domain of ophthalmology, offering non-invasive imaging with high resolution for ocular disease detection. OCT, which is frequently used in diagnosing fundamental ocular pathologies, such as glaucoma and age-related macular degeneration (AMD), plays an important role in the widespread adoption of this technology. Apart from glaucoma and AMD, we will also investigate pertinent pathologies, such as epiretinal membrane (ERM), macular hole (MH), macular dystrophy (MD), vitreomacular traction (VMT), diabetic maculopathy (DMP), cystoid macular edema (CME), central serous chorioretinopathy (CSC), diabetic macular edema (DME), diabetic retinopathy (DR), drusen, glaucomatous optic neuropathy (GON), neovascular AMD (nAMD), myopia macular degeneration (MMD) and choroidal neovascularization (CNV) diseases. This comprehensive review examines the role that OCT-derived images play in detecting, characterizing, and monitoring eye diseases.

METHOD

The 2020 PRISMA guideline was used to structure a systematic review of research on various eye conditions using machine learning (ML) or deep learning (DL) techniques. A thorough search across IEEE, PubMed, Web of Science, and Scopus databases yielded 1787 publications, of which 1136 remained after removing duplicates. Subsequent exclusion of conference papers, review papers, and non-open-access articles reduced the selection to 511 articles. Further scrutiny led to the exclusion of 435 more articles due to lower-quality indexing or irrelevance, resulting in 76 journal articles for the review.

RESULTS

During our investigation, we found that a major challenge for ML-based decision support is the abundance of features and the determination of their significance. In contrast, DL-based decision support is characterized by a plug-and-play nature rather than relying on a trial-and-error approach. Furthermore, we observed that pre-trained networks are practical and especially useful when working on complex images such as OCT. Consequently, pre-trained deep networks were frequently utilized for classification tasks. Currently, medical decision support aims to reduce the workload of ophthalmologists and retina specialists during routine tasks. In the future, it might be possible to create continuous learning systems that can predict ocular pathologies by identifying subtle changes in OCT images.

Self-supervised contrastive learning improves machine learning discrimination of full thickness macular holes from epiretinal membranes in retinal OCT scans

There is a growing interest in using computer-assisted models for the detection of macular conditions using optical coherence tomography (OCT) data. As the quantity of clinical scan data of specific conditions is limited, these models are typically developed by fine-tuning a generalized network to classify specific macular conditions of interest. Full thickness macular holes (FTMH) present a condition requiring urgent surgical repair to prevent vision loss. Other works on automated FTMH classification have tended to use supervised ImageNet pre-trained networks with good results but leave room for improvement. In this paper, we develop a model for FTMH classification using OCT B-scans around the central foveal region to pre-train a naïve network using contrastive self-supervised learning. We found that self-supervised pre-trained networks outperform ImageNet pre-trained networks despite a small training set size (284 eyes total, 51 FTMH+ eyes, 3 B-scans from each eye). On three replicate data splits, 3D spatial contrast pre-training yields a model with an average F1-score of 1.0 on holdout data (50 eyes total, 10 FTMH+), compared to an average F1-score of 0.831 for FTMH detection by ImageNet pre-trained models. These results demonstrate that even limited data may be applied toward self-supervised pre-training to substantially improve performance for FTMH classification, indicating applicability toward other OCT-based problems.

Artificial Intelligence, Digital Imaging, and Robotics Technologies for Surgical Vitreoretinal Diseases

OBJECTIVE

To review recent technological advancement in imaging, surgical visualization, robotics technology, and the use of artificial intelligence in surgical vitreoretinal (VR) diseases.

BACKGROUND

Technological advancements in imaging enhance both preoperative and intraoperative management of surgical VR diseases. Widefield imaging in fundal photography and OCT can improve assessment of peripheral retinal disorders such as retinal detachments, degeneration, and tumors. OCT angiography provides a rapid and noninvasive imaging of the retinal and choroidal vasculature. Surgical visualization has also improved with intraoperative OCT providing a detailed real-time assessment of retinal layers to guide surgical decisions. Heads-up display and head-mounted display utilize 3-dimensional technology to provide surgeons with enhanced visual guidance and improved ergonomics during surgery. Intraocular robotics technology allows for greater surgical precision and is shown to be useful in retinal vein cannulation and subretinal drug delivery. In addition, deep learning techniques leverage on diverse data including widefield retinal photography and OCT for better predictive accuracy in classification, segmentation, and prognostication of many surgical VR diseases.

CONCLUSION

This review article summarized the latest updates in these areas and highlights the importance of continuous innovation and improvement in technology within the field. These advancements have the potential to reshape management of surgical VR diseases in the very near future and to ultimately improve patient care.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Three-Dimension Epithelial Segmentation in Optical Coherence Tomography of the Oral Cavity Using Deep Learning

This paper aims to simplify the application of optical coherence tomography (OCT) for the examination of subsurface morphology in the oral cavity and reduce barriers towards the adoption of OCT as a biopsy guidance device. The aim of this work was to develop automated software tools for the simplified analysis of the large volume of data collected during OCT. Imaging and corresponding histopathology were acquired in-clinic using a wide-field endoscopic OCT system. An annotated dataset (n = 294 images) from 60 patients (34 male and 26 female) was assembled to train four unique neural networks. A deep learning pipeline was built using convolutional and modified u-net models to detect the imaging field of view (network 1), detect artifacts (network 2), identify the tissue surface (network 3), and identify the presence and location of the epithelial-stromal boundary (network 4). The area under the curve of the image and artifact detection networks was 1.00 and 0.94, respectively. The Dice similarity score for the surface and epithelial-stromal boundary segmentation networks was 0.98 and 0.83, respectively. Deep learning (DL) techniques can identify the location and variations in the epithelial surface and epithelial-stromal boundary in OCT images of the oral mucosa. Segmentation results can be synthesized into accessible en face maps to allow easier visualization of changes.

Interpretable detection of epiretinal membrane from optical coherence tomography with deep neural networks

This study aimed to automatically detect epiretinal membranes (ERM) in various OCT-scans of the central and paracentral macula region and classify them by size using deep-neural-networks (DNNs). To this end, 11,061 OCT-images were included and graded according to the presence of an ERM and its size (small 100-1000 µm, large > 1000 µm). The data set was divided into training, validation and test sets (75%, 10%, 15% of the data, respectively). An ensemble of DNNs was trained and saliency maps were generated using Guided-Backprob. OCT-scans were also transformed into a one-dimensional-value using t-SNE analysis. The DNNs' receiver-operating-characteristics on the test set showed a high performance for no-ERM, small-ERM and large-ERM cases (AUC: 0.99, 0.92, 0.99, respectively; 3-way accuracy: 89%), with small-ERMs being the most difficult ones to detect. t-SNE analysis sorted cases by size and, in particular, revealed increased classification uncertainty at the transitions between groups. Saliency maps reliably highlighted ERM, regardless of the presence of other OCT features (i.e. retinal-thickening, intraretinal pseudo-cysts, epiretinal-proliferation) and entities such as ERM-retinoschisis, macular-pseudohole and lamellar-macular-hole. This study showed therefore that DNNs can reliably detect and grade ERMs according to their size not only in the fovea but also in the paracentral region. This is also achieved in cases of hard-to-detect, small-ERMs. In addition, the generated saliency maps can be used to highlight small-ERMs that might otherwise be missed. The proposed model could be used for screening-programs or decision-support-systems in the future.

Clinical evaluation of deep learning systems for assisting in the diagnosis of the epiretinal membrane grade in general ophthalmologists

BACKGROUND

Epiretinal membrane (ERM) is a common age-related retinal disease detected by optical coherence tomography (OCT), with a prevalence of 34.1% among people over 60 years old. This study aims to develop artificial intelligence (AI) systems to assist in the diagnosis of ERM grade using OCT images and to clinically evaluate the potential benefits and risks of our AI systems with a comparative experiment.

METHODS

A segmentation deep learning (DL) model that segments retinal features associated with ERM severity and a classification DL model that grades the severity of ERM were developed based on an OCT dataset obtained from three hospitals. A comparative experiment was conducted to compare the performance of four general ophthalmologists with and without assistance from the AI in diagnosing ERM severity.

RESULTS

The segmentation network had a pixel accuracy (PA) of 0.980 and a mean intersection over union (MIoU) of 0.873, while the six-classification network had a total accuracy of 81.3%. The diagnostic accuracy scores of the four ophthalmologists increased with AI assistance from 81.7%, 80.7%, 78.0%, and 80.7% to 87.7%, 86.7%, 89.0%, and 91.3%, respectively, while the corresponding time expenditures were reduced. The specific results of the study as well as the misinterpretations of the AI systems were analysed.

CONCLUSION

Through our comparative experiment, the AI systems proved to be valuable references for medical diagnosis and demonstrated the potential to accelerate clinical workflows. Systematic efforts are needed to ensure the safe and rapid integration of AI systems into ophthalmic practice.

Classification of diabetic maculopathy based on optical coherence tomography images using a Vision Transformer model

PURPOSE

To develop a Vision Transformer model to detect different stages of diabetic maculopathy (DM) based on optical coherence tomography (OCT) images.

METHODS

After removing images with poor quality, a total of 3319 OCT images were extracted from the Eye Center of the Renmin Hospital of Wuhan University and randomly split the images into training and validation sets in a 7:3 ratio. All macular cross-sectional scan OCT images were collected retrospectively from the eyes of DM patients from 2016 to 2022. One of the OCT stages of DM, including early diabetic macular oedema (DME), advanced DME, severe DME and atrophic maculopathy, was labelled on the collected images, respectively. A deep learning (DL) model based on Vision Transformer was trained to detect four OCT grading of DM.

RESULTS

The model proposed in our paper can provide an impressive detection performance. We achieved an accuracy of 82.00%, an F1 score of 83.11%, an area under the receiver operating characteristic curve (AUC) of 0.96. The AUC for the detection of four OCT grading (ie, early DME, advanced DME, severe DME and atrophic maculopathy) was 0.96, 0.95, 0.87 and 0.98, respectively, with an accuracy of 90.87%, 89.96%, 94.42% and 95.13%, respectively, a precision of 88.46%, 80.31%, 89.42% and 87.74%, respectively, a sensitivity of 87.03%, 88.18%, 63.39% and 89.42%, respectively, a specificity of 93.02%, 90.72%, 98.40% and 96.66%, respectively and an F1 score of 87.74%, 84.06%, 88.18% and 88.57%, respectively.

CONCLUSION

Our DL model based on Vision Transformer demonstrated a relatively high accuracy in the detection of OCT grading of DM, which can help with patients in a preliminary screening to identify groups with serious conditions. These patients need a further test for an accurate diagnosis, and a timely treatment to obtain a good visual prognosis. These results emphasised the potential of artificial intelligence in assisting clinicians in developing therapeutic strategies with DM in the future.

Rapid On-Site AI-Assisted Grading for Lung Surgery Based on Optical Coherence Tomography

The determination of resection extent traditionally relies on the microscopic invasiveness of frozen sections (FSs) and is crucial for surgery of early lung cancer with preoperatively unknown histology. While previous research has shown the value of optical coherence tomography (OCT) for instant lung cancer diagnosis, tumor grading through OCT remains challenging. Therefore, this study proposes an interactive human-machine interface (HMI) that integrates a mobile OCT system, deep learning algorithms, and attention mechanisms. The system is designed to mark the lesion's location on the image smartly and perform tumor grading in real time, potentially facilitating clinical decision making. Twelve patients with a preoperatively unknown tumor but a final diagnosis of adenocarcinoma underwent thoracoscopic resection, and the artificial intelligence (AI)-designed system mentioned above was used to measure fresh specimens. Results were compared to FSs benchmarked on permanent pathologic reports. Current results show better differentiating power among minimally invasive adenocarcinoma (MIA), invasive adenocarcinoma (IA), and normal tissue, with an overall accuracy of 84.9%, compared to 20% for FSs. Additionally, the sensitivity and specificity, the sensitivity and specificity were 89% and 82.7% for MIA and 94% and 80.6% for IA, respectively. The results suggest that this AI system can potentially produce rapid and efficient diagnoses and ultimately improve patient outcomes.

Microsurgery Robots: Applications, Design, and Development

Microsurgical techniques have been widely utilized in various surgical specialties, such as ophthalmology, neurosurgery, and otolaryngology, which require intricate and precise surgical tool manipulation on a small scale. In microsurgery, operations on delicate vessels or tissues require high standards in surgeons' skills. This exceptionally high requirement in skills leads to a steep learning curve and lengthy training before the surgeons can perform microsurgical procedures with quality outcomes. The microsurgery robot (MSR), which can improve surgeons' operation skills through various functions, has received extensive research attention in the past three decades. There have been many review papers summarizing the research on MSR for specific surgical specialties. However, an in-depth review of the relevant technologies used in MSR systems is limited in the literature. This review details the technical challenges in microsurgery, and systematically summarizes the key technologies in MSR with a developmental perspective from the basic structural mechanism design, to the perception and human-machine interaction methods, and further to the ability in achieving a certain level of autonomy. By presenting and comparing the methods and technologies in this cutting-edge research, this paper aims to provide readers with a comprehensive understanding of the current state of MSR research and identify potential directions for future development in MSR.

OCT-based deep-learning models for the identification of retinal key signs

A new system based on binary Deep Learning (DL) convolutional neural networks has been developed to recognize specific retinal abnormality signs on Optical Coherence Tomography (OCT) images useful for clinical practice. Images from the local hospital database were retrospectively selected from 2017 to 2022. Images were labeled by two retinal specialists and included central fovea cross-section OCTs. Nine models were developed using the Visual Geometry Group 16 architecture to distinguish healthy versus abnormal retinas and to identify eight different retinal abnormality signs. A total of 21,500 OCT images were screened, and 10,770 central fovea cross-section OCTs were included in the study. The system achieved high accuracy in identifying healthy retinas and specific pathological signs, ranging from 93 to 99%. Accurately detecting abnormal retinal signs from OCT images is crucial for patient care. This study aimed to identify specific signs related to retinal pathologies, aiding ophthalmologists in diagnosis. The high-accuracy system identified healthy retinas and pathological signs, making it a useful diagnostic aid. Labelled OCT images remain a challenge, but our approach reduces dataset creation time and shows DL models' potential to improve ocular pathology diagnosis and clinical decision-making.

FIT-Net: Feature Interaction Transformer Network for Pathologic Myopia Diagnosis

Automatic and accurate classification of retinal optical coherence tomography (OCT) images is essential to assist physicians in diagnosing and grading pathological changes in pathologic myopia (PM). Clinically, due to the obvious differences in the position, shape, and size of the lesion structure in different scanning directions, ophthalmologists usually need to combine the lesion structure in the OCT images in the horizontal and vertical scanning directions to diagnose the type of pathological changes in PM. To address these challenges, we propose a novel feature interaction Transformer network (FIT-Net) to diagnose PM using OCT images, which consists of two dual-scale Transformer (DST) blocks and an interactive attention (IA) unit. Specifically, FIT-Net divides image features of different scales into a series of feature block sequences. In order to enrich the feature representation, we propose an IA unit to realize the interactive learning of class token in feature sequences of different scales. The interaction between feature sequences of different scales can effectively integrate different scale image features, and hence FIT-Net can focus on meaningful lesion regions to improve the PM classification performance. Finally, by fusing the dual-view image features in the horizontal and vertical scanning directions, we propose six dual-view feature fusion methods for PM diagnosis. The extensive experimental results based on the clinically obtained datasets and three publicly available datasets demonstrate the effectiveness and superiority of the proposed method. Our code is avaiable at: https://github.com/chenshaobin/FITNet.

Domain Adaptation-Based Automated Detection of Retinal Diseases from Optical Coherence Tomography Images

PURPOSE

To verify the effectiveness of domain adaptation in generalizing a deep learning-based anomaly detection model to unseen optical coherence tomography (OCT) images.

METHODS

Two datasets (source and target, where labelled training data was only available for the source) captured by two different OCT facilities were collected to train the model. We defined the model containing a feature extractor and a classifier as Model One and trained it with only labeled source data. The proposed domain adaptation model was defined as Model Two, which has the same feature extractor and classifier as Model One but has an additional domain critic in the training phase. We trained the Model Two with both the source and target datasets; the feature extractor was trained to extract domain-invariant features while the domain critic learned to capture the domain discrepancy. Finally, a well-trained feature extractor was used to extract domain-invariant features and a classifier was used to detect images with retinal pathologies in the two domains.

RESULTS

The target data consisted of 3,058 OCT B-scans captured from 163 participants. Model One achieved an area under the curve (AUC) of 0.912 [95% confidence interval (CI), 0.895-0.962], while Model Two achieved an overall AUC of 0.989 [95% CI, 0.982-0.993] for detecting pathological retinas from healthy samples. Moreover, Model Two achieved an average retinopathies detection accuracy of 94.52%. Heat maps showed that the algorithm focused on the area with pathological changes during processing, similar to manual grading in daily clinical work.

CONCLUSIONS

The proposed domain adaptation model showed a strong ability in reducing the domain distance between different OCT datasets.

Deep learning for automated detection of neovascular leakage on ultra-widefield fluorescein angiography in diabetic retinopathy

Diabetic retinopathy is a leading cause of blindness in working-age adults worldwide. Neovascular leakage on fluorescein angiography indicates progression to the proliferative stage of diabetic retinopathy, which is an important distinction that requires timely ophthalmic intervention with laser or intravitreal injection treatment to reduce the risk of severe, permanent vision loss. In this study, we developed a deep learning algorithm to detect neovascular leakage on ultra-widefield fluorescein angiography images obtained from patients with diabetic retinopathy. The algorithm, an ensemble of three convolutional neural networks, was able to accurately classify neovascular leakage and distinguish this disease marker from other angiographic disease features. With additional real-world validation and testing, our algorithm could facilitate identification of neovascular leakage in the clinical setting, allowing timely intervention to reduce the burden of blinding diabetic eye disease.

An Explainable Fully Dense Fusion Neural Network with Deep Support Vector Machine for Retinal Disease Determination

Retinal issues are crucial because they result in visual loss. Early diagnosis can aid physicians in initiating treatment and preventing visual loss. Optical coherence tomography (OCT), which portrays retinal morphology cross-sectionally and noninvasively, is used to identify retinal abnormalities. The process of analyzing OCT images, on the other hand, takes time. This study has proposed a hybrid approach based on a fully dense fusion neural network (FD-CNN) and dual preprocessing to identify retinal diseases, such as choroidal neovascularization, diabetic macular edema, drusen from OCT images. A dual preprocessing methodology, in other words, a hybrid speckle reduction filter was initially used to diminish speckle noise present in OCT images. Secondly, the FD-CNN architecture was trained, and the features obtained from this architecture were extracted. Then Deep Support Vector Machine (D-SVM) and Deep K-Nearest Neighbor (D-KNN) classifiers were proposed to reclassify those features and tested on University of California San Diego (UCSD) and Duke OCT datasets. D-SVM demonstrated the best performance in both datasets. D-SVM achieved 99.60% accuracy, 99.60% sensitivity, 99.87% specificity, 99.60% precision and 99.60% F1 score in the UCSD dataset. It achieved 97.50% accuracy, 97.64% sensitivity, 98.91% specificity, 96.61% precision, and 97.03% F1 score in Duke dataset. Additionally, the results were compared to state-of-the-art works on the both datasets. The D-SVM was demonstrated to be an efficient and productive strategy for improving the robustness of automatic retinal disease classification. Also, in this study, it is shown that the unboxing of how AI systems' black-box choices is made by generating heat maps using the local interpretable model-agnostic explanation method, which is an explainable artificial intelligence (XAI) technique. Heat maps, in particular, may contribute to the development of more stable deep learning-based systems, as well as enhancing the confidence in the diagnosis of retinal disease in the analysis of OCT image for ophthalmologists.

An interpretable transformer network for the retinal disease classification using optical coherence tomography

Retinal illnesses such as age-related macular degeneration and diabetic macular edema will lead to irreversible blindness. With optical coherence tomography (OCT), doctors are able to see cross-sections of the retinal layers and provide patients with a diagnosis. Manual reading of OCT images is time-consuming, labor-intensive and even error-prone. Computer-aided diagnosis algorithms improve efficiency by automatically analyzing and diagnosing retinal OCT images. However, the accuracy and interpretability of these algorithms can be further improved through effective feature extraction, loss optimization and visualization analysis. In this paper, we propose an interpretable Swin-Poly Transformer network for performing automatically retinal OCT image classification. By shifting the window partition, the Swin-Poly Transformer constructs connections between neighboring non-overlapping windows in the previous layer and thus has the flexibility to model multi-scale features. Besides, the Swin-Poly Transformer modifies the importance of polynomial bases to refine cross entropy for better retinal OCT image classification. In addition, the proposed method also provides confidence score maps, assisting medical practitioners to understand the models' decision-making process. Experiments in OCT2017 and OCT-C8 reveal that the proposed method outperforms both the convolutional neural network approach and ViT, with an accuracy of 99.80% and an AUC of 99.99%.

A review of deep learning-based multiple-lesion recognition from medical images: classification, detection and segmentation

Deep learning-based methods have become the dominant methodology in medical image processing with the advancement of deep learning in natural image classification, detection, and segmentation. Deep learning-based approaches have proven to be quite effective in single lesion recognition and segmentation. Multiple-lesion recognition is more difficult than single-lesion recognition due to the little variation between lesions or the too wide range of lesions involved. Several studies have recently explored deep learning-based algorithms to solve the multiple-lesion recognition challenge. This paper includes an in-depth overview and analysis of deep learning-based methods for multiple-lesion recognition developed in recent years, including multiple-lesion recognition in diverse body areas and recognition of whole-body multiple diseases. We discuss the challenges that still persist in the multiple-lesion recognition tasks by critically assessing these efforts. Finally, we outline existing problems and potential future research areas, with the hope that this review will help researchers in developing future approaches that will drive additional advances.

Deep Learning-Based System for Disease Screening and Pathologic Region Detection From Optical Coherence Tomography Images

Purpose

This study was designed to apply deep learning models in retinal disease screening and lesion detection based on optical coherence tomography (OCT) images.

Methods

We collected 37,138 OCT images from 775 patients and labelled by ophthalmologists. Multiple deep learning models including ResNet50 and YOLOv3 were developed to identify the types and locations of diseases or lesions based on the images.

Results

The model were evaluated using patient-based independent holdout set. For binary classification of OCT images with or without lesions, the performance accuracy was 98.5%, sensitivity was 98.7%, specificity was 98.4%, and the F1 score was 97.7%. For multiclass multilabel disease classification, the models was able to detect vitreomacular traction syndrome and age-related macular degeneration both with an accuracy of more than 99%, sensitivity of more than 98%, specificity of more than 98%, and an F1 score of more than 97%. For lesion location detection, the recalls for different lesion types ranged from 87.0% (epiretinal membrane) to 98.2% (macular pucker).

Conclusions

Deep learning-based models have potentials to aid retinal disease screening, classification and diagnosis with excellent performance, which may serve as useful references for ophthalmologists.

Translational Relevance

The deep learning-based models are capable of identifying and predicting different eye diseases and lesions from OCT images and may have potential clinical application to assist the ophthalmologists for fast and accuracy retinal disease screening.

Artificial intelligence method based on multi-feature fusion for automatic macular edema (ME) classification on spectral-domain optical coherence tomography (SD-OCT) images

Purpose

A common ocular manifestation, macular edema (ME) is the primary cause of visual deterioration. In this study, an artificial intelligence method based on multi-feature fusion was introduced to enable automatic ME classification on spectral-domain optical coherence tomography (SD-OCT) images, to provide a convenient method of clinical diagnosis.

Methods

First, 1,213 two-dimensional (2D) cross-sectional OCT images of ME were collected from the Jiangxi Provincial People's Hospital between 2016 and 2021. According to OCT reports of senior ophthalmologists, there were 300 images with diabetic (DME), 303 images with age-related macular degeneration (AMD), 304 images with retinal-vein occlusion (RVO), and 306 images with central serous chorioretinopathy (CSC). Then, traditional omics features of the images were extracted based on the first-order statistics, shape, size, and texture. After extraction by the alexnet, inception_v3, resnet34, and vgg13 models and selected by dimensionality reduction using principal components analysis (PCA), the deep-learning features were fused. Next, the gradient-weighted class-activation map (Grad-CAM) was used to visualize the-deep-learning process. Finally, the fusion features set, which was fused from the traditional omics features and the deep-fusion features, was used to establish the final classification models. The performance of the final models was evaluated by accuracy, confusion matrix, and the receiver operating characteristic (ROC) curve.

Results

Compared with other classification models, the performance of the support vector machine (SVM) model was best, with an accuracy of 93.8%. The area under curves AUC of micro- and macro-averages were 99%, and the AUC of the AMD, DME, RVO, and CSC groups were 100, 99, 98, and 100%, respectively.

Conclusion

The artificial intelligence model in this study could be used to classify DME, AME, RVO, and CSC accurately from SD-OCT images.

A Deep Learning-Based Framework for Retinal Disease Classification

This study addresses the problem of the automatic detection of disease states of the retina. In order to solve the abovementioned problem, this study develops an artificially intelligent model. The model is based on a customized 19-layer deep convolutional neural network called VGG-19 architecture. The model (VGG-19 architecture) is empowered by transfer learning. The model is designed so that it can learn from a large set of images taken with optical coherence tomography (OCT) and classify them into four conditions of the retina: (1) choroidal neovascularization, (2) drusen, (3) diabetic macular edema, and (4) normal form. The training datasets (taken from publicly available sources) consist of 84,568 instances of OCT retinal images. The datasets exhibit all four classes of retinal disease mentioned above. The proposed model achieved a 99.17% classification accuracy with 0.995 specificities and 0.99 sensitivity, making it better than the existing models. In addition, the proper statistical evaluation is done on the predictions using such performance measures as (1) area under the receiver operating characteristic curve, (2) Cohen's kappa parameter, and (3) confusion matrix. Experimental results show that the proposed VGG-19 architecture coupled with transfer learning is an effective technique for automatically detecting the disease state of a retina.

iERM: An Interpretable Deep Learning System to Classify Epiretinal Membrane for Different Optical Coherence Tomography Devices: A Multi-Center Analysis

Background: Epiretinal membranes (ERM) have been found to be common among individuals >50 years old. However, the severity grading assessment for ERM based on optical coherence tomography (OCT) images has remained a challenge due to lacking reliable and interpretable analysis methods. Thus, this study aimed to develop a two-stage deep learning (DL) system named iERM to provide accurate automatic grading of ERM for clinical practice. Methods: The iERM was trained based on human segmentation of key features to improve classification performance and simultaneously provide interpretability to the classification results. We developed and tested iERM using a total of 4547 OCT B-Scans of four different commercial OCT devices that were collected from nine international medical centers. Results: As per the results, the integrated network effectively improved the grading performance by 1−5.9% compared with the traditional classification DL model and achieved high accuracy scores of 82.9%, 87.0%, and 79.4% in the internal test dataset and two external test datasets, respectively. This is comparable to retinal specialists whose average accuracy scores are 87.8% and 79.4% in two external test datasets. Conclusion: This study proved to be a benchmark method to improve the performance and enhance the interpretability of the traditional DL model with the implementation of segmentation based on prior human knowledge. It may have the potential to provide precise guidance for ERM diagnosis and treatment.

Prediction of Visual Impairment in Epiretinal Membrane and Feature Analysis: A Deep Learning Approach Using Optical Coherence Tomography

PURPOSE

The aim was to develop a deep learning model for predicting the extent of visual impairment in epiretinal membrane (ERM) using optical coherence tomography (OCT) images, and to analyze the associated features.

METHODS

Six hundred macular OCT images from eyes with ERM and no visually significant media opacity or other retinal diseases were obtained. Those with best-corrected visual acuity ≤20/50 were classified as "profound visual impairment," while those with best-corrected visual acuity >20/50 were classified as "less visual impairment." Ninety percent of images were used as the training data set and 10% were used for testing. Two convolutional neural network models (ResNet-50 and ResNet-18) were adopted for training. The t-distributed stochastic neighbor-embedding approach was used to compare their performances. The Grad-CAM technique was used in the heat map generative phase for feature analysis.

RESULTS

During the model development, the training accuracy was 100% in both convolutional neural network models, while the testing accuracy was 70% and 80% for ResNet-18 and ResNet-50, respectively. The t-distributed stochastic neighbor-embedding approach found that the deeper structure (ResNet-50) had better discrimination on OCT characteristics for visual impairment than the shallower structure (ResNet-18). The heat maps indicated that the key features for visual impairment were located mostly in the inner retinal layers of the fovea and parafoveal regions.

CONCLUSIONS

Deep learning algorithms could assess the extent of visual impairment from OCT images in patients with ERM. Changes in inner retinal layers were found to have a greater impact on visual acuity than the outer retinal changes.

Optical coherence tomography image based eye disease detection using deep convolutional neural network

Over the past few decades, health care industries and medical practitioners faced a lot of obstacles to diagnosing medical-related problems due to inadequate technology and availability of equipment. In the present era, computer science technologies such as IoT, Cloud Computing, Artificial Intelligence and its allied techniques, etc. play a crucial role in the identification of medical diseases, especially in the domain of Ophthalmology. Despite this, ophthalmologists have to perform the various disease diagnosis task manually which is time-consuming and the chances of error are also very high because some of the abnormalities of eye diseases possess the same symptoms. Furthermore, multiple autonomous systems also exist to categorize the diseases but their prediction rate does not accomplish state-of-art accuracy. In the proposed approach by implementing the concept of Attention, Transfer Learning with the Deep Convolution Neural Network, the model accomplished an accuracy of 97.79% and 95.6% on the training and testing data respectively. This autonomous model efficiently classifies the various oscular disorders namely Choroidal Neovascularization, Diabetic Macular Edema, Drusen from the Optical Coherence Tomography images. It may provide a realistic solution to the healthcare sector to bring down the ophthalmologist burden in the screening of Diabetic Retinopathy.

Generative adversarial network-based deep learning approach in classification of retinal conditions with optical coherence tomography images

PURPOSE

To determine whether a deep learning approach using generative adversarial networks (GANs) is beneficial for the classification of retinal conditions with Optical coherence tomography (OCT) images.

METHODS

Our study utilized 84,452 retinal OCT images obtained from a publicly available dataset (Kermany Dataset). Employing GAN, synthetic OCT images are produced to balance classes of retinal disorders. A deep learning classification model is constructed using pretrained deep neural networks (DNNs), and outcomes are evaluated using 2082 images collected from patients who visited the Department of Ophthalmology and the Department of Endocrinology and Metabolism at the Tri-service General Hospital in Taipei from January 2017 to December 2021.

RESULTS

The highest classification accuracies accomplished by deep learning machines trained on the unbalanced dataset for its training set, validation set, fivefold cross validation (CV), Kermany test set, and TSGH test set were 97.73%, 96.51%, 97.14%, 99.59%, and 81.03%, respectively. The highest classification accuracies accomplished by deep learning machines trained on the synthesis-balanced dataset for its training set, validation set, fivefold CV, Kermany test set, and TSGH test set were 98.60%, 98.41%, 98.52%, 99.38%, and 84.92%, respectively. In comparing the highest accuracies, deep learning machines trained on the synthesis-balanced dataset outperformed deep learning machines trained on the unbalanced dataset for the training set, validation set, fivefold CV, and TSGH test set.

CONCLUSIONS

Overall, deep learning machines on a synthesis-balanced dataset demonstrated to be advantageous over deep learning machines trained on an unbalanced dataset for the classification of retinal conditions.

A review of generative adversarial network applications in optical coherence tomography image analysis

Optical coherence tomography (OCT) has revolutionized ophthalmic clinical practice and research, as a result of the high-resolution images that the method is able to capture in a fast, non-invasive manner. Although clinicians can interpret OCT images qualitatively, the ability to quantitatively and automatically analyse these images represents a key goal for eye care by providing clinicians with immediate and relevant metrics to inform best clinical practice. The range of applications and methods to analyse OCT images is rich and rapidly expanding. With the advent of deep learning methods, the field has experienced significant progress with state-of-the-art-performance for several OCT image analysis tasks. Generative adversarial networks (GANs) represent a subfield of deep learning that allows for a range of novel applications not possible in most other deep learning methods, with the potential to provide more accurate and robust analyses. In this review, the progress in this field and clinical impact are reviewed and the potential future development of applications of GANs to OCT image processing are discussed.

Application of Deep Learning for Automated Detection of Polypoidal Choroidal Vasculopathy in Spectral Domain Optical Coherence Tomography

Objective

To develop an automated polypoidal choroidal vasculopathy (PCV) screening model to distinguish PCV from wet age-related macular degeneration (wet AMD).

Methods

A retrospective review of spectral domain optical coherence tomography (SD-OCT) images was undertaken. The included SD-OCT images were classified into two distinct categories (PCV or wet AMD) prior to the development of the PCV screening model. The automated detection of PCV using the developed model was compared with the results of gold-standard fundus fluorescein angiography and indocyanine green (FFA + ICG) angiography. A framework of SHapley Additive exPlanations was used to interpret the results from the model.

Results

A total of 2334 SD-OCT images were enrolled for training purposes, and an additional 1171 SD-OCT images were used for external validation. The ResNet attention model yielded superior performance with average area under the curve values of 0.8 and 0.81 for the training and external validation data sets, respectively. The sensitivity/specificity calculated at a patient level was 100%/60% and 85%/71% for the training and external validation data sets, respectively.

Conclusions

A conventional FFA + ICG investigation to differentiate PCV from wet AMD requires intense health care resources and adversely affects patients. A deep learning algorithm is proposed to automatically distinguish PCV from wet AMD. The developed algorithm exhibited promising performance for further development into an alternative PCV screening tool. Enhancement of the model's performance with additional data is needed prior to implementation of this diagnostic tool in real-world clinical practice. The invisibility of disease signs within SD-OCT images is the main limitation of the proposed model.

Translational Relevance

Basic research of deep learning algorithms was applied to differentiate PCV from wet AMD based on OCT images, benefiting a diagnosis process and minimizing a risk of ICG angiogram.

Predicting Visual Improvement After Macular Hole Surgery: A Combined Model Using Deep Learning and Clinical Features

Purpose

The purpose of this study was to assess the feasibility of deep learning (DL) methods to enhance the prediction of visual acuity (VA) improvement after macular hole (MH) surgery from a combined model using DL on high-definition optical coherence tomography (HD-OCT) B-scans and clinical features.

Methods

We trained a DL convolutional neural network (CNN) using pre-operative HD-OCT B-scans of the macula and combined with a logistic regression model of pre-operative clinical features to predict VA increase ≥15 Early Treatment Diabetic Retinopathy Study (ETDRS) letters at 6 months post-vitrectomy in closed MHs. A total of 121 MHs with 242 HD-OCT B-scans and 484 clinical data points were used to train, validate, and test the model. Prediction of VA increase was evaluated using the area under the receiver operating characteristic curve (AUROC) and F1 scores. We also extracted the weight of each input feature in the hybrid model.

Results

All performances are reported on the held-out test set, matching results obtained with cross-validation. Using a regression on clinical features, the AUROC was 80.6, with an F1 score of 79.7. For the CNN, relying solely on the HD-OCT B-scans, the AUROC was 72.8 ± 14.6, with an F1 score of 61.5 ± 23.7. For our hybrid regression model using clinical features and CNN prediction, the AUROC was 81.9 ± 5.2, with an F1 score of 80.4 ± 7.7. In the hybrid model, the baseline VA was the most important feature (weight = 59.1 ± 6.9%), while the weight of HD-OCT prediction was 9.6 ± 4.2%.

Conclusions

Both the clinical data and HD-OCT models can predict postoperative VA improvement in patients undergoing vitrectomy for a MH with good discriminative performances. Combining them into a hybrid model did not significantly improve performance.

Translational Relevance

OCT-based DL models can predict postoperative VA improvement following vitrectomy for MH but fusing those models with clinical data might not provide improved predictive performance.

Identification of Sex and Age from Macular Optical Coherence Tomography and Feature Analysis Using Deep Learning

PURPOSE

To develop deep learning models for identification of sex and age from macular optical coherence tomography (OCT) and to analyze the features for differentiation of sex and age.

DESIGN

Algorithm development using database of macular OCT.

METHODS

We reviewed 6147 sets of macular OCT images from the healthy eyes of 3134 individuals from a single eye center in Taiwan. Deep learning-based algorithms were used to develop models for the identification of sex and age, and 10-fold cross-validation was applied. Gradient-weighted class activation mapping was used for feature analysis.

RESULTS

The accuracy for sex prediction using deep learning from macular OCT was 85.6% ± 2.1% compared with accuracy of 61.9% using macular thickness and 61.4% ± 4.0% using deep learning from infrared fundus photography (P < .001 for both). The mean absolute error for age prediction using deep learning from macular OCT was 5.78 ± 0.29 years. A thorough analysis of the prediction accuracy and the gradient-weighted class activation mapping showed that the cross-sectional foveal contour lead to a better sex distinction than macular thickness or fundus photography, and the age-related characteristics of macula were on the whole layers of retina rather than the choroid.

CONCLUSIONS

Sex and age could be identified from macular OCT using deep learning with good accuracy. The main sexual difference of macula lies in the foveal contour, and the whole layers of retina differ with aging. These novel findings provide useful information for further investigation in the pathogenesis of sex- and age-related macular structural diseases.

Anatomical prognosis after idiopathic macular hole surgery: machine learning based-predection

To develop a machine learning (ML) model for the prediction of the idiopathic macular hole (MH) status at 9 months after vitrectomy and inverted flap internal limiting membrane (ILM) peeling surgery. This single center was conducted at Department A, Institute Hedi Raies of Ophthalmology, Tunis, Tunisia. The study included 114 patients. In total, 120 eyes underwent optical coherence tomography (OCT) and inverted flap ILM peeling for surgery. Then 510 B scan of macular OCT was acquired 9 months after surgery. MH diameter, basal MH diameter (b), nasal and temporal arm lengths and macular hole angle were measured. Indices including hole form factor, MH index, diameter hole index (DHI) and tractional hole, MH area index and MH volume index were calculated. Receiver operating characteristic (ROC) curves and cut‑off values were derived for each indices predicting closure or not of the MH. The area under the receiver operating characteristic curve (AUC) and kappa value were calculated to evaluate performance of the medical decision support system (MDSS) in predicting the MH closure. From the ROC curve analysis, it was derived that MH indices like MH diameter, diameter hole index (DHI), MH index, and hole formation factor were capable of successfully predicting MH closure while basal diameter, DHI and MH area index predicted none closure MH. The MDSS achieved an AUC of 0.984 with a kappa value of 0.934. Based on the preoperative OCT parameters, our ML model achieved remarkable accuracy in predicting MH outcomes after pars plana vitrectomy and inverted flap ILM peeling. Therefore, MDSS may help optimize surgical planning for full thickness macular hole patients in the future.

Learning to Discover Explainable Clinical Features With Minimum Supervision

Purpose

To compare supervised transfer learning to semisupervised learning for their ability to learn in-depth knowledge with limited data in the optical coherence tomography (OCT) domain.

Methods

Transfer learning with EfficientNet-B4 and semisupervised learning with SimCLR are used in this work. The largest public OCT dataset, consisting of 108,312 images and four categories (choroidal neovascularization, diabetic macular edema, drusen, and normal) is used. In addition, two smaller datasets are constructed, containing 31,200 images for the limited version and 4000 for the mini version of the dataset. To illustrate the effectiveness of the developed models, local interpretable model-agnostic explanations and class activation maps are used as explainability techniques.

Results

The proposed transfer learning approach using the EfficientNet-B4 model trained on the limited dataset achieves an accuracy of 0.976 (95% confidence interval [CI], 0.963, 0.983), sensitivity of 0.973 and specificity of 0.991. The semisupervised based solution with SimCLR using 10% labeled data and the limited dataset performs with an accuracy of 0.946 (95% CI, 0.932, 0.960), sensitivity of 0.941, and specificity of 0.983.

Conclusions

Semisupervised learning has a huge potential for datasets that contain both labeled and unlabeled inputs, generally, with a significantly smaller number of labeled samples. The semisupervised based solution provided with merely 10% labeled data achieves very similar performance to the supervised transfer learning that uses 100% labeled samples.

Translational Relevance

Semisupervised learning enables building performant models while requiring less expertise effort and time by using to good advantage the abundant amount of available unlabeled data along with the labeled samples.

Development and Validation of an Explainable Artificial Intelligence Framework for Macular Disease Diagnosis Based on OCT Images

PURPOSE

To develop and validate an artificial intelligence framework for identifying multiple retinal lesions at image-level and performing an explainable macular disease diagnosis at eye-level in optical coherence tomography (OCT) images.

METHODS

26,815 OCT images were collected from 865 eyes, and 9 retinal lesions and 3 macular diseases were labelled by ophthalmologists, including diabetic macular edema (DME) and dry/wet age-related macular degeneration (dry/wet AMD). We applied deep learning to classify retinal lesion at image-level and random forests to achieve an explainable disease diagnosis at eye-level. The performance of the integrated two-stage framework was evaluated and compared with human experts.

RESULTS

On testing dataset of 2,480 OCT images from 80 eyes, deep learning model achieved average Area Under Curve (AUC) of 0.978 (95% CI, 0.971-0.983) for lesion classification. And random forests performed accurate disease diagnosis with 0% error rate, which achieved the same accuracy as one of human experts and was better than the other 3 experts. It also revealed that the detection of specific lesions in the center of macular region had more contribution to macular disease diagnosis.

CONCLUSIONS

The integrated method achieved high accuracy and interpretability in retinal lesion classification and macular disease diagnosis in OCT images, and could have the potential to facilitate the clinical diagnosis.

Artificial intelligence-based detection of epimacular membrane from color fundus photographs

Epiretinal membrane (ERM) is a common ophthalmological disorder of high prevalence. Its symptoms include metamorphopsia, blurred vision, and decreased visual acuity. Early diagnosis and timely treatment of ERM is crucial to preventing vision loss. Although optical coherence tomography (OCT) is regarded as a de facto standard for ERM diagnosis due to its intuitiveness and high sensitivity, ophthalmoscopic examination or fundus photographs still have the advantages of price and accessibility. Artificial intelligence (AI) has been widely applied in the health care industry for its robust and significant performance in detecting various diseases. In this study, we validated the use of a previously trained deep neural network based-AI model in ERM detection based on color fundus photographs. An independent test set of fundus photographs was labeled by a group of ophthalmologists according to their corresponding OCT images as the gold standard. Then the test set was interpreted by other ophthalmologists and AI model without knowing their OCT results. Compared with manual diagnosis based on fundus photographs alone, the AI model had comparable accuracy (AI model 77.08% vs. integrated manual diagnosis 75.69%, χ² = 0.038, P = 0.845, McNemar’s test), higher sensitivity (75.90% vs. 63.86%, χ² = 4.500, P = 0.034, McNemar’s test), under the cost of lower but reasonable specificity (78.69% vs. 91.80%, χ² = 6.125, P = 0.013, McNemar’s test). Thus our AI model can serve as a possible alternative for manual diagnosis in ERM screening.

Student becomes teacher: training faster deep learning lightweight networks for automated identification of optical coherence tomography B-scans of interest using a student-teacher framework

This work explores a student-teacher framework that leverages unlabeled images to train lightweight deep learning models with fewer parameters to perform fast automated detection of optical coherence tomography B-scans of interest. Twenty-seven lightweight models (LWMs) from four families of models were trained on expert-labeled B-scans (∼70 K) as either "abnormal" or "normal", which established a baseline performance for the models. Then the LWMs were trained from random initialization using a student-teacher framework to incorporate a large number of unlabeled B-scans (∼500 K). A pre-trained ResNet50 model served as the teacher network. The ResNet50 teacher model achieved 96.0% validation accuracy and the validation accuracy achieved by the LWMs ranged from 89.6% to 95.1%. The best performing LWMs were 2.53 to 4.13 times faster than ResNet50 (0.109s to 0.178s vs. 0.452s). All LWMs benefitted from increasing the training set by including unlabeled B-scans in the student-teacher framework, with several models achieving validation accuracy of 96.0% or higher. The three best-performing models achieved comparable sensitivity and specificity in two hold-out test sets to the teacher network. We demonstrated the effectiveness of a student-teacher framework for training fast LWMs for automated B-scan of interest detection leveraging unlabeled, routinely-available data.

Contactless optical coherence tomography of the eyes of freestanding individuals with a robotic scanner

Clinical systems for optical coherence tomography (OCT) are used routinely to diagnose and monitor patients with a range of ocular diseases. They are large tabletop instruments operated by trained staff, and require mechanical stabilization of the head of the patient for positioning and motion reduction. Here we report the development and performance of a robot-mounted OCT scanner for the autonomous contactless imaging, at safe distances, of the eyes of freestanding individuals without the need for operator intervention or head stabilization. The scanner uses robotic positioning to align itself with the eye to be imaged, as well as optical active scanning to locate the pupil and to attenuate physiological eye motion. We show that the scanner enables the acquisition of OCT volumetric datasets, comparable in quality to those of clinical tabletop systems, that resolve key anatomic structures relevant for the management of common eye conditions. Robotic OCT scanners may enable the diagnosis and monitoring of patients with eye conditions in non-specialist clinics.