Classification of optical coherence tomography images using a capsule network

Integration of Optical Coherence Tomography Images and Real-Life Clinical Data for Deep Learning Modeling: A Unified Approach in Prognostication of Diabetic Macular Edema

The primary ocular effect of diabetes is diabetic retinopathy (DR), which is associated with diabetic microangiopathy. Diabetic macular edema (DME) can cause vision loss for people with DR. For this reason, deciding on the appropriate treatment and follow-up has a critical role in terms of curing the disease. Current artificial intelligence (AI) approaches focus on OCT images and may ignore clinical, laboratory, and demographic information obtained by the specialist. This study presents a novel deep learning (DL) framework for evaluating the visual outcome of the TREX anti-VEGF intravitreal injection regimen. DL models are trained to extract deep features from OCT and ILM topographic images and the obtained deep features are combined with patients' demographic, clinical, and laboratory findings to predict the direction of the treatment process. When the ResNet-18 network is used, the proposed DL framework is able to predict the prognosis status of patients with the highest accuracy.

Advanced retinal disease detection from OCT images using a hybrid squeeze and excitation enhanced model

BACKGROUND

Retinal problems are critical because they can cause severe vision loss if not treated. Traditional methods for diagnosing retinal disorders often rely heavily on manual interpretation of optical coherence tomography (OCT) images, which can be time-consuming and dependent on the expertise of ophthalmologists. This leads to challenges in early diagnosis, especially as retinal diseases like diabetic macular edema (DME), Drusen, and Choroidal neovascularization (CNV) become more prevalent. OCT helps ophthalmologists diagnose patients more accurately by allowing for early detection. This paper offers a hybrid SE (Squeeze-and-Excitation)-Enhanced Hybrid Model for detecting retinal disorders from OCT images, including DME, Drusen, and CNV, using artificial intelligence and deep learning.

METHODS

The model integrates SE blocks with EfficientNetB0 and Xception architectures, which provide high success in image classification tasks. EfficientNetB0 achieves high accuracy with fewer parameters through model scaling strategies, while Xception offers powerful feature extraction using deep separable convolutions. The combination of these architectures enhances both the efficiency and classification performance of the model, enabling more accurate detection of retinal disorders from OCT images. Additionally, SE blocks increase the representational ability of the network by adaptively recalibrating per-channel feature responses.

RESULTS

The combined features from EfficientNetB0 and Xception are processed via fully connected layers and categorized using the Softmax algorithm. The methodology was tested on UCSD and Duke's OCT datasets and produced excellent results. The proposed SE-Improved Hybrid Model outperformed the current best-known approaches, with accuracy rates of 99.58% on the UCSD dataset and 99.18% on the Duke dataset.

CONCLUSION

These findings emphasize the model's ability to effectively diagnose retinal disorders using OCT images and indicate substantial promise for the development of computer-aided diagnostic tools in the field of ophthalmology.

Uncertainty-aware regression model to predict post-operative visual acuity in patients with macular holes

Full-thickness macular holes are a relatively common and visually disabling condition with a prevalence of approximately 0.5% in the over-40-year-old age group. If left untreated, the hole typically enlarges, reducing visual acuity (VA) below the definition of blindness in the eye affected. They are now routinely treated with surgery, which can close the hole and improve vision in most cases. The extent of improvement, however, is variable and dependent on the size of the hole and other features which can be discerned in spectral-domain optical coherence tomography imaging, which is now routinely available in eye clinics globally. Artificial intelligence (AI) models have been developed to enable surgical decision-making and have achieved relatively high predictive performance. However, their black-box behavior is opaque to users and uncertainty associated with their predictions is not typically stated, leading to a lack of trust among clinicians and patients. In this paper, we describe an uncertainty-aware regression model (U-ARM) for predicting VA for people undergoing macular hole surgery using preoperative spectral-domain optical coherence tomography images, achieving an MAE of 6.07, RMSE of 9.11 and R2 of 0.47 in internal tests, and an MAE of 6.49, RMSE of 9.49, and R2 of 0.42 in external tests. In addition to predicting VA following surgery, U-ARM displays its associated uncertainty, a p-value of <0.005 in internal and external tests, showing the predictions are not due to random chance. We then qualitatively evaluated the performance of U-ARM. Lastly, we demonstrate out-of-sample data performance, generalizing well to data outside the training distribution, low-quality images, and unseen instances not encountered during training. The results show that U-ARM outperforms commonly used methods in terms of prediction and reliability. U-ARM is thus a promising approach for clinical settings and can improve the reliability of AI models in predicting VA.

VLFATRollout: Fully transformer-based classifier for retinal OCT volumes

BACKGROUND AND OBJECTIVE

Despite the promising capabilities of 3D transformer architectures in video analysis, their application to high-resolution 3D medical volumes encounters several challenges. One major limitation is the high number of 3D patches, which reduces the efficiency of the global self-attention mechanisms of transformers. Additionally, background information can distract vision transformers from focusing on crucial areas of the input image, thereby introducing noise into the final representation. Moreover, the variability in the number of slices per volume complicates the development of models capable of processing input volumes of any resolution while simple solutions like subsampling may risk losing essential diagnostic details.

METHODS

To address these challenges, we introduce an end-to-end transformer-based framework, variable length feature aggregator transformer rollout (VLFATRollout), to classify volumetric data. The proposed VLFATRollout enjoys several merits. First, the proposed VLFATRollout can effectively mine slice-level fore-background information with the help of transformer's attention matrices. Second, randomization of volume-wise resolution (i.e. the number of slices) during training enhances the learning capacity of the learnable positional embedding (PE) assigned to each volume slice. This technique allows the PEs to generalize across neighboring slices, facilitating the handling of high-resolution volumes at the test time.

RESULTS

VLFATRollout was thoroughly tested on the retinal optical coherence tomography (OCT) volume classification task, demonstrating a notable average improvement of 5.47% in balanced accuracy over the leading convolutional models for a 5-class diagnostic task. These results emphasize the effectiveness of our framework in enhancing slice-level representation and its adaptability across different volume resolutions, paving the way for advanced transformer applications in medical image analysis. The code is available at https://github.com/marziehoghbaie/VLFATRollout/.

A comparative evaluation of deep learning approaches for ophthalmology

There is a growing number of publicly available ophthalmic imaging datasets and open-source code for Machine Learning algorithms. This allows ophthalmic researchers and practitioners to independently perform various deep-learning tasks. With the advancement in artificial intelligence (AI) and in the field of imaging, the choice of the most appropriate AI architecture for different tasks will vary greatly. The best-performing AI-dataset combination will depend on the specific problem that needs to be solved and the type of data available. The article discusses different machine learning models and deep learning architectures currently used for various ophthalmic imaging modalities and for different machine learning tasks. It also proposes the most appropriate models based on accuracy and other important factors such as training time, the ability to deploy the model on clinical devices/smartphones, heatmaps that enhance the self-explanatory nature of classification decisions, and the ability to train/adapt on small image datasets to determine if further data collection is worthwhile. The article extensively reviews the existing state-of-the-art AI methods focused on useful machine-learning applications for ophthalmology. It estimates their performance and viability through training and evaluating architectures with different public and private image datasets of different modalities, such as full-color retinal images, OCT images, and 3D OCT scans. The article is expected to benefit the readers by enriching their knowledge of artificial intelligence applied to ophthalmology.

Global research trends and future directions in diabetic macular edema research: A bibliometric and visualized analysis

BACKGROUND

Diabetic Macular Edema (DME) significantly impairs vision in diabetics, with varied patient responses to current treatments like anti-vascular endothelial growth factor (VEGF) therapy underscoring the necessity for continued research into more effective strategies. This study aims to evaluate global research trends and identify emerging frontiers in DME to guide future research and clinical management.

METHODS

A qualitative and quantitative analysis of publications related to diabetic macular edema retrieved from the Web of Science Core Collection (WoSCC) between its inception and September 4, 2023, was conducted. Microsoft Excel, CiteSpace, VOSviewer, Bibliometrix Package, and Tableau were used for the bibliometric analysis and visualization. This encompasses an examination of the overall distribution of annual output, major countries, regions, institutions, authors, core journals, co-cited references, and keyword analyses.

RESULTS

Overall, 5624 publications were analyzed, indicating an increasing trend in DME research. The United States was identified as the leading country in DME research, with the highest h-index of 135 and 91,841 citations. Francesco Bandello emerged as the most prolific author with 97 publications. Neil M. Bressler has the highest h-index and highest total citation count of 46 and 9692, respectively. The journals "Retina - the Journal of Retinal and Vitreous Diseases" and "Ophthalmology" were highlighted as the most prominent in this field. "Retina" leads with 354 publications, a citation count of 11,872, and an h-index of 59. Meanwhile, "Ophthalmology" stands out with the highest overall citation count of 31,558 and the highest h-index of 90. The primary research focal points in diabetic macular edema included "prevalence and risk factors," "pathological mechanisms," "imaging modalities," "treatment strategies," and "clinical trials." Emerging research areas encompassed "deep learning and artificial intelligence," "novel treatment modalities," and "biomarkers."

CONCLUSION

Our bibliometric analysis delineates the leading role of the United States in DME research. We identified current research hotspots, including epidemiological studies, pathophysiological mechanisms, imaging advancements, and treatment innovations. Emerging trends, such as the integration of artificial intelligence and novel therapeutic approaches, highlight future directions. These insights underscore the importance of collaborative and interdisciplinary approaches in advancing DME research and clinical management.

A new computer-aided diagnosis tool based on deep learning methods for automatic detection of retinal disorders from OCT images

PURPOSE

Early detection of retinal disorders using optical coherence tomography (OCT) images can prevent vision loss. Since manual screening can be time-consuming, tedious, and fallible, we present a reliable computer-aided diagnosis (CAD) software based on deep learning. Also, we made efforts to increase the interpretability of the deep learning methods, overcome their vague and black box nature, and also understand their behavior in the diagnosis.

METHODS

We propose a novel method to improve the interpretability of the used deep neural network by embedding the rich semantic information of abnormal areas based on the ophthalmologists' interpretations and medical descriptions in the OCT images. Finally, we trained the classification network on a small subset of the online publicly available University of California San Diego (UCSD) dataset with an overall of 29,800 OCT images.

RESULTS

The experimental results on the 1000 test OCT images show that the proposed method achieves the overall precision, accuracy, sensitivity, and f1-score of 97.6%, 97.6%, 97.6%, and 97.59%, respectively. Also, the heat map images provide a clear region of interest which indicates that the interpretability of the proposed method is increased dramatically.

CONCLUSION

The proposed software can help ophthalmologists in providing a second opinion to make a decision, and primitive automated diagnoses of retinal diseases and even it can be used as a screening tool, in eye clinics. Also, the improvement of the interpretability of the proposed method causes to increase in the model generalization, and therefore, it will work properly on a wide range of other OCT datasets.

Classification of diabetic maculopathy based on optical coherence tomography images using a Vision Transformer model

PURPOSE

To develop a Vision Transformer model to detect different stages of diabetic maculopathy (DM) based on optical coherence tomography (OCT) images.

METHODS

After removing images with poor quality, a total of 3319 OCT images were extracted from the Eye Center of the Renmin Hospital of Wuhan University and randomly split the images into training and validation sets in a 7:3 ratio. All macular cross-sectional scan OCT images were collected retrospectively from the eyes of DM patients from 2016 to 2022. One of the OCT stages of DM, including early diabetic macular oedema (DME), advanced DME, severe DME and atrophic maculopathy, was labelled on the collected images, respectively. A deep learning (DL) model based on Vision Transformer was trained to detect four OCT grading of DM.

RESULTS

The model proposed in our paper can provide an impressive detection performance. We achieved an accuracy of 82.00%, an F1 score of 83.11%, an area under the receiver operating characteristic curve (AUC) of 0.96. The AUC for the detection of four OCT grading (ie, early DME, advanced DME, severe DME and atrophic maculopathy) was 0.96, 0.95, 0.87 and 0.98, respectively, with an accuracy of 90.87%, 89.96%, 94.42% and 95.13%, respectively, a precision of 88.46%, 80.31%, 89.42% and 87.74%, respectively, a sensitivity of 87.03%, 88.18%, 63.39% and 89.42%, respectively, a specificity of 93.02%, 90.72%, 98.40% and 96.66%, respectively and an F1 score of 87.74%, 84.06%, 88.18% and 88.57%, respectively.

CONCLUSION

Our DL model based on Vision Transformer demonstrated a relatively high accuracy in the detection of OCT grading of DM, which can help with patients in a preliminary screening to identify groups with serious conditions. These patients need a further test for an accurate diagnosis, and a timely treatment to obtain a good visual prognosis. These results emphasised the potential of artificial intelligence in assisting clinicians in developing therapeutic strategies with DM in the future.

Few-shot out-of-distribution detection for automated screening in retinal OCT images using deep learning

Deep neural networks have been increasingly proposed for automated screening and diagnosis of retinal diseases from optical coherence tomography (OCT), but often provide high-confidence predictions on out-of-distribution (OOD) cases, compromising their clinical usage. With this in mind, we performed an in-depth comparative analysis of the state-of-the-art uncertainty estimation methods for OOD detection in retinal OCT imaging. The analysis was performed within the use-case of automated screening and staging of age-related macular degeneration (AMD), one of the leading causes of blindness worldwide, where we achieved a macro-average area under the curve (AUC) of 0.981 for AMD classification. We focus on a few-shot Outlier Exposure (OE) method and the detection of near-OOD cases that share pathomorphological characteristics with the inlier AMD classes. Scoring the OOD case based on the Cosine distance in the feature space from the penultimate network layer proved to be a robust approach for OOD detection, especially in combination with the OE. Using Cosine distance and only 8 outliers exposed per class, we were able to improve the near-OOD detection performance of the OE with Reject Bucket method by [Formula: see text] 10% compared to without OE, reaching an AUC of 0.937. The Cosine distance served as a robust metric for OOD detection of both known and unknown classes and should thus be considered as an alternative to the reject bucket class probability in OE approaches, especially in the few-shot scenario. The inclusion of these methodologies did not come at the expense of classification performance, and can substantially improve the reliability and trustworthiness of the resulting deep learning-based diagnostic systems in the context of retinal OCT.

Optical coherence tomography imaging biomarkers associated with neovascular age-related macular degeneration: a systematic review

The aim of this systematic literature review is twofold, (1) detail the impact of retinal biomarkers identifiable via optical coherence tomography (OCT) on disease progression and response to treatment in neovascular age-related macular degeneration (nAMD) and (2) establish which biomarkers are currently identifiable by artificial intelligence (AI) models and the utilisation of this technology. Following the PRISMA guidelines, PubMed was searched for peer-reviewed publications dated between January 2016 and January 2022.

POPULATION

Patients diagnosed with nAMD with OCT imaging.

SETTINGS

Comparable settings to NHS hospitals.

STUDY DESIGNS

Randomised controlled trials, prospective/retrospective cohort studies and review articles. From 228 articles, 130 were full-text reviewed, 50 were removed for falling outside the scope of this review with 10 added from the author's inventory, resulting in the inclusion of 90 articles. From 9 biomarkers identified; intraretinal fluid (IRF), subretinal fluid, pigment epithelial detachment, subretinal hyperreflective material (SHRM), retinal pigmental epithelial (RPE) atrophy, drusen, outer retinal tabulation (ORT), hyperreflective foci (HF) and retinal thickness, 5 are considered pertinent to nAMD disease progression; IRF, SHRM, drusen, ORT and HF. A number of these biomarkers can be classified using current AI models. Significant retinal biomarkers pertinent to disease activity and progression in nAMD are identifiable via OCT; IRF being the most important in terms of the significant impact on visual outcome. Incorporating AI into ophthalmology practice is a promising advancement towards automated and reproducible analyses of OCT data with the ability to diagnose disease and predict future disease conversion.

SYSTEMATIC REVIEW REGISTRATION

This review has been registered with PROSPERO (registration ID: CRD42021233200).

Multi-Stage Classification of Retinal OCT Using Multi-Scale Ensemble Deep Architecture

Accurate noninvasive diagnosis of retinal disorders is required for appropriate treatment or precision medicine. This work proposes a multi-stage classification network built on a multi-scale (pyramidal) feature ensemble architecture for retinal image classification using optical coherence tomography (OCT) images. First, a scale-adaptive neural network is developed to produce multi-scale inputs for feature extraction and ensemble learning. The larger input sizes yield more global information, while the smaller input sizes focus on local details. Then, a feature-rich pyramidal architecture is designed to extract multi-scale features as inputs using DenseNet as the backbone. The advantage of the hierarchical structure is that it allows the system to extract multi-scale, information-rich features for the accurate classification of retinal disorders. Evaluation on two public OCT datasets containing normal and abnormal retinas (e.g., diabetic macular edema (DME), choroidal neovascularization (CNV), age-related macular degeneration (AMD), and Drusen) and comparison against recent networks demonstrates the advantages of the proposed architecture's ability to produce feature-rich classification with average accuracy of 97.78%, 96.83%, and 94.26% for the first (binary) stage, second (three-class) stage, and all-at-once (four-class) classification, respectively, using cross-validation experiments using the first dataset. In the second dataset, our system showed an overall accuracy, sensitivity, and specificity of 99.69%, 99.71%, and 99.87%, respectively. Overall, the tangible advantages of the proposed network for enhanced feature learning might be used in various medical image classification tasks where scale-invariant features are crucial for precise diagnosis.

A bibliometric analysis of artificial intelligence applications in macular edema: exploring research hotspots and Frontiers

Background: Artificial intelligence (AI) is used in ophthalmological disease screening and diagnostics, medical image diagnostics, and predicting late-disease progression rates. We reviewed all AI publications associated with macular edema (ME) research Between 2011 and 2022 and performed modeling, quantitative, and qualitative investigations. Methods: On 1st February 2023, we screened the Web of Science Core Collection for AI applications related to ME, from which 297 studies were identified and analyzed (2011-2022). We collected information on: publications, institutions, country/region, keywords, journal name, references, and research hotspots. Literature clustering networks and Frontier knowledge bases were investigated using bibliometrix-BiblioShiny, VOSviewer, and CiteSpace bibliometric platforms. We used the R "bibliometrix" package to synopsize our observations, enumerate keywords, visualize collaboration networks between countries/regions, and generate a topic trends plot. VOSviewer was used to examine cooperation between institutions and identify citation relationships between journals. We used CiteSpace to identify clustering keywords over the timeline and identify keywords with the strongest citation bursts. Results: In total, 47 countries published AI studies related to ME; the United States had the highest H-index, thus the greatest influence. China and the United States cooperated most closely between all countries. Also, 613 institutions generated publications - the Medical University of Vienna had the highest number of studies. This publication record and H-index meant the university was the most influential in the ME field. Reference clusters were also categorized into 10 headings: retinal Optical Coherence Tomography (OCT) fluid detection, convolutional network models, deep learning (DL)-based single-shot predictions, retinal vascular disease, diabetic retinopathy (DR), convolutional neural networks (CNNs), automated macular pathology diagnosis, dry age-related macular degeneration (DARMD), class weight, and advanced DL architecture systems. Frontier keywords were represented by diabetic macular edema (DME) (2021-2022). Conclusion: Our review of the AI-related ME literature was comprehensive, systematic, and objective, and identified future trends and current hotspots. With increased DL outputs, the ME research focus has gradually shifted from manual ME examinations to automatic ME detection and associated symptoms. In this review, we present a comprehensive and dynamic overview of AI in ME and identify future research areas.

Deep learning-based detection of diabetic macular edema using optical coherence tomography and fundus images: A meta-analysis

Diabetic macular edema (DME) is an important cause of visual impairment in the working-age group. Deep learning methods have been developed to detect DME from two-dimensional retinal images and also from optical coherence tomography (OCT) images. The performances of these algorithms vary and often create doubt regarding their clinical utility. In resource-constrained health-care systems, these algorithms may play an important role in determining referral and treatment. The survey provides a diversified overview of macular edema detection methods, including cutting-edge research, with the objective of providing pertinent information to research groups, health-care professionals, and diabetic patients about the applications of deep learning in retinal image detection and classification process. Electronic databases such as PubMed, IEEE Explore, BioMed, and Google Scholar were searched from inception to March 31, 2022, and the reference lists of published papers were also searched. The study followed the preferred reporting items for systematic review and meta-analysis (PRISMA) reporting guidelines. Examination of various deep learning models and their exhibition regarding precision, epochs, their capacity to detect anomalies for less training data, concepts, and challenges that go deep into the applications were analyzed. A total of 53 studies were included that evaluated the performance of deep learning models in a total of 1,414,169°CT volumes, B-scans, patients, and 472,328 fundus images. The overall area under the receiver operating characteristic curve (AUROC) was 0.9727. The overall sensitivity for detecting DME using OCT images was 96% (95% confidence interval [CI]: 0.94-0.98). The overall sensitivity for detecting DME using fundus images was 94% (95% CI: 0.90-0.96).

An Explainable Fully Dense Fusion Neural Network with Deep Support Vector Machine for Retinal Disease Determination

Retinal issues are crucial because they result in visual loss. Early diagnosis can aid physicians in initiating treatment and preventing visual loss. Optical coherence tomography (OCT), which portrays retinal morphology cross-sectionally and noninvasively, is used to identify retinal abnormalities. The process of analyzing OCT images, on the other hand, takes time. This study has proposed a hybrid approach based on a fully dense fusion neural network (FD-CNN) and dual preprocessing to identify retinal diseases, such as choroidal neovascularization, diabetic macular edema, drusen from OCT images. A dual preprocessing methodology, in other words, a hybrid speckle reduction filter was initially used to diminish speckle noise present in OCT images. Secondly, the FD-CNN architecture was trained, and the features obtained from this architecture were extracted. Then Deep Support Vector Machine (D-SVM) and Deep K-Nearest Neighbor (D-KNN) classifiers were proposed to reclassify those features and tested on University of California San Diego (UCSD) and Duke OCT datasets. D-SVM demonstrated the best performance in both datasets. D-SVM achieved 99.60% accuracy, 99.60% sensitivity, 99.87% specificity, 99.60% precision and 99.60% F1 score in the UCSD dataset. It achieved 97.50% accuracy, 97.64% sensitivity, 98.91% specificity, 96.61% precision, and 97.03% F1 score in Duke dataset. Additionally, the results were compared to state-of-the-art works on the both datasets. The D-SVM was demonstrated to be an efficient and productive strategy for improving the robustness of automatic retinal disease classification. Also, in this study, it is shown that the unboxing of how AI systems' black-box choices is made by generating heat maps using the local interpretable model-agnostic explanation method, which is an explainable artificial intelligence (XAI) technique. Heat maps, in particular, may contribute to the development of more stable deep learning-based systems, as well as enhancing the confidence in the diagnosis of retinal disease in the analysis of OCT image for ophthalmologists.

An interpretable transformer network for the retinal disease classification using optical coherence tomography

Retinal illnesses such as age-related macular degeneration and diabetic macular edema will lead to irreversible blindness. With optical coherence tomography (OCT), doctors are able to see cross-sections of the retinal layers and provide patients with a diagnosis. Manual reading of OCT images is time-consuming, labor-intensive and even error-prone. Computer-aided diagnosis algorithms improve efficiency by automatically analyzing and diagnosing retinal OCT images. However, the accuracy and interpretability of these algorithms can be further improved through effective feature extraction, loss optimization and visualization analysis. In this paper, we propose an interpretable Swin-Poly Transformer network for performing automatically retinal OCT image classification. By shifting the window partition, the Swin-Poly Transformer constructs connections between neighboring non-overlapping windows in the previous layer and thus has the flexibility to model multi-scale features. Besides, the Swin-Poly Transformer modifies the importance of polynomial bases to refine cross entropy for better retinal OCT image classification. In addition, the proposed method also provides confidence score maps, assisting medical practitioners to understand the models' decision-making process. Experiments in OCT2017 and OCT-C8 reveal that the proposed method outperforms both the convolutional neural network approach and ViT, with an accuracy of 99.80% and an AUC of 99.99%.

Diseased thyroid tissue classification in OCT images using deep learning: Towards surgical decision support

Intraoperative guidance tools for thyroid surgery based on optical coherence tomography (OCT) could aid distinguish between normal and diseased tissue. However, OCT images are difficult to interpret, thus, real-time automatic analysis could support the clinical decision-making. In this study, several deep learning models were investigated for thyroid disease classification on 2D and 3D OCT data obtained from ex vivo specimens of 22 patients undergoing surgery and diagnosed with several thyroid pathologies. Additionally, two open-access datasets were used to evaluate the custom models. On the thyroid dataset, the best performance was achieved by the 3D vision transformer model with a Matthew's correlation coefficient (MCC) of 0.79 (accuracy = 0.90) for the normal-versus-abnormal classification. On the open-access datasets, the custom models achieved the best performance (MCC > 0.88, accuracy > 0.96). Results obtained for the normal-versus-abnormal classification suggest OCT, complemented with deep learning-based analysis, as a tool for real-time automatic diseased tissue identification in thyroid surgery.

Artificial intelligence based detection of age-related macular degeneration using optical coherence tomography with unique image preprocessing

PURPOSE

The aim of the study is to improve the accuracy of age related macular degeneration (AMD) disease in its earlier phases with proposed Capsule Network (CapsNet) architecture trained on speckle noise reduced spectral domain optical coherence tomography (SD-OCT) images based on an optimized Bayesian non-local mean (OBNLM) filter augmentation techniques.

METHODS

A total of 726 local SD-OCT images were collected and labelled as 159 drusen, 145 dry AMD, 156 wet AMD and 266 normal. Region of interest (ROI) was identified. Speckle noise in SD-OCT images were reduced based on OBNLM filter. The processed images were fed to proposed CapsNet architecture to clasify SD-OCT images. Accuracy rates were calculated in both public and local dataset.

RESULTS

Accuracy rate of local SD-OCT image dataset classification was achieved to a value of 96.39% after performing data augmentation and speckle noise reduction with OBNLM. The performance of proposed CapsNet was also evaluated on the public Kaggle dataset under the same processing procedures and the accuracy rate was calculated as 98.07%. The sensitivity and specificity rates were 96.72% and 99.98%, respectively.

CONCLUSIONS

The classification success of proposed CapsNet may be improved with robust pre-processing steps like; determination of ROI and denoised SD-OCT images based on OBNLM. These impactful image preprocessing steps yielded higher accuracy rates for determining different types of AMD including its precursor lesion on the both local and public dataset with proposed CapsNet architecture.

Deep learning algorithms for detection of diabetic macular edema in OCT images: A systematic review and meta-analysis

PURPOSE

Artificial intelligence (AI) can detect diabetic macular edema (DME) from optical coherence tomography (OCT) images. We aimed to evaluate the performance of deep learning neural networks in DME detection.

METHODS

Embase, Pubmed, the Cochrane Library, and IEEE Xplore were searched up to August 14, 2021. We included studies using deep learning algorithms to detect DME from OCT images. Two reviewers extracted the data independently, and the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool was applied to assess the risk of bias. The study is reported according to Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA).

RESULTS

Ninteen studies involving 41005 subjects were included. The pooled sensitivity and specificity were 96.0% (95% confidence interval (CI): 93.9% to 97.3%) and 99.3% (95% CI: 98.2% to 99.7%), respectively. Subgroup analyses found that data set selection, sample size of training set and the choice of OCT devices contributed to the heterogeneity (all P < 0.05). While there was no association between the diagnostic accuracy and transfer learning adoption or image management (all P > 0.05).

CONCLUSIONS

Deep learning methods, particularly the convolutional neural networks (CNNs) could effectively detect clinically significant DME, which can provide referral suggestions to the patients.

Automated artificial intelligence-based system for clinical follow-up of patients with age-related macular degeneration

PURPOSE

In this study, we investigate the potential of a novel artificial intelligence-based system for autonomous follow-up of patients treated for neovascular age-related macular degeneration (AMD).

METHODS

A temporal deep learning model was trained on a data set of 84 489 optical coherence tomography scans from AMD patients to recognize disease activity, and its performance was compared with a published non-temporal model trained on the same data (Acta Ophthalmol, 2021). An autonomous follow-up system was created by augmenting the AI model with deterministic logic to suggest treatment according to the observe-and-plan regimen. To validate the AI-based system, a data set comprising clinical decisions and imaging data from 200 follow-up consultations was collected prospectively. In each case, both the autonomous AI decision and original clinical decision were compared with an expert panel consensus.

RESULTS

The temporal AI model proved superior at detecting disease activity compared with the model without temporal input (area under the curve 0.900 (95% CI 0.894-0.906) and 0.857 (95% CI 0.846-0.867) respectively). The AI-based follow-up system could make an autonomous decision in 73% of the cases, 91.8% of which were in agreement with expert consensus. This was on par with the 87.7% agreement rate between decisions made in the clinic and expert consensus (p = 0.33).

CONCLUSIONS

The proposed autonomous follow-up system was shown to be safe and compliant with expert consensus on par with clinical practice. The system could in the future ease the pressure on public ophthalmology services from an increasing number of AMD patients.

Generative adversarial network-based deep learning approach in classification of retinal conditions with optical coherence tomography images

PURPOSE

To determine whether a deep learning approach using generative adversarial networks (GANs) is beneficial for the classification of retinal conditions with Optical coherence tomography (OCT) images.

METHODS

Our study utilized 84,452 retinal OCT images obtained from a publicly available dataset (Kermany Dataset). Employing GAN, synthetic OCT images are produced to balance classes of retinal disorders. A deep learning classification model is constructed using pretrained deep neural networks (DNNs), and outcomes are evaluated using 2082 images collected from patients who visited the Department of Ophthalmology and the Department of Endocrinology and Metabolism at the Tri-service General Hospital in Taipei from January 2017 to December 2021.

RESULTS

The highest classification accuracies accomplished by deep learning machines trained on the unbalanced dataset for its training set, validation set, fivefold cross validation (CV), Kermany test set, and TSGH test set were 97.73%, 96.51%, 97.14%, 99.59%, and 81.03%, respectively. The highest classification accuracies accomplished by deep learning machines trained on the synthesis-balanced dataset for its training set, validation set, fivefold CV, Kermany test set, and TSGH test set were 98.60%, 98.41%, 98.52%, 99.38%, and 84.92%, respectively. In comparing the highest accuracies, deep learning machines trained on the synthesis-balanced dataset outperformed deep learning machines trained on the unbalanced dataset for the training set, validation set, fivefold CV, and TSGH test set.

CONCLUSIONS

Overall, deep learning machines on a synthesis-balanced dataset demonstrated to be advantageous over deep learning machines trained on an unbalanced dataset for the classification of retinal conditions.

Inflation of test accuracy due to data leakage in deep learning-based classification of OCT images

In the application of deep learning on optical coherence tomography (OCT) data, it is common to train classification networks using 2D images originating from volumetric data. Given the micrometer resolution of OCT systems, consecutive images are often very similar in both visible structures and noise. Thus, an inappropriate data split can result in overlap between the training and testing sets, with a large portion of the literature overlooking this aspect. In this study, the effect of improper dataset splitting on model evaluation is demonstrated for three classification tasks using three OCT open-access datasets extensively used, Kermany's and Srinivasan's ophthalmology datasets, and AIIMS breast tissue dataset. Results show that the classification performance is inflated by 0.07 up to 0.43 in terms of Matthews Correlation Coefficient (accuracy: 5% to 30%) for models tested on datasets with improper splitting, highlighting the considerable effect of dataset handling on model evaluation. This study intends to raise awareness on the importance of dataset splitting given the increased research interest in implementing deep learning on OCT data.

Etiology of Macular Edema Defined by Deep Learning in Optical Coherence Tomography Scans

Purpose

To develop an automated method based on deep learning (DL) to classify macular edema (ME) from the evaluation of optical coherence tomography (OCT) scans.

Methods

A total of 4230 images were obtained from data repositories of patients attended in an ophthalmology clinic in Colombia and two free open-access databases. They were annotated with four biomarkers (BMs) as intraretinal fluid, subretinal fluid, hyperreflective foci/tissue, and drusen. Then the scans were labeled as control or ocular disease among diabetic macular edema (DME), neovascular age-related macular degeneration (nAMD), and retinal vein occlusion (RVO) by two expert ophthalmologists. Our method was developed by following four consecutive phases: segmentation of BMs, the combination of BMs, feature extraction with convolutional neural networks to achieve binary classification for each disease, and, finally, multiclass classification of diseases and control images.

Results

The accuracy of our model for nAMD was 97%, and for DME, RVO, and control were 94%, 93%, and 93%, respectively. Area under curve values were 0.99, 0.98, 0.96, and 0.97, respectively. The mean Cohen's kappa coefficient for the multiclass classification task was 0.84.

Conclusions

The proposed DL model may identify OCT scans as normal and ME. In addition, it may classify its cause among three major exudative retinal diseases with high accuracy and reliability.

Translational Relevance

Our DL approach can optimize the efficiency and timeliness of appropriate etiological diagnosis of ME, thus improving patient access and clinical decision making. It could be useful in places with a shortage of specialists and for readers that evaluate OCT scans remotely.

Automatic detection of microaneurysms in optical coherence tomography images of retina using convolutional neural networks and transfer learning

Microaneurysms (MAs) are pathognomonic signs that help clinicians to detect diabetic retinopathy (DR) in the early stages. Automatic detection of MA in retinal images is an active area of research due to its application in screening processes for DR which is one of the main reasons of blindness amongst the working-age population. The focus of these works is on the automatic detection of MAs in en face retinal images like fundus color and Fluorescein Angiography (FA). On the other hand, detection of MAs from Optical Coherence Tomography (OCT) images has 2 main advantages: first, OCT is a non-invasive imaging technique that does not require injection, therefore is safer. Secondly, because of the proven application of OCT in detection of Age-Related Macular Degeneration, Diabetic Macular Edema, and normal cases, thanks to detecting MAs in OCT, extensive information is obtained by using this imaging technique. In this research, the concentration is on the diagnosis of MAs using deep learning in the OCT images which represent in-depth structure of retinal layers. To this end, OCT B-scans should be divided into strips and MA patterns should be searched in the resulted strips. Since we need a dataset comprising OCT image strips with suitable labels and such large labelled datasets are not yet available, we have created it. For this purpose, an exact registration method is utilized to align OCT images with FA photographs. Then, with the help of corresponding FA images, OCT image strips are created from OCT B-scans in four labels, namely MA, normal, abnormal, and vessel. Once the dataset of image strips is prepared, a stacked generalization (stacking) ensemble of four fine-tuned, pre-trained convolutional neural networks is trained to classify the strips of OCT images into the mentioned classes. FA images are used once to create OCT strips for training process and they are no longer needed for subsequent steps. Once the stacking ensemble model is obtained, it will be used to classify the OCT strips in the test process. The results demonstrate that the proposed framework classifies overall OCT image strips and OCT strips containing MAs with accuracy scores of 0.982 and 0.987, respectively.

HCTNet: A Hybrid ConvNet-Transformer Network for Retinal Optical Coherence Tomography Image Classification

Automatic and accurate optical coherence tomography (OCT) image classification is of great significance to computer-assisted diagnosis of retinal disease. In this study, we propose a hybrid ConvNet-Transformer network (HCTNet) and verify the feasibility of a Transformer-based method for retinal OCT image classification. The HCTNet first utilizes a low-level feature extraction module based on the residual dense block to generate low-level features for facilitating the network training. Then, two parallel branches of the Transformer and the ConvNet are designed to exploit the global and local context of the OCT images. Finally, a feature fusion module based on an adaptive re-weighting mechanism is employed to combine the extracted global and local features for predicting the category of OCT images in the testing datasets. The HCTNet combines the advantage of the convolutional neural network in extracting local features and the advantage of the vision Transformer in establishing long-range dependencies. A verification on two public retinal OCT datasets shows that our HCTNet method achieves an overall accuracy of 91.56% and 86.18%, respectively, outperforming the pure ViT and several ConvNet-based classification methods.

Fast and Efficient Method for Optical Coherence Tomography Images Classification Using Deep Learning Approach

The use of optical coherence tomography (OCT) in medical diagnostics is now common. The growing amount of data leads us to propose an automated support system for medical staff. The key part of the system is a classification algorithm developed with modern machine learning techniques. The main contribution is to present a new approach for the classification of eye diseases using the convolutional neural network model. The research concerns the classification of patients on the basis of OCT B-scans into one of four categories: Diabetic Macular Edema (DME), Choroidal Neovascularization (CNV), Drusen, and Normal. Those categories are available in a publicly available dataset of above 84,000 images utilized for the research. After several tested architectures, our 5-layer neural network gives us a promising result. We compared them to the other available solutions which proves the high quality of our algorithm. Equally important for the application of the algorithm is the computational time, which is reduced by the limited size of the model. In addition, the article presents a detailed method of image data augmentation and its impact on the classification results. The results of the experiments were also presented for several derived models of convolutional network architectures that were tested during the research. Improving processes in medical treatment is important. The algorithm cannot replace a doctor but, for example, can be a valuable tool for speeding up the process of diagnosis during screening tests.

SISE-PC: Semi-supervised Image Subsampling for Explainable Pathology Classification

Although automated pathology classification using deep learning (DL) has proved to be predictively efficient, DL methods are found to be data and compute cost intensive. In this work, we aim to reduce DL training costs by pre-training a ResNet feature extractor using SimCLR contrastive loss for latent encoding of OCT images. We propose a novel active learning framework that identifies a minimal sub-sampled dataset containing the most uncertain OCT image samples using label propagation on the SimCLR latent encodings. The pre-trained ResNet model is then fine-tuned with the labelled minimal sub-sampled data and the underlying pathological sites are visually explained. Our framework identifies upto 2% of OCT images to be most uncertain that need prioritized specialist attention and that can fine-tune a ResNet model to achieve upto 97% classification accuracy. The proposed method can be extended to other medical images to minimize prediction costs.

Narrative review of artificial intelligence in diabetic macular edema: Diagnosis and predicting treatment response using optical coherence tomography

Diabetic macular edema (DME), being a frequent manifestation of DR, disrupts the retinal symmetry. This event is particularly triggered by vascular endothelial growth factors (VEGF). Intravitreal injections of anti-VEGFs have been the most practiced treatment but an expensive option. A major challenge associated with this treatment is determining an optimal treatment regimen and differentiating patients who do not respond to anti-VEGF. As it has a significant burden for both the patient and the health care providers if the patient is not responding, any clinically acceptable method to predict the treatment outcomes holds huge value in the efficient management of DME. In such situations, artificial intelligence (AI) or machine learning (ML)-based algorithms come useful as they can analyze past clinical details of the patients and help clinicians to predict the patient's response to an anti-VEGF agent. The work presented here attempts to review the literature that is available from the peer research community to discuss solutions provided by AI/ML methodologies to tackle challenges in DME management. Lastly, a possibility for using two different types of data has been proposed, which is believed to be the key differentiators as compared to the similar and recent contributions from the peer research community.

A Multitask Deep-Learning System to Classify Diabetic Macular Edema for Different Optical Coherence Tomography Devices: A Multicenter Analysis

OBJECTIVE

Diabetic macular edema (DME) is the primary cause of vision loss among individuals with diabetes mellitus (DM). We developed, validated, and tested a deep learning (DL) system for classifying DME using images from three common commercially available optical coherence tomography (OCT) devices.

RESEARCH DESIGN AND METHODS

We trained and validated two versions of a multitask convolution neural network (CNN) to classify DME (center-involved DME [CI-DME], non-CI-DME, or absence of DME) using three-dimensional (3D) volume scans and 2D B-scans, respectively. For both 3D and 2D CNNs, we used the residual network (ResNet) as the backbone. For the 3D CNN, we used a 3D version of ResNet-34 with the last fully connected layer removed as the feature extraction module. A total of 73,746 OCT images were used for training and primary validation. External testing was performed using 26,981 images across seven independent data sets from Singapore, Hong Kong, the U.S., China, and Australia.

RESULTS

In classifying the presence or absence of DME, the DL system achieved area under the receiver operating characteristic curves (AUROCs) of 0.937 (95% CI 0.920-0.954), 0.958 (0.930-0.977), and 0.965 (0.948-0.977) for the primary data set obtained from CIRRUS, SPECTRALIS, and Triton OCTs, respectively, in addition to AUROCs >0.906 for the external data sets. For further classification of the CI-DME and non-CI-DME subgroups, the AUROCs were 0.968 (0.940-0.995), 0.951 (0.898-0.982), and 0.975 (0.947-0.991) for the primary data set and >0.894 for the external data sets.

CONCLUSIONS

We demonstrated excellent performance with a DL system for the automated classification of DME, highlighting its potential as a promising second-line screening tool for patients with DM, which may potentially create a more effective triaging mechanism to eye clinics.

Deep Learning-Based Optical Coherence Tomography and Optical Coherence Tomography Angiography Image Analysis: An Updated Summary

ABSTRACT

Deep learning (DL) is a subset of artificial intelligence based on deep neural networks. It has made remarkable breakthroughs in medical imaging, particularly for image classification and pattern recognition. In ophthalmology, there are rising interests in applying DL methods to analyze optical coherence tomography (OCT) and optical coherence tomography angiography (OCTA) images. Studies showed that OCT and OCTA image evaluation by DL algorithms achieved good performance for disease detection, prognosis prediction, and image quality control, suggesting that the incorporation of DL technology could potentially enhance the accuracy of disease evaluation and the efficiency of clinical workflow. However, substantial issues, such as small training sample size, data preprocessing standardization, model robustness, results explanation, and performance cross-validation, are yet to be tackled before deploying these DL models in real-time clinics. This review summarized recent studies on DL-based image analysis models for OCT and OCTA images and discussed the potential challenges of clinical deployment and future research directions.