Macular OCT Classification Using a Multi-Scale Convolutional Neural Network Ensemble

Integrating lightweight convolutional neural network with entropy-informed channel attention and adaptive spatial attention for OCT-based retinal disease classification

This article proposes an effective and lightweight contextual convolutional neural network architecture called LOCT-Net for classifying retinal diseases. The LOCT-Net adopts nested residual blocks to capture the local patterns from the optical coherence tomography brightness scans and facilitate gradient flow throughout the network. The multi-scale feature enhancement module incorporates dilation-integrated depthwise strip convolutions to extract fine-grained contextual patterns with an expanded receptive field and a gating mechanism. The extracted features are then refined by a novel feature refinement network consisting of the entropy-informed channel attention module, followed by the adaptive spatial attention module. The entropy-informed channel attention module uses the frequency distribution of pixel values to compute attention weights for spatial analysis. The adaptive spatial attention module focuses on relevant clinical regions and further refines the feature maps in a multi-kernel setting. Additionally, post-explainable artificial intelligence methods are used to provide explanations of LOCT-Net's decision-making and predictions. The LOCT-Net model has been evaluated on six benchmark datasets, demonstrating an efficient balance between performance and computational cost. With just 2.32 M trainable parameters, the proposed model addresses key challenges in retinal disease classification tasks using OCT B-scans and surpasses previous state-of-the-art methods, achieving an F1 score of 92.98%, 92.34%, 100%, 99.58%, 94.50%, and 97.14% in the OCTID, OCTDL, DUKE, SD-OCT Noor, NEH, and UCSD datasets, respectively.

Multidimensional Directionality-Enhanced Segmentation via large vision model

Optical Coherence Tomography (OCT) facilitates a comprehensive examination of macular edema and associated lesions. Manual delineation of retinal fluid is labor-intensive and error-prone, necessitating an automated diagnostic and therapeutic planning mechanism. Conventional supervised learning models are hindered by dataset limitations, while Transformer-based large vision models exhibit challenges in medical image segmentation, particularly in detecting small, subtle lesions in OCT images. This paper introduces the Multidimensional Directionality-Enhanced Retinal Fluid Segmentation framework (MD-DERFS), which reduces the limitations inherent in conventional supervised models by adapting a transformer-based large vision model for macular edema segmentation. The proposed MD-DERFS introduces a Multi-Dimensional Feature Re-Encoder Unit (MFU) to augment the model's proficiency in recognizing specific textures and pathological features through directional prior extraction and an Edema Texture Mapping Unit (ETMU), a Cross-scale Directional Insight Network (CDIN) furnishes a holistic perspective spanning local to global details, mitigating the large vision model's deficiencies in capturing localized feature information. Additionally, the framework is augmented by a Harmonic Minutiae Segmentation Equilibrium loss (LHMSE) that can address the challenges of data imbalance and annotation scarcity in macular edema datasets. Empirical validation on the MacuScan-8k dataset shows that MD-DERFS surpasses existing segmentation methodologies, demonstrating its efficacy in adapting large vision models for boundary-sensitive medical imaging tasks. The code is publicly available at https://github.com/IMOP-lab/MD-DERFS-Pytorch.git.

Publicly available imaging datasets for age-related macular degeneration: Evaluation according to the Findable, Accessible, Interoperable, Reusable (FAIR) principles

Age-related macular degeneration (AMD), a leading cause of vision loss among older adults, affecting more than 200 million people worldwide. With no cure currently available and a rapidly increasing prevalence, emerging approaches such as artificial intelligence (AI) and machine learning (ML) hold promise for advancing the study of AMD. The effective utilization of AI and ML in AMD research is highly dependent on access to high-quality and reusable clinical data. The Findable, Accessible, Interoperable, Reusable (FAIR) principles, published in 2016, provide a framework for sharing data that is easily useable by both humans and machines. However, it is unclear how these principles are implemented with regards to ophthalmic imaging datasets for AMD research. We evaluated openly available AMD-related datasets containing optical coherence tomography (OCT) data against the FAIR principles. The assessment revealed that none of the datasets were fully compliant with FAIR principles. Specifically, compliance rates were 5 % for Findable, 82 % for Accessible, 73 % for Interoperable, and 0 % for Reusable. The low compliance rates can be attributed to the relatively recent emergence of these principles and the lack of established standards for data and metadata formatting in the AMD research community. This article presents our findings and offers guidelines for adopting FAIR practices to enhance data sharing in AMD research.

OCT5k: A dataset of multi-disease and multi-graded annotations for retinal layers

Publicly available open-access OCT datasets for retinal layer segmentation have been limited in scope, often being small in size, specific to a single disease, or containing only one grading. This dataset improves upon this with multi-grader and multi-disease labels for training machine learning-based algorithms. The proposed dataset covers three subsets of scans (Age-related Macular Degeneration, Diabetic Macular Edema, and healthy) and annotations for two types of tasks (semantic segmentation and object detection). This dataset compiled 5016 pixel-wise manual labels for 1672 OCT scans featuring 5 layer boundaries for three different disease classes to support development of automatic techniques. A subset of data (566 scans across 9 classes of disease biomarkers) was subsequently labeled for disease features for 4698 bounding box annotations. To minimize bias, images were shuffled and distributed among graders. Retinal layers were corrected, and outliers identified using the interquartile range (IQR). This step was iterated three times, improving layer annotations' quality iteratively, ensuring a reliable dataset for automated retinal image analysis.

CircWaveDL: Modeling of optical coherence tomography images based on a new supervised tensor-based dictionary learning for classification of macular abnormalities

Modeling Optical Coherence Tomography (OCT) images is crucial for numerous image processing applications and aids ophthalmologists in the early detection of macular abnormalities. Sparse representation-based models, particularly dictionary learning (DL), play a pivotal role in image modeling. Traditional DL methods often transform higher-order tensors into vectors and then aggregate them into a matrix, which overlooks the inherent multi-dimensional structure of the data. To address this limitation, tensor-based DL approaches have been introduced. In this study, we present a novel tensor-based DL algorithm, CircWaveDL, for OCT classification, where both the training data and the dictionary are modeled as higher-order tensors. We named our approach CircWaveDL to reflect the use of CircWave atoms for dictionary initialization, rather than random initialization. CircWave has previously shown effectiveness in OCT classification, making it a fitting basis function for our DL method. The algorithm employs CANDECOMP/PARAFAC (CP) decomposition to factorize each tensor into lower dimensions. We then learn a sub-dictionary for each class using its respective training tensor. For testing, a test tensor is reconstructed with each sub-dictionary, and each test B-scan is assigned to the class that yields the minimal residual error. To evaluate the model's generalizability, we tested it across three distinct databases. Additionally, we introduce a new heatmap generation technique based on averaging the most significant atoms of the learned sub-dictionaries. This approach highlights that selecting an appropriate sub-dictionary for reconstructing test B-scans improves reconstructions, emphasizing the distinctive features of different classes. CircWaveDL demonstrated strong generalizability across external validation datasets, outperforming previous classification methods. It achieved accuracies of 92.5 %, 86.1 %, and 89.3 % on datasets 1, 2, and 3, respectively, showcasing its efficacy in OCT image classification.

A Vision Language Correlation Framework for Screening Disabled Retina

Retinopathy is a group of retinal disabilities that causes severe visual impairments or complete blindness. Due to the capability of optical coherence tomography to reveal early retinal abnormalities, many researchers have utilized it to develop autonomous retinal screening systems. However, to the best of our knowledge, most of these systems rely only on mathematical features, which might not be helpful to clinicians since they do not encompass the clinical manifestations of screening the underlying diseases. Such clinical manifestations are critically important to be considered within the autonomous screening systems to match the grading of ophthalmologists within the clinical settings. To overcome these limitations, we present a novel framework that exploits the fusion of vision language correlation between the retinal imagery and the set of clinical prompts to recognize the different types of retinal disabilities. The proposed framework is rigorously tested on six public datasets, where, across each dataset, the proposed framework outperformed state-of-the-art methods in various metrics. Moreover, the clinical significance of the proposed framework is also tested under strict blind testing experiments, where the proposed system achieved a statistically significant correlation coefficient of 0.9185 and 0.9529 with the two expert clinicians. These blind test experiments highlight the potential of the proposed framework to be deployed in the real world for accurate screening of retinal diseases.

Retinal OCT image classification based on MGR-GAN

Accurately classifying optical coherence tomography (OCT) images is essential for diagnosing and treating ophthalmic diseases. This paper introduces a novel generative adversarial network framework called MGR-GAN. The masked image modeling (MIM) method is integrated into the GAN model's generator, enhancing its ability to synthesize more realistic images by reconstructing them based on unmasked patches. A ResNet-structured discriminator is employed to determine whether the image is generated by the generator. Through the unique game process of the generative adversarial network (GAN) model, the discriminator acquires high-level discriminant features, essential for precise OCT classification. Experimental results demonstrate that MGR-GAN achieves a classification accuracy of 98.4% on the original UCSD dataset. As the trained generator can synthesize OCT images with higher precision, and owing to category imbalances in the UCSD dataset, the generated OCT images are leveraged to address this imbalance. After balancing the UCSD dataset, the classification accuracy further improves to 99%.

A Comprehensive CNN Model for Age-Related Macular Degeneration Classification Using OCT: Integrating Inception Modules, SE Blocks, and ConvMixer

Background/Objectives: Age-related macular degeneration (AMD) is a significant cause of vision loss in older adults, often progressing without early noticeable symptoms. Deep learning (DL) models, particularly convolutional neural networks (CNNs), demonstrate potential in accurately diagnosing and classifying AMD using medical imaging technologies like optical coherence to-mography (OCT) scans. This study introduces a novel CNN-based DL method for AMD diagnosis, aiming to enhance computational efficiency and classification accuracy. Methods: The proposed method (PM) combines modified Inception modules, Depthwise Squeeze-and-Excitation Blocks, and ConvMixer architecture. Its effectiveness was evaluated on two datasets: a private dataset with 2316 images and the public Noor dataset. Key performance metrics, including accuracy, precision, recall, and F1 score, were calculated to assess the method's diagnostic performance. Results: On the private dataset, the PM achieved outstanding performance: 97.98% accuracy, 97.95% precision, 97.77% recall, and 97.86% F1 score. When tested on the public Noor dataset, the method reached 100% across all evaluation metrics, outperforming existing DL approaches. Conclusions: These results highlight the promising role of AI-based systems in AMD diagnosis, of-fering advanced feature extraction capabilities that can potentially enable early detection and in-tervention, ultimately improving patient care and outcomes. While the proposed model demon-strates promising performance on the datasets tested, the study is limited by the size and diversity of the datasets. Future work will focus on external clinical validation to address these limita-tions.

Multiscale attention-over-attention network for retinal disease recognition in OCT radiology images

Retinal disease recognition using Optical Coherence Tomography (OCT) images plays a pivotal role in the early diagnosis and treatment of conditions. However, the previous attempts relied on extracting single-scale features often refined by stacked layered attentions. This paper presents a novel deep learning-based Multiscale Feature Enhancement via a Dual Attention Network specifically designed for retinal disease recognition in OCT images. Our approach leverages the EfficientNetB7 backbone to extract multiscale features from OCT images, ensuring a comprehensive representation of global and local retinal structures. To further refine feature extraction, we propose a Pyramidal Attention mechanism that integrates Multi-Head Self-Attention (MHSA) with Dense Atrous Spatial Pyramid Pooling (DASPP), effectively capturing long-range dependencies and contextual information at multiple scales. Additionally, Efficient Channel Attention (ECA) and Spatial Refinement modules are introduced to enhance channel-wise and spatial feature representations, enabling precise localization of retinal abnormalities. A comprehensive ablation study confirms the progressive impact of integrated blocks and attention mechanisms that enhance overall performance. Our findings underscore the potential of advanced attention mechanisms and multiscale processing, highlighting the effectiveness of the network. Extensive experiments on two benchmark datasets demonstrate the superiority of the proposed network over existing state-of-the-art methods.

Uncertainty-aware multiple-instance learning for reliable classification: Application to optical coherence tomography

Deep learning classification models for medical image analysis often perform well on data from scanners that were used to acquire the training data. However, when these models are applied to data from different vendors, their performance tends to drop substantially. Artifacts that only occur within scans from specific scanners are major causes of this poor generalizability. We aimed to enhance the reliability of deep learning classification models using a novel method called Uncertainty-Based Instance eXclusion (UBIX). UBIX is an inference-time module that can be employed in multiple-instance learning (MIL) settings. MIL is a paradigm in which instances (generally crops or slices) of a bag (generally an image) contribute towards a bag-level output. Instead of assuming equal contribution of all instances to the bag-level output, UBIX detects instances corrupted due to local artifacts on-the-fly using uncertainty estimation, reducing or fully ignoring their contributions before MIL pooling. In our experiments, instances are 2D slices and bags are volumetric images, but alternative definitions are also possible. Although UBIX is generally applicable to diverse classification tasks, we focused on the staging of age-related macular degeneration in optical coherence tomography. Our models were trained on data from a single scanner and tested on external datasets from different vendors, which included vendor-specific artifacts. UBIX showed reliable behavior, with a slight decrease in performance (a decrease of the quadratic weighted kappa (κw) from 0.861 to 0.708), when applied to images from different vendors containing artifacts; while a state-of-the-art 3D neural network without UBIX suffered from a significant detriment of performance (κw from 0.852 to 0.084) on the same test set. We showed that instances with unseen artifacts can be identified with OOD detection. UBIX can reduce their contribution to the bag-level predictions, improving reliability without retraining on new data. This potentially increases the applicability of artificial intelligence models to data from other scanners than the ones for which they were developed. The source code for UBIX, including trained model weights, is publicly available through https://github.com/qurAI-amsterdam/ubix-for-reliable-classification.

Artificial intelligence in retinal screening using OCT images: A review of the last decade (2013-2023)

BACKGROUND AND OBJECTIVES

Optical coherence tomography (OCT) has ushered in a transformative era in the domain of ophthalmology, offering non-invasive imaging with high resolution for ocular disease detection. OCT, which is frequently used in diagnosing fundamental ocular pathologies, such as glaucoma and age-related macular degeneration (AMD), plays an important role in the widespread adoption of this technology. Apart from glaucoma and AMD, we will also investigate pertinent pathologies, such as epiretinal membrane (ERM), macular hole (MH), macular dystrophy (MD), vitreomacular traction (VMT), diabetic maculopathy (DMP), cystoid macular edema (CME), central serous chorioretinopathy (CSC), diabetic macular edema (DME), diabetic retinopathy (DR), drusen, glaucomatous optic neuropathy (GON), neovascular AMD (nAMD), myopia macular degeneration (MMD) and choroidal neovascularization (CNV) diseases. This comprehensive review examines the role that OCT-derived images play in detecting, characterizing, and monitoring eye diseases.

METHOD

The 2020 PRISMA guideline was used to structure a systematic review of research on various eye conditions using machine learning (ML) or deep learning (DL) techniques. A thorough search across IEEE, PubMed, Web of Science, and Scopus databases yielded 1787 publications, of which 1136 remained after removing duplicates. Subsequent exclusion of conference papers, review papers, and non-open-access articles reduced the selection to 511 articles. Further scrutiny led to the exclusion of 435 more articles due to lower-quality indexing or irrelevance, resulting in 76 journal articles for the review.

RESULTS

During our investigation, we found that a major challenge for ML-based decision support is the abundance of features and the determination of their significance. In contrast, DL-based decision support is characterized by a plug-and-play nature rather than relying on a trial-and-error approach. Furthermore, we observed that pre-trained networks are practical and especially useful when working on complex images such as OCT. Consequently, pre-trained deep networks were frequently utilized for classification tasks. Currently, medical decision support aims to reduce the workload of ophthalmologists and retina specialists during routine tasks. In the future, it might be possible to create continuous learning systems that can predict ocular pathologies by identifying subtle changes in OCT images.

A novel approach for automatic classification of macular degeneration OCT images

Age-related macular degeneration (AMD) and diabetic macular edema (DME) are significant causes of blindness worldwide. The prevalence of these diseases is steadily increasing due to population aging. Therefore, early diagnosis and prevention are crucial for effective treatment. Classification of Macular Degeneration OCT Images is a widely used method for assessing retinal lesions. However, there are two main challenges in OCT image classification: incomplete image feature extraction and lack of prominence in important positional features. To address these challenges, we proposed a deep learning neural network model called MSA-Net, which incorporates our proposed multi-scale architecture and spatial attention mechanism. Our multi-scale architecture is based on depthwise separable convolution, which ensures comprehensive feature extraction from multiple scales while minimizing the growth of model parameters. The spatial attention mechanism is aim to highlight the important positional features in the images, which emphasizes the representation of macular region features in OCT images. We test MSA-NET on the NEH dataset and the UCSD dataset, performing three-class (CNV, DURSEN, and NORMAL) and four-class (CNV, DURSEN, DME, and NORMAL) classification tasks. On the NEH dataset, the accuracy, sensitivity, and specificity are 98.1%, 97.9%, and 98.0%, respectively. After fine-tuning on the UCSD dataset, the accuracy, sensitivity, and specificity are 96.7%, 96.7%, and 98.9%, respectively. Experimental results demonstrate the excellent classification performance and generalization ability of our model compared to previous models and recent well-known OCT classification models, establishing it as a highly competitive intelligence classification approach in the field of macular degeneration.

A Comprehensive Review of AI Diagnosis Strategies for Age-Related Macular Degeneration (AMD)

The rapid advancement of computational infrastructure has led to unprecedented growth in machine learning, deep learning, and computer vision, fundamentally transforming the analysis of retinal images. By utilizing a wide array of visual cues extracted from retinal fundus images, sophisticated artificial intelligence models have been developed to diagnose various retinal disorders. This paper concentrates on the detection of Age-Related Macular Degeneration (AMD), a significant retinal condition, by offering an exhaustive examination of recent machine learning and deep learning methodologies. Additionally, it discusses potential obstacles and constraints associated with implementing this technology in the field of ophthalmology. Through a systematic review, this research aims to assess the efficacy of machine learning and deep learning techniques in discerning AMD from different modalities as they have shown promise in the field of AMD and retinal disorders diagnosis. Organized around prevalent datasets and imaging techniques, the paper initially outlines assessment criteria, image preprocessing methodologies, and learning frameworks before conducting a thorough investigation of diverse approaches for AMD detection. Drawing insights from the analysis of more than 30 selected studies, the conclusion underscores current research trajectories, major challenges, and future prospects in AMD diagnosis, providing a valuable resource for both scholars and practitioners in the domain.

Stitched vision transformer for age-related macular degeneration detection using retinal optical coherence tomography images

Age-related macular degeneration (AMD) is an eye disease that leads to the deterioration of the central vision area of the eye and can gradually result in vision loss in elderly individuals. Early identification of this disease can significantly impact patient treatment outcomes. Furthermore, given the increasing elderly population globally, the importance of automated methods for rapidly monitoring at-risk individuals and accurately diagnosing AMD is growing daily. One standard method for diagnosing AMD is using optical coherence tomography (OCT) images as a non-invasive imaging technology. In recent years, numerous deep neural networks have been proposed for the classification of OCT images. Utilizing pre-trained neural networks can speed up model deployment in related tasks without compromising accuracy. However, most previous methods overlook the feasibility of leveraging pre-existing trained networks to search for an optimal architecture for AMD staging on a new target dataset. In this study, our objective was to achieve an optimal architecture in the efficiency-accuracy trade-off for classifying retinal OCT images. To this end, we employed pre-trained medical vision transformer (MedViT) models. MedViT combines convolutional and transformer neural networks, explicitly designed for medical image classification. Our approach involved pre-training two distinct MedViT models on a source dataset with labels identical to those in the target dataset. This pre-training was conducted in a supervised manner. Subsequently, we evaluated the performance of the pre-trained MedViT models for classifying retinal OCT images from the target Noor Eye Hospital (NEH) dataset into the normal, drusen, and choroidal neovascularization (CNV) classes in zero-shot settings and through five-fold cross-validation. Then, we proposed a stitching approach to search for an optimal model from two MedViT family models. The proposed stitching method is an efficient architecture search algorithm known as stitchable neural networks. Stitchable neural networks create a candidate model in search space for each pair of stitchable layers by inserting a linear layer between them. A pair of stitchable layers consists of layers, each selected from one input model. While stitchable neural networks had previously been tested on more extensive and general datasets, this study demonstrated that stitching networks could also be helpful in smaller medical datasets. The results of this approach indicate that when pre-trained models were available for OCT images from another dataset, it was possible to achieve a model in 100 epochs with an accuracy of over 94.9% in classifying images from the NEH dataset. The results of this study demonstrate the efficacy of stitchable neural networks as a fine-tuning method for OCT image classification. This approach not only leads to higher accuracy but also considers architecture optimization at a reasonable computational cost.

Multibranch CNN With MLP-Mixer-Based Feature Exploration for High-Performance Disease Diagnosis

Deep learning-based diagnosis is becoming an indispensable part of modern healthcare. For high-performance diagnosis, the optimal design of deep neural networks (DNNs) is a prerequisite. Despite its success in image analysis, existing supervised DNNs based on convolutional layers often suffer from their rudimentary feature exploration ability caused by the limited receptive field and biased feature extraction of conventional convolutional neural networks (CNNs), which compromises the network performance. Here, we propose a novel feature exploration network named manifold embedded multilayer perceptron (MLP) mixer (ME-Mixer), which utilizes both supervised and unsupervised features for disease diagnosis. In the proposed approach, a manifold embedding network is employed to extract class-discriminative features; then, two MLP-Mixer-based feature projectors are adopted to encode the extracted features with the global reception field. Our ME-Mixer network is quite general and can be added as a plugin to any existing CNN. Comprehensive evaluations on two medical datasets are performed. The results demonstrate that their approach greatly enhances the classification accuracy in comparison with different configurations of DNNs with acceptable computational complexity.

Multi-Scale-Denoising Residual Convolutional Network for Retinal Disease Classification Using OCT

Macular pathologies can cause significant vision loss. Optical coherence tomography (OCT) images of the retina can assist ophthalmologists in diagnosing macular diseases. Traditional deep learning networks for retinal disease classification cannot extract discriminative features under strong noise conditions in OCT images. To address this issue, we propose a multi-scale-denoising residual convolutional network (MS-DRCN) for classifying retinal diseases. Specifically, the MS-DRCN includes a soft-denoising block (SDB), a multi-scale context block (MCB), and a feature fusion block (FFB). The SDB can determine the threshold for soft thresholding automatically, which removes speckle noise features efficiently. The MCB is designed to capture multi-scale context information and strengthen extracted features. The FFB is dedicated to integrating high-resolution and low-resolution features to precisely identify variable lesion areas. Our approach achieved classification accuracies of 96.4% and 96.5% on the OCT2017 and OCT-C4 public datasets, respectively, outperforming other classification methods. To evaluate the robustness of our method, we introduced Gaussian noise and speckle noise with varying PSNRs into the test set of the OCT2017 dataset. The results of our anti-noise experiments demonstrate that our approach exhibits superior robustness compared with other methods, yielding accuracy improvements ranging from 0.6% to 2.9% when compared with ResNet under various PSNR noise conditions.

A new convolutional neural network based on combination of circlets and wavelets for macular OCT classification

Artificial intelligence (AI) algorithms, encompassing machine learning and deep learning, can assist ophthalmologists in early detection of various ocular abnormalities through the analysis of retinal optical coherence tomography (OCT) images. Despite considerable progress in these algorithms, several limitations persist in medical imaging fields, where a lack of data is a common issue. Accordingly, specific image processing techniques, such as time-frequency transforms, can be employed in conjunction with AI algorithms to enhance diagnostic accuracy. This research investigates the influence of non-data-adaptive time-frequency transforms, specifically X-lets, on the classification of OCT B-scans. For this purpose, each B-scan was transformed using every considered X-let individually, and all the sub-bands were utilized as the input for a designed 2D Convolutional Neural Network (CNN) to extract optimal features, which were subsequently fed to the classifiers. Evaluating per-class accuracy shows that the use of the 2D Discrete Wavelet Transform (2D-DWT) yields superior outcomes for normal cases, whereas the circlet transform outperforms other X-lets for abnormal cases characterized by circles in their retinal structure (due to the accumulation of fluid). As a result, we propose a novel transform named CircWave by concatenating all sub-bands from the 2D-DWT and the circlet transform. The objective is to enhance the per-class accuracy of both normal and abnormal cases simultaneously. Our findings show that classification results based on the CircWave transform outperform those derived from original images or any individual transform. Furthermore, Grad-CAM class activation visualization for B-scans reconstructed from CircWave sub-bands highlights a greater emphasis on circular formations in abnormal cases and straight lines in normal cases, in contrast to the focus on irrelevant regions in original B-scans. To assess the generalizability of our method, we applied it to another dataset obtained from a different imaging system. We achieved promising accuracies of 94.5% and 90% for the first and second datasets, respectively, which are comparable with results from previous studies. The proposed CNN based on CircWave sub-bands (i.e. CircWaveNet) not only produces superior outcomes but also offers more interpretable results with a heightened focus on features crucial for ophthalmologists.

CLAHE-CapsNet: Efficient retina optical coherence tomography classification using capsule networks with contrast limited adaptive histogram equalization

Manual detection of eye diseases using retina Optical Coherence Tomography (OCT) images by Ophthalmologists is time consuming, prone to errors and tedious. Previous researchers have developed a computer aided system using deep learning-based convolutional neural networks (CNNs) to aid in faster detection of the retina diseases. However, these methods find it difficult to achieve better classification performance due to noise in the OCT image. Moreover, the pooling operations in CNN reduce resolution of the image that limits the performance of the model. The contributions of the paper are in two folds. Firstly, this paper makes a comprehensive literature review to establish current-state-of-act methods successfully implemented in retina OCT image classifications. Additionally, this paper proposes a capsule network coupled with contrast limited adaptive histogram equalization (CLAHE-CapsNet) for retina OCT image classification. The CLAHE was implemented as layers to minimize the noise in the retina image for better performance of the model. A three-layer convolutional capsule network was designed with carefully chosen hyperparameters. The dataset used for this study was presented by University of California San Diego (UCSD). The dataset consists of 84,495 X-Ray images (JPEG) and 4 categories (NORMAL, CNV, DME, and DRUSEN). The images went through a grading system consisting of multiple layers of trained graders of expertise for verification and correction of image labels. Evaluation experiments were conducted and comparison of results was done with state-of-the-art models to find out the best performing model. The evaluation metrics; accuracy, sensitivity, precision, specificity, and AUC are used to determine the performance of the models. The evaluation results show that the proposed model achieves the best performing model of accuracies of 97.7%, 99.5%, and 99.3% on overall accuracy (OA), overall sensitivity (OS), and overall precision (OP), respectively. The results obtained indicate that the proposed model can be adopted and implemented to help ophthalmologists in detecting retina OCT diseases.

Automated deep learning-based AMD detection and staging in real-world OCT datasets (PINNACLE study report 5)

Real-world retinal optical coherence tomography (OCT) scans are available in abundance in primary and secondary eye care centres. They contain a wealth of information to be analyzed in retrospective studies. The associated electronic health records alone are often not enough to generate a high-quality dataset for clinical, statistical, and machine learning analysis. We have developed a deep learning-based age-related macular degeneration (AMD) stage classifier, to efficiently identify the first onset of early/intermediate (iAMD), atrophic (GA), and neovascular (nAMD) stage of AMD in retrospective data. We trained a two-stage convolutional neural network to classify macula-centered 3D volumes from Topcon OCT images into 4 classes: Normal, iAMD, GA and nAMD. In the first stage, a 2D ResNet50 is trained to identify the disease categories on the individual OCT B-scans while in the second stage, four smaller models (ResNets) use the concatenated B-scan-wise output from the first stage to classify the entire OCT volume. Classification uncertainty estimates are generated with Monte-Carlo dropout at inference time. The model was trained on a real-world OCT dataset, 3765 scans of 1849 eyes, and extensively evaluated, where it reached an average ROC-AUC of 0.94 in a real-world test set.

Wavelet scattering transform application in classification of retinal abnormalities using OCT images

To assist ophthalmologists in diagnosing retinal abnormalities, Computer Aided Diagnosis has played a significant role. In this paper, a particular Convolutional Neural Network based on Wavelet Scattering Transform (WST) is used to detect one to four retinal abnormalities from Optical Coherence Tomography (OCT) images. Predefined wavelet filters in this network decrease the computation complexity and processing time compared to deep learning methods. We use two layers of the WST network to obtain a direct and efficient model. WST generates a sparse representation of the images which is translation-invariant and stable concerning local deformations. Next, a Principal Component Analysis classifies the extracted features. We evaluate the model using four publicly available datasets to have a comprehensive comparison with the literature. The accuracies of classifying the OCT images of the OCTID dataset into two and five classes were [Formula: see text] and [Formula: see text], respectively. We achieved an accuracy of [Formula: see text] in detecting Diabetic Macular Edema from Normal ones using the TOPCON device-based dataset. Heidelberg and Duke datasets contain DME, Age-related Macular Degeneration, and Normal classes, in which we achieved accuracy of [Formula: see text] and [Formula: see text], respectively. A comparison of our results with the state-of-the-art models shows that our model outperforms these models for some assessments or achieves nearly the best results reported so far while having a much smaller computational complexity.

Comparative analysis of alignment algorithms for macular optical coherence tomography imaging

BACKGROUND

Optical coherence tomography (OCT) is the most important and commonly utilized imaging modality in ophthalmology and is especially crucial for the diagnosis and management of macular diseases. Each OCT volume is typically only available as a series of cross-sectional images (B-scans) that are accessible through proprietary software programs which accompany the OCT machines. To maximize the potential of OCT imaging for machine learning purposes, each OCT image should be analyzed en bloc as a 3D volume, which requires aligning all the cross-sectional images within a particular volume.

METHODS

A dataset of OCT B-scans obtained from 48 age-related macular degeneration (AMD) patients and 50 normal controls was used to evaluate five registration algorithms. After alignment of B-scans from each patient, an en face surface map was created to measure the registration quality, based on an automatically generated Laplace difference of the surface map-the smoother the surface map, the smaller the average Laplace difference. To demonstrate the usefulness of B-scan alignment, we trained a 3D convolutional neural network (CNN) to detect age-related macular degeneration (AMD) on OCT images and compared the performance of the model with and without B-scan alignment.

RESULTS

The mean Laplace difference of the surface map before registration was 27 ± 4.2 pixels for the AMD group and 26.6 ± 4 pixels for the control group. After alignment, the smoothness of the surface map was improved, with a mean Laplace difference of 5.5 ± 2.7 pixels for Advanced Normalization Tools Symmetric image Normalization (ANTs-SyN) registration algorithm in the AMD group and a mean Laplace difference of 4.3 ± 1.4.2 pixels for ANTs in the control group. Our 3D CNN achieved superior performance in detecting AMD, when aligned OCT B-scans were used (AUC 0.95 aligned vs. 0.89 unaligned).

CONCLUSIONS

We introduced a novel metric to quantify OCT B-scan alignment and compared the effectiveness of five alignment algorithms. We confirmed that alignment could be improved in a statistically significant manner with readily available alignment algorithms that are available to the public, and the ANTs algorithm provided the most robust performance overall. We further demonstrated that alignment of OCT B-scans will likely be useful for training 3D CNN models.

Few-shot out-of-distribution detection for automated screening in retinal OCT images using deep learning

Deep neural networks have been increasingly proposed for automated screening and diagnosis of retinal diseases from optical coherence tomography (OCT), but often provide high-confidence predictions on out-of-distribution (OOD) cases, compromising their clinical usage. With this in mind, we performed an in-depth comparative analysis of the state-of-the-art uncertainty estimation methods for OOD detection in retinal OCT imaging. The analysis was performed within the use-case of automated screening and staging of age-related macular degeneration (AMD), one of the leading causes of blindness worldwide, where we achieved a macro-average area under the curve (AUC) of 0.981 for AMD classification. We focus on a few-shot Outlier Exposure (OE) method and the detection of near-OOD cases that share pathomorphological characteristics with the inlier AMD classes. Scoring the OOD case based on the Cosine distance in the feature space from the penultimate network layer proved to be a robust approach for OOD detection, especially in combination with the OE. Using Cosine distance and only 8 outliers exposed per class, we were able to improve the near-OOD detection performance of the OE with Reject Bucket method by [Formula: see text] 10% compared to without OE, reaching an AUC of 0.937. The Cosine distance served as a robust metric for OOD detection of both known and unknown classes and should thus be considered as an alternative to the reject bucket class probability in OE approaches, especially in the few-shot scenario. The inclusion of these methodologies did not come at the expense of classification performance, and can substantially improve the reliability and trustworthiness of the resulting deep learning-based diagnostic systems in the context of retinal OCT.

FIT-Net: Feature Interaction Transformer Network for Pathologic Myopia Diagnosis

Automatic and accurate classification of retinal optical coherence tomography (OCT) images is essential to assist physicians in diagnosing and grading pathological changes in pathologic myopia (PM). Clinically, due to the obvious differences in the position, shape, and size of the lesion structure in different scanning directions, ophthalmologists usually need to combine the lesion structure in the OCT images in the horizontal and vertical scanning directions to diagnose the type of pathological changes in PM. To address these challenges, we propose a novel feature interaction Transformer network (FIT-Net) to diagnose PM using OCT images, which consists of two dual-scale Transformer (DST) blocks and an interactive attention (IA) unit. Specifically, FIT-Net divides image features of different scales into a series of feature block sequences. In order to enrich the feature representation, we propose an IA unit to realize the interactive learning of class token in feature sequences of different scales. The interaction between feature sequences of different scales can effectively integrate different scale image features, and hence FIT-Net can focus on meaningful lesion regions to improve the PM classification performance. Finally, by fusing the dual-view image features in the horizontal and vertical scanning directions, we propose six dual-view feature fusion methods for PM diagnosis. The extensive experimental results based on the clinically obtained datasets and three publicly available datasets demonstrate the effectiveness and superiority of the proposed method. Our code is avaiable at: https://github.com/chenshaobin/FITNet.

Optical Coherence Tomography Image Classification Using Hybrid Deep Learning and Ant Colony Optimization

Optical coherence tomography (OCT) is widely used to detect and classify retinal diseases. However, OCT-image-based manual detection by ophthalmologists is prone to errors and subjectivity. Thus, various automation methods have been proposed; however, improvements in detection accuracy are required. Particularly, automated techniques using deep learning on OCT images are being developed to detect various retinal disorders at an early stage. Here, we propose a deep learning-based automatic method for detecting and classifying retinal diseases using OCT images. The diseases include age-related macular degeneration, branch retinal vein occlusion, central retinal vein occlusion, central serous chorioretinopathy, and diabetic macular edema. The proposed method comprises four main steps: three pretrained models, DenseNet-201, InceptionV3, and ResNet-50, are first modified according to the nature of the dataset, after which the features are extracted via transfer learning. The extracted features are improved, and the best features are selected using ant colony optimization. Finally, the best features are passed to the k-nearest neighbors and support vector machine algorithms for final classification. The proposed method, evaluated using OCT retinal images collected from Soonchunhyang University Bucheon Hospital, demonstrates an accuracy of 99.1% with the incorporation of ACO. Without ACO, the accuracy achieved is 97.4%. Furthermore, the proposed method exhibits state-of-the-art performance and outperforms existing techniques in terms of accuracy.

Multi-Stage Classification of Retinal OCT Using Multi-Scale Ensemble Deep Architecture

Accurate noninvasive diagnosis of retinal disorders is required for appropriate treatment or precision medicine. This work proposes a multi-stage classification network built on a multi-scale (pyramidal) feature ensemble architecture for retinal image classification using optical coherence tomography (OCT) images. First, a scale-adaptive neural network is developed to produce multi-scale inputs for feature extraction and ensemble learning. The larger input sizes yield more global information, while the smaller input sizes focus on local details. Then, a feature-rich pyramidal architecture is designed to extract multi-scale features as inputs using DenseNet as the backbone. The advantage of the hierarchical structure is that it allows the system to extract multi-scale, information-rich features for the accurate classification of retinal disorders. Evaluation on two public OCT datasets containing normal and abnormal retinas (e.g., diabetic macular edema (DME), choroidal neovascularization (CNV), age-related macular degeneration (AMD), and Drusen) and comparison against recent networks demonstrates the advantages of the proposed architecture's ability to produce feature-rich classification with average accuracy of 97.78%, 96.83%, and 94.26% for the first (binary) stage, second (three-class) stage, and all-at-once (four-class) classification, respectively, using cross-validation experiments using the first dataset. In the second dataset, our system showed an overall accuracy, sensitivity, and specificity of 99.69%, 99.71%, and 99.87%, respectively. Overall, the tangible advantages of the proposed network for enhanced feature learning might be used in various medical image classification tasks where scale-invariant features are crucial for precise diagnosis.

Deep learning model for automatic image quality assessment in PET

BACKGROUND

A variety of external factors might seriously degrade PET image quality and lead to inconsistent results. The aim of this study is to explore a potential PET image quality assessment (QA) method with deep learning (DL).

METHODS

A total of 89 PET images were acquired from Peking Union Medical College Hospital (PUMCH) in China in this study. Ground-truth quality for images was assessed by two senior radiologists and classified into five grades (grade 1, grade 2, grade 3, grade 4, and grade 5). Grade 5 is the best image quality. After preprocessing, the Dense Convolutional Network (DenseNet) was trained to automatically recognize optimal- and poor-quality PET images. Accuracy (ACC), sensitivity, specificity, receiver operating characteristic curve (ROC), and area under the ROC Curve (AUC) were used to evaluate the diagnostic properties of all models. All indicators of models were assessed using fivefold cross-validation. An image quality QA tool was developed based on our deep learning model. A PET QA report can be automatically obtained after inputting PET images.

RESULTS

Four tasks were generated. Task2 showed worst performance in AUC,ACC, specificity and sensitivity among 4 tasks, and task1 showed unstable performance between training and testing and task3 showed low specificity in both training and testing. Task 4 showed the best diagnostic properties and discriminative performance between poor image quality (grade 1, grade 2) and good quality (grade 3, grade 4, grade 5) images. The automated quality assessment of task 4 showed ACC = 0.77, specificity = 0.71, and sensitivity = 0.83, in the train set; ACC = 0.85, specificity = 0.79, and sensitivity = 0.91, in the test set, respectively. The ROC measuring performance of task 4 had an AUC of 0.86 in the train set and 0.91 in the test set. The image QA tool could output basic information of images, scan and reconstruction parameters, typical instances of PET images, and deep learning score.

CONCLUSIONS

This study highlights the feasibility of the assessment of image quality in PET images using a deep learning model, which may assist with accelerating clinical research by reliably assessing image quality.

Improved Detection of Dry Age-Related Macular Degeneration from Optical Coherence Tomography Images using Adaptive Window Based Feature Extraction and Weighted Ensemble Based Classification Approach

BACKGROUND

Dry Age-related macular degeneration (AMD), which affects the older population, can lead to blindness when left untreated. Preventing vision loss in elderly needs early identification. Dry-AMD diagnosis is still time-consuming and very subjective, depending on the ophthalmologist. Setting up a thorough eye-screening system to find Dry-AMD is a very difficult task.

METHODOLOGY

This study aims to develop a weighted majority voting (WMV) ensemble-based prediction model to diagnose Dry-AMD. The WMV approach combines the predictions from base-classifiers and chooses the class with greatest vote based on assigned weights to each classifier. A novel feature extraction method is used along the retinal pigment epithelium (RPE) layer, with the number of windows calculated for each picture playing an important part in identifying Dry-AMD/normal images using the WMV methodology. Pre-processing using hybrid-median filter followed by scale-invariant feature transform based segmentation of RPE layer and curvature flattening of retina is employed to measure exact thickness of RPE layer.

RESULT

The proposed model is trained on 70% of the OCT image database (OCTID) and evaluated on remaining OCTID and SD-OCT Noor dataset. Model has achieved accuracy of 96.15% and 96.94%, respectively. The suggested algorithm's effectiveness in Dry-AMD identification is demonstrated by comparison with alternative approaches. Even though the suggested model is only trained on the OCTID, it has performed well when tested on additional dataset.

CONCLUSION

The suggested architecture can be used for quick eye-screening for early identification of Dry-AMD. The recommended method may be applied in real-time since it requires fewer complexity and learning-variables.

A bibliometric analysis of artificial intelligence applications in macular edema: exploring research hotspots and Frontiers

Background: Artificial intelligence (AI) is used in ophthalmological disease screening and diagnostics, medical image diagnostics, and predicting late-disease progression rates. We reviewed all AI publications associated with macular edema (ME) research Between 2011 and 2022 and performed modeling, quantitative, and qualitative investigations. Methods: On 1st February 2023, we screened the Web of Science Core Collection for AI applications related to ME, from which 297 studies were identified and analyzed (2011-2022). We collected information on: publications, institutions, country/region, keywords, journal name, references, and research hotspots. Literature clustering networks and Frontier knowledge bases were investigated using bibliometrix-BiblioShiny, VOSviewer, and CiteSpace bibliometric platforms. We used the R "bibliometrix" package to synopsize our observations, enumerate keywords, visualize collaboration networks between countries/regions, and generate a topic trends plot. VOSviewer was used to examine cooperation between institutions and identify citation relationships between journals. We used CiteSpace to identify clustering keywords over the timeline and identify keywords with the strongest citation bursts. Results: In total, 47 countries published AI studies related to ME; the United States had the highest H-index, thus the greatest influence. China and the United States cooperated most closely between all countries. Also, 613 institutions generated publications - the Medical University of Vienna had the highest number of studies. This publication record and H-index meant the university was the most influential in the ME field. Reference clusters were also categorized into 10 headings: retinal Optical Coherence Tomography (OCT) fluid detection, convolutional network models, deep learning (DL)-based single-shot predictions, retinal vascular disease, diabetic retinopathy (DR), convolutional neural networks (CNNs), automated macular pathology diagnosis, dry age-related macular degeneration (DARMD), class weight, and advanced DL architecture systems. Frontier keywords were represented by diabetic macular edema (DME) (2021-2022). Conclusion: Our review of the AI-related ME literature was comprehensive, systematic, and objective, and identified future trends and current hotspots. With increased DL outputs, the ME research focus has gradually shifted from manual ME examinations to automatic ME detection and associated symptoms. In this review, we present a comprehensive and dynamic overview of AI in ME and identify future research areas.

RetiFluidNet: A Self-Adaptive and Multi-Attention Deep Convolutional Network for Retinal OCT Fluid Segmentation

Optical coherence tomography (OCT) helps ophthalmologists assess macular edema, accumulation of fluids, and lesions at microscopic resolution. Quantification of retinal fluids is necessary for OCT-guided treatment management, which relies on a precise image segmentation step. As manual analysis of retinal fluids is a time-consuming, subjective, and error-prone task, there is increasing demand for fast and robust automatic solutions. In this study, a new convolutional neural architecture named RetiFluidNet is proposed for multi-class retinal fluid segmentation. The model benefits from hierarchical representation learning of textural, contextual, and edge features using a new self-adaptive dual-attention (SDA) module, multiple self-adaptive attention-based skip connections (SASC), and a novel multi-scale deep self-supervision learning (DSL) scheme. The attention mechanism in the proposed SDA module enables the model to automatically extract deformation-aware representations at different levels, and the introduced SASC paths further consider spatial-channel interdependencies for concatenation of counterpart encoder and decoder units, which improve representational capability. RetiFluidNet is also optimized using a joint loss function comprising a weighted version of dice overlap and edge-preserved connectivity-based losses, where several hierarchical stages of multi-scale local losses are integrated into the optimization process. The model is validated based on three publicly available datasets: RETOUCH, OPTIMA, and DUKE, with comparisons against several baselines. Experimental results on the datasets prove the effectiveness of the proposed model in retinal OCT fluid segmentation and reveal that the suggested method is more effective than existing state-of-the-art fluid segmentation algorithms in adapting to retinal OCT scans recorded by various image scanning instruments.

A novel sequential structure for lightweight multi-scale feature learning under limited available images

Although multi-scale feature learning can improve the performances of deep models, its parallel structure quadratically increases the model parameters and causes deep models to become larger and larger when enlarging the receptive fields (RFs). This leads to deep models easily suffering from over-fitting issue in many practical applications where the available training samples are always insufficient or limited. In addition, under this limited situation, although lightweight models (with fewer model parameters) can effectively reduce over-fitting, they may suffer from under-fitting because of insufficient training data for effective feature learning. In this work, a lightweight model called Sequential Multi-scale Feature Learning Network (SMF-Net) is proposed to alleviate these two issues simultaneously using a novel sequential structure of multi-scale feature learning. Compared to both deep and lightweight models, the proposed sequential structure in SMF-Net can easily extract features with larger RFs for multi-scale feature learning only with a few and linearly increased model parameters. The experimental results on both classification and segmentation tasks demonstrate that our SMF-Net only has 1.25M model parameters (5.3% of Res2Net50) with 0.7G FLOPS (14.6% of Res2Net50) for classification and 1.54M parameters (8.9% of UNet) with 3.35G FLOPs (10.9% of UNet) for segmentation but achieves higher accuracy than SOTA deep models and lightweight models, even when the training data is very limited available.

A review of deep learning-based multiple-lesion recognition from medical images: classification, detection and segmentation

Deep learning-based methods have become the dominant methodology in medical image processing with the advancement of deep learning in natural image classification, detection, and segmentation. Deep learning-based approaches have proven to be quite effective in single lesion recognition and segmentation. Multiple-lesion recognition is more difficult than single-lesion recognition due to the little variation between lesions or the too wide range of lesions involved. Several studies have recently explored deep learning-based algorithms to solve the multiple-lesion recognition challenge. This paper includes an in-depth overview and analysis of deep learning-based methods for multiple-lesion recognition developed in recent years, including multiple-lesion recognition in diverse body areas and recognition of whole-body multiple diseases. We discuss the challenges that still persist in the multiple-lesion recognition tasks by critically assessing these efforts. Finally, we outline existing problems and potential future research areas, with the hope that this review will help researchers in developing future approaches that will drive additional advances.

Deep ensemble learning for automated non-advanced AMD classification using optimized retinal layer segmentation and SD-OCT scans

BACKGROUND

Accurate retinal layer segmentation in optical coherence tomography (OCT) images is crucial for quantitatively analyzing age-related macular degeneration (AMD) and monitoring its progression. However, previous retinal segmentation models depend on experienced experts and manually annotating retinal layers is time-consuming. On the other hand, accuracy of AMD diagnosis is directly related to the segmentation model's performance. To address these issues, we aimed to improve AMD detection using optimized retinal layer segmentation and deep ensemble learning.

METHOD

We integrated a graph-cut algorithm with a cubic spline to automatically annotate 11 retinal boundaries. The refined images were fed into a deep ensemble mechanism that combined a Bagged Tree and end-to-end deep learning classifiers. We tested the developed deep ensemble model on internal and external datasets.

RESULTS

The total error rates for our segmentation model using the boundary refinement approach was significantly lower than OCT Explorer segmentations (1.7% vs. 7.8%, p-value = 0.03). We utilized the refinement approach to quantify 169 imaging features using Zeiss SD-OCT volume scans. The presence of drusen and thickness of total retina, neurosensory retina, and ellipsoid zone to inner-outer segment (EZ-ISOS) thickness had higher contributions to AMD classification compared to other features. The developed ensemble learning model obtained a higher diagnostic accuracy in a shorter time compared with two human graders. The area under the curve (AUC) for normal vs. early AMD was 99.4%.

CONCLUSION

Testing results showed that the developed framework is repeatable and effective as a potentially valuable tool in retinal imaging research.

Distinctions between Choroidal Neovascularization and Age Macular Degeneration in Ocular Disease Predictions via Multi-Size Kernels ξcho-Weighted Median Patterns

Age-related macular degeneration is a visual disorder caused by abnormalities in a part of the eye's retina and is a leading source of blindness. The correct detection, precise location, classification, and diagnosis of choroidal neovascularization (CNV) may be challenging if the lesion is small or if Optical Coherence Tomography (OCT) images are degraded by projection and motion. This paper aims to develop an automated quantification and classification system for CNV in neovascular age-related macular degeneration using OCT angiography images. OCT angiography is a non-invasive imaging tool that visualizes retinal and choroidal physiological and pathological vascularization. The presented system is based on new retinal layers in the OCT image-specific macular diseases feature extractor, including Multi-Size Kernels ξcho-Weighted Median Patterns (MSKξMP). Computer simulations show that the proposed method: (i) outperforms current state-of-the-art methods, including deep learning techniques; and (ii) achieves an overall accuracy of 99% using ten-fold cross-validation on the Duke University dataset and over 96% on the noisy Noor Eye Hospital dataset. In addition, MSKξMP performs well in binary eye disease classifications and is more accurate than recent works in image texture descriptors.

Classification of Retinal Diseases in Optical Coherence Tomography Images Using Artificial Intelligence and Firefly Algorithm

In recent years, the number of studies for the automatic diagnosis of biomedical diseases has increased. Many of these studies have used Deep Learning, which gives extremely good results but requires a vast amount of data and computing load. If the processor is of insufficient quality, this takes time and places an excessive load on the processor. On the other hand, Machine Learning is faster than Deep Learning and does not have a much-needed computing load, but it does not provide as high an accuracy value as Deep Learning. Therefore, our goal is to develop a hybrid system that provides a high accuracy value, while requiring a smaller computing load and less time to diagnose biomedical diseases such as the retinal diseases we chose for this study. For this purpose, first, retinal layer extraction was conducted through image preprocessing. Then, traditional feature extractors were combined with pre-trained Deep Learning feature extractors. To select the best features, we used the Firefly algorithm. In the end, multiple binary classifications were conducted instead of multiclass classification with Machine Learning classifiers. Two public datasets were used in this study. The first dataset had a mean accuracy of 0.957, and the second dataset had a mean accuracy of 0.954.

A Deep Learning-Based Framework for Retinal Disease Classification

This study addresses the problem of the automatic detection of disease states of the retina. In order to solve the abovementioned problem, this study develops an artificially intelligent model. The model is based on a customized 19-layer deep convolutional neural network called VGG-19 architecture. The model (VGG-19 architecture) is empowered by transfer learning. The model is designed so that it can learn from a large set of images taken with optical coherence tomography (OCT) and classify them into four conditions of the retina: (1) choroidal neovascularization, (2) drusen, (3) diabetic macular edema, and (4) normal form. The training datasets (taken from publicly available sources) consist of 84,568 instances of OCT retinal images. The datasets exhibit all four classes of retinal disease mentioned above. The proposed model achieved a 99.17% classification accuracy with 0.995 specificities and 0.99 sensitivity, making it better than the existing models. In addition, the proper statistical evaluation is done on the predictions using such performance measures as (1) area under the receiver operating characteristic curve, (2) Cohen's kappa parameter, and (3) confusion matrix. Experimental results show that the proposed VGG-19 architecture coupled with transfer learning is an effective technique for automatically detecting the disease state of a retina.

Cyst identification in retinal optical coherence tomography images using hidden Markov model

Optical Coherence Tomography (OCT) is a useful imaging modality facilitating the capturing process from retinal layers. In the salient diseases of retina, cysts are formed in retinal layers. Therefore, the identification of cysts in the retinal layers is of great importance. In this paper, a new method is proposed for the rapid detection of cystic OCT B-scans. In the proposed method, a Hidden Markov Model (HMM) is used for mathematically modelling the existence of cyst. In fact, the existence of cyst in the image can be considered as a hidden state. Since the existence of cyst in an OCT B-scan depends on the existence of cyst in the previous B-scans, HMM is an appropriate tool for modelling this process. In the first phase, a number of features are extracted which are Harris, KAZE, HOG, SURF, FAST, Min-Eigen and feature extracted by deep AlexNet. It is shown that the feature with the best discriminating power is the feature extracted by AlexNet. The features extracted in the first phase are used as observation vectors to estimate the HMM parameters. The evaluation results show the improved performance of HMM in terms of accuracy.

Structure-Oriented Transformer for retinal diseases grading from OCT images

Retinal diseases are the leading causes of vision temporary or permanent loss. Precise retinal disease grading is a prerequisite for early intervention or specific therapeutic schedules. Existing works based on Convolutional Neural Networks (CNN) focus on typical locality structures and cannot capture long-range dependencies. But retinal disease grading relies more on the relationship between the local lesion and the whole retina, which is consistent with the self-attention mechanism. Therefore, the paper proposes a novel Structure-Oriented Transformer (SoT) framework to further construct the relationship between lesions and retina on clinical datasets. To reduce the dependence on the amount of data, we design structure guidance as a model-oriented filter to emphasize the whole retina structure and guide relation construction. Then, we adopt the pre-trained vision transformer that efficiently models all feature patches' relationships via transfer learning. Besides, to make the best of all output tokens, a Token vote classifier is proposed to obtain the final grading results. We conduct extensive experiments on one clinical neovascular Age-related Macular Degeneration (nAMD) dataset. The experiments demonstrate the effectiveness of SoT components and improve the ability of relation construction between lesion and retina, which outperforms the state-of-the-art methods for nAMD grading. Furthermore, we evaluate our SoT on one publicly available retinal diseases dataset, which proves our algorithm has classification superiority and good generality.

Predicting effectiveness of anti-VEGF injection through self-supervised learning in OCT images

Anti-vascular endothelial growth factor (Anti-VEGF) therapy has become a standard way for choroidal neovascularization (CNV) and cystoid macular edema (CME) treatment. However, anti-VEGF injection is a long-term therapy with expensive cost and may be not effective for some patients. Therefore, predicting the effectiveness of anti-VEGF injection before the therapy is necessary. In this study, a new optical coherence tomography (OCT) images based self-supervised learning (OCT-SSL) model for predicting the effectiveness of anti-VEGF injection is developed. In OCT-SSL, we pre-train a deep encoder-decoder network through self-supervised learning to learn the general features using a public OCT image dataset. Then, model fine-tuning is performed on our own OCT dataset to learn the discriminative features to predict the effectiveness of anti-VEGF. Finally, classifier trained by the features from fine-tuned encoder as a feature extractor is built to predict the response. Experimental results on our private OCT dataset demonstrated that the proposed OCT-SSL can achieve an average accuracy, area under the curve (AUC), sensitivity and specificity of 0.93, 0.98, 0.94 and 0.91, respectively. Meanwhile, it is found that not only the lesion region but also the normal region in OCT image is related to the effectiveness of anti-VEGF.

CLASSIFICATION OF AGE-RELATED MACULAR DEGENERATION USING DAG-CNN ARCHITECTURE

Age-related Macular Degeneration (AMD) is the prime reason for vision impairment observed in major countries worldwide. Hence an accurate early detection of the disease is vital for more research in this area. Also, having a thorough eye diagnosis to detect AMD is a complex job. This paper introduces a Directed Acyclic Graph (DAG) structure-based Convolutional Neural network (CNN) architecture to better classify Dry or Wet AMD. The DAG architecture can combine features from multiple layers to provide better results. The DAG model also has the capacity to learn multi-level visual properties to increase classification accuracy. Fine tuning of DAG-based CNN model helps in improving the performance of the network. The training and testing of the proposed model are carried out with the Mendeley data set and achieved an accuracy of 99.2% with an AUC value of 0.9999. The proposed model also obtains better results for other parameters such as precision, recall and F1-score. Performance of the proposed network is also compared to that of the related works performed on the same data set. This shows ability of the proposed method to grade AMD images to help early detection of the disease. The model also performs computationally efficient for real-time applications as it does the classification process with few learnable parameters and fewer Floating-Point Operations (FLOPs).

Inflation of test accuracy due to data leakage in deep learning-based classification of OCT images

In the application of deep learning on optical coherence tomography (OCT) data, it is common to train classification networks using 2D images originating from volumetric data. Given the micrometer resolution of OCT systems, consecutive images are often very similar in both visible structures and noise. Thus, an inappropriate data split can result in overlap between the training and testing sets, with a large portion of the literature overlooking this aspect. In this study, the effect of improper dataset splitting on model evaluation is demonstrated for three classification tasks using three OCT open-access datasets extensively used, Kermany's and Srinivasan's ophthalmology datasets, and AIIMS breast tissue dataset. Results show that the classification performance is inflated by 0.07 up to 0.43 in terms of Matthews Correlation Coefficient (accuracy: 5% to 30%) for models tested on datasets with improper splitting, highlighting the considerable effect of dataset handling on model evaluation. This study intends to raise awareness on the importance of dataset splitting given the increased research interest in implementing deep learning on OCT data.

Two-Stage Selective Ensemble of CNN via Deep Tree Training for Medical Image Classification

Medical image classification is an important task in computer-aided diagnosis systems. Its performance is critically determined by the descriptiveness and discriminative power of features extracted from images. With rapid development of deep learning, deep convolutional neural networks (CNNs) have been widely used to learn the optimal high-level features from the raw pixels of images for a given classification task. However, due to the limited amount of labeled medical images with certain quality distortions, such techniques crucially suffer from the training difficulties, including overfitting, local optimums, and vanishing gradients. To solve these problems, in this article, we propose a two-stage selective ensemble of CNN branches via a novel training strategy called deep tree training (DTT). In our approach, DTT is adopted to jointly train a series of networks constructed from the hidden layers of CNN in a hierarchical manner, leading to the advantage that vanishing gradients can be mitigated by supplementing gradients for hidden layers of CNN, and intrinsically obtain the base classifiers on the middle-level features with minimum computation burden for an ensemble solution. Moreover, the CNN branches as base learners are combined into the optimal classifier via the proposed two-stage selective ensemble approach based on both accuracy and diversity criteria. Extensive experiments on CIFAR-10 benchmark and two specific medical image datasets illustrate that our approach achieves better performance in terms of accuracy, sensitivity, specificity, and F1 score measurement.

Automatic detection of microaneurysms in optical coherence tomography images of retina using convolutional neural networks and transfer learning

Microaneurysms (MAs) are pathognomonic signs that help clinicians to detect diabetic retinopathy (DR) in the early stages. Automatic detection of MA in retinal images is an active area of research due to its application in screening processes for DR which is one of the main reasons of blindness amongst the working-age population. The focus of these works is on the automatic detection of MAs in en face retinal images like fundus color and Fluorescein Angiography (FA). On the other hand, detection of MAs from Optical Coherence Tomography (OCT) images has 2 main advantages: first, OCT is a non-invasive imaging technique that does not require injection, therefore is safer. Secondly, because of the proven application of OCT in detection of Age-Related Macular Degeneration, Diabetic Macular Edema, and normal cases, thanks to detecting MAs in OCT, extensive information is obtained by using this imaging technique. In this research, the concentration is on the diagnosis of MAs using deep learning in the OCT images which represent in-depth structure of retinal layers. To this end, OCT B-scans should be divided into strips and MA patterns should be searched in the resulted strips. Since we need a dataset comprising OCT image strips with suitable labels and such large labelled datasets are not yet available, we have created it. For this purpose, an exact registration method is utilized to align OCT images with FA photographs. Then, with the help of corresponding FA images, OCT image strips are created from OCT B-scans in four labels, namely MA, normal, abnormal, and vessel. Once the dataset of image strips is prepared, a stacked generalization (stacking) ensemble of four fine-tuned, pre-trained convolutional neural networks is trained to classify the strips of OCT images into the mentioned classes. FA images are used once to create OCT strips for training process and they are no longer needed for subsequent steps. Once the stacking ensemble model is obtained, it will be used to classify the OCT strips in the test process. The results demonstrate that the proposed framework classifies overall OCT image strips and OCT strips containing MAs with accuracy scores of 0.982 and 0.987, respectively.

HCTNet: A Hybrid ConvNet-Transformer Network for Retinal Optical Coherence Tomography Image Classification

Automatic and accurate optical coherence tomography (OCT) image classification is of great significance to computer-assisted diagnosis of retinal disease. In this study, we propose a hybrid ConvNet-Transformer network (HCTNet) and verify the feasibility of a Transformer-based method for retinal OCT image classification. The HCTNet first utilizes a low-level feature extraction module based on the residual dense block to generate low-level features for facilitating the network training. Then, two parallel branches of the Transformer and the ConvNet are designed to exploit the global and local context of the OCT images. Finally, a feature fusion module based on an adaptive re-weighting mechanism is employed to combine the extracted global and local features for predicting the category of OCT images in the testing datasets. The HCTNet combines the advantage of the convolutional neural network in extracting local features and the advantage of the vision Transformer in establishing long-range dependencies. A verification on two public retinal OCT datasets shows that our HCTNet method achieves an overall accuracy of 91.56% and 86.18%, respectively, outperforming the pure ViT and several ConvNet-based classification methods.

Automatic Classification of Macular Diseases from OCT Images Using CNN Guided with Edge Convolutional Layer

Optical Coherence Tomography (OCT) is a non-invasive imaging technology that is widely applied for the diagnosis of retinal pathologies. In general, the structural information of retinal layers plays an important role in the diagnosis of various eye diseases by ophthalmologists. In this paper, by focusing on this information, we first introduce a new layer called the edge convolutional layer (ECL) to accurately extract the retinal boundaries in different sizes and angles with a much smaller number of parameters than the conventional convolutional layer. Then, using this layer, we propose the ECL-guided convolutional neural network (ECL-CNN) method for the automatic classification of the OCT images. For the assessment of the proposed method, we utilize a publicly available data comprising 45 OCT volumes with 15 age-related macular degeneration (AMD), 15 diabetic macular edema (DME), and 15 normal volumes, captured by using the Heidelberg OCT imaging device. Experimental results demonstrate that the suggested ECL-CNN approach has an outstanding performance in OCT image classification, which achieves an average precision of 99.43% as a three-class classification work. Clinical Relevance - The objective of this research is to introduce a new approach based on CNN for the automated classification of retinal OCT images. Clinically, the ophthalmologists should manually check each cross-sectional B-scan and classify retinal pathologies from B-scan images. This manual process is tedious and time-consuming in general. Hence, an automatic computer-assisted technique for retinal OCT image classification is demanded.

Detection of Retinal Abnormalities in OCT Images Using Wavelet Scattering Network

Diagnosis retinal abnormalities in Optical Coherence Tomography (OCT) images assist ophthalmologist in the early detection and treatment of patients. To do this, different Computer Aided Diagnosis (CAD) methods based on machine learning and deep learning algorithms have been proposed. In this paper, wavelet scattering network is used to identify normal retina and four pathologies namely, Central Serous Retinopathy (CSR), Macular Hole (MH), Age-related Macular Degeneration (AMD) and Diabetic Retinopathy (DR). Wavelet scattering network is a particular convolutional network which is formed from cascading wavelet transform with nonlinear modulus and averaging operators. This transform generates sparse, translation invariant and deformation stable representations of signals. Filters in the layers of this network are predefined wavelets and not need to be learned which causes decreasing the processing time and complexity. The extracted features are fed to a Principal Component Analysis (PCA) classifier. The results of this research show the accuracy of 97.4% and 100% in diagnosis abnormal retina and DR from normal ones, respectively. We also achieved the accuracy of 84.2% in classifying OCT images to five classes of normal, CSR, MH, AMD and DR which outperforms other state of the art methods with high computational complexity. Clinical Relevance- Clinically, the manually checking of each OCT B-scan by ophthalmologists is tedious and time consuming and may lead to an erroneous decision specially for multiclass problems. In this study, a low complexity CAD system for retinal OCT image classification based on wavelet scattering network is introduced which can be learned by a small number of data.

FN-OCT: Disease Detection Algorithm for Retinal Optical Coherence Tomography Based on a Fusion Network

Optical coherence tomography (OCT) is a new type of tomography that has experienced rapid development and potential in recent years. It is playing an increasingly important role in retinopathy diagnoses. At present, due to the uneven distributions of medical resources in various regions, the uneven proficiency levels of doctors in grassroots and remote areas, and the development needs of rare disease diagnosis and precision medicine, artificial intelligence technology based on deep learning can provide fast, accurate, and effective solutions for the recognition and diagnosis of retinal OCT images. To prevent vision damage and blindness caused by the delayed discovery of retinopathy, a fusion network (FN)-based retinal OCT classification algorithm (FN-OCT) is proposed in this paper to improve upon the adaptability and accuracy of traditional classification algorithms. The InceptionV3, Inception-ResNet, and Xception deep learning algorithms are used as base classifiers, a convolutional block attention mechanism (CBAM) is added after each base classifier, and three different fusion strategies are used to merge the prediction results of the base classifiers to output the final prediction results (choroidal neovascularization (CNV), diabetic macular oedema (DME), drusen, normal). The results show that in a classification problem involving the UCSD common retinal OCT dataset (108,312 OCT images from 4,686 patients), compared with that of the InceptionV3 network model, the prediction accuracy of FN-OCT is improved by 5.3% (accuracy = 98.7%, area under the curve (AUC) = 99.1%). The predictive accuracy and AUC achieved on an external dataset for the classification of retinal OCT diseases are 92 and 94.5%, respectively, and gradient-weighted class activation mapping (Grad-CAM) is used as a visualization tool to verify the effectiveness of the proposed FNs. This finding indicates that the developed fusion algorithm can significantly improve the performance of classifiers while providing a powerful tool and theoretical support for assisting with the diagnosis of retinal OCT.

Recognition Method of Corn and Rice Crop Growth State Based on Computer Image Processing Technology

The agriculture field is one of the most important fields where computational techniques play an imperative role for decision-making whether it is the automation of watering of plants, controlling of humidity levels, and detection of plant diseases and growth of plants. There are problems in the conventional methods where newer computational techniques and image processing methods are not used to keep track of growth of plants. The traditional image capturing and processing models have problems of large image segmentation error, excessive feature extraction time, and poor recognition output. In order to overcome the problems in the traditional plant growth methods based on image processing automations, computer image processing with computational method has been proposed to analyze the plant growth by utilizing state recognition method for corn and rice crops. An image acquisition platform is established on the basis of CMOS image sensor for crop image acquisition. The binary processing is performed, and then the images are segmented to reduce error of segmentation results in the traditional methods. To extract image features of corn and rice crops, convolution neural network (CNN) with newer architecture is used. According to contour information of images, the block wavelet transform method is used for feature adaptive matching. The binary tree structure is used to divide the growth period of corn and rice crops. The fuzzy mathematical model is also devised to identify the characteristics of crops in different growth periods and to complete the identification of growth state. Experimental results show that the proposed method effectively improves problems of traditional methods with better image recognition effect and reduces the time of feature recognition. The time to extract features by the proposed method is 1.4 seconds, whereas comparative methods such as random forest (RF) take 3.8 s and other traditional techniques take 4.9 s. Segmentation result error of the recognition method is also reduced significantly.