Deep learning architecture "LightOCT" for diagnostic decision support using optical coherence tomography images of biological samples

A Vision Language Correlation Framework for Screening Disabled Retina

Retinopathy is a group of retinal disabilities that causes severe visual impairments or complete blindness. Due to the capability of optical coherence tomography to reveal early retinal abnormalities, many researchers have utilized it to develop autonomous retinal screening systems. However, to the best of our knowledge, most of these systems rely only on mathematical features, which might not be helpful to clinicians since they do not encompass the clinical manifestations of screening the underlying diseases. Such clinical manifestations are critically important to be considered within the autonomous screening systems to match the grading of ophthalmologists within the clinical settings. To overcome these limitations, we present a novel framework that exploits the fusion of vision language correlation between the retinal imagery and the set of clinical prompts to recognize the different types of retinal disabilities. The proposed framework is rigorously tested on six public datasets, where, across each dataset, the proposed framework outperformed state-of-the-art methods in various metrics. Moreover, the clinical significance of the proposed framework is also tested under strict blind testing experiments, where the proposed system achieved a statistically significant correlation coefficient of 0.9185 and 0.9529 with the two expert clinicians. These blind test experiments highlight the potential of the proposed framework to be deployed in the real world for accurate screening of retinal diseases.

Artificial intelligence for diagnosing exudative age-related macular degeneration

BACKGROUND

Age-related macular degeneration (AMD) is a retinal disorder characterized by central retinal (macular) damage. Approximately 10% to 20% of non-exudative AMD cases progress to the exudative form, which may result in rapid deterioration of central vision. Individuals with exudative AMD (eAMD) need prompt consultation with retinal specialists to minimize the risk and extent of vision loss. Traditional methods of diagnosing ophthalmic disease rely on clinical evaluation and multiple imaging techniques, which can be resource-consuming. Tests leveraging artificial intelligence (AI) hold the promise of automatically identifying and categorizing pathological features, enabling the timely diagnosis and treatment of eAMD.

OBJECTIVES

To determine the diagnostic accuracy of artificial intelligence (AI) as a triaging tool for exudative age-related macular degeneration (eAMD).

SEARCH METHODS

We searched CENTRAL, MEDLINE, Embase, three clinical trials registries, and Data Archiving and Networked Services (DANS) for gray literature. We did not restrict searches by language or publication date. The date of the last search was April 2024.

SELECTION CRITERIA

Included studies compared the test performance of algorithms with that of human readers to detect eAMD on retinal images collected from people with AMD who were evaluated at eye clinics in community or academic medical centers, and who were not receiving treatment for eAMD when the images were taken. We included algorithms that were either internally or externally validated or both.

DATA COLLECTION AND ANALYSIS

Pairs of review authors independently extracted data and assessed study quality using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool with revised signaling questions. For studies that reported more than one set of performance results, we extracted only one set of diagnostic accuracy data per study based on the last development stage or the optimal algorithm as indicated by the study authors. For two-class algorithms, we collected data from the 2x2 table whenever feasible. For multi-class algorithms, we first consolidated data from all classes other than eAMD before constructing the corresponding 2x2 tables. Assuming a common positivity threshold applied by the included studies, we chose random-effects, bivariate logistic models to estimate summary sensitivity and specificity as the primary performance metrics.

MAIN RESULTS

We identified 36 eligible studies that reported 40 sets of algorithm performance data, encompassing over 16,000 participants and 62,000 images. We included 28 studies (78%) that reported 31 algorithms with performance data in the meta-analysis. The remaining nine studies (25%) reported eight algorithms that lacked usable performance data; we reported them in the qualitative synthesis. Study characteristics and risk of bias Most studies were conducted in Asia, followed by Europe, the USA, and collaborative efforts spanning multiple countries. Most studies identified study participants from the hospital setting, while others used retinal images from public repositories; a few studies did not specify image sources. Based on four of the 36 studies reporting demographic information, the age of the study participants ranged from 62 to 82 years. The included algorithms used various retinal image types as model input, such as optical coherence tomography (OCT) images (N = 15), fundus images (N = 6), and multi-modal imaging (N = 7). The predominant core method used was deep neural networks. All studies that reported externally validated algorithms were at high risk of bias mainly due to potential selection bias from either a two-gate design or the inappropriate exclusion of potentially eligible retinal images (or participants). Findings Only three of the 40 included algorithms were externally validated (7.5%, 3/40). The summary sensitivity and specificity were 0.94 (95% confidence interval (CI) 0.90 to 0.97) and 0.99 (95% CI 0.76 to 1.00), respectively, when compared to human graders (3 studies; 27,872 images; low-certainty evidence). The prevalence of images with eAMD ranged from 0.3% to 49%. Twenty-eight algorithms were reportedly either internally validated (20%, 8/40) or tested on a development set (50%, 20/40); the pooled sensitivity and specificity were 0.93 (95% CI 0.89 to 0.96) and 0.96 (95% CI 0.94 to 0.98), respectively, when compared to human graders (28 studies; 33,409 images; low-certainty evidence). We did not identify significant sources of heterogeneity among these 28 algorithms. Although algorithms using OCT images appeared more homogeneous and had the highest summary specificity (0.97, 95% CI 0.93 to 0.98), they were not superior to algorithms using fundus images alone (0.94, 95% CI 0.89 to 0.97) or multimodal imaging (0.96, 95% CI 0.88 to 0.99; P for meta-regression = 0.239). The median prevalence of images with eAMD was 30% (interquartile range [IQR] 22% to 39%). We did not include eight studies that described nine algorithms (one study reported two sets of algorithm results) to distinguish eAMD from normal images, images of other AMD, or other non-AMD retinal lesions in the meta-analysis. Five of these algorithms were generally based on smaller datasets (range 21 to 218 participants per study) yet with a higher prevalence of eAMD images (range 33% to 66%). Relative to human graders, the reported sensitivity in these studies ranged from 0.95 and 0.97, while the specificity ranged from 0.94 to 0.99. Similarly, using small datasets (range 46 to 106), an additional four algorithms for detecting eAMD from other retinal lesions showed high sensitivity (range 0.96 to 1.00) and specificity (range 0.77 to 1.00).

AUTHORS' CONCLUSIONS

Low- to very low-certainty evidence suggests that an algorithm-based test may correctly identify most individuals with eAMD without increasing unnecessary referrals (false positives) in either the primary or the specialty care settings. There were significant concerns for applying the review findings due to variations in the eAMD prevalence in the included studies. In addition, among the included algorithm-based tests, diagnostic accuracy estimates were at risk of bias due to study participants not reflecting real-world characteristics, inadequate model validation, and the likelihood of selective results reporting. Limited quality and quantity of externally validated algorithms highlighted the need for high-certainty evidence. This evidence will require a standardized definition for eAMD on different imaging modalities and external validation of the algorithm to assess generalizability.

Dense Convolutional Neural Network-Based Deep Learning Pipeline for Pre-Identification of Circular Leaf Spot Disease of Diospyros kaki Leaves Using Optical Coherence Tomography

Circular leaf spot (CLS) disease poses a significant threat to persimmon cultivation, leading to substantial harvest reductions. Existing visual and destructive inspection methods suffer from subjectivity, limited accuracy, and considerable time consumption. This study presents an automated pre-identification method of the disease through a deep learning (DL) based pipeline integrated with optical coherence tomography (OCT), thereby addressing the highlighted issues with the existing methods. The investigation yielded promising outcomes by employing transfer learning with pre-trained DL models, specifically DenseNet-121 and VGG-16. The DenseNet-121 model excels in differentiating among three stages of CLS disease (healthy (H), apparently healthy (or healthy-infected (HI)), and infected (I)). The model achieved precision values of 0.7823 for class-H, 0.9005 for class-HI, and 0.7027 for class-I, supported by recall values of 0.8953 for class-HI and 0.8387 for class-I. Moreover, the performance of CLS detection was enhanced by a supplemental quality inspection model utilizing VGG-16, which attained an accuracy of 98.99% in discriminating between low-detail and high-detail images. Moreover, this study employed a combination of LAMP and A-scan for the dataset labeling process, significantly enhancing the accuracy of the models. Overall, this study underscores the potential of DL techniques integrated with OCT to enhance disease identification processes in agricultural settings, particularly in persimmon cultivation, by offering efficient and objective pre-identification of CLS and enabling early intervention and management strategies.

Machine Learning-Assisted Short-Wave InfraRed (SWIR) Techniques for Biomedical Applications: Towards Personalized Medicine

Personalized medicine transforms healthcare by adapting interventions to individuals' unique genetic, molecular, and clinical profiles. To maximize diagnostic and/or therapeutic efficacy, personalized medicine requires advanced imaging devices and sensors for accurate assessment and monitoring of individual patient conditions or responses to therapeutics. In the field of biomedical optics, short-wave infrared (SWIR) techniques offer an array of capabilities that hold promise to significantly enhance diagnostics, imaging, and therapeutic interventions. SWIR techniques provide in vivo information, which was previously inaccessible, by making use of its capacity to penetrate biological tissues with reduced attenuation and enable researchers and clinicians to delve deeper into anatomical structures, physiological processes, and molecular interactions. Combining SWIR techniques with machine learning (ML), which is a powerful tool for analyzing information, holds the potential to provide unprecedented accuracy for disease detection, precision in treatment guidance, and correlations of complex biological features, opening the way for the data-driven personalized medicine field. Despite numerous biomedical demonstrations that utilize cutting-edge SWIR techniques, the clinical potential of this approach has remained significantly underexplored. This paper demonstrates how the synergy between SWIR imaging and ML is reshaping biomedical research and clinical applications. As the paper showcases the growing significance of SWIR imaging techniques that are empowered by ML, it calls for continued collaboration between researchers, engineers, and clinicians to boost the translation of this technology into clinics, ultimately bridging the gap between cutting-edge technology and its potential for personalized medicine.

Generating OCT B-Scan DME images using optimized Generative Adversarial Networks (GANs)

Diabetic Macular Edema (DME) represents a significant visual impairment among individuals with diabetes, leading to a dramatic reduction in visual acuity and potentially resulting in irreversible vision loss. Optical Coherence Tomography (OCT), a technique that produces high-resolution retinal images, plays a vital role in the clinical assessment of this condition. Physicians typically rely on OCT B-Scan images to evaluate DME severity. However, manual interpretation of these images is susceptible to errors, which can lead to detrimental consequences, such as misdiagnosis and improper treatment strategies. Hence, there is a critical need for more reliable diagnostic methods. This study aims to address this gap by proposing an automated model based on Generative Adversarial Networks (GANs) to generate OCT B-Scan images of DME. The model synthesizes images from patients' baseline OCT B-Scan images, which could potentially enhance the robustness of DME detection systems. We employ five distinct GANs in this study: Deep Convolutional GAN, Conditional GAN, CycleGAN, StyleGAN2, and StyleGAN3, drawing comparisons across their performance. Subsequently, the hyperparameters of the best-performing GAN are fine-tuned using Particle Swarm Optimization (PSO) to produce more realistic OCT images. This comparative analysis not only serves to improve the detection of DME severity using OCT images but also provides insights into the appropriate choice of GANs for the effective generation of realistic OCT images from the baseline OCT datasets.

The application of artificial intelligence in glaucoma diagnosis and prediction

Artificial intelligence is a multidisciplinary and collaborative science, the ability of deep learning for image feature extraction and processing gives it a unique advantage in dealing with problems in ophthalmology. The deep learning system can assist ophthalmologists in diagnosing characteristic fundus lesions in glaucoma, such as retinal nerve fiber layer defects, optic nerve head damage, optic disc hemorrhage, etc. Early detection of these lesions can help delay structural damage, protect visual function, and reduce visual field damage. The development of deep learning led to the emergence of deep convolutional neural networks, which are pushing the integration of artificial intelligence with testing devices such as visual field meters, fundus imaging and optical coherence tomography to drive more rapid advances in clinical glaucoma diagnosis and prediction techniques. This article details advances in artificial intelligence combined with visual field, fundus photography, and optical coherence tomography in the field of glaucoma diagnosis and prediction, some of which are familiar and some not widely known. Then it further explores the challenges at this stage and the prospects for future clinical applications. In the future, the deep cooperation between artificial intelligence and medical technology will make the datasets and clinical application rules more standardized, and glaucoma diagnosis and prediction tools will be simplified in a single direction, which will benefit multiple ethnic groups.

Point-of-care devices based on fluorescence imaging and spectroscopy for tumor margin detection during breast cancer surgery: Towards breast conservation treatment

OBJECTIVE

Fluorescence-based methods are highly specific and sensitive and have potential in breast cancer detection. Simultaneous fluorescence imaging and spectroscopy during intraoperative procedures of breast cancer have great advantages in detection of tumor margin as well as in classification of tumor to healthy tissues. Intra-operative real-time confirmation of breast cancer tumor margin is the aim of surgeons, and therefore, there is an urgent need for such techniques and devices which fulfill the surgeon's priorities.

METHODS

In this article, we propose the development of fluorescence-based smartphone imaging and spectroscopic point-of-care multi-modal devices for detection of invasive ductal carcinoma in tumor margin during removal of tumor. These multimodal devices are portable, cost-effective, noninvasive, and user-friendly. Molecular level sensitivity of fluorescence process shows different behavior in normal, cancerous and marginal tissues. We observed significant spectral changes, such as, red-shift, full-width half maximum (FWHM), and increased intensity as we go towards tumor center from normal tissue. High contrast in fluorescence images and spectra are also recorded for cancer tissues compared to healthy tissues. Preliminary results for the initial trial of the devices are reported in this article.

RESULTS

A total 44 spectra from 11 patients of invasive ductal carcinoma (11 spectra for invasive ductal carcinoma and rest are normal and negative margins) are used. Principle component analysis is used for the classification of invasive ductal carcinoma with an accuracy of 93%, specificity of 75% and sensitivity of 92.8%. We obtained an average 6.17 ± 1.66 nm red shift for IDC with respect to normal tissue. The red shift and maximum fluorescence intensity indicates p < 0.01. These results described here are supported by histopathological examination of the same sample.

CONCLUSION

In the present manuscript, simultaneous fluorescence-based imaging and spectroscopy is accomplished for the classification of IDC tissues and breast cancer margin detection.

Diseased thyroid tissue classification in OCT images using deep learning: Towards surgical decision support

Intraoperative guidance tools for thyroid surgery based on optical coherence tomography (OCT) could aid distinguish between normal and diseased tissue. However, OCT images are difficult to interpret, thus, real-time automatic analysis could support the clinical decision-making. In this study, several deep learning models were investigated for thyroid disease classification on 2D and 3D OCT data obtained from ex vivo specimens of 22 patients undergoing surgery and diagnosed with several thyroid pathologies. Additionally, two open-access datasets were used to evaluate the custom models. On the thyroid dataset, the best performance was achieved by the 3D vision transformer model with a Matthew's correlation coefficient (MCC) of 0.79 (accuracy = 0.90) for the normal-versus-abnormal classification. On the open-access datasets, the custom models achieved the best performance (MCC > 0.88, accuracy > 0.96). Results obtained for the normal-versus-abnormal classification suggest OCT, complemented with deep learning-based analysis, as a tool for real-time automatic diseased tissue identification in thyroid surgery.

Inflation of test accuracy due to data leakage in deep learning-based classification of OCT images

In the application of deep learning on optical coherence tomography (OCT) data, it is common to train classification networks using 2D images originating from volumetric data. Given the micrometer resolution of OCT systems, consecutive images are often very similar in both visible structures and noise. Thus, an inappropriate data split can result in overlap between the training and testing sets, with a large portion of the literature overlooking this aspect. In this study, the effect of improper dataset splitting on model evaluation is demonstrated for three classification tasks using three OCT open-access datasets extensively used, Kermany's and Srinivasan's ophthalmology datasets, and AIIMS breast tissue dataset. Results show that the classification performance is inflated by 0.07 up to 0.43 in terms of Matthews Correlation Coefficient (accuracy: 5% to 30%) for models tested on datasets with improper splitting, highlighting the considerable effect of dataset handling on model evaluation. This study intends to raise awareness on the importance of dataset splitting given the increased research interest in implementing deep learning on OCT data.

Multi-class classification of breast tissue using optical coherence tomography and attenuation imaging combined via deep learning

We demonstrate a convolutional neural network (CNN) for multi-class breast tissue classification as adipose tissue, benign dense tissue, or malignant tissue, using multi-channel optical coherence tomography (OCT) and attenuation images, and a novel Matthews correlation coefficient (MCC)-based loss function that correlates more strongly with performance metrics than the commonly used cross-entropy loss. We hypothesized that using multi-channel images would increase tumor detection performance compared to using OCT alone. 5,804 images from 29 patients were used to fine-tune a pre-trained ResNet-18 network. Adding attenuation images to OCT images yields statistically significant improvements in several performance metrics, including benign dense tissue sensitivity (68.0% versus 59.6%), malignant tissue positive predictive value (PPV) (79.4% versus 75.5%), and total accuracy (85.4% versus 83.3%), indicating that the additional contrast from attenuation imaging is most beneficial for distinguishing between benign dense tissue and malignant tissue.

Non-transfer Deep Learning of Optical Coherence Tomography for Post-hoc Explanation of Macular Disease Classification

Deep transfer learning is a popular choice for classifying monochromatic medical images using models that are pretrained by natural images with color channels. This choice may introduce unnecessarily redundant model complexity that can limit explanations of such model behavior and outcomes in the context of medical imaging. To investigate this hypothesis, we develop a configurable deep convolutional neural network (CNN) to classify four macular disease conditions using retinal optical coherence tomography (OCT) images. Our proposed non-transfer deep CNN model (acc: 97.9%) outperforms existing transfer learning models such as ResNet-50 (acc: 89.0%), ResNet-101 (acc: 96.7%), VGG-19 (acc: 93.3%), Inception-V3 (acc: 95.8%) in the same retinal OCT image classification task. We perform post-hoc analysis of the trained model and model extracted image features, which reveals that only eight out of 256 filter kernels are active at our final convolutional layer. The convolutional responses of these selective eight filters yield image features that efficiently separate four macular disease classes even when projected onto two-dimensional principal component space. Our findings suggest that many deep learning parameters and their computations are redundant and expensive for retinal OCT image classification, which are expected to be more intense when using transfer learning. Additionally, we provide clinical interpretations of our misclassified test images identifying manifest artifacts, shadowing of useful texture, false texture representing fluids, and other confounding factors. These clinical explanations along with model optimization via kernel selection can improve the classification accuracy, computational costs, and explainability of model outcomes.

Differentiation of breast tissue types for surgical margin assessment using machine learning and polarization-sensitive optical coherence tomography

We report an automated differentiation model for classifying malignant tumor, fibro-adipose, and stroma in human breast tissues based on polarization-sensitive optical coherence tomography (PS-OCT). A total of 720 PS-OCT images from 72 sites of 41 patients with H&E histology-confirmed diagnoses as the gold standard were employed in this study. The differentiation model is trained by the features extracted from both one standard OCT-based metric (i.e., intensity) and four PS-OCT-based metrics (i.e., phase difference between two channels (PD), phase retardation (PR), local phase retardation (LPR), and degree of polarization uniformity (DOPU)). Further optimized by forward searching and validated by leave-one-site-out-cross-validation (LOSOCV) method, the best feature subset was acquired with the highest overall accuracy of 93.5% for the model. Furthermore, to show the superiority of our differentiation model based on PS-OCT images over standard OCT images, the best model trained by intensity-only features (usually obtained by standard OCT systems) was also obtained with an overall accuracy of 82.9%, demonstrating the significance of the polarization information in breast tissue differentiation. The high performance of our differentiation model suggests the potential of using PS-OCT for intraoperative human breast tissue differentiation during the surgical resection of breast cancer.