Unraveling the deep learning gearbox in optical coherence tomography image segmentation towards explainable artificial intelligence

Human selection bias drives the linear nature of the more ground truth effect in explainable deep learning optical coherence tomography image segmentation

Supervised deep learning (DL) algorithms are highly dependent on training data for which human graders are assigned, for example, for optical coherence tomography (OCT) image annotation. Despite the tremendous success of DL, due to human judgment, these ground truth labels can be inaccurate and/or ambiguous and cause a human selection bias. We therefore investigated the impact of the size of the ground truth and variable numbers of graders on the predictive performance of the same DL architecture and repeated each experiment three times. The largest training dataset delivered a prediction performance close to that of human experts. All DL systems utilized were highly consistent. Nevertheless, the DL under-performers could not achieve any further autonomous improvement even after repeated training. Furthermore, a quantifiable linear relationship between ground truth ambiguity and the beneficial effect of having a larger amount of ground truth data was detected and marked as the more-ground-truth effect.

Discrimination of multiple sclerosis using OCT images from two different centers

BACKGROUND

Multiple sclerosis (MS) is one of the most prevalent chronic inflammatory diseases caused by demyelination and axonal damage in the central nervous system. Structural retinal imaging via optical coherence tomography (OCT) shows promise as a noninvasive biomarker for monitoring of MS. There are successful reports regarding the application of Artificial Intelligence (AI) in the analysis of cross-sectional OCTs in ophthalmologic diseases. However, the alteration of thicknesses of various retinal layers in MS is noticeably subtle compared to other ophthalmologic diseases. Therefore, raw cross-sectional OCTs are replaced with multilayer segmented OCTs for discrimination of MS and healthy controls (HCs).

METHODS

To conform to the principles of trustworthy AI, interpretability is provided by visualizing the regional layer contribution to classification performance with the proposed occlusion sensitivity approach. The robustness of the classification is also guaranteed by showing the effectiveness of the algorithm while being tested on the new independent dataset. The most discriminative features from different topologies of the multilayer segmented OCTs are selected by the dimension reduction method. Support vector machine (SVM), random forest (RF), and artificial neural network (ANN) are used for classification. Patient-wise cross-validation (CV) is utilized to evaluate the performance of the algorithm, where the training and test folds contain records from different subjects.

RESULTS

The most discriminative topology is determined to square with a size of 40 pixels and the most influential layers are the ganglion cell and inner plexiform layer (GCIPL) and inner nuclear layer (INL). Linear SVM resulted in 88% Accuracy (with standard deviation (std) = 0.49 in 10 times of execution to indicate the repeatability), 78% precision (std=1.48), and 63% recall (std=1.35) in the discrimination of MS and HCs using macular multilayer segmented OCTs.

CONCLUSION

The proposed classification algorithm is expected to help neurologists in the early diagnosis of MS. This paper distinguishes itself from other studies by employing two distinct datasets, which enhances the robustness of its findings in comparison with previous studies with lack of external validation. This study aims to circumvent the utilization of deep learning methods due to the limited quantity of the available data and convincingly demonstrates that favorable outcomes can be achieved without relying on deep learning techniques.

Explainable AI in radiology: a white paper of the Italian Society of Medical and Interventional Radiology

The term Explainable Artificial Intelligence (xAI) groups together the scientific body of knowledge developed while searching for methods to explain the inner logic behind the AI algorithm and the model inference based on knowledge-based interpretability. The xAI is now generally recognized as a core area of AI. A variety of xAI methods currently are available to researchers; nonetheless, the comprehensive classification of the xAI methods is still lacking. In addition, there is no consensus among the researchers with regards to what an explanation exactly is and which are salient properties that must be considered to make it understandable for every end-user. The SIRM introduces an xAI-white paper, which is intended to aid Radiologists, medical practitioners, and scientists in the understanding an emerging field of xAI, the black-box problem behind the success of the AI, the xAI methods to unveil the black-box into a glass-box, the role, and responsibilities of the Radiologists for appropriate use of the AI-technology. Due to the rapidly changing and evolution of AI, a definitive conclusion or solution is far away from being defined. However, one of our greatest responsibilities is to keep up with the change in a critical manner. In fact, ignoring and discrediting the advent of AI a priori will not curb its use but could result in its application without awareness. Therefore, learning and increasing our knowledge about this very important technological change will allow us to put AI at our service and at the service of the patients in a conscious way, pushing this paradigm shift as far as it will benefit us.

Quantitative optical coherence microscopy of neuron morphology in human entorhinal cortex

Introduction

The size and shape of neurons are important features indicating aging and the pathology of neurodegenerative diseases. Despite the significant advances of optical microscopy, quantitative analysis of the neuronal features in the human brain remains largely incomplete. Traditional histology on thin slices bears tremendous distortions in three-dimensional reconstruction, the magnitude of which are often greater than the structure of interest. Recently development of tissue clearing techniques enable the whole brain to be analyzed in small animals; however, the application in the human remains challenging.

Methods

In this study, we present a label-free quantitative optical coherence microscopy (OCM) technique to obtain the morphological parameters of neurons in human entorhinal cortex (EC). OCM uses the intrinsic back-scattering property of tissue to identify individual neurons in 3D. The area, length, width, and orientation of individual neurons are quantified and compared between layer II and III in EC.

Results

The high-resolution mapping of neuron size, shape, and orientation shows significant differences between layer II and III neurons in EC. The results are validated by standard Nissl staining of the same samples.

Discussion

The quantitative OCM technique in our study offers a new solution to analyze variety of neurons and their organizations in the human brain, which opens new insights in advancing our understanding of neurodegenerative diseases.

Volumetric subfield analysis of cynomolgus monkey’s choroid derived from hybrid machine learning optical coherence tomography segmentation

This study aimed to provide volumetric choroidal readings regarding sex, origin, and eye side from healthy cynomolgus monkey eyes as a reference database using optical coherence tomography (OCT) imaging. A machine learning (ML) algorithm was used to extract the choroid from the volumetric OCT data. Classical computer vision methods were then applied to automatically identify the deepest location in the foveolar depression. The choroidal thickness was determined from this reference point. A total of 374 eyes of 203 cynomolgus macaques from Asian and Mauritius origin were included in the analysis. The overall subfoveolar mean choroidal volume in zone 1, in the region of the central bouquet, was 0.156 mm³ (range, 0.131–0.193 mm³). For the central choroid volume, the coefficient of variation (CV) was found of 6.3%, indicating relatively little variation. Our results show, based on analyses of variance, that monkey origin (Asian or Mauritius) does not influence choroid volumes. Sex had a significant influence on choroidal volumes in the superior-inferior axis (p ≤ 0.01), but not in the fovea centralis. A homogeneous foveolar choroidal architecture was also observed.

Cynomolgus monkey's choroid reference database derived from hybrid deep learning optical coherence tomography segmentation

Cynomolgus monkeys exhibit human-like features, such as a fovea, so they are often used in non-clinical research. Nevertheless, little is known about the natural variation of the choroidal thickness in relation to origin and sex. A combination of deep learning and a deterministic computer vision algorithm was applied for automatic segmentation of foveolar optical coherence tomography images in cynomolgus monkeys. The main evaluation parameters were choroidal thickness and surface area directed from the deepest point on OCT images within the fovea, marked as the nulla with regard to sex and origin. Reference choroid landmarks were set underneath the nulla and at 500 µm intervals laterally up to a distance of 2000 µm nasally and temporally, complemented by a sub-analysis of the central bouquet of cones. 203 animals contributed 374 eyes for a reference choroid database. The overall average central choroidal thickness was 193 µm with a coefficient of variation of 7.8%, and the overall mean surface area of the central bouquet temporally was 19,335 µm² and nasally was 19,283 µm². The choroidal thickness of the fovea appears relatively homogeneous between the sexes and the studied origins. However, considerable natural variation has been observed, which needs to be appreciated.

Updates in deep learning research in ophthalmology

Ophthalmology has been one of the early adopters of artificial intelligence (AI) within the medical field. Deep learning (DL), in particular, has garnered significant attention due to the availability of large amounts of data and digitized ocular images. Currently, AI in Ophthalmology is mainly focused on improving disease classification and supporting decision-making when treating ophthalmic diseases such as diabetic retinopathy, age-related macular degeneration (AMD), glaucoma and retinopathy of prematurity (ROP). However, most of the DL systems (DLSs) developed thus far remain in the research stage and only a handful are able to achieve clinical translation. This phenomenon is due to a combination of factors including concerns over security and privacy, poor generalizability, trust and explainability issues, unfavorable end-user perceptions and uncertain economic value. Overcoming this challenge would require a combination approach. Firstly, emerging techniques such as federated learning (FL), generative adversarial networks (GANs), autonomous AI and blockchain will be playing an increasingly critical role to enhance privacy, collaboration and DLS performance. Next, compliance to reporting and regulatory guidelines, such as CONSORT-AI and STARD-AI, will be required to in order to improve transparency, minimize abuse and ensure reproducibility. Thirdly, frameworks will be required to obtain patient consent, perform ethical assessment and evaluate end-user perception. Lastly, proper health economic assessment (HEA) must be performed to provide financial visibility during the early phases of DLS development. This is necessary to manage resources prudently and guide the development of DLS.

Uncovering of intraspecies macular heterogeneity in cynomolgus monkeys using hybrid machine learning optical coherence tomography image segmentation

The fovea is a depression in the center of the macula and is the site of the highest visual acuity. Optical coherence tomography (OCT) has contributed considerably in elucidating the pathologic changes in the fovea and is now being considered as an accompanying imaging method in drug development, such as antivascular endothelial growth factor and its safety profiling. Because animal numbers are limited in preclinical studies and automatized image evaluation tools have not yet been routinely employed, essential reference data describing the morphologic variations in macular thickness in laboratory cynomolgus monkeys are sparse to nonexistent. A hybrid machine learning algorithm was applied for automated OCT image processing and measurements of central retina thickness and surface area values. Morphological variations and the effects of sex and geographical origin were determined. Based on our findings, the fovea parameters are specific to the geographic origin. Despite morphological similarities among cynomolgus monkeys, considerable variations in the foveolar contour, even within the same species but from different geographic origins, were found. The results of the reference database show that not only the entire retinal thickness, but also the macular subfields, should be considered when designing preclinical studies and in the interpretation of foveal data.

Deep Learning-Based Optical Coherence Tomography and Optical Coherence Tomography Angiography Image Analysis: An Updated Summary

ABSTRACT

Deep learning (DL) is a subset of artificial intelligence based on deep neural networks. It has made remarkable breakthroughs in medical imaging, particularly for image classification and pattern recognition. In ophthalmology, there are rising interests in applying DL methods to analyze optical coherence tomography (OCT) and optical coherence tomography angiography (OCTA) images. Studies showed that OCT and OCTA image evaluation by DL algorithms achieved good performance for disease detection, prognosis prediction, and image quality control, suggesting that the incorporation of DL technology could potentially enhance the accuracy of disease evaluation and the efficiency of clinical workflow. However, substantial issues, such as small training sample size, data preprocessing standardization, model robustness, results explanation, and performance cross-validation, are yet to be tackled before deploying these DL models in real-time clinics. This review summarized recent studies on DL-based image analysis models for OCT and OCTA images and discussed the potential challenges of clinical deployment and future research directions.

Unraveling the deep learning gearbox in optical coherence tomography image segmentation towards explainable artificial intelligence

Machine learning has greatly facilitated the analysis of medical data, while the internal operations usually remain intransparent. To better comprehend these opaque procedures, a convolutional neural network for optical coherence tomography image segmentation was enhanced with a Traceable Relevance Explainability (T-REX) technique. The proposed application was based on three components: ground truth generation by multiple graders, calculation of Hamming distances among graders and the machine learning algorithm, as well as a smart data visualization ('neural recording'). An overall average variability of 1.75% between the human graders and the algorithm was found, slightly minor to 2.02% among human graders. The ambiguity in ground truth had noteworthy impact on machine learning results, which could be visualized. The convolutional neural network balanced between graders and allowed for modifiable predictions dependent on the compartment. Using the proposed T-REX setup, machine learning processes could be rendered more transparent and understandable, possibly leading to optimized applications.