Image Projection Network: 3D to 2D Image Segmentation in OCTA Images

Application of medical imaging methods and artificial intelligence in tissue engineering and organ-on-a-chip

Organ-on-a-chip (OOC) is a new type of biochip technology. Various types of OOC systems have been developed rapidly in the past decade and found important applications in drug screening and precision medicine. However, due to the complexity in the structure of both the chip-body itself and the engineered-tissue inside, the imaging and analysis of OOC have still been a big challenge for biomedical researchers. Considering that medical imaging is moving towards higher spatial and temporal resolution and has more applications in tissue engineering, this paper aims to review medical imaging methods, including CT, micro-CT, MRI, small animal MRI, and OCT, and introduces the application of 3D printing in tissue engineering and OOC in which medical imaging plays an important role. The achievements of medical imaging assisted tissue engineering are reviewed, and the potential applications of medical imaging in organoids and OOC are discussed. Moreover, artificial intelligence - especially deep learning - has demonstrated its excellence in the analysis of medical imaging; we will also present the application of artificial intelligence in the image analysis of 3D tissues, especially for organoids developed in novel OOC systems.

Diagnosing Diabetic Retinopathy in OCTA Images Based on Multilevel Information Fusion Using a Deep Learning Framework

Objective

As an extension of optical coherence tomography (OCT), optical coherence tomographic angiography (OCTA) provides information on the blood flow status at the microlevel and is sensitive to changes in the fundus vessels. However, due to the distinct imaging mechanism of OCTA, existing models, which are primarily used for analyzing fundus images, do not work well on OCTA images. Effectively extracting and analyzing the information in OCTA images remains challenging. To this end, a deep learning framework that fuses multilevel information in OCTA images is proposed in this study. The effectiveness of the proposed model was demonstrated in the task of diabetic retinopathy (DR) classification.

Method

First, a U-Net-based segmentation model was proposed to label the boundaries of large retinal vessels and the foveal avascular zone (FAZ) in OCTA images. Then, we designed an isolated concatenated block (ICB) structure to extract and fuse information from the original OCTA images and segmentation results at different fusion levels.

Results

The experiments were conducted on 301 OCTA images. Of these images, 244 were labeled by ophthalmologists as normal images, and 57 were labeled as DR images. An accuracy of 93.1% and a mean intersection over union (mIOU) of 77.1% were achieved using the proposed large vessel and FAZ segmentation model. In the ablation experiment with 6-fold validation, the proposed deep learning framework that combines the proposed isolated and concatenated convolution process significantly improved the DR diagnosis accuracy. Moreover, inputting the merged images of the original OCTA images and segmentation results further improved the model performance. Finally, a DR diagnosis accuracy of 88.1% (95%CI ± 3.6%) and an area under the curve (AUC) of 0.92 were achieved using our proposed classification model, which significantly outperforms the state-of-the-art classification models. As a comparison, an accuracy of 83.7 (95%CI ± 1.5%) and AUC of 0.76 were obtained using EfficientNet. Significance. The visualization results show that the FAZ and the vascular region close to the FAZ provide more information for the model than the farther surrounding area. Furthermore, this study demonstrates that a clinically sophisticated designed deep learning model is not only able to effectively assist in the diagnosis but also help to locate new indicators for certain illnesses.

A Self-Supervised Model Advance OCTA Image Disease Diagnosis

Due to the lack of medical image datasets, transfer learning/fine-tuning is generally used to realize disease detection (mainly the ImageNet transfer model). Significant differences of dominance between natural and medical images seriously restrict the performance of the model. In this paper, a contrastive learning method (BY-OCTA) combined with patient metadata is proposed to detect the pathology in fundus OCTA images. This method uses the patient’s metadata to construct positive sample pairs. By introducing super-parameters into the loss function, we can reasonably adjust the approximate proportion of the same patient metadata sample pair, so as to produce a better representation and initialization model. This paper evaluates the performance of downstream tasks by fine-tuning the multi-layer perceptron of the model. Experiments show that the linear model pretrained by BY-OCTA is better than that pretrained by ImageNet and BYOL on multiple datasets. Furthermore, in the case of limited labeled training data, BY-OCTA provides the most significant benefit. This shows that the BY-OCTA pretraining model has better characterization extraction ability and transferability. This method allows a flexible combination of medical opinions and uses metadata to construct positive sample pairs, which can be widely used in medical image interpretation.

Linking Function and Structure with ReSensNet: Predicting Retinal Sensitivity from OCT using Deep Learning

PURPOSE

The currently used measures of retinal function are limited by being subjective, nonlocalized, or taxing for patients. To address these limitations, we sought to develop and evaluate a deep learning (DL) method to automatically predict the functional end point (retinal sensitivity) based on structural OCT images.

DESIGN

Retrospective, cross-sectional study.

SUBJECTS

In total, 714 volumes of 289 patients were used in this study.

METHODS

A DL algorithm was developed to automatically predict a comprehensive retinal sensitivity map from an OCT volume. Four hundred sixty-three spectral-domain OCT volumes from 174 patients and their corresponding microperimetry examinations (Nidek MP-1) were used for development and internal validation, with a total of 15 563 retinal sensitivity measurements. The patients presented with a healthy macula, early or intermediate age-related macular degeneration, choroidal neovascularization, or geographic atrophy. In addition, an external validation was performed using 251 volumes of 115 patients, comprising 3 different patient populations: those with diabetic macular edema, retinal vein occlusion, or epiretinal membrane.

MAIN OUTCOME MEASURES

We evaluated the performance of the algorithm using the mean absolute error (MAE), limits of agreement (LoA), and correlation coefficients of point-wise sensitivity (PWS) and mean sensitivity (MS).

RESULTS

The algorithm achieved an MAE of 2.34 dB and 1.30 dB, an LoA of 5.70 and 3.07, a Pearson correlation coefficient of 0.66 and 0.84, and a Spearman correlation coefficient of 0.68 and 0.83 for PWS and MS, respectively. In the external test set, the method achieved an MAE of 2.73 dB and 1.66 dB for PWS and MS, respectively.

CONCLUSIONS

The proposed approach allows the prediction of retinal function at each measured location directly based on an OCT scan, demonstrating how structural imaging can serve as a surrogate of visual function. Prospectively, the approach may help to complement retinal function measures, explore the association between image-based information and retinal functionality, improve disease progression monitoring, and provide objective surrogate measures for future clinical trials.

Artificial Intelligence Segmentation Algorithm-Based Optical Coherence Tomography Image in Evaluation of Binocular Retinopathy

On account of optical coherence tomography (OCT) images with intelligent segmentation algorithm, this article investigated the clinical efficacy and safety of docetaxel combined with fluorouracil. In this study, 60 patients with retinopathy treated in hospital were selected as the research objects. There were 30 cases in each group, the control group was treated with conventional images, and the observation group was treated with algorithm-based OCT images. Intelligent segmentation boundary detection algorithm, boundary tracking, and contour localization were proposed and applied to the OCT images of patients to analyze features and measure corneal thickness in OCT images with high signal-to-noise ratio and noise and artifacts. Objects in the control group were treated with semiconductor laser, and those in the observation group were treated with OCT images with algorithm in addition to the treatment of the control group. The results showed that the number of images with relative error of 2 was more, and the number of images with relative error of -2 was the least. The average thickness of high-quality images was 562.7 μm, and the average thickness of images with noise and artifacts was 573.8 μm. The total effective rate of the observation group was 96.67%, which was significantly higher than that of the control group (80%), and the curative effect and physical improvement rate of the observation group were significantly better than that of the control group (P < 0.05). All in all, the feature extraction of OCT images and corneal measurement proposed in this study had a good measurement effect, and the method had the advantages of strong anti-interference ability and high measurement accuracy.

Dual-consistency semi-supervision combined with self-supervision for vessel segmentation in retinal OCTA images

Optical coherence tomography angiography(OCTA) is an advanced noninvasive vascular imaging technique that has important implications in many vision-related diseases. The automatic segmentation of retinal vessels in OCTA is understudied, and the existing segmentation methods require large-scale pixel-level annotated images. However, manually annotating labels is time-consuming and labor-intensive. Therefore, we propose a dual-consistency semi-supervised segmentation network incorporating multi-scale self-supervised puzzle subtasks(DCSS-Net) to tackle the challenge of limited annotations. First, we adopt a novel self-supervised task in assisting semi-supervised networks in training to learn better feature representations. Second, we propose a dual-consistency regularization strategy that imposed data-based and feature-based perturbation to effectively utilize a large number of unlabeled data, alleviate the overfitting of the model, and generate more accurate segmentation predictions. Experimental results on two OCTA retina datasets validate the effectiveness of our DCSS-Net. With very little labeled data, the performance of our method is comparable with fully supervised methods trained on the entire labeled dataset.

Priors-guided convolutional neural network for 3D foveal avascular zone segmentation

The foveal avascular zone (FAZ) is sensitive to retinal pathological process in the macular fovea area. For the purpose of efficient FAZ 3D quantification, we firstly propose a priors-guided convolutional neural network (CNN) to provide a tailor-made solution for 3D FAZ segmentation for optical coherence tomography angiography (OCTA) images. Location and topology priors are taken into account. The random central crop module is utilized to restrict the region to be processed, while the non-local attention gates are contained in the network to capture long-range dependency. The topological consistency constraint is calculated on maximum and mean projection maps through persistent homology to keep topological correctness of the model's prediction. Our method was evaluated on two OCTA datasets with 478 eyes and the experimental results demonstrate that our method can not only alleviate the over-segmentation prominently but also fit better on the contour of FAZ region.

SS-net: split and spatial attention network for vessel segmentation of retinal OCT angiography

Optical coherence tomography angiography (OCTA) has been widely used in clinical fields because of its noninvasive, high-resolution qualities. Accurate vessel segmentation on OCTA images plays an important role in disease diagnosis. Most deep learning methods are based on region segmentation, which may lead to inaccurate segmentation for the extremely complex curve structure of retinal vessels. We propose a U-shaped network called SS-Net that is based on the attention mechanism to solve the problem of continuous segmentation of discontinuous vessels of a retinal OCTA. In this SS-Net, the improved SRes Block combines the residual structure and split attention to prevent the disappearance of gradient and gives greater weight to capillary features to form a backbone with an encoder and decoder architecture. In addition, spatial attention is applied to extract key information from spatial dimensions. To enhance the credibility, we use several indicators to evaluate the function of the SS-Net. In two datasets, the important indicators of accuracy reach 0.9258/0.9377, respectively, and a Dice coefficient is achieved, with an improvement of around 3% compared to state-of-the-art models in segmentation.

RPS-Net: An effective retinal image projection segmentation network for retinal vessels and foveal avascular zone based on OCTA data

BACKGROUND

Optical coherence tomography angiography (OCTA) is an advanced imaging technology that can present the three-dimensional (3D) structure of retinal vessels (RVs). Quantitative analysis of retinal vessel density and foveal avascular zone (FAZ) area is of great significance in clinical diagnosis and the automatic semantic segmentation at the pixel level helps quantitative analysis. The existing segmentation methods cannot effectively use the volume data and projection map data of the OCTA image at the same time and lack the trade-off between global perception and local details, which lead to problems such as discontinuity of segmentation results and deviation of morphological estimation.

PURPOSE

In order to better assist physicians in clinical diagnosis and treatment, the segmentation accuracy of RVs and FAZ needs to be further improved. In this work, we propose an effective retinal image projection segmentation network (RPS-Net) to achieve accurate RVs and FAZ segmentation. Experiments show that this network exhibits good performance and outperforms other existing methods.

METHODS

Our method considers three aspects. First, we use two parallel projection paths to learn global perceptual features and local supplementary details. Secondly, we use the dual-way projection learning module (DPLM) to reduce the depth of the 3D data and learn image spatial features. Finally, we merged the two-dimensional features learned from the volume data with the two-dimensional projection data, and used a U-shaped network to further learn and generate the final result.

RESULTS

We validated our model on the OCTA-500, which is a large multi-modal, multi-task retinal dataset. The experimental results showed that our method achieved state-of-the-art performance, the mean Dice coefficients for RVs are 89.89 ± 2.60 (%) and 91.40 ± 9.18 (%) on the two subsets, while the Dice coefficients for FAZ are 91.55 ± 2.05 (%) and 97.80 ± 2.75 (%), respectively.

CONCLUSIONS

Our method can make full use of the information of 3D data and 2D data to generate segmented images with higher continuity and accuracy. Code is available at https://github.com/hchuanZ/MFFN/tree/master. This article is protected by copyright. All rights reserved.

Artificial intelligence in OCT angiography

Optical coherence tomographic angiography (OCTA) is a non-invasive imaging modality that provides three-dimensional, information-rich vascular images. With numerous studies demonstrating unique capabilities in biomarker quantification, diagnosis, and monitoring, OCTA technology has seen rapid adoption in research and clinical settings. The value of OCTA imaging is significantly enhanced by image analysis tools that provide rapid and accurate quantification of vascular features and pathology. Today, the most powerful image analysis methods are based on artificial intelligence (AI). While AI encompasses a large variety of techniques, machine-learning-based, and especially deep-learning-based, image analysis provides accurate measurements in a variety of contexts, including different diseases and regions of the eye. Here, we discuss the principles of both OCTA and AI that make their combination capable of answering new questions. We also review contemporary applications of AI in OCTA, which include accurate detection of pathologies such as choroidal neovascularization, precise quantification of retinal perfusion, and reliable disease diagnosis.

Imaging depth adaptive resolution enhancement for optical coherence tomography via deep neural network with external attention

Optical coherence tomography (OCT) is a promising non-invasive imaging technique that owns many biomedical applications. In this paper, a deep neural network is proposed for enhancing the spatial resolution of OCTen faceimages. Different from the previous reports, the proposed can recover high-resolutionen faceimages from low-resolutionen faceimages at arbitrary imaging depth. This kind of imaging depth adaptive resolution enhancement is achieved through an external attention mechanism, which takes advantage of morphological similarity between the arbitrary-depth and full-depthen faceimages. Firstly, the deep feature maps are extracted by a feature extraction network from the arbitrary-depth and full-depthen faceimages. Secondly, the morphological similarity between the deep feature maps is extracted and utilized to emphasize the features strongly correlated to the vessel structures by using the external attention network. Finally, the SR image is recovered from the enhanced feature map through an up-sampling network. The proposed network is tested on a clinical skin OCT data set and an open-access retinal OCT dataset. The results show that the proposed external attention mechanism can suppress invalid features and enhance significant features in our tasks. For all tests, the proposed SR network outperformed the traditional image interpolation method, e.g. bi-cubic method, and the state-of-the-art image super-resolution networks, e.g. enhanced deep super-resolution network, residual channel attention network, and second-order attention network. The proposed method may increase the quantitative clinical assessment of micro-vascular diseases which is limited by OCT imaging device resolution.

An end-to-end network for segmenting the vasculature of three retinal capillary plexuses from OCT angiographic volumes

The segmentation of en face retinal capillary angiograms from volumetric optical coherence tomographic angiography (OCTA) usually relies on retinal layer segmentation, which is time-consuming and error-prone. In this study, we developed a deep-learning-based method to segment vessels in the superficial vascular plexus (SVP), intermediate capillary plexus (ICP), and deep capillary plexus (DCP) directly from volumetric OCTA data. The method contains a three-dimensional convolutional neural network (CNN) for extracting distinct retinal layers, a custom projection module to generate three vascular plexuses from OCTA data, and three parallel CNNs to segment vasculature. Experimental results on OCTA data from rat eyes demonstrated the feasibility of the proposed method. This end-to-end network has the potential to simplify OCTA data processing on retinal vasculature segmentation. The main contribution of this study is that we propose a custom projection module to connect retinal layer segmentation and vasculature segmentation modules and automatically convert data from three to two dimensions, thus establishing an end-to-end method to segment three retinal capillary plexuses from volumetric OCTA without any human intervention.

ROSE: A Retinal OCT-Angiography Vessel Segmentation Dataset and New Model

Optical Coherence Tomography Angiography (OCTA) is a non-invasive imaging technique that has been increasingly used to image the retinal vasculature at capillary level resolution. However, automated segmentation of retinal vessels in OCTA has been under-studied due to various challenges such as low capillary visibility and high vessel complexity, despite its significance in understanding many vision-related diseases. In addition, there is no publicly available OCTA dataset with manually graded vessels for training and validation of segmentation algorithms. To address these issues, for the first time in the field of retinal image analysis we construct a dedicated Retinal OCTA SEgmentation dataset (ROSE), which consists of 229 OCTA images with vessel annotations at either centerline-level or pixel level. This dataset with the source code has been released for public access to assist researchers in the community in undertaking research in related topics. Secondly, we introduce a novel split-based coarse-to-fine vessel segmentation network for OCTA images (OCTA-Net), with the ability to detect thick and thin vessels separately. In the OCTA-Net, a split-based coarse segmentation module is first utilized to produce a preliminary confidence map of vessels, and a split-based refined segmentation module is then used to optimize the shape/contour of the retinal microvasculature. We perform a thorough evaluation of the state-of-the-art vessel segmentation models and our OCTA-Net on the constructed ROSE dataset. The experimental results demonstrate that our OCTA-Net yields better vessel segmentation performance in OCTA than both traditional and other deep learning methods. In addition, we provide a fractal dimension analysis on the segmented microvasculature, and the statistical analysis demonstrates significant differences between the healthy control and Alzheimer's Disease group. This consolidates that the analysis of retinal microvasculature may offer a new scheme to study various neurodegenerative diseases.