Segmentation of Coronary Calcified Plaque in Intravascular OCT Images Using a Two-Step Deep Learning Approach

Deep learning model DeepNeo predicts neointimal tissue characterization using optical coherence tomography

BACKGROUND

Accurate interpretation of optical coherence tomography (OCT) pullbacks is critical for assessing vascular healing after percutaneous coronary intervention (PCI). Manual analysis is time-consuming and subjective, highlighting the need for a fully automated solution.

METHODS

In this study, 1148 frames from 92 OCT pullbacks were manually annotated to classify neointima as homogeneous, heterogeneous, neoatherosclerosis, or not analyzable on a quadrant level. Stent and lumen contours were annotated in 305 frames for segmentation of the lumen, stent struts, and neointima. We used these annotations to train a deep learning algorithm called DeepNeo. Performance was further evaluated in an animal model (male New Zealand White Rabbits) of neoatherosclerosis using co-registered histopathology images as the gold standard.

RESULTS

DeepNeo demonstrates a strong classification performance for neointimal tissue, achieving an overall accuracy of 75%, which is comparable to manual classification accuracy by two clinical experts (75% and 71%). In the animal model of neoatherosclerosis, DeepNeo achieves an accuracy of 87% when compared with histopathological findings. For segmentation tasks in human pullbacks, the algorithm shows strong performance with mean Dice overlap scores of 0.99 for the lumen, 0.66 for stent struts, and 0.86 for neointima.

CONCLUSIONS

To the best of our knowledge, DeepNeo is the first deep learning algorithm enabling fully automated segmentation and classification of neointimal tissue with performance comparable to human experts. It could standardize vascular healing assessments after PCI, support therapeutic decisions, and improve risk detection for cardiac events.

Artificial intelligence for the analysis of intracoronary optical coherence tomography images: a systematic review

Intracoronary optical coherence tomography (OCT) is a valuable tool for, among others, periprocedural guidance of percutaneous coronary revascularization and the assessment of stent failure. However, manual OCT image interpretation is challenging and time-consuming, which limits widespread clinical adoption. Automated analysis of OCT frames using artificial intelligence (AI) offers a potential solution. For example, AI can be employed for automated OCT image interpretation, plaque quantification, and clinical event prediction. Many AI models for these purposes have been proposed in recent years. However, these models have not been systematically evaluated in terms of model characteristics, performances, and bias. We performed a systematic review of AI models developed for OCT analysis to evaluate the trends and performances, including a systematic evaluation of potential sources of bias in model development and evaluation.

Plaque Characteristics Derived from Intravascular Optical Coherence Tomography That Predict Cardiovascular Death

This study aimed to investigate whether plaque characteristics derived from intravascular optical coherence tomography (IVOCT) could predict a long-term cardiovascular (CV) death. This study was a single-center, retrospective study on 104 patients who had undergone IVOCT-guided percutaneous coronary intervention. Plaque characterization was performed using Optical Coherence TOmography PlaqUe and Stent (OCTOPUS) software developed by our group. A total of 31 plaque features, including lesion length, lumen, calcium, fibrous cap (FC), and vulnerable plaque features (e.g., microchannel), were computed from the baseline IVOCT images. The discriminatory power for predicting CV death was determined using univariate/multivariate logistic regressions. Of 104 patients, CV death was identified in 24 patients (23.1%). Univariate logistic regression revealed that lesion length, calcium angle, calcium thickness, FC angle, FC area, and FC surface area were significantly associated with CV death (p < 0.05). In the multivariate logistic analysis, only the FC surface area (OR 2.38, CI 0.98-5.83, p < 0.05) was identified as a significant determinant for CV death, highlighting the importance of the 3D lesion analysis. The AUC of FC surface area for predicting CV death was 0.851 (95% CI 0.800-0.927, p < 0.05). Patients with CV death had distinct plaque characteristics (i.e., large FC surface area) in IVOCT. Studies such as this one might someday lead to recommendations for pharmaceutical and interventional approaches.

Coronary artery calcification and cardiovascular outcome as assessed by intravascular OCT and artificial intelligence

Coronary artery calcification (CAC) is a marker of atherosclerosis and is thought to be associated with worse clinical outcomes. However, evidence from large-scale high-resolution imaging data is lacking. We proposed a novel deep learning method that can automatically identify and quantify CAC in massive intravascular OCT data trained using efficiently generated sparse labels. 1,106,291 OCT images from 1,048 patients were collected and utilized to train and evaluate the method. The Dice similarity coefficient for CAC segmentation and the accuracy for CAC classification are 0.693 and 0.932, respectively, close to human-level performance. Applying the method to 1259 ST-segment elevated myocardial infarction patients imaged with OCT, we found that patients with a greater extent and more severe calcification in the culprit vessels were significantly more likely to have major adverse cardiovascular and cerebrovascular events (MACCE) (p < 0.05), while the CAC in non-culprit vessels did not differ significantly between MACCE and non-MACCE groups.

Co-registered optical coherence tomography and X-ray angiography for the prediction of fractional flow reserve

Cardiovascular disease (CVD) stands as the leading global cause of mortality, and coronary artery disease (CAD) has the highest prevalence, contributing to 42% of these fatalities. Recognizing the constraints inherent in the anatomical assessment of CAD, Fractional Flow Reserve (FFR) has emerged as a pivotal functional diagnostic metric. Herein, we assess the potential of employing an ensemble approach with deep neural networks (DNN) to predict invasively measured Fractional Flow Reserve (FFR) using raw anatomical data extracted from both optical coherence tomography (OCT) and X-ray coronary angiography (XA). In this study, we used a challenging dataset, with 46% of the lesions falling within the FFR range of 0.75 to 0.85. Despite this complexity, our model achieved an accuracy of 84.3%, demonstrating a sensitivity of 87.5% and a specificity of 81.4%. Our results demonstrate that incorporating both OCT and XA signals, co-registered, as inputs for the DNN model leads to an important increase in overall accuracy.

A transformer-based pyramid network for coronary calcified plaque segmentation in intravascular optical coherence tomography images

Characterizing coronary calcified plaque (CCP) provides essential insight into diagnosis and treatment of atherosclerosis. Intravascular optical coherence tomography (OCT) offers significant advantages for detecting CCP and even automated segmentation with recent advances in deep learning techniques. Most of current methods have achieved promising results by adopting existing convolution neural networks (CNNs) in computer vision domain. However, their performance can be detrimentally affected by unseen plaque patterns and artifacts due to inherent limitation of CNNs in contextual reasoning. To overcome this obstacle, we proposed a Transformer-based pyramid network called AFS-TPNet for robust, end-to-end segmentation of CCP from OCT images. Its encoder is built upon CSWin Transformer architecture, allowing for better perceptual understanding of calcified arteries at a higher semantic level. Specifically, an augmented feature split (AFS) module and residual convolutional position encoding (RCPE) mechanism are designed to effectively enhance the capability of Transformer in capturing both fine-grained features and global contexts. Extensive experiments showed that AFS-TPNet trained using Lovasz Loss achieved superior performance in segmentation CCP under various contexts, surpassing prior state-of-the-art CNN and Transformer architectures by more than 6.58% intersection over union (IoU) score. The application of this promising method to extract CCP features is expected to enhance clinical intervention and translational research using OCT.

Deep learning segmentation of fibrous cap in intravascular optical coherence tomography images

Thin-cap fibroatheroma (TCFA) is a prominent risk factor for plaque rupture. Intravascular optical coherence tomography (IVOCT) enables identification of fibrous cap (FC), measurement of FC thicknesses, and assessment of plaque vulnerability. We developed a fully-automated deep learning method for FC segmentation. This study included 32,531 images across 227 pullbacks from two registries (TRANSFORM-OCT and UHCMC). Images were semi-automatically labeled using our OCTOPUS with expert editing using established guidelines. We employed preprocessing including guidewire shadow detection, lumen segmentation, pixel-shifting, and Gaussian filtering on raw IVOCT (r,θ) images. Data were augmented in a natural way by changing θ in spiral acquisitions and by changing intensity and noise values. We used a modified SegResNet and comparison networks to segment FCs. We employed transfer learning from our existing much larger, fully-labeled calcification IVOCT dataset to reduce deep-learning training. Postprocessing with a morphological operation enhanced segmentation performance. Overall, our method consistently delivered better FC segmentation results (Dice: 0.837 ± 0.012) than other deep-learning methods. Transfer learning reduced training time by 84% and reduced the need for more training samples. Our method showed a high level of generalizability, evidenced by highly-consistent segmentations across five-fold cross-validation (sensitivity: 85.0 ± 0.3%, Dice: 0.846 ± 0.011) and the held-out test (sensitivity: 84.9%, Dice: 0.816) sets. In addition, we found excellent agreement of FC thickness with ground truth (2.95 ± 20.73 µm), giving clinically insignificant bias. There was excellent reproducibility in pre- and post-stenting pullbacks (average FC angle: 200.9 ± 128.0°/202.0 ± 121.1°). Our fully automated, deep-learning FC segmentation method demonstrated excellent performance, generalizability, and reproducibility on multi-center datasets. It will be useful for multiple research purposes and potentially for planning stent deployments that avoid placing a stent edge over an FC.

Validation of biomechanical assessment of coronary plaque vulnerability based on intravascular optical coherence tomography and digital subtraction angiography

Background

It has been suggested that biomechanical factors may influence plaque development. However, key determinants for assessing plaque vulnerability remain speculative.

Methods

In this study, a two-dimensional (2D) structural mechanical analysis and a three-dimensional (3D) fluid-structure interaction (FSI) analysis were conducted based on intravascular optical coherence tomography (IV-OCT) and digital subtraction angiography (DSA) data sets. In the 2D study, 103 IV-OCT slices were analyzed. An in-depth morpho-mechanic analysis and a weighted least absolute shrinkage and selection operator (LASSO) regression analysis were conducted to identify the crucial features related to plaque vulnerability via the tuning parameter (λ). In the 3D study, the coronary model was reconstructed by fusing the IV-OCT and DSA data, and a FSI analysis was subsequently performed. The relationship between vulnerable plaque and wall shear stress (WSS) was investigated.

Results

The influential factors were selected using the minimum criteria (λ-min) and one-standard error criteria (λ-1se). In addition to the common vulnerable factor of the minimum fibrous cap thickness (FCTmin), four biomechanical factors were selected by λ-min, including the average/maximal displacements and average/maximal stress, and two biomechanical factors were selected by λ-1se, including the average/maximal displacements. Additionally, the positions of the vulnerable plaques were consistent with the sites of high WSS.

Conclusions

Functional indices are crucial for plaque status assessment. An evaluation based on biomechanical simulations might provide insights into risk identification and guide therapeutic decisions.

A novel deep learning model for a computed tomography diagnosis of coronary plaque erosion

Patients with acute coronary syndromes caused by plaque erosion might be managed conservatively without stenting. Currently, the diagnosis of plaque erosion requires an invasive imaging procedure. We sought to develop a deep learning (DL) model that enables an accurate diagnosis of plaque erosion using coronary computed tomography angiography (CTA). A total of 532 CTA scans from 395 patients were used to develop a DL model: 426 CTA scans from 316 patients for training and internal validation, and 106 separate scans from 79 patients for validation. Momentum Distillation-enhanced Composite Transformer Attention (MD-CTA), a novel DL model that can effectively process the entire set of CTA scans to diagnose plaque erosion, was developed. The novel DL model, compared to the convolution neural network, showed significantly improved AUC (0.899 [0.841-0.957] vs. 0.724 [0.622-0.826]), sensitivity (87.1 [70.2-96.4] vs. 71.0 [52.0-85.8]), and specificity (85.3 [75.3-92.4] vs. 68.0 [56.2-78.3]), respectively, for the patient-level prediction. Similar results were obtained at the slice-level prediction AUC (0.897 [0.890-0.904] vs. 0.757 [0.744-0.770]), sensitivity (82.2 [79.8-84.3] vs. 68.9 [66.2-71.6]), and specificity (80.1 [79.1-81.0] vs. 67.3 [66.3-68.4]), respectively. This newly developed DL model enables an accurate CT diagnosis of plaque erosion, which might enable cardiologists to provide tailored therapy without invasive procedures.Clinical Trial Registration: http://www.clinicaltrials.gov , NCT04523194.

Prediction of stent under-expansion in calcified coronary arteries using machine learning on intravascular optical coherence tomography images

It can be difficult/impossible to fully expand a coronary artery stent in a heavily calcified coronary artery lesion. Under-expanded stents are linked to later complications. Here we used machine/deep learning to analyze calcifications in pre-stent intravascular optical coherence tomography (IVOCT) images and predicted the success of vessel expansion. Pre- and post-stent IVOCT image data were obtained from 110 coronary lesions. Lumen and calcifications in pre-stent images were segmented using deep learning, and lesion features were extracted. We analyzed stent expansion along the lesion, enabling frame, segmental, and whole-lesion analyses. We trained regression models to predict the post-stent lumen area and then computed the stent expansion index (SEI). Best performance (root-mean-square-error = 0.04 ± 0.02 mm2, r = 0.94 ± 0.04, p < 0.0001) was achieved when we used features from both lumen and calcification to train a Gaussian regression model for segmental analysis of 31 frames in length. Stents with minimum SEI > 80% were classified as "well-expanded;" others were "under-expanded." Under-expansion classification results (e.g., AUC = 0.85 ± 0.02) were significantly improved over a previous, simple calculation, as well as other machine learning solutions. Promising results suggest that such methods can identify lesions at risk of under-expansion that would be candidates for intervention lesion preparation (e.g., atherectomy).

The Role of Intracoronary Imaging for the Management of Calcified Lesions

Interventional cardiologists in everyday practice are often confronted with calcified coronary lesions indicated for percutaneous transluminal coronary angioplasty (PTCA). PTCA of calcified lesions is associated with diverse technical challenges resulting in suboptimal coronary stenting and adverse long-term clinical outcomes. Angiography itself offers limited information regarding coronary calcification, and the adjuvant use of intracoronary imaging such as intravascular ultrasound (IVUS) and Optical Coherence Tomography (OCT) can guide the treatment of calcified coronary lesions, optimizing the different stages of the procedure. This review offers a description of why, when, and how to use intracoronary imaging for PTCA of calcified coronary lesions in order to obtain the most favorable results. We used the PubMed and Google Scholar databases to search for relevant articles. Keywords were calcified coronary lesions, intracoronary imaging, IVUS, OCT, coronary calcium modification techniques, PTCA, and artificial intelligence in intracoronary imaging. A total of 192 articles were identified. Ninety-one were excluded because of repetitive or non-important information.

Pericoronary Adipose Tissue Radiomics from Coronary Computed Tomography Angiography Identifies Vulnerable Plaques

Pericoronary adipose tissue (PCAT) features on Computed Tomography (CT) have been shown to reflect local inflammation and increased cardiovascular risk. Our goal was to determine whether PCAT radiomics extracted from coronary CT angiography (CCTA) images are associated with intravascular optical coherence tomography (IVOCT)-identified vulnerable-plaque characteristics (e.g., microchannels (MC) and thin-cap fibroatheroma (TCFA)). The CCTA and IVOCT images of 30 lesions from 25 patients were registered. The vessels with vulnerable plaques were identified from the registered IVOCT images. The PCAT-radiomics features were extracted from the CCTA images for the lesion region of interest (PCAT-LOI) and the entire vessel (PCAT-Vessel). We extracted 1356 radiomic features, including intensity (first-order), shape, and texture features. The features were reduced using standard approaches (e.g., high feature correlation). Using stratified three-fold cross-validation with 1000 repeats, we determined the ability of PCAT-radiomics features from CCTA to predict IVOCT vulnerable-plaque characteristics. In the identification of TCFA lesions, the PCAT-LOI and PCAT-Vessel radiomics models performed comparably (Area Under the Curve (AUC) ± standard deviation 0.78 ± 0.13, 0.77 ± 0.14). For the identification of MC lesions, the PCAT-Vessel radiomics model (0.89 ± 0.09) was moderately better associated than the PCAT-LOI model (0.83 ± 0.12). In addition, both the PCAT-LOI and the PCAT-Vessel radiomics model identified coronary vessels thought to be highly vulnerable to a similar standard (i.e., both TCFA and MC; 0.88 ± 0.10, 0.91 ± 0.09). The most favorable radiomic features tended to be those describing the texture and size of the PCAT. The application of PCAT radiomics can identify coronary vessels with TCFA or MC, consistent with IVOCT. Furthermore, the use of CCTA radiomics may improve risk stratification by noninvasively detecting vulnerable-plaque characteristics that are only visible with IVOCT.

Toward reliable calcification detection: calibration of uncertainty in object detection from coronary optical coherence tomography images

SIGNIFICANCE

Optical coherence tomography (OCT) has become increasingly essential in assisting the treatment of coronary artery disease (CAD). However, unidentified calcified regions within a narrowed artery could impair the outcome of the treatment. Fast and objective identification is paramount to automatically procuring accurate readings on calcifications within the artery.

AIM

We aim to rapidly identify calcification in coronary OCT images using a bounding box and reduce the prediction bias in automated prediction models.

APPROACH

We first adopt a deep learning-based object detection model to rapidly draw the calcified region from coronary OCT images using a bounding box. We measure the uncertainty of predictions based on the expected calibration errors, thus assessing the certainty level of detection results. To calibrate confidence scores of predictions, we implement dependent logistic calibration using each detection result's confidence and center coordinates.

RESULTS

We implemented an object detection module to draw the boundary of the calcified region at a rate of 140 frames per second. With the calibrated confidence score of each prediction, we lower the uncertainty of predictions in calcification detection and eliminate the estimation bias from various object detection methods. The calibrated confidence of prediction results in a confidence error of ∼ 0.13 , suggesting that the confidence calibration on calcification detection could provide a more trustworthy result.

CONCLUSIONS

Given the rapid detection and effective calibration of the proposed work, we expect that it can assist in clinical evaluation of treating the CAD during the imaging-guided procedure.

OCTOPUS - Optical coherence tomography plaque and stent analysis software

Background and objective

Compared with other imaging modalities, intravascular optical coherence tomography (IVOCT) has significant advantages for guiding percutaneous coronary interventions, assessing their outcomes, and characterizing plaque components. To aid IVOCT research studies, we developed the Optical Coherence TOmography PlaqUe and Stent (OCTOPUS) analysis software, which provides highly automated, comprehensive analysis of coronary plaques and stents in IVOCT images.

Methods

User specifications for OCTOPUS were obtained from detailed, iterative discussions with IVOCT analysts in the Cardiovascular Imaging Core Laboratory at University Hospitals Cleveland Medical Center, a leading laboratory for IVOCT image analysis. To automate image analysis results, the software includes several important algorithmic steps: pre-processing, deep learning plaque segmentation, machine learning identification of stent struts, and registration of pullbacks for sequential comparisons. Intuitive, interactive visualization and manual editing of segmentations were included in the software. Quantifications include stent deployment characteristics (e.g., stent area and stent strut malapposition), strut level analysis, calcium angle, and calcium thickness measurements. Interactive visualizations include (x,y) anatomical, en face, and longitudinal views with optional overlays (e.g., segmented calcifications). To compare images over time, linked visualizations were enabled to display up to four registered vessel segments at a time.

Results

OCTOPUS has been deployed for nearly 1 year and is currently being used in multiple IVOCT studies. Underlying plaque segmentation algorithm yielded excellent pixel-wise results (86.2% sensitivity and 0.781 F1 score). Using OCTOPUS on 34 new pullbacks, we determined that following automated segmentation, only 13% and 23% of frames needed any manual touch up for detailed lumen and calcification labeling, respectively. Only up to 3.8% of plaque pixels were modified, leading to an average editing time of only 7.5 s/frame, an approximately 80% reduction compared to manual analysis. Regarding stent analysis, sensitivity and precision were both greater than 90%, and each strut was successfully classified as either covered or uncovered with high sensitivity (94%) and specificity (90%). We demonstrated use cases for sequential analysis. To analyze plaque progression, we loaded multiple pullbacks acquired at different points (e.g., pre-stent, 3-month follow-up, and 18-month follow-up) and evaluated frame-level development of in-stent neo-atherosclerosis. In ex vivo cadaver experiments, the OCTOPUS software enabled visualization and quantitative evaluation of irregular stent deployment in the presence of calcifications identified in pre-stent images.

Conclusions

We introduced and evaluated the clinical application of a highly automated software package, OCTOPUS, for quantitative plaque and stent analysis in IVOCT images. The software is currently used as an offline tool for research purposes; however, the software's embedded algorithms may also be useful for real-time treatment planning.

Pericoronary adipose tissue radiomics from coronary CT angiography identifies vulnerable plaques characteristics in intravascular OCT

Pericoronary adipose tissue (PCAT) features on CT have been shown to reflect local inflammation, and signals increased cardiovascular risk. Our goal was to determine if PCAT radiomics extracted from coronary CT angiography (CCTA) images are associated with intravascular optical coherence tomography (IVOCT)-identified vulnerable plaque characteristics (e.g., microchannels [MC] and thin-cap fibroatheroma [TCFA]). CCTA and IVOCT images of 30 lesions from 25 patients were registered. Vessels with vulnerable plaques were identified from the registered IVOCT images. PCAT radiomics features were extracted from CCTA images for the lesion region of interest (PCAT-LOI) and the entire vessel (PCAT-Vessel). We extracted 1356 radiomics features, including intensity (first-order), shape, and texture features. Features were reduced using standard approaches (e.g., high feature correlation). Using stratified three-fold cross-validation with 1000 repeats, we determined the ability of PCAT radiomics features from CCTA to predict IVOCT vulnerable plaque characteristics. In identification of TCFA lesions, PCAT-LOI and PCAT-Vessel radiomics models performed comparably (AUC±standard deviation 0.78±0.13, 0.77±0.14). For identification of MC lesions, PCAT-Vessel radiomics model (0.89±0.09) was moderately better associated than that of PCAT-LOI model (0.83±0.12). Both PCAT-LOI and PCAT-Vessel radiomics models also similarly identified coronary vessels thought to be highly vulnerable (i.e., both TCFA and MC) (0.88±0.10, 0.91±0.09). Favorable radiomics features tended to be those describing texture and size of PCAT. PCAT radiomics can identify coronary vessels with TCFA or MC, consistent with IVOCT. CCTA radiomics may improve risk stratification by noninvasively detecting vulnerable plaque characteristics that are only visible with IVOCT.

Neoatherosclerosis prediction using plaque markers in intravascular optical coherence tomography images

Introduction

In-stent neoatherosclerosis has emerged as a crucial factor in post-stent complications including late in-stent restenosis and very late stent thrombosis. In this study, we investigated the ability of quantitative plaque characteristics from intravascular optical coherence tomography (IVOCT) images taken just prior to stent implantation to predict neoatherosclerosis after implantation.

Methods

This was a sub-study of the TRiple Assessment of Neointima Stent FOrmation to Reabsorbable polyMer with Optical Coherence Tomography (TRANSFORM-OCT) trial. Images were obtained before and 18 months after stent implantation. Final analysis included images of 180 lesions from 90 patients; each patient had images of two lesions in different coronary arteries. A total of 17 IVOCT plaque features, including lesion length, lumen (e.g., area and diameter); calcium (e.g., angle and thickness); and fibrous cap (FC) features (e.g., thickness, surface area, and burden), were automatically extracted from the baseline IVOCT images before stenting using dedicated software developed by our group (OCTOPUS). The predictive value of baseline IVOCT plaque features for neoatherosclerosis development after stent implantation was assessed using univariate/multivariate logistic regression and receiver operating characteristic (ROC) analyses.

Results

Follow-up IVOCT identified stents with (n = 19) and without (n = 161) neoatherosclerosis. Greater lesion length and maximum calcium angle and features related to FC were associated with a higher prevalence of neoatherosclerosis after stent implantation (p < 0.05). Hierarchical clustering identified six clusters with the best prediction p-values. In univariate logistic regression analysis, maximum calcium angle, minimum calcium thickness, maximum FC angle, maximum FC area, FC surface area, and FC burden were significant predictors of neoatherosclerosis. Lesion length and features related to the lumen were not significantly different between the two groups. In multivariate logistic regression analysis, only larger FC surface area was strongly associated with neoatherosclerosis (odds ratio 1.38, 95% confidence interval [CI] 1.05-1.80, p < 0.05). The area under the ROC curve was 0.901 (95% CI 0.859-0.946, p < 0.05) for FC surface area.

Conclusion

Post-stent neoatherosclerosis can be predicted by quantitative IVOCT imaging of plaque characteristics prior to stent implantation. Our findings highlight the additional clinical benefits of utilizing IVOCT imaging in the catheterization laboratory to inform treatment decision-making and improve outcomes.

Automated analysis of fibrous cap in intravascular optical coherence tomography images of coronary arteries

Thin-cap fibroatheroma (TCFA) and plaque rupture have been recognized as the most frequent risk factor for thrombosis and acute coronary syndrome. Intravascular optical coherence tomography (IVOCT) can identify TCFA and assess cap thickness, which provides an opportunity to assess plaque vulnerability. We developed an automated method that can detect lipidous plaque and assess fibrous cap thickness in IVOCT images. This study analyzed a total of 4360 IVOCT image frames of 77 lesions among 41 patients. Expert cardiologists manually labeled lipidous plaque based on established criteria. To improve segmentation performance, preprocessing included lumen segmentation, pixel-shifting, and noise filtering on the raw polar (r, θ) IVOCT images. We used the DeepLab-v3 plus deep learning model to classify lipidous plaque pixels. After lipid detection, we automatically detected the outer border of the fibrous cap using a special dynamic programming algorithm and assessed the cap thickness. Our method provided excellent discriminability of lipid plaque with a sensitivity of 85.8% and A-line Dice coefficient of 0.837. By comparing lipid angle measurements between two analysts following editing of our automated software, we found good agreement by Bland-Altman analysis (difference 6.7° ± 17°; mean ~ 196°). Our method accurately detected the fibrous cap from the detected lipid plaque. Automated analysis required a significant modification for only 5.5% frames. Furthermore, our method showed a good agreement of fibrous cap thickness between two analysts with Bland-Altman analysis (4.2 ± 14.6 µm; mean ~ 175 µm), indicating little bias between users and good reproducibility of the measurement. We developed a fully automated method for fibrous cap quantification in IVOCT images, resulting in good agreement with determinations by analysts. The method has great potential to enable highly automated, repeatable, and comprehensive evaluations of TCFAs.

Automated Segmentation of Microvessels in Intravascular OCT Images Using Deep Learning

Microvessels in vascular plaque are associated with plaque progression and are found in plaque rupture and intra-plaque hemorrhage. To analyze this characteristic of vulnerability, we developed an automated deep learning method for detecting microvessels in intravascular optical coherence tomography (IVOCT) images. A total of 8403 IVOCT image frames from 85 lesions and 37 normal segments were analyzed. Manual annotation was performed using a dedicated software (OCTOPUS) previously developed by our group. Data augmentation in the polar (r,θ) domain was applied to raw IVOCT images to ensure that microvessels appear at all possible angles. Pre-processing methods included guidewire/shadow detection, lumen segmentation, pixel shifting, and noise reduction. DeepLab v3+ was used to segment microvessel candidates. A bounding box on each candidate was classified as either microvessel or non-microvessel using a shallow convolutional neural network. For better classification, we used data augmentation (i.e., angle rotation) on bounding boxes with a microvessel during network training. Data augmentation and pre-processing steps improved microvessel segmentation performance significantly, yielding a method with Dice of 0.71 ± 0.10 and pixel-wise sensitivity/specificity of 87.7 ± 6.6%/99.8 ± 0.1%. The network for classifying microvessels from candidates performed exceptionally well, with sensitivity of 99.5 ± 0.3%, specificity of 98.8 ± 1.0%, and accuracy of 99.1 ± 0.5%. The classification step eliminated the majority of residual false positives and the Dice coefficient increased from 0.71 to 0.73. In addition, our method produced 698 image frames with microvessels present, compared with 730 from manual analysis, representing a 4.4% difference. When compared with the manual method, the automated method improved microvessel continuity, implying improved segmentation performance. The method will be useful for research purposes as well as potential future treatment planning.

Towards a Deep-Learning Approach for Prediction of Fractional Flow Reserve from Optical Coherence Tomography

Cardiovascular disease (CVD) is the number one cause of death worldwide, and coronary artery disease (CAD) is the most prevalent CVD, accounting for 42% of these deaths. In view of the limitations of the anatomical evaluation of CAD, Fractional Flow Reserve (FFR) has been introduced as a functional diagnostic index. Herein, we evaluate the feasibility of using deep neural networks (DNN) in an ensemble approach to predict the invasively measured FFR from raw anatomical information that is extracted from optical coherence tomography (OCT). We evaluate the performance of various DNN architectures under different formulations: regression, classification—standard, and few-shot learning (FSL) on a dataset containing 102 intermediate lesions from 80 patients. The FSL approach that is based on a convolutional neural network leads to slightly better results compared to the standard classification: the per-lesion accuracy, sensitivity, and specificity were 77.5%, 72.9%, and 81.5%, respectively. However, since the 95% confidence intervals overlap, the differences are statistically not significant. The main findings of this study can be summarized as follows: (1) Deep-learning (DL)-based FFR prediction from reduced-order raw anatomical data is feasible in intermediate coronary artery lesions; (2) DL-based FFR prediction provides superior diagnostic performance compared to baseline approaches that are based on minimal lumen diameter and percentage diameter stenosis; and (3) the FFR prediction performance increases quasi-linearly with the dataset size, indicating that a larger train dataset will likely lead to superior diagnostic performance.

In vivo detection of plaque erosion by intravascular optical coherence tomography using artificial intelligence

Plaque erosion is one of the most common underlying mechanisms for acute coronary syndrome (ACS). Optical coherence tomography (OCT) allows in vivo diagnosis of plaque erosion. However, challenge remains due to high inter- and intra-observer variability. We developed an artificial intelligence method based on deep learning for fully automated detection of plaque erosion in vivo, which achieved a recall of 0.800 ± 0.175, a precision of 0.734 ± 0.254, and an area under the precision-recall curve (AUC) of 0.707. Our proposed method is in good agreement with physicians, and can help improve the clinical diagnosis of plaque erosion and develop individualized treatment strategies for optimal management of ACS patients.

Automated Coronary Optical Coherence Tomography Feature Extraction with Application to Three-Dimensional Reconstruction

Coronary optical coherence tomography (OCT) is an intravascular, near-infrared light-based imaging modality capable of reaching axial resolutions of 10-20 µm. This resolution allows for accurate determination of high-risk plaque features, such as thin cap fibroatheroma; however, visualization of morphological features alone still provides unreliable positive predictive capability for plaque progression or future major adverse cardiovascular events (MACE). Biomechanical simulation could assist in this prediction, but this requires extracting morphological features from intravascular imaging to construct accurate three-dimensional (3D) simulations of patients' arteries. Extracting these features is a laborious process, often carried out manually by trained experts. To address this challenge, numerous techniques have emerged to automate these processes while simultaneously overcoming difficulties associated with OCT imaging, such as its limited penetration depth. This systematic review summarizes advances in automated segmentation techniques from the past five years (2016-2021) with a focus on their application to the 3D reconstruction of vessels and their subsequent simulation. We discuss four categories based on the feature being processed, namely: coronary lumen; artery layers; plaque characteristics and subtypes; and stents. Areas for future innovation are also discussed as well as their potential for future translation.

Comprehensive Assessment of Coronary Calcification in Intravascular OCT Using a Spatial-Temporal Encoder-Decoder Network

Coronary calcification is a strong indicator of coronary artery disease and a key determinant of the outcome of percutaneous coronary intervention. We propose a fully automated method to segment and quantify coronary calcification in intravascular OCT (IVOCT) images based on convolutional neural networks (CNN). All possible calcified plaques were segmented from IVOCT pullbacks using a spatial-temporal encoder-decoder network by exploiting the 3D continuity information of the plaques, which were then screened and classified by a DenseNet network to reduce false positives. A novel data augmentation method based on the IVOCT image acquisition pattern was also proposed to improve the performance and robustness of the segmentation. Clinically relevant metrics including calcification area, depth, angle, thickness, volume, and stent-deployment calcification score, were automatically computed. 13844 IVOCT images with 2627 calcification slices from 45 clinical OCT pullbacks were collected and used to train and test the model. The proposed method performed significantly better than existing state-of-the-art 2D and 3D CNN methods. The data augmentation method improved the Dice similarity coefficient for calcification segmentation from 0.615±0.332 to 0.756±0.222, reaching human-level inter-observer agreement. Our proposed region-based classifier improved image-level calcification classification precision and F1-score from 0.725±0.071 and 0.791±0.041 to 0.964±0.002 and 0.883±0.008, respectively. Bland-Altman analysis showed close agreement between manual and automatic calcification measurements. Our proposed method is valuable for automated assessment of coronary calcification lesions and in-procedure planning of stent deployment.

Artificial Intelligence—A Good Assistant to Multi-Modality Imaging in Managing Acute Coronary Syndrome

Acute coronary syndrome is the leading cause of cardiac death and has a significant impact on patient prognosis. Early identification and proper management are key to ensuring better outcomes and have improved significantly with the development of various cardiovascular imaging modalities. Recently, the use of artificial intelligence as a method of enhancing the capability of cardiovascular imaging has grown. AI can inform the decision-making process, as it enables existing modalities to perform more efficiently and make more accurate diagnoses. This review demonstrates recent applications of AI in cardiovascular imaging to facilitate better patient care.

Automatic microchannel detection using deep learning in intravascular optical coherence tomography images

Microchannel formation is known to be a significant marker of plaque vulnerability, plaque rupture, and intraplaque hemorrhage, which are responsible for plaque progression. We developed a fully-automated method for detecting microchannels in intravascular optical coherence tomography (IVOCT) images using deep learning. A total of 3,075 IVOCT image frames across 41 patients having 62 microchannel segments were analyzed. Microchannel was manually annotated by expert cardiologists, according to previously established criteria. In order to improve segmentation performance, pre-processing including guidewire detection/removal, lumen segmentation, pixel-shifting, and noise filtering was applied to the raw (r,θ) IVOCT image. We used the DeepLab-v3 plus deep learning model with the Xception backbone network for identifying microchannel candidates. After microchannel candidate detection, each candidate was classified as either microchannel or no-microchannel using a convolutional neural network (CNN) classification model. Our method provided excellent segmentation of microchannel with a Dice coefficient of 0.811, sensitivity of 92.4%, and specificity of 99.9%. We found that pre-processing and data augmentation were very important to improve results. In addition, a CNN classification step was also helpful to rule out false positives. Furthermore, automated analysis missed only 3% of frames having microchannels and showed no false positives. Our method has great potential to enable highly automated, objective, repeatable, and comprehensive evaluations of vulnerable plaques and treatments. We believe that this method is promising for both research and clinical applications.

Recent Trends in Artificial Intelligence-Assisted Coronary Atherosclerotic Plaque Characterization

Coronary artery disease is a major cause of morbidity and mortality worldwide. Its underlying histopathology is the atherosclerotic plaque, which comprises lipid, fibrous and—when chronic—calcium components. Intravascular ultrasound (IVUS) and intravascular optical coherence tomography (IVOCT) performed during invasive coronary angiography are reference standards for characterizing the atherosclerotic plaque. Fine image spatial resolution attainable with contemporary coronary computed tomographic angiography (CCTA) has enabled noninvasive plaque assessment, including identifying features associated with vulnerable plaques known to presage acute coronary events. Manual interpretation of IVUS, IVOCT and CCTA images demands scarce physician expertise and high time cost. This has motivated recent research into and development of artificial intelligence (AI)-assisted methods for image processing, feature extraction, plaque identification and characterization. We performed parallel searches of the medical and technical literature from 1995 to 2021 focusing respectively on human plaque characterization using various imaging modalities and the use of AI-assisted computer aided diagnosis (CAD) to detect and classify atherosclerotic plaques, including their composition and the presence of high-risk features denoting vulnerable plaques. A total of 122 publications were selected for evaluation and the analysis was summarized in terms of data sources, methods—machine versus deep learning—and performance metrics. Trends in AI-assisted plaque characterization are detailed and prospective research challenges discussed. Future directions for the development of accurate and efficient CAD systems to characterize plaque noninvasively using CCTA are proposed.