A new nonparametric statistical approach to detect lumen and Media-Adventitia borders in intravascular ultrasound frames

Transformer and Attention-Based Architectures for Segmentation of Coronary Arterial Walls in Intravascular Ultrasound: A Narrative Review

Background: The leading global cause of death is coronary artery disease (CAD), necessitating early and precise diagnosis. Intravascular ultrasound (IVUS) is a sophisticated imaging technique that provides detailed visualization of coronary arteries. However, the methods for segmenting walls in the IVUS scan into internal wall structures and quantifying plaque are still evolving. This study explores the use of transformers and attention-based models to improve diagnostic accuracy for wall segmentation in IVUS scans. Thus, the objective is to explore the application of transformer models for wall segmentation in IVUS scans to assess their inherent biases in artificial intelligence systems for improving diagnostic accuracy. Methods: By employing the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework, we pinpointed and examined the top strategies for coronary wall segmentation using transformer-based techniques, assessing their traits, scientific soundness, and clinical relevancy. Coronary artery wall thickness is determined by using the boundaries (inner: lumen-intima and outer: media-adventitia) through cross-sectional IVUS scans. Additionally, it is the first to investigate biases in deep learning (DL) systems that are associated with IVUS scan wall segmentation. Finally, the study incorporates explainable AI (XAI) concepts into the DL structure for IVUS scan wall segmentation. Findings: Because of its capacity to automatically extract features at numerous scales in encoders, rebuild segmented pictures via decoders, and fuse variations through skip connections, the UNet and transformer-based model stands out as an efficient technique for segmenting coronary walls in IVUS scans. Conclusions: The investigation underscores a deficiency in incentives for embracing XAI and pruned AI (PAI) models, with no UNet systems attaining a bias-free configuration. Shifting from theoretical study to practical usage is crucial to bolstering clinical evaluation and deployment.

POLYCORE: Polygon-based contour refinement for improved Intravascular Ultrasound Segmentation

Segmentation of the coronary vessel wall in intravascular ultrasound is a fundamental step in guiding coronary intervention. However, it is an challenging task, even for highly skilled cardiologists, due to image artefacts and shadowed regions caused by calcified plaque, guide wires and vessel side branches. Recently, dense-based neural networks have been applied to this task, however, they often fail to predict anatomically plausible contours in these low-signal areas. We propose a novel methodology called Polygon-based Contour Refiner (POLYCORE) that addresses topological error in dense-based segmentation networks using a relational inductive bias through higher-order connections between vertices to learn anatomically rational contours. Our approach remedies the over-smoothing phenomena common in polygon networks by introducing a new vector field refinement module which enables pixel-level detail to be added in an iterative process. POLYCORE is enhanced with augmented polygon aggregation which we show is more effective than typical dense-based test-time augmentation strategies. We achieve state-of-the-art results on two diverse datasets, observing particular improvements when segmenting the lumen structure and in topologically-challenging regions containing shadow artefacts. Our source code is available here: https://github.com/kitbransby/POLYCORE.

Distraction-aware hierarchical learning for vascular structure segmentation in intravascular ultrasound images

Vascular structure segmentation in intravascular ultrasound (IVUS) images plays an important role in pre-procedural evaluation of percutaneous coronary intervention (PCI). However, vascular structure segmentation in IVUS images has the challenge of structure-dependent distractions. Structure-dependent distractions are categorized into two cases, structural intrinsic distractions and inter-structural distractions. Traditional machine learning methods often rely solely on low-level features, overlooking high-level features. This way limits the generalization of these methods. The existing semantic segmentation methods integrate low-level and high-level features to enhance generalization performance. But these methods also introduce additional interference, which is harmful to solving structural intrinsic distractions. Distraction cue methods attempt to address structural intrinsic distractions by removing interference from the features through a unique decoder. However, they tend to overlook the problem of inter-structural distractions. In this paper, we propose distraction-aware hierarchical learning (DHL) for vascular structure segmentation in IVUS images. Inspired by distraction cue methods for removing interference in a decoder, the DHL is designed as a hierarchical decoder that gradually removes structure-dependent distractions. The DHL includes global perception process, distraction perception process and structural perception process. The global perception process and distraction perception process remove structural intrinsic distractions then the structural perception process removes inter-structural distractions. In the global perception process, the DHL searches for the coarse structural region of the vascular structures on the slice of IVUS sequence. In the distraction perception process, the DHL progressively refines the coarse structural region of the vascular structures to remove structural distractions. In the structural perception process, the DHL detects regions of inter-structural distractions in fused structure features then separates them. Extensive experiments on 361 subjects show that the DHL is effective (e.g., the average Dice is greater than 0.95), and superior to ten state-of-the-art IVUS vascular structure segmentation methods.

POST-IVUS: A perceptual organisation-aware selective transformer framework for intravascular ultrasound segmentation

Intravascular ultrasound (IVUS) is recommended in guiding coronary intervention. The segmentation of coronary lumen and external elastic membrane (EEM) borders in IVUS images is a key step, but the manual process is time-consuming and error-prone, and suffers from inter-observer variability. In this paper, we propose a novel perceptual organisation-aware selective transformer framework that can achieve accurate and robust segmentation of the vessel walls in IVUS images. In this framework, temporal context-based feature encoders extract efficient motion features of vessels. Then, a perceptual organisation-aware selective transformer module is proposed to extract accurate boundary information, supervised by a dedicated boundary loss. The obtained EEM and lumen segmentation results will be fused in a temporal constraining and fusion module, to determine the most likely correct boundaries with robustness to morphology. Our proposed methods are extensively evaluated in non-selected IVUS sequences, including normal, bifurcated, and calcified vessels with shadow artifacts. The results show that the proposed methods outperform the state-of-the-art, with a Jaccard measure of 0.92 for lumen and 0.94 for EEM on the IVUS 2011 open challenge dataset. This work has been integrated into a software QCU-CMS2 to automatically segment IVUS images in a user-friendly environment.

A State-Of-The-Art Review on Coronary Artery Border Segmentation Algorithms for Intravascular Ultrasound (IVUS) Images

Intravascular Ultrasound images (IVUS) is a useful guide for medical practitioners to identify the vascular status of coronary arteries in human beings. IVUS is a unique intracoronary imaging modality that is used as an adjunct to angioplasty to view vessel structures using a catheter with high resolutions. Segmentation of IVUS images has always remained a challenging task due to various impediments, for example, similar tissue components, vessel structures, and artifacts imposed during the acquisition process. Many researchers have applied various techniques to develop standard methods of image interpretation, however, the ultimate goal is still elusive to most researchers. This challenge was presented at the MICCAI- Computing and Visualization for (Intra)Vascular Imaging (CVII) workshop in 2011. This paper presents a major review of recently reported work in the field, with a detailed analysis of various segmentation techniques applied in IVUS, and highlights the directions for future research. The findings recommend a reference database with a larger number of samples acquired at varied transducer frequencies with special consideration towards complex lesions, suitable validation metrics, and ground-truth definition as a standard against which to compare new and current algorithms.

Multilevel structure-preserved GAN for domain adaptation in intravascular ultrasound analysis

The poor generalizability of intravascular ultrasound (IVUS) analysis methods caused by the great diversity of IVUS datasets is hopefully addressed by the domain adaptation strategy. However, existing domain adaptation models underperform in intravascular structural preservation, because of the complex pathology and low contrast in IVUS images. Losing structural information during the domain adaptation would lead to inaccurate analyses of vascular states. In this paper, we propose a Multilevel Structure-Preserved Generative Adversarial Network (MSP-GAN) for transferring IVUS domains while maintaining intravascular structures. On the generator-discriminator baseline, the MSP-GAN integrates the transformer, contrastive restraint, and self-ensembling strategy, for effectively preserving structures in multi-levels, including global, local, and fine levels. For the global-level pathology maintenance, the generator explores long-range dependencies in IVUS images via an incorporated vision transformer. For the local-level anatomy consistency, a region-to-region correspondence is forced between the translated and source images via a superpixel-wise multiscale contrastive (SMC) constraint. For reducing distortions of fine-level structures, a self-ensembling mean teacher generates the pixel-wise pseudo-label and restricts the translated image via an uncertainty-aware teacher-student consistency (TSC) constraint. Experiments were conducted on 20 MHz and 40 MHz IVUS datasets from different medical centers. Ablation studies illustrate that each innovation contributes to intravascular structural preservation. Comparisons with representative domain adaptation models illustrate the superiority of the MSP-GAN in the structural preservation. Further comparisons with the state-of-the-art IVUS analysis accuracy demonstrate that the MSP-GAN is effective in enlarging the generalizability of diverse IVUS analysis methods and promoting accurate vessel and lumen segmentation and stenosis-related parameter quantification.

Capsule networks for segmentation of small intravascular ultrasound image datasets

PURPOSE

Intravascular ultrasound (IVUS) imaging is crucial for planning and performing percutaneous coronary interventions. Automatic segmentation of lumen and vessel wall in IVUS images can thus help streamlining the clinical workflow. State-of-the-art results in image segmentation are achieved with data-driven methods like convolutional neural networks (CNNs). These need large amounts of training data to perform sufficiently well but medical image datasets are often rather small. A possibility to overcome this problem is exploiting alternative network architectures like capsule networks.

METHODS

We systematically investigated different capsule network architecture variants and optimized the performance on IVUS image segmentation. We then compared our capsule network with corresponding CNNs under varying amounts of training images and network parameters.

RESULTS

Contrary to previous works, our capsule network performs best when doubling the number of capsule types after each downsampling stage, analogous to typical increase rates of feature maps in CNNs. Maximum improvements compared to the baseline CNNs are 20.6% in terms of the Dice coefficient and 87.2% in terms of the average Hausdorff distance.

CONCLUSION

Capsule networks are promising candidates when it comes to segmentation of small IVUS image datasets. We therefore assume that this also holds for ultrasound images in general. A reasonable next step would be the investigation of capsule networks for few- or even single-shot learning tasks.

Extraction of media adventitia and luminal intima borders by reconstructing intravascular ultrasound image sequences with vascular structural continuity

PURPOSE

Most published methods directly achieve vessel membrane border detection on cross-sectional intravascular ultrasound (IVUS) images. The vascular structural continuity that exists in entire IVUS image sequences has been overlooked. However, this continuity can have a helpful role in the delineation of vessel membrane contours. To achieve the vessel membrane segmentation more effectively through employing this continuity, a strategy, referred to as multiangle reconstruction, segmentation, and recovery (RSR), is proposed in this paper.

METHODS

Four main steps are contained in the multiangle-RSR: first, a combination of sampling and interpolation is employed to reconstruct long-axis-model IVUS frames, in which continuity information becomes available. Second, a clustering algorithm is conducted on long-axis-model IVUS frames to roughly extract the media-adventitia (MA) and lumen-intima (LI) boundaries. Third, the segmentation results of cross-sectional IVUS frames are recovered based on the rough results of long-axis-model IVUS frames, and an optimization process that combines downsampling, fitting and smoothing is designed to reduce the interference of bifurcation and side vessels.

RESULTS

Multiangle-RSR is tested on a public dataset, and the Hausdorff distance (HD), Jaccard measure (JM), and percentage of area difference (PAD) are utilized as quantitative evaluation metrics. Mean HDs of 0.34 and 0.29 mm are obtained for MA border detection and LI border detection, respectively, which decrease by 43.3% and 9.4%, respectively, compared with their counterparts in previously published approaches. Furthermore, the mean JM is 0.87 for both MA border detection and LI border detection. The mean PADs of the MA contour extraction and the LI contour extraction are 0.10 and 0.11, respectively.

CONCLUSION

The results indicate that the proposed strategy effectively introduces vascular structural continuity by reconstructing long-axis-model IVUS frames and achieves more precise extraction of MA and LI borders.

Automatic Lumen Border Detection in IVUS Images Using Deep Learning Model and Handcrafted Features

In the clinical analysis of Intravascular ultrasound (IVUS) images, the lumen size is an important indicator of coronary atherosclerosis, and is also the premise of coronary artery disease diagnosis and interventional treatment. In this study, a fully automatic method based on deep learning model and handcrafted features is presented for the detection of the lumen borders in IVUS images. First, 193 handcrafted features are extracted from the IVUS images. Then hybrid feature vectors are constructed by combining handcrafted features with 64 high-level features extracted from U-Net. In order to obtain the feature subsets with larger contribution, we employ the extended binary cuckoo search for feature selection. Finally, the selected 36-dimensional hybrid feature subset is used to classify the test images using dictionary learning based on kernel sparse coding. The proposed algorithm is tested on the publicly available dataset and evaluated using three indicators. Through ablation experiments, mean value of the experimental results (Jaccard: 0.88, Hausdorff distance: 0.36, Percentage of the area difference: 0.06) prove to be effective improving lumen border detection. Furthermore, compared with the recent methods used on the same dataset, the proposed method shows good performance and high accuracy.

Vessel membrane segmentation and calcification location in intravascular ultrasound images using a region detector and an effective selection strategy

BACKGROUND AND OBJECTIVE

Segmenting vessel membranes and locating the calcific region in intravascular ultrasound (IVUS) images aid physicians in the diagnosis of atherosclerosis. However, the manual extraction of the media adventitia (MA)/lumen border and calcification location are cumbersome due to the excessive number of IVUS frames. Moreover, most existing (semi-)automatic detection methods cannot achieve both vessel membrane extraction and calcification location simultaneously, and they are unable to detect vessel membranes in IVUS frames from different acquisition systems.

METHOD

A fully automatic approach is proposed based on extremal regions and a flexible selection strategy to extract vessel membranes in different IVUS frames and locate the calcific region in high-frequency ones. Three main steps are included in the algorithm. First, a region detector is employed to extract extremal regions from an IVUS image. Then, according to the selection strategy, a part of the extracted regions is selected. At the same time, the calcification is located according to its special acoustic properties. Next, approximate MA and lumen border segmentation is achieved based on the selected extremal regions and the located calcification in polar coordinates. Finally, the final segmentation results are obtained by smoothing the approximate values.

RESULT

To demonstrate the feasibility of the method, it was evaluated based on a standard public dataset. Furthermore, to quantitatively evaluate the segmentation performance, the Hausdorff distance (HD), Jaccard measure (JM) and percentage of area difference (PAD) were used. The results show that a mean HD of 1.13/1.21 mm, a mean JM of 0.83/0.77 and a mean PAD of 0.11/0.23 are achieved for MA/lumen border detection in 77 40-MHz IVUS images. For MA/lumen border extraction in 435 20-MHz IVUS frames, the average HD, JM and PAD values are 0.47/0.28 mm, 0.84/0.89 and 0.13/0.10, respectively. In addition, the approach successfully achieves calcification location in 40-MHz IVUS frames. In comparison with other published methods, the method proposed in this study is competitive.

CONCLUSION

According to these results, our strategy can extract MA/lumen borders in different IVUS frames and effectively locate calcification in high-frequency IVUS frames.

Privileged Modality Distillation for Vessel Border Detection in Intracoronary Imaging

Intracoronary imaging is a crucial imaging technology in coronary disease diagnosis as it visualizes the internal tissue morphologies of coronary arteries. Vessel border detection in intracoronary images (VBDI) is desired because it can help the succeeding procedures of computer-aided disease diagnosis. However, existing VDBI methods suffer from the challenge of vessel-environment variability (i.e. high intra- and inter-subject diversity of vessels and their surrounding tissues appeared in images). This challenge leads to the ineffectiveness in the vessel region representation for hand-crafted features, in the receptive field extraction for deeply-represented features, as well as performance suppression derived from clinical data limitation. To solve this challenge, we propose a novel privileged modality distillation (PMD) framework for VBDI. PMD transforms the single-input-single-task (SIST) learning problem in the single-mode VBDI to a multiple-input-multiple-task (MIMT) problem by using the privileged image modality to help the learning model in the target modality. This learns the enriched high-level knowledge with similar semantics and generalizes PMD on diversity-increased low-level image features for improving the model adaptation to diverse vessel environments. Moreover, PMD refines MIMT to SIST by distilling the learned knowledge from multiple to one modality. This eliminates the reliance on privileged modality in the test phase, and thus enables the applicability to each of different intracoronary modalities. A structure-deformable neural network is proposed as an elaborately-designed implementation of PMD. It expands a conventional SIST network structure to the MIMT structure, and then recovers it to the final SIST structure. The PMD is validated on intravascular ultrasound imaging and optical coherence tomography imaging. One modality is the target, and the other one can be considered as the privileged modality owing to their semantic relatedness. The experiments show that our PMD is effective in VBDI (e.g. the Dice index is larger than 0.95), as well as superior to six state-of-the-art VBDI methods.