A novel framework for differentiating vessel-like objects in coronarography images

Coronary Artery Disease is the leading cause of death worldwide. Its prevalence will grow while access to specialized medical care will be further limited due to staff shortages. Therefore, any facilitation of diagnosis or treatment is of paramount importance. The diagnosis based on Coronary Angiography can be automated to perform a quantitative evaluation of lesions. This requires precise segmentation of coronary arteries. At the moment, the state-of-the-art algorithms fail to eliminate vessel-like artifacts that are wrongly included in segmentation results (e.g. catheters, stitches). This is a bottleneck for the automatization of the diagnosis workflow that precedes clinical action. In this paper, we propose a 2-step post-segmentation refinement algorithm. A binary segmentation of the coronary arteries is used to extract image features - inputs for an XGBoost Classifier. Its predictions are improved by a neighborhood filter that leverages contextual information to assign correct labels. The algorithm is primarily concerned with differentiating vessels from other vessel-like objects and does so with a 99% accuracy rate. It takes advantage of an original local description of Tamura features, which proved to be one of the most influential factors in decision-making. As a result, the segmentation of coronary arteries is cleaned from artifacts, enabling AI-supported diagnosis workflows to be automated. After re-training, the proposed method can be used to eliminate post-segmentation artifacts in other medical domains.Clinical relevance- The algorithm proposed in this paper allows for the development of software that could automatically calculate the Syntax Score in real time. This would shorten diagnostics time and allow for immediate action in critical cases.

Deep learning-based end-to-end automated stenosis classification and localization on catheter coronary angiography

Background

Automatic coronary angiography (CAG) assessment may help in faster screening and diagnosis of stenosis in patients with atherosclerotic disease. We aimed to provide an end-to-end workflow that separates cases with normal or mild stenoses from those with higher stenosis severities to facilitate safety screening of a large volume of the CAG images.

Methods

A deep learning-based end-to-end workflow was employed as follows: (1) Candidate frame selection from CAG videograms with Convolutional Neural Network (CNN) + Long Short Term Memory (LSTM) network, (2) Stenosis classification with Inception-v3 using 2 or 3 categories (<25%, >25%, and/or total occlusion) with and without redundancy training, and (3) Stenosis localization with two methods of class activation map (CAM) and anchor-based feature pyramid network (FPN). Overall 13,744 frames from 230 studies were used for the stenosis classification training and fourfold cross-validation for image-, artery-, and per-patient-level. For the stenosis localization training and fourfold cross-validation, 690 images with > 25% stenosis were used.

Results

Our model achieved an accuracy of 0.85, sensitivity of 0.96, and AUC of 0.86 in per-patient level stenosis classification. Redundancy training was effective to improve classification performance. Stenosis position localization was adequate with better quantitative results in anchor-based FPN model, achieving global-sensitivity for left coronary artery (LCA) and right coronary artery (RCA) of 0.68 and 0.70.

Conclusion

We demonstrated a fully automatic end-to-end deep learning-based workflow that eliminates the vessel extraction and segmentation step in coronary artery stenosis classification and localization on CAG images. This tool may be useful to facilitate safety screening in high-volume centers and in clinical trial settings.

Towards real time guide wire shape extraction in fluoroscopic sequences: A two phase deep learning scheme to extract sparse curvilinear structures

BACKGROUND AND OBJECTIVE

Real time localization and shape extraction of guide wire in fluoroscopic images plays a significant role in the image guided navigation during cerebral and cardiovascular interventions. Given the complexity of the non-rigid and sparse characteristics of guide wire structures, and the low SNR(Signal Noise Ratio) of fluoroscopic images, traditional handcrafted guide wire tracking methods such as Frangi filter, Hessian Matrix, or open active contour usually produce insufficient accuracy with high computational cost, and may require extra human intervention for proper initialization or correction. The application of deep learning techniques to guide wire tracking is reported to produce significant improvement in guide wire localization accuracy, but the heavy calculation cost is still a concern.

METHOD

In this paper we propose a two phase deep learning scheme for accurate and real time guide wire shape extraction in fluoroscopic sequences. In the first phase we train a guide wire localization network to pick image regions containing guide wire structures. From the picked image regions, we train a guide wire shape extraction network in the second phase to mark the guide wire pixels.

RESULTS

We report that our proposed method can accurately distinguish about 99% of the guide wire structure pixels, and the falsely detected pixels in the background are close to 0, the average offset from the ground truth is less than 1 pixel. For extreme cases where traditional handcrafted method may fail, our proposed method can still extract guide wire completely and accurately. The processing time for a 512 × 512 pixels image is 78 ms.

CONCLUSION

Compared with the traditional filtering based method from our previous work, we show that our proposed method can achieve more accurate and stable performance. Compared with other deep learning methods, our proposed method significantly improve calculation efficiency to meet the real time requirement of clinical applications.

AngioNet: a convolutional neural network for vessel segmentation in X-ray angiography

Coronary Artery Disease (CAD) is commonly diagnosed using X-ray angiography, in which images are taken as radio-opaque dye is flushed through the coronary vessels to visualize the severity of vessel narrowing, or stenosis. Cardiologists typically use visual estimation to approximate the percent diameter reduction of the stenosis, and this directs therapies like stent placement. A fully automatic method to segment the vessels would eliminate potential subjectivity and provide a quantitative and systematic measurement of diameter reduction. Here, we have designed a convolutional neural network, AngioNet, for vessel segmentation in X-ray angiography images. The main innovation in this network is the introduction of an Angiographic Processing Network (APN) which significantly improves segmentation performance on multiple network backbones, with the best performance using Deeplabv3+ (Dice score 0.864, pixel accuracy 0.983, sensitivity 0.918, specificity 0.987). The purpose of the APN is to create an end-to-end pipeline for image pre-processing and segmentation, learning the best possible pre-processing filters to improve segmentation. We have also demonstrated the interchangeability of our network in measuring vessel diameter with Quantitative Coronary Angiography. Our results indicate that AngioNet is a powerful tool for automatic angiographic vessel segmentation that could facilitate systematic anatomical assessment of coronary stenosis in the clinical workflow.

Automatic tool segmentation and tracking during robotic intravascular catheterization for cardiac interventions

Background

Cardiovascular diseases resulting from aneurism, thrombosis, and atherosclerosis in the cardiovascular system are major causes of global mortality. Recent treatment methods have been based on catheterization of flexible endovascular tools with imaging guidance. While advances in robotic intravascular catheterization have led to modeling tool navigation approaches with data sensing and feedback, proper adaptation of image-based guidance for robotic navigation requires the development of sensitive segmentation and tracking models without specificity loss. Several methods have been developed to tackle non-uniform illumination, low contrast; however, presence of untargeted body organs commonly found in X-ray frames taken during angiography procedures still presents some major issues to be solved.

Methods

In this study, a segmentation method was developed for automatic detection and tracking of guidewire pixels in X-ray angiograms. Image frames were acquired during robotic intravascular catheterization for cardiac interventions. For segmentation, multiscale enhancement filtering was applied on preprocessed X-ray angiograms, while morphological operations and filters were applied to refine the frames for pixel intensity adjustment and vesselness measurement. Minima and maxima extrema of the pixels were obtained to detect guidewire pixels in the X-ray frames. Lastly, morphological operation was applied for guidewire pixel connectivity and tracking in segmented pixels. Method validation was performed on 12 X-ray angiogram sequences which were acquired during in vivo intravascular catheterization trials in rabbits.

Results

The study outcomes showed that an overall accuracy of 0.995±0.001 was achieved for segmentation. Tracking performance was characterized with displacement and orientation errors observed as 1.938±2.429 mm and 0.039±0.040°, respectively. Evaluation studies performed against 9 existing methods revealed that this proposed method provides more accurate segmentation with 0.753±0.074 area under curve. Simultaneously, high tracking accuracy of 0.995±0.001 with low displacement and orientation errors of 1.938±2.429 mm and 0.039±0.040°, respectively, were achieved. Also, the method demonstrated higher sensitivity and specificity values compared to the 9 existing methods, with a relatively faster exaction time.

Conclusions

The proposed method has the capability to enhance robotic intravascular catheterization during percutaneous coronary interventions (PCIs). Thus, interventionists can be provided with better tool tracking and visualization systems while also reducing their exposure to operational hazards during intravascular catheterization for cardiac interventions.

Automatic stenosis recognition from coronary angiography using convolutional neural networks

BACKGROUND AND OBJECTIVE

Coronary artery disease, which is mostly caused by atherosclerotic narrowing of the coronary artery lumen, is a leading cause of death. Coronary angiography is the standard method to estimate the severity of coronary artery stenosis, but is frequently limited by intra- and inter-observer variations. We propose a deep-learning algorithm that automatically recognizes stenosis in coronary angiographic images.

METHODS

The proposed method consists of key frame detection, deep learning model training for classification of stenosis on each key frame, and visualization of the possible location of the stenosis. Firstly, we propose an algorithm that automatically extracts key frames essential for diagnosis from 452 right coronary artery angiography movie clips. Our deep learning model is then trained with image-level annotations to classify the areas narrowed by over 50 %. To make the model focus on the salient features, we apply a self-attention mechanism. The stenotic locations are visualized using the activated area of feature maps with gradient-weighted class activation mapping.

RESULTS

The automatically detected key frame was very close to the manually selected key frame (average distance (1.70 ± 0.12) frame per clip). The model was trained with key frames on internal datasets, and validated with internal and external datasets. Our training method achieved high frame-wise area-under-the-curve of 0.971, frame-wise accuracy of 0.934, and clip-wise accuracy of 0.965 in the average values of cross-validation evaluations. The external validation results showed high performances with the mean frame-wise area-under-the-curve of (0.925 and 0.956) in the single and ensemble model, respectively. Heat map visualization shows the location for different types of stenosis in both internal and external data sets. With the self-attention mechanism, the stenosis could be precisely localized, which helps to accurately classify the stenosis by type.

CONCLUSIONS

Our automated classification algorithm could recognize and localize coronary artery stenosis highly accurately. Our approach might provide the basis for a screening and assistant tool for the interpretation of coronary angiography.

An automatic multi-class coronary atherosclerosis plaque detection and classification framework

Detection of different classes of atherosclerotic plaques is important for early intervention of coronary artery diseases. However, previous methods focused either on the detection of a specific class of coronary plaques or on the distinction between plaques and normal arteries, neglecting the classification of different classes of plaques. Therefore, we proposed an automatic multi-class coronary atherosclerosis plaque detection and classification framework. Firstly, we retrieved the transverse cross sections along centerlines from the computed tomography angiography. Secondly, we extracted the region of interests based on coarse segmentation. Thirdly, we extracted a random radius symmetry (RRS) feature vector, which incorporates multiple descriptions into a random strategy and greatly augments the training data. Finally, we fed the RRS feature vector into the multi-class coronary plaque classifier. In experiments, we compared our proposed framework with other methods on the cross sections of Rotterdam Coronary Datasets, including 729 non-calcified plaques, 511 calcified plaques, and 546 mixed plaques. Our RRS with support vector machine outperforms the intensity feature vector and the random forest classifier, with the average precision of 92.6 ± 1.9% and average recall of 94.3 ± 2.1%. The proposed framework provides a computer-aided diagnostic method for multi-class plaque detection and classification. Graphical abstract Diagram of the proposed automatic multi-class coronary atherosclerosis plaque detection and classification framework. ᅟ.

Vessel segmentation and catheter detection in X-ray angiograms using superpixels

Coronary artery disease (CAD) is the leading cause of death around the world. One of the most common imaging methods for diagnosing CAD is the X-ray angiography (XRA). Diagnosing using XRA images is usually challenging due to some reasons such as, non-uniform illumination, low contrast, presence of other body tissues, and presence of catheter. These challenges make the diagnosis task hard and more prone to misdiagnosis. In this paper, we propose a new method for coronary artery segmentation, catheter detection, and centerline extraction in X-ray angiography images. For the segmentation, initially, three different superpixel scales are exploited, and a measure for vesselness probability of each superpixel is determined. A voting mechanism is used for obtaining an initial segmentation map from the three superpixel scales. The initial segmentation is refined by finding the orthogonal line on each ridge pixel of vessel region. The catheter is detected in the first frame of the angiography sequence and is tracked in other frames by fitting a second order polynomial on it. Also, we use the image ridges for extracting the coronary artery centerlines. We evaluated and compared our method with one of the previous well-known coronary artery segmentation methods on two challenging datasets. The results show that our method can segment the vessels and also detect and track the catheter in the XRA sequences. In general, the results assessed by a cardiologist show that 83% of the images processed by our proposed segmentation method were labeled as good or excellent, while this score for the compared method is 48%. Also, the evaluation results show that our method performs 67% faster than the compared method. Graphical abstract Proposed framework for coronary artery detection.

A graph-based approach for spatio-temporal segmentation of coronary arteries in X-ray angiographic sequences

The segmentation and tracking of coronary arteries (CAs) are critical steps for the computation of biophysical measurements in pediatric interventional cardiology. In the literature, most methods are focused on either segmenting the vessel lumen or on tracking the vessel centerline. However, they do not simultaneously combine the segmentation and tracking of a specific CA. This paper introduces a novel algorithm for CA segmentation and tracking from 2D X-ray angiography sequences. The proposed algorithm is based on the Temporal Vessel Walker (TVW) segmentation method, which combines graph-based formulation and temporal priors. Moreover, superpixel groups are used by TVW as image primitives to ensure a better extraction of the CA. The proposed algorithm, TVW with superpixels (SP-TVW), returns an accurate result to segment and track the artery along the angiogram. Quantitative results over 12 sequences of young patients show the accuracy of the proposed framework. The results return a mean recall of 84% in the dataset. In addition, the proposed method returned a Dice index of 70% in segmenting and tracking right coronary arteries and circumflex arteries. The performance of the proposed method surpasses the existing polyline method in tracking the centerline of CA with a more precise localization of the centerline, resulting in a smaller distance error of 0.23mm compared to 0.94mm.

Robust catheter identification and tracking in X-ray angiographic sequences

Coronary artery disease (CAD) is one of the major causes of death worldwide. Today X-ray angiography is a standard method for CAD diagnosis. Usually, the quality of these images is not good enough. Noise, camera and heart motions, non-uniform illumination and even the presence of catheter are sources of quality degradation. The existence of catheter can produce difficulties in vessel extraction methods because catheter is structurally similar to arteries. In this paper we propose a fully automatic method for catheter detection and tracking during the whole angiography sequence. In this method with a vesselness map, we smooth each frame using guided filter. The catheter is detected in the first frame using Hough transform. We then fit a second order polynomial on the catheter and accurately track it throughout the sequence. Our method is tested on 25 X-ray angiography sequences where a precision of 0.9597 is achieved.

Computer Vision Techniques for Transcatheter Intervention

Minimally invasive transcatheter technologies have demonstrated substantial promise for the diagnosis and the treatment of cardiovascular diseases. For example, transcatheter aortic valve implantation is an alternative to aortic valve replacement for the treatment of severe aortic stenosis, and transcatheter atrial fibrillation ablation is widely used for the treatment and the cure of atrial fibrillation. In addition, catheter-based intravascular ultrasound and optical coherence tomography imaging of coronary arteries provides important information about the coronary lumen, wall, and plaque characteristics. Qualitative and quantitative analysis of these cross-sectional image data will be beneficial to the evaluation and the treatment of coronary artery diseases such as atherosclerosis. In all the phases (preoperative, intraoperative, and postoperative) during the transcatheter intervention procedure, computer vision techniques (e.g., image segmentation and motion tracking) have been largely applied in the field to accomplish tasks like annulus measurement, valve selection, catheter placement control, and vessel centerline extraction. This provides beneficial guidance for the clinicians in surgical planning, disease diagnosis, and treatment assessment. In this paper, we present a systematical review on these state-of-the-art methods. We aim to give a comprehensive overview for researchers in the area of computer vision on the subject of transcatheter intervention. Research in medical computing is multi-disciplinary due to its nature, and hence, it is important to understand the application domain, clinical background, and imaging modality, so that methods and quantitative measurements derived from analyzing the imaging data are appropriate and meaningful. We thus provide an overview on the background information of the transcatheter intervention procedures, as well as a review of the computer vision techniques and methodologies applied in this area.

3D multimodal cardiac data reconstruction using angiography and computerized tomographic angiography registration

BACKGROUND

Computerized tomographic angiography (3D data representing the coronary arteries) and X-ray angiography (2D X-ray image sequences providing information about coronary arteries and their stenosis) are standard and popular assessment tools utilized for medical diagnosis of coronary artery diseases. At present, the results of both modalities are individually analyzed by specialists and it is difficult for them to mentally connect the details of these two techniques. The aim of this work is to assist medical diagnosis by providing specialists with the relationship between computerized tomographic angiography and X-ray angiography.

METHODS

In this study, coronary arteries from two modalities are registered in order to create a 3D reconstruction of the stenosis position. The proposed method starts with coronary artery segmentation and labeling for both modalities. Then, stenosis and relevant labeled artery in X-ray angiography image are marked by a specialist. Proper control points for the marked artery in both modalities are automatically detected and normalized. Then, a geometrical transformation function is computed using these control points. Finally, this function is utilized to register the marked artery from the X-ray angiography image on the computerized tomographic angiography and get the 3D position of the stenosis lesion.

RESULTS

The result is a 3D informative model consisting of stenosis and coronary arteries' information from the X-ray angiography and computerized tomographic angiography modalities. The results of the proposed method for coronary artery segmentation, labeling and 3D reconstruction are evaluated and validated on the dataset containing both modalities.

CONCLUSIONS

The advantage of this method is to aid specialists to determine a visual relationship between the correspondent coronary arteries from two modalities and also set up a connection between stenosis points from an X-ray angiography along with their 3D positions on the coronary arteries from computerized tomographic angiography. Moreover, another benefit of this work is that the medical acquisition standards remain unchanged, which means that no calibration in the acquisition devices is required. It can be applied on most computerized tomographic angiography and angiography devices.

Diaphragm border detection in coronary X-ray angiographies: New method and applications

X-ray angiography is widely used in cardiac disease diagnosis during or prior to intravascular interventions. The diaphragm motion and the heart beating induce gray-level changes, which are one of the main obstacles in quantitative analysis of myocardial perfusion. In this paper we focus on detecting the diaphragm border in both single images or whole X-ray angiography sequences. We show that the proposed method outperforms state of the art approaches. We extend a previous publicly available data set, adding new ground truth data. We also compose another set of more challenging images, thus having two separate data sets of increasing difficulty. Finally, we show three applications of our method: (1) a strategy to reduce false positives in vessel enhanced images; (2) a digital diaphragm removal algorithm; (3) an improvement in Myocardial Blush Grade semi-automatic estimation.