Constraint-Aware Learning for Fractional Flow Reserve Pullback Curve Estimation From Invasive Coronary Imaging

Estimation of the fractional flow reserve (FFR) pullback curve from invasive coronary imaging is important for the intraoperative guidance of coronary intervention. Machine/deep learning has been proven effective in FFR pullback curve estimation. However, the existing methods suffer from inadequate incorporation of intrinsic geometry associations and physics knowledge. In this paper, we propose a constraint-aware learning framework to improve the estimation of the FFR pullback curve from invasive coronary imaging. It incorporates both geometrical and physical constraints to approximate the relationships between the geometric structure and FFR values along the coronary artery centerline. Our method also leverages the power of synthetic data in model training to reduce the collection costs of clinical data. Moreover, to bridge the domain gap between synthetic and real data distributions when testing on real-world imaging data, we also employ a diffusion-driven test-time data adaptation method that preserves the knowledge learned in synthetic data. Specifically, this method learns a diffusion model of the synthetic data distribution and then projects real data to the synthetic data distribution at test time. Extensive experimental studies on a synthetic dataset and a real-world dataset of 382 patients covering three imaging modalities have shown the better performance of our method for FFR estimation of stenotic coronary arteries, compared with other machine/deep learning-based FFR estimation models and computational fluid dynamics-based model. The results also provide high agreement and correlation between the FFR predictions of our method and the invasively measured FFR values. The plausibility of FFR predictions along the coronary artery centerline is also validated.

Patient-specific in silico 3D coronary model in cardiac catheterisation laboratories

Coronary artery disease is caused by the buildup of atherosclerotic plaque in the coronary arteries, affecting the blood supply to the heart, one of the leading causes of death around the world. X-ray coronary angiography is the most common procedure for diagnosing coronary artery disease, which uses contrast material and x-rays to observe vascular lesions. With this type of procedure, blood flow in coronary arteries is viewed in real-time, making it possible to detect stenoses precisely and control percutaneous coronary interventions and stent insertions. Angiograms of coronary arteries are used to plan the necessary revascularisation procedures based on the calculation of occlusions and the affected segments. However, their interpretation in cardiac catheterisation laboratories presently relies on sequentially evaluating multiple 2D image projections, which limits measuring lesion severity, identifying the true shape of vessels, and analysing quantitative data. In silico modelling, which involves computational simulations of patient-specific data, can revolutionise interventional cardiology by providing valuable insights and optimising treatment methods. This paper explores the challenges and future directions associated with applying patient-specific in silico models in catheterisation laboratories. We discuss the implications of the lack of patient-specific in silico models and how their absence hinders the ability to accurately predict and assess the behaviour of individual patients during interventional procedures. Then, we introduce the different components of a typical patient-specific in silico model and explore the potential future directions to bridge this gap and promote the development and utilisation of patient-specific in silico models in the catheterisation laboratories.

A multi-stage neural network approach for coronary 3D reconstruction from uncalibrated X-ray angiography images

We present a multi-stage neural network approach for 3D reconstruction of coronary artery trees from uncalibrated 2D X-ray angiography images. This method uses several binarized images from different angles to reconstruct a 3D coronary tree without any knowledge of image acquisition parameters. The method consists of a single backbone network and separate stages for vessel centerline and radius reconstruction. The output is an analytical matrix representation of the coronary tree suitable for downstream applications such as hemodynamic modeling of local vessel narrowing (i.e., stenosis). The network was trained using a dataset of synthetic coronary trees from a vessel generator informed by both clinical image data and literature values on coronary anatomy. Our multi-stage network achieved sub-pixel accuracy in reconstructing vessel radius (RMSE = 0.16 ± 0.07 mm) and stenosis radius (MAE = 0.27 ± 0.18 mm), the most important feature used to inform diagnostic decisions. The network also led to 52% and 38% reduction in vessel centerline reconstruction errors compared to a single-stage network and projective geometry-based methods, respectively. Our method demonstrated robustness to overcome challenges such as vessel foreshortening or overlap in the input images. This work is an important step towards automated analysis of anatomic and functional disease severity in the coronary arteries.

A Simple Method for Automatic 3D Reconstruction of Coronary Arteries From X-Ray Angiography

Automatic three-dimensional (3-D) reconstruction of the coronary arteries (CA) from medical imaging modalities is still a challenging task. In this study, we present a deep learning-based method of automatic identification of the two ends of the vessel from X-ray coronary angiography (XCA). We also present a method of using template models of CA in matching the two-dimensional segmented vessels from two different angles of XCA. For the deep learning network, we used a U-net consisting of an encoder (Resnet) and a decoder. The two ends of the vessel were manually labeled to generate training images. The network was trained with 2,342, 1,907, and 1,523 labeled images for the left anterior descending (LAD), left circumflex (LCX), and right coronary artery (RCA), respectively. For template models of CA, ten reconstructed 3-D models were averaged for each artery. The accuracy of correspondence using template models was compared with that of manual matching. The deep learning network pointed the proximal region (20% of the total length) in 97.7, 97.5, and 96.4% of 315, 201, and 167 test images for LAD, LCX, and RCA, respectively. The success rates in pointing the distal region were 94.9, 89.8, and 94.6%, respectively. The average distances between the projected points from the reconstructed 3-D model to the detector and the points on the segmented vessels were not statistically different between the template and manual matchings. The computed FFR was not significantly different between the two matchings either. Deep learning methodology is feasible in identifying the two ends of the vessel in XCA, and the accuracy of using template models is comparable to that of manual correspondence in matching the segmented vessels from two angles.

Ordered multi-path propagation for vessel centerline extraction

Vessel centerline extraction from x-ray angiography images is essential for vessel structure analysis in the diagnosis of coronary artery disease. However, complete and continuous centerline extraction remains a challenging task due to image noise, poor contrast, and complexity of vessel structure. Thus, an iterative multi-path search framework for automatic vessel centerline extraction is proposed. First, the seed points of the vessel structure are detected and sorted by confidence. With the ordered seed points, multi-bifurcation centerline is searched through multi-path propagation of wavefront and accumulated voting. Finally, the centerline is further extended piecewise by wavefront propagation on the basis of keypoint detection. The latter two steps are performed alternately to obtain the final centerline result. The proposed method is qualitatively and quantitatively evaluated on 1260 synthetic images and 50 clinical angiography images. The results demonstrate that our method has a highF1score of 87.8% ± 2.7% for the angiography images and achieves accurate and continuous results of vessel centerline extraction.

Local-global active contour model based on tensor-based representation for 3D ultrasound vessel segmentation

Three-dimensional (3D) vessel segmentation can provide full spatial information about an anatomic structure to help physicians gain increased understanding of vascular structures, which plays an utmost role in many medical image-processing and analysis applications. The purpose of this paper aims to develop a 3D vessel-segmentation method that can improve segmentation accuracy in 3D ultrasound (US) images. We propose a 3D tensor-based active contour model method for accurate 3D vessel segmentation. With our method, the contrast-independent multiscale bottom-hat tensor representation and local-global information are captured. This strategy ensures the effective extraction of the boundaries of vessels from inhomogeneous and homogeneous regions without being affected by the noise and low-contrast of the 3D US images. Experimental results in clinical 3D US and public 3D Multiphoton Microscopy datasets are used for quantitative and qualitative comparison with several state-of-the-art vessel segmentation methods. Clinical experiments demonstrate that our method can achieve a smoother and more accurate boundary of the vessel object than competing methods. The mean SE, SP and ACC of the proposed method are: 0.7768 ± 0.0597, 0.9978 ± 0.0013 and 0.9971 ± 0.0015 respectively. Experiments on the public dataset show that our method can segment complex vessels in different medical images with noise and low- contrast.

Motion-flow-guided recurrent network for respiratory signal estimation of x-ray angiographic image sequences

Motion compensation can eliminate inconsistencies of respiratory movement during image acquisitions for precise vascular reconstruction in the clinical diagnosis of vascular disease from x-ray angiographic image sequences. In x-ray-based vascular interventional therapy, motion modeling can simulate the process of organ deformation driven by motion signals to display a dynamic organ on angiograms without contrast agent injection. Automatic respiratory signal estimation from x-ray angiographic image sequences is essential for motion compensation and modeling. The effects of respiratory motion, cardiac impulses, and tremors on structures in the chest and abdomen bring difficulty in extracting accurate respiratory signals individually. In this study, an end-to-end deep learning framework based on a motion-flow-guided recurrent network is proposed to address the aforementioned problem. The proposed method utilizes a convolutional neural network to learn the spatial features of every single frame, and a recurrent neural network to learn the temporal features of the entire sequence. The combination of the two networks can effectively analyze the image sequence to realize respiratory signal estimation. In addition, the motion-flow between consecutive frames is introduced to provide a dynamic constraint of spatial features, which enables the recurrent network to learn better temporal features from dynamic spatial features than from static spatial features. We demonstrate the advantages of our approach on designed datasets which contain coronary and hepatic angiographic sequences with diaphragm structures, and coronary angiographic sequences without diaphragm structures. Our method improves over state-of-the-art manifold-learning-based methods by 85.7%, 81.5% and 75.3% in respiratory signal accuracy metric on these datasets. The results demonstrate that the proposed method can effectively estimate respiratory signals from multiple motion patterns.

Direct Quantification of Coronary Artery Stenosis Through Hierarchical Attentive Multi-View Learning

Quantification of coronary artery stenosis on X-ray angiography (XRA) images is of great importance during the intraoperative treatment of coronary artery disease. It serves to quantify the coronary artery stenosis by estimating the clinical morphological indices, which are essential in clinical decision making. However, stenosis quantification is still a challenging task due to the overlapping, diversity and small-size region of the stenosis in the XRA images. While efforts have been devoted to stenosis quantification through low-level features, these methods have difficulty in learning the real mapping from these features to the stenosis indices. These methods are still cumbersome and unreliable for the intraoperative procedures due to their two-phase quantification, which depends on the results of segmentation or reconstruction of the coronary artery. In this work, we are proposing a hierarchical attentive multi-view learning model (HEAL) to achieve a direct quantification of coronary artery stenosis, without the intermediate segmentation or reconstruction. We have designed a multi-view learning model to learn more complementary information of the stenosis from different views. For this purpose, an intra-view hierarchical attentive block is proposed to learn the discriminative information of stenosis. Additionally, a stenosis representation learning module is developed to extract the multi-scale features from the keyframe perspective for considering the clinical workflow. Finally, the morphological indices are directly estimated based on the multi-view feature embedding. Extensive experiment studies on clinical multi-manufacturer dataset consisting of 228 subjects show the superiority of our HEAL against nine comparing methods, including direct quantification methods and multi-view learning methods. The experimental results demonstrate the better clinical agreement between the ground truth and the prediction, which endows our proposed method with a great potential for the efficient intraoperative treatment of coronary artery disease.

Validation study of a novel method for the 3D reconstruction of coronary bifurcations

Quantitative Coronary Angiography (QCA) is an important tool in the study of coronary artery disease. Validation of this technique is crucial for their ongoing development and refinement although it is difficult due to several factors such as potential sources of error. The present work aims to a further validation of a new semi-automated method for three-dimensional (3D) reconstruction of coronary bifurcations arteries based on X-Ray Coronary Angiographies (CA). In a dataset of 40 patients (79 angiographic views), we used the aforementioned method to reconstruct them in 3D space. The validation was based on the comparison of these 3D models with the true silhouette of 2D models annotated by an expert using specific metrics. The obtained results indicate a good accuracy for the most parameters (≥ 90 %). Comparison with similar works shows that our new method is a promising tool for the 3D reconstruction of coronary bifurcations and for application in everyday clinical use.

A new method for the 3D reconstruction of coronary bifurcations pre and post the angioplasty procedure using the QCA

The aim of this study is to propose a new semi-automated method for three-dimensional (3D) reconstruction of coronary bifurcations arteries using X-ray Coronary Angiographies (CA). Considering two monoplane angiographic views as the input data, the method is based on a four-step approach. In the first step, the image pre-processing and the vessel segmentation is performed. In the second step the 3D centerline is reconstructed by implementing the back-projection algorithm. In the third step, the lumen borders are reconstructed around the centerline to result to the fourth step, during which the 3D point cloud of the side branch is adjusted to the main branch, to produce the final 3D model of the coronary bifurcation artery. Imaging data from seven patients (pre and post-stenting) were reconstructed in the 3D space. The validation of the proposed methodology was based on the comparison of the 3D model with the 2D CA. Blood flow simulations were performed both for the vessels before and after the angioplasty procedure. Decreased Endothelial Shear Stress (ESS) values were observed for the vessels after the Percutaneous Transluminal Coronary Intervention (PTCI).

Point-Cloud Method for Automated 3D Coronary Tree Reconstruction From Multiple Non-Simultaneous Angiographic Projections

X-ray angiography is the most commonly used imaging modality for the detection of coronary stenoses due to its high spatial and temporal resolution of lumen contour and its utility to guide coronary interventions in real time. However, the high inter- and intra-observer variability in interpreting the geometry of 3D vascular structure based on multiple 2D image projections is a limitation in the accurate determination of lesion severity. This could be addressed by the 3D reconstruction of the coronary arterial (CA) tree. The automated reconstruction of 3D CA tree from 2D projections is challenging due to the existence of several imaging artifacts, such as vessel overlap, foreshortening, and most importantly respiratory and cardiac motion. Along with these artifacts, the acquisition geometry introduces the possibility of generating false vessel segments in the reconstruction. Our approach aims to reduce the motion artifacts in angiographic projections by developing a new method for rigid and non-rigid motion correction. A novel point-cloud based approach is subsequently introduced for reconstruction of 3D vessel centerlines by iteratively minimizing the reconstruction error. The performance of the proposed 3D reconstruction is evaluated using angiographic projections from 45 patients, producing average reprojection errors of 0.092 ±0.055 mm and 0.910 ±0.352 mm for 3D centerlines reconstruction, when co-registered with the parent vessels on projection planes that were/were not used to derive the 3D reconstruction, respectively. A comparison of the reconstructed 3D lumen surface with optical coherence tomography (OCT) measurements has been performed, showing no statistically significant difference in the luminal cross-sections reconstructed with our method, compared to OCT.

Deep motion tracking from multiview angiographic image sequences for synchronization of cardiac phases

In the diagnosis and interventional treatment of coronary artery disease, the 3D[Formula: see text]time reconstruction of the coronary artery on the basis of x-ray angiographic image sequences can provide dynamic structural information. The synchronization of cardiac phases in the sequences is essential for minimizing the influence of cardiorespiratory motion and realizing precise 3D[Formula: see text]time reconstruction. Key points are initially extracted from the first image of a sequence. Matching grid points between consecutive images in the sequence are extracted by a multi-layer matching strategy. Then deep motion tracking (DMT) of key points is achieved by local deformation based on the neighboring grid points of key points. The local deformation is optimized by the Random sample consensus (RANSAC) algorithm. Then, a simple harmonic motion (SHM) model is utilized to distinguish cardiac motion from other motion sources (e.g. respiratory, patient movement, etc). Next, the signal which is composed of cardiac motions is filtered by a band-pass filter to reconstruct the cardiac phases. Finally, the synchronization of cardiac phases from different imaging angles is realized by a piece-wise linear transformation. The proposed method was evaluated using clinical x-ray angiographic image sequences from 13 patients. [Formula: see text] matching points can be accurately computed by the DMT method. The mean peak temporal distance (MPTD) between the reconstructed cardiac phases and the electrocardiograph signal is [Formula: see text] s. The correlation between the cardiac phases of the same patient is over [Formula: see text]. Compared with three other state-of-the-art methods, the proposed method accurately reconstructs and synchronizes the cardiac phases from different sequences of the same patient. The proposed DMT method is robust and highly effective in synchronizing cardiac phases of angiographic image sequences captured from different imaging angles.

Image-Aligned Dynamic Liver Reconstruction Using Intra-Operative Field of Views for Minimal Invasive Surgery

During hepatic minimal invasive surgery (MIS), 3-D reconstruction of a liver surface by interpreting the geometry of its soft tissues is achieving attractions. One of the major issues to be addressed in MIS is liver deformation. Moreover, it severely inhibits free sight and dexterity of tissue manipulation, which causes its intra-operative morphology and soft tissue motion altered as compared to its pre-operative shape. While many applications focus on 3-D reconstruction of rigid or semi-rigid scenes, the techniques applied in hepatic MIS must be able to cope with a dynamic and deformable environment. We propose an efficient technique for liver surface reconstruction based on the structure from motion to handle liver deformation. The reconstructed liver will assist surgeons to visualize liver surface more efficiently with better depth perception. We use the intra-operative field of views to generate 3-D template mesh from a dense keypoint cloud. We estimate liver deformation by finding best correspondence between 3-D templates and reconstruct a liver image to calculate translation and rotational motions. Our technique then finely tunes deformed surface by adding smoothness using shading cues. Up till now, this technique is not used for solving the human liver deformation problem. Our approach is tested and validated with synthetic as well as real in vivo data, which reveal that the reconstruction accuracy can be enhanced using our approach even in challenging laparoscopic environments.

Hybrid constraint optimization for 3D subcutaneous vein reconstruction by near-infrared images

BACKGROUND AND OBJECTIVE

The development of biometric identification technology and intelligent medication has enabled researchers to analyze subcutaneous veins from near-infrared images. However, the stereo reconstruction of subcutaneous veins has not been well studied, and the results are difficult to utilize in clinical practice.

METHODS

We present a hybrid constraint optimization (HCO) matching algorithm for vein reconstruction to solve the matching failure problems caused by the incomplete segmentation of vein structures captured from different views. This algorithm initially introduces the existence of the epipolar and homography constraints in the subcutaneous vein matching. Then, the HCO matching algorithm of the vascular centerline is established by homography point-to-point matching, homography matrix optimization, and vascular section matching. Finally, the 3D subcutaneous vein is reconstructed on the basis of the principle of triangulation and system calibration parameters.

RESULTS

To validate the performance of the proposed matching method, we designed a series of experiments to evaluate the effectiveness of the hybrid constraint optimization method. The experiments were performed on simulated and real datasets. 42 real vascular images were analyzed on the basis of different matching strategies. Experimental result shows that the matching accuracy increased significantly with the proposed optimization matching method. To calculate the reconstruction accuracy, we reconstructed seven simulated cardboards and measured 10 vascular distances in each simulated cardboard. The average vascular distance error of each simulated image was within 1.0 mm, and the distance errors of 75% feature points were less than 1.5 mm. Also, we printed a 3D simulated vein model to improve the illustration of this system. The reconstruction error extends from -3.58 mm to 1.94 mm with a standard deviation of 0.68 mm and a mean of 0.07 mm.

CONCLUSIONS

The algorithm is validated in terms of homography optimization, matching efficiency, and simulated vascular reconstruction error. The experimental results demonstrate that the veins captured from the left and right views can be accurately matched through the proposed algorithm.

Three-dimensional reconstruction and NURBS-based structured meshing of coronary arteries from the conventional X-ray angiography projection images

Despite its two-dimensional nature, X-ray angiography (XRA) has served as the gold standard imaging technique in the interventional cardiology for over five decades. Accordingly, demands for tools that could increase efficiency of the XRA procedure for the quantitative analysis of coronary arteries (CA) are constantly increasing. The aim of this study was to propose a novel procedure for three-dimensional modeling of CA from uncalibrated XRA projections. A comprehensive mathematical model of the image formation was developed and used with a robust genetic algorithm optimizer to determine the calibration parameters across XRA views. The frames correspondences between XRA acquisitions were found using a partial-matching approach. Using the same matching method, an efficient procedure for vessel centerline reconstruction was developed. Finally, the problem of meshing complex CA trees was simplified to independent reconstruction and meshing of connected branches using the proposed nonuniform rational B-spline (NURBS)-based method. Because it enables structured quadrilateral and hexahedral meshing, our method is suitable for the subsequent computational modelling of CA physiology (i.e. coronary blood flow, fractional flow reverse, virtual stenting and plaque progression). Extensive validations using digital, physical, and clinical datasets showed competitive performances and potential for further application on a wider scale.

A novel hybrid approach for reconstruction of coronary bifurcations using angiography and OCT

The aim of this study is to present a new method for three-dimensional (3D) reconstruction of coronary bifurcations using biplane Coronary Angiographies and Optical Coherence Tomography (OCT) imaging. The method is based on a five step approach by improving a previous validated work in order to reconstruct coronary arterial bifurcations. In the first step the lumen borders are detected on the Frequency Domain (FD) OCT images. In the second step a semi-automated method is implemented on two angiographies for the extraction of the 2D bifurcation coronary artery centerline. In the third step the 3D path of the bifurcation artery is extracted based on a back projection algorithm. In the fourth step the lumen borders are placed onto the 3D catheter path. Finally, in the fifth step the intersection of the main and side branches produces the reconstructed model of the coronary bifurcation artery. Data from three patients are acquired for the validation of the proposed methodology and the results are compared against a reconstruction method using quantitative coronary angiography (QCA). The comparison between the two methods is achieved using morphological measures of the vessels as well as comparison of the wall shear stress (WSS) mean values.

Augmented reality based real-time subcutaneous vein imaging system

A novel 3D reconstruction and fast imaging system for subcutaneous veins by augmented reality is presented. The study was performed to reduce the failure rate and time required in intravenous injection by providing augmented vein structures that back-project superimposed veins on the skin surface of the hand. Images of the subcutaneous vein are captured by two industrial cameras with extra reflective near-infrared lights. The veins are then segmented by a multiple-feature clustering method. Vein structures captured by the two cameras are matched and reconstructed based on the epipolar constraint and homographic property. The skin surface is reconstructed by active structured light with spatial encoding values and fusion displayed with the reconstructed vein. The vein and skin surface are both reconstructed in the 3D space. Results show that the structures can be precisely back-projected to the back of the hand for further augmented display and visualization. The overall system performance is evaluated in terms of vein segmentation, accuracy of vein matching, feature points distance error, duration times, accuracy of skin reconstruction, and augmented display. All experiments are validated with sets of real vein data. The imaging and augmented system produces good imaging and augmented reality results with high speed.

Reconstruction of coronary arteries from X-ray angiography: A review

Despite continuous progress in X-ray angiography systems, X-ray coronary angiography is fundamentally limited by its 2D representation of moving coronary arterial trees, which can negatively impact assessment of coronary artery disease and guidance of percutaneous coronary intervention. To provide clinicians with 3D/3D+time information of coronary arteries, methods computing reconstructions of coronary arteries from X-ray angiography are required. Because of several aspects (e.g. cardiac and respiratory motion, type of X-ray system), reconstruction from X-ray coronary angiography has led to vast amount of research and it still remains as a challenging and dynamic research area. In this paper, we review the state-of-the-art approaches on reconstruction of high-contrast coronary arteries from X-ray angiography. We mainly focus on the theoretical features in model-based (modelling) and tomographic reconstruction of coronary arteries, and discuss the evaluation strategies. We also discuss the potential role of reconstructions in clinical decision making and interventional guidance, and highlight areas for future research.