3D-2D registration in endovascular image-guided surgery: evaluation of state-of-the-art methods on cerebral angiograms

Comparison of global and local optimization methods for intensity-based 2D-3D registration

Intensity-based 2D-3D registration methods are commonly used in musculoskeletal research and image-guided therapy to align 2D X-ray images with 3D CT scans. However, their success rate (SR) is limited by local optimization methods, which often cause the optimization of the underlying cost function to get stuck at a local minimum, resulting in false alignments. Global optimization methods aim to mitigate this problem, but despite their increasing popularity, the existing literature lacks consensus on which one is the most appropriate. In this work, we compare 11 global and 4 local optimization methods on thousands of typical registration examples of single- and dual-plane fluoroscopy, including three datasets of varying complexity. In addition, we evaluate the differences between global and local methods, determine the best overall method, and validate its suitability for real clinical data. The results demonstrate that global methods that require a large number of function evaluations (NFEV) are generally the most robust. Furthermore, hyperparameter tuning can significantly improve their performance and is generalizable across datasets. Evolutionary strategy (ES) is the best-performing optimization method in our study, achieving a mean SR of ∼95% for all test models, ∼77% for the knee bones, and ∼95-100% for cerebral angiograms when using dual-plane registration setup. Nevertheless, in cases where good initialization is available, local methods are still preferable due to their reduced NFEV. The use of global optimization improves the overall robustness and ease-of-use of 2D-3D registration, potentially accelerating its adaptation in routine medical practice and biomedical research.

3D/2D Vessel Registration Based on Monte Carlo Tree Search and Manifold Regularization

The augmented intra-operative real-time imaging in vascular interventional surgery, which is generally performed by projecting preoperative computed tomography angiography images onto intraoperative digital subtraction angiography (DSA) images, can compensate for the deficiencies of DSA-based navigation, such as lack of depth information and excessive use of toxic contrast agents. 3D/2D vessel registration is the critical step in image augmentation. A 3D/2D registration method based on vessel graph matching is proposed in this study. For rigid registration, the matching of vessel graphs can be decomposed into continuous states, thus 3D/2D vascular registration is formulated as a search tree problem. The Monte Carlo tree search method is applied to find the optimal vessel matching associated with the highest rigid registration score. For nonrigid registration, we propose a novel vessel deformation model based on manifold regularization. This model incorporates the smoothness constraint of vessel topology into the objective function. Furthermore, we derive simplified gradient formulas that enable fast registration. The proposed technique undergoes evaluation against seven rigid and three nonrigid methods using a variety of data - simulated, algorithmically generated, and manually annotated - across three vascular anatomies: the hepatic artery, coronary artery, and aorta. Our findings show the proposed method's resistance to pose variations, noise, and deformations, outperforming existing methods in terms of registration accuracy and computational efficiency. The proposed method demonstrates average registration errors of 2.14 mm and 0.34 mm for rigid and nonrigid registration, and an average computation time of 0.51 s.

Image-to-Graph Convolutional Network for 2D/3D Deformable Model Registration of Low-Contrast Organs

Organ shape reconstruction based on a single-projection image during treatment has wide clinical scope, e.g., in image-guided radiotherapy and surgical guidance. We propose an image-to-graph convolutional network that achieves deformable registration of a three-dimensional (3D) organ mesh for a low-contrast two-dimensional (2D) projection image. This framework enables simultaneous training of two types of transformation: from the 2D projection image to a displacement map, and from the sampled per-vertex feature to a 3D displacement that satisfies the geometrical constraint of the mesh structure. Assuming application to radiation therapy, the 2D/3D deformable registration performance is verified for multiple abdominal organs that have not been targeted to date, i.e., the liver, stomach, duodenum, and kidney, and for pancreatic cancer. The experimental results show shape prediction considering relationships among multiple organs can be used to predict respiratory motion and deformation from digitally reconstructed radiographs with clinically acceptable accuracy.

Full-Resolution Network and Dual-Threshold Iteration for Retinal Vessel and Coronary Angiograph Segmentation

Vessel segmentation is critical for disease diagnosis and surgical planning. Recently, the vessel segmentation method based on deep learning has achieved outstanding performance. However, vessel segmentation remains challenging due to thin vessels with low contrast that easily lose spatial information in the traditional U-shaped segmentation network. To alleviate this problem, we propose a novel and straightforward full-resolution network (FR-UNet) that expands horizontally and vertically through a multiresolution convolution interactive mechanism while retaining full image resolution. In FR-UNet, the feature aggregation module integrates multiscale feature maps from adjacent stages to supplement high-level contextual information. The modified residual blocks continuously learn multiresolution representations to obtain a pixel-level accuracy prediction map. Moreover, we propose the dual-threshold iterative algorithm (DTI) to extract weak vessel pixels for improving vessel connectivity. The proposed method was evaluated on retinal vessel datasets (DRIVE, CHASE_DB1, and STARE) and coronary angiography datasets (DCA1 and CHUAC). The results demonstrate that FR-UNet outperforms state-of-the-art methods by achieving the highest Sen, AUC, F1, and IOU on most of the above-mentioned datasets with fewer parameters, and that DTI enhances vessel connectivity while greatly improving sensitivity. The code is available at: https://github.com/lseventeen/FR-UNet.

Instrument localisation for endovascular aneurysm repair: Comparison of two methods based on tracking systems or using imaging

BACKGROUND

In endovascular aneuysm repair (EVAR) procedures, medical instruments are currently navigated with a two-dimensional imaging based guidance requiring X-rays and contrast agent.

METHODS

Novel approaches for obtaining the three-dimensional instrument positions are introduced. Firstly, a method based on fibre optical shape sensing, one electromagnetic sensor and a preoperative computed tomography (CT) scan is described. Secondly, an approach based on image processing using one 2D fluoroscopic image and a preoperative CT scan is introduced.

RESULTS

For the tracking based method, average errors from 1.81 to 3.13 mm and maximum errors from 3.21 to 5.46 mm were measured. For the image-based approach, average errors from 3.07 to 6.02 mm and maximum errors from 8.05 to 15.75 mm were measured.

CONCLUSION

The tracking based method is promising for usage in EVAR procedures. For the image-based approach are applications in smaller vessels more suitable, since its errors increase with the vessel diameter.

Multilevel 2D-3D Intensity-Based Image Registration

2D-3D image registration is an important task for computer-aided minimally invasive vascular therapies. A crucial component for practical image registration is the use of multilevel strategies to avoid local optima and to speed-up runtime. However, due to the different dimensionalities of the 2D fixed and 3D moving image, the setup of multilevel strategies is not straightforward.

In this work, we propose an intensity-driven 2D-3D multiresolution registration approach using the normalized gradient fields (NGF) distance measure. We discuss and empirically analyze the impact on the choice of 2D and 3D image resolutions. Furthermore, we show that our approach produces results that are comparable or superior to other state-of-the-art methods.

Robust Multi-View 2-D/3-D Registration Using Point-To-Plane Correspondence Model

In minimally invasive procedures, the clinician relies on image guidance to observe and navigate the operation site. In order to show structures which are not visible in the live X-ray images, such as vessels or planning annotations, X-ray images can be augmented with pre-operatively acquired images. Accurate image alignment is needed and can be provided by 2-D/3-D registration. In this paper, a multi-view registration method based on the point-to-plane correspondence model is proposed. The correspondence model is extended to be independent of the used camera coordinates and different multi-view registration schemes are introduced and compared. Evaluation is performed for a wide range of clinically relevant registration scenarios. We show for different applications that registration using correspondences from both views simultaneously provides accurate and robust registration, while the performance of the other schemes varies considerably. Our method also outperforms the state-of-the-art method for cerebral angiography registration, achieving a capture range of 18 mm and an accuracy of 0.22±0.07 mm. Furthermore, investigations on the minimum angle between the views are performed in order to provide accurate and robust registration, while minimizing the obstruction to the clinical workflow. We show that small angles around 30° are sufficient to provide reliable registration results.