A novel approach to 2D/3D registration of X-ray images using Grangeat's relation

Real-Time 3D Instrument Tip Tracking Using 2D X-Ray Fluoroscopy With Vessel Deformation Correction Under Free Breathing

OBJECTIVE

Accurate localization of the instrument tip within the hepatic vein is crucial for the success of transjugular intrahepatic portosystemic shunt (TIPS) procedures. Real-time tracking of the instrument tip in X-ray images is greatly influenced by vessel deformation due to patient's pose variation, respiratory motion, and puncture manipulation, frequently resulting in failed punctures.

METHOD

We propose a novel framework called deformable instrument tip tracking (DITT) to obtain the real-time tip positioning within the 3D deformable vasculature. First, we introduce a pose alignment module to improve the rigid matching between the preoperative vessel centerline and the intraoperative instrument centerline, in which the accurate matching of 3D/2D centerline features is implemented with an adaptive point sampling strategy. Second, a respiration compensation module using monoplane X-ray image sequences is constructed and provides the motion prior to predict intraoperative liver movement. Third, a deformation correction module is proposed to rectify the vessel deformation during procedures, in which a manifold regularization and the maximum likelihood-based acceleration are introduced to obtain the accurate and fast deformation learning.

RESULTS

Experimental results on simulated and clinical datasets show an average tracking error of 1.59 0.57 mm and 1.67 0.54 mm, respectively.

CONCLUSION

Our framework can track the tip in 3D vessel and dynamically overlap the branch roadmapping onto X-ray images to provide real-time guidance.

SIGNIFICANCE

Accurate and fast (43ms per frame) tip tracking with the proposed framework possesses a good potential for improving the outcomes of TIPS treatment and minimizes the usage of contrast agent.

A Robust Method for Real Time Intraoperative 2D and Preoperative 3D X-Ray Image Registration Based on an Enhanced Swin Transformer Framework

In image-guided surgery (IGS) practice, combining intraoperative 2D X-ray images with preoperative 3D X-ray images from computed tomography (CT) enables the rapid and accurate localization of lesions, which allows for a more minimally invasive and efficient surgery, and also reduces the risk of secondary injuries to nerves and vessels. Conventional optimization-based methods for 2D X-ray and 3D CT matching are limited in speed and precision due to non-convex optimization spaces and a constrained searching range. Recently, deep learning (DL) approaches have demonstrated remarkable proficiency in solving complex nonlinear 2D-3D registration. In this paper, a fast and robust DL-based registration method is proposed that takes an intraoperative 2D X-ray image as input, compares it with the preoperative 3D CT, and outputs their relative pose in x, y, z and pitch, yaw, roll. The method employs a dual-channel Swin transformer feature extractor equipped with attention mechanisms and feature pyramid to facilitate the correlation between features of the 2D X-ray and anatomical pose of CT. Tests on three different regions of interest acquired from open-source datasets show that our method can achieve high pose estimation accuracy (mean rotation and translation error of 0.142° and 0.362 mm, respectively) in a short time (0.02 s). Robustness tests indicate that our proposed method can maintain zero registration failures across varying levels of noise. This generalizable learning-based 2D (X-ray) and 3D (CT) registration algorithm owns promising applications in surgical navigation, targeted radiotherapy, and other clinical operations, with substantial potential for enhancing the accuracy and efficiency of image-guided surgery.

Isocentric fixed angle irradiation-based DRR: a novel approach to enhance x-ray and CT image registration

Objective.Digitally reconstructed radiography (DRR) plays an important role in the registration of intraoperative x-ray and preoperative CT images. However, existing DRR algorithms often neglect the critical isocentric fixed angle irradiation (IFAI) principle in C-arm imaging, resulting in inaccurate simulation of x-ray images. This limitation degrades registration algorithms relying on DRR image libraries or employing DRR images (DRRs) to train neural network models. To address this issue, we propose a novel IFAI-based DRR method that accurately captures the true projection transformation during x-ray imaging of the human body.Approach.By strictly adhering to the IFAI principle and utilizing known parameters from intraoperative x-ray images paired with CT scans, our method successfully simulates the real projection transformation and generates DRRs that closely resemble actual x-ray images.Main result.Experimental results validate the effectiveness of our IFAI-based DRR method by successfully registering intraoperative x-ray images with preoperative CT images from multiple patients who underwent thoracic endovascular aortic procedures.Significance. The proposed IFAI-based DRR method enhances the quality of DRR images, significantly accelerates the construction of DRR image libraries, and thereby improves the performance of x-ray and CT image registration. Additionally, the method has the generality of registering CT and x-ray images generated by large C-arm devices.

Deformable 3D/3D CT-to-digital-tomosynthesis image registration in image-guided bronchoscopy interventions

Traditional navigational bronchoscopy procedures rely on preprocedural computed tomography (CT) and intraoperative chest radiography and cone-beam CT (CBCT) to biopsy peripheral lung lesions. This navigational approach is challenging due to the projective nature of radiography, and the high radiation dose, long imaging time, and large footprints of CBCT. Digital tomosynthesis (DTS) is considered an attractive alternative combining the advantages of radiography and CBCT. Only the depth resolution cannot match a full CBCT image due to the limited angle acquisition. To address this issue, preoperative CT is a good auxiliary in guiding bronchoscopy interventions. Nevertheless, CT-to-body divergence caused by anatomic changes and respiratory motion, hinders the effective use of CT imaging. To mitigate CT-to-body divergence, we propose a novel deformable 3D/3D CT-to-DTS registration algorithm employing a multistage, multiresolution approach and using affine and elastic B-spline transformation models with bone and lung mask images. A multiresolution strategy with a Gaussian image pyramid and a multigrid strategy within the B-spline model are applied. The normalized correlation coefficient is included in the cost function for the affine model and a multimetric weighted cost function is used for the B-spline model, with weights determined heuristically. Tested on simulated and real patient bronchoscopy data, the algorithm yields promising results. Assessed qualitatively by visual inspection and quantitatively by computing the Dice coefficient (DC) and the average symmetric surface distance (ASSD), the algorithm achieves mean DC of 0.82±0.05 and 0.74±0.05, and mean ASSD of 0.65±0.29mm and 0.93±0.43mm for simulated and real data, respectively. This algorithm lays the groundwork for CT-aided intraoperative DTS imaging in image-guided bronchoscopy interventions with future studies focusing on automated metric weight setting.

Volumetric tumor tracking from a single cone-beam X-ray projection image enabled by deep learning

Radiotherapy serves as a pivotal treatment modality for malignant tumors. However, the accuracy of radiotherapy is significantly compromised due to respiratory-induced fluctuations in the size, shape, and position of the tumor. To address this challenge, we introduce a deep learning-anchored, volumetric tumor tracking methodology that employs single-angle X-ray projection images. This process involves aligning the intraoperative two-dimensional (2D) X-ray images with the pre-treatment three-dimensional (3D) planning Computed Tomography (CT) scans, enabling the extraction of the 3D tumor position and segmentation. Prior to therapy, a bespoke patient-specific tumor tracking model is formulated, leveraging a hybrid data augmentation, style correction, and registration network to create a mapping from single-angle 2D X-ray images to the corresponding 3D tumors. During the treatment phase, real-time X-ray images are fed into the trained model, producing the respective 3D tumor positioning. Rigorous validation conducted on actual patient lung data and lung phantoms attests to the high localization precision of our method at lowered radiation doses, thus heralding promising strides towards enhancing the precision of radiotherapy.

Enhancing the depth perception of DSA images with 2D-3D registration

Objective

Today, cerebrovascular disease has become an important health hazard. Therefore, it is necessary to perform a more accurate and less time-consuming registration of preoperative three-dimensional (3D) images and intraoperative two-dimensional (2D) projection images which is very important for conducting cerebrovascular disease interventions. The 2D-3D registration method proposed in this study is designed to solve the problems of long registration time and large registration errors in 3D computed tomography angiography (CTA) images and 2D digital subtraction angiography (DSA) images.

Methods

To make a more comprehensive and active diagnosis, treatment and surgery plan for patients with cerebrovascular diseases, we propose a weighted similarity measure function, the normalized mutual information-gradient difference (NMG), which can evaluate the 2D-3D registration results. Then, using a multi-resolution fusion optimization strategy, the multi-resolution fused regular step gradient descent optimization (MR-RSGD) method is presented to attain the optimal value of the registration results in the process of the optimization algorithm.

Result

In this study, we adopt two datasets of the brain vessels to validate and obtain similarity metric values which are 0.0037 and 0.0003, respectively. Using the registration method proposed in this study, the time taken for the experiment was calculated to be 56.55s and 50.8070s, respectively, for the two sets of data. The results show that the registration methods proposed in this study are both better than the Normalized Mutual (NM) and Normalized Mutual Information (NMI).

Conclusion

The experimental results in this study show that in the 2D-3D registration process, to evaluate the registration results more accurately, we can use the similarity metric function containing the image gray information and spatial information. To improve the efficiency of the registration process, we can choose the algorithm with gradient optimization strategy. Our method has great potential to be applied in practical interventional treatment for intuitive 3D navigation.

2D/3D Non-Rigid Image Registration via Two Orthogonal X-ray Projection Images for Lung Tumor Tracking

Two-dimensional (2D)/three-dimensional (3D) registration is critical in clinical applications. However, existing methods suffer from long alignment times and high doses. In this paper, a non-rigid 2D/3D registration method based on deep learning with orthogonal angle projections is proposed. The application can quickly achieve alignment using only two orthogonal angle projections. We tested the method with lungs (with and without tumors) and phantom data. The results show that the Dice and normalized cross-correlations are greater than 0.97 and 0.92, respectively, and the registration time is less than 1.2 seconds. In addition, the proposed model showed the ability to track lung tumors, highlighting the clinical potential of the proposed method.

A Novel Method for Improving the Accuracy of MR-derived Patient-specific Vascular Models using X-ray Angiography

MR imaging, a noninvasive radiation-free imaging modality commonly used during clinical follow up, has been widely utilized to reconstruct realistic 3D vascular models for patient-specific analysis. In recent work, we used patient-specific hemodynamic analysis of the circle of Willis to noninvasively assess stroke risk in pediatric Moyamoya disease (MMD)-a progressive steno-occlusive cerebrovascular disorder that leads to recurrent stroke. The objective was to identify vascular regions with critically high wall shear rate (WSR) that signifies elevated stroke risk. However, sources of error such as insufficient resolution of MR images can negatively impact vascular model accuracy, especially in areas of severe pathological narrowing, and thus diminish clinical relevance of simulation results, as local hemodynamics are sensitive to vessel geometry. To improve the accuracy of MR-derived vascular models, we have developed a novel method for adjusting model vessel geometry utilizing 2D X-ray angiography (XA), which is considered the gold standard for clinically assessing vessel caliber. In this workflow, "virtual angiographies" (VAs) of 3D MR-derived vascular models are conducted, producing 2D projections that are compared with corresponding XA images to guide the local adjustment of modeled vessels. This VA-comparison-adjustment loop is iterated until the two agree, as confirmed by an expert neuroradiologist. Using this method, we generated models of the circle of Willis of two patients with a history of unilateral stroke. Blood flow simulations were performed using a Navier-Stokes solver within an isogeometric analysis framework, and WSR distributions were quantified. Results for one patient show as much as 45% underestimation of local WSR in the stenotic left anterior cerebral artery (LACA), and up to a 56% underestimation in the right anterior cerebral artery when using the initial MR-derived model compared to the XA-adjusted model. To evaluate whether XA-based adjustment improves model accuracy, vessel cross-sectional areas of the pre- and post-adjustment models were compared to those seen in 3D CTA images of the same patient. CTA has superior resolution and signal-to-noise ratio compared to MR imaging but is not commonly used in the clinic due to radiation exposure concerns, especially in pediatric patients. While the vessels in the initial model had normalized root mean squared deviations (NRMSDs) ranging from 26% to 182% and 31% to 69% in two patients with respect to CTA, the adjusted vessel NRMSDs were comparatively smaller (32% to 53% and 11% to 42%). In the mildly stenotic LACA of patient 1, the NRMSDs for the pre- and post-adjusted models were 49% and 32%, respectively. These findings suggest that our XA-based adjustment method can considerably improve the accuracy of vascular models, and thus, stroke-risk prediction. An accurate, individualized assessment of stroke risk would be of substantial help in guiding the timing of preventive surgical interventions in pediatric MMD patients.

Long-range medical image registration through generalized mutual information (GMI): towards a fully automatic volumetric alignment

Objective. Mutual information (MI) is consolidated as a robust similarity metric often used for medical image registration. Although MI provides a robust registration, it usually fails when the transform needed to register an image is too large due to MI local minima traps. This paper proposes and evaluates Generalized MI (GMI), using Tsallis entropy, to improve affine registration. Approach. We assessed the GMI metric output space using separable affine transforms to seek a better gradient space. The range used was 150 mm for translations, 360° for rotations, [0.5, 2] for scaling, and [−1, 1] for skewness. The data were evaluated using 3D visualization of gradient and contour curves. A simulated gradient descent algorithm was also used to calculate the registration capability. The improvements detected were then tested through Monte Carlo simulation of actual registrations with brain T1 and T2 MRI from the HCP dataset. Main results. Results show significantly prolonged registration ranges, without local minima in the metric space, with a registration capability of 100% for translations, 88.2% for rotations, 100% for scaling and 100% for skewness. Tsallis entropy obtained 99.75% success in the Monte Carlo simulation of 2000 translation registrations with 1113 double randomized subjects T1 and T2 brain MRI against 56.5% success for the Shannon entropy. Significance. Tsallis entropy can improve brain MRI MI affine registration with long-range translation registration, lower-cost interpolation, and faster registrations through a better gradient space.