Deformable 2D-3D registration of the pelvis with a limited field of view, using shape statistics

The accuracy of statistical shape models in predicting bone shape: A systematic review

BACKGROUND

This systematic review aims to ascertain how accurately 3D models can be predicted from two-dimensional (2D) imaging utilising statistical shape modelling.

METHODS

A systematic search of published literature was conducted in September 2022. All papers which assessed the accuracy of 3D models predicted from 2D imaging utilising statistical shape models and which validated the models against the ground truth were eligible.

RESULTS

2127 papers were screened and a total of 34 studies were included for final data extraction. The best overall achievable accuracy was 0.45 mm (root mean square error) and 0.16 mm (average error).

CONCLUSION

Statistical shape modelling can predict detailed 3D anatomical models from minimal 2D imaging. Future studies should report the intended application domain of the model, the level of accuracy required, the underlying demographics of subjects, and the method in which accuracy was calculated, with root mean square error recommended if appropriate.

Realistic C-arm to pCT registration for vertebral localization in spine surgery

Abstract

Spine surgeries are vulnerable to wrong-level surgeries and postoperative complications because of their complex structure. Unavailability of the 3D intraoperative imaging device, low-contrast intraoperative X-ray images, variable clinical and patient conditions, manual analyses, lack of skilled technicians, and human errors increase the chances of wrong-site or wrong-level surgeries. State of the art work refers 3D-2D image registration systems and other medical image processing techniques to address the complications associated with spine surgeries. Intensity-based 3D-2D image registration systems had been widely practiced across various clinical applications. However, these frameworks are limited to specific clinical conditions such as anatomy, dimension of image correspondence, and imaging modalities. Moreover, there are certain prerequisites for these frameworks to function in clinical application, such as dataset requirement, speed of computation, requirement of high-end system configuration, limited capture range, and multiple local maxima. A simple and effective registration framework was designed with a study objective of vertebral level identification and its pose estimation from intraoperative fluoroscopic images by combining intensity-based and iterative control point (ICP)–based 3D-2D registration. A hierarchical multi-stage registration framework was designed that comprises coarse and finer registration. The coarse registration was performed in two stages, i.e., intensity similarity-based spatial localization and source-to-detector localization based on the intervertebral distance correspondence between vertebral centroids in projected and intraoperative X-ray images. Finally, to speed up target localization in the intraoperative application, based on 3D-2D vertebral centroid correspondence, a rigid ICP-based finer registration was performed. The mean projection distance error (mPDE) measurement and visual similarity between projection image at finer registration point and intraoperative X-ray image and surgeons’ feedback were held accountable for the quality assurance of the designed registration framework. The average mPDE after peak signal to noise ratio (PSNR)–based coarse registration was 20.41mm. After the coarse registration in spatial region and source to detector direction, the average mPDE reduced to 12.18mm. On finer ICP-based registration, the mean mPDE was finally reduced to 0.36 mm. The approximate mean time required for the coarse registration, finer registration, and DRR image generation at the final registration point were 10 s, 15 s, and 1.5 min, respectively. The designed registration framework can act as a supporting tool for vertebral level localization and its pose estimation in an intraoperative environment. The framework was designed with the future perspective of intraoperative target localization and its pose estimation irrespective of the target anatomy.

Graphical abstract

Pose Estimation of Periacetabular Osteotomy Fragments With Intraoperative X-Ray Navigation

OBJECTIVE

State-of-the-art navigation systems for pelvic osteotomies use optical systems with external fiducials. In this paper, we propose the use of X-ray navigation for pose estimation of periacetabular fragments without fiducials.

METHODS

A two-dimensional/three-dimensional (2-D/3-D) registration pipeline was developed to recover fragment pose. This pipeline was tested through an extensive simulation study and six cadaveric surgeries. Using osteotomy boundaries in the fluoroscopic images, the preoperative plan was refined to more accurately match the intraoperative shape.

RESULTS

In simulation, average fragment pose errors were 1.3 ° /1.7 mm when the planned fragment matched the intraoperative fragment, 2.2 ° /2.1 mm when the plan was not updated to match the true shape, and 1.9 ° /2.0 mm when the fragment shape was intraoperatively estimated. In cadaver experiments, the average pose errors were 2.2  ° /2.2 mm, 3.8 ° /2.5 mm, and 3.5  ° /2.2 mm when registering with the actual fragment shape, a preoperative plan, and an intraoperatively refined plan, respectively. Average errors of the lateral center edge angle were less than 2 ° for all fragment shapes in simulation and cadaver experiments.

CONCLUSION

The proposed pipeline is capable of accurately reporting femoral head coverage within a range clinically identified for long-term joint survivability.

SIGNIFICANCE

Human interpretation of fragment pose is challenging and usually restricted to rotation about a single anatomical axis. The proposed pipeline provides an intraoperative estimate of rigid pose with respect to all anatomical axes, is compatible with minimally invasive incisions, and has no dependence on external fiducials.

Review of 2-D/3-D Reconstruction Using Statistical Shape and Intensity Models and X-Ray Image Synthesis: Toward a Unified Framework

Patient-specific three-dimensional (3-D) bone models are useful for a number of clinical applications such as surgery planning, postoperative evaluation, as well as implant and prosthesis design. Two-dimensional-to-3-D (2-D/3-D) reconstruction, also known as model-to-modality or atlas-based 2-D/3-D registration, provides a means of obtaining a 3-D model of a patient's bones from their 2-D radiographs when 3-D imaging modalities are not available. The preferred approach for estimating both shape and density information (that would be present in a patient's computed tomography data) for 2-D/3-D reconstruction makes use of digitally reconstructed radiographs and deformable models in an iterative, non-rigid, intensity-based approach. Based on a large number of state-of-the-art 2-D/3-D bone reconstruction methods, a unified mathematical formulation of the problem is proposed in a common conceptual framework, using unambiguous terminology. In addition, shortcomings, recent adaptations, and persisting challenges are discussed along with insights for future research.

3X-Knee: A Novel Technology for 3D Preoperative Planning and Postoperative Evaluation of TKA Based on 2D X-Rays

This chapter introduces a solution called "3X-knee" that can robustly derive 3D models of the lower extremity from 2D long leg standing X-ray radiographs for preoperative planning and postoperative treatment evaluation of total knee arthroplasty (TKA). There are three core components in 3X-knee technology: (1) a knee joint immobilization apparatus, (2) an X-ray image calibration phantom, and (3) a statistical shape model-based 2D-3D reconstruction algorithm. These three components are integrated in a systematic way in 3X-knee to derive 3D models of the complete lower extremity from 2D long leg standing X-ray radiographs acquired in weight-bearing position. More specifically, the knee joint immobilization apparatus will be used to rigidly fix the X-ray calibration phantom with respect to the underlying anatomy during the image acquisition. The calibration phantom then serves two purposes. For one side, the phantom will allow one to calibrate the projection parameters of any acquired X-ray image. For the other side, the phantom also allowsone to track positions of multiple X-ray images of the underlying anatomy without using any additional positional tracker, which is a prerequisite condition for the third component to compute patient-specific 3D models from 2D X-ray images and the associated statistical shape models. Validation studies conducted on both simulated X-ray images and on patients' X-ray data demonstrate the efficacy of the present solution.

Pose-aware C-arm for automatic re-initialization of interventional 2D/3D image registration

PURPOSE

In minimally invasive interventions assisted by C-arm imaging, there is a demand to fuse the intra-interventional 2D C-arm image with pre-interventional 3D patient data to enable surgical guidance. The commonly used intensity-based 2D/3D registration has a limited capture range and is sensitive to initialization. We propose to utilize an opto/X-ray C-arm system which allows to maintain the registration during intervention by automating the re-initialization for the 2D/3D image registration. Consequently, the surgical workflow is not disrupted and the interaction time for manual initialization is eliminated.

METHODS

We utilize two distinct vision-based tracking techniques to estimate the relative poses between different C-arm arrangements: (1) global tracking using fused depth information and (2) RGBD SLAM system for surgical scene tracking. A highly accurate multi-view calibration between RGBD and C-arm imaging devices is achieved using a custom-made multimodal calibration target.

RESULTS

Several in vitro studies are conducted on pelvic-femur phantom that is encased in gelatin and covered with drapes to simulate a clinically realistic scenario. The mean target registration errors (mTRE) for re-initialization using depth-only and RGB [Formula: see text] depth are 13.23 mm and 11.81 mm, respectively. 2D/3D registration yielded 75% success rate using this automatic re-initialization, compared to a random initialization which yielded only 23% successful registration.

CONCLUSION

The pose-aware C-arm contributes to the 2D/3D registration process by globally re-initializing the relationship of C-arm image and pre-interventional CT data. This system performs inside-out tracking, is self-contained, and does not require any external tracking devices.

Fully automatic reconstruction of personalized 3D volumes of the proximal femur from 2D X-ray images

PURPOSE

Accurate preoperative planning is crucial for the outcome of total hip arthroplasty. Recently, 2D pelvic X-ray radiographs have been replaced by 3D CT. However, CT suffers from relatively high radiation dosage and cost. An alternative is to reconstruct a 3D patient-specific volume data from 2D X-ray images.

METHODS

In this paper, based on a fully automatic image segmentation algorithm, we propose a new control point-based 2D-3D registration approach for a deformable registration of a 3D volumetric template to a limited number of 2D calibrated X-ray images and show its application to personalized reconstruction of 3D volumes of the proximal femur. The 2D-3D registration is done with a hierarchical two-stage strategy: the scaled-rigid 2D-3D registration stage followed by a regularized deformable B-spline 2D-3D registration stage. In both stages, a set of control points with uniform spacing are placed over the domain of the 3D volumetric template first. The registration is then driven by computing updated positions of these control points with intensity-based 2D-2D image registrations of the input X-ray images with the associated digitally reconstructed radiographs, which allows computing the associated registration transformation at each stage.

RESULTS

Evaluated on datasets of 44 patients, our method achieved an overall surface reconstruction accuracy of [Formula: see text] and an average Dice coefficient of [Formula: see text]. We further investigated the cortical bone region reconstruction accuracy, which is important for planning cementless total hip arthroplasty. An average cortical bone region Dice coefficient of [Formula: see text] and an inner cortical bone surface reconstruction accuracy of [Formula: see text] were found.

CONCLUSIONS

In summary, we developed a new approach for reconstruction of 3D personalized volumes of the proximal femur from 2D X-ray images. Comprehensive experiments demonstrated the efficacy of the present approach.

Known-component 3D-2D registration for quality assurance of spine surgery pedicle screw placement

A 3D-2D image registration method is presented that exploits knowledge of interventional devices (e.g. K-wires or spine screws-referred to as 'known components') to extend the functionality of intraoperative radiography/fluoroscopy by providing quantitative measurement and quality assurance (QA) of the surgical product. The known-component registration (KC-Reg) algorithm uses robust 3D-2D registration combined with 3D component models of surgical devices known to be present in intraoperative 2D radiographs. Component models were investigated that vary in fidelity from simple parametric models (e.g. approximation of a screw as a simple cylinder, referred to as 'parametrically-known' component [pKC] registration) to precise models based on device-specific CAD drawings (referred to as 'exactly-known' component [eKC] registration). 3D-2D registration from three intraoperative radiographs was solved using the covariance matrix adaptation evolution strategy (CMA-ES) to maximize image-gradient similarity, relating device placement relative to 3D preoperative CT of the patient. Spine phantom and cadaver studies were conducted to evaluate registration accuracy and demonstrate QA of the surgical product by verification of the type of devices delivered and conformance within the 'acceptance window' of the spinal pedicle. Pedicle screws were successfully registered to radiographs acquired from a mobile C-arm, providing TRE 1-4 mm and  <5° using simple parametric (pKC) models, further improved to  <1 mm and  <1° using eKC registration. Using advanced pKC models, screws that did not match the device models specified in the surgical plan were detected with an accuracy of  >99%. Visualization of registered devices relative to surgical planning and the pedicle acceptance window provided potentially valuable QA of the surgical product and reliable detection of pedicle screw breach. 3D-2D registration combined with 3D models of known surgical devices offers a novel method for intraoperative QA. The method provides a near-real-time independent check against pedicle breach, facilitating revision within the same procedure if necessary and providing more rigorous verification of the surgical product.

Principal component analysis in construction of 3D human knee joint models using a statistical shape model method

The statistical shape model (SSM) method that uses 2D images of the knee joint to predict the three-dimensional (3D) joint surface model has been reported in the literature. In this study, we constructed a SSM database using 152 human computed tomography (CT) knee joint models, including the femur, tibia and patella and analysed the characteristics of each principal component of the SSM. The surface models of two in vivo knees were predicted using the SSM and their 2D bi-plane fluoroscopic images. The predicted models were compared to their CT joint models. The differences between the predicted 3D knee joint surfaces and the CT image-based surfaces were 0.30 ± 0.81 mm, 0.34 ± 0.79 mm and 0.36 ± 0.59 mm for the femur, tibia and patella, respectively (average ± standard deviation). The computational time for each bone of the knee joint was within 30 s using a personal computer. The analysis of this study indicated that the SSM method could be a useful tool to construct 3D surface models of the knee with sub-millimeter accuracy in real time. Thus, it may have a broad application in computer-assisted knee surgeries that require 3D surface models of the knee.

Automatic localization of vertebral levels in x-ray fluoroscopy using 3D-2D registration: a tool to reduce wrong-site surgery

Surgical targeting of the incorrect vertebral level (wrong-level surgery) is among the more common wrong-site surgical errors, attributed primarily to the lack of uniquely identifiable radiographic landmarks in the mid-thoracic spine. The conventional localization method involves manual counting of vertebral bodies under fluoroscopy, is prone to human error and carries additional time and dose. We propose an image registration and visualization system (referred to as LevelCheck), for decision support in spine surgery by automatically labeling vertebral levels in fluoroscopy using a GPU-accelerated, intensity-based 3D-2D (namely CT-to-fluoroscopy) registration. A gradient information (GI) similarity metric and a CMA-ES optimizer were chosen due to their robustness and inherent suitability for parallelization. Simulation studies involved ten patient CT datasets from which 50 000 simulated fluoroscopic images were generated from C-arm poses selected to approximate the C-arm operator and positioning variability. Physical experiments used an anthropomorphic chest phantom imaged under real fluoroscopy. The registration accuracy was evaluated as the mean projection distance (mPD) between the estimated and true center of vertebral levels. Trials were defined as successful if the estimated position was within the projection of the vertebral body (namely mPD <5 mm). Simulation studies showed a success rate of 99.998% (1 failure in 50 000 trials) and computation time of 4.7 s on a midrange GPU. Analysis of failure modes identified cases of false local optima in the search space arising from longitudinal periodicity in vertebral structures. Physical experiments demonstrated the robustness of the algorithm against quantum noise and x-ray scatter. The ability to automatically localize target anatomy in fluoroscopy in near-real-time could be valuable in reducing the occurrence of wrong-site surgery while helping to reduce radiation exposure. The method is applicable beyond the specific case of vertebral labeling, since any structure defined in pre-operative (or intra-operative) CT or cone-beam CT can be automatically registered to the fluoroscopic scene.

Intraoperative image-based multiview 2D/3D registration for image-guided orthopaedic surgery: incorporation of fiducial-based C-arm tracking and GPU-acceleration

Intraoperative patient registration may significantly affect the outcome of image-guided surgery (IGS). Image-based registration approaches have several advantages over the currently dominant point-based direct contact methods and are used in some industry solutions in image-guided radiation therapy with fixed X-ray gantries. However, technical challenges including geometric calibration and computational cost have precluded their use with mobile C-arms for IGS. We propose a 2D/3D registration framework for intraoperative patient registration using a conventional mobile X-ray imager combining fiducial-based C-arm tracking and graphics processing unit (GPU)-acceleration. The two-stage framework 1) acquires X-ray images and estimates relative pose between the images using a custom-made in-image fiducial, and 2) estimates the patient pose using intensity-based 2D/3D registration. Experimental validations using a publicly available gold standard dataset, a plastic bone phantom and cadaveric specimens have been conducted. The mean target registration error (mTRE) was 0.34 ± 0.04 mm (success rate: 100%, registration time: 14.2 s) for the phantom with two images 90° apart, and 0.99 ± 0.41 mm (81%, 16.3 s) for the cadaveric specimen with images 58.5° apart. The experimental results showed the feasibility of the proposed registration framework as a practical alternative for IGS routines.

Personalized X-ray reconstruction of the proximal femur via intensity-based non-rigid 2D-3D registration

This paper presents a new approach for reconstructing a patient-specific shape model and internal relative intensity distribution of the proximal femur from a limited number (e.g., 2) of calibrated C-arm images or X-ray radiographs. Our approach uses independent shape and appearance models that are learned from a set of training data to encode the a priori information about the proximal femur. An intensity-based non-rigid 2D-3D registration algorithm is then proposed to deformably fit the learned models to the input images. The fitting is conducted iteratively by minimizing the dissimilarity between the input images and the associated digitally reconstructed radiographs of the learned models together with regularization terms encoding the strain energy of the forward deformation and the smoothness of the inverse deformation. Comprehensive experiments conducted on images of cadaveric femurs and on clinical datasets demonstrate the efficacy of the present approach.

Construction of 3D human distal femoral surface models using a 3D statistical deformable model

Construction of 3D geometric surface models of human knee joint is always a challenge in biomedical engineering. This study introduced an improved statistical shape model (SSM) method that only uses 2D images of a joint to predict the 3D joint surface model. The SSM was constructed using 40 distal femur models of human knees. In this paper, a series validation and parametric analysis suggested that more than 25 distal femur models are needed to construct the SSM; each distal femur should be described using at least 3000 nodes in space; and two 2D fluoroscopic images taken in 45° directions should be used for the 3D surface shape prediction. Using this SSM method, ten independent distal femurs from 10 independent living subjects were predicted using their 2D plane fluoroscopic images. The predicted models were compared to their native 3D distal femur models constructed using their 3D MR images. The results demonstrated that using two fluoroscopic images of the knee, the overall difference between the predicted distal femur surface and the MR image-based surface was 0.16±1.16 mm. These data indicated that the SSM method could be a powerful method for construction of 3D surface geometries of the distal femur.

Hybrid cone-beam tomographic reconstruction: incorporation of prior anatomical models to compensate for missing data

We propose a method for improving the quality of cone-beam tomographic reconstruction done with a C-arm. C-arm scans frequently suffer from incomplete information due to image truncation, limited scan length, or other limitations. Our proposed "hybrid reconstruction" method injects information from a prior anatomical model, derived from a subject-specific computed tomography (CT) or from a statistical database (atlas), where the C-arm X-ray data is missing. This significantly reduces reconstruction artifacts with little loss of true information from the X-ray projections. The methods consist of constructing anatomical models, fast rendering of digitally reconstructed radiograph (DRR) projections of the models, rigid or deformable registration of the model and the X-ray images, and fusion of the DRR and X-ray projections, all prior to a conventional filtered back-projection algorithm. Our experiments, conducted with a mobile image intensifier C-arm, demonstrate visually and quantitatively the contribution of data fusion to image quality, which we assess through comparison to a "ground truth" CT. Importantly, we show that a significantly improved reconstruction can be obtained from a C-arm scan as short as 90° by complementing the observed projections with DRRs of two prior models, namely an atlas and a preoperative same-patient CT. The hybrid reconstruction principles are applicable to other types of C-arms as well.

Enhancement of mobile C-arm cone-beam reconstruction using prior anatomical models

We demonstrate an improvement to cone-beam tomographic imaging by using a prior anatomical model. A protocol for scanning and reconstruction has been designed and implemented for a conventional mobile C-arm: a 9 inch image-intensifier OEC-9600. Due to the narrow field of view (FOV), the reconstructed image contains strong truncation artifacts. We propose to improve the reconstructed images by fusing the observed x-ray data with computed projections of a prior 3D anatomical model, derived from a subject-specific CT or from a statistical database (atlas), and co-registered (3D/2D) to the x-rays.The prior model contains a description of geometry and radiodensity as a tetrahedral mesh shape and density polynomials, respectively. A CT-based model can be created by segmentation, meshing and polynomial fitting of the object's CT study. The statistical atlas is created through principal component analysis (PCA) of a collection of mesh instances deformably-registered (3D/3D) to patient datasets.The 3D/2D registration method optimizes a pixel-based similarity score (mutual information) between the observed x-rays and the prior. The transformation involves translation, rotation and shape deformation based on the atlas. After registration, the image intensities of observed and prior projections are matched and adjusted, and the two information sources are blended as inputs to a reconstruction algorithm.We demonstrate recostruction results of three cadaveric specimens, and the effect of fusing prior data to compensate for truncation. Further uses of hybrid reconstruction, such as compensation for the scan's limited arc length, are suggested for future research.