Intraoperative image-based multiview 2D/3D registration for image-guided orthopaedic surgery: incorporation of fiducial-based C-arm tracking and GPU-acceleration

A Novel System for Markerless Intra-Operative Bone and Bone Fragment Tracking

Fluoroscopic guidance is an integral tool in modern orthopedic surgery often used to track bones and/or bone fragments during a surgical procedure. However, relying upon this intra-operative 2D projective imaging modality for this purpose can challenge a surgeon's ability to interpret 3D position and orientation of any but the simplest bony anatomy. A number of object-tracking technologies have been developed to aid surgeons, but they have failed to be generalizable to a wider array of procedures, have required an unrealistic amount of time and effort to implement, or have unacceptably changed the flow of the surgery. This work describes a novel, general-purpose system for markerless, intra-operative bone tracking that seamlessly integrates into a surgical setting. The system uses a unique calibration object placed next to the patient, which provides a common reference for aligning multiple fluoroscopic images. This approach enables robust and expedient 3D object registration from only two semi-orthogonal 2D fluoroscopic images.

Toward Semantically-Consistent Deformable 2D-3D Registration for 3D Craniofacial Structure Estimation From a Single-View Lateral Cephalometric Radiograph

The deep neural networks combined with the statistical shape model have enabled efficient deformable 2D-3D registration and recovery of 3D anatomical structures from a single radiograph. However, the recovered volumetric image tends to lack the volumetric fidelity of fine-grained anatomical structures and explicit consideration of cross-dimensional semantic correspondence. In this paper, we introduce a simple but effective solution for semantically-consistent deformable 2D-3D registration and detailed volumetric image recovery by inferring a voxel-wise registration field between the cone-beam computed tomography and a single lateral cephalometric radiograph (LC). The key idea is to refine the initial statistical model-based registration field with craniofacial structural details and semantic consistency from the LC. Specifically, our framework employs a self-supervised scheme to learn a voxel-level refiner of registration fields to provide fine-grained craniofacial structural details and volumetric fidelity. We also present a weakly supervised semantic consistency measure for semantic correspondence, relieving the requirements of volumetric image collections and annotations. Experiments showcase that our method achieves deformable 2D-3D registration with performance gains over state-of-the-art registration and radiograph-based volumetric reconstruction methods. The source code is available at https://github.com/Jyk-122/SC-DREG.

Soft-tissue sound-speed-aware ultrasound-CT registration method for computer-assisted orthopedic surgery

Ultrasound (US) has been introduced to computer-assisted orthopedic surgery for bone registration owing to its advantages of nonionizing radiation, low cost, and noninvasiveness. However, the registration accuracy is limited by US image distortion caused by variations in the acoustic properties of soft tissues. This paper proposes a soft-tissue sound-speed-aware registration method to overcome the above challenge. First, the feature enhancement strategy of multi-channel overlay is proposed for U2-net to improve bone segmentation performance. Secondly, the sound speed of soft tissue is estimated by simulating the bone surface distance map for the update of US-derived points. Finally, an iterative registration strategy is adopted to optimize the registration result. A phantom experiment was conducted using different registration methods for the femur and tibia/fibula. The fiducial registration error (femur, 0.98 ± 0.08 mm (mean ± SD); tibia/fibula, 1.29 ± 0.19 mm) and the target registration error (less than 2.11 mm) showed the high accuracy of the proposed method. The experimental results suggest that the proposed method can be integrated into navigation systems that provide surgeons with accurate 3D navigation information.

Feature-based multimodal registration framework for vertebral pose estimation

PURPOSE

The reliable estimation of the vertebral body posture helps to aid a safe and effective spine surgery. The proposed work aims to present an MR to X-ray image registration to assess the 3D pose of the vertebral body during spine surgery. The 3D assessment of vertebral pose assists in analyzing the position and orientation of the vertebral body to provide information during various clinical diagnosis conditions such as curvature estimation and pedicle screw insertion surgery.

METHODS

The proposed feature-based registration framework extracted vertebral end plates to avoid the mismatch between the intensities of MR and X-ray images. Using the projection matrix, the segmented MRI is forward projected and then registered to the X-ray image using binary image matching similarity and the CMA-ES optimizer.

RESULTS

The proposed method estimated the vertebral pose by registering the simulated X-ray onto pre-operative MRI. To evaluate the efficacy of the proposed approach, a certain number of experiments are carried out on the simulated dataset.

CONCLUSION

The proposed method is a fast and accurate registration method that can provide 3D information about the vertebral body. This 3D information is useful to improve accuracy during various clinical diagnoses.

Scan-Free and Fully Automatic Tracking of Native Knee Anatomy from Dynamic Stereo-Radiography with Statistical Shape and Intensity Models

Kinematic tracking of native anatomy from stereo-radiography provides a quantitative basis for evaluating human movement. Conventional tracking procedures require significant manual effort and call for acquisition and annotation of subject-specific volumetric medical images. The current work introduces a framework for fully automatic tracking of native knee anatomy from dynamic stereo-radiography which forgoes reliance on volumetric scans. The method consists of three computational steps. First, captured radiographs are annotated with segmentation maps and anatomic landmarks using a convolutional neural network. Next, a non-convex polynomial optimization problem formulated from annotated landmarks is solved to acquire preliminary anatomy and pose estimates. Finally, a global optimization routine is performed for concurrent refinement of anatomy and pose. An objective function is maximized which quantifies similarities between masked radiographs and digitally reconstructed radiographs produced from statistical shape and intensity models. The proposed framework was evaluated against manually tracked trials comprising dynamic activities, and additional frames capturing a static knee phantom. Experiments revealed anatomic surface errors routinely below 1.0 mm in both evaluation cohorts. Median absolute errors of individual bone pose estimates were below 1.0∘ or mm for 15 out of 18 degrees of freedom in both evaluation cohorts. Results indicate that accurate pose estimation of native anatomy from stereo-radiography may be performed with significantly reduced manual effort, and without reliance on volumetric scans.

Fast digitally reconstructed radiograph generation using particle-based statistical shape and intensity model

Purpose

Statistical shape and intensity models (SSIMs) and digitally reconstructed radiographs (DRRs) were introduced for non-rigid 2D-3D registration and skeletal geometry/density reconstruction studies. The computation of DRRs takes most of the time during registration or reconstruction. The goal of this study is to propose a particle-based method for composing an SSIM and a DRR image generation scheme and analyze the quality of the images compared with previous DRR generation methods.

Approach

Particle-based SSIMs consist of densely scattered particles on the surface and inside of an object, with each particle having an intensity value. Generating the DRR resembles ray tracing, which counts the particles that are binned with each ray and calculates the radiation attenuation. The distance between adjacent particles was considered to be the radiologic path during attenuation integration, and the mean linear attenuation coefficient of the two particles was multiplied. The proposed method was compared with the DRR of CT projection. The mean squared error and peak signal-to-noise ratio (PSNR) were calculated between the DRR images from the proposed method and those of existing methods of projecting tetrahedral-based SSIMs or computed tomography (CT) images to verify the accuracy of the proposed scheme.

Results

The suggested method was about 600 times faster than the tetrahedral-based SSIM without using the hardware acceleration technique. The PSNR was 37.59 dB, and the root mean squared error of the normalized pixel intensities was 0.0136.

Conclusions

The proposed SSIM and DRR generation procedure showed high temporal performance while maintaining image quality, and particle-based SSIM is a feasible form for representing a 3D volume and generating the DRR images.

Reproduction of forearm rotation dynamic using intensity-based biplane 2D-3D registration matching method

This study aimed to reproduce and analyse the in vivo dynamic rotational motion of the forearm and to clarify forearm motion involvement and the anatomical function of the interosseous membrane (IOM). The dynamic forearm rotational motion of the radius and ulna was analysed in vivo using a novel image-matching method based on fluoroscopic and computed tomography images for intensity-based biplane two-dimensional-three-dimensional registration. Twenty upper limbs from 10 healthy volunteers were included in this study. The mean range of forearm rotation was 150 ± 26° for dominant hands and 151 ± 18° for non-dominant hands, with no significant difference observed between the two. The radius was most proximal to the maximum pronation relative to the ulna, moved distally toward 60% of the rotation range from maximum pronation, and again proximally toward supination. The mean axial translation of the radius relative to the ulna during forearm rotation was 1.8 ± 0.8 and 1.8 ± 0.9 mm for dominant and non-dominant hands, respectively. The lengths of the IOM components, excluding the central band (CB), changed rotation. The transverse CB length was maximal at approximately 50% of the rotation range from maximum pronation. Summarily, this study describes a detailed method for evaluating in vivo dynamic forearm motion and provides valuable insights into forearm kinematics and IOM function.

Fast X-ray/CT image registration based on perspective projection triangular features

X-ray/CT image registration plays a pivotal role in enhancing surgical navigation success rates. However, challenges stemming from sparse and noisy X-ray image features, coupled with the complexities of multi-parameter optimization, impose limitations on existing methods in terms of registration accuracy and efficiency. In response, this paper presents an innovative approach-a fast X-ray/CT image registration method based on perspective projection triangular features(F-PPTF). By leveraging the conformal nature of perspective projection, the proposed method constructs perspective projection triangular features with rotation, translation, and scale invariance using point feature descriptors. Diverging from multi-parameter iterative optimization techniques, this approach achieves the decoupling of the six transformation parameters. This decoupling simplifies computational intricacies, thereby facilitating swift registration. Experimental evaluations conducted on synthetic and real X-ray images reveal an average rotational absolute error of 0.41°, an average translational absolute error of 1.16 mm, and an average registration time of 14.89 s. In comparison to conventional registration methodologies, the method presented in this paper demonstrates pronounced superiority in terms of both registration accuracy and efficiency, thereby exhibiting heightened potential for broader applicability.

Fully automatic tracking of native knee kinematics from stereo-radiography with digitally reconstructed radiographs

Precise measurement of joint-level motion from stereo-radiography facilitates understanding of human movement. Conventional procedures for kinematic tracking require significant manual effort and are time intensive. The current work introduces a method for fully automatic tracking of native knee kinematics from stereo-radiography sequences. The framework consists of three computational steps. First, biplanar radiograph frames are annotated with segmentation maps and key points using a convolutional neural network. Next, initial bone pose estimates are acquired by solving a polynomial optimization problem constructed from annotated key points and anatomic landmarks from digitized models. A semidefinite relaxation is formulated to realize the global minimum of the non-convex problem. Pose estimates are then refined by registering computed tomography-based digitally reconstructed radiographs to masked radiographs. A novel rendering method is also introduced which enables generating digitally reconstructed radiographs from computed tomography scans with inconsistent slice widths. The automatic tracking framework was evaluated with stereo-radiography trials manually tracked with model-image registration, and with frames which capture a synthetic leg phantom. The tracking method produced pose estimates which were consistently similar to manually tracked values; and demonstrated pose errors below 1.0 degree or millimeter for all femur and tibia degrees of freedom in phantom trials. Results indicate the described framework may benefit orthopaedics and biomechanics applications through acceleration of kinematic tracking.

The effect of forearm rotation on the bone mineral density measurements of the distal radius

INTRODUCTION

Forearm dual-energy X-ray absorptiometry (DXA) is often performed in clinics where central DXA is unavailable. Accurate bone mineral density (BMD) measurement is crucial for clinical assessment. Forearm rotation can affect BMD measurements, but this effect remains uncertain. Thus, we aimed to conduct a simulation study using CT images to clarify the effect of forearm rotation on BMD measurements.

MATERIALS AND METHODS

Forearm CT images of 60 women were analyzed. BMD was measured at the total, ultra-distal (UD), mid-distal (MD), and distal 33% radius regions with the radius located at the neutral position using digitally reconstructed radiographs generated from CT images. Then, the rotation was altered from - 30° to 30° (supination set as positive) with a one-degree increment, and the percent BMD changes from the neutral position were quantified for all regions at each angle for each patient.

RESULTS

The maximum mean BMD changes were 5.8%, 7.0%, 6.2%, and 7.2% for the total, UD, MD, and distal 33% radius regions, respectively. The analysis of the absolute values of the percent BMD changes from the neutral position showed that BMD changes of all patients remained within 2% when the rotation was between - 5° and 7° for the total region, between - 3° and 2° for the UD region, between - 4° and 3° for the MD region, and between - 3° and 1° for the distal 33% radius region.

CONCLUSION

Subtle rotational changes affected the BMD measurement of each region. The results showed the importance of forearm positioning when measuring the distal radius BMD.

Bone mineral density estimation from a plain X-ray image by learning decomposition into projections of bone-segmented computed tomography

Osteoporosis is a prevalent bone disease that causes fractures in fragile bones, leading to a decline in daily living activities. Dual-energy X-ray absorptiometry (DXA) and quantitative computed tomography (QCT) are highly accurate for diagnosing osteoporosis; however, these modalities require special equipment and scan protocols. To frequently monitor bone health, low-cost, low-dose, and ubiquitously available diagnostic methods are highly anticipated. In this study, we aim to perform bone mineral density (BMD) estimation from a plain X-ray image for opportunistic screening, which is potentially useful for early diagnosis. Existing methods have used multi-stage approaches consisting of extraction of the region of interest and simple regression to estimate BMD, which require a large amount of training data. Therefore, we propose an efficient method that learns decomposition into projections of bone-segmented QCT for BMD estimation under limited datasets. The proposed method achieved high accuracy in BMD estimation, where Pearson correlation coefficients of 0.880 and 0.920 were observed for DXA-measured BMD and QCT-measured BMD estimation tasks, respectively, and the root mean square of the coefficient of variation values were 3.27 to 3.79% for four measurements with different poses. Furthermore, we conducted extensive validation experiments, including multi-pose, uncalibrated-CT, and compression experiments toward actual application in routine clinical practice.

Development and validation of an open-source tool for opportunistic screening of osteoporosis from hip CT images

Aims

This study aimed to develop and validate a fully automated system that quantifies proximal femoral bone mineral density (BMD) from CT images.

Methods

The study analyzed 978 pairs of hip CT and dual-energy X-ray absorptiometry (DXA) measurements of the proximal femur (DXA-BMD) collected from three institutions. From the CT images, the femur and a calibration phantom were automatically segmented using previously trained deep-learning models. The Hounsfield units of each voxel were converted into density (mg/cm3). Then, a deep-learning model trained by manual landmark selection of 315 cases was developed to select the landmarks at the proximal femur to rotate the CT volume to the neutral position. Finally, the CT volume of the femur was projected onto the coronal plane, and the areal BMD of the proximal femur (CT-aBMD) was quantified. CT-aBMD correlated to DXA-BMD, and a receiver operating characteristic (ROC) analysis quantified the accuracy in diagnosing osteoporosis.

Results

CT-aBMD was successfully measured in 976/978 hips (99.8%). A significant correlation was found between CT-aBMD and DXA-BMD (r = 0.941; p < 0.001). In the ROC analysis, the area under the curve to diagnose osteoporosis was 0.976. The diagnostic sensitivity and specificity were 88.9% and 96%, respectively, with the cutoff set at 0.625 g/cm2.

Conclusion

Accurate DXA-BMD measurements and diagnosis of osteoporosis were performed from CT images using the system developed herein. As the models are open-source, clinicians can use the proposed system to screen osteoporosis and determine the surgical strategy for hip surgery.

Fully automatic tracking of native glenohumeral kinematics from stereo-radiography

The current work introduces a system for fully automatic tracking of native glenohumeral kinematics in stereo-radiography sequences. The proposed method first applies convolutional neural networks to obtain segmentation and semantic key point predictions in biplanar radiograph frames. Preliminary bone pose estimates are computed by solving a non-convex optimization problem with semidefinite relaxations to register digitized bone landmarks to semantic key points. Initial poses are then refined by registering computed tomography-based digitally reconstructed radiographs to captured scenes, which are masked by segmentation maps to isolate the shoulder joint. A particular neural net architecture which exploits subject-specific geometry is also introduced to improve segmentation predictions and increase robustness of subsequent pose estimates. The method is evaluated by comparing predicted glenohumeral kinematics to manually tracked values from 17 trials capturing 4 dynamic activities. Median orientation differences between predicted and ground truth poses were 1.7∘ and 8.6∘ for the scapula and humerus, respectively. Joint-level kinematics differences were less than 2∘ in 65%, 13%, and 63% of frames for XYZ orientation DoFs based on Euler angle decompositions. Automation of kinematic tracking can increase scalability of tracking workflows in research, clinical, or surgical applications.

Decomposition of musculoskeletal structures from radiographs using an improved CycleGAN framework

This paper presents methods of decomposition of musculoskeletal structures from radiographs into multiple individual muscle and bone structures. While existing solutions require dual-energy scan for the training dataset and are mainly applied to structures with high-intensity contrast, such as bones, we focused on multiple superimposed muscles with subtle contrast in addition to bones. The decomposition problem is formulated as an image translation problem between (1) a real X-ray image and (2) multiple digitally reconstructed radiographs, each of which contains a single muscle or bone structure, and solved using unpaired training based on the CycleGAN framework. The training dataset was created via automatic computed tomography (CT) segmentation of muscle/bone regions and virtually projecting them with geometric parameters similar to the real X-ray images. Two additional features were incorporated into the CycleGAN framework to achieve a high-resolution and accurate decomposition: hierarchical learning and reconstruction loss with the gradient correlation similarity metric. Furthermore, we introduced a new diagnostic metric for muscle asymmetry directly measured from a plain X-ray image to validate the proposed method. Our simulation and real-image experiments using real X-ray and CT images of 475 patients with hip diseases suggested that each additional feature significantly enhanced the decomposition accuracy. The experiments also evaluated the accuracy of muscle volume ratio measurement, which suggested a potential application to muscle asymmetry assessment from an X-ray image for diagnostic and therapeutic assistance. The improved CycleGAN framework can be applied for investigating the decomposition of musculoskeletal structures from single radiographs.

The effect of patient positioning on measurements of bone mineral density of the proximal femur: a simulation study using computed tomographic images

The patient's position may affect the bone mineral density (BMD) measurements; however, the extent of this effect is undefined. This CT image-based simulation study quantified changes in BMD induced by hip flexion, adduction, and rotations to recommend appropriate patient positioning when acquiring dual-energy x-ray absorptiometry images.

PURPOSE

Several studies have analyzed the effect of hip rotation on the measurement of bone mineral density (BMD) of the proximal femur by dual-energy x-ray absorptiometry (DXA). However, as the effects of hip flexion and abduction on BMD measurements remain uncertain, a computational simulation study using CT images was performed in this study.

METHODS

Hip CT images of 120 patients (33 men and 87 women; mean age, 82.1 ± 9.4 years) were used for analysis. Digitally reconstructed radiographs of the proximal femur region were generated from CT images to calculate the BMD of the proximal femur region. BMD at the neutral position was quantified, and the percent changes in BMD when hip internal rotation was altered from -30° to 15°, when hip flexion was altered from 0° to 30°, and when hip abduction was altered from -15° to 30° were quantified. Analyses were automatically performed with a 1° increment in each direction using computer programming.

RESULTS

The alteration of hip angles in each direction affected BMD measurements, with the largest changes found for hip flexion (maximum change of 17.7% at 30° flexion) and the smallest changes found for hip rotation (maximum change of 2.2% at 15° internal rotation). The BMD measurements increased by 0.34% for each 1° of hip abduction, and the maximum change was 12.3% at 30° abduction.

CONCLUSION

This simulation study quantified the amount of BMD change induced by altering the hip position. Based on these results, we recommend that patients be positioned carefully when acquiring DXA images.

A Surgical Robotic System for Osteoporotic Hip Augmentation: System Development and Experimental Evaluation

Minimally-invasive Osteoporotic Hip Augmentation (OHA) by injecting bone cement is a potential treatment option to reduce the risk of hip fracture. This treatment can significantly benefit from computer-assisted planning and execution system to optimize the pattern of cement injection. We present a novel robotic system for the execution of OHA that consists of a 6-DOF robotic arm and integrated drilling and injection component. The minimally-invasive procedure is performed by registering the robot and preoperative images to the surgical scene using multiview image-based 2D/3D registration with no external fiducial attached to the body. The performance of the system is evaluated through experimental sawbone studies as well as cadaveric experiments with intact soft tissues. In the cadaver experiments, distance errors of 3.28mm and 2.64mm for entry and target points and orientation error of 2.30° are calculated. Moreover, the mean surface distance error of 2.13mm with translational error of 4.47mm is reported between injected and planned cement profiles. The experimental results demonstrate the first application of the proposed Robot-Assisted combined Drilling and Injection System (RADIS), incorporating biomechanical planning and intraoperative fiducial-less 2D/3D registration on human cadavers with intact soft tissues.

Development of a system to assess the two- and three-dimensional bone mineral density of the lumbar vertebrae from clinical quantitative CT images

This study developed a system to quantify the lumbar spine's bone mineral density (BMD) in two and three dimensions for osteoporosis screening using quantitative CT images. Measuring the two-dimensional BMD could reproduce the BMD measurement performed in dual-energy X-ray absorptiometry, and an accurate diagnosis of osteoporosis was possible.

PURPOSE

To date, the assessment of bone mineral density (BMD) using CT images has been made in three dimensions, leading to errors in detecting osteoporosis based on the two-dimensional assessments of BMD using dual-energy X-ray absorptiometry (DXA-BMD). Herein, we aimed to develop a system that measures two- and three-dimensional lumbar BMD from quantitative CT images and validated the accuracy of the system in diagnosing osteoporosis with regard to the DXA classification.

METHODS

Fifty-nine pairs of spinal CT and DXA images were analyzed. First, the three-dimensional BMD was measured at the axial slice of the L1 vertebra on CT images (L1-vBMD). Then, the L1-L4 vertebrae were segmented from the CT images to measure the three-dimensional BMD at the trabecular region of the L1-L4 vertebral bodies (CT-vBMD). Lastly, the segmented vertebrae were projected onto the coronal plane to measure the two-dimensional BMD (CT-aBMD). Each parameter was correlated with DXA-BMD, and the receiver operating characteristic (ROC) curve to diagnose osteoporosis was assessed.

RESULTS

The correlation coefficients of DXA-BMD with L1-vBMD, CT-vBMD, and CT-aBMD were 0.364, 0.456, and 0.911, respectively (all p < 0.01). In the ROC curve analysis to diagnose osteoporosis, the area under the curve for CT-aBMD (0.941) was significantly higher than those for L1-vBMD (0.582) and CT-vBMD (0.657) (both p < 0.01).

CONCLUSION

Compared with L1-vBMD and CT-vBMD, CT-aBMD could accurately predict DXA-BMD and detect patients with osteoporosis. Given that our method can quantify BMD in both two and three dimensions, it could be used to screen for osteoporosis from quantitative CT images.

Realistic C-arm to pCT registration for vertebral localization in spine surgery

Abstract

Spine surgeries are vulnerable to wrong-level surgeries and postoperative complications because of their complex structure. Unavailability of the 3D intraoperative imaging device, low-contrast intraoperative X-ray images, variable clinical and patient conditions, manual analyses, lack of skilled technicians, and human errors increase the chances of wrong-site or wrong-level surgeries. State of the art work refers 3D-2D image registration systems and other medical image processing techniques to address the complications associated with spine surgeries. Intensity-based 3D-2D image registration systems had been widely practiced across various clinical applications. However, these frameworks are limited to specific clinical conditions such as anatomy, dimension of image correspondence, and imaging modalities. Moreover, there are certain prerequisites for these frameworks to function in clinical application, such as dataset requirement, speed of computation, requirement of high-end system configuration, limited capture range, and multiple local maxima. A simple and effective registration framework was designed with a study objective of vertebral level identification and its pose estimation from intraoperative fluoroscopic images by combining intensity-based and iterative control point (ICP)–based 3D-2D registration. A hierarchical multi-stage registration framework was designed that comprises coarse and finer registration. The coarse registration was performed in two stages, i.e., intensity similarity-based spatial localization and source-to-detector localization based on the intervertebral distance correspondence between vertebral centroids in projected and intraoperative X-ray images. Finally, to speed up target localization in the intraoperative application, based on 3D-2D vertebral centroid correspondence, a rigid ICP-based finer registration was performed. The mean projection distance error (mPDE) measurement and visual similarity between projection image at finer registration point and intraoperative X-ray image and surgeons’ feedback were held accountable for the quality assurance of the designed registration framework. The average mPDE after peak signal to noise ratio (PSNR)–based coarse registration was 20.41mm. After the coarse registration in spatial region and source to detector direction, the average mPDE reduced to 12.18mm. On finer ICP-based registration, the mean mPDE was finally reduced to 0.36 mm. The approximate mean time required for the coarse registration, finer registration, and DRR image generation at the final registration point were 10 s, 15 s, and 1.5 min, respectively. The designed registration framework can act as a supporting tool for vertebral level localization and its pose estimation in an intraoperative environment. The framework was designed with the future perspective of intraoperative target localization and its pose estimation irrespective of the target anatomy.

Graphical abstract

2D/3D Multimode Medical Image Registration Based on Normalized Cross-Correlation

Image-guided surgery (IGS) can reduce the risk of tissue damage and improve the accuracy and targeting of lesions by increasing the surgery’s visual field. Three-dimensional (3D) medical images can provide spatial location information to determine the location of lesions and plan the operation process. For real-time tracking and adjusting the spatial position of surgical instruments, two-dimensional (2D) images provide real-time intraoperative information. In this experiment, 2D/3D medical image registration algorithm based on the gray level is studied, and the registration based on normalized cross-correlation is realized. The Gaussian Laplacian second-order differential operator is introduced as a new similarity measure to increase edge information and internal detail information to solve single information and small convergence regions of the normalized cross-correlation algorithm. The multiresolution strategy improves the registration accuracy and efficiency to solve the low efficiency of the normalized cross-correlation algorithm.

Development of an open-source measurement system to assess the areal bone mineral density of the proximal femur from clinical CT images

Commercial software is generally needed to measure the areal bone mineral density (aBMD) of the proximal femur from clinical computed tomography (CT) images. This study developed and verified an open-source reproducible system to quantify CT-aBMD to screen osteoporosis using clinical CT images.

PURPOSE

For existing CT images acquired for various reasons other than osteoporosis, it might be beneficial to estimate areal BMD as assessed by dual-energy X-ray absorptiometry (DXA-based BMD) to ascertain the bone status based on DXA. In this study, we aimed to (1) develop an open-source reproducible measurement system to quantify DXA-based BMD from CT images and (2) validate its accuracy.

METHODS

This study analyzed 75 pairs of hip CT and DXA images of women that were acquired for the preoperative assessment of total hip arthroplasty. From the CT images, the femur and a calibration phantom were automatically segmented using pre-trained codes/models available at https://github.com/keisuke-uemura . The proximal femoral region was isolated by manually selected landmarks and was projected onto the coronal plane to measure the areal density (CT-aHU). The calibration phantom was employed to convert the CT-aHU into CT-aBMD. Each parameter was correlated with DXA-based BMD, and the residual errors of CT images to estimate the T-scores in DXA were calculated using the standard error of estimate (SEE).

RESULTS

The correlation coefficients of DXA-based BMD with CT-aHU and CT-aBMD were 0.947 and 0.950, respectively (both p < 0.001). The SEE for quantifying the T-scores in DXA were 0.51 and 0.50 for CT-aHU and CT-aBMD, respectively.

CONCLUSION

With the method developed herein, CT permits estimation of the DXA-based BMD of the proximal femur within the standard DXA total hip region of interest with an SEE of 0.5 in T-scores. The radiation dose for CT acquisition needs consideration; therefore, our data do not provide a rationale for performing CT for screening osteoporosis. However, on CT images already acquired for clinical indications other than osteoporosis, researchers may use this open-source system to investigate osteoporosis status through the estimated DXA-based BMD of the proximal femur.

Three-Dimensional Subject-Specific Knee Shape Reconstruction with Asynchronous Fluoroscopy Images Using Statistical Shape Modeling

Background and objectives: Statistical shape modeling (SSM) based on computerized tomography (CT) datasets has enabled reasonably accurate reconstructions of subject-specific 3D bone morphology from one or two synchronous radiographs for clinical applications. Increasing the number of radiographic images may increase the reconstruction accuracy, but errors related to the temporal and spatial asynchronization of clinical alternating bi-plane fluoroscopy may also increase. The current study aimed to develop a new approach for subject-specific 3D knee shape reconstruction from multiple asynchronous fluoroscopy images from 2, 4, and 6 X-ray detector views using a CT-based SSM model; and to determine the optimum number of planar images for best accuracy via computer simulations and in vivo experiments.

Methods: A CT-based SSM model of the knee was established from 60 training models in a healthy young Chinese male population. A new two-phase optimization approach for 3D subject-specific model reconstruction from multiple asynchronous clinical fluoroscopy images using the SSM was developed, and its performance was evaluated via computer simulation and in vivo experiments using one, two and three image pairs from an alternating bi-plane fluoroscope.

Results: The computer simulation showed that subject-specific 3D shape reconstruction using three image pairs had the best accuracy with RMSE of 0.52 ± 0.09 and 0.63 ± 0.085 mm for the femur and tibia, respectively. The corresponding values for the in vivo study were 0.64 ± 0.084 and 0.69 ± 0.069 mm, respectively, which was significantly better than those using one image pair (0.81 ± 0.126 and 0.83 ± 0.108 mm). No significant differences existed between using two and three image pairs.

Conclusion: A new two-phase optimization approach was developed for SSM-based 3D subject-specific knee model reconstructions using more than one asynchronous fluoroscopy image pair from widely available alternating bi-plane fluoroscopy systems in clinical settings. A CT-based SSM model of the knee was also developed for a healthy young Chinese male population. The new approach was found to have high mode reconstruction accuracy, and those for both two and three image pairs were much better than for a single image pair. Thus, two image pairs may be used when considering computational costs and radiation dosage. The new approach will be useful for generating patient-specific knee models for clinical applications using multiple asynchronous images from alternating bi-plane fluoroscopy widely available in clinical settings. The current SSM model will serve as a basis for further inclusion of training models with a wider range of sizes and morphological features for broader applications.

Deformable registration of lateral cephalogram and cone-beam computed tomography image

PURPOSE

This study aimed to design and evaluate a novel method for the registration of 2D lateral cephalograms and 3D craniofacial cone-beam computed tomography (CBCT) images, providing patient-specific 3D structures from a 2D lateral cephalogram without additional radiation exposure.

METHODS

We developed a cross-modal deformable registration model based on a deep convolutional neural network. Our approach took advantage of a low-dimensional deformation field encoding and an iterative feedback scheme to infer coarse-to-fine volumetric deformations. In particular, we constructed a statistical subspace of deformation fields and parameterized the nonlinear mapping function from an image pair, consisting of the target 2D lateral cephalogram and the reference volumetric CBCT, to a latent encoding of the deformation field. Instead of the one-shot registration by the learned mapping function, a feedback scheme was introduced to progressively update the reference volumetric image and to infer coarse-to-fine deformations fields, accounting for the shape variations of anatomical structures. A total of 220 clinically obtained CBCTs were used to train and validate the proposed model, among which 120 CBCTs were used to generate a training dataset with 24k paired synthetic lateral cephalograms and CBCTs. The proposed approach was evaluated on the deformable 2D-3D registration of clinically obtained lateral cephalograms and CBCTs from growing and adult orthodontic patients.

RESULTS

Strong structural consistencies were observed between the deformed CBCT and the target lateral cephalogram in all criteria. The proposed method achieved state-of-the-art performances with the mean contour deviation of 0.41 ± 0.12 mm on the anterior cranial base, 0.48 ± 0.17 mm on the mandible, and 0.35 ± 0.08 mm on the maxilla, respectively. The mean surface mesh ranged from 0.78 to 0.97 mm on various craniofacial structures, and the LREs ranged from 0.83 to 1.24 mm on the growing datasets regarding 14 landmarks. The proposed iterative feedback scheme handled the structural details and improved the registration. The resultant deformed volumetric image was consistent with the target lateral cephalogram in both 2D projective planes and 3D volumetric space regarding the multicategory craniofacial structures.

CONCLUSIONS

The results suggest that the deep learning-based 2D-3D registration model enables the deformable alignment of 2D lateral cephalograms and CBCTs and estimates patient-specific 3D craniofacial structures.

2D-3D reconstruction of distal forearm bone from actual X-ray images of the wrist using convolutional neural networks

The purpose of the study was to develop a deep learning network for estimating and constructing highly accurate 3D bone models directly from actual X-ray images and to verify its accuracy. The data used were 173 computed tomography (CT) images and 105 actual X-ray images of a healthy wrist joint. To compensate for the small size of the dataset, digitally reconstructed radiography (DRR) images generated from CT were used as training data instead of actual X-ray images. The DRR-like images were generated from actual X-ray images in the test and adapted to the network, and high-accuracy estimation of a 3D bone model from a small data set was possible. The 3D shape of the radius and ulna were estimated from actual X-ray images with accuracies of 1.05 ± 0.36 and 1.45 ± 0.41 mm, respectively.

Bayesian Fully Convolutional Networks for Brain Image Registration

The purpose of medical image registration is to find geometric transformations that align two medical images so that the corresponding voxels on two images are spatially consistent. Nonrigid medical image registration is a key step in medical image processing, such as image comparison, data fusion, target recognition, and pathological change analysis. Existing registration methods only consider registration accuracy but largely neglect the uncertainty of registration results. In this work, a method based on the Bayesian fully convolutional neural network is proposed for nonrigid medical image registration. The proposed method can generate a geometric uncertainty map to calculate the uncertainty of registration results. This uncertainty can be interpreted as a confidence interval, which is essential for judging whether the source data are abnormal. Moreover, the proposed method introduces group normalization, which is conducive to the network convergence of the Bayesian neural network. Some representative learning-based image registration methods are compared with the proposed method on different image datasets. Experimental results show that the registration accuracy of the proposed method is better than that of the methods, and its antifolding performance is comparable to that of fast image registration and VoxelMorph. Furthermore, the proposed method can evaluate the uncertainty of registration results.

A level-wise spine registration framework to account for large pose changes

PURPOSES

Accurate and efficient spine registration is crucial to success of spine image guidance. However, changes in spine pose cause intervertebral motion that can lead to significant registration errors. In this study, we develop a geometrical rectification technique via nonlinear principal component analysis (NLPCA) to achieve level-wise vertebral registration that is robust to large changes in spine pose.

METHODS

We used explanted porcine spines and live pigs to develop and test our technique. Each sample was scanned with preoperative CT (pCT) in an initial pose and rescanned with intraoperative stereovision (iSV) in a different surgical posture. Patient registration rectified arbitrary spinal postures in pCT and iSV into a common, neutral pose through a parameterized moving-frame approach. Topologically encoded depth projection 2D images were then generated to establish invertible point-to-pixel correspondences. Level-wise point correspondences between pCT and iSV vertebral surfaces were generated via 2D image registration. Finally, closed-form vertebral level-wise rigid registration was obtained by directly mapping 3D surface point pairs. Implanted mini-screws were used as fiducial markers to measure registration accuracy.

RESULTS

In seven explanted porcine spines and two live animal surgeries (maximum in-spine pose change of 87.5 mm and 32.7 degrees averaged from all spines), average target registration errors (TRE) of 1.70  ±  0.15 mm and 1.85 ± 0.16 mm were achieved, respectively. The automated spine rectification took 3-5 min, followed by an additional 30 secs for depth image projection and level-wise registration.

CONCLUSIONS

Accuracy and efficiency of the proposed level-wise spine registration support its application in human open spine surgeries. The registration framework, itself, may also be applicable to other intraoperative imaging modalities such as ultrasound and MRI, which may expand utility of the approach in spine registration in general.

Extended Capture Range of Rigid 2D/3D Registration by Estimating Riemannian Pose Gradients

Traditional intensity-based 2D/3D registration requires near-perfect initialization in order for image similarity metrics to yield meaningful updates of X-ray pose and reduce the likelihood of getting trapped in a local minimum. The conventional approaches strongly depend on image appearance rather than content, and therefore, fail in revealing large pose offsets that substantially alter the appearance of the same structure. We complement traditional similarity metrics with a convolutional neural network-based (CNN-based) registration solution that captures large-range pose relations by extracting both local and contextual information, yielding meaningful X-ray pose updates without the need for accurate initialization. To register a 2D X-ray image and a 3D CT scan, our CNN accepts a target X-ray image and a digitally reconstructed radiograph at the current pose estimate as input and iteratively outputs pose updates in the direction of the pose gradient on the Riemannian Manifold. Our approach integrates seamlessly with conventional image-based registration frameworks, where long-range relations are captured primarily by our CNN-based method while short-range offsets are recovered accurately with an image similarity-based method. On both synthetic and real X-ray images of the human pelvis, we demonstrate that the proposed method can successfully recover large rotational and translational offsets, irrespective of initialization.

Fiducial-Free 2D/3D Registration for Robot-Assisted Femoroplasty

Femoroplasty is a proposed alternative therapeutic method for preventing osteoporotic hip fractures in the elderly. Previously developed navigation system for femoroplasty required the attachment of an external X-ray fiducial to the femur. We propose a fiducial-free 2D/3D registration pipeline using fluoroscopic images for robot-assisted femoroplasty. Intraoperative fluoroscopic images are taken from multiple views to perform registration of the femur and drilling/injection device. The proposed method was tested through comprehensive simulation and cadaveric studies. Performance was evaluated on the registration error of the femur and the drilling/injection device. In simulations, the proposed approach achieved a mean accuracy of 1.26±0.74 mm for the relative planned injection entry point; 0.63±0.21° and 0.17±0.19° for the femur injection path direction and device guide direction, respectively. In the cadaver studies, a mean error of 2.64 ± 1.10 mm was achieved between the planned entry point and the device guide tip. The biomechanical analysis showed that even with a 4 mm translational deviation from the optimal injection path, the yield load prior to fracture increased by 40.7%. This result suggests that the fiducial-less 2D/3D registration is sufficiently accurate to guide robot assisted femoroplasty.

Reproducibility of pelvic sagittal inclination while acquiring radiographs in supine and standing postures

PURPOSE

Pelvic position on the sagittal plane is usually evaluated with the pelvic sagittal inclination (PSI) angle from a single radiograph. However, the reproducibility of pelvic positioning has not been investigated, and thus, the validity of measuring the PSI from a single film/time point is not understood. Herein, the reproducibility of a patient's pelvic positions in supine and standing postures was analyzed.

METHODS

A total of 34 patients who underwent either a pelvic osteotomy or total hip arthroplasty were enrolled in this study. Preoperative radiographs in both supine and standing postures were acquired twice (first X-ray and second X-ray) within 6 months; preoperative computed tomography (CT) images of the full pelvis were also acquired in a supine posture (preop-CT). To eliminate measurement variability, each PSI was automatically measured from radiographs and CT images through the use of CT segmentation and landmark localization followed by intensity-based 2D-3D registration. The absolute difference of PSI among each image was calculated and the intra-class correlation coefficient (ICC) in each posture was also analyzed.

RESULTS

The median absolute differences of PSI in the supine posture were 1.3° between the first and second X-rays, 1.2° between the first X-ray and preop-CT, and 1.3° between the second X-ray and preop-CT. The median absolute difference of PSI in the standing posture was 1.5°. The ICC was 0.965 (95% CI: 0.939-0.981) in supine and 0.977 (95% CI: 0.954-0.988) during standing.

CONCLUSIONS

Pelvic positions in supine and standing postures are reproducible. Thus, measuring the PSI from a single radiograph is reliable.

Fully automatic estimation of pelvic sagittal inclination from anterior-posterior radiography image using deep learning framework

BACKGROUND AND OBJECTIVE

Malposition of the acetabular component causes dislocation and prosthetic impingement after Total Hip Arthroplasty (THA), which significantly affects the postoperative quality of life and implant longevity. The position of the acetabular component is determined by the Pelvic Sagittal Inclination (PSI), which not only varies among different people but also changes in different positions. It is important to recognize individual dynamic changes of the PSI for patient-specific planning of the THA. Previously PSI was estimated by registering the CT and radiography images. In this study, we introduce a new method for accurate estimation of functional PSI without requiring CT image in order to lower radiation exposure of the patient which opens up the possibility of increasing its application in a larger number of hospitals where CT is not acquired as a routine protocol.

METHODS

The proposed method consists of two main steps: First, the Mask R-CNN framework was employed to segment the pelvic shape from the background in the radiography images. Then, following the segmentation network, another convolutional network regressed the PSI angle. We employed a transfer learning paradigm where the network weights were initialized by non-medical images followed by fine-tuning using radiography images. Furthermore, in the training process, augmented data was generated to improve the performance of both networks. We analyzed the role of segmentation network in our system and investigated the Mask R-CNN performance in comparison with the U-Net, which is commonly used for the medical image segmentation.

RESULTS

In this study, the Mask R-CNN utilizing multi-task learning, transfer learning, and data augmentation techniques achieve 0.960 ± 0.008 DICE coefficient, which significantly outperforms the U-Net. The cascaded system is capable of estimating the PSI with 4.04° ± 3.39° error for the radiography images.

CONCLUSIONS

The proposed framework suggests a fully automatic and robust estimation of the PSI using only an anterior-posterior radiography image.

Pose Estimation of Periacetabular Osteotomy Fragments With Intraoperative X-Ray Navigation

OBJECTIVE

State-of-the-art navigation systems for pelvic osteotomies use optical systems with external fiducials. In this paper, we propose the use of X-ray navigation for pose estimation of periacetabular fragments without fiducials.

METHODS

A two-dimensional/three-dimensional (2-D/3-D) registration pipeline was developed to recover fragment pose. This pipeline was tested through an extensive simulation study and six cadaveric surgeries. Using osteotomy boundaries in the fluoroscopic images, the preoperative plan was refined to more accurately match the intraoperative shape.

RESULTS

In simulation, average fragment pose errors were 1.3 ° /1.7 mm when the planned fragment matched the intraoperative fragment, 2.2 ° /2.1 mm when the plan was not updated to match the true shape, and 1.9 ° /2.0 mm when the fragment shape was intraoperatively estimated. In cadaver experiments, the average pose errors were 2.2  ° /2.2 mm, 3.8 ° /2.5 mm, and 3.5  ° /2.2 mm when registering with the actual fragment shape, a preoperative plan, and an intraoperatively refined plan, respectively. Average errors of the lateral center edge angle were less than 2 ° for all fragment shapes in simulation and cadaver experiments.

CONCLUSION

The proposed pipeline is capable of accurately reporting femoral head coverage within a range clinically identified for long-term joint survivability.

SIGNIFICANCE

Human interpretation of fragment pose is challenging and usually restricted to rotation about a single anatomical axis. The proposed pipeline provides an intraoperative estimate of rigid pose with respect to all anatomical axes, is compatible with minimally invasive incisions, and has no dependence on external fiducials.

Significance of preoperative planning for prophylactic augmentation of osteoporotic hip: A computational modeling study

A potential effective treatment for prevention of osteoporotic hip fractures is augmentation of the mechanical properties of the femur by injecting it with bone cement. This therapy, however, is only in research stage and can benefit substantially from computational simulations to optimize the pattern of cement injection. Some studies have considered a patient-specific planning paradigm for Osteoporotic Hip Augmentation (OHA). Despite their biomechanical advantages, customized plans require advanced surgical systems for implementation. Other studies, therefore, have suggested a more generalized injection strategy. The goal of this study is to investigate as to whether the additional computational overhead of the patient-specific planning can significantly improve the bone strength as compared to the generalized injection strategies attempted in the literature. For this purpose, numerical models were developed from high resolution CT images (n = 4). Through finite element analysis and hydrodynamic simulations, we compared the biomechanical efficiency of the customized cement-based augmentation along with three generalized injection strategies developed previously. Two series of simulations were studied, one with homogeneous and one with inhomogeneous material properties for the osteoporotic bone. The customized cement-based augmentation inhomogeneous models showed that injection of only 10 ml of bone cement can significantly increase the yield load (79.6%, P < 0.01) and yield energy (199%, P < 0.01) of an osteoporotic femur. This increase is significantly higher than those of the generalized injections proposed previously (23.8% on average). Our findings suggest that OHA can significantly benefit from a patient-specific plan that determines the pattern and volume of the injected cement.

A BIOMECHANICAL AND THERMAL ANALYSIS FOR BONE AUGMENTATION OF THE PROXIMAL FEMUR

In this study, we aim to create and validate a Finite Element (FE) model to estimate the bone temperature after cement injection and compare the simulation temperature results with experimental data in three key locations of the proximal femur. Simulation results suggest that the maximum temperature-rise measured at the bone surface is 10°C which occurs about 12 minutes after the injection. Temperature profiles measured during the experiment showed an agreement with those of the simulation with an average error of 1.73°C Although additional experiments are required to further validate the model, results of this study suggest that this model is a promising tool for bone augmentation planning to lower the risk of thermal necrosis.

Feasibility of A-mode ultrasound based intraoperative registration in computer-aided orthopedic surgery: A simulation and experimental study

PURPOSE

A fast and accurate intraoperative registration method is important for Computer-Aided Orthopedic Surgery (CAOS). A-mode ultrasound (US) is able to acquire bone surface data in a non-invasive manner. To utilize A-mode US in CAOS, a suitable registration algorithm is necessary with a small number of registration points and the presence of measurement errors. Therefore, we investigated the effects of (1) the number of registration points and (2) the Ultrasound Point Localization Error (UPLE) on the overall registration accuracy.

METHODS

We proposed a new registration method (ICP-PS), including the Iterative Closest Points (ICP) algorithm and a Perturbation Search algorithm. This method enables to avoid getting stuck in the local minimum of ICP iterations and to find the adjacent global minimum. This registration method was subsequently validated in a numerical simulation and a cadaveric experiment using a 3D-tracked A-mode US system.

RESULTS

The results showed that ICP-PS outperformed the standard ICP algorithm. The registration accuracy improved with the addition of ultrasound registration points. In the numerical simulation, for 25 sample points with zero UPLE, the averaged registration error of ICP-PS reached 0.25 mm, while 1.71 mm for ICP, decreasing by 85.38%. In the cadaver experiment, using 25 registration points, ICP-PS achieved an RMSE of 2.81 mm relative to 5.84 mm for the ICP, decreasing by 51.88%.

CONCLUSIONS

The simulation approach provided a well-defined framework for estimating the necessary number of ultrasound registration points and acceptable level of UPLE for a given required level of accuracy for intraoperative registration in CAOS. ICP-PS method is suitable for A-mode US based intraoperative registration. This study would facilitate the application of A-mode US probe in registering the point cloud to a known shape model, which also has the potential for accurately estimating bone position and orientation for skeletal motion tracking and surgical navigation.

Influence of the quality of intraoperative fluoroscopic images on the spatial positioning accuracy of a CAOS system

BACKGROUND

Spatial positioning accuracy is a key issue in a computer-assisted orthopaedic surgery (CAOS) system. Since intraoperative fluoroscopic images are one of the most important input data to the CAOS system, the quality of these images should have a significant influence on the accuracy of the CAOS system. But the regularities and mechanism of the influence of the quality of intraoperative images on the accuracy of a CAOS system have yet to be studied.

METHODS

Two typical spatial positioning methods - a C-arm calibration-based method and a bi-planar positioning method - are used to study the influence of different image quality parameters, such as resolution, distortion, contrast and signal-to-noise ratio, on positioning accuracy. The error propagation rules of image error in different spatial positioning methods are analyzed by the Monte Carlo method.

RESULTS

Correlation analysis showed that resolution and distortion had a significant influence on spatial positioning accuracy. In addition the C-arm calibration-based method was more sensitive to image distortion, while the bi-planar positioning method was more susceptible to image resolution. The image contrast and signal-to-noise ratio have no significant influence on the spatial positioning accuracy. The result of Monte Carlo analysis proved that generally the bi-planar positioning method was more sensitive to image quality than the C-arm calibration-based method.

CONCLUSIONS

The quality of intraoperative fluoroscopic images is a key issue in the spatial positioning accuracy of a CAOS system. Although the 2 typical positioning methods have very similar mathematical principles, they showed different sensitivities to different image quality parameters. The result of this research may help to create a realistic standard for intraoperative fluoroscopic images for CAOS systems.

Can Anatomic Measurements of Stem Anteversion Angle Be Considered as the Functional Anteversion Angle?

BACKGROUND

Stem anteversion angle is important in the combined anteversion theory to avoid implant impingement after total hip arthroplasty (THA). However, anatomic measurements of stem anteversion angle may not represent functional anteversion of the femur if the femur undergoes axial rotation. Herein, the femoral rotational angle (FRA) was measured in supine and standing positions before and after THA to evaluate the difference between anatomic and functional measurements.

METHODS

A total of 191 hips (174 patients) treated with THA for osteoarthritis were analyzed in this retrospective, case-controlled study. The FRA was measured as the angle between the posterior condylar line and the line through the bilateral anterior superior iliac spines (positive for external rotation) and was measured preoperatively and postoperatively in supine and standing positions with computed tomography segmentation and landmark localization of the pelvis and the femur followed by intensity-based 2D-3D registration. The number of cases in which the absolute FRA remained <15° in both positions was also calculated.

RESULTS

The average ± standard deviation preoperative FRA was 0.3° ± 8.3° in the supine position and -4.5° ± 8.8° during standing; the postoperative FRA was -3.8° ± 9.0° in supine and -14.3° ± 8.3° during standing. There were 134 cases (70%) in which the preoperative absolute FRA remained <15° in both positions while only 85 hips (45%) remained <15°, postoperatively.

CONCLUSION

Substantial variability was seen in the FRA, especially during the postoperative period. These results suggest that the anatomic stem anteversion angle may not represent the functional anteversion of the femur.

A Novel Ultrasound-Based Lower Extremity Motion Tracking System

Tracking joint motion of the lower extremity is important for human motion analysis. In this study, we present a novel ultrasound-based motion tracking system for measuring three-dimensional (3D) position and orientation of the femur and tibia in 3D space and quantifying tibiofemoral kinematics under dynamic conditions. As ultrasound is capable of detecting underlying bone surface noninvasively through multiple layers of soft tissues, an integration of multiple A-mode ultrasound transducers with a conventional motion tracking system provides a new approach to track the motion of bone segments during dynamic conditions. To demonstrate the technical and clinical feasibilities of this concept, an in vivo experiment was conducted. For this purpose the kinematics of healthy individuals were determined in treadmill walking conditions and stair descending tasks. The results clearly demonstrated the potential of tracking skeletal motion of the lower extremity and measuring six-degrees-of-freedom (6-DOF) tibiofemoral kinematics and related kinematic alterations caused by a variety of gait parameters. It was concluded that this prototyping system has great potential to measure human kinematics in an ambulant, non-radiative, and noninvasive manner.

Image-Guided Surgical Robotic System for Percutaneous Reduction of Joint Fractures

Complex joint fractures often require an open surgical procedure, which is associated with extensive soft tissue damages and longer hospitalization and rehabilitation time. Percutaneous techniques can potentially mitigate these risks but their application to joint fractures is limited by the current sub-optimal 2D intra-operative imaging (fluoroscopy) and by the high forces involved in the fragment manipulation (due to the presence of soft tissue, e.g., muscles) which might result in fracture malreduction. Integration of robotic assistance and 3D image guidance can potentially overcome these issues. The authors propose an image-guided surgical robotic system for the percutaneous treatment of knee joint fractures, i.e., the robot-assisted fracture surgery (RAFS) system. It allows simultaneous manipulation of two bone fragments, safer robot-bone fixation system, and a traction performing robotic manipulator. This system has led to a novel clinical workflow and has been tested both in laboratory and in clinically relevant cadaveric trials. The RAFS system was tested on 9 cadaver specimens and was able to reduce 7 out of 9 distal femur fractures (T- and Y-shape 33-C1) with acceptable accuracy (≈1 mm, ≈5°), demonstrating its applicability to fix knee joint fractures. This study paved the way to develop novel technologies for percutaneous treatment of complex fractures including hip, ankle, and shoulder, thus representing a step toward minimally-invasive fracture surgeries.

Change in Pelvic Sagittal Inclination From Supine to Standing Position Before Hip Arthroplasty

BACKGROUND

Cup anteversion and inclination are important for avoiding implant impingement and dislocation in total hip arthroplasty. However, functional cup anteversion and cup inclination also change as the pelvic sagittal inclination (PSI) changes. Therefore, PSI in both supine and standing positions was measured in a large cohort in this study.

METHODS

A total of 422 patients (median age, 61; range, 15-87) who underwent total hip arthroplasty were the subjects of this study. There were 83 patients with primary osteoarthritis (OA), 274 patients with developmental dysplasia-derived secondary OA, 48 patients with osteonecrosis, and 17 patients with rapidly destructive coxopathy (RDC). Preoperative PSI in supine and standing positions was measured by automated computed topography segmentation and landmark localization of the pelvis followed by intensity-based 2D-3D registration, and the number of cases in which PSI changed more than 10° posteriorly was calculated. Hip disease, sex, and age were analyzed if they were related to a PSI change of more than 10°.

RESULTS

The median PSI was 5.1° (interquartile range, 0.4°-9.4°) in supine and -1.3° (interquartile range, -6.5° to 4.2°) in standing position. There were 79 cases (19%) in which the PSI changed more than 10° posteriorly from supine to standing. Elder age and patients with primary OA and RDC were revealed to be the related factors.

CONCLUSION

PSI changed more than 10° posteriorly from supine to standing in 19% of cases. Age and diagnosis of primary OA and RDC were the significant factors for the posterior rotation.

Pose-aware C-arm for automatic re-initialization of interventional 2D/3D image registration

PURPOSE

In minimally invasive interventions assisted by C-arm imaging, there is a demand to fuse the intra-interventional 2D C-arm image with pre-interventional 3D patient data to enable surgical guidance. The commonly used intensity-based 2D/3D registration has a limited capture range and is sensitive to initialization. We propose to utilize an opto/X-ray C-arm system which allows to maintain the registration during intervention by automating the re-initialization for the 2D/3D image registration. Consequently, the surgical workflow is not disrupted and the interaction time for manual initialization is eliminated.

METHODS

We utilize two distinct vision-based tracking techniques to estimate the relative poses between different C-arm arrangements: (1) global tracking using fused depth information and (2) RGBD SLAM system for surgical scene tracking. A highly accurate multi-view calibration between RGBD and C-arm imaging devices is achieved using a custom-made multimodal calibration target.

RESULTS

Several in vitro studies are conducted on pelvic-femur phantom that is encased in gelatin and covered with drapes to simulate a clinically realistic scenario. The mean target registration errors (mTRE) for re-initialization using depth-only and RGB [Formula: see text] depth are 13.23 mm and 11.81 mm, respectively. 2D/3D registration yielded 75% success rate using this automatic re-initialization, compared to a random initialization which yielded only 23% successful registration.

CONCLUSION

The pose-aware C-arm contributes to the 2D/3D registration process by globally re-initializing the relationship of C-arm image and pre-interventional CT data. This system performs inside-out tracking, is self-contained, and does not require any external tracking devices.

Motion compensation in extremity cone-beam CT using a penalized image sharpness criterion

Cone-beam CT (CBCT) for musculoskeletal imaging would benefit from a method to reduce the effects of involuntary patient motion. In particular, the continuing improvement in spatial resolution of CBCT may enable tasks such as quantitative assessment of bone microarchitecture (0.1 mm-0.2 mm detail size), where even subtle, sub-mm motion blur might be detrimental. We propose a purely image based motion compensation method that requires no fiducials, tracking hardware or prior images. A statistical optimization algorithm (CMA-ES) is used to estimate a motion trajectory that optimizes an objective function consisting of an image sharpness criterion augmented by a regularization term that encourages smooth motion trajectories. The objective function is evaluated using a volume of interest (VOI, e.g. a single bone and surrounding area) where the motion can be assumed to be rigid. More complex motions can be addressed by using multiple VOIs. Gradient variance was found to be a suitable sharpness metric for this application. The performance of the compensation algorithm was evaluated in simulated and experimental CBCT data, and in a clinical dataset. Motion-induced artifacts and blurring were significantly reduced across a broad range of motion amplitudes, from 0.5 mm to 10 mm. Structure similarity index (SSIM) against a static volume was used in the simulation studies to quantify the performance of the motion compensation. In studies with translational motion, the SSIM improved from 0.86 before compensation to 0.97 after compensation for 0.5 mm motion, from 0.8 to 0.94 for 2 mm motion and from 0.52 to 0.87 for 10 mm motion (~70% increase). Similar reduction of artifacts was observed in a benchtop experiment with controlled translational motion of an anthropomorphic hand phantom, where SSIM (against a reconstruction of a static phantom) improved from 0.3 to 0.8 for 10 mm motion. Application to a clinical dataset of a lower extremity showed dramatic reduction of streaks and improvement in delineation of tissue boundaries and trabecular structures throughout the whole volume. The proposed method will support new applications of extremity CBCT in areas where patient motion may not be sufficiently managed by immobilization, such as imaging under load and quantitative assessment of subchondral bone architecture.

Intra-operative fiducial-based CT/fluoroscope image registration framework for image-guided robot-assisted joint fracture surgery

Purpose

Joint fractures must be accurately reduced minimising soft tissue damages to avoid negative surgical outcomes. To this regard, we have developed the RAFS surgical system, which allows the percutaneous reduction of intra-articular fractures and provides intra-operative real-time 3D image guidance to the surgeon. Earlier experiments showed the effectiveness of the RAFS system on phantoms, but also key issues which precluded its use in a clinical application. This work proposes a redesign of the RAFS’s navigation system overcoming the earlier version’s issues, aiming to move the RAFS system into a surgical environment.

Methods

The navigation system is improved through an image registration framework allowing the intra-operative registration between pre-operative CT images and intra-operative fluoroscopic images of a fractured bone using a custom-made fiducial marker. The objective of the registration is to estimate the relative pose between a bone fragment and an orthopaedic manipulation pin inserted into it intra-operatively. The actual pose of the bone fragment can be updated in real time using an optical tracker, enabling the image guidance.

Results

Experiments on phantom and cadavers demonstrated the accuracy and reliability of the registration framework, showing a reduction accuracy (sTRE) of about \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.88~\pm 0.2\,\hbox {mm}$$\end{document}0.88±0.2mm (phantom) and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.15\pm 0.8\,\hbox {mm}$$\end{document}1.15±0.8mm (cadavers). Four distal femur fractures were successfully reduced in cadaveric specimens using the improved navigation system and the RAFS system following the new clinical workflow (reduction error \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.2\pm 0.3\,\hbox {mm}$$\end{document}1.2±0.3mm, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2\pm 1{^{\circ }})$$\end{document}2±1∘).

Conclusion

Experiments showed the feasibility of the image registration framework. It was successfully integrated into the navigation system, allowing the use of the RAFS system in a realistic surgical application.

3D Catheter Shape Reconstruction Using Electromagnetic and Image Sensors

In current practice, fluoroscopy remains the gold standard for guiding surgeons during endovascular catheterization. The poor visibility of anatomical structures and the absence of depth information make accurate catheter localization and manipulation a difficult task. Overexposure to radiation and use of risk-prone contrast agent also compromise surgeons’ and patients’ health. Alternative approaches using embedded electromagnetic (EM) sensors have been developed to overcome the limitations of fluoroscopy-based interventions. As only a finite number of sensors can be integrated within a catheter, methods that rely on such sensors require the use of interpolation schemes to recover the catheter shape. Since EM sensors are sensitive to external interferences, the outcome is not robust. This paper introduces a probabilistic framework that improves the catheter localization and reduces the dependency on fluoroscopy and contrast agents. Within this framework, the dense 2D information extracted from fluoroscopic images is combined with the discrete pose information of EM sensors to provide a reliable reconstruction of the full three-dimensional catheter shape. Validation in a physics-based simulation environment and in a real-world experimental setup provides promising results and indicates that the proposed framework allows reconstructing the 3D catheter shape with a median root-mean-square error of 3.7[Formula: see text]mm with an interquartile range of 0.3[Formula: see text]mm.

Sparse 3D Radon Space Rigid Registration of CT Scans: Method and Validation Study

We present a new method for rigid registration of CT datasets in 3D Radon space based on sparse sampling of scanning projections. The inputs are the two 3D Radon transforms of the CT scans, one densely sampled and the other sparsely sampled (limited number of scan angles/ranges). The output is the rigid transformation that best matches them. The method first finds the best matching between each projection direction vector in the sparse transform and the corresponding direction vector in the dense transform. It then solves a system of linear equations derived from the direction vector pairs (parallel-beam projections) or finds a solution by non-linear optimization (fan-beam and cone-beam projections). Experimental studies show that our method for 3D parallel beam registration outperforms image space registration in terms of convergence range with significantly reduced X-ray dose compared to a full conventional CT scan.