Fast generation of virtual X-ray images for reconstruction of 3D anatomy

Comparison of global and local optimization methods for intensity-based 2D-3D registration

Intensity-based 2D-3D registration methods are commonly used in musculoskeletal research and image-guided therapy to align 2D X-ray images with 3D CT scans. However, their success rate (SR) is limited by local optimization methods, which often cause the optimization of the underlying cost function to get stuck at a local minimum, resulting in false alignments. Global optimization methods aim to mitigate this problem, but despite their increasing popularity, the existing literature lacks consensus on which one is the most appropriate. In this work, we compare 11 global and 4 local optimization methods on thousands of typical registration examples of single- and dual-plane fluoroscopy, including three datasets of varying complexity. In addition, we evaluate the differences between global and local methods, determine the best overall method, and validate its suitability for real clinical data. The results demonstrate that global methods that require a large number of function evaluations (NFEV) are generally the most robust. Furthermore, hyperparameter tuning can significantly improve their performance and is generalizable across datasets. Evolutionary strategy (ES) is the best-performing optimization method in our study, achieving a mean SR of ∼95% for all test models, ∼77% for the knee bones, and ∼95-100% for cerebral angiograms when using dual-plane registration setup. Nevertheless, in cases where good initialization is available, local methods are still preferable due to their reduced NFEV. The use of global optimization improves the overall robustness and ease-of-use of 2D-3D registration, potentially accelerating its adaptation in routine medical practice and biomedical research.

2D-to-3DMaxiDeform: A computer-aided approach for 3D construction of maxillary sinus from PA and lateral X-ray images

3D volume construction of the maxillary sinus is important for understanding the 3D surface morphology of the maxillary sinuses and detecting changes or obstruction in sinuses. It is important to detect the pathological conditions affecting the sinuses and to determine the treatment outcomes. The cases of sinusitis and various other pathoses in maxillary sinuses are getting comparatively higher than in other sinuses. Therefore, analysis of maxillary sinus structure has some clinical importance. 3D volume construction from Computed Tomography (CT) can help in the volumetric study of the maxillary sinus. Nonetheless, CT imaging is expensive and has a higher radiation dose than X-ray imaging. Thus, the 3D construction from X-ray images is a challenging problem. This paper proposed a 3D construction method for maxillary sinus from lateral and Posterior-anterior (PA) 2D X-ray images. A novel 3D maxillary sinus construction model (2D-to-3D-MaxiDeform) is proposed which evolves the shape and size of the maxillary sinus providing the required volume, and other linear measurements in mesh format. The 2D-to-3DMaxiDeform uses a 3D point cloud template of the maxillary sinus and applies for registration on the template model. The deformation moves the 3D template points to the required positions of the X-ray image contours. The registration of the template model before deformation into the required shape makes sure the size, position, and angle of the template model are mapped with the input X-ray contours. The evaluation of the 2D-to-3DMaxiDeform achieves an average accuracy of 0.83, mDSC (mean Dice similarity coefficient) of 0.80, mIoU (mean Intersection over Union) of 0.67, recall of 0.90, precision of 0.74, specificity of 0.78 and RMSE of 2.3 mm. The proposed method is the first and novel approach for maxillary sinus 3D construction from X-ray images. The result shows that the proposed method could be a valuable tool for generating 3D models of the maxillary sinus to be used in a clinical setting.

Scan-Free and Fully Automatic Tracking of Native Knee Anatomy from Dynamic Stereo-Radiography with Statistical Shape and Intensity Models

Kinematic tracking of native anatomy from stereo-radiography provides a quantitative basis for evaluating human movement. Conventional tracking procedures require significant manual effort and call for acquisition and annotation of subject-specific volumetric medical images. The current work introduces a framework for fully automatic tracking of native knee anatomy from dynamic stereo-radiography which forgoes reliance on volumetric scans. The method consists of three computational steps. First, captured radiographs are annotated with segmentation maps and anatomic landmarks using a convolutional neural network. Next, a non-convex polynomial optimization problem formulated from annotated landmarks is solved to acquire preliminary anatomy and pose estimates. Finally, a global optimization routine is performed for concurrent refinement of anatomy and pose. An objective function is maximized which quantifies similarities between masked radiographs and digitally reconstructed radiographs produced from statistical shape and intensity models. The proposed framework was evaluated against manually tracked trials comprising dynamic activities, and additional frames capturing a static knee phantom. Experiments revealed anatomic surface errors routinely below 1.0 mm in both evaluation cohorts. Median absolute errors of individual bone pose estimates were below 1.0∘ or mm for 15 out of 18 degrees of freedom in both evaluation cohorts. Results indicate that accurate pose estimation of native anatomy from stereo-radiography may be performed with significantly reduced manual effort, and without reliance on volumetric scans.

Fast digitally reconstructed radiograph generation using particle-based statistical shape and intensity model

Purpose

Statistical shape and intensity models (SSIMs) and digitally reconstructed radiographs (DRRs) were introduced for non-rigid 2D-3D registration and skeletal geometry/density reconstruction studies. The computation of DRRs takes most of the time during registration or reconstruction. The goal of this study is to propose a particle-based method for composing an SSIM and a DRR image generation scheme and analyze the quality of the images compared with previous DRR generation methods.

Approach

Particle-based SSIMs consist of densely scattered particles on the surface and inside of an object, with each particle having an intensity value. Generating the DRR resembles ray tracing, which counts the particles that are binned with each ray and calculates the radiation attenuation. The distance between adjacent particles was considered to be the radiologic path during attenuation integration, and the mean linear attenuation coefficient of the two particles was multiplied. The proposed method was compared with the DRR of CT projection. The mean squared error and peak signal-to-noise ratio (PSNR) were calculated between the DRR images from the proposed method and those of existing methods of projecting tetrahedral-based SSIMs or computed tomography (CT) images to verify the accuracy of the proposed scheme.

Results

The suggested method was about 600 times faster than the tetrahedral-based SSIM without using the hardware acceleration technique. The PSNR was 37.59 dB, and the root mean squared error of the normalized pixel intensities was 0.0136.

Conclusions

The proposed SSIM and DRR generation procedure showed high temporal performance while maintaining image quality, and particle-based SSIM is a feasible form for representing a 3D volume and generating the DRR images.

3D biplanar reconstruction of lower limbs using nonlinear statistical models

Three-dimensional (3D) reconstruction of lower limbs is of great interest in surgical planning, computer assisted surgery, and for biomechanical applications. The use of 3D imaging modalities such as computed tomography (CT) scan and magnetic resonance imaging (MRI) has limitations such as high radiation and expense. Therefore, three-dimensional reconstruction methods from biplanar X-ray images represent an attractive alternative. In this paper, we present a new unsupervised 3D reconstruction method for the patella, talus, and pelvis using calibrated biplanar (45- and 135-degree oblique) radiographic images and a prior information on the geometric/anatomical structure of these complex bones. A multidimensional scaling (MDS)-based nonlinear dimensionality reduction algorithm is applied to exploit this prior geometric/anatomical information. It represents relevant deformations existing in the training set. Our method is based on a hybrid-likelihood using regions and contours. The edge-based notion represents the relation between the external contours of the bone projections and an edge potential field estimated on the radiographic images. Region-based notion is the non-overlapping ratio between segmented and projected bone regions of interest (RoIs). Our automatic 3D reconstruction model entails stochastically minimizing an energy function allowing an estimation of deformation parameters of the bone shape. This 3D reconstruction method has been successfully tested on 13 biplanar radiographic image pairs, yielding very promising results.

BOSS: Bones, organs and skin shape model

A virtual anatomical model of a patient can be a valuable tool for enhancing clinical tasks such as workflow automation, patient-specific X-ray dose optimization, markerless tracking, positioning, and navigation assistance in image-guided interventions. For these tasks, it is highly desirable that the patient's surface and internal organs are of high quality for any pose and shape estimate. At present, the majority of statistical shape models (SSMs) are restricted to a small number of organs or bones or do not adequately represent the general population. To address this, we propose a deformable human shape and pose model that combines skin, internal organs, and bones, learned from CT images. By modeling the statistical variations in a pose-normalized space using probabilistic PCA while also preserving joint kinematics, our approach offers a holistic representation of the body that can be beneficial for automation in various medical applications. In an interventional setup, our model could, for example, facilitate automatic system/patient positioning, organ-specific iso-centering, automated collimation or collision prediction. We assessed our model's performance on a registered dataset, utilizing the unified shape space, and noted an average error of 3.6 mm for bones and 8.8 mm for organs. By utilizing solely skin surface data or patient metadata like height and weight, we find that the overall combined error for bone-organ measurement is 8.68 mm and 8.11 mm, respectively. To further verify our findings, we conducted additional tests on publicly available datasets with multi-part segmentations, which confirmed the effectiveness of our model. In the diverse TotalSegmentator dataset, the errors for bones and organs are observed to be 5.10mm and 8.72mm, respectively. Our work shows that anatomically parameterized statistical shape models can be created accurately and in a computationally efficient manner. The proposed approach enables the construction of shape models that can be directly integrated into to various medical applications.

An autonomous X-ray image acquisition and interpretation system for assisting percutaneous pelvic fracture fixation

PURPOSE

Percutaneous fracture fixation involves multiple X-ray acquisitions to determine adequate tool trajectories in bony anatomy. In order to reduce time spent adjusting the X-ray imager's gantry, avoid excess acquisitions, and anticipate inadequate trajectories before penetrating bone, we propose an autonomous system for intra-operative feedback that combines robotic X-ray imaging and machine learning for automated image acquisition and interpretation, respectively.

METHODS

Our approach reconstructs an appropriate trajectory in a two-image sequence, where the optimal second viewpoint is determined based on analysis of the first image. A deep neural network is responsible for detecting the tool and corridor, here a K-wire and the superior pubic ramus, respectively, in these radiographs. The reconstructed corridor and K-wire pose are compared to determine likelihood of cortical breach, and both are visualized for the clinician in a mixed reality environment that is spatially registered to the patient and delivered by an optical see-through head-mounted display.

RESULTS

We assess the upper bounds on system performance through in silico evaluation across 11 CTs with fractures present, in which the corridor and K-wire are adequately reconstructed. In post hoc analysis of radiographs across 3 cadaveric specimens, our system determines the appropriate trajectory to within 2.8 ± 1.3 mm and 2.7 ± 1.8[Formula: see text].

CONCLUSION

An expert user study with an anthropomorphic phantom demonstrates how our autonomous, integrated system requires fewer images and lower movement to guide and confirm adequate placement compared to current clinical practice. Code and data are available.

X-Ray to DRR Images Translation for Efficient Multiple Objects Similarity Measures in Deformable Model 3D/2D Registration

The robustness and accuracy of the intensity-based 3D/2D registration of a 3D model on planar X-ray image(s) is related to the quality of the image correspondences between the digitally reconstructed radiographs (DRR) generated from the 3D models (varying image) and the X-ray images (fixed target). While much effort may be devoted to generating realistic DRR that are similar to real X-rays (using complex X-ray simulation, adding densities information in 3D models, etc.), significant differences still remain between DRR and real X-ray images. Differences such as the presence of adjacent or superimposed soft tissue and bony or foreign structures lead to image matching difficulties and decrease the 3D/2D registration performance. In the proposed method, the X-ray images were converted into DRR images using a GAN-based cross-modality image-to-images translation. With this added prior step of XRAY-to-DRR translation, standard similarity measures become efficient even when using simple and fast DRR projection. For both images to match, they must belong to the same image domain and essentially contain the same kind of information. The XRAY-to-DRR translation also addresses the well-known issue of registering an object in a scene composed of multiple objects by separating the superimposed or/and adjacent objects to avoid mismatching across similar structures. We applied the proposed method to the 3D/2D fine registration of vertebra deformable models to biplanar radiographs of the spine. We showed that the XRAY-to-DRR translation enhances the registration results, by increasing the capture range and decreasing dependence on the similarity measure choice since the multi-modal registration becomes mono-modal.

Dynamic multi feature-class Gaussian process models

In model-based medical image analysis, three relevant features are the shape of structures of interest, their relative pose, and image intensity profiles representative of some physical properties. Often, these features are modelled separately through statistical models by decomposing the object's features into a set of basis functions through principal geodesic analysis or principal component analysis. However, analysing articulated objects in an image using independent single object models may lead to large uncertainties and impingement, especially around organ boundaries. Questions that come to mind are the feasibility of building a unique model that combines all three features of interest in the same statistical space, and what advantages can be gained for image analysis. This study presents a statistical modelling method for automatic analysis of shape, pose and intensity features in medical images which we call the Dynamic multi feature-class Gaussian process models (DMFC-GPM). The DMFC-GPM is a Gaussian process (GP)-based model with a shared latent space that encodes linear and non-linear variations. Our method is defined in a continuous domain with a principled way to represent shape, pose and intensity feature-classes in a linear space, based on deformation fields. A deformation field-based metric is adapted in the method for modelling shape and intensity variation as well as for comparing rigid transformations (pose). Moreover, DMFC-GPMs inherit properties intrinsic to GPs including marginalisation and regression. Furthermore, they allow for adding additional pose variability on top of those obtained from the image acquisition process; what we term as permutation modelling. For image analysis tasks using DMFC-GPMs, we adapt Metropolis-Hastings algorithms making the prediction of features fully probabilistic. We validate the method using controlled synthetic data and we perform experiments on bone structures from CT images of the shoulder to illustrate the efficacy of the model for pose and shape prediction. The model performance results suggest that this new modelling paradigm is robust, accurate, accessible, and has potential applications in a multitude of scenarios including the management of musculoskeletal disorders, clinical decision making and image processing.

3D shape reconstruction of the femur from planar X-ray images using statistical shape and appearance models

Major trauma is a condition that can result in severe bone damage. Customised orthopaedic reconstruction allows for limb salvage surgery and helps to restore joint alignment. For the best possible outcome three dimensional (3D) medical imaging is necessary, but its availability and access, especially in developing countries, can be challenging. In this study, 3D bone shapes of the femur reconstructed from planar radiographs representing bone defects were evaluated for use in orthopaedic surgery. Statistical shape and appearance models generated from 40 cadaveric X-ray computed tomography (CT) images were used to reconstruct 3D bone shapes. The reconstruction simulated bone defects of between 0% and 50% of the whole bone, and the prediction accuracy using anterior-posterior (AP) and anterior-posterior/medial-lateral (AP/ML) X-rays were compared. As error metrics for the comparison, measures evaluating the distance between contour lines of the projections as well as a measure comparing similarities in image intensities were used. The results were evaluated using the root-mean-square distance for surface error as well as differences in commonly used anatomical measures, including bow, femoral neck, diaphyseal-condylar and version angles between reconstructed surfaces from the shape model and the intact shape reconstructed from the CT image. The reconstructions had average surface errors between 1.59 and 3.59 mm with reconstructions using the contour error metric from the AP/ML directions being the most accurate. Predictions of bow and femoral neck angles were well below the clinical threshold accuracy of 3°, diaphyseal-condylar angles were around the threshold of 3° and only version angle predictions of between 5.3° and 9.3° were above the clinical threshold, but below the range reported in clinical practice using computer navigation (i.e., 17° internal to 15° external rotation). This study shows that the reconstructions from partly available planar images using statistical shape and appearance models had an accuracy which would support their potential use in orthopaedic reconstruction.

2D-3D reconstruction of the proximal femur from DXA scans: Evaluation of the 3D-Shaper software

Introduction: Osteoporosis is currently diagnosed based on areal bone mineral density (aBMD) computed from 2D DXA scans. However, aBMD is a limited surrogate for femoral strength since it does not account for 3D bone geometry and density distribution. QCT scans combined with finite element (FE) analysis can deliver improved femoral strength predictions. However, non-negligible radiation dose and high costs prevent a systematic usage of this technique for screening purposes. As an alternative, the 3D-Shaper software (3D-Shaper Medical, Spain) reconstructs the 3D shape and density distribution of the femur from 2D DXA scans. This approach could deliver a more accurate estimation of femoral strength than aBMD by using FE analysis on the reconstructed 3D DXA. Methods: Here we present the first independent evaluation of the software, using a dataset of 77 ex vivo femora. We extend a prior evaluation by including the density distribution differences, the spatial correlation of density values and an FE analysis. Yet, cortical thickness is left out of this evaluation, since the cortex is not resolved in our FE models. Results: We found an average surface distance of 1.16 mm between 3D DXA and QCT images, which shows a good reconstruction of the bone geometry. Although BMD values obtained from 3D DXA and QCT correlated well (r ² = 0.92), the 3D DXA BMD were systematically lower. The average BMD difference amounted to 64 mg/cm³, more than one-third of the 3D DXA BMD. Furthermore, the low correlation (r ² = 0.48) between density values of both images indicates a limited reconstruction of the 3D density distribution. FE results were in good agreement between QCT and 3D DXA images, with a high coefficient of determination (r ² = 0.88). However, this correlation was not statistically different from a direct prediction by aBMD. Moreover, we found differences in the fracture patterns between the two image types. QCT-based FE analysis resulted mostly in femoral neck fractures and 3D DXA-based FE in subcapital or pertrochanteric fractures. Discussion: In conclusion, 3D-Shaper generates an altered BMD distribution compared to QCT but, after careful density calibration, shows an interesting potential for deriving a standardized femoral strength from a DXA scan.

Development of pre-procedure virtual simulation for challenging interventional procedures: an experimental study with clinical application

Background

Most pain management techniques for challenging procedures are still performed under the guidance of the C-arm fluoroscope although it is sometimes difficult for even experienced clinicians to understand the modified three-dimensional anatomy as a two-dimensional X-ray image. To overcome these difficulties, the development of a virtual simulator may be helpful. Therefore, in this study, the authors developed a virtual simulator and presented its clinical application cases.

Methods

We developed a computer program to simulate the actual environment of the procedure. Computed tomography (CT) Digital Imaging and Communications in Medicine (DICOM) data were used for the simulations. Virtual needle placement was simulated at the most appropriate position for a successful block. Using a virtual C-arm, the authors searched for the position of the C-arm at which the needle was visualized as a point. The positional relationships between the anatomy of the patient and the needle were identified.

Results

For the simulations, the CT DICOM data of patients who visited the outpatient clinic was used. When the patients revisited the clinic, images similar to the simulated images were obtained by manipulating the C-arm. Transforaminal epidural injection, which was difficult to perform due to severe spinal deformity, and the challenging procedures of the superior hypogastric plexus block and Gasserian ganglion block, were successfully performed with the help of the simulation.

Conclusions

We created a pre-procedural virtual simulation and demonstrated its successful application in patients who are expected to undergo challenging procedures.

Rapid X-Ray-Based 3-D Finite Element Modeling of Medial Knee Joint Cartilage Biomechanics During Walking

Finite element (FE) modeling is becoming an increasingly popular method for analyzing knee joint mechanics and biomechanical mechanisms leading to osteoarthritis (OA). The most common and widely available imaging method for knee OA diagnostics is planar X-ray imaging, while more sophisticated imaging methods, e.g., magnetic resonance imaging (MRI) and computed tomography (CT), are seldom used. Hence, the capability to produce accurate biomechanical knee joint models directly from X-ray imaging would bring FE modeling closer to clinical use. Here, we extend our atlas-based framework by generating FE knee models from X-ray images (N = 28). Based on measured anatomical landmarks from X-ray and MRI, knee joint templates were selected from the atlas library. The cartilage stresses and strains of the X-ray-based model were then compared with the MRI-based model during the stance phase of the gait. The biomechanical responses were statistically not different between MRI- vs. X-ray-based models when the template obtained from X-ray imaging was the same as the MRI template. However, if this was not the case, the peak values of biomechanical responses were statistically different between X-ray and MRI models. The developed X-ray-based framework may pave the way for a clinically feasible approach for knee joint FE modeling.

Shape, Pose and Density Statistical Model for 3D Reconstruction of Articulated Structures from X-Ray Images

This article proposes a joint statistical model, to describe the volumetric shape + pose + density information, and a reconstruction algorithm to simultaneously recover the volumetric information of several anatomical structures from biplanar radiographs. A PCA-based representation is proposed as compact model representation and a hybrid AAM search and genetic optimization is used to perform the reconstruction. A study was conducted to recover a 3D volume grid containing a human knee mesh from 2 orthogonal simulated radiographs. The model was computed on a data set of 200 subjects and the reconstruction test was performed on 18 subjects, leading to a surface distance RMSE of 0.7 ± 0.31 mm for the distal femur, 0.9 ± 0.3 mm for the proximal tibia and 0.8 ± 0.3 mm for the fibula. These results demonstrate the feasibility and the pertinence of the proposed approach, the next step being its application in a clinical context.

DeepOrganNet: On-the-Fly Reconstruction and Visualization of 3D / 4D Lung Models from Single-View Projections by Deep Deformation Network

This paper introduces a deep neural network based method, i.e., DeepOrganNet, to generate and visualize fully high-fidelity 3D / 4D organ geometric models from single-view medical images with complicated background in real time. Traditional 3D / 4D medical image reconstruction requires near hundreds of projections, which cost insufferable computational time and deliver undesirable high imaging / radiation dose to human subjects. Moreover, it always needs further notorious processes to segment or extract the accurate 3D organ models subsequently. The computational time and imaging dose can be reduced by decreasing the number of projections, but the reconstructed image quality is degraded accordingly. To our knowledge, there is no method directly and explicitly reconstructing multiple 3D organ meshes from a single 2D medical grayscale image on the fly. Given single-view 2D medical images, e.g., 3D / 4D-CT projections or X-ray images, our end-to-end DeepOrganNet framework can efficiently and effectively reconstruct 3D / 4D lung models with a variety of geometric shapes by learning the smooth deformation fields from multiple templates based on a trivariate tensor-product deformation technique, leveraging an informative latent descriptor extracted from input 2D images. The proposed method can guarantee to generate high-quality and high-fidelity manifold meshes for 3D / 4D lung models; while, all current deep learning based approaches on the shape reconstruction from a single image cannot. The major contributions of this work are to accurately reconstruct the 3D organ shapes from 2D single-view projection, significantly improve the procedure time to allow on-the-fly visualization, and dramatically reduce the imaging dose for human subjects. Experimental results are evaluated and compared with the traditional reconstruction method and the state-of-the-art in deep learning, by using extensive 3D and 4D examples, including both synthetic phantom and real patient datasets. The efficiency of the proposed method shows that it only needs several milliseconds to generate organ meshes with 10K vertices, which has great potential to be used in real-time image guided radiation therapy (IGRT).

In Silico Optimization of Femoral Fixator Position and Configuration by Parametric CAD Model

Structural analysis, based on the finite element method, and structural optimization, can help surgery planning or decrease the probability of fixator failure during bone healing. Structural optimization implies the creation of many finite element model instances, usually built using a computer-aided design (CAD) model of the bone-fixator assembly. The three most important features of such CAD models are: parameterization, robustness and bidirectional associativity with finite elements (FE) models. Their significance increases with the increase in the complexity of the modeled fixator. The aim of this study was to define an automated procedure for the configuration and placement of fixators used in the treatment of long bone fractures. Automated and robust positioning of the selfdynamisable internal fixator on the femur was achieved and sensitivity analysis of fixator stress on the change of major design parameters was performed. The application of the proposed methodology is considered to be beneficial in the preparation of CAD models for automated structural optimization procedures used in long bone fixation.

Method for quantitative assessment of acetabular bone defects

The objective of the study was to suggest a novel quantitative assessment of acetabular bone defects based on a statistical shape model, validate the method, and present preliminary results. Two exemplary CT-data sets with acetabular bone defects were segmented to obtain a solid model of each defect pelvis. The pathological areas around the acetabulum were excluded and a statistical shape model was fitted to the remaining healthy bone structures. The excluded areas were extrapolated such that a solid model of the native pelvis per specimen resulted (i.e., each pelvis without defect). The validity of the reconstruction was tested by a leave-one-out study. Validation results showed median reconstruction errors of 3.0 mm for center of rotation, 1.7 mm for acetabulum diameter, 2.1° for inclination, 2.5° for anteversion, and 3.3 mm³ for bone volume around the acetabulum. By applying Boolean operations on the solid models of defect and native pelvis, bone loss and bone formation in four different sectors were assessed. For both analyzed specimens, bone loss and bone formation per sector were calculated and were consistent with the visual impression. In specimen_1 bone loss was predominant in the medial wall (10.8 ml; 79%), in specimen_2 in the posterior column (15.6 ml; 46%). This study showed the feasibility of a quantitative assessment of acetabular bone defects using a statistical shape model-based reconstruction method. Validation results showed acceptable reconstruction accuracy, also when less healthy bone remains. The method could potentially be used for implant development, pre-clinical testing, pre-operative planning, and intra-operative navigation. © 2018 The Authors. Journal of Orthopaedic Research® Published by Wiley Periodicals, Inc. on behalf of Orthopaedic Research Society. J Orthop Res 9999:1-9, 2018.

Statistical Shape Models: Understanding and Mastering Variation in Anatomy

In our chapter we are describing how to reconstruct three-dimensional anatomy from medical image data and how to build Statistical 3D Shape Models out of many such reconstructions yielding a new kind of anatomy that not only allows quantitative analysis of anatomical variation but also a visual exploration and educational visualization. Future digital anatomy atlases will not only show a static (average) anatomy but also its normal or pathological variation in three or even four dimensions, hence, illustrating growth and/or disease progression.Statistical Shape Models (SSMs) are geometric models that describe a collection of semantically similar objects in a very compact way. SSMs represent an average shape of many three-dimensional objects as well as their variation in shape. The creation of SSMs requires a correspondence mapping, which can be achieved e.g. by parameterization with a respective sampling. If a corresponding parameterization over all shapes can be established, variation between individual shape characteristics can be mathematically investigated.We will explain what Statistical Shape Models are and how they are constructed. Extensions of Statistical Shape Models will be motivated for articulated coupled structures. In addition to shape also the appearance of objects will be integrated into the concept. Appearance is a visual feature independent of shape that depends on observers or imaging techniques. Typical appearances are for instance the color and intensity of a visual surface of an object under particular lighting conditions, or measurements of material properties with computed tomography (CT) or magnetic resonance imaging (MRI). A combination of (articulated) Statistical Shape Models with statistical models of appearance lead to articulated Statistical Shape and Appearance Models (a-SSAMs).After giving various examples of SSMs for human organs, skeletal structures, faces, and bodies, we will shortly describe clinical applications where such models have been successfully employed. Statistical Shape Models are the foundation for the analysis of anatomical cohort data, where characteristic shapes are correlated to demographic or epidemiologic data. SSMs consisting of several thousands of objects offer, in combination with statistical methods or machine learning techniques, the possibility to identify characteristic clusters, thus being the foundation for advanced diagnostic disease scoring.

Review of 2-D/3-D Reconstruction Using Statistical Shape and Intensity Models and X-Ray Image Synthesis: Toward a Unified Framework

Patient-specific three-dimensional (3-D) bone models are useful for a number of clinical applications such as surgery planning, postoperative evaluation, as well as implant and prosthesis design. Two-dimensional-to-3-D (2-D/3-D) reconstruction, also known as model-to-modality or atlas-based 2-D/3-D registration, provides a means of obtaining a 3-D model of a patient's bones from their 2-D radiographs when 3-D imaging modalities are not available. The preferred approach for estimating both shape and density information (that would be present in a patient's computed tomography data) for 2-D/3-D reconstruction makes use of digitally reconstructed radiographs and deformable models in an iterative, non-rigid, intensity-based approach. Based on a large number of state-of-the-art 2-D/3-D bone reconstruction methods, a unified mathematical formulation of the problem is proposed in a common conceptual framework, using unambiguous terminology. In addition, shortcomings, recent adaptations, and persisting challenges are discussed along with insights for future research.

Radiative Transport Based Flame Volume Reconstruction from Videos

We introduce a novel approach for flame volume reconstruction from videos using inexpensive charge-coupled device (CCD) consumer cameras. The approach includes an economical data capture technique using inexpensive CCD cameras. Leveraging the smear feature of the CCD chip, we present a technique for synchronizing CCD cameras while capturing flame videos from different views. Our reconstruction is based on the radiative transport equation which enables complex phenomena such as emission, extinction, and scattering to be used in the rendering process. Both the color intensity and temperature reconstructions are implemented using the CUDA parallel computing framework, which provides real-time performance and allows visualization of reconstruction results after every iteration. We present the results of our approach using real captured data and physically-based simulated data. Finally, we also compare our approach against the other state-of-the-art flame volume reconstruction methods and demonstrate the efficacy and efficiency of our approach in four different applications: (1) rendering of reconstructed flames in virtual environments, (2) rendering of reconstructed flames in augmented reality, (3) flame stylization, and (4) reconstruction of other semitransparent phenomena.

Association of incident hip fracture with the estimated femoral strength by finite element analysis of DXA scans in the Osteoporotic Fractures in Men (MrOS) study

Finite element model can estimate bone strength better than BMD. This study used such a model to determine its association with hip fracture risk and found that the strength estimate provided limited improvement over the hip BMDs in predicting femoral neck (FN) fracture risk only.

INTRODUCTION

Bone fractures occur only when it is loaded beyond its ultimate strength. The goal of this study was to determine the association of femoral strength, as estimated by finite element (FE) analysis of DXA scans, with incident hip fracture as a single condition or with femoral neck (FN) and trochanter (TR) fractures separately in older men.

METHODS

This prospective case-cohort study included 91 FN and 64 TR fracture cases and a random sample of 500 men (14 had a hip fracture) from the Osteoporotic Fractures in Men study during a mean ± SD follow-up of 7.7 ± 2.2 years. We analysed the baseline DXA scans of the hip using a validated plane-stress, linear-elastic FE model of the proximal femur and estimated the femoral strength during a sideways fall.

RESULTS

The estimated strength was significantly (P < 0.05) associated with hip fracture independent of the TR and total hip (TH) BMDs but not FN BMD, and combining the strength with BMD did not improve the hip fracture prediction. The strength estimate was associated with FN fractures independent of the FN, TR and TH BMDs; the age-BMI-BMD adjusted hazard ratio (95% CI) per SD decrease of the strength was 1.68 (1.07-2.64), 2.38 (1.57, 3.61) and 2.04 (1.34, 3.11), respectively. This association with FN fracture was as strong as FN BMD (Harrell's C index for the strength 0.81 vs. FN BMD 0.81) and stronger than TR and TH BMDs (0.8 vs. 0.78 and 0.81 vs. 0.79). The strength's association with TR fracture was not independent of hip BMD.

CONCLUSIONS

Although the strength estimate provided additional information over the hip BMDs, its improvement in predictive ability over the hip BMDs was confined to FN fracture only and limited.

Reliable landmarks for precise topographical analyses of pathological structural changes of the ovine tibial plateau in 2-D and 3-D subspaces

Selecting identical topographical locations to analyse pathological structural changes of the osteochondral unit in translational models remains difficult. The specific aim of the study was to provide objectively defined reference points on the ovine tibial plateau based on 2-D sections of micro-CT images useful for reproducible sample harvesting and as standardized landmarks for landmark-based 3-D image registration. We propose 5 reference points, 11 reference lines and 12 subregions that are detectable macroscopically and on 2-D micro-CT sections. Their value was confirmed applying landmark-based rigid and affine 3-D registration methods. Intra- and interobserver comparison showed high reliabilities, and constant positions (standard errors < 1%). Spatial patterns of the thicknesses of the articular cartilage and subchondral bone plate were revealed by measurements in 96 individual points of the tibial plateau. As a case study, pathological phenomena 6 months following OA induction in vivo such as osteophytes and areas of OA development were mapped to the individual subregions. These new reference points and subregions are directly identifiable on tibial plateau specimens or macroscopic images, enabling a precise topographical location of pathological structural changes of the osteochondral unit in both 2-D and 3-D subspaces in a region-appropriate fashion relevant for translational investigations.

Planning, guidance, and quality assurance of pelvic screw placement using deformable image registration

Percutaneous pelvic screw placement is challenging due to narrow bone corridors surrounded by vulnerable structures and difficult visual interpretation of complex anatomical shapes in 2D x-ray projection images. To address these challenges, a system for planning, guidance, and quality assurance (QA) is presented, providing functionality analogous to surgical navigation, but based on robust 3D-2D image registration techniques using fluoroscopy images already acquired in routine workflow. Two novel aspects of the system are investigated: automatic planning of pelvic screw trajectories and the ability to account for deformation of surgical devices (K-wire deflection). Atlas-based registration is used to calculate a patient-specific plan of screw trajectories in preoperative CT. 3D-2D registration aligns the patient to CT within the projective geometry of intraoperative fluoroscopy. Deformable known-component registration (dKC-Reg) localizes the surgical device, and the combination of plan and device location is used to provide guidance and QA. A leave-one-out analysis evaluated the accuracy of automatic planning, and a cadaver experiment compared the accuracy of dKC-Reg to rigid approaches (e.g. optical tracking). Surgical plans conformed within the bone cortex by 3-4 mm for the narrowest corridor (superior pubic ramus) and  >5 mm for the widest corridor (tear drop). The dKC-Reg algorithm localized the K-wire tip within 1.1 mm and 1.4° and was consistently more accurate than rigid-body tracking (errors up to 9 mm). The system was shown to automatically compute reliable screw trajectories and accurately localize deformed surgical devices (K-wires). Such capability could improve guidance and QA in orthopaedic surgery, where workflow is impeded by manual planning, conventional tool trackers add complexity and cost, rigid tool assumptions are often inaccurate, and qualitative interpretation of complex anatomy from 2D projections is prone to trial-and-error with extended fluoroscopy time.

3D-DXA: Assessing the Femoral Shape, the Trabecular Macrostructure and the Cortex in 3D from DXA images

The 3D distribution of the cortical and trabecular bone mass in the proximal femur is a critical component in determining fracture resistance that is not taken into account in clinical routine Dual-energy X-ray Absorptiometry (DXA) examination. In this paper, a statistical shape and appearance model together with a 3D-2D registration approach are used to model the femoral shape and bone density distribution in 3D from an anteroposterior DXA projection. A model-based algorithm is subsequently used to segment the cortex and build a 3D map of the cortical thickness and density. Measurements characterising the geometry and density distribution were computed for various regions of interest in both cortical and trabecular compartments. Models and measurements provided by the "3D-DXA" software algorithm were evaluated using a database of 157 study subjects, by comparing 3D-DXA analyses (using DXA scanners from three manufacturers) with measurements performed by Quantitative Computed Tomography (QCT). The mean point-to-surface distance between 3D-DXA and QCT femoral shapes was 0.93 mm. The mean absolute error between cortical thickness and density estimates measured by 3D-DXA and QCT was 0.33 mm and 72 mg/cm3. Correlation coefficients (R) between the 3D-DXA and QCT measurements were 0.86, 0.93, and 0.95 for the volumetric bone mineral density at the trabecular, cortical, and integral compartments respectively, and 0.91 for the mean cortical thickness. 3D-DXA provides a detailed analysis of the proximal femur, including a separate assessment of the cortical layer and trabecular macrostructure, which could potentially improve osteoporosis management while maintaining DXA as the standard routine modality.

Parallel generation of digitally reconstructed radiographs on heterogeneous multi-GPU workstations

The growing importance of three-dimensional radiotherapy treatment has been associated with the active presence of advanced computational workflows that can simulate conventional x-ray films from computed tomography (CT) volumetric data to create digitally reconstructed radiographs (DRR). These simulated x-ray images are used to continuously verify the patient alignment in image-guided therapies with 2D-3D image registration. The present DRR rendering pipelines are quite limited to handle huge imaging stacks generated by recent state-of-the-art CT imaging modalities. We present a high performance x-ray rendering pipeline that is capable of generating high quality DRRs from large scale CT volumes. The pipeline is designed to harness the immense computing power of all the heterogeneous computing platforms that are connected to the system relying on OpenCL. Load-balancing optimization is also addressed to equalize the rendering load across the entire system. The performance benchmarks demonstrate the capability of our pipeline to generate high quality DRRs from relatively large CT volumes at interactive frame rates using cost-effective multi-GPU workstations. A 5122 DRR frame can be rendered from 1024 × 2048 × 2048 CT volumes at 85 frames per second.

Accuracy of an adjustable patient-specific guide for acetabular alignment in hip replacement surgery (Optihip)

Implant malalignment in hip arthroplasty increases the risk of revision surgery due to problems such as hip instability, wear, and impingement. Traditional instrumentation lacks accuracy and does not individualize the goal. Computer-assisted surgery (CAS) and patient-specific solutions improve accuracy but add considerably to the cost, amongst other drawbacks. We developed an adjustable mechanical device, called Optihip, which is set to a patient-specific goal preoperatively and is independent of pelvis position intraoperatively. The purpose of the present study was to evaluate Optihip's accuracy ex vivo. Acetabular components were implanted into six cadaveric specimens, 12 hips, by two surgeons, with the device individually adjusted according to preoperative templating on computed tomography (CT) images relative to defined acetabular rim landmarks; options also exist for templating on single or biplanar X-rays. Intraoperatively, the device was positioned on the corresponding anatomical landmarks allowing the insertion of a guide pin, which then defined the desired orientation of the acetabular cup during impaction. Mean absolute difference between the preoperatively planned cup alignment and final acetabular cup orientation, measured from postoperative CT images, was 2.5±1.2° for inclination and 2.5±2.2° for version with maximum values of 4.7° and 6.8°, respectively. Compared with previous in vivo reports, Optihip guided the acetabular cup orientation more accurately than conventional hip arthroplasty, and comparably to CAS or patient-specific systems, while fitting into the normal surgical workflow. Although clinical testing is required to confirm these experimental results, the positive ex vivo accuracy suggests good potential for improving revision rates and patient functional outcome.