A new validation method for X-ray mammogram registration algorithms using a projection model of breast X-ray compression

Towards CNN-Based Registration of Craniocaudal and Mediolateral Oblique 2-D X-ray Mammographic Images

We investigate methodologies for the automated registration of pairs of 2-D X-ray mammographic images, taken from the two standard mammographic angles. We present two exploratory techniques, based on Convolutional Neural Networks, to examine their potential for co-registration of findings on the two standard mammographic views. To test algorithm performance, our analysis uses a synthetic, surrogate data set for performing controlled experiments, as well as real 2-D X-ray mammogram imagery. The preliminary results are promising, and provide insights into how the proposed techniques may support multi-view X-ray mammography image registration currently and as technology evolves in the future.

A review of biomechanically informed breast image registration

Breast radiology encompasses the full range of imaging modalities from routine imaging via x-ray mammography, magnetic resonance imaging and ultrasound (both two- and three-dimensional), to more recent technologies such as digital breast tomosynthesis, and dedicated breast imaging systems for positron emission mammography and ultrasound tomography. In addition new and experimental modalities, such as Photoacoustics, Near Infrared Spectroscopy and Electrical Impedance Tomography etc, are emerging. The breast is a highly deformable structure however, and this greatly complicates visual comparison of imaging modalities for the purposes of breast screening, cancer diagnosis (including image guided biopsy), tumour staging, treatment monitoring, surgical planning and simulation of the effects of surgery and wound healing etc. Due primarily to the challenges posed by these gross, non-rigid deformations, development of automated methods which enable registration, and hence fusion, of information within and across breast imaging modalities, and between the images and the physical space of the breast during interventions, remains an active research field which has yet to translate suitable methods into clinical practice. This review describes current research in the field of breast biomechanical modelling and identifies relevant publications where the resulting models have been incorporated into breast image registration and simulation algorithms. Despite these developments there remain a number of issues that limit clinical application of biomechanical modelling. These include the accuracy of constitutive modelling, implementation of representative boundary conditions, failure to meet clinically acceptable levels of computational cost, challenges associated with automating patient-specific model generation (i.e. robust image segmentation and mesh generation) and the complexity of applying biomechanical modelling methods in routine clinical practice.

Deformable registration for quantifying longitudinal tumor changes during neoadjuvant chemotherapy

PURPOSE

To evaluate DRAMMS, an attribute-based deformable registration algorithm, compared to other intensity-based algorithms, for longitudinal breast MRI registration, and to show its applicability in quantifying tumor changes over the course of neoadjuvant chemotherapy.

METHODS

Breast magnetic resonance images from 14 women undergoing neoadjuvant chemotherapy were analyzed. The accuracy of DRAMMS versus five intensity-based deformable registration methods was evaluated based on 2,380 landmarks independently annotated by two experts, for the entire image volume, different image subregions, and patient subgroups. The registration method with the smallest landmark error was used to quantify tumor changes, by calculating the Jacobian determinant maps of the registration deformation.

RESULTS

DRAMMS had the smallest landmark errors (6.05 ± 4.86 mm), followed by the intensity-based methods CC-FFD (8.07 ± 3.86 mm), NMI-FFD (8.21 ± 3.81 mm), SSD-FFD (9.46 ± 4.55 mm), Demons (10.76 ± 6.01 mm), and Diffeomorphic Demons (10.82 ± 6.11 mm). Results show that registration accuracy also depends on tumor versus normal tissue regions and different patient subgroups.

CONCLUSIONS

The DRAMMS deformable registration method, driven by attribute-matching and mutual-saliency, can register longitudinal breast magnetic resonance images with a higher accuracy than several intensity-matching methods included in this article. As such, it could be valuable for more accurately quantifying heterogeneous tumor changes as a marker of response to treatment.

Simulation of mammographic breast compression in 3D MR images using ICP-based B-spline deformation for multimodality breast cancer diagnosis

PURPOSE

Multimodality mammography using conventional 2D mammography and dynamic contrast-enhanced 3D magnetic resonance imaging (DCE-MRI) is frequently performed for breast cancer detection and diagnosis. Combination of both imaging modalities requires superimposition of corresponding structures in mammograms and MR images. This task is challenging due to large differences in (1) dimensionality and spatial resolution, (2) variations in tissue contrast, as well as (3) differences in breast orientation and deformation during the image acquisition. A new method for multimodality breast image registration was developed and tested.

METHODS

Combined diagnosis of mammograms and MRI datasets was achieved by simulation of mammographic breast compression to overcome large differences in breast deformation. Surface information was extracted from the 3D MR image, and back-projection of the 2D breast contour in the mammogram was done. B-spline-based 3D/3D surface-based registration was then used to approximate mammographic breast compression. This breast deformation simulation was performed on 14 MRI datasets with 19 corresponding mammograms. The results were evaluated by comparison with distances between corresponding structures identified by an expert observer.

RESULTS

The evaluation revealed an average distance of 6.46 mm between corresponding structures, when an optimized initial alignment between both image datasets is performed. Without the optimization, the accuracy is 9.12 mm.

CONCLUSION

A new surface-based method that approximates the mammographic deformation due to breast compression without using a specific complex model needed for finite-element-based methods was developed and tested with favorable results. The simulated compression can serve as foundation for a point-to-line correspondence between 2D mammograms and 3D MR image data.

Deformable medical image registration: a survey

Deformable image registration is a fundamental task in medical image processing. Among its most important applications, one may cite: 1) multi-modality fusion, where information acquired by different imaging devices or protocols is fused to facilitate diagnosis and treatment planning; 2) longitudinal studies, where temporal structural or anatomical changes are investigated; and 3) population modeling and statistical atlases used to study normal anatomical variability. In this paper, we attempt to give an overview of deformable registration methods, putting emphasis on the most recent advances in the domain. Additional emphasis has been given to techniques applied to medical images. In order to study image registration methods in depth, their main components are identified and studied independently. The most recent techniques are presented in a systematic fashion. The contribution of this paper is to provide an extensive account of registration techniques in a systematic manner.

Breast lesion co-localisation between X-ray and MR images using finite element modelling

This paper presents a novel X-ray and MR image registration technique based on individual-specific biomechanical finite element (FE) models of the breasts. Information from 3D magnetic resonance (MR) images was registered to X-ray mammographic images using non-linear FE models subject to contact mechanics constraints to simulate the large compressive deformations between the two imaging modalities. A physics-based perspective ray-casting algorithm was used to generate 2D pseudo-X-ray projections of the FE-warped 3D MR images. Unknown input parameters to the FE models, such as the location and orientation of the compression plates, were optimised to provide the best match between the pseudo and clinical X-ray images. The methods were validated using images taken before and during compression of a breast-shaped phantom, for which 12 inclusions were tracked between imaging modalities. These methods were then applied to X-ray and MR images from six breast cancer patients. Error measures (such as centroid and surface distances) of segmented tumours in simulated and actual X-ray mammograms were used to assess the accuracy of the methods. Sensitivity analysis of the lesion co-localisation accuracy to rotation about the anterior-posterior axis was then performed. For 10 of the 12 X-ray mammograms, lesion localisation accuracies of 14 mm and less were achieved. This analysis on the rotation about the anterior-posterior axis indicated that, in cases where the lesion lies in the plane parallel to the mammographic compression plates, that cuts through the nipple, such rotations have relatively minor effects.This has important implications for clinical applicability of this multi-modality lesion registration technique, which will aid in the diagnosis and treatment of breast cancer.

Breast image analysis for risk assessment, detection, diagnosis, and treatment of cancer

The role of breast image analysis in radiologists' interpretation tasks in cancer risk assessment, detection, diagnosis, and treatment continues to expand. Breast image analysis methods include segmentation, feature extraction techniques, classifier design, biomechanical modeling, image registration, motion correction, and rigorous methods of evaluation. We present a review of the current status of these task-based image analysis methods, which are being developed for the various image acquisition modalities of mammography, tomosynthesis, computed tomography, ultrasound, and magnetic resonance imaging. Depending on the task, image-based biomarkers from such quantitative image analysis may include morphological, textural, and kinetic characteristics and may depend on accurate modeling and registration of the breast images. We conclude with a discussion of future directions.

Simulations and phantom evaluations of magnetic resonance electrical impedance tomography (MREIT) for breast cancer detection

MREIT is a new imaging modality that can be used to reconstruct high-resolution conductivity images of the human body. Since conductivity values of cancerous tissues in the breast are significantly higher than those of surrounding normal tissues, breast imaging using MREIT may provide a new noninvasive way of detecting early stage of cancer. In this paper, we present results of experimental and numerical simulation studies of breast MREIT. We built a realistic three-dimensional model of the human breast connected to a simplified model of the chest including the heart and evaluated the ability of MREIT to detect cancerous anomalies in a background material with similar electrical properties to breast tissue. We performed numerical simulations of various scenarios in breast MREIT including assessment of the effects of fat inclusions and effects related to noise levels, such as changing the amplitude of injected currents, effect of added noise and number of averages. Phantom results showed straightforward detection of cancerous anomalies in a background was possible with low currents and few averages. The simulation results showed it should be possible to detect a cancerous anomaly in the breast, while restricting the maximal current density in the heart below published levels for nerve excitation.

Accuracy validation for medical image registration algorithms: a review

Accuracy validation is essential to clinical application of medical image registration techniques. Registration validation remains a challenging problem in practice mainly due to lack of 'ground truth'.In this paper, an overview of current validation methods for medical image registration is presented with detailed discussion of their benefits and drawbacks.Special focus is on non-rigid registration validation. Promising solution is also discussed.

Regularising limited view tomography using anatomical reference images and information theoretic similarity metrics

This paper is concerned with limited view tomography. Inspired by the application of digital breast tomosynthesis (DBT), which is but one of an increasing number of applications of limited view tomography, we concentrate primarily on cases where the angular range is restricted to a narrow wedge of approximately ±30°, and the number of views is restricted to 10-30. The main challenge posed by these conditions is undersampling, also known as the null space problem. As a consequence of the Fourier Slice Theorem, a limited angular range leaves large swathes of the object's Fourier space unsampled, leaving a large space of possible solutions, reconstructed volumes, for a given set of inputs. We explore the feasibility of using same- or different-modality images as anatomical priors to constrain the null space, hence the solution. To allow for different-modality priors, we choose information theoretic measures to quantify the similarity between reconstructions and their priors. We demonstrate the limitations of two popular choices, namely mutual information and joint entropy, and propose robust alternatives that overcome their limitations. One of these alternatives is essentially a joint mixture model of the image and its prior. Promising mitigation of the data insufficiency problem is demonstrated using 2D synthetic as well as clinical phantoms. This work initially assumes a priori registered priors, and is then extended to allow for the registration to be performed simultaneously with the reconstruction.

Large breast compressions: observations and evaluation of simulations

PURPOSE

Several methods have been proposed to simulate large breast compressions such as those occurring during x-ray mammography. However, the evaluation of these methods against real data is rare. The aim of this study is to learn more about the deformation behavior of breasts and to assess a simulation method.

METHODS

Magnetic resonance (MR) images of 11 breasts before and after applying a relatively large in vivo compression in the medial direction were acquired. Nonrigid registration was employed to study the deformation behavior. Optimal material properties for finite element modeling were determined and their prediction performance was assessed. The realism of simulated compressions was evaluated by comparing the breast shapes on simulated and real mammograms.

RESULTS

Following image registration, 19 breast compressions from 8 women were studied. An anisotropic deformation behavior, with a reduced elongation in the anterior-posterior direction and an increased stretch in the inferior-superior direction was observed. Using finite element simulations, the performance of isotropic and transverse isotropic material models to predict the displacement of internal landmarks was compared. Isotropic materials reduced the mean displacement error of the landmarks from 23.3 to 4.7 mm, on average, after optimizing material properties with respect to breast surface alignment and image similarity. Statistically significantly smaller errors were achieved with transverse isotropic materials (4.1 mm, P=0.0045). Homogeneous material models performed substantially worse (transverse isotropic: 5.5 mm; isotropic: 6.7 mm). Of the parameters varied, the amount of anisotropy had the greatest influence on the results. Optimal material properties varied less when grouped by patient rather than by compression magnitude (mean: 0.72 vs. 1.44). Employing these optimal materials for simulating mammograms from ten MR breast images of a different cohort resulted in more realistic breast shapes than when using established material models.

CONCLUSIONS

Breasts in the prone position exhibited an anisotropic compression behavior. Transverse isotropic materials with an increased stiffness in the anterior-posterior direction improved the prediction of these deformations and produced more realistic mammogram simulations from MR images.

Motion correction and attenuation correction for respiratory gated PET images

Positron emission tomography (PET) is a molecular imaging technique which provides important functional information about the human body. However, thoracic PET images are often substantially degraded by respiratory motion, which adversely impacts on subsequent diagnosis. In this paper, a motion correction and attenuation correction method is proposed to correct for motion in respiratory gated PET images and to yield an accurate distribution of the radioactivity concentration. Experimental results show that this method can effectively correct for motion and improve PET image quality. The method is able to provide improved diagnostic information without increasing the acquisition time or the radiation burden.

Computational simulation of breast compression based on segmented breast and fibroglandular tissues on magnetic resonance images

This study presents a finite element-based computational model to simulate the three-dimensional deformation of a breast and fibroglandular tissues under compression. The simulation was based on 3D MR images of the breast, and craniocaudal and mediolateral oblique compression, as used in mammography, was applied. The geometry of the whole breast and the segmented fibroglandular tissues within the breast were reconstructed using triangular meshes by using the Avizo 6.0 software package. Due to the large deformation in breast compression, a finite element model was used to simulate the nonlinear elastic tissue deformation under compression, using the MSC.Marc software package. The model was tested in four cases. The results showed a higher displacement along the compression direction compared to the other two directions. The compressed breast thickness in these four cases at a compression ratio of 60% was in the range of 5-7 cm, which is a typical range of thickness in mammography. The projection of the fibroglandular tissue mesh at a compression ratio of 60% was compared to the corresponding mammograms of two women, and they demonstrated spatially matched distributions. However, since the compression was based on magnetic resonance imaging (MRI), which has much coarser spatial resolution than the in-plane resolution of mammography, this method is unlikely to generate a synthetic mammogram close to the clinical quality. Whether this model may be used to understand the technical factors that may impact the variations in breast density needs further investigation. Since this method can be applied to simulate compression of the breast at different views and different compression levels, another possible application is to provide a tool for comparing breast images acquired using different imaging modalities--such as MRI, mammography, whole breast ultrasound and molecular imaging--that are performed using different body positions and under different compression conditions.

A Biomechanical model of spiculated tumours under mammographic compressions

The aim of this paper is to introduce effects well known to clinicians -but neglected to date- in the biomechanical modelling of breast malignant tumours. We develop a model of an isolated stellate breast tumour under mammographic compression forces. We study a range of reported mechanical properties, both linear elastic and hyperelastic. We also introduce different volumes of increased density/stiffness around the tumour. We show that each of these issues has a non-negligible effect on stresses and strains/deformations.