A nonlinear biomechanical model based registration method for aligning prone and supine MR breast images

FEM simulation of breast deformation with semi-fluid representation

PURPOSE

In image-guided surgery for breast cancer, the representation of the breast deformation between planning and surgery plays a key role. The breast deforms significantly and behaves as a fluid with some constraints. Concretely, the deep fat layer in the breast deforms fluidly due to its incomplete fixation to the chest wall, while the anchoring structures by fascia avoid excessive deformation. In this study, we propose a method to simulate the semi-fluid deformation of the breast, considering the fluidic properties of the adipose tissue under the constraints of the anchoring structures.

METHODS

The proposed method prioritizes anatomical features of the breast, enhancing tissue mobility near the chest wall and modeling the anchoring structure of the fascia along the inframammary fold. To simulate semi-fluid deformation, constraint force from anchoring structure is applied to prone-positioned breast model, using a finite element method.

RESULTS

The results of the evaluation indicate a tumor center registration error of 11.87 ± 4.05 mm. Additionally, we verified how semi-fluid representation affects the registration error. The tumor's Hausdorff distance decreased from 12.89 ± 6.24 mm to 11.50 ± 4.38 mm with considering semi-fluidity.

CONCLUSION

The results showed that the use of semi-fluid representation tends to reduce registration errors. Therefore, it was suggested that the proposed method could improve the accuracy of breast posture conversion.

Evaluation of breast skin and tissue stiffness using a non-invasive aspiration device and impact of clinical predictors

A personalized 3D breast model could present a real benefit for preoperative discussion with patients, surgical planning, and guidance. Breast tissue biomechanical properties have been poorly studied in vivo, although they are important for breast deformation simulation. The main objective of our study was to determine breast skin thickness and breast skin and adipose/fibroglandular tissue stiffness. The secondary objective was to assess clinical predictors of elasticity and thickness: age, smoking status, body mass index, contraception, pregnancies, breastfeeding, menopausal status, history of radiotherapy or breast surgery. Participants were included at the Montpellier University Breast Surgery Department from March to May 2022. Breast skin thickness was measured by ultrasonography, breast skin and adipose/fibroglandular tissue stiffnesses were determined with a VLASTIC non-invasive aspiration device at three different sites (breast segments I-III). Multivariable linear models were used to assess clinical predictors of elasticity and thickness. In this cohort of 196 women, the mean breast skin and adipose/fibroglandular tissue stiffness values were 39 and 3 kPa, respectively. The mean breast skin thickness was 1.83 mm. Only menopausal status was significantly correlated with breast skin thickness and adipose/fibroglandular tissue stiffness. The next step will be to implement these stiffness and thickness values in a biomechanical breast model and to evaluate its capacity to predict breast tissue deformations.

Breast simulation pipeline: From medical imaging to patient-specific simulations

BACKGROUND

Breast-conserving surgery is the most acceptable operation for breast cancer removal from an invasive and psychological point of view. Before the surgical procedure, a preoperative MRI is performed in the prone configuration, while the surgery is achieved in the supine position. This leads to a considerable movement of the breast, including the tumor, between the two poses, complicating the surgeon's task.

METHODS

In this work, a simulation pipeline allowing the computation of patient-specific geometry and the prediction of personalized breast material properties was put forward. Through image segmentation, a finite element model including the subject-specific geometry is established. By first computing an undeformed state of the breast, the geometrico-material model is calibrated by surface acquisition in the intra-operative stance.

FINDINGS

Using an elastic corotational formulation, the patient-specific mechanical properties of the breast and skin were identified to obtain the best estimates of the supine configuration. The final results are a shape-fitting closest point residual of 4.00 mm for the mechanical parameters Ebreast=0.32 kPa and Eskin=22.72 kPa, congruent with the current state-of-the-art. The Covariance Matrix Adaptation Evolution Strategy optimizer converges on average between 5 to 30 min depending on the initial parameters, reaching a simulation speed of 20 s. To our knowledge, our model offers one of the best compromises between accuracy and speed.

INTERPRETATION

Satisfactory results were obtained for the estimation of breast deformation from preoperative to intra-operative configuration. Furthermore, we have demonstrated the clinical feasibility of such applications using a simulation framework that aims at the smallest disturbance of the actual surgical pipeline.

Estimation of the biomechanical mammographic deformation of the breast using machine learning models

BACKGROUND

A typical problem in the registration of MRI and X-ray mammography is the nonlinear deformation applied to the breast during mammography. We have developed a method for virtual deformation of the breast using a biomechanical model automatically constructed from MRI. The virtual deformation is applied in two steps: unloaded state estimation and compression simulation. The finite element method is used to solve the deformation process. However, the extensive computational cost prevents its usage in clinical routine.

METHODS

We propose three machine learning models to overcome this problem: an extremely randomized tree (first model), extreme gradient boosting (second model), and deep learning-based bidirectional long short-term memory with an attention layer (third model) to predict the deformation of a biomechanical model. We evaluated our methods with 516 breasts with realistic compression ratios up to 76%.

FINDINGS

We first applied one-fold validation, in which the second and third models performed better than the first model. We then applied ten-fold validation. For the unloaded state estimation, the median RMSE for the second and third models is 0.8 mm and 1.2 mm, respectively. For the compression, the median RMSE is 3.4 mm for both models. We evaluated correlations between model accuracy and characteristics of the clinical datasets such as compression ratio, breast volume, and tissue types.

INTERPRETATION

Using the proposed models, we achieved accurate results comparable to the finite element model, with a speedup of factor 240 using the extreme gradient boosting model. These proposed models can replace the finite element model simulation, enabling clinically relevant real-time application.

Incorporating heterogeneity and anisotropy for surgical applications in breast deformation modeling

BACKGROUND

Simulating soft-tissue breast deformations is of interest for many applications including image fusion, longitudinal registration, and image-guided surgery. For the surgical use case, positional changes cause breast deformations that compromise the use of preoperative imaging to inform tumor excision. Even when acquiring imaging in the supine position, which better reflects surgical presentation, deformations still occur due to arm motion and orientation changes. A biomechanical modeling approach to simulate supine breast deformations for surgical applications must be both accurate and compatible with the clinical workflow.

METHODS

A supine MR breast imaging dataset from n = 11 healthy volunteers was used to simulate surgical deformations by acquiring images in arm-down and arm-up positions. Three linear-elastic modeling approaches with varying levels of complexity were used to predict deformations caused by this arm motion: a homogeneous isotropic model, a heterogeneous isotropic model, and a heterogeneous anisotropic model using a transverse-isotropic constitutive model.

FINDINGS

The average target registration errors for subsurface anatomical features were 5.4 ± 1.5 mm for the homogeneous isotropic model, 5.3 ± 1.5 mm for the heterogeneous isotropic model, and 4.7 ± 1.4 mm for the heterogeneous anisotropic model. A statistically significant improvement in target registration error was observed between the heterogeneous anisotropic model and both the homogeneous and the heterogeneous isotropic models (P < 0.01).

INTERPRETATION

While a model that fully incorporates all constitutive complexities of anatomical structure likely achieves the best accuracy, a computationally tractable heterogeneous anisotropic model provided significant improvement and may be applicable for image-guided breast surgeries.

Ultrasound-guided breast biopsy using an adapted automated cone-based ultrasound scanner: a feasibility study

BACKGROUND

Among available breast biopsy techniques, ultrasound (US)-guided biopsy is preferable because it is relatively inexpensive and provides live imaging feedback. The availability of magnetic resonance imaging (MRI)-3D US image fusion would facilitate US-guided biopsy even for US occult lesions to reduce the need for expensive and time-consuming MRI-guided biopsy. In this paper, we propose a novel Automated Cone-based Breast Ultrasound Scanning and Biopsy System (ACBUS-BS) to scan and biopsy breasts of women in prone position. It is based on a previously developed system, called ACBUS, that facilitates MRI-3D US image fusion imaging of the breast employing a conical container filled with coupling medium.

PURPOSE

The purpose of this study was to introduce the ABCUS-BS system and demonstrate its feasibility for biopsy of US occult lesions.

METHOD

The biopsy procedure with the ACBUS-BS comprises four steps: target localization, positioning, preparation, and biopsy. The biopsy outcome can be impacted by 5 types of errors: due to lesion segmentation, MRI-3D US registration, navigation, lesion tracking during repositioning, and US inaccuracy (due to sound speed difference between the sample and the one used for image reconstruction). For the quantification, we use a soft custom-made polyvinyl alcohol phantom (PVA) containing eight lesions (three US-occult and five US-visible lesions of 10 mm in diameter) and a commercial breast mimicking phantom with a median stiffness of 7.6 and 28 kPa, respectively. Errors of all types were quantified using the custom-made phantom. The error due to lesion tracking was also quantified with the commercial phantom. Finally, the technology was validated by biopsying the custom-made phantom and comparing the size of the biopsied material to the original lesion size. The average size of the 10-mm-sized lesions in the biopsy specimen was 7.00 ± 0.92 mm (6.33 ± 1.16 mm for US occult lesions, and 7.40 ± 0.55 mm for US-visible lesions).

RESULTS

For the PVA phantom, the errors due to registration, navigation, lesion tracking during repositioning, and US inaccuracy were 1.33, 0.30, 2.12, and 0.55 mm. The total error was 4.01 mm. For the commercial phantom, the error due to lesion tracking was estimated at 1.10 mm, and the total error was 4.11 mm. Given these results, the system is expected to successfully biopsy lesions larger than 8.22 mm in diameter. Patient studies will have to be carried out to confirm this in vivo.

CONCLUSION

The ACBUS-BS facilitates US-guided biopsy of lesions detected in pre-MRI and therefore might offer a low-cost alternative to MRI-guided biopsy. We demonstrated the feasibility of the approach by successfully taking biopsies of five US-visible and three US-occult lesions embedded in a soft breast-shaped phantom.

Multimodal Patient-Specific Registration for Breast Imaging Using Biomechanical Modeling with Reference to AI Evaluation of Breast Tumor Change

Background: The strategy to combat the problem associated with large deformations in the breast due to the difference in the medical imaging of patient posture plays a vital role in multimodal medical image registration with artificial intelligence (AI) initiatives. How to build a breast biomechanical model simulating the large-scale deformation of soft tissue remains a challenge but is highly desirable. Methods: This study proposed a hybrid individual-specific registration model of the breast combining finite element analysis, property optimization, and affine transformation to register breast images. During the registration process, the mechanical properties of the breast tissues were individually assigned using an optimization process, which allowed the model to become patient specific. Evaluation and results: The proposed method has been extensively tested on two datasets collected from two independent institutions, one from America and another from Hong Kong. Conclusions: Our method can accurately predict the deformation of breasts from the supine to prone position for both the Hong Kong and American samples, with a small target registration error of lesions.

The Biomechanics of the Fibrocystic Breasts at Finite Compressive Deformation

The deformation of the human breast, especially that of the female, under variable pressure conditions, has been a recent focus for researchers, both in the computational biomechanics, computational biology and the health sector. When the deformation of the breast is large, it hampers suitable cyst tracing as a mammographic biopsy precontrive data. Finite element methods (FEM) has been instrumental in the currently studied practices to trail nodules dislocation. However, the effect of breast material constitution, especially that of a fibrocystic composition, on the biomechanical response of these nodules has gained less attention. The present study is aimed at developing a finite element fibrocystic breast model within the frame of biosolid mechanics and material hyperelasticity to model the breast deformation at finite strain. The geometry of a healthy stress‐free breast is modelled from a magnetic resonance image (MRI) using tissues deformations measurements and solid modelling technology. Results show that the incompressible Neo-Hookean and Mooney-Rivlin constitutive models can approximate large deformation of a stressed breast. In addition to the areola (i.e. nipple base), the surrounding area of the cyst together with its interface with the breast tissue is the maximum stressed region when the breast is subjected to compressive pressure. This effect can lead to an internal tear of the breast that could degenerate to malignant tissue.

Integrated Biophysical Modeling and Image Analysis: Application to Neuro-Oncology

Central nervous system (CNS) tumors come with vastly heterogeneous histologic, molecular, and radiographic landscapes, rendering their precise characterization challenging. The rapidly growing fields of biophysical modeling and radiomics have shown promise in better characterizing the molecular, spatial, and temporal heterogeneity of tumors. Integrative analysis of CNS tumors, including clinically acquired multi-parametric magnetic resonance imaging (mpMRI) and the inverse problem of calibrating biophysical models to mpMRI data, assists in identifying macroscopic quantifiable tumor patterns of invasion and proliferation, potentially leading to improved (a) detection/segmentation of tumor subregions and (b) computer-aided diagnostic/prognostic/predictive modeling. This article presents a summary of (a) biophysical growth modeling and simulation,(b) inverse problems for model calibration, (c) these models' integration with imaging workflows, and (d) their application to clinically relevant studies. We anticipate that such quantitative integrative analysis may even be beneficial in a future revision of the World Health Organization (WHO) classification for CNS tumors, ultimately improving patient survival prospects.

Characterization of human female breast and abdominal skin elasticity using a bulge test

Characterization of material properties of human skin is required to develop a physics-based biomechanical model that can predict deformation of female breast after cosmetic and reconstructive surgery. In this paper, we have adopted an experimental approach to characterize the biaxial response of human skin using bulge tests. Skin specimens were harvested from breast and abdominal skin of female subjects who underwent mastectomy and/or reconstruction at The University of Texas MD Anderson Cancer Center and who provided informed consent. The specimens were tested within 2 h of harvest, and after freezing for different time periods but not exceeding 6 months. Our experimental results show that storage in a freezer at -20 °C for up to about 40 days does not lead to changes in the mechanical response of the skin beyond statistical variation. Moreover, displacement at the apex of the bulged specimen versus applied pressure varies significantly between different specimens from the same subject and from different subjects. The bulge test results were used in an inverse optimization procedure in order to calibrate two different constitutive material models - the angular integration model proposed by Lanir (1983) and the generalized structure tensor formulation of Gasser et al. (2006). The material parameters were estimated through a cost function that penalized deviations of the displacement and principal curvatures at the apex. Generally, acceptable fits were obtained with both models, although the angular integration model was able to fit the curvatures slightly better than the Gasser et al. model. The range of the model parameters has been extracted for use in physics-based biomechanical models of the breast.

An automated computational biomechanics workflow for improving breast cancer diagnosis and treatment

Clinicians face many challenges when diagnosing and treating breast cancer. These challenges include interpreting and co-locating information between different medical imaging modalities that are used to identify tumours and predicting where these tumours move to during different treatment procedures. We have developed a novel automated breast image analysis workflow that integrates state-of-the-art image processing and machine learning techniques, personalized three-dimensional biomechanical modelling and population-based statistical analysis to assist clinicians during breast cancer detection and treatment procedures. This paper summarizes our recent research to address the various technical and implementation challenges associated with creating a fully automated system. The workflow is applied to predict the repositioning of tumours from the prone position, where diagnostic magnetic resonance imaging is performed, to the supine position where treatment procedures are performed. We discuss our recent advances towards addressing challenges in identifying the mechanical properties of the breast and evaluating the accuracy of the biomechanical models. We also describe our progress in implementing a prototype of this workflow in clinical practice. Clinical adoption of these state-of-the-art modelling techniques has significant potential for reducing the number of misdiagnosed breast cancers, while also helping to improve the treatment of patients.

A biomechanical breast model evaluated with respect to MRI data collected in three different positions

BACKGROUND

Mammography is a specific type of breast imaging that uses low-dose X-rays to detect cancer in early stage. During the exam, the women breast is compressed between two plates in order to even out the breast thickness and to spread out the soft tissues. This technique improves exam quality but can be uncomfortable for the patient. The perceived discomfort can be assessed by the means of a breast biomechanical model. Alternative breast compression techniques may be computationally investigated trough finite elements simulations.

METHODS

The aim of this work is to develop and evaluate a new biomechanical Finite Element (FE) breast model. The complex breast anatomy is considered including adipose and glandular tissues, muscle, skin, suspensory ligaments and pectoral fascias. Material hyper-elasticity is modeled using the Neo-Hookean material models. The stress-free breast geometry and subject-specific constitutive models are derived using tissues deformations measurements from MR images.

FINDINGS

The breast geometry in three breast configurations were computed using the breast stress-free geometry together with the estimated set of equivalent Young's modulus (E_breast^r = 0.3 kPa, E_breast^l = 0.2 kPa, E_skin = 4 kPa, E_fascia = 120 kPa). The Hausdorff distance between estimated and measured breast geometries for prone, supine and supine tilted configurations is equal to 2.17 mm, 1.72 mm and 5.90 mm respectively.

INTERPRETATION

A subject-specific breast model allows a better characterization of breast mechanics. However, the model presents some limitations when estimating the supine tilted breast configuration. The results show clearly the difficulties to characterize soft tissues mechanics at large strain ranges with Neo-Hookean material models.

Iterative simulations to estimate the elastic properties from a series of MRI images followed by MRI-US validation

The modeling of breast deformations is of interest in medical applications such as image-guided biopsy, or image registration for diagnostic purposes. In order to have such information, it is needed to extract the mechanical properties of the tissues. In this work, we propose an iterative technique based on finite element analysis that estimates the elastic modulus of realistic breast phantoms, starting from MRI images acquired in different positions (prone and supine), when deformed only by the gravity force. We validated the method using both a single-modality evaluation in which we simulated the effect of the gravity force to generate four different configurations (prone, supine, lateral, and vertical) and a multi-modality evaluation in which we simulated a series of changes in orientation (prone to supine). Validation is performed, respectively, on surface points and lesions using as ground-truth data from MRI images, and on target lesions inside the breast phantom compared with the actual target segmented from the US image. The use of pre-operative images is limited at the moment to diagnostic purposes. By using our method we can compute patient-specific mechanical properties that allow compensating deformations. Graphical Abstract Workflow of the proposed method and comparative results of the prone-to-supine simulation (red volumes) validated using MRI data (blue volumes).

Analytical derivation of elasticity in breast phantoms for deformation tracking

PURPOSE

Patient-specific biomedical modeling of the breast is of interest for medical applications such as image registration, image guided procedures and the alignment for biopsy or surgery purposes. The computation of elastic properties is essential to simulate deformations in a realistic way. This study presents an innovative analytical method to compute the elastic modulus and evaluate the elasticity of a breast using magnetic resonance (MRI) images of breast phantoms.

METHODS

An analytical method for elasticity computation was developed and subsequently validated on a series of geometric shapes, and on four physical breast phantoms that are supported by a planar frame. This method can compute the elasticity of a shape directly from a set of MRI scans. For comparison, elasticity values were also computed numerically using two different simulation software packages.

RESULTS

Application of the different methods on the geometric shapes shows that the analytically derived elongation differs from simulated elongation by less than 9% for cylindrical shapes, and up to 18% for other shapes that are also substantially vertically supported by a planar base. For the four physical breast phantoms, the analytically derived elasticity differs from numeric elasticity by 18% on average, which is in accordance with the difference in elongation estimation for the geometric shapes. The analytic method has shown to be multiple orders of magnitude faster than the numerical methods.

CONCLUSION

It can be concluded that the analytical elasticity computation method has good potential to supplement or replace numerical elasticity simulations in gravity-induced deformations, for shapes that are substantially supported by a planar base perpendicular to the gravitational field. The error is manageable, while the calculation procedure takes less than one second as opposed to multiple minutes with numerical methods. The results will be used in the MRI and Ultrasound Robotic Assisted Biopsy (MURAB) project.

A Regression Model for Predicting Shape Deformation after Breast Conserving Surgery

Breast cancer treatments can have a negative impact on breast aesthetics, in case when surgery is intended to intersect tumor. For many years mastectomy was the only surgical option, but more recently breast conserving surgery (BCS) has been promoted as a liable alternative to treat cancer while preserving most part of the breast. However, there is still a significant number of BCS intervened patients who are unpleasant with the result of the treatment, which leads to self-image issues and emotional overloads. Surgeons recognize the value of a tool to predict the breast shape after BCS to facilitate surgeon/patient communication and allow more educated decisions; however, no such tool is available that is suited for clinical usage. These tools could serve as a way of visually sensing the aesthetic consequences of the treatment. In this research, it is intended to propose a methodology for predict the deformation after BCS by using machine learning techniques. Nonetheless, there is no appropriate dataset containing breast data before and after surgery in order to train a learning model. Therefore, an in-house semi-synthetic dataset is proposed to fulfill the requirement of this research. Using the proposed dataset, several learning methodologies were investigated, and promising outcomes are obtained.

Towards an in-plane methodology to track breast lesions using mammograms and patient-specific finite-element simulations

In breast cancer screening or diagnosis, it is usual to combine different images in order to locate a lesion as accurately as possible. These images are generated using a single or several imaging techniques. As x-ray-based mammography is widely used, a breast lesion is located in the same plane of the image (mammogram), but tracking it across mammograms corresponding to different views is a challenging task for medical physicians. Accordingly, simulation tools and methodologies that use patient-specific numerical models can facilitate the task of fusing information from different images. Additionally, these tools need to be as straightforward as possible to facilitate their translation to the clinical area. This paper presents a patient-specific, finite-element-based and semi-automated simulation methodology to track breast lesions across mammograms. A realistic three-dimensional computer model of a patient's breast was generated from magnetic resonance imaging to simulate mammographic compressions in cranio-caudal (CC, head-to-toe) and medio-lateral oblique (MLO, shoulder-to-opposite hip) directions. For each compression being simulated, a virtual mammogram was obtained and posteriorly superimposed to the corresponding real mammogram, by sharing the nipple as a common feature. Two-dimensional rigid-body transformations were applied, and the error distance measured between the centroids of the tumors previously located on each image was 3.84 mm and 2.41 mm for CC and MLO compression, respectively. Considering that the scope of this work is to conceive a methodology translatable to clinical practice, the results indicate that it could be helpful in supporting the tracking of breast lesions.

Robust MR image segmentation using the trimmed likelihood estimator in asymmetric Student's-t mixture model

Finite mixture model (FMM) has been widely used for unsupervised segmentation of magnetic resonance (MR) images in recent years. However, in real applications, the distribution of the observed data usually contains an unknown fraction of outliers, which would interfere with the estimation of the parameters of the mixture model. The statistical model-based technique which provides a theoretically well segmentation criterion in presence of outliers is the mixture modeling and the trimming approach. Therefore, in this paper, a robust estimation of asymmetric Student's-t mixture model (ASMM) using the trimmed likelihood estimator for MR image segmentation has been proposed. The proposed method is supposed to discard the outliers, and then to estimate the parameters of the ASMM with the remaining samples. The advantages of the proposed algorithm are that its robustness to dispose the disturbance of outliers and its flexibility to describe various shapes of data. Finally, expectation-maximization (EM) algorithm is adopted to maximize the log-likelihood and to obtain the estimation of the parameters. The experimental results show that the proposed method has a better performance on the segmentation of synthetic data and real MR images.

A hybrid patient-specific biomechanical model based image registration method for the motion estimation of lungs

This paper presents a new hybrid biomechanical model-based non-rigid image registration method for lung motion estimation. In the proposed method, a patient-specific biomechanical modelling process captures major physically realistic deformations with explicit physical modelling of sliding motion, whilst a subsequent non-rigid image registration process compensates for small residuals. The proposed algorithm was evaluated with 10 4D CT datasets of lung cancer patients. The target registration error (TRE), defined as the Euclidean distance of landmark pairs, was significantly lower with the proposed method (TRE = 1.37 mm) than with biomechanical modelling (TRE = 3.81 mm) and intensity-based image registration without specific considerations for sliding motion (TRE = 4.57 mm). The proposed method achieved a comparable accuracy as several recently developed intensity-based registration algorithms with sliding handling on the same datasets. A detailed comparison on the distributions of TREs with three non-rigid intensity-based algorithms showed that the proposed method performed especially well on estimating the displacement field of lung surface regions (mean TRE = 1.33 mm, maximum TRE = 5.3 mm). The effects of biomechanical model parameters (such as Poisson's ratio, friction and tissue heterogeneity) on displacement estimation were investigated. The potential of the algorithm in optimising biomechanical models of lungs through analysing the pattern of displacement compensation from the image registration process has also been demonstrated.

The feasibility of manual parameter tuning for deformable breast MR image registration from a multi-objective optimization perspective

Deformable image registration is typically formulated as an optimization problem involving a linearly weighted combination of terms that correspond to objectives of interest (e.g. similarity, deformation magnitude). The weights, along with multiple other parameters, need to be manually tuned for each application, a task currently addressed mainly via trial-and-error approaches. Such approaches can only be successful if there is a sensible interplay between parameters, objectives, and desired registration outcome. This, however, is not well established. To study this interplay, we use multi-objective optimization, where multiple solutions exist that represent the optimal trade-offs between the objectives, forming a so-called Pareto front. Here, we focus on weight tuning. To study the space a user has to navigate during manual weight tuning, we randomly sample multiple linear combinations. To understand how these combinations relate to desirability of registration outcome, we associate with each outcome a mean target registration error (TRE) based on expert-defined anatomical landmarks. Further, we employ a multi-objective evolutionary algorithm that optimizes the weight combinations, yielding a Pareto front of solutions, which can be directly navigated by the user. To study how the complexity of manual weight tuning changes depending on the registration problem, we consider an easy problem, prone-to-prone breast MR image registration, and a hard problem, prone-to-supine breast MR image registration. Lastly, we investigate how guidance information as an additional objective influences the prone-to-supine registration outcome. Results show that the interplay between weights, objectives, and registration outcome makes manual weight tuning feasible for the prone-to-prone problem, but very challenging for the harder prone-to-supine problem. Here, patient-specific, multi-objective weight optimization is needed, obtaining a mean TRE of 13.6 mm without guidance information reduced to 7.3 mm with guidance information, but also providing a Pareto front that exhibits an intuitively sensible interplay between weights, objectives, and registration outcome, allowing outcome selection.

3D volume reconstruction from serial breast specimen radiographs for mapping between histology and 3D whole specimen imaging

PURPOSE

In breast imaging, radiological in vivo images, such as x-ray mammography and magnetic resonance imaging (MRI), are used for tumor detection, diagnosis, and size determination. After excision, the specimen is typically sliced into slabs and a small subset is sampled. Histopathological imaging of the stained samples is used as the gold standard for characterization of the tumor microenvironment. A 3D volume reconstruction of the whole specimen from the 2D slabs could facilitate bridging the gap between histology and in vivo radiological imaging. This task is challenging, however, due to the large deformation that the breast tissue undergoes after surgery and the significant undersampling of the specimen obtained in histology. In this work, we present a method to reconstruct a coherent 3D volume from 2D digital radiographs of the specimen slabs.

METHODS

To reconstruct a 3D breast specimen volume, we propose the use of multiple target neighboring slices, when deforming each 2D slab radiograph in the volume, rather than performing pairwise registrations. The algorithm combines neighborhood slice information with free-form deformations, which enables a flexible, nonlinear deformation to be computed subject to the constraint that a coherent 3D volume is obtained. The neighborhood information provides adequate constraints, without the need for any additional regularization terms.

RESULTS

The volume reconstruction algorithm is validated on clinical mastectomy samples using a quantitative assessment of the volume reconstruction smoothness and a comparison with a whole specimen 3D image acquired for validation before slicing. Additionally, a target registration error of 5 mm (comparable to the specimen slab thickness of 4 mm) was obtained for five cases. The error was computed using manual annotations from four observers as gold standard, with interobserver variability of 3.4 mm. Finally, we illustrate how the reconstructed volumes can be used to map histology images to a 3D specimen image of the whole sample (either MRI or CT).

CONCLUSIONS

Qualitative and quantitative assessment has illustrated the benefit of using our proposed methodology to reconstruct a coherent specimen volume from serial slab radiographs. To our knowledge, this is the first method that has been applied to clinical breast cases, with the goal of reconstructing a whole specimen sample. The algorithm can be used as part of the pipeline of mapping histology images to ex vivo and ultimately in vivo radiological images of the breast.

Systematic feasibility analysis of a quantitative elasticity estimation for breast anatomy using supine/prone patient postures

PURPOSE

Breast elastography is a critical tool for improving the targeted radiotherapy treatment of breast tumors. Current breast radiotherapy imaging protocols only involve prone and supine CT scans. There is a lack of knowledge on the quantitative accuracy with which breast elasticity can be systematically measured using only prone and supine CT datasets. The purpose of this paper is to describe a quantitative elasticity estimation technique for breast anatomy using only these supine/prone patient postures. Using biomechanical, high-resolution breast geometry obtained from CT scans, a systematic assessment was performed in order to determine the feasibility of this methodology for clinically relevant elasticity distributions.

METHODS

A model-guided inverse analysis approach is presented in this paper. A graphics processing unit (GPU)-based linear elastic biomechanical model was employed as a forward model for the inverse analysis with the breast geometry in a prone position. The elasticity estimation was performed using a gradient-based iterative optimization scheme and a fast-simulated annealing (FSA) algorithm. Numerical studies were conducted to systematically analyze the feasibility of elasticity estimation. For simulating gravity-induced breast deformation, the breast geometry was anchored at its base, resembling the chest-wall/breast tissue interface. Ground-truth elasticity distributions were assigned to the model, representing tumor presence within breast tissue. Model geometry resolution was varied to estimate its influence on convergence of the system. A priori information was approximated and utilized to record the effect on time and accuracy of convergence. The role of the FSA process was also recorded. A novel error metric that combined elasticity and displacement error was used to quantify the systematic feasibility study. For the authors' purposes, convergence was set to be obtained when each voxel of tissue was within 1 mm of ground-truth deformation.

RESULTS

The authors' analyses showed that a ∼97% model convergence was systematically observed with no-a priori information. Varying the model geometry resolution showed no significant accuracy improvements. The GPU-based forward model enabled the inverse analysis to be completed within 10-70 min. Using a priori information about the underlying anatomy, the computation time decreased by as much as 50%, while accuracy improved from 96.81% to 98.26%. The use of FSA was observed to allow the iterative estimation methodology to converge more precisely.

CONCLUSIONS

By utilizing a forward iterative approach to solve the inverse elasticity problem, this work indicates the feasibility and potential of the fast reconstruction of breast tissue elasticity using supine/prone patient postures.

From clinical imaging and computational models to personalised medicine and image guided interventions

This short paper describes the development of the UCL Centre for Medical Image Computing (CMIC) from 2006 to 2016, together with reference to historical developments of the Computational Imaging sciences Group (CISG) at Guy's Hospital. Key early work in automated image registration led to developments in image guided surgery and improved cancer diagnosis and therapy. The work is illustrated with examples from neurosurgery, laparoscopic liver and gastric surgery, diagnosis and treatment of prostate cancer and breast cancer, and image guided radiotherapy for lung cancer.

A review of biomechanically informed breast image registration

Breast radiology encompasses the full range of imaging modalities from routine imaging via x-ray mammography, magnetic resonance imaging and ultrasound (both two- and three-dimensional), to more recent technologies such as digital breast tomosynthesis, and dedicated breast imaging systems for positron emission mammography and ultrasound tomography. In addition new and experimental modalities, such as Photoacoustics, Near Infrared Spectroscopy and Electrical Impedance Tomography etc, are emerging. The breast is a highly deformable structure however, and this greatly complicates visual comparison of imaging modalities for the purposes of breast screening, cancer diagnosis (including image guided biopsy), tumour staging, treatment monitoring, surgical planning and simulation of the effects of surgery and wound healing etc. Due primarily to the challenges posed by these gross, non-rigid deformations, development of automated methods which enable registration, and hence fusion, of information within and across breast imaging modalities, and between the images and the physical space of the breast during interventions, remains an active research field which has yet to translate suitable methods into clinical practice. This review describes current research in the field of breast biomechanical modelling and identifies relevant publications where the resulting models have been incorporated into breast image registration and simulation algorithms. Despite these developments there remain a number of issues that limit clinical application of biomechanical modelling. These include the accuracy of constitutive modelling, implementation of representative boundary conditions, failure to meet clinically acceptable levels of computational cost, challenges associated with automating patient-specific model generation (i.e. robust image segmentation and mesh generation) and the complexity of applying biomechanical modelling methods in routine clinical practice.

Symmetric Biomechanically Guided Prone-to-Supine Breast Image Registration

Prone-to-supine breast image registration has potential application in the fields of surgical and radiotherapy planning, image guided interventions, and multi-modal cancer diagnosis, staging, and therapy response prediction. However, breast image registration of three dimensional images acquired in different patient positions is a challenging problem, due to large deformations induced to the soft breast tissue caused by the change in gravity loading. We present a symmetric, biomechanical simulation based registration framework which aligns the images in a central, virtually unloaded configuration. The breast tissue is modelled as a neo-Hookean material and gravity is considered as the main source of deformation in the original images. In addition to gravity, our framework successively applies image derived forces directly into the unloading simulation in place of a subsequent image registration step. This results in a biomechanically constrained deformation. Using a finite difference scheme avoids an explicit meshing step and enables simulations to be performed directly in the image space. The explicit time integration scheme allows the motion at the interface between chest and breast to be constrained along the chest wall. The feasibility and accuracy of the approach presented here was assessed by measuring the target registration error (TRE) using a numerical phantom with known ground truth deformations, nine clinical prone MRI and supine CT image pairs, one clinical prone-supine CT image pair and four prone-supine MRI image pairs. The registration reduced the mean TRE for the numerical phantom experiment from initially 19.3 to 0.9 mm and the combined mean TRE for all fourteen clinical data sets from 69.7 to 5.6 mm.

Realization of a biomechanical model-assisted image guidance system for breast cancer surgery using supine MRI

PURPOSE

Unfortunately, the current re-excision rates for breast conserving surgeries due to positive margins average 20-40 %. The high re-excision rates arise from difficulty in localizing tumor boundaries intraoperatively and lack of real-time information on the presence of residual disease. The work presented here introduces the use of supine magnetic resonance (MR) images, digitization technology, and biomechanical models to investigate the capability of using an image guidance system to localize tumors intraoperatively.

METHODS

Preoperative supine MR images were used to create patient-specific biomechanical models of the breast tissue, chest wall, and tumor. In a mock intraoperative setup, a laser range scanner was used to digitize the breast surface and tracked ultrasound was used to digitize the chest wall and tumor. Rigid registration combined with a novel nonrigid registration routine was used to align the preoperative and intraoperative patient breast and tumor. The registration framework is driven by breast surface data (laser range scan of visible surface), ultrasound chest wall surface, and MR-visible fiducials. Tumor localizations by tracked ultrasound were only used to evaluate the fidelity of aligning preoperative MR tumor contours to physical patient space. The use of tracked ultrasound to digitize subsurface features to constrain our nonrigid registration approach and to assess the fidelity of our framework makes this work unique. Two patient subjects were analyzed as a preliminary investigation toward the realization of this supine image-guided approach.

RESULTS

An initial rigid registration was performed using adhesive MR-visible fiducial markers for two patients scheduled for a lumpectomy. For patient 1, the rigid registration resulted in a root-mean-square fiducial registration error (FRE) of 7.5 mm and the difference between the intraoperative tumor centroid as visualized with tracked ultrasound imaging and the registered preoperative MR counterpart was 6.5 mm. Nonrigid correction resulted in a decrease in FRE to 2.9 mm and tumor centroid difference to 5.5 mm. For patient 2, rigid registration resulted in a FRE of 8.8 mm and a 3D tumor centroid difference of 12.5 mm. Following nonrigid correction for patient 2, the FRE was reduced to 7.4 mm and the 3D tumor centroid difference was reduced to 5.3 mm.

CONCLUSION

Using our prototype image-guided surgery platform, we were able to align intraoperative data with preoperative patient-specific models with clinically relevant accuracy; i.e., tumor centroid localizations of approximately 5.3-5.5 mm.

An Inexact Newton-Krylov Algorithm for Constrained Diffeomorphic Image Registration

We propose numerical algorithms for solving large deformation diffeomorphic image registration problems. We formulate the nonrigid image registration problem as a problem of optimal control. This leads to an infinite-dimensional partial differential equation (PDE) constrained optimization problem. The PDE constraint consists, in its simplest form, of a hyperbolic transport equation for the evolution of the image intensity. The control variable is the velocity field. Tikhonov regularization on the control ensures well-posedness. We consider standard smoothness regularization based on H¹- or H²-seminorms. We augment this regularization scheme with a constraint on the divergence of the velocity field (control variable) rendering the deformation incompressible (Stokes regularization scheme) and thus ensuring that the determinant of the deformation gradient is equal to one, up to the numerical error. We use a Fourier pseudospectral discretization in space and a Chebyshev pseudospectral discretization in time. The latter allows us to reduce the number of unknowns and enables the time-adaptive inversion for nonstationary velocity fields. We use a preconditioned, globalized, matrix-free, inexact Newton-Krylov method for numerical optimization. A parameter continuation is designed to estimate an optimal regularization parameter. Regularity is ensured by controlling the geometric properties of the deformation field. Overall, we arrive at a black-box solver that exploits computational tools that are precisely tailored for solving the optimality system. We study spectral properties of the Hessian, grid convergence, numerical accuracy, computational efficiency, and deformation regularity of our scheme. We compare the designed Newton-Krylov methods with a globalized Picard method (preconditioned gradient descent). We study the influence of a varying number of unknowns in time. The reported results demonstrate excellent numerical accuracy, guaranteed local deformation regularity, and computational efficiency with an optional control on local mass conservation. The Newton-Krylov methods clearly outperform the Picard method if high accuracy of the inversion is required. Our method provides equally good results for stationary and nonstationary velocity fields for two-image registration problems.