Registering preprocedure volumetric images with intraprocedure 3-D ultrasound using an ultrasound imaging model

An improved Fourier Ptychography algorithm for ultrasonic array imaging

Inspired by the optical imaging algorithm, the Fourier Ptychography (FP) algorithm is adopted to improve the resolution of ultrasonic array imaging. In the FP algorithm, the steady-state spectrum is utilized to recover the high-resolution ultrasonic images. Meanwhile, the parameters of FP algorithm are empirical, which can affect the imaging quality of ultrasonic array. Then the particle swarm optimization (PSO) algorithm is used to optimize the parameters of FP algorithm to further improve the imaging quality of ultrasonic array. The tungsten imaging experiments and pig eye imaging experiments are conducted to demonstrate the feasibility and effectiveness of the developed algorithm. In addition, the proposed algorithm and the coherent wave superposition (CWS) algorithm are both based on single plane wave (SPW) algorithms and they are then compared. The results show that the CWS algorithm and FP algorithm have good longitudinal and lateral resolutions, respectively. The particle swarm optimization-based FP (PSOFP) imaging algorithm has both excellent lateral and longitudinal resolutions. The average lateral resolution of PSOFP imaging algorithm is improved by 34.47% compared with CWS imaging algorithm in the tungsten wires experiments, and the lateral boundary structure width of the lens is improved by 49.48% in the pig eye experiments. The proposed algorithm can effectively improve the ultrasonic imaging quality for medical application.

Self-Supervised Ultrasound to MRI Fetal Brain Image Synthesis

Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for second-trimester anomaly screening, for which ultrasound (US) is employed. Although expert sonographers are adept at reading US images, MR images which closely resemble anatomical images are much easier for non-experts to interpret. Thus in this article we propose to generate MR-like images directly from clinical US images. In medical image analysis such a capability is potentially useful as well, for instance for automatic US-MRI registration and fusion. The proposed model is end-to-end trainable and self-supervised without any external annotations. Specifically, based on an assumption that the US and MRI data share a similar anatomical latent space, we first utilise a network to extract the shared latent features, which are then used for MRI synthesis. Since paired data is unavailable for our study (and rare in practice), pixel-level constraints are infeasible to apply. We instead propose to enforce the distributions to be statistically indistinguishable, by adversarial learning in both the image domain and feature space. To regularise the anatomical structures between US and MRI during synthesis, we further propose an adversarial structural constraint. A new cross-modal attention technique is proposed to utilise non-local spatial information, by encouraging multi-modal knowledge fusion and propagation. We extend the approach to consider the case where 3D auxiliary information (e.g., 3D neighbours and a 3D location index) from volumetric data is also available, and show that this improves image synthesis. The proposed approach is evaluated quantitatively and qualitatively with comparison to real fetal MR images and other approaches to synthesis, demonstrating its feasibility of synthesising realistic MR images.

Ultrasonographic changes in the liver tumors as indicators of adequate tumor coverage with electric field for effective electrochemotherapy

Background The aim of the study was to characterize ultrasonographic (US) findings during and after electrochem-otherapy of liver tumors to determine the actual ablation zone and to verify the coverage of the treated tumor with a sufficiently strong electric field for effective electrochemotherapy. Patients and methods US findings from two representative patients that describe immediate and delayed tumor changes after electrochemotherapy of colorectal liver metastases are presented. Results The US findings were interrelated with magnetic resonance imaging (MRI). Electrochemotherapy-treated tumors were exposed to electric pulses based on computational treatment planning. The US findings indicate immediate appearance of hyperechogenic microbubbles along the electrode tracks. Within minutes, the tumors became evenly hyperechogenic, and simultaneously, an oedematous rim was formed presenting as a hypoechogenic formation which persisted for several hours after treatment. The US findings overlapped with computed electric field distribution in the treated tissue, indicating adequate coverage of tumors with sufficiently strong electric field, which may predict an effective treatment outcome. Conclusions US provides a tool for assessment of appropriate electrode insertion for intraoperative electrochemo-therapy of liver tumors and assessment of the appropriate coverage of a tumor with a sufficiently strong electric field and can serve as predictor of the response of tumors.

Phase-aware echocardiogram stabilization using keyframes

This paper presents an echocardiogram stabilization method designed to compensate for unwanted auxilliary motion. Echocardiograms contain both deformable cardiac motion and approximately rigid motion due to a number of factors. The goal of this work is to stabilize the video, while preserving the informative deformable cardiac motion. Our approach incorporates synchronized side information, extracted from electrocardiography (ECG), which provides a proxy for cardiac phase. To avoid the computational expense of pairwise alignment, we propose an efficient strategy for keyframe selection, formulated as a submodular optimization problem. We evaluate our approach quantitatively on synthetic data and demonstrate its benefit as a preprocessing step for two common echocardiogram applications: denoising and left ventricle segmentation. In both cases, preprocessing with our method improved the performance compared to no preprocessing or other alignment approaches.

Development and validation of real-time simulation of X-ray imaging with respiratory motion

We present a framework that combines evolutionary optimisation, soft tissue modelling and ray tracing on GPU to simultaneously compute the respiratory motion and X-ray imaging in real-time. Our aim is to provide validated building blocks with high fidelity to closely match both the human physiology and the physics of X-rays. A CPU-based set of algorithms is presented to model organ behaviours during respiration. Soft tissue deformation is computed with an extension of the Chain Mail method. Rigid elements move according to kinematic laws. A GPU-based surface rendering method is proposed to compute the X-ray image using the Beer-Lambert law. It is provided as an open-source library. A quantitative validation study is provided to objectively assess the accuracy of both components: (i) the respiration against anatomical data, and (ii) the X-ray against the Beer-Lambert law and the results of Monte Carlo simulations. Our implementation can be used in various applications, such as interactive medical virtual environment to train percutaneous transhepatic cholangiography in interventional radiology, 2D/3D registration, computation of digitally reconstructed radiograph, simulation of 4D sinograms to test tomography reconstruction tools.

Automatic ultrasound-MRI registration for neurosurgery using the 2D and 3D LC(2) Metric

To enable image guided neurosurgery, the alignment of pre-interventional magnetic resonance imaging (MRI) and intra-operative ultrasound (US) is commonly required. We present two automatic image registration algorithms using the similarity measure Linear Correlation of Linear Combination (LC(2)) to align either freehand US slices or US volumes with MRI images. Both approaches allow an automatic and robust registration, while the three dimensional method yields a significantly improved percentage of optimally aligned registrations for randomly chosen clinically relevant initializations. This study presents a detailed description of the methodology and an extensive evaluation showing an accuracy of 2.51mm, precision of 0.85mm and capture range of 15mm (>95% convergence) using 14 clinical neurosurgical cases.

MRI-3D ultrasound-X-ray image fusion with electromagnetic tracking for transendocardial therapeutic injections: in-vitro validation and in-vivo feasibility

Myocardial infarction (MI) is one of the leading causes of death in the world. Small animal studies have shown that stem-cell therapy offers dramatic functional improvement post-MI. An endomyocardial catheter injection approach to therapeutic agent delivery has been proposed to improve efficacy through increased cell retention. Accurate targeting is critical for reaching areas of greatest therapeutic potential while avoiding a life-threatening myocardial perforation. Multimodal image fusion has been proposed as a way to improve these procedures by augmenting traditional intra-operative imaging modalities with high resolution pre-procedural images. Previous approaches have suffered from a lack of real-time tissue imaging and dependence on X-ray imaging to track devices, leading to increased ionizing radiation dose. In this paper, we present a new image fusion system for catheter-based targeted delivery of therapeutic agents. The system registers real-time 3D echocardiography, magnetic resonance, X-ray, and electromagnetic sensor tracking within a single flexible framework. All system calibrations and registrations were validated and found to have target registration errors less than 5 mm in the worst case. Injection accuracy was validated in a motion enabled cardiac injection phantom, where targeting accuracy ranged from 0.57 to 3.81 mm. Clinical feasibility was demonstrated with in-vivo swine experiments, where injections were successfully made into targeted regions of the heart.

Global registration of ultrasound to MRI using the LC2 metric for enabling neurosurgical guidance

Automatic and robust registration of pre-operative magnetic resonance imaging (MRI) and intra-operative ultrasound (US) is essential to neurosurgery. We reformulate and extend an approach which uses a Linear Correlation of Linear Combination (LC2)-based similarity metric, yielding a novel algorithm which allows for fully automatic US-MRI registration in the matter of seconds. It is invariant with respect to the unknown and locally varying relationship between US image intensities and both MRI intensity and its gradient. The overall method based on this both recovers global rigid alignment, as well as the parameters of a free-form-deformation (FFD) model. The algorithm is evaluated on 14 clinical neurosurgical cases with tumors, with an average landmark-based error of 2.52 mm for the rigid transformation. In addition, we systematically study the accuracy, precision, and capture range of the algorithm, as well as its sensitivity to different choices of parameters.

A learning-based approach for fast and robust vessel tracking in long ultrasound sequences

We propose a learning-based method for robust tracking in long ultrasound sequences for image guidance applications. The framework is based on a scale-adaptive block-matching and temporal realignment driven by the image appearance learned from an initial training phase. The latter is introduced to avoid error accumulation over long sequences. The vessel tracking performance is assessed on long 2D ultrasound sequences of the liver of 9 volunteers under free breathing. We achieve a mean tracking accuracy of 0.96 mm. Without learning, the error increases significantly (2.19 mm, p<0.001).

Medical image registration: a review

This paper presents a review of automated image registration methodologies that have been used in the medical field. The aim of this paper is to be an introduction to the field, provide knowledge on the work that has been developed and to be a suitable reference for those who are looking for registration methods for a specific application. The registration methodologies under review are classified into intensity or feature based. The main steps of these methodologies, the common geometric transformations, the similarity measures and accuracy assessment techniques are introduced and described.

Registration of 3D trans-esophageal echocardiography to X-ray fluoroscopy using image-based probe tracking

Two-dimensional (2D) X-ray imaging is the dominant imaging modality for cardiac interventions. However, the use of X-ray fluoroscopy alone is inadequate for the guidance of procedures that require soft-tissue information, for example, the treatment of structural heart disease. The recent availability of three-dimensional (3D) trans-esophageal echocardiography (TEE) provides cardiologists with real-time 3D imaging of cardiac anatomy. Increasingly X-ray imaging is now supported by using intra-procedure 3D TEE imaging. We hypothesize that the real-time co-registration and visualization of 3D TEE and X-ray fluoroscopy data will provide a powerful guidance tool for cardiologists. In this paper, we propose a novel, robust and efficient method for performing this registration. The major advantage of our method is that it does not rely on any additional tracking hardware and therefore can be deployed straightforwardly into any interventional laboratory. Our method consists of an image-based TEE probe localization algorithm and a calibration procedure. While the calibration needs to be done only once, the GPU-accelerated registration takes approximately from 2 to 15s to complete depending on the number of X-ray images used in the registration and the image resolution. The accuracy of our method was assessed using a realistic heart phantom. The target registration error (TRE) for the heart phantom was less than 2mm. In addition, we assess the accuracy and the clinical feasibility of our method using five patient datasets, two of which were acquired from cardiac electrophysiology procedures and three from trans-catheter aortic valve implantation procedures. The registration results showed our technique had mean registration errors of 1.5-4.2mm and 95% capture range of 8.7-11.4mm in terms of TRE.

Keep breathing! Common motion helps multi-modal mapping

We propose an unconventional approach for transferring of information between multi-modal images. It exploits the temporal commonality of multi-modal images acquired from the same organ during free-breathing. Strikingly there is no need for capturing the same region by the modalities. The method is based on extracting a low-dimensional description of the image sequences, selecting the common cause signal (breathing) for both modalities and finding the most similar sub-sequences for predicting image feature location. The approach was evaluated for 3 volunteers on sequences of 2D MRI and 2D US images of the liver acquired at different locations. Simultaneous acquisition of these images allowed for quantitative evaluation (predicted versus ground truth MRI feature locations). The best performance was achieved with signal extraction by slow feature analysis resulting in an average error of 2.6 mm (4.2 mm) for sequences acquired at the same (a different) time.