Semiautomatic registration of volumetric ultrasound scans

Unsupervised Deep Learning Network with Self-Attention Mechanism for Non-Rigid Registration of 3D Brain MR Images

In non-rigid registration for medical imaging analysis, computation is complicated, and the high accuracy and robustness needed for registration are difficult to obtain. Recently, many studies have been conducted for nonrigid registration via unsupervised learning networks. This study proposes a method to improve the performance of this unsupervised learning network approach, through the use of a self-attention mechanism. In this paper, the self-attention mechanism is combined with deep learning networks to identify information of higher importance, among large amounts of data, and thereby solve specific tasks. Furthermore, the proposed method extracts both local and non-local information so that the network can create feature vectors with more information. As a result, the limitation of the existing network is addressed: alignment based solely on the entire silhouette of the brain is mitigated in favor of a network which also learns to perform registration of the parts of the brain that have internal structural characteristics. To the best of our knowledge, this is the first such utilization of the attention mechanism in this unsupervised learning network for non-rigid registration. The proposed attention network performs registration that takes into account the overall characteristics of the data, thus yielding more accurate matching results than those of the existing methods. In particular, matching is achieved with especially high accuracy in the gray matter and cortical ventricle areas, since these areas contain many of the structural features of the brain. The experiment was performed on 3D magnetic resonance images of the brains of 50 people. The measured average dice similarity coefficient after registration was 70.40%, which is an improvement of 17.48% compared to that before registration. This improvement indicates that application of the attention block can further improve the performance by an additional 8.5%, as relative to that without attention block. Ultimately, through implementation of non-rigid registration via the attention block method, the internal structure and overall shape of the brain can be addressed, without additional data input. Additionally, attention blocks have the advantage of being able to easily connect to existing networks without a significant computational overhead. Furthermore, by producing an attention map, the area of the brain around which registration was more performed can be visualized. This approach can be used for non-rigid registration with various types of medical imaging data.

M3VR-A multi-stage, multi-resolution, and multi-volumes-of-interest volume registration method applied to 3D endovaginal ultrasound

Heterogeneity of echo-texture and lack of sharply delineated tissue boundaries in diagnostic ultrasound images make three-dimensional (3D) registration challenging, especially when the volumes to be registered are considerably different due to local changes. We implemented a novel computational method that optimally registers volumetric ultrasound image data containing significant and local anatomical differences. It is A Multi-stage, Multi-resolution, and Multi-volumes-of-interest Volume Registration Method. A single region registration is optimized first for a close initial alignment to avoid convergence to a locally optimal solution. Multiple sub-volumes of interest can then be selected as target alignment regions to achieve confident consistency across the volume. Finally, a multi-resolution rigid registration is performed on these sub-volumes associated with different weights in the cost function. We applied the method on 3D endovaginal ultrasound image data acquired from patients during biopsy procedure of the pelvic floor muscle. Systematic assessment of our proposed method through cross validation demonstrated its accuracy and robustness. The algorithm can also be applied on medical imaging data of other modalities for which the traditional rigid registration methods would fail.

Automated Breast Ultrasound: Dual-Sided Compared with Single-Sided Imaging

The design and performance of a mammographically configured, dual-sided, automated breast ultrasound (ABUS) 3-D imaging system are described. Dual-sided imaging (superior and inferior) is compared with single-sided imaging to aid decisions on clinical implementation of the more complex, but potentially higher-quality dual-sided imaging. Marked improvement in image quality and coverage of the breast is obtained in dual-sided ultrasound over single-sided ultrasound. Among hypo-echoic masses imaged, there are increases in the mean contrast-to-noise ratio of 57% and 79%, respectively, for spliced dual-sided versus superior or inferior single-sided imaging. The fractional breast volume coverage, defined as the percentage volume in the transducer field of view that is imaged with clinically acceptable quality, is improved from 59% in both superior and inferior single-sided imaging to 89% in dual-sided imaging. Applying acoustic coupling to the breast requires more effort or sophisticated methods in dual-sided imaging than in single-sided imaging.

3D Breast Image Registration — A Review

Image registration is an important problem in breast imaging. It is used in a wide variety of applications that include better visualization of lesions on pre- and post-contrast breast MRI images, speckle tracking and image compounding in breast ultrasound images, alignment of positron emission, and standard mammography images on hybrid machines et cetera. It is a prerequisite to align images taken at different times to isolate small interval lesions. Image registration also has useful applications in monitoring cancer therapy. The field of breast image registration has gained considerable interest in recent years. While the primary focus of interest continues to be the registration of pre- and post-contrast breast MRI images, other areas like breast ultrasound registration have gained more attention in recent years. The focus of registration algorithms has also shifted from control point based semiautomated techniques, to more sophisticated voxel based automated techniques that use mutual information as a similarity measure. This paper visits the problem of breast image registration and provides an overview of the current state-of-the-art in this area.

Temporal compounding: a novel implementation and its impact on quality and diagnostic value in echocardiography

Temporal compounding can be used to suppress acoustic noise in transthoracic cardiac ultrasound by spatially averaging partially decorrelated images acquired over consecutive cardiac cycles. However, the reliable spatial and temporal alignment of the corresponding frames in consecutive cardiac cycles is vital for effective implementation of temporal compounding. This study introduces a novel, efficient, accurate and robust technique for the spatiotemporal alignment of consecutive cardiac cycles with variable temporal characteristics. Furthermore, optimal acquisition parameters, such as the number of consecutive cardiac cycles used, are derived. The effect of the proposed implementation of temporal compounding on cardiac ultrasound images is quantitatively assessed (32 clinical data sets providing a representative range of image qualities and diagnostic values) using measures such as tissue signal-to-noise ratio, chamber signal-to-noise ratio, tissue/chamber contrast and detectability index, as well as a range of clinical measurements, such as chamber diameter and wall thickness, performed during routine echocardiographic examinations. Temporal compounding (as implemented) consistently improved the image quality and diagnostic value of the processed images, when compared with the original data by: (i) increasing tissue and cavity signal-to-noise ratios as well as tissue/cavity detectability index, (ii) improving the corresponding clinical measurement repeatability and inter-operator measurement agreement, while (iii) reducing the number of omitted measurements caused by data corruption.

Machine learning and radiology

In this paper, we give a short introduction to machine learning and survey its applications in radiology. We focused on six categories of applications in radiology: medical image segmentation, registration, computer aided detection and diagnosis, brain function or activity analysis and neurological disease diagnosis from fMR images, content-based image retrieval systems for CT or MRI images, and text analysis of radiology reports using natural language processing (NLP) and natural language understanding (NLU). This survey shows that machine learning plays a key role in many radiology applications. Machine learning identifies complex patterns automatically and helps radiologists make intelligent decisions on radiology data such as conventional radiographs, CT, MRI, and PET images and radiology reports. In many applications, the performance of machine learning-based automatic detection and diagnosis systems has shown to be comparable to that of a well-trained and experienced radiologist. Technology development in machine learning and radiology will benefit from each other in the long run. Key contributions and common characteristics of machine learning techniques in radiology are discussed. We also discuss the problem of translating machine learning applications to the radiology clinical setting, including advantages and potential barriers.

Medical image registration: a review

This paper presents a review of automated image registration methodologies that have been used in the medical field. The aim of this paper is to be an introduction to the field, provide knowledge on the work that has been developed and to be a suitable reference for those who are looking for registration methods for a specific application. The registration methodologies under review are classified into intensity or feature based. The main steps of these methodologies, the common geometric transformations, the similarity measures and accuracy assessment techniques are introduced and described.

The ultrasound brain helmet: new transducers and volume registration for in vivo simultaneous multi-transducer 3-D transcranial imaging

Because stroke remains an important and time-sensitive health concern in developed nations, we present a system capable of fusing 3-D transcranial ultrasound volumes acquired from two sides of the head. This system uses custom sparse array transducers built on flexible multilayer circuits that can be positioned for simultaneous imaging through both temporal acoustic windows, allowing for potential registration of multiple real-time 3-D scans of cerebral vasculature. We examine hardware considerations for new matrix arrays-transducer design and interconnects-in this application. Specifically, it is proposed that SNR may be increased by reducing the length of probe cables. This claim is evaluated as part of the presented system through simulation, experimental data, and in vivo imaging. Ultimately, gains in SNR of 7 dB are realized by replacing a standard probe cable with a much shorter flex interconnect; higher gains may be possible using ribbon-based probe cables. In vivo images are presented, showing cerebral arteries with and without the use of microbubble contrast agent; they have been registered and fused using a simple algorithm which maximizes normalized cross-correlation.

Spatial registration of temporally separated whole breast 3D ultrasound images

The purpose of this study was to evaluate the potential for use of image volume based registration (IVBaR) to aid in measurement of changes in the tumor during chemotherapy of breast cancer. Successful IVBaR could aid in the detection of such changes in response to neoadjuvant chemotherapy and potentially be useful for routine breast cancer screening and diagnosis. IVBaR was employed in a new method of automated estimation of tumor volume in studies following the radiologist identification of the tumor region in the prechemotherapy scan. The authors have also introduced a new semiautomated method for validation of registration based on Doppler ultrasound (U.S.) signals that are independent of the grayscale signals used for registration. This Institutional Review Board approved study was conducted on 10 patients undergoing chemotherapy and 14 patients with a suspicious/unknown mass scheduled to undergo biopsy. Reasonably reproducible mammographic positioning and nearly whole breast U.S. imaging were achieved. The image volume was registered offline with a mutual information cost function and global interpolation based on a thin-plate spline using MIAMI FUSE software developed at the University of Michigan. The success and accuracy of registration of the three dimensional (3D) U.S. image volume were measured by means of mean registration error (MRE). IVBaR was successful with MRE of 4.3 +/- 1.7 mm in 9 out of 10 reproducibility automated breast ultrasound (ABU) studies and in 12 out of 17 ABU image pairs collected before, during, or after 115 +/- 14 days of chemotherapy. Semiautomated tumor volume estimation was performed on registered image volumes giving 86 +/- 8% mean accuracy compared to the radiologist hand-segmented tumor volume on seven cases. Doppler studies yielded fractional volume of color pixels in the region surrounding the lesion and its change with changing breast compression. The Doppler study of patients with detectable blood flow included five patients with suspicious masses and three undergoing chemotherapy. Spatial alignment of the 3D blood vessel data from the Doppler studies provided independent measures for the validation of registration. In 15 Doppler image volume pairs scanned with differing breast compression, the mean centerline separation value was 1.5 +/- 0.6 mm, while MRE based on a few identifiable structural points common to the two grayscale image volumes was 1.1 +/- 0.6 mm. Another measure, the overlap ratio of blood vessels, was shown to increase from 0.32 to 0.59 (+84%) with IVBaR for pairs at various compression levels. These results show that successful registration of ABU scans may be accomplished for comparison and integration of information.

A real-time intrasubject elastic registration algorithm for dynamic 2-D ultrasound images

A new dynamic registration technique is designed to elastically align a sequence of 2-D ultrasound (US) images. The proposed algorithm tracks features over an image sequence in real-time, as opposed to our previous technique which registers images without utilizing prior information. The registration results were evaluated using a customized validation framework for elastic US registration algorithms. Experiments were performed on 600 simulated images as well as 20 image sequences obtained from 10 volunteer subjects, each sequence containing 50 images. Strategies for qualitative and quantitative evaluation consisted of visual assessment, feature overlap, similarity measures, and simulation experiments. The registration method has comparable performance to our previous registration technique; however, has the advantage of lower computational requirements and hence, is potentially more suitable for clinical applications. Rigorous performance evaluations attest to the fast speed of registration at an average of 5.5 frames per second on a conventional computing platform.

Reconstruction of volumetric ultrasound panorama based on improved 3D SIFT

Registration of ultrasound volumes is a key issue for the reconstruction of volumetric ultrasound panorama. In this paper, we propose an improved three-dimensional (3D) scale invariant feature transform (SIFT) algorithm to globally register ultrasound volumes acquired from dedicated ultrasound probe, where local deformations are corrected by block-based warping algorithm. Original SIFT algorithm is extended to 3D and improved by combining the SIFT detector with Rohr3D detector to extract complementary features and applying the diffusion distance algorithm for robust feature comparison. Extensive experiments have been performed on both phantom and clinical data sets to demonstrate the effectiveness and robustness of our approach.

Quantitative ultrasound backscatter for pulsed cavitational ultrasound therapy- histotripsy

Histotripsy is a well-controlled ultrasonic tissue ablation technology that mechanically and progressively fractionates tissue structures using cavitation. The fractionated tissue volume can be monitored with ultrasound imaging because a significant ultrasound backscatter reduction occurs.This paper correlates the ultrasound backscatter reduction with the degree of tissue fractionation characterized by the percentage of remaining normal-appearing cell nuclei on histology.Different degrees of tissue fractionation were generated in vitro in freshly excised porcine kidneys by varying the number of therapeutic ultrasound pulses from 100 to 2000 pulses per treatment location. All ultrasound pulses were 15 cycles at 1 MHz delivered at 100 Hz pulse repetition frequency and 19 MPa peak negative pressure. The results showed that the normalized backscatter intensity decreased exponentially with increasing number of pulses. Correspondingly, the percentage of normal appearing nuclei in the treated area decreased exponentially as well. A linear correlation existed between the normalized backscatter intensity and the percentage of normal appearing cell nuclei in the treated region. This suggests that the normalized backscatter intensity may be a potential quantitative real-time feedback parameter for histotripsy-induced tissue fractionation. This quantitative feedback may allow the prediction of local clinical outcomes, i.e., when a tissue volume has been sufficiently treated.

Breast ultrasound image improvement by pixel compounding of compression sequence

Pixel compounding is a technique that synthesizes the information of an image sequence involving slow decorrelation of the speckle to form a detail-recovered and speckle reduced image. To avoid extra data acquisition time and patient exposure, reuse of the existing data is desirable. In the procedure of elasticity imaging, a set of B-mode images with slight changes due to deformation is produced, which provides an ideal input for the pixel compounding. The improvement in image quality is evaluated quantitatively using a figure-of-merit (FOM) that indicates the quality of boundary information recovery and the contrast-to-noise ratio (CNR) over the phantom images. The increase in average CNR is from 0.4 in the original images to 0.8 in the pixel compounded images. The improvement in average FOM is from 0.15 to more than 0.5 on a scale of 0 to 1. In vivo results with a breast cyst, a fibroadenoma, and a breast cancer1 are also presented and the image quality improvement is subjectively evaluated. The results suggest that B-mode breast images from compression procedures are suitable data for pixel compounding, and that a speckle-reduced and detail-recovered or detail-maintained image can be produced. The improved imaging may provide alternative or better information for detection and diagnosis. A similar approach could be extended to elasticity imaging with other modalities.

Image registration for detection and quantification of change on digital tomosynthesis mammographic volumes

OBJECTIVE

The purpose of this study was to achieve 3D registration of digital tomosynthesis mammographic volumes using mutual information.

CONCLUSION

Registration of digital breast tomosynthesis mammographic volumes was achieved with an average error of 1.8 +/- 1.4 mm.

Mutual-information-based image to patient re-registration using intraoperative ultrasound in image-guided neurosurgery

An image-based re-registration scheme has been developed and evaluated that uses fiducial registration as a starting point to maximize the normalized mutual information (nMI) between intraoperative ultrasound (iUS) and preoperative magnetic resonance images (pMR). We show that this scheme significantly (p<0.001) reduces tumor boundary misalignment between iUS pre-durotomy and pMR from an average of 2.5 mm to 1.0 mm in six resection surgeries. The corrected tumor alignment before dural opening provides a more accurate reference for assessing subsequent intraoperative tumor displacement, which is important for brain shift compensation as surgery progresses. In addition, we report the translational and rotational capture ranges necessary for successful convergence of the nMI registration technique (5.9 mm and 5.2 deg, respectively). The proposed scheme is automatic, sufficiently robust, and computationally efficient (<2 min), and holds promise for routine clinical use in the operating room during image-guided neurosurgical procedures.

Sparse registration for three-dimensional stress echocardiography

Three-dimensional (3-D) stress echocardiography is a novel technique for diagnosing cardiac dysfunction. It involves evaluating wall motion of the left ventricle, by visually analyzing ultrasound images obtained in rest and in different stages of stress. Since the acquisitions are performed minutes apart, variabilities may exist in the visualized cross-sections. To improve anatomical correspondence between rest and stress, aligning the images is essential. We developed a new intensity-based, sparse registration method to retrieve standard anatomical views from 3-D stress images that were equivalent to the manually selected views in the rest images. Using sparse image planes, the influence of common image artifacts could be reduced. We investigated different similarity measures and different levels of sparsity. The registration was tested using data of 20 patients and quantitatively evaluated based on manually defined anatomical landmarks. Alignment was best using sparse registration with two long-axis and two short-axis views; registration errors were reduced significantly, to the range of interobserver variabilities. In 91% of the cases, the registration result was qualitatively assessed as better than or equal to the manual alignment. In conclusion, sparse registration improves the alignment of rest and stress images, with a performance similar to manual alignment. This is an important step towards objective quantification in 3-D stress echocardiography.

A "twisting and bending" model-based nonrigid image registration technique for 3-D ultrasound carotid images

Atherosclerosis at the carotid bifurcation resulting in cerebral emboli is a major cause of ischemic stroke. Most strokes associated with carotid atherosclerosis can be prevented by lifestyle/dietary changes and pharmacological treatments if identified early by monitoring carotid plaque changes. Registration of 3-D ultrasound (US) images of carotid plaque obtained at different time points is essential for sensitive monitoring of plaque changes in volume and surface morphology. This registration technique should be nonrigid, since different head positions during image acquisition sessions cause relative bending and torsion in the neck, producing nonlinear deformations between the images. We modeled the movement of the neck using a "twisting and bending" model with only six parameters for nonrigid registration. We evaluated the algorithm using 3-D US carotid images acquired at two different head positions to simulate images acquired at different times. We calculated the mean registration error (MRE) between the segmented vessel surfaces in the target image and the registered image using a distance-based error metric after applying our "twisting and bending" model-based nonrigid registration algorithm. We achieved an average registration error of 0.80 +/-0.26 mm using our nonrigid registration technique, which was a significant improvement in registration accuracy over rigid registration, even with reduced degrees-of-freedom compared to the other nonrigid registration algorithms.

Non-rigid registration of three-dimensional (3D) grayscale and Doppler ultrasound breast images

The purpose of this study is to evaluate the accuracy of image volume based registration (IVBaR) of 3D ultrasound (US) image volumes of the whole breast acquired at different times. Successful IVBaR could aid in detection of tumor changes in response to neoadjuvant chemotherapy and potentially be useful for routine breast cancer screening and diagnosis. IVBaR was successful in 9 of 10 reproducibility studies, 11 of 15 image pairs collected before and after approximately 45 days of chemotherapy. Doppler study yielded volume of blood flow to the region surrounding the lesion and its change when reducing breast compression. The color flow vessels provided independent measures for validation of registration of the grayscale portion of those images.

Experimental validation of a 4D elastic registration algorithm

This paper presents an extensive validation study of an elastic registration algorithm for dynamic 3D ultrasound images (also known as a 4D image). The registration algorithm uses attribute vectors from both a fixed and previous moving images to perform feature-based alignment of a series of images. The 4D method reduces computational requirements and increases the effective search space for the location of corresponding features, resulting in enhanced registration speed when compared to a static 3D registration technique. Experimental analysis revealed up to 32% improvement in speed when using the 4D method, which makes the algorithm attractive for real-time applications.

Towards efficient registration of medical images

In this paper we propose a Bayesian based mutual information technique for image registration, combined with an established affine transformation model. Classical affine models allow the images to be approximately aligned. However, inefficiency and inaccuracy has appeared when using these affine models in rigorous circumstances, such as low-resolution images. To challenge this problem, we conduct mutual information measures with importance sampling to the images in an attempt to simulate the probability distribution of intensity similarity across the images. The entire registration adopts a stopping criterion as discovered in the context of differential equations. Finally, experimental results demonstrate the favorable performance of the proposed algorithm.

Breast image registration techniques: a survey

Breast cancer is the most common type of cancer in women worldwide. Image registration plays an important role in breast cancer detection. This paper gives an overview of the current state-of-the-art in the breast image registration techniques. For the intramodality registration techniques, X-ray, MRI, and ultrasound are the primary focuses of interest. Intermodality techniques will cover the combination of different modalities. Validation of breast registration methods is also discussed.

Many infants prenatally exposed to high levels of alcohol show one particular anomaly of the corpus callosum

INTRODUCTION

Effects of prenatal alcohol exposure on the brain are seen at every age. The earlier they can be quantified, the better the prognosis for the affected child. Here we show measurable alcohol effects at birth on a structure currently used for nosology only much later in life.

METHODS

Midline shape of the corpus callosum was imaged in infants via averaged unwarped transfontanelle ultrasound. We compared measures of these shapes among 23 infants prenatally exposed to high levels of alcohol and 21 infants unexposed to alcohol or only lightly exposed.

RESULTS

A particular feature of the corpus callosum, the appearance of a "hook" (obtuse angle) between the splenium and the long diameter of the arch in this plane, is strongly associated with prenatal alcohol exposure. In half of the high-exposed infants, the splenium angle is larger than in any of the unexposed brains. Simply characterizing this angle as less than or greater than 90 degrees detects 12 of the 23 exposed infants as anomalous with only 1 false positive among the unexposed. This apparently direct effect of prenatal ethanol exposure on the details of the callosum in about half the at-risk subjects cannot be attributed to any of several plausible competing exposures or other confounding factors applying during or after gestation.

CONCLUSION

An average of the images for the unexposed subjects has the geometry of textbook images of normal babies; but the average for the subgroup of high-angle subjects may serve as a template or guide to this regional damage parallel to the familiar photographic exemplars that help to assess facial signs.

Registration of ultrasonic cardiac images

We describe an efficient approach to the registration problem of ultrasonic cardiac images using an affine transformation model, incorporating a mutual information based error minimisation. The affine motion model provides an analytic solution which has fast and stable convergence. The geometric and intensity constancy constraint is combined with the smoothness constraint, which allow us to have approximate image alignment. Due to complexity of a realistic motion, we then minimise the error in intensity between the registered source and the target image in the context of mutual information. In fact, image registration can be reached when the amount of information buried in the images is maximised. The experimental results demonstrate that this hybrid framework is highly efficient in registering different cardiac images.

A system for ultrasound-guided computer-assisted orthopaedic surgery

Current computer-assisted orthopedic surgery (CAOS) systems typically use preoperative computed tomography (CT) and intraoperative fluoroscopy as their imaging modalities. Because these imaging tools use X-rays, both patients and surgeons are exposed to ionizing radiation that may cause long-term health damage. To register the patient with the preoperative surgical plan, these techniques require tracking of the targeted anatomy by invasively mounting a tracking device on the patient, which results in extra pain and may prolong recovery time. The mounting procedure also leads to a major difficulty of using these approaches to track small bones or mobile fractures. Furthermore, it is practically impossible to mount a heavy tracking device on a small bone, which thus restricts the use of CAOS techniques. This article presents a novel CAOS method that employs 2D ultrasound (US) as the imaging modality. Medical US is non-ionizing and real-time, and our proposed method does not require any invasive mounting procedures. Experiments have shown that the proposed registration technique has sub-millimetric accuracy in localizing the best match between the intraoperative and preoperative images, demonstrating great potential for orthopedic applications. This method has some significant advantages over previously reported US-guided CAOS techniques: it requires no segmentation and employs only a few US images to accurately and robustly localize the patient. Preliminary laboratory results on both a radius-bone phantom and human subjects are presented.

Three-dimensional extended field-of-view ultrasound

Three-dimensional (3-D) extended field-of-view ultrasound creates a mosaic view from a set of volumes acquired from a dedicated 3-D ultrasound machine combined with a position tracker. A simple compounding technique can be used to combine the volumes together using only the position measurements, but some misalignment remains. Two different registration methods were developed to correct these errors in the overlapping regions. The first method divides the overlap into smaller blocks and warps the blocks to best align the features. The second method is similar, but uses rigid body registration of the blocks. Experiments in vitro and in vivo showed that block-based registration with warping produced the most reproducible results and the greatest increase in similarity among the overlapping regions. It also produced the best reconstruction accuracy, with a mean distance error of 0.4 mm measured across 101.78 mm in a phantom, representing 0.4% error.

Evaluation on Similarity Measures of a Surface-to-Image Registration Technique for Ultrasound Images

Ultrasound is a universal guidance tool for many medical procedures, whereas it is of poor image quality and resolution. Merging high-contrast image information from other image modalities enhances the guidance capability of ultrasound. However, few registration methods work well for it. In this paper we present a surface-to-image registration technique for mono- or multimodal medical data concerning ultrasound. This approach is able to automatically register the object surface to its counterpart in image volume. Three similarity measurements are investigated in the rigid registration experiments of the pubic arch in transrectal ultrasound images. It shown that the selection of the similarity function is related to the ultrasound characteristics of the object to be registered.

Preliminary evidence that prenatal alcohol damage may be visible in averaged ultrasound images of the neonatal human corpus callosum

Brain damage consequent to prenatal alcohol exposure can be detected by measurements of the corpus callosum in the midline magnetic resonance (MR) brain image in adolescents and adults. The present article extends this finding into the neonatal period, when the power of detection to ameliorate the quality of the child's future life is greatest. The midline corpus callosum of the very young infant can be located reliably in multiple frames of clinical transfontanelle ultrasound. We studied a sample of 18 children aged 17 weeks or less, 7 of whom were exposed to high levels of alcohol prenatally and 11 of whom were not exposed or only minimally exposed. The midline callosum of each child was imaged up to 50 times by a standard clinical device, and coplanar subsets of these series were averaged with reference to fiducial image structures. On each average image four semilandmark points were set and their configuration quantified by standard landmark methods. The angle between the terminal bulb of splenium and the long axis of the callosal outline classifies four of the seven exposed infants as different from all 11 of the unexposed infants. This simple angle measurement upon averaged ultrasound images of the human neonatal midline corpus callosum, perhaps a version of the long-sought "biomarker of prenatal alcohol damage," may be able to discriminate baby brains affected by prenatal alcohol exposure from those that were unaffected.

Processing and visualizing three-dimensional ultrasound data

This paper describes techniques for the visualization and processing of three-dimensional (3D) ultrasound data. The nature of such data demands specialized algorithms, which differ from those employed for other medical imaging modalities. In this paper, the emphasis is placed on generic processing techniques, which are relevant across a wide range of 3D ultrasound application domains.

F-information measures in medical image registration

A measure for registration of medical images that currently draws much attention is mutual information. The measure originates from information theory, but has been shown to be successful for image registration as well. Information theory, however, offers many more measures that may be suitable for image registration. These all measure the divergence of the joint distribution of the images' grey values from the joint distribution that would have been found had the images been completely independent. This paper compares the performance of mutual information as a registration measure with that of other F-information measures. The measures are applied to rigid registration of positron emission tomography (PET)/magnetic resonance (MR) and MR/computed tomography (CT) images, for 35 and 41 image pairs, respectively. An accurate gold standard transformation is available for the images, based on implanted markers. The registration performance, robustness and accuracy of the measures are studied. Some of the measures are shown to perform poorly on all aspects. The majority of measures produces results similar to those of mutual information. An important finding, however, is that several measures, although slightly more difficult to optimize, can potentially yield significantly more accurate results than mutual information.

Adaptive registration using local information measures

Rapidly advancing registration methods increasingly employ warping transforms. High degrees of freedom (DOF) warpings can be specified by manually placing control points or instantiating a regular, dense grid of control points everywhere. The former approach is laborious and prone to operator bias, whereas the latter is computationally expensive. We propose to improve upon the latter approach by adaptively placing control points where they are needed. Local estimates of mutual information (MI) and entropy are used to identify local regions requiring additional DOF.

Registration of real-time 3-D ultrasound images of the heart for novel 3-D stress echocardiography

Stress echocardiography is a routinely used clinical procedure to diagnose cardiac dysfunction by comparing wall motion information in prestress and poststress ultrasound images. Incomplete data, complicated imaging protocols and misaligned prestress and poststress views, however, are known limitations of conventional stress echocardiography. We discuss how the first two limitations are overcome via the use of real-time three-dimensional (3-D) ultrasound imaging, an emerging modality, and have called the new procedure "3-D stress echocardiography." We also show that the problem of misaligned views can be solved by registration of prestress and poststress 3-D image sequences. Such images are misaligned because of variations in placing the ultrasound transducer and stress-induced anatomical changes. We have developed a technique to temporally align 3-D images of the two sequences first and then to spatially register them to rectify probe placement error while preserving the stress-induced changes. The 3-D spatial registration is mutual information-based. Image registration used in conjunction with 3-D stress echocardiography can potentially improve the diagnostic accuracy of stress testing.

High-Speed Registration of Three- and Four-dimensional Medical Images by Using Voxel Similarity

A generalized, accurate, automatic, retrospective method of image registration for three-dimensional images has been developed. The method is based on mutual information, a specific measure of voxel similarity, and is applicable to a wide range of imaging modalities and organs, rigid or deformable. A drawback of mutual information-based image registration is long execution times. To overcome the speed problem, low-cost, customized hardware to accelerate this computationally intensive task was developed. Individual hardware accelerator units (each, in principle, 25-fold faster than a comparable software implementation) can be concatenated to perform image registration at any user-desired speed. A first-generation prototype board with two processing units provided a 12- to 16-fold increase in speed. Enhancements for increasing the speed further are being developed. These advances have enabled many nontraditional applications of image registration and have made the traditional applications more efficient. Clinical applications include fusion of computed tomographic (CT), magnetic resonance, and positron emission tomographic (PET) images of the brain; fusion of whole-body CT and PET images; fusion of four-dimensional spatiotemporal ultrasonographic (US) and single photon emission CT images of the heart; and correction of misalignment between pre- and poststress four-dimensional US images.

Rapid registration for wide field of view freehand three-dimensional ultrasound

A freehand scanning protocol is the only way to acquire arbitrary large volumes of three-dimensional ultrasound (US) data. For some applications, multiple freehand sweeps are required to cover the area of interest. Aligning these multiple sweeps is difficult, typically requiring nonrigid image-based registration as well as the readings from the spatial locator attached to the US probe. Conventionally, nonrigid warps are achieved through general elastic spline deformations, which are expensive to compute and difficult to constrain. This paper presents an alternative registration technique, where the warp's degrees of freedom are carefully linked to the mechanics of the freehand scanning process. The technique is assessed through an extensive series of in vivo experiments, which reveal a registration precision of a few pixels with comparatively little computational load.

Mutual-information-based registration of medical images: a survey

An overview is presented of the medical image processing literature on mutual-information-based registration. The aim of the survey is threefold: an introduction for those new to the field, an overview for those working in the field, and a reference for those searching for literature on a specific application. Methods are classified according to the different aspects of mutual-information-based registration. The main division is in aspects of the methodology and of the application. The part on methodology describes choices made on facets such as preprocessing of images, gray value interpolation, optimization, adaptations to the mutual information measure, and different types of geometrical transformations. The part on applications is a reference of the literature available on different modalities, on interpatient registration and on different anatomical objects. Comparison studies including mutual information are also considered. The paper starts with a description of entropy and mutual information and it closes with a discussion on past achievements and some future challenges.

A new approach to elastography using mutual information and finite elements

Historically, increased mechanical stiffness during tissue palpation exams has been associated with assessing organ health as well as with detecting the growth of a potentially life-threatening cell mass. As such, techniques to image elasticity parameters (i.e., elastography) have recently become of great interest to scientists. In this work, a new method of elastography will be introduced within the context of mammographic imaging. The elastography method proposed represents a non-rigid iterative image registration algorithm that varies material properties within a finite element model to improve registration. More specifically, regional measures of image similarity are used within an objective function minimization framework to reconstruct elasticity images of tissue stiffness. Numerical simulations illustrate: (1) the encoding of stiffness information within the context of a regional image similarity criterion, (2) the methodology for an iterative elastographic imaging framework and (3) elasticity reconstruction simulations. The real strength in this approach is that images from any modality (e.g., magnetic resonance, computed tomography, ultrasound. etc) that have sufficient anatomically-based intensity heterogeneity and remain consistent from a pre- to a post-deformed state could be used in this paradigm.

Rapid elastic image registration for 3-D ultrasound

A Subvolume-based algorithm for elastic Ultrasound REgistration (SURE) was developed and evaluated. Designed primarily to improve spatial resolution in three-dimensional compound imaging, the algorithm registers individual image volumes nonlinearly before combination into compound volumes. SURE works in one or two stages, optionally using MIAMI Fuse software first to determine a global affine registration before iteratively dividing the volume into subvolumes and computing local rigid registrations in the second stage. Connectivity of the entire volume is ensured by global interpolation using thin-plate splines after each iteration. The performance of SURE was quantified in 20 synthetically deformed in vivo ultrasound volumes, and in two phantom scans, one of which was distorted at acquisition by placing an aberrating layer in the sound path. The aberrating layer was designed to induce beam aberrations reported for the female breast. Synthetic deformations of 1.5-2.5 mm were reduced by over 85% when SURE was applied to register the distorted image volumes with the original ones. Registration times were below 5 min on a 500-MHz CPU for an average data set size of 13 MB. In the aberrated phantom scans, SURE reduced the average deformation between the two volumes from 1.01 to 0.30 mm. This was a statistically significant (P = 0.01) improvement over rigid and affine registration transformations, which produced reductions to 0.59 and 0.50 mm, respectively.

Differentiation of breast masses using 3-D sonographic and echo-enhancer-based evaluation of the vascular pattern: initial experiences

To evaluate the potential of combined 3-D B-mode and color Doppler (CD) data sets in the differentiation of breast masses, in 50 patients with histologically proven solid breast lesions, 3-D datasets were acquired. A 3-D display was created and volume calculation of tumors, their periphery and vasculature was performed. Time-intensity curves of enhancement after administration of a contrast agent were analyzed. Volumetry of tumor vasculature yielded no significant differences between malignant and benign tumors regarding vascularization of the center (2.60 vs. 2.88%) and periphery (6.66 vs. 3.78%). Only the mean values for the rise time in the center of the tumor, fibroadenoma (FA): 5.7 s and ductal invasive carcinoma (DIC): 15.8s; p = 0.05, and the time to peak in the periphery, FA: 21.0 s and DIC: 31.6 s; p = 0.03, differed significantly. The 3-D ultrasound (US) technique was of no additional value in differentiating breast masses. The calculation of time-intensity curves after administration of a contrast agent may be helpful in differentiating FA and DIC.

Ultrasound imaging in lower limb prosthetics

The biomechanical interaction between the residual limb and the prosthetic socket determines the quality of fit of the socket in lower limb prosthetics. An understanding of this interaction and the development of quantitative measures to predict the quality of fit of the socket are important for optimal socket design. Finite-element modeling is used widely for biomechanical modeling of the limb/socket interaction and requires information on the internal and external geometry of the residual limb. Volumetric imaging methods such as X-ray computed tomography, magnetic resonance imaging, and ultrasound have been used to obtain residual limb shape information. Of these modalities, ultrasound has been introduced most recently and its development for visualization in prosthetics is the least mature. This paper reviews ultrasound image acquisition and processing methods as they have been applied in lower limb prosthetics.

Mutual information-based rigid and nonrigid registration of ultrasound volumes

We investigated the registration of ultrasound volumes based on the mutual information measure, a technique originally applied to multimodality registration of brain images. A prerequisite for successful registration is a smooth, quasi-convex mutual information surface with an unambiguous maximum. We discuss the necessary preprocessing to create such a surface for ultrasound volumes. Abdominal and thoracic organs imaged with ultrasound typically move relative to the exterior of the body and are deformable. Consequently, four specific instances of image registration involving progressively generalized transformations were studied: rigid-body, rigid-body + uniform scaling, rigid-body + nonuniform scaling, and affine. Registration was applied to clinically acquired volumetric images. The accuracy was comparable with the voxel dimension for all transformation modes, although it degraded as the transformation grew more complex. Likewise, the capture range became narrower with the complexity of transformation. As the use of real-time three-dimensional ultrasound becomes more prevalent, the method we present should work well for a variety of applications examining serial anatomic and physiologic changes. Developers of these clinical applications would match the deformation model of their problem to one of the four transformation models presented here.

Evaluation of voxel-based registration of 3-D power Doppler ultrasound and 3-D magnetic resonance angiographic images of carotid arteries

Spatial registration and fusion of ultrasound (US) images with other modalities may aid clinical interpretation. We implemented and evaluated on patient data an automated retrospective registration of magnetic resonance angiography (MRA) carotid bifurcation images with 3-D power Doppler ultrasound (PD US) and indirectly with 3-D B-mode US. Volumes were initially thresholded to reduce the uncorrelated noise signals. The registration algorithm subsequently maximized the mutual information measure between the PD US and 3-D MRA via iterative simplex search to find best "rigid body" transformation. We rated the performance of the algorithm visually on (n = 5) clinical MRA and 3-D PD US datasets. We also evaluated quantitatively the effect of thresholding, initial misalignment of the paired volumes and the reproducibility registration. We investigated the effect of image artefacts by simulation experiments. Preregistration misalignments of up to 5 mm in the transaxial plane, up to 10 mm along the axis of the carotids and up to 40 degrees resulted in 107 of 110 successful registrations, with translational and rotational errors of 0.32 mm +/- 0.3 mm and 1.6 +/- 2.1 degrees. The algorithm was not affected by missing arterial segments of up to 8 mm in length. The average registration time was 4 min. We conclude that the algorithm could be applied to 3-D US PD and MRA data for automated multimodality registration of carotid vessels without the use of fiducials.

Serial measurement of cross-sectional area in peripheral vein grafts using three-dimensional ultrasound

Frequent surveillance of bypass grafts placed in the lower limbs can provide early detection of stenoses. A three-dimensional (3-D) ultrasound (US) imaging system has been used to produce serial surface reconstructions of regions of interest in vein grafts in the lower extremities. Using anatomical reference points, data sets from serial studies are registered in a common 3-D coordinate system. Cross-sectional area measurements are extracted from the surface reconstructions in planes normal to the vessel center axis. These measurements are compared at matched sites over time to track changes in the vessel configuration. The quantitative measurements are paired with surface displays of the vessels for a complete depiction of the changing geometry. Example studies from three patients are shown, for time periods up to 38 weeks. The cross-sectional area measurements highlight regions of remodeling and developing stenoses within the grafts.

3D spatial compounding of ultrasound images using image-based nonrigid registration

Medical ultrasound images are often distorted enough to significantly limit resolution during compounding (i.e., summation of images from multiple views). A new, volumetric image registration technique has been used successfully to enable high spatial resolution in three-dimensional (3D) spatial compounding of ultrasound images. Volumetric ultrasound data were acquired by scanning a linear matrix array probe in the elevational direction in a focal lesion phantom and in a breast in vivo. To obtain partly uncorrelated views, the volume of interest was scanned at five different transducer tilt angles separated by 4 degrees to 6 degrees. Pairs of separate views were registered by an automatic procedure based on a mutual information metric, using global full affine and thin-plate spline warping transformations. Registration accuracy was analyzed automatically in the phantom data, and manually in vivo, yielding average registration errors of 0.31 mm and 0.65 mm, respectively. In the vicinity of the warping control points, registrations obtained with warping transformations were significantly more accurate than full affine registrations. Compounded images displayed the expected reduction in speckle noise and increase in contrast-to-noise ratio (CNR), as well as better delineation of connective tissues and reduced shadowing. Compounding also revealed some apparent low contrast lobulations that were not visible in the single-sweep images. Given expected algorithmic and hardware enhancements, nonrigid, image-based registration shows great promise for reducing tissue motion and refraction artifacts in 3D spatial compounding.