Interactive 3-D registration of ultrasound and magnetic resonance images based on a magnetic position sensor

A localization method for wireless capsule endoscopy using side wall cameras and IMU sensor

Localizing the endoscopy capsule inside gastrointestinal (GI) system provides key information which leads to GI abnormality tracking and precision medical delivery. In this paper, we have proposed a new method to localize the capsule inside human GI track. We propose to equip the capsule with four side wall cameras and an Inertial Measurement Unit (IMU), that consists of 9 Degree-Of-Freedom (DOF) including a gyroscope, an accelerometer and a magnetometer to monitor the capsule’s orientation and direction of travel. The low resolution mono-chromatic cameras, installed along the wide wall, are responsible to measure the actual capsule movement, not the involuntary motion of the small intestine. Finally, a fusion algorithm is used to combine all data to derive the traveled path and plot the trajectory. Compared to other methods, the presented system is resistive to surrounding conditions, such as GI nonhomogeneous structure and involuntary small bowel movements. In addition, it does not require external antenna or arrays. Therefore, GI tracking can be achieved without disturbing patients’ daily activities.

EFFICIENT MAGNETIC LOCALIZATION AND ORIENTATION TECHNIQUE FOR CAPSULE ENDOSCOPY

To build a new wireless robotic capsule endoscope with external guidance for controllable and interactive GI tract examination, a sensing system is needed for tracking 3D location and 2D orientation of the capsule endoscope movement. An appropriate sensing method is to enclose a small permanent magnet in the capsule. The intensities of the magnetic field produced by the magnet in different spatial points can be measured by the magnetic sensors outside the patient's body. With the sensing data of magnetic sensor array, the 3D location and 2D orientation of the capsule can be calculated. Higher calculation accuracy can be obtained if more sensors and optimal algorithm are applied. In this paper, different nonlinear optimization algorithms were evaluated to solve the magnet's 5D parameters, e.g. Powell's, Downhill Simplex, DIRECT, Multilevel Coordinate Search, and Levenberg Marquardt method. We have found that Levenberg-Marquardt method provides satisfactory calculation accuracy and faster speed. Simulations were done for investigating the de-noise ability of this algorithm based on different sensor arrays. Also the real experiment shows that the results are satisfactory with high accuracy (average localization error is 5.6 mm).

Biplane US-Guided Real-Time Volumetric Target Pose Estimation Method for Theragnostic HIFU System

This paper presents a real-time pose estimation method as a part of robotic HIFU treatment system for moving volumetric targets. For the acquired biplane US images, current pose of the preoperative model is calculated by iterative segmentation and registration. Seed contours for the segmentation in each iteration is provided by previously registered preoperative 3-D model. The segmented boundary points then update the pose of 3-D model. The boundary outlier-removal makes the algorithm robust against partially noisy boundaries as well as the spatial boundary points accelerates the algorithm to be calculated in real-time. By the phantom experiments, registration accuracy for a biplane US image data was evaluated, and the processing time was also investigated.

Adaptive spatial calibration of a 3D ultrasound system

PURPOSE

The authors present a method devised to calibrate the spatial relationship between a 3D ultrasound scanhead and its tracker completely automatically and reliably. The user interaction is limited to collecting ultrasound data on which the calibration is based.

METHODS

The method of calibration is based on images of a fixed plane of unknown location with respect to the 3D tracking system. This approach has, for advantage, to eliminate the measurement of the plane location as a source of error. The devised method is sufficiently general and adaptable to calibrate scanheads for 2D images and 3D volume sets using the same approach. The basic algorithm for both types of scanheads is the same and can be run unattended fully automatically once the data are collected. The approach was devised by seeking the simplest and most robust solutions for each of the steps required. These are the identification of the plane intersection within the images or volumes and the optimization method used to compute a calibration transformation matrix. The authors use adaptive algorithms in these two steps to eliminate data that would otherwise prevent the convergence of the procedure, which contributes to the robustness of the method.

RESULTS

The authors have run tests amounting to 57 runs of the calibration on two a scanhead that produce 3D imaging volumes, at all the available scales. The authors evaluated the system on two criteria: Robustness and accuracy. The program converged to useful values unattended for every one of the tests (100%). Its accuracy, based on the measured location of a reference plane, was estimated to be 0.7 +/- 0.6 mm for all tests combined.

CONCLUSIONS

The system presented is robust and allows unattended computations of the calibration parameters required for freehand tracked ultrasound based on either 2D or 3D imaging systems.

Computer-assisted needle positioning for liver tumour radiofrequency ablation (RFA)

BACKGROUND

The RFA procedures rely on a precise positioning of the radiofrequency electrode and the complete destruction of the tumour. This article presents new optimization techniques to improve such surgical procedures.

METHODS

A method to optimize the coverage of the tumour by successive RFA destructions and an in vitro procedure with simulated tumours have been developed.

RESULTS

The guidance system and optimization coverage have been tested on 3D simulation and by the surgeon in vitro on a heifer liver. In this context, the RFA electrode is optically tracked and guided.

CONCLUSIONS

The optimization method provides needle placements that ensure a complete theoretical ablation of the tumour, and the guidance system helps the surgeon to reach each position of destruction.

A new method for registration-based medical image interpolation

A new technique is presented for interpolating between grey-scale images in a medical data set. Registration between neighboring slices is achieved with a modified control grid interpolation algorithm that selectively accepts displacement field updates in a manner optimized for performance. A cubic interpolator is then applied to pixel intensities correlated by the displacement fields. Special considerations are made for efficiency, interpolation quality, and compression in the implementation of the algorithm. Experimental results show that the new method achieves good quality, while offering dramatic improvement in efficiency relative to the best competing method.

A neurosurgical navigation system based on intraoperative tumour remnant estimation

This paper proposes a method to intra-operatively visualize the process of tumour resection until complete resection is accomplished. A fuzzy connectedness method that is robust against image noises was used to identify the tumour position and volume. Based on the tumour segmentation results, the removed area and the residual tumour tissues were examined with reference to the electrocautery trace log. Unique processes that are specific to glioma resection were introduced in the method to improve the accuracy of estimation. Invalid data in the trace log were excluded, and the tumour region surrounded by valid log points was included in the removed area. The proposed system also produces an alarm to indicate whether the electrocautery is being accurately performed within the tumour area. Thus, this surgical navigation system can assist surgeons in intuitively monitoring the tumour resection process and properly removing tumour remnants. Preliminary experiments and a clinical pilot study showed the feasible application of this method.

A method to register intra-treatment ultrasound images to pre-treatment images of prostate

This paper describes a method to register ultrasound images (US) to pre-treatment images. The aim of the work is the information transfer between the pre-treatment imaging modality (MR or CT) and the intra-treatment imaging (US). Ultrasound images are spatially tracked by a stereo-vision system and the prostate boundaries are automatically extracted using a method that combines morphological and adaptive speckle suppression and a priori knowledge. MR/CT images are merged to construct a volume of pelvis using fuzzy logic algorithm and an MPR virtual slice corresponding to the orientation of the US image is generated from the volume. The prostate is segmented from the slice by a model-based method and rigidly registered by ICP algorithm to the US contour. Preliminary experiences gave satisfactory results with short computing time.

Interventional navigation for abdominal therapy based on simultaneous use of MRI and ultrasound

An interventional navigation system designed for percutaneous abdominal therapies was proposed, and a pilot study was carried out to assess the proposed system. Integration of US to MRI-based segmentation and 3D display of tumours can help physicians deal with instabilities such as respiratory motion and soft tissue shift that are inherent in abdominal interventions. In addition to the 3D display of the needle and tumours, we adapted the system for the abdominal applications and incorporated a process to correct the mismatch in needle path between MRI and US. The preliminary results of phantom and animal experiments indicated that the proposed method could combine the advantages of both MRI and US. The time required to determine the optimal needle insertion path by using this system was significantly less than that required when either US or MRI guidance alone was employed. The developed system was applied in two patients who underwent PEIT therapy, and its clinical feasibility was partially confirmed.

Rigid overlay of volume sonography and MR image data of the female pelvic floor using a fiducial based alignment—feasibility due to a case series

The visual combination of different medical image acquisition techniques (modalities) can lead to new modalities with enhanced informative content. In this paper, we present an overlay technique of magnetic resonance (MR) and 3D US image data sets of the female anal canal (internal and external sphincter) as a base for a new diagnostic modality. It is a new field of the application of the overlay technique. Three corresponding MR and US volume data sets from the female pelvic floor region were filtered using adaptive filtering techniques and overlayed (=registered rigidly) with a landmark based alignment method.

Virtual fetal touch through a haptic interface decreases maternal anxiety and salivary cortisol

OBJECTIVE

To evaluate whether a virtual reality workstation (Fetouch system) offering three-dimensional (3D) fetal visual and kinesthetic interaction may affect maternal stress.

METHODS

Maternal-fetal visual and kinesthetic interaction was obtained through a haptic interface based on 3D reconstruction of sequencial bi-dimensional ultrasound images of the fetus. Maternal stress was assessed before and after visual/kinesthetic interaction with the fetus: 1) by using the State Trait Anxiety Inventory-Form Y (STAI) test, and 2) by measuring salivary cortisol levels. Statistical analysis was performed by paired t test and analysis of variance for repeated measures.

RESULTS

After the fetal visual and kinesthetic experiences, a significant reduction was observed in anxiety (low state anxiety group, P < .0034; high state anxiety group, P < .0108), as well as in salivary cortisol concentration (P < .0004).

CONCLUSION

Physical interaction with the fetus through a 3D model may reduce maternal stress.

Redundant system of passive markers for ultrasound scanhead tracking

Scanhead tracking by opto-electronic (OE) systems allows high accuracy in three-dimensional (3-D) freehand ultrasound imaging. In this paper, a new set of methods is proposed and compared with the standard approach [Gram-Schmidt method (GS)]. Three redundancy-based algorithms are introduced to compensate for possible loss of markers during data acquisition: regression plane (RP), multiple Gram-Schmidt (MGS), and center of mass least square (CMLS). When combined with the ultrasound instrument, the root-mean-squared (RMS) uncertainty in locating target points, over a working volume of 420 mm x 490 mm x 100 mm, improved by 7% and 24% using MGS and CMLS method respectively, compared to GS. A lower improvement was obtained with RP methods (5%), using the best marker configuration. In conclusion, CMLS method provides a robust and accurate procedure for 3-D freehand ultrasound scanhead tracking, able to manage possible loss of markers, with interesting perspectives for image fusion and body referenced 3-D ultrasound.

Ultrasound-to-computer-tomography registration for image-guided laparoscopic liver surgery

BACKGROUND

The application of image-guided surgery (IGS) to laparoscopic liver resection and ablation is currently limited, but it would assist in intraoperative decision making regarding oncologic margins, ablation probe placement, and ablation tracking.

METHODS

Eight spherical surface targets on a liver phantom were imaged with an optically tracked laparoscopic ultrasound (US) probe. Ten US images of each target were registered to computer tomography (CT) images of the phantoms and then mapped to the CT scans. Accuracy of the registration was assessed by comparing the distance between the predicted target location and the position obtained directly from CT.

RESULTS

The average localization error was 5.3 mm. The errors resulted primarily from inaccurate US probe tracking but were otherwise insensitive to the variability that arises from manually identifying targets in US and CT images.

CONCLUSIONS

The results obtained for US-to-CT registration in a phantom model suggest that further investigations into its clinical use are warranted and that other IGS technologies could be applied to laparoscopic liver surgery as well.

Elastic registration of 3D ultrasound images

3D registration of ultrasound images is an important and fast-growing research area with various medical applications, such as image-guided radiotherapy and surgery. However, this registration process is extremely challenging due to the deformation of soft tissue and the existence of speckles in these images. This paper presents a novel intra-modality elastic registration technique for 3D ultrasound images. It uses the general concept of attribute vectors to find the corresponding voxels in the fixed and moving images. The method does not require any pre-segmentation and does not employ any numerical optimization procedure. Therefore, the computational requirements are very low and it has the potential to be used for real-time applications. The technique is implemented and tested for 3D ultrasound images of liver, captured by a 3D ultrasound transducer. The results show that the method is sufficiently accurate and robust and does not easily get trapped with local minima.

Surgical Navigation by Autostereoscopic Image Overlay of Integral Videography

This paper describes an autostereoscopic image overlay technique that is integrated into a surgical navigation system to superimpose a real three-dimensional (3-D) image onto the patient via a half-silvered mirror. The images are created by employing a modified version of integral videography (IV), which is an animated extension of integral photography. IV records and reproduces 3-D images using a microconvex lens array and flat display; it can display geometrically accurate 3-D autostereoscopic images and reproduce motion parallax without the need for special devices. The use of semitransparent display devices makes it appear that the 3-D image is inside the patient's body. This is the first report of applying an autostereoscopic display with an image overlay system in surgical navigation. Experiments demonstrated that the fast IV rendering technique and patient-image registration method produce an average registration accuracy of 1.13 mm. Experiments using a target in phantom agar showed that the system can guide a needle toward a target with an average error of 2.6 mm. Improvement in the quality of the IV display will make this system practical and its use will increase surgical accuracy and reduce invasiveness.

Probe calibration for freehand 3-D ultrasound

Ultrasound (US) probe calibration establishes the rigid body transformation between the US image and a tracking device attached to the probe. This is an important requirement for correct 3-D reconstruction of freehand US images and, thus, for accurate surgical navigation based on US. In this study, we evaluated three methods for probe calibration, based on a single-point phantom, a wire-cross phantom requiring 2-D alignment and a wire phantom for freehand scanning. The processing of acquired data is fairly common to these methods and, to a great extent, based on automated procedures. The evaluation is based on quality measures in 2-D and 3-D reconstructed data. With each of the three methods, we calibrated a linear-array probe, a phased-array sector probe and an intraoperative probe. The freehand method performed best, with a 3-D navigation accuracy of 0.6 mm for one of the probes. This indicates that clinical accuracy in the order of 1 mm may be achieved in US-based surgical navigation.

Displacement estimation with co-registered ultrasound for image guided neurosurgery: a quantitative in vivo porcine study

Brain shift during open cranial surgery presents a challenge for maintaining registration with image-guidance systems. Ultrasound (US) is a convenient intraoperative imaging modality that may be a useful tool in detecting tissue shift and updating preoperative images based on intraoperative measurements of brain deformation. We have quantitatively evaluated the ability of spatially tracked freehand US to detect displacement of implanted markers in a series of three in vivo porcine experiments, where both US and computed tomography (CT) image acquisitions were obtained before and after deforming the brain. Marker displacements ranged from 0.5 to 8.5 mm. Comparisons between CT and US measurements showed a mean target localization error of 1.5 mm, and a mean vector error for displacement of 1.1 mm. Mean error in the magnitude of displacement was 0.6 mm. For one of the animals studied, the US data was used in conjunction with a biomechanical model to nonrigidly re-register a baseline CT to the deformed brain. The mean error between the actual and deformed CT's was found to be on average 1.2 and 1.9 mm at the marker locations depending on the extent of the deformation induced. These findings indicate the potential accuracy in coregistered freehand US displacement tracking in brain tissue and suggest that the resulting information can be used to drive a modeling re-registration strategy to comparable levels of agreement.

A hybrid breast biopsy system combining ultrasound and MRI

System design and initial phantom accuracy results for a novel biopsy system integrating both magnetic resonance (MR) and ultrasound (US) imaging modalities are presented. A phantom experiment was performed to investigate the efficacy of this hybrid guidance biopsy technique in a breast tissue mimicking phantom. A comparison between MR-guided core biopsy verses MR/US-guided core biopsy of phantom targets was realized using a scoring system based on the consistency of the acquired core samples (14 gauge). It was determined that the addition of US to guide needle placement improved the accuracy from an average score of 7.4 out of 10 (MRI guidance alone), to 9.6 (MRI/US guidance) over 21 trials. The average amount of needle tip correction resulting from the additional US information was determined to be 3.7 mm. This correction value is substantial, equal to approximately one radius of the intended targets. Hybrid US/MRI guided biopsy appears to offer a simple means to ensure accurate breast tissue sampling without the need for repeat MRI scans for verification or the need for real-time imaging in open MRI geometries.

Rigid registration of 3-D ultrasound with MR images: a new approach combining intensity and gradient information

We present a new image-based technique to rigidly register intraoperative three-dimensional ultrasound (US) with preoperative magnetic resonance (MR) images. Automatic registration is achieved by maximization of a similarity measure which generalizes the correlation ratio, and whose novelty is to incorporate multivariate information from the MR data (intensity and gradient). In addition, the similarity measure is built upon a robust intensity-based distance measure, which makes it possible to handle a variety of US artifacts. A cross-validation study has been carried out using a number of phantom and clinical data. This indicates that the method is quite robust and that the worst registration errors are of the order of the MR image resolution.

A fast calibration method for 3-D tracking of ultrasound images using a spatial localizer

We have developed a fast calibration method for computing the position and orientation of 2-D ultrasound (US) images in 3-D space where a position sensor is mounted on the US probe. This calibration is required in the fields of 3-D ultrasound and registration of ultrasound with other imaging modalities. Most of the existing calibration methods require a complex and tedious experimental procedure. Our method is simple and it is based on a custom-built phantom. Thirty N-fiducials (markers in the shape of the letter "N") embedded in the phantom provide the basis for our calibration procedure. We calibrated a 3.5-MHz sector phased-array probe with a magnetic position sensor, and we studied the accuracy and precision of our method. A typical calibration procedure requires approximately 2 min. We conclude that we can achieve accurate and precise calibration using a single US image, provided that a large number (approximately ten) of N-fiducials are captured within the US image, enabling a representative sampling of the imaging plane.