Automatic tracking of implanted fiducial markers in cone beam CT projection images

Unsupervised learning for deformable registration of thoracic CT and cone-beam CT based on multiscale features matching with spatially adaptive weighting

PURPOSE

Cone-beam computed tomography (CBCT) is a common on-treatment imaging widely used in image-guided radiotherapy. Fast and accurate registration between the on-treatment CBCT and planning CT is significant for and precise adaptive radiotherapy treatment (ART). However, existing CT-CBCT registration methods, which are mostly affine or time-consuming intensity- based deformation registration, still need further study due to the considerable CT-CBCT intensity discrepancy and the artifacts in low-quality CBCT images. In this paper, we propose a deep learning-based CT-CBCT registration model to promote rapid and accurate CT-CBCT registration for radiotherapy.

METHODS

The proposed CT-CBCT registration model consists of a registration network and an innovative deep similarity metric network. The registration network is a novel fully convolution network adapted specially for patch-wise CT-CBCT registration. The metric network, going beyond intensity, automatically evaluates the high-dimensional attribute-based dissimilarity between the registered CT and CBCT images. In addition, considering the artifacts in low-quality CBCT images, we add spatial weighting (SW) block to adaptively attach more importance to those informative voxels while inhibit the interference of artifact regions. Such SW-based metric network is expected to extract the most meaningful and discriminative deep features, and form a more reliable CT-CBCT similarity measure to train the registration network.

RESULTS

We evaluate the proposed method on clinical thoracic CBCT and CT dataset, and compare the registration results with some other common image similarity metrics and some state-of-the-art registration algorithms. The proposed method provides the highest Structural Similarity index (86.17 ± 5.09), minimum Target Registration Error of landmarks (2.37 ± 0.32 mm), and the best DSC coefficient (78.71 ± 10.95) of tumor volumes. Moreover, our model also obtains comparable distance error of lung surfaces (1.75 ± 0.35 mm).

CONCLUSION

The proposed model shows both efficiency and efficacy for reliable thoracic CT-CBCT registration, and can generate the matched CT and CBCT images within few seconds, which is of great significance to clinical radiotherapy.

Real-time intrafraction motion monitoring in external beam radiotherapy

Radiotherapy (RT) aims to deliver a spatially conformal dose of radiation to tumours while maximizing the dose sparing to healthy tissues. However, the internal patient anatomy is constantly moving due to respiratory, cardiac, gastrointestinal and urinary activity. The long term goal of the RT community to 'see what we treat, as we treat' and to act on this information instantaneously has resulted in rapid technological innovation. Specialized treatment machines, such as robotic or gimbal-steered linear accelerators (linac) with in-room imaging suites, have been developed specifically for real-time treatment adaptation. Additional equipment, such as stereoscopic kilovoltage (kV) imaging, ultrasound transducers and electromagnetic transponders, has been developed for intrafraction motion monitoring on conventional linacs. Magnetic resonance imaging (MRI) has been integrated with cobalt treatment units and more recently with linacs. In addition to hardware innovation, software development has played a substantial role in the development of motion monitoring methods based on respiratory motion surrogates and planar kV or Megavoltage (MV) imaging that is available on standard equipped linacs. In this paper, we review and compare the different intrafraction motion monitoring methods proposed in the literature and demonstrated in real-time on clinical data as well as their possible future developments. We then discuss general considerations on validation and quality assurance for clinical implementation. Besides photon RT, particle therapy is increasingly used to treat moving targets. However, transferring motion monitoring technologies from linacs to particle beam lines presents substantial challenges. Lessons learned from the implementation of real-time intrafraction monitoring for photon RT will be used as a basis to discuss the implementation of these methods for particle RT.

Recommendation of fiducial marker implantation for better target tracking using MV imager in prostate radiotherapy

Purpose

The aim of this study was to develop a model that optimizes the fiducial marker locations in the prostate to increase detectability of the markers in the projected EPID images during VMAT treatments.

Methods and Materials

The fiducial marker tracking capability for each arc was evaluated through a proposed formula. The output of the formula, a detectability score, was calculated with the in‐house developed software written in MATLAB (The Mathworks, Inc., Natick, MA, USA). Three unique weighting factors were added to penalize the detectability score. The detectability scores of four different patterns containing 40 combinations of simulated fiducial marker locations were evaluated with 101 previously treated prostate treatment plans (containing 202 individual arcs). The results were analyzed for each pattern group and each marker separation distance on the transverse plane.

Results

The maximum detectability of the markers occurred when they were placed between 10 and 15 mm from the center of the prostate in the transverse plane and 6–13 mm in the superior–inferior direction. The detectability decreased when the markers were placed beyond 20 mm in both directions.

Conclusions

The fiducial marker‐based detectability score can be used to predict the real‐time tracking capability. Suggestions for optimal insertion locations were given to improve prostate motion management using MV imaging.

Automated patient setup and gating using cone beam computed tomography projections

In radiation therapy, fiducial markers are often implanted near tumors and used for patient positioning and respiratory gating purposes. These markers are then used to manually align the patients by matching the markers in the cone beam computed tomography (CBCT) reconstruction to those in the planning CT. This step is time-intensive and user-dependent, and often results in a suboptimal patient setup. We propose a fully automated, robust method based on dynamic programming (DP) for segmenting radiopaque fiducial markers in CBCT projection images, which are then used to automatically optimize the treatment couch position and/or gating window bounds. The mean of the absolute 2D segmentation error of our DP algorithm is 1.3 ± 1.0 mm for 87 markers on 39 patients. Intrafraction images were acquired every 3 s during treatment at two different institutions. For gated patients from Institution A (8 patients, 40 fractions), the DP algorithm increased the delivery accuracy (96 ± 6% versus 91 ± 11%, p < 0.01) compared to the manual setup using kV fluoroscopy. For non-gated patients from Institution B (6 patients, 16 fractions), the DP algorithm performed similarly (1.5 ± 0.8 mm versus 1.6 ± 0.9 mm, p = 0.48) compared to the manual setup matching the fiducial markers in the CBCT to the mean position. Our proposed automated patient setup algorithm only takes 1-2 s to run, requires no user intervention, and performs as well as or better than the current clinical setup.

Ultrasonic tracking of shear waves using a particle filter

PURPOSE

This paper discusses an application of particle filtering for estimating shear wave velocity in tissue using ultrasound elastography data. Shear wave velocity estimates are of significant clinical value as they help differentiate stiffer areas from softer areas which is an indicator of potential pathology.

METHODS

Radio-frequency ultrasound echo signals are used for tracking axial displacements and obtaining the time-to-peak displacement at different lateral locations. These time-to-peak data are usually very noisy and cannot be used directly for computing velocity. In this paper, the denoising problem is tackled using a hidden Markov model with the hidden states being the unknown (noiseless) time-to-peak values. A particle filter is then used for smoothing out the time-to-peak curve to obtain a fit that is optimal in a minimum mean squared error sense.

RESULTS

Simulation results from synthetic data and finite element modeling suggest that the particle filter provides lower mean squared reconstruction error with smaller variance as compared to standard filtering methods, while preserving sharp boundary detail. Results from phantom experiments show that the shear wave velocity estimates in the stiff regions of the phantoms were within 20% of those obtained from a commercial ultrasound scanner and agree with estimates obtained using a standard method using least-squares fit. Estimates of area obtained from the particle filtered shear wave velocity maps were within 10% of those obtained from B-mode ultrasound images.

CONCLUSIONS

The particle filtering approach can be used for producing visually appealing SWV reconstructions by effectively delineating various areas of the phantom with good image quality properties comparable to existing techniques.

Automated marker tracking using noisy X-ray images degraded by the treatment beam

This study demonstrates the feasibility of automated marker tracking for the real-time detection of intrafractional target motion using noisy kilovoltage (kV) X-ray images degraded by the megavoltage (MV) treatment beam. The authors previously introduced the in-line imaging geometry, in which the flat-panel detector (FPD) is mounted directly underneath the treatment head of the linear accelerator. They found that the 121 kVp image quality was severely compromised by the 6 MV beam passing through the FPD at the same time. Specific MV-induced artefacts present a considerable challenge for automated marker detection algorithms. For this study, the authors developed a new imaging geometry by re-positioning the FPD and the X-ray tube. This improved the contrast-to-noise-ratio between 40% and 72% at the 1.2 mAs/image exposure setting. The increase in image quality clearly facilitates the quick and stable detection of motion with the aid of a template matching algorithm. The setup was tested with an anthropomorphic lung phantom (including an artificial lung tumour). In the tumour one or three Calypso beacons were embedded to achieve better contrast during MV radiation. For a single beacon, image acquisition and automated marker detection typically took around 76 ± 6 ms. The success rate was found to be highly dependent on imaging dose and gantry angle. To eliminate possible false detections, the authors implemented a training phase prior to treatment beam irradiation and also introduced speed limits for motion between subsequent images.

Automatic tracking of arbitrarily shaped implanted markers in kilovoltage projection images: a feasibility study

PURPOSE

Certain types of commonly used fiducial markers take on irregular shapes upon implantation in soft tissue. This poses a challenge for methods that assume a predefined shape of markers when automatically tracking such markers in kilovoltage (kV) radiographs. The authors have developed a method of automatically tracking regularly and irregularly shaped markers using kV projection images and assessed its potential for detecting intrafractional target motion during rotational treatment.

METHODS

Template-based matching used a normalized cross-correlation with simplex minimization. Templates were created from computed tomography (CT) images for phantom studies and from end-expiration breath-hold planning CT for patient studies. The kV images were processed using a Sobel filter to enhance marker visibility. To correct for changes in intermarker relative positions between simulation and treatment that can introduce errors in automatic matching, marker offsets in three dimensions were manually determined from an approximately orthogonal pair of kV images. Two studies in anthropomorphic phantom were carried out, one using a gold cylindrical marker representing regular shape, another using a Visicoil marker representing irregular shape. Automatic matching of templates to cone beam CT (CBCT) projection images was performed to known marker positions in phantom. In patient data, automatic matching was compared to manual matching as an approximate ground truth. Positional discrepancy between automatic and manual matching of less than 2 mm was assumed as the criterion for successful tracking. Tracking success rates were examined in kV projection images from 22 CBCT scans of four pancreas, six gastroesophageal junction, and one lung cancer patients. Each patient had at least one irregularly shaped radiopaque marker implanted in or near the tumor. In addition, automatic tracking was tested in intrafraction kV images of three lung cancer patients with irregularly shaped markers during 11 volumetric modulated arc treatments. Purpose-built software developed at our institution was used to create marker templates and track the markers embedded in kV images.

RESULTS

Phantom studies showed mean ± standard deviation measurement uncertainty of automatic registration to be 0.14 ± 0.07 mm and 0.17 ± 0.08 mm for Visicoil and gold cylindrical markers, respectively. The mean success rate of automatic tracking with CBCT projections (11 frames per second, fps) of pancreas, gastroesophageal junction, and lung cancer patients was 100%, 99.1% (range 98%-100%), and 100%, respectively. With intrafraction images (approx. 0.2 fps) of lung cancer patients, the success rate was 98.2% (range 97%-100%), and 94.3% (range 93%-97%) using templates from 1.25 mm and 2.5 mm slice spacing CT scans, respectively. Correction of intermarker relative position was found to improve the success rate in two out of eight patients analyzed.

CONCLUSIONS

The proposed method can track arbitrary marker shapes in kV images using templates generated from a breath-hold CT acquired at simulation. The studies indicate its feasibility for tracking tumor motion during rotational treatment. Investigation of the causes of misregistration suggests that its rate of incidence can be reduced with higher frequency of image acquisition, templates made from smaller CT slice spacing, and correction of changes in intermarker relative positions when they occur.

Fiducial marker-based correction for involuntary motion in weight-bearing C-arm CT scanning of knees. Part I. Numerical model-based optimization

PURPOSE

Human subjects in standing positions are apt to show much more involuntary motion than in supine positions. The authors aimed to simulate a complicated realistic lower body movement using the four-dimensional (4D) digital extended cardiac-torso (XCAT) phantom. The authors also investigated fiducial marker-based motion compensation methods in two-dimensional (2D) and three-dimensional (3D) space. The level of involuntary movement-induced artifacts and image quality improvement were investigated after applying each method.

METHODS

An optical tracking system with eight cameras and seven retroreflective markers enabled us to track involuntary motion of the lower body of nine healthy subjects holding a squat position at 60° of flexion. The XCAT-based knee model was developed using the 4D XCAT phantom and the optical tracking data acquired at 120 Hz. The authors divided the lower body in the XCAT into six parts and applied unique affine transforms to each so that the motion (6 degrees of freedom) could be synchronized with the optical markers' location at each time frame. The control points of the XCAT were tessellated into triangles and 248 projection images were created based on intersections of each ray and monochromatic absorption. The tracking data sets with the largest motion (Subject 2) and the smallest motion (Subject 5) among the nine data sets were used to animate the XCAT knee model. The authors defined eight skin control points well distributed around the knees as pseudo-fiducial markers which functioned as a reference in motion correction. Motion compensation was done in the following ways: (1) simple projection shifting in 2D, (2) deformable projection warping in 2D, and (3) rigid body warping in 3D. Graphics hardware accelerated filtered backprojection was implemented and combined with the three correction methods in order to speed up the simulation process. Correction fidelity was evaluated as a function of number of markers used (4-12) and marker distribution in three scenarios.

RESULTS

Average optical-based translational motion for the nine subjects was 2.14 mm (± 0.69 mm) and 2.29 mm (± 0.63 mm) for the right and left knee, respectively. In the representative central slices of Subject 2, the authors observed 20.30%, 18.30%, and 22.02% improvements in the structural similarity (SSIM) index with 2D shifting, 2D warping, and 3D warping, respectively. The performance of 2D warping improved as the number of markers increased up to 12 while 2D shifting and 3D warping were insensitive to the number of markers used. The minimum required number of markers for 2D shifting, 2D warping, and 3D warping was 4-6, 12, and 8, respectively. An even distribution of markers over the entire field of view provided robust performance for all three correction methods.

CONCLUSIONS

The authors were able to simulate subject-specific realistic knee movement in weight-bearing positions. This study indicates that involuntary motion can seriously degrade the image quality. The proposed three methods were evaluated with the numerical knee model; 3D warping was shown to outperform the 2D methods. The methods are shown to significantly reduce motion artifacts if an appropriate marker setup is chosen.

Development and clinical evaluation of automatic fiducial detection for tumor tracking in cine megavoltage images during volumetric modulated arc therapy

PURPOSE

Real-time tracking of implanted fiducials in cine megavoltage (MV) imaging during volumetric modulated arc therapy (VMAT) delivery is complicated due to the inherent low contrast of MV images and potential blockage of dynamic leaves configurations. The purpose of this work is to develop a clinically practical autodetection algorithm for motion management during VMAT.

METHODS

The expected field-specific segments and the planned fiducial position from the Eclipse (Varian Medical Systems, Palo Alto, CA) treatment planning system were projected onto the MV images. The fiducials were enhanced by applying a Laplacian of Gaussian filter in the spatial domain for each image, with a blob-shaped object as the impulse response. The search of implanted fiducials was then performed on a region of interest centered on the projection of the fiducial when it was within an open field including the case when it was close to the field edge or partially occluded by the leaves. A universal template formula was proposed for template matching and normalized cross correlation was employed for its simplicity and computational efficiency. The search region for every image was adaptively updated through a prediction model that employed the 3D position of the fiducial estimated from the localized positions in previous images. This prediction model allowed the actual fiducial position to be tracked dynamically and was used to initialize the search region. The artifacts caused by electronic interference during the acquisition were effectively removed. A score map was computed by combining both morphological information and image intensity. The pixel location with the highest score was selected as the detected fiducial position. The sets of cine MV images taken during treatment were analyzed with in-house developed software written in MATLAB (The Mathworks, Inc., Natick, MA). Five prostate patients were analyzed to assess the algorithm performance by measuring their positioning accuracy during treatment.

RESULTS

The algorithm was able to accurately localize the fiducial position on MV images with success rates of more than 90% per case. The percentage of images in which each fiducial was localized in the studied cases varied between 23% and 65%, with at least one fiducial having been localized between 40% and 95% of the images. This depended mainly on the modulation of the plan and fiducial blockage. The prostate movement in the presented cases varied between 0.8 and 3.5 mm (mean values). The maximum displacement detected among all patients was of 5.7 mm.

CONCLUSIONS

An algorithm for automatic detection of fiducial markers in cine MV images has been developed and tested with five clinical cases. Despite the challenges posed by complex beam aperture shapes, fiducial localization close to the field edge, partial occlusion of fiducials, fast leaf and gantry movement, and inherently low MV image quality, good localization results were achieved in patient images. This work provides a technique for enabling real-time accurate fiducial detection and tumor tracking during VMAT treatments without the use of extra imaging dose.

Real-time automatic fiducial marker tracking in low contrast cine-MV images

PURPOSE

To develop a real-time automatic method for tracking implanted radiographic markers in low-contrast cine-MV patient images used in image-guided radiation therapy (IGRT).

METHODS

Intrafraction motion tracking using radiotherapy beam-line MV images have gained some attention recently in IGRT because no additional imaging dose is introduced. However, MV images have much lower contrast than kV images, therefore a robust and automatic algorithm for marker detection in MV images is a prerequisite. Previous marker detection methods are all based on template matching or its derivatives. Template matching needs to match object shape that changes significantly for different implantation and projection angle. While these methods require a large number of templates to cover various situations, they are often forced to use a smaller number of templates to reduce the computation load because their methods all require exhaustive search in the region of interest. The authors solve this problem by synergetic use of modern but well-tested computer vision and artificial intelligence techniques; specifically the authors detect implanted markers utilizing discriminant analysis for initialization and use mean-shift feature space analysis for sequential tracking. This novel approach avoids exhaustive search by exploiting the temporal correlation between consecutive frames and makes it possible to perform more sophisticated detection at the beginning to improve the accuracy, followed by ultrafast sequential tracking after the initialization. The method was evaluated and validated using 1149 cine-MV images from two prostate IGRT patients and compared with manual marker detection results from six researchers. The average of the manual detection results is considered as the ground truth for comparisons.

RESULTS

The average root-mean-square errors of our real-time automatic tracking method from the ground truth are 1.9 and 2.1 pixels for the two patients (0.26 mm/pixel). The standard deviations of the results from the 6 researchers are 2.3 and 2.6 pixels. The proposed framework takes about 128 ms to detect four markers in the first MV images and about 23 ms to track these markers in each of the subsequent images.

CONCLUSIONS

The unified framework for tracking of multiple markers presented here can achieve marker detection accuracy similar to manual detection even in low-contrast cine-MV images. It can cope with shape deformations of fiducial markers at different gantry angles. The fast processing speed reduces the image processing portion of the system latency, therefore can improve the performance of real-time motion compensation.

Robust methods for automatic image-to-world registration in cone-beam CT interventional guidance

PURPOSE

Real-time surgical navigation relies on accurate image-to-world registration to align the coordinate systems of the image and patient. Conventional manual registration can present a workflow bottleneck and is prone to manual error and intraoperator variability. This work reports alternative means of automatic image-to-world registration, each method involving an automatic registration marker (ARM) used in conjunction with C-arm cone-beam CT (CBCT). The first involves a Known-Model registration method in which the ARM is a predefined tool, and the second is a Free-Form method in which the ARM is freely configurable.

METHODS

Studies were performed using a prototype C-arm for CBCT and a surgical tracking system. A simple ARM was designed with markers comprising a tungsten sphere within infrared reflectors to permit detection of markers in both x-ray projections and by an infrared tracker. The Known-Model method exercised a predefined specification of the ARM in combination with 3D-2D registration to estimate the transformation that yields the optimal match between forward projection of the ARM and the measured projection images. The Free-Form method localizes markers individually in projection data by a robust Hough transform approach extended from previous work, backprojected to 3D image coordinates based on C-arm geometric calibration. Image-domain point sets were transformed to world coordinates by rigid-body point-based registration. The robustness and registration accuracy of each method was tested in comparison to manual registration across a range of body sites (head, thorax, and abdomen) of interest in CBCT-guided surgery, including cases with interventional tools in the radiographic scene.

RESULTS

The automatic methods exhibited similar target registration error (TRE) and were comparable or superior to manual registration for placement of the ARM within ∼200 mm of C-arm isocenter. Marker localization in projection data was robust across all anatomical sites, including challenging scenarios involving the presence of interventional tools. The reprojection error of marker localization was independent of the distance of the ARM from isocenter, and the overall TRE was dominated by the configuration of individual fiducials and distance from the target as predicted by theory. The median TRE increased with greater ARM-to-isocenter distance (e.g., for the Free-Form method, TRE increasing from 0.78 mm to 2.04 mm at distances of ∼75 mm and 370 mm, respectively). The median TRE within ∼200 mm distance was consistently lower than that of the manual method (TRE = 0.82 mm). Registration performance was independent of anatomical site (head, thorax, and abdomen). The Free-Form method demonstrated a statistically significant improvement (p = 0.0044) in reproducibility compared to manual registration (0.22 mm versus 0.30 mm, respectively).

CONCLUSIONS

Automatic image-to-world registration methods demonstrate the potential for improved accuracy, reproducibility, and workflow in CBCT-guided procedures. A Free-Form method was shown to exhibit robustness against anatomical site, with comparable or improved TRE compared to manual registration. It was also comparable or superior in performance to a Known-Model method in which the ARM configuration is specified as a predefined tool, thereby allowing configuration of fiducials on the fly or attachment to the patient.