A key-point based real-time tracking of lung tumor in x-ray image sequence by using difference of Gaussians filtering and optical flow

Simultaneous object detection and segmentation for patient-specific markerless lung tumor tracking in simulated radiographs with deep learning

BACKGROUND

Real-time tumor tracking is one motion management method to address motion-induced uncertainty. To date, fiducial markers are often required to reliably track lung tumors with X-ray imaging, which carries risks of complications and leads to prolonged treatment time. A markerless tracking approach is thus desirable. Deep learning-based approaches have shown promise for markerless tracking, but systematic evaluation and procedures to investigate applicability in individual cases are missing. Moreover, few efforts have been made to provide bounding box prediction and mask segmentation simultaneously, which could allow either rigid or deformable multi-leaf collimator tracking.

PURPOSE

The purpose of this study was to implement a deep learning-based markerless lung tumor tracking model exploiting patient-specific training which outputs both a bounding box and a mask segmentation simultaneously. We also aimed to compare the two kinds of predictions and to implement a specific procedure to understand the feasibility of markerless tracking on individual cases.

METHODS

We first trained a Retina U-Net baseline model on digitally reconstructed radiographs (DRRs) generated from a public dataset containing 875 CT scans and corresponding lung nodule annotations. Afterwards, we used an independent cohort of 97 lung patients to develop a patient-specific refinement procedure. In order to determine the optimal hyperparameters for automatic patient-specific training, we selected 13 patients for validation where the baseline model predicted a bounding box on planning CT (PCT)-DRR with intersection over union (IoU) with the ground-truth higher than 0.7. The final test set contained the remaining 84 patients with varying PCT-DRR IoU. For each testing patient, the baseline model was refined on the PCT-DRR to generate a patient-specific model, which was then tested on a separate 10-phase 4DCT-DRR to mimic the intrafraction motion during treatment. A template matching algorithm served as benchmark model. The testing results were evaluated by four metrics: the center of mass (COM) error and the Dice similarity coefficient (DSC) for segmentation masks, and the center of box (COB) error and the DSC for bounding box detections. Performance was compared to the benchmark model including statistical testing for significance.

RESULTS

A PCT-DRR IoU value of 0.2 was shown to be the threshold dividing inconsistent (68%) and consistent (100%) success (defined as mean bounding box DSC > 0.6) of PS models on 4DCT-DRRs. Thirty-seven out of the eighty-four testing cases had a PCT-DRR IoU above 0.2. For these 37 cases, the mean COM error was 2.6 mm, the mean segmentation DSC was 0.78, the mean COB error was 2.7 mm, and the mean box DSC was 0.83. Including the validation cases, the model was applicable to 50 out of 97 patients when using the PCT-DRR IoU threshold of 0.2. The inference time per frame was 170 ms. The model outperformed the benchmark model on all metrics, and the comparison was significant (p < 0.001) over the 37 PCT-DRR IoU > 0.2 cases, but not over the undifferentiated 84 testing cases.

CONCLUSIONS

The implemented patient-specific refinement approach based on a pre-trained baseline model was shown to be applicable to markerless tumor tracking in simulated radiographs for lung cases.

Improved Tumor Image Estimation in X-Ray Fluoroscopic Images by Augmenting 4DCT Data for Radiotherapy

Measurement of tumor position is important for the radiotherapy of lung tumors with respiratory motion. Although tumors can be observed using X-ray fluoroscopy during radiotherapy, it is often difficult to measure tumor position from X-ray image sequences accurately because of overlapping organs. To measure tumor position accurately, a method for extracting tumor intensities from X-ray image sequences using a hidden Markov model (HMM) has been proposed. However, the performance of tumor intensity extraction depends on limited knowledge regarding the tumor motion observed in the four-dimensional computed tomography (4DCT) data used to construct the HMM. In this study, we attempted to improve the performance of tumor intensity extraction by augmenting 4DCT data. The proposed method was tested using simulated datasets of X-ray image sequences. The experimental results indicated that the HMM using the augmentation method could improve tumor-tracking performance when the range of tumor movement during treatment differed from that in the 4DCT data.

Vector-field dynamic x-ray (VF-DXR) using optical flow method in patients with chronic obstructive pulmonary disease

BACKGROUND

We assessed the difference in lung motion during inspiration/expiration between chronic obstructive pulmonary disease (COPD) patients and healthy volunteers using vector-field dynamic x-ray (VF-DXR) with optical flow method (OFM).

METHODS

We enrolled 36 COPD patients and 47 healthy volunteers, classified according to pulmonary function into: normal, COPD mild, and COPD severe. Contrast gradient was obtained from sequential dynamic x-ray (DXR) and converted to motion vector using OFM. VF-DXR images were created by projection of the vertical component of lung motion vectors onto DXR images. The maximum magnitude of lung motion vectors in tidal inspiration/expiration, forced inspiration/expiration were selected and defined as lung motion velocity (LMV). Correlations between LMV with demographics and pulmonary function and differences in LMV between COPD patients and healthy volunteers were investigated.

RESULTS

Negative correlations were confirmed between LMV and % forced expiratory volume in one second (%FEV1) in the tidal inspiration in the right lung (Spearman's rank correlation coefficient, rs = -0.47, p < 0.001) and the left lung (rs = -0.32, p = 0.033). A positive correlation between LMV and %FEV1 in the tidal expiration was observed only in the right lung (rs = 0.25, p = 0.024). LMVs among normal, COPD mild and COPD severe groups were different in the tidal respiration. COPD mild group showed a significantly larger magnitude of LMV compared with the normal group.

CONCLUSIONS

In the tidal inspiration, the lung parenchyma moved faster in COPD patients compared with healthy volunteers. VF-DXR was feasible for the assessment of lung parenchyma using LMV.

Vector-Field dynamic X-ray (VF-DXR) using Optical Flow Method

OBJECTIVES

To explore the feasibility of Vector-Field DXR (VF-DXR) using optical flow method (OFM).

METHODS

Five healthy volunteers and five COPD patients were studied. DXR was performed in the standing position using a prototype X-ray system (Konica Minolta Inc., Tokyo, Japan). During the examination, participants took several tidal breaths and one forced breath. DXR image file was converted to the videos with different frames per second (fps): 15 fps, 7.5 fps, five fps, three fps, and 1.5 fps. Pixel-value gradient was calculated by the serial change of pixel value, which was subsequently converted mathematically to motion vector using OFM. Color-coding map and vector projection into horizontal and vertical components were also tested.

RESULTS

Dynamic motion of lung and thorax was clearly visualized using VF-DXR with an optimal frame rate of 5 fps. Color-coding map and vector projection into horizontal and vertical components were also presented. VF-DXR technique was also applied in COPD patients.

CONCLUSION

The feasibility of VF-DXR was demonstrated with small number of healthy subjects and COPD patients.

ADVANCES IN KNOWLEDGE

A new Vector-Field Dynamic X-ray (VF-DXR) technique is feasible for dynamic visualization of lung, diaphragms, thoracic cage, and cardiac contour.