A systematic literature review: deep learning techniques for synthetic medical image generation and their applications in radiotherapy

The aim of this systematic review is to determine whether Deep Learning (DL) algorithms can provide a clinically feasible alternative to classic algorithms for synthetic Computer Tomography (sCT). The following categories are presented in this study: ∙ MR-based treatment planning and synthetic CT generation techniques. ∙ Generation of synthetic CT images based on Cone Beam CT images. ∙ Low-dose CT to High-dose CT generation. ∙ Attenuation correction for PET images. To perform appropriate database searches, we reviewed journal articles published between January 2018 and June 2023. Current methodology, study strategies, and results with relevant clinical applications were analyzed as we outlined the state-of-the-art of deep learning based approaches to inter-modality and intra-modality image synthesis. This was accomplished by contrasting the provided methodologies with traditional research approaches. The key contributions of each category were highlighted, specific challenges were identified, and accomplishments were summarized. As a final step, the statistics of all the cited works from various aspects were analyzed, which revealed that DL-based sCTs have achieved considerable popularity, while also showing the potential of this technology. In order to assess the clinical readiness of the presented methods, we examined the current status of DL-based sCT generation.

An optimized framework for cone-beam computed tomography-based online evaluation for proton therapy

BACKGROUND

Clinical evidence has demonstrated that proton therapy can achieve comparable tumor control probabilities compared to conventional photon therapy but with the added benefit of sparing healthy tissues. However, proton therapy is sensitive to inter-fractional anatomy changes. Online pre-fraction evaluation can effectively verify proton dose before delivery to patients, but there is a lack of guidelines for implementing this workflow.

PURPOSE

The purpose of this study is to develop a cone-beam CT-based (CBCT) online evaluation framework for proton therapy that enables knowledge transparency and evaluates the efficiency and accuracy of each essential component.

METHODS

Twenty-three patients with various lesion sites were included to conduct a retrospective study of implementing the proposed CBCT evaluation framework for the clinic. The framework was implemented on the RayStation 11B Research platform. Two synthetic CT (sCT) methods, corrected CBCT (cCBCT), and virtual CT (vCT), were used, and the ground truth images were acquired from the same-day deformed quality assurance CT (dQACT) for the comparisons. The evaluation metrics for the framework include time efficiency, dose-difference distributions (gamma passing rates), and water equivalent thickness (WET) distributions.

RESULTS

The mean online CBCT evaluation times were 1.6 ± 0.3 min and 1.9 ± 0.4 min using cCBCT and vCT, respectively. The dose calculation and deformable image registration dominated the evaluation efficiency, and accounted for 33% and 30% of the total evaluation time, respectively. The sCT generation took another 19% of the total time. Gamma passing rates were greater than 91% and 97% using 1%/1 mm and 2%/2 mm criteria, respectively. When the appropriate sCT was chosen, the target mean WET difference from the reference were less than 0.5 mm. The appropriate sCT method choice determined the uncertainty for the framework, with the cCBCT being superior for head-and-neck patient evaluation and vCT being better for lung patient evaluation.

CONCLUSIONS

An online CBCT evaluation framework was proposed to identify the use of the optimal sCT algorithm regarding efficiency and dosimetry accuracy. The framework is extendable to adopt advanced imaging methods and has the potential to support online adaptive radiotherapy to enhance patient benefits. It could be implemented into clinical use in the future.

An Unsupervised Learning-Based Regional Deformable Model for Automated Multi-Organ Contour Propagation

The aim of this study is to evaluate a regional deformable model based on a deep unsupervised learning model for automatic contour propagation in breast cone-beam computed tomography-guided adaptive radiation therapy. A deep unsupervised learning model was introduced to map breast's tumor bed, clinical target volume, heart, left lung, right lung, and spinal cord from planning computed tomography to cone-beam CT. To improve the traditional image registration method's performance, we used a regional deformable framework based on the narrow-band mapping, which can mitigate the effect of the image artifacts on the cone-beam CT. We retrospectively selected 373 anonymized cone-beam CT volumes from 111 patients with breast cancer. The cone-beam CTs are divided into three sets. 311 / 20 / 42 cone-beam CT images were used for training, validating, and testing. The manual contour was used as reference for the testing set. We compared the results between the reference and the model prediction for evaluating the performance. The mean Dice between manual reference segmentations and the model predicted segmentations for breast tumor bed, clinical target volume, heart, left lung, right lung, and spinal cord were 0.78 ± 0.09, 0.90 ± 0.03, 0.88 ± 0.04, 0.94 ± 0.03, 0.95 ± 0.02, and 0.77 ± 0.07, respectively. The results demonstrated a good agreement between the reference and the proposed contours. The proposed deep learning-based regional deformable model technique can automatically propagate contours for breast cancer adaptive radiotherapy. Deep learning in contour propagation was promising, but further investigation was warranted.

The geometric and dosimetric accuracy of kilovoltage cone beam computed tomography images for adaptive treatment: a systematic review

Objectives

To provide an overview and meta-analysis of different techniques adopted to accomplish kVCBCT for dose calculation and automated segmentation.

Methods

A systematic review and meta-analysis were performed on eligible studies demonstrating kVCBCT-based dose calculation and automated contouring of different tumor features. Meta-analysis of the performance was accomplished on the reported γ analysis and dice similarity coefficient (DSC) score of both collected results as three subgroups (head and neck, chest, and abdomen).

Results

After the literature scrutinization (n = 1008), 52 papers were recognized for the systematic review. Nine studies of dosimtric studies and eleven studies of geometric analysis were suitable for inclusion in meta-analysis. Using kVCBCT for treatment replanning depends on a method used. Deformable Image Registration (DIR) methods yielded small dosimetric error (≤2%), γ pass rate (≥90%) and DSC (≥0.8). Hounsfield Unit (HU) override and calibration curve-based methods also achieved satisfactory yielded small dosimetric error (≤2%) and γ pass rate ((≥90%), but they are prone to error due to their sensitivity to a vendor-specific variation in kVCBCT image quality.

Conclusions

Large cohorts of patients ought to be undertaken to validate methods achieving low levels of dosimetric and geometric errors. Quality guidelines should be established when reporting on kVCBCT, which include agreed metrics for reporting on the quality of corrected kVCBCT and defines protocols of new site-specific standardized imaging used when obtaining kVCBCT images for adaptive radiotherapy.

Advances in knowledge

This review gives useful knowledge about methods making kVCBCT feasible for kVCBCT-based adaptive radiotherapy, simplifying patient pathway and reducing concomitant imaging dose to the patient.

Evaluation of generalization ability for deep learning-based auto-segmentation accuracy in limited field of view CBCT of male pelvic region

PURPOSE

The aim of this study was to evaluate generalization ability of segmentation accuracy for limited FOV CBCT in the male pelvic region using a full-image CNN. Auto-segmentation accuracy was evaluated using various datasets with different intensity distributions and FOV sizes.

METHODS

A total of 171 CBCT datasets from patients with prostate cancer were enrolled. There were 151, 10, and 10 CBCT datasets acquired from Vero4DRT, TrueBeam STx, and Clinac-iX, respectively. The FOV for Vero4DRT, TrueBeam STx, and Clinac-iX was 20, 26, and 25 cm, respectively. The ROIs, including the bladder, prostate, rectum, and seminal vesicles, were manually delineated. The U² -Net CNN network architecture was used to train the segmentation model. A total of 131 limited FOV CBCT datasets from Vero4DRT were used for training (104 datasets) and validation (27 datasets); thereafter the rest were for testing. The training routine was set to save the best weight values when the DSC in the validation set was maximized. Segmentation accuracy was qualitatively and quantitatively evaluated between the ground truth and predicted ROIs in the different testing datasets.

RESULTS

The mean scores ± standard deviation of visual evaluation for bladder, prostate, rectum, and seminal vesicle in all treatment machines were 1.0 ± 0.7, 1.5 ± 0.6, 1.4 ± 0.6, and 2.1 ± 0.8 points, respectively. The median DSC values for all imaging devices were ≥0.94 for the bladder, 0.84-0.87 for the prostate and rectum, and 0.48-0.69 for the seminal vesicles. Although the DSC values for the bladder and seminal vesicles were significantly different among the three imaging devices, the DSC value of the bladder changed by less than 1% point. The median MSD values for all imaging devices were ≤1.2 mm for the bladder and 1.4-2.2 mm for the prostate, rectum, and seminal vesicles. The MSD values for the seminal vesicles were significantly different between the three imaging devices.

CONCLUSION

The proposed method is effective for testing datasets with different intensity distributions and FOV from training datasets.

CDFRegNet: A cross-domain fusion registration network for CT-to-CBCT image registration

BACKGROUND AND OBJECTIVE

Computer tomography (CT) to cone-beam computed tomography (CBCT) image registration plays an important role in radiotherapy treatment placement, dose verification, and anatomic changes monitoring during radiotherapy. However, fast and accurate CT-to-CBCT image registration is still very challenging due to the intensity differences, the poor image quality of CBCT images, and inconsistent structure information.

METHODS

To address these problems, a novel unsupervised network named cross-domain fusion registration network (CDFRegNet) is proposed. First, a novel edge-guided attention module (EGAM) is designed, aiming at capturing edge information based on the gradient prior images and guiding the network to model the spatial correspondence between two image domains. Moreover, a novel cross-domain attention module (CDAM) is proposed to improve the network's ability to guide the network to effectively map and fuse the domain-specific features.

RESULTS

Extensive experiments on a real clinical dataset were carried out, and the experimental results verify that the proposed CDFRegNet can register CT to CBCT images effectively and obtain the best performance, while compared with other representative methods, with a mean DSC of 80.01±7.16%, a mean TRE of 2.27±0.62 mm, and a mean MHD of 1.50±0.32 mm. The ablation experiments also proved that our EGAM and CDAM can further improve the accuracy of the registration network and they can generalize well to other registration networks.

CONCLUSION

This paper proposed a novel CT-to-CBCT registration method based on EGAM and CDAM, which has the potential to improve the accuracy of multi-domain image registration.

MIRSIG position paper: the use of image registration and fusion algorithms in radiotherapy

The report of the American Association of Physicists in Medicine (AAPM) Task Group No. 132 published in 2017 reviewed rigid image registration and deformable image registration (DIR) approaches and solutions to provide recommendations for quality assurance and quality control of clinical image registration and fusion techniques in radiotherapy. However, that report did not include the use of DIR for advanced applications such as dose warping or warping of other matrices of interest. Considering that DIR warping tools are now readily available, discussions were hosted by the Medical Image Registration Special Interest Group (MIRSIG) of the Australasian College of Physical Scientists & Engineers in Medicine in 2018 to form a consensus on best practice guidelines. This position statement authored by MIRSIG endorses the recommendations of the report of AAPM task group 132 and expands on the best practice advice from the ‘Deforming to Best Practice’ MIRSIG publication to provide guidelines on the use of DIR for advanced applications.

Head and neck synthetic CT generated from ultra-low-dose cone-beam CT following Image Gently Protocol using deep neural network

PURPOSE

Image guidance is used to improve accuracy of radiation therapy delivery but results in increased dose to patients. This is of particular concern in children who need be treated per Pediatric Image Gently Protocols due to long term risks from radiation exposure. The purpose of this study is to design a deep neural network (DNN) architecture and loss function for improving soft-tissue contrast and preserving small anatomical features in ultra-low-dose cone-beam CTs (CBCT) of head and neck cancer (HNC) imaging.

METHODS

A 2-D compound U-Net architecture (modified U-Net++) with different depths was proposed to enhance the network capability of capturing small-volume structures. A mask weighted loss function (Mask-Loss) was applied to enhance soft-tissue contrast. Fifty-five paired CBCT and CT images of HNC patients were retrospectively collected for network training and testing. The output enhanced CBCT images from the present study were evaluated with quantitative metrics including mean absolute error (MAE), signal-to-noise ratio (SNR), and structural similarity (SSIM), and compared with those from the previously proposed network architectures (U-Net and wide U-Net) using MAE loss functions. A visual assessment of ten selected structures in the enhanced CBCT images of each patient was performed to evaluate image quality improvement, blindly scored by an experienced radiation oncologist specialized in HN cancer.

RESULTS

All the enhanced CBCT images showed reduced artifactual distortion and image noise. U-Net++ outperformed the U-Net and wide U-Net in terms of MAE, contrast near structure boundaries, and small structures. The proposed Mask-Loss improved image contrast and accuracy of the soft-tissue regions. The enhanced CBCT images predicted by U-Net++ and Mask-Loss demonstrated improvement compared to the U-Net in terms of average MAE (52.41 vs. 42.85 HU), SNR (14.14 vs. 15.07 dB), and SSIM (0.84 vs. 0.87), respectively (p < 0.01, in all paired t-tests). The visual assessment showed that the proposed U-Net++ and Mask-Loss significantly improved original CBCTs (p < 0.01), compared to the U-Net and MAE loss.

CONCLUSIONS

The proposed network architecture and loss function effectively improved image quality in soft-tissue contrast, organ boundary, and small structures preservation for ultra-low-dose CBCT following Image Gently Protocol. This method has potential to provide sufficient anatomical representation on the enhanced CBCT images for accurate treatment delivery and potentially fast online-adaptive re-planning for HN cancer patients. This article is protected by copyright. All rights reserved.

Onboard cone-beam CT-based replan evaluation for head and neck proton therapy

Purpose

Quality assurance computed tomography (QACT) is the current clinical practice in proton therapy to evaluate the needs for replan. QACT could falsely indicate replan because of setup issues that would be solved on the treatment machine. Deforming the treatment planning CT (TPCT) to the pretreatment CBCT may eliminate this issue. We investigated the performance of replan evaluation based on deformed TPCT (TPCTdir) for proton head and neck (H&N) therapy.

Methods and materials

Twenty‐eight H&N datasets along with pretreatment CBCT and QACT were used to validate the method. The changes in body volume were analyzed between the no‐replan and replan groups. The dose on the TPCTdir, the deformed QACT (QACTdir), and the QACT were calculated by applying the clinical plans to these image sets. Dosimetric parameters’ changes, including ΔD95, ΔDmean, and ΔD1 for the clinical target volumes (CTVs) were calculated. Receiver operating characteristic curves for replan evaluation based on ΔD95 on QACT and TPCTdir were calculated, using ΔD95 on QACTdir as the reference. A threshold for replan based on ΔD95 on TPCTdir is proposed. The specificities for the proposed method were calculated.

Results

The changes in the body contour were 95.8 ± 83.8 cc versus 305.0 ± 235.0 cc (p < 0.01) for the no‐replan and replan groups, respectively. The ΔD95, ΔDmean, and ΔD1 are all comparable for all the evaluations. The differences between TPCTdir and QACTdir evaluations were 0.30% ± 0.86%, 0.00 ± 0.22 Gy, and −0.17 ± 0.61 Gy for CTV ΔD95, ΔDmean, and ΔD1, respectively. The corresponding differences between the QACT and QACTdir were 0.12% ± 1.1%, 0.02 ± 0.32 Gy, and −0.01 ± 0.71 Gy. CTV ΔD95 > 2.6% in TPCTdir was chosen as the threshold to trigger QACT/replan. The corresponding specificity was 94% and 98% for the clinical practice and the proposed method, respectively.

Conclusions

The replan evaluation based on TPCTdir provides better specificity than that based on the QACT.

Human-level comparable control volume mapping with a deep unsupervised-learning model for image-guided radiation therapy

PURPOSE

To develop a deep unsupervised learning method with control volume (CV) mapping from patient positioning daily CT (dCT) to planning computed tomography (pCT) for precise patient positioning.

METHODS

We propose an unsupervised learning framework, which maps CVs from dCT to pCT to automatically generate the couch shifts, including translation and rotation dimensions. The network inputs are dCT, pCT and CV positions in the pCT. The output is the transformation parameter of the dCT used to setup the head and neck cancer (HNC) patients. The network is trained to maximize image similarity between the CV in the pCT and the CV in the dCT. A total of 554 CT scans from 158 HNC patients were used for the evaluation of the proposed model. At different points in time, each patient had many CT scans. Couch shifts are calculated for the testing by averaging the translation and rotation from the CVs. The ground-truth of the shifts come from bone landmarks determined by an experienced radiation oncologist.

RESULTS

The system positioning errors of translation and rotation are less than 0.47 mm and 0.17°, respectively. The random positioning errors of translation and rotation are less than 1.13 mm and 0.29°, respectively. The proposed method enhanced the proportion of cases registered within a preset tolerance (2.0 mm/1.0°) from 66.67% to 90.91% as compared to standard registrations.

CONCLUSIONS

We proposed a deep unsupervised learning architecture for patient positioning with inclusion of CVs mapping, which weights the CVs regions differently to mitigate any potential adverse influence of image artifacts on the registration. Our experimental results show that the proposed method achieved efficient and effective HNC patient positioning.

Geometric and Dosimetric Evaluation of Deep Learning-Based Automatic Delineation on CBCT-Synthesized CT and Planning CT for Breast Cancer Adaptive Radiotherapy: A Multi-Institutional Study

Purpose

We developed a deep learning model to achieve automatic multitarget delineation on planning CT (pCT) and synthetic CT (sCT) images generated from cone-beam CT (CBCT) images. The geometric and dosimetric impact of the model was evaluated for breast cancer adaptive radiation therapy.

Methods

We retrospectively analyzed 1,127 patients treated with radiotherapy after breast-conserving surgery from two medical institutions. The CBCT images for patient setup acquired utilizing breath-hold guided by optical surface monitoring system were used to generate sCT with a generative adversarial network. Organs at risk (OARs), clinical target volume (CTV), and tumor bed (TB) were delineated automatically with a 3D U-Net model on pCT and sCT images. The geometric accuracy of the model was evaluated with metrics, including Dice similarity coefficient (DSC) and 95% Hausdorff distance (HD95). Dosimetric evaluation was performed by quick dose recalculation on sCT images relying on gamma analysis and dose-volume histogram (DVH) parameters. The relationship between ΔD95, ΔV95 and DSC-CTV was assessed to quantify the clinical impact of the geometric changes of CTV.

Results

The ranges of DSC and HD95 were 0.73–0.97 and 2.22–9.36 mm for pCT, 0.63–0.95 and 2.30–19.57 mm for sCT from institution A, 0.70–0.97 and 2.10–11.43 mm for pCT from institution B, respectively. The quality of sCT was excellent with an average mean absolute error (MAE) of 71.58 ± 8.78 HU. The mean gamma pass rate (3%/3 mm criterion) was 91.46 ± 4.63%. DSC-CTV down to 0.65 accounted for a variation of more than 6% of V95 and 3 Gy of D95. DSC-CTV up to 0.80 accounted for a variation of less than 4% of V95 and 2 Gy of D95. The mean ΔD90/ΔD95 of CTV and TB were less than 2Gy/4Gy, 4Gy/5Gy for all the patients. The cardiac dose difference in left breast cancer cases was larger than that in right breast cancer cases.

Conclusions

The accurate multitarget delineation is achievable on pCT and sCT via deep learning. The results show that dose distribution needs to be considered to evaluate the clinical impact of geometric variations during breast cancer radiotherapy.

Evaluation of the relationship between the range of radiation-induced lung injury on CT images after IMRT for stage I lung cancer and dosimetric parameters

BACKGROUND

This study evaluated the correlation between radiation-induced lung injury (RILI) and dosimetric parameters on computed tomography (CT) images of stage I non-small cell lung cancer (NSCLC) patients undergoing intensity-modulated radiotherapy (IMRT).

MATERIALS AND METHODS

Sixty-three stage I NSLC patients who underwent IMRT were enrolled in the study. The patients underwent CT within 6 months (acute phase) and 1.5 years (late phase) after radiotherapy. These were fused with the planned irradiation CT. The range of RILI was measured from 10% to 100%, with an IC in 10% increments.

RESULTS

The median interval from completion of radiotherapy to acute and late phase CT was 92 and 440 days, respectively. The median RILI ranges of the acute and late phases were in the 80% (20-100%) and 70% dose regions (20-100%), respectively. The significantly narrower range of RILI when lung V20 in the acute phase was less than 19.2% and that of V5 in the late phase was less than 27.6% at the time of treatment planning.

CONCLUSIONS

This study showed that RILI occurred in a localized range in stage I NSCLC patients who underwent IMRT. The range of RILI was correlated with V20 in the acute phase and V5 in the late phase. KEY MESSAGES RILI correlated with V20 in acute and V5 in late phase. The shadow of RILI occurred in 80% dose region in acute and 70% in late phase. No relationship exists between radiographic changes in RILI and PTV volume.

Clinical Enhancement in AI-Based Post-processed Fast-Scan Low-Dose CBCT for Head and Neck Adaptive Radiotherapy

Purpose: To assess image quality and uncertainty in organ-at-risk segmentation on cone beam computed tomography (CBCT) enhanced by deep-learning convolutional neural network (DCNN) for head and neck cancer.

Methods: An in-house DCNN was trained using forty post-operative head and neck cancer patients with their planning CT and first-fraction CBCT images. Additional fifteen patients with repeat simulation CT (rCT) and CBCT scan taken on the same day (oCBCT) were used for validation and clinical utility assessment. Enhanced CBCT (eCBCT) images were generated from the oCBCT using the in-house DCNN. Quantitative imaging quality improvement was evaluated using HU accuracy, signal-to-noise-ratio (SNR), and structural similarity index measure (SSIM). Organs-at-risk (OARs) were delineated on o/eCBCT and compared with manual structures on the same day rCT. Contour accuracy was assessed using dice similarity coefficient (DSC), Hausdorff distance (HD), and center of mass (COM) displacement. Qualitative assessment of users’ confidence in manual segmenting OARs was performed on both eCBCT and oCBCT by visual scoring.

Results: eCBCT organs-at-risk had significant improvement on mean pixel values, SNR (p < 0.05), and SSIM (p < 0.05) compared to oCBCT images. Mean DSC of eCBCT-to-rCT (0.83 ± 0.06) was higher than oCBCT-to-rCT (0.70 ± 0.13). Improvement was observed for mean HD of eCBCT-to-rCT (0.42 ± 0.13 cm) vs. oCBCT-to-rCT (0.72 ± 0.25 cm). Mean COM was less for eCBCT-to-rCT (0.28 ± 0.19 cm) comparing to oCBCT-to-rCT (0.44 ± 0.22 cm). Visual scores showed OAR segmentation was more accessible on eCBCT than oCBCT images.

Conclusion: DCNN improved fast-scan low-dose CBCT in terms of the HU accuracy, image contrast, and OAR delineation accuracy, presenting potential of eCBCT for adaptive radiotherapy.

Automated contour propagation of the prostate from pCT to CBCT images via deep unsupervised learning

PURPOSE

To develop and evaluate a deep unsupervised learning (DUL) framework based on a regional deformable model for automated prostate contour propagation from planning computed tomography (pCT) to cone-beam CT (CBCT).

METHODS

We introduce a DUL model to map the prostate contour from pCT to on-treatment CBCT. The DUL framework used a regional deformable model via narrow-band mapping to augment the conventional strategy. Two hundred and fifty-one anonymized CBCT images from prostate cancer patients were retrospectively selected and divided into three sets: 180 were used for training, 12 for validation, and 59 for testing. The testing dataset was divided into two groups. Group 1 contained 50 CBCT volumes, with one physician-generated prostate contour on CBCT image. Group 2 contained nine CBCT images, each including prostate contours delineated by four independent physicians and a consensus contour generated using the STAPLE method. Results were compared between the proposed DUL and physician-generated contours through the Dice similarity coefficients (DSCs), the Hausdorff distances, and the distances of the center-of-mass.

RESULTS

The average DSCs between DUL-based prostate contours and reference contours for test data in group 1 and group 2 consensus were 0.83 ± 0.04, and 0.85 ± 0.04, respectively. Correspondingly, the mean center-of-mass distances were 3.52 mm ± 1.15 mm, and 2.98 mm ± 1.42 mm, respectively.

CONCLUSIONS

This novel DUL technique can automatically propagate the contour of the prostate from pCT to CBCT. The proposed method shows that highly accurate contour propagation for CBCT-guided adaptive radiotherapy is achievable via the deep learning technique.

Head-and-neck organs-at-risk auto-delineation using dual pyramid networks for CBCT-guided adaptive radiotherapy

Organ-at-risk (OAR) delineation is a key step for cone-beam CT (CBCT) based adaptive radiotherapy planning that can be a time-consuming, labor-intensive, and subject-to-variability process. We aim to develop a fully automated approach aided by synthetic MRI for rapid and accurate CBCT multi-organ contouring in head-and-neck (HN) cancer patients. MRI has superb soft-tissue contrasts, while CBCT offers bony-structure contrasts. Using the complementary information provided by MRI and CBCT is expected to enable accurate multi-organ segmentation in HN cancer patients. In our proposed method, MR images are firstly synthesized using a pre-trained cycle-consistent generative adversarial network given CBCT. The features of CBCT and synthetic MRI (sMRI) are then extracted using dual pyramid networks for final delineation of organs. CBCT images and their corresponding manual contours were used as pairs to train and test the proposed model. Quantitative metrics including Dice similarity coefficient (DSC), Hausdorff distance 95% (HD95), mean surface distance, and residual mean square distance (RMS) were used to evaluate the proposed method. The proposed method was evaluated on a cohort of 65 HN cancer patients. CBCT images were collected from those patients who received proton therapy. Overall, DSC values of 0.87 ± 0.03, 0.79 ± 0.10/0.79 ± 0.11, 0.89 ± 0.08/0.89 ± 0.07, 0.90 ± 0.08, 0.75 ± 0.06/0.77 ± 0.06, 0.86 ± 0.13, 0.66 ± 0.14, 0.78 ± 0.05/0.77 ± 0.04, 0.96 ± 0.04, 0.89 ± 0.04/0.89 ± 0.04, 0.83 ± 0.02, and 0.84 ± 0.07 for commonly used OARs for treatment planning including brain stem, left/right cochlea, left/right eye, larynx, left/right lens, mandible, optic chiasm, left/right optic nerve, oral cavity, left/right parotid, pharynx, and spinal cord, respectively, were achieved. This study provides a rapid and accurate OAR auto-delineation approach, which can be used for adaptive radiation therapy.

A review on medical imaging synthesis using deep learning and its clinical applications

This paper reviewed the deep learning-based studies for medical imaging synthesis and its clinical application. Specifically, we summarized the recent developments of deep learning-based methods in inter- and intra-modality image synthesis by listing and highlighting the proposed methods, study designs, and reported performances with related clinical applications on representative studies. The challenges among the reviewed studies were then summarized with discussion.

An Automated Workflow to Improve Efficiency in Radiation Therapy Treatment Planning by Prioritizing Organs at Risk

PURPOSE

Manual delineation (MD) of organs at risk (OAR) is time and labor intensive. Auto-delineation (AD) can reduce the need for MD, but because current algorithms are imperfect, manual review and modification is still typically used. Recognizing that many OARs are sufficiently far from important dose levels that they do not pose a realistic risk, we hypothesize that some OARs can be excluded from MD and manual review with no clinical effect. The purpose of this study was to develop a method that automatically identifies these OARs and enables more efficient workflows that incorporate AD without degrading clinical quality.

METHODS AND MATERIALS

Preliminary dose map estimates were generated for n = 10 patients with head and neck cancers using only prescription and target-volume information. Conservative estimates of clinical OAR objectives were computed using AD structures with spatial expansion buffers to account for potential delineation uncertainties. OARs with estimated dose metrics below clinical tolerances were deemed low priority and excluded from MD and/or manual review. Final plans were then optimized using high-priority MD OARs and low-priority AD OARs and compared with reference plans generated using all MD OARs. Multiple different spatial buffers were used to accommodate different potential delineation uncertainties.

RESULTS

Sixty-seven out of 201 total OARs were identified as low-priority using the proposed methodology, which permitted a 33% reduction in structures requiring manual delineation/review. Plans optimized using low-priority AD OARs without review or modification met all planning objectives that were met when all MD OARs were used, indicating clinical equivalence.

CONCLUSIONS

Prioritizing OARs using estimated dose distributions allowed a substantial reduction in required MD and review without affecting clinically relevant dosimetry.

Unsupervised learning for deformable registration of thoracic CT and cone-beam CT based on multiscale features matching with spatially adaptive weighting

PURPOSE

Cone-beam computed tomography (CBCT) is a common on-treatment imaging widely used in image-guided radiotherapy. Fast and accurate registration between the on-treatment CBCT and planning CT is significant for and precise adaptive radiotherapy treatment (ART). However, existing CT-CBCT registration methods, which are mostly affine or time-consuming intensity- based deformation registration, still need further study due to the considerable CT-CBCT intensity discrepancy and the artifacts in low-quality CBCT images. In this paper, we propose a deep learning-based CT-CBCT registration model to promote rapid and accurate CT-CBCT registration for radiotherapy.

METHODS

The proposed CT-CBCT registration model consists of a registration network and an innovative deep similarity metric network. The registration network is a novel fully convolution network adapted specially for patch-wise CT-CBCT registration. The metric network, going beyond intensity, automatically evaluates the high-dimensional attribute-based dissimilarity between the registered CT and CBCT images. In addition, considering the artifacts in low-quality CBCT images, we add spatial weighting (SW) block to adaptively attach more importance to those informative voxels while inhibit the interference of artifact regions. Such SW-based metric network is expected to extract the most meaningful and discriminative deep features, and form a more reliable CT-CBCT similarity measure to train the registration network.

RESULTS

We evaluate the proposed method on clinical thoracic CBCT and CT dataset, and compare the registration results with some other common image similarity metrics and some state-of-the-art registration algorithms. The proposed method provides the highest Structural Similarity index (86.17 ± 5.09), minimum Target Registration Error of landmarks (2.37 ± 0.32 mm), and the best DSC coefficient (78.71 ± 10.95) of tumor volumes. Moreover, our model also obtains comparable distance error of lung surfaces (1.75 ± 0.35 mm).

CONCLUSION

The proposed model shows both efficiency and efficacy for reliable thoracic CT-CBCT registration, and can generate the matched CT and CBCT images within few seconds, which is of great significance to clinical radiotherapy.

Gum-Net: Unsupervised Geometric Matching for Fast and Accurate 3D Subtomogram Image Alignment and Averaging

We propose a Geometric unsupervised matching Network (Gum-Net) for finding the geometric correspondence between two images with application to 3D subtomogram alignment and averaging. Subtomogram alignment is the most important task in cryo-electron tomography (cryo-ET), a revolutionary 3D imaging technique for visualizing the molecular organization of unperturbed cellular landscapes in single cells. However, subtomogram alignment and averaging are very challenging due to severe imaging limits such as noise and missing wedge effects. We introduce an end-to-end trainable architecture with three novel modules specifically designed for preserving feature spatial information and propagating feature matching information. The training is performed in a fully unsupervised fashion to optimize a matching metric. No ground truth transformation information nor category-level or instance-level matching supervision information is needed. After systematic assessments on six real and nine simulated datasets, we demonstrate that Gum-Net reduced the alignment error by 40 to 50% and improved the averaging resolution by 10%. Gum-Net also achieved 70 to 110 times speedup in practice with GPU acceleration compared to state-of-the-art subtomogram alignment methods. Our work is the first 3D unsupervised geometric matching method for images of strong transformation variation and high noise level. The training code, trained model, and datasets are available in our open-source software AITom.

Pelvic multi-organ segmentation on cone-beam CT for prostate adaptive radiotherapy

BACKGROUND AND PURPOSE

The purpose of this study is to develop a deep learning-based approach to simultaneously segment five pelvic organs including prostate, bladder, rectum, left and right femoral heads on cone-beam CT (CBCT), as required elements for prostate adaptive radiotherapy planning.

MATERIALS AND METHODS

We propose to utilize both CBCT and CBCT-based synthetic MRI (sMRI) for the segmentation of soft tissue and bony structures, as they provide complementary information for pelvic organ segmentation. CBCT images have superior bony structure contrast and sMRIs have superior soft tissue contrast. Prior to segmentation, sMRI was generated using a cycle-consistent adversarial networks (CycleGAN), which was trained using paired CBCT-MR images. To combine the advantages of both CBCT and sMRI, we developed a cross-modality attention pyramid network with late feature fusion. Our method processes CBCT and sMRI inputs separately to extract CBCT-specific and sMRI-specific features prior to combining them in a late-fusion network for final segmentation. The network was trained and tested using 100 patients' datasets, with each dataset including the CBCT and manual physician contours. For comparison, we trained another two networks with different network inputs and architectures. The segmentation results were compared to manual contours for evaluations.

RESULTS

For the proposed method, dice similarity coefficients and mean surface distances between the segmentation results and the ground truth were 0.96 ± 0.03, 0.65 ± 0.67 mm; 0.91 ± 0.08, 0.93 ± 0.96 mm; 0.93 ± 0.04, 0.72 ± 0.61 mm; 0.95 ± 0.05, 1.05 ± 1.40 mm; and 0.95 ± 0.05, 1.08 ± 1.48 mm for bladder, prostate, rectum, left and right femoral heads, respectively. As compared to the other two competing methods, our method has shown superior performance in terms of the segmentation accuracy.

CONCLUSION

We developed a deep learning-based segmentation method to rapidly and accurately segment five pelvic organs simultaneously from daily CBCTs. The proposed method could be used in the clinic to support rapid target and organs-at-risk contouring for prostate adaptive radiation therapy.

"Dose of the day" based on cone beam computed tomography and deformable image registration for lung cancer radiotherapy

Purpose

Adaptive radiotherapy (ART) has potential to reduce toxicity and facilitate safe dose escalation. Dose calculations with the planning CT deformed to cone beam CT (CBCT) have shown promise for estimating the “dose of the day”. The purpose of this study is to investigate the “dose of the day” calculation accuracy based on CBCT and deformable image registration (DIR) for lung cancer radiotherapy.

Methods

A total of 12 lung cancer patients were identified, for which daily CBCT imaging was performed for treatment positioning. A re‐planning CT (rCT) was acquired after 20 Gy for all patients. A virtual CT (vCT) was created by deforming initial planning CT (pCT) to the simulated CBCT that was generated from deforming CBCT to rCT acquired on the same day. Treatment beams from the initial plan were copied to the vCT and rCT for dose calculation. Dosimetric agreement between vCT‐based and rCT‐based accumulated doses was evaluated using the Bland‐Altman analysis.

Results

Mean differences in dose‐volume metrics between vCT and rCT were smaller than 1.5%, and most discrepancies fell within the range of ± 5% for the target volume, lung, esophagus, and heart. For spinal cord D_max, a large mean difference of −5.55% was observed, which was largely attributed to very limited CBCT image quality (e.g., truncation artifacts).

Conclusion

This study demonstrated a reasonable agreement in dose‐volume metrics between dose accumulation based on vCT and rCT, with the exception for cases with poor CBCT image quality. These findings suggest potential utility of vCT for providing a reasonable estimate of the “dose of the day”, and thus facilitating the process of ART for lung cancer.

Deformable image registration for radiation therapy: principle, methods, applications and evaluation

Background: Deformable image registration (DIR) is increasingly used in the field of radiation therapy (RT) to account for anatomical deformations. The aims of this paper are to describe the main applications of DIR in RT and discuss current DIR evaluation methods. Methods: Articles on DIR published from January 2000 to October 2018 were extracted from PubMed and Science Direct. Our search was restricted to articles that report data obtained from humans, were written in English, and address DIR methods for RT. A total of 207 articles were selected from among 2506 identified in the search process. Results: At planning, DIR is used for organ delineation using atlas-based segmentation, deformation-based planning target volume definition, functional planning and magnetic resonance imaging-based dose calculation. In image-guided RT, DIR is used for contour propagation and dose calculation on per-treatment imaging. DIR is also used to determine the accumulated dose from fraction to fraction in external beam RT and brachytherapy, both for dose reporting and adaptive RT. In the case of re-irradiation, DIR can be used to estimate the cumulated dose of the two irradiations. Finally, DIR can be used to predict toxicity in voxel-wise population analysis. However, the evaluation of DIR remains an open issue, especially when dealing with complex cases such as the disappearance of matter. To quantify DIR uncertainties, most evaluation methods are limited to geometry-based metrics. Software companies have now integrated DIR tools into treatment planning systems for clinical use, such as contour propagation and fraction dose accumulation. Conclusions: DIR is increasingly important in RT applications, from planning to toxicity prediction. DIR is routinely used to reduce the workload of contour propagation. However, its use for complex dosimetric applications must be carefully evaluated by combining quantitative and qualitative analyses.

Dosimetric study on learning-based cone-beam CT correction in adaptive radiation therapy

INTRODUCTION

Cone-beam CT (CBCT) image quality is important for its quantitative analysis in adaptive radiation therapy. However, due to severe artifacts, the CBCTs are primarily used for verifying patient setup only so far. We have developed a learning-based image quality improvement method which could provide CBCTs with image quality comparable to planning CTs (pCTs). The accuracy of dose calculations based on these CBCTs is unknown. In this study, we aim to investigate the dosimetric accuracy of our corrected CBCT (CCBCT) in brain stereotactic radiosurgery (SRS) and pelvic radiotherapy.

MATERIALS AND METHODS

We retrospectively investigated a total of 32 treatment plans from 22 patients, each of whom with both original treatment pCTs and CBCTs acquired during treatment setup. The CCBCT and original CBCT (OCBCT) were registered to the pCT for generating CCBCT-based and OCBCT-based treatment plans. The original pCT-based plans served as ground truth. Clinically-relevant dose volume histogram (DVH) metrics were extracted from the ground truth, OCBCT-based and CCBCT-based plans for comparison. Gamma analysis was also performed to compare the absorbed dose distributions between the pCT-based and OCBCT/CCBCT-based plans of each patient.

RESULTS

CCBCTs demonstrated better image contrast and more accurate HU ranges when compared side-by-side with OCBCTs. For pelvic radiotherapy plans, the mean dose error in DVH metrics for planning target volume (PTV), bladder and rectum was significantly reduced, from 1% to 0.3%, after CBCT correction. The gamma analysis showed the average pass rate increased from 94.5% before correction to 99.0% after correction. For brain SRS treatment plans, both original and corrected CBCT images were accurate enough for dose calculation, though CCBCT featured higher image quality.

CONCLUSION

CCBCTs can provide a level of dose accuracy comparable to traditional pCTs for brain and prostate radiotherapy planning and the correction method proposed here can be useful in CBCT-guided adaptive radiotherapy.

Application of aSi-kVCBCT for Volume Assessment and Dose Estimation: An Offline Adaptive Study for Prostate Radiotherapy

Objective:

The purpose of this study is to develop a method to estimate the dose using amorphous silicon detector panel cone beam computed tomography (aSi-kVCBCT) for the OARs and targets in prostate radiotherapy and to compare with the actual planned dose.

Methods:

The aSi-kVCBCT is used widely in radiotherapy to verify the patient position before treatment. The advancement in aSi-kVCBCT combined with adaptive software allows us to verify the dose distribution in daily acquired CBCT images. CBCT images from 10 patients undergoing radical prostate radiotherapy were included in this study. Patients received total dose of 65Gy in 25 fractions using volumetric modulated arc therapy (VMAT). aSi-kVCBCT scans were acquired before daily treatment and exported to smart adapt software for image adaptation. The planning CT is adapted to daily aSi-kVCBCT images in terms of HU mapping. The primary VMAT plans were copied on to the adapted planning CT images and dose was calculated using Anisotropic Analytic Algorithm (AAA). The DVH is then used to evaluate the volume changes of organs at risk (OAR), the actual dose received by OARs, CTV and PTV during a single fraction.

Results:

The normalized volume of the bladder and rectum ranged from 0.70–1.66 and 0.70–1.16 respectively. The cumulative mean Sorensen–Dice coefficient values of bladder and rectum were 0.89±0.04 and 0.79±0.06 respectively. The maximum dose differences for CTV and PTV were 2.5% and -4.7% and minimum were 0.1% and 0.1% respectively.

Conclusion:

The adapted planning CT obtained from daily imaging using aSi-kVCBCT and SmartAdapt^® can be used as an effective tool to estimate the volume changes and dose difference in prostate radiotherapy.

A density assignment method for dose monitoring in head-and-neck radiotherapy

BACKGROUND AND PURPOSE

During head-and-neck (H&N) radiotherapy, the parotid glands (PGs) may be overdosed; thus, a tool is required to monitor the delivered dose. This study aimed to assess the dose accuracy of a patient-specific density assignment method (DAM) for dose calculation to monitor the dose to PGs during treatment.

PATIENTS AND METHODS

Forty patients with H&N cancer received an intensity modulated radiation therapy (IMRT), among whom 15 had weekly CTs. Dose distributions were calculated either on the CTs (CT_ref), on one-class CTs (1C-CT, water), or on three-class CTs (3C-CT, water-air-bone). The inter- and intra-patient DAM uncertainties were evaluated by the difference between doses calculated on CT_ref and 1C-CTs or 3C-CTs. PG mean dose (D_mean) and spinal cord maximum dose (D_2%) were considered. The cumulated dose to the PGs was estimated by the mean D_mean of the weekly CTs.

RESULTS

The mean (maximum) inter-patient DAM dose uncertainties for the PGs (in cGy) were 23 (75) using 1C-CTs and 12 (50) using 3C-CTs (p ≤ 0.001). For the spinal cord, these uncertainties were 118 (245) and 15 (67; p ≤ 0.001). The mean (maximum) DAM dose uncertainty between cumulated doses calculated on CTs and 3C-CTs was 7 cGy (45 cGy) for the PGs. Considering the difference between the planned and cumulated doses, 53% of the ipsilateral and 80% of the contralateral PGs were overdosed by +3.6 Gy (up to 8.2 Gy) and +1.9 Gy (up to 5.2 Gy), respectively.

CONCLUSION

The uncertainty of the three-class DAM appears to be clinically non-significant (<0.5 Gy) compared with the PG overdose (up to 8.2 Gy). This DAM could therefore be used to monitor PG doses and trigger replanning.

Design and development of a nonrigid phantom for the quantitative evaluation of DIR-based mapping of simulated pulmonary ventilation

PURPOSE

The validation of deformable image registration (DIR)-based pulmonary ventilation mapping is time consuming and prone to inaccuracies and is also affected by deformation parameters. In this study, we developed a nonrigid phantom as a quality assurance (QA) tool that simulates ventilation to evaluate DIR-based images quantitatively.

METHODS

The phantom consists of an acrylic cylinder filled with polyurethane foam designed to simulate pulmonic alveoli. A polyurethane membrane is attached to the inferior end of the phantom to simulate the diaphragm. In addition, tracheobronchial-tree-shaped polyurethane tubes are inserted through the foam and converge outside the phantom to simulate the trachea. Solid polyurethane is also used to model arteries, which closely follow the model airways. Two three-dimensional (3D) CT scans were performed during exhalation and inhalation phases using xenon (Xe) gas as the inhaled contrast agent. The exhalation 3D-CT image is deformed to an inhalation 3D-CT image using our in-house program based on the NiftyReg open-source package. The target registration error (TRE) between the two images was calculated for 16 landmarks located in the simulated lung volume. The DIR-based ventilation image was generated using Jacobian determinant (JD) metrics. Subsequently, differences in the Hounsfield unit (HU) values between the two images were measured. The correlation coefficient between the JD and HU differences was calculated. In addition, three 4D-CT scans are performed to evaluate the reproducibility of the phantom motion and Xe gas distribution.

RESULTS

The phantom exhibited a variety of displacements for each landmark (range: 1-20 mm). The reproducibility analysis indicated that the location differences were <1 mm for all landmarks, and the HU variation in the Xe gas distribution was close to zero. The mean TRE in the evaluation of spatial accuracy according to the DIR software was 1.47 ± 0.71 mm (maximum: 2.6 mm). The relationship between the JD and HU differences had a large correlation (R = -0.71) for the DIR software.

CONCLUSION

The phantom implemented new features, namely, deformation and simulated ventilation. To assess the accuracy of the DIR-based mapping of the simulated pulmonary ventilation, the phantom allows for simulation of Xe gas wash-in and wash-out. The phantom may be an effective QA tool, because the DIR algorithm can be quickly changed and its accuracy evaluated with a high degree of precision.

A graphical approach to optimizing variable-kernel smoothing parameters for improved deformable registration of CT and cone beam CT images

A systematic method is presented for determining optimal parameters in variable-kernel deformable image registration of cone beam CT and CT images, in order to improve accuracy and convergence for potential use in online adaptive radiotherapy. Assessed conditions included the noise constant (symmetric force demons), the kernel reduction rate, the kernel reduction percentage, and the kernel adjustment criteria. Four such parameters were tested in conjunction with reductions of 5, 10, 15, 20, 30, and 40%. Noise constants ranged from 1.0 to 1.9 for pelvic images in ten prostate cancer patients. A total of 516 tests were performed and assessed using the structural similarity index. Registration accuracy was plotted as a function of iteration number and a least-squares regression line was calculated, which implied an average improvement of 0.0236% per iteration. This baseline was used to determine if a given set of parameters under- or over-performed. The most accurate parameters within this range were applied to contoured images. The mean Dice similarity coefficient was calculated for bladder, prostate, and rectum with mean values of 98.26%, 97.58%, and 96.73%, respectively; corresponding to improvements of 2.3%, 9.8%, and 1.2% over previously reported values for the same organ contours. This graphical approach to registration analysis could aid in determining optimal parameters for Demons-based algorithms. It also establishes expectation values for convergence rates and could serve as an indicator of non-physical warping, which often occurred in cases  >0.6% from the regression line.

Transmission characteristics of a two dimensional antiscatter grid prototype for CBCT

AIM

High fraction of scattered radiation in cone-beam CT (CBCT) imaging degrades CT number accuracy and visualization of low contrast objects. To suppress scatter in CBCT projections, we developed a focused, two-dimensional antiscatter grid (2DASG) prototype. In this work, we report on the primary and scatter transmission characteristics of the 2DASG prototype aimed for linac mounted, offset detector geometry CBCT systems in radiation therapy, and compared its performance to a conventional one-dimensional ASG (1DASG).

METHODS

The 2DASG is an array of through-holes separated by 0.1 mm septa that was fabricated from tungsten using additive manufacturing techniques. Through-holes' focusing geometry was designed for offset detector CBCT in Varian TrueBeam system. Two types of ASGs were evaluated: (a) a conventional 1DASG with a grid ratio of 10, (b) the 2DASG prototype with a grid ratio of 8.2. To assess the scatter suppression performance of both ASGs, Scatter-to-primary ratio (SPR) and scatter transmission fraction (Ts) were measured using the beam stop method. Scatter and primary intensities were modulated by varying the phantom thickness between 10 and 40 cm. Additionally, the effect of air gap and bow tie (BT) filter on SPR and Ts were evaluated. Average primary transmission fraction (T_P ) and pixel specific primary transmission were also measured for both ASGs. To assess the effect of transmission characteristics on projection image signal-to-noise ratio (SNR), SNR improvement factor was calculated. Improvement in contrast to noise ratio (CNR) was demonstrated using a low contrast object.

RESULTS

In comparison to 1DASG, 2DASG reduced SPRs by a factor of 3 to 6 across the range of phantom setups investigated. Ts values for 1D and 2DASGs were in the range of 21 to 29%, and 5 to 14% respectively. 2DASG continued to provide lower SPR and Ts at increased air gap and with BT filter. Tp of 1D and 2DASGs were 70.6% and 84.7% respectively. Due to the septal shadow of the 2DASG, its pixel specific primary transmission values varied between 32.5% and 99.1%. With respect to 1DASG, 2DASG provided up to factor of 1.7 more improvement in SNR across the SPR range investigated. Moreover, 2DASG provided improved visualization of low contrast objects with respect to 1DASG and NOASG setups.

CONCLUSIONS

When compared to a conventional 1DASG, 2DASG prototype provided noticeably lower SPR and Ts values, indicating its superior scatter suppression performance. 2DASG also provided 19% higher average primary transmission that was attributed to the absence of interseptal spacers and optimized grid geometry. Our results indicate that the combined effect of lower scatter and higher primary transmission provided by 2DASG may potentially translate into more accurate CT numbers and improved contrast resolution in CBCT images.

Evaluation of deformable image registration for contour propagation between CT and cone-beam CT images in adaptive head and neck radiotherapy

Deformable image registration (DIR) is a critical technic in adaptive radiotherapy (ART) to propagate contours between planning computerized tomography (CT) images and treatment CT/Cone-beam CT (CBCT) image to account for organ deformation for treatment re-planning. To validate the ability and accuracy of DIR algorithms in organ at risk (OAR) contours mapping, seven intensity-based DIR strategies are tested on the planning CT and weekly CBCT images from six Head & Neck cancer patients who underwent a 6 ∼ 7 weeks intensity-modulated radiation therapy (IMRT). Three similarity metrics, i.e. the Dice similarity coefficient (DSC), the percentage error (PE) and the Hausdorff distance (HD), are employed to measure the agreement between the propagated contours and the physician delineated ground truths. It is found that the performance of all the evaluated DIR algorithms declines as the treatment proceeds. No statistically significant performance difference is observed between different DIR algorithms (p> 0.05), except for the double force demons (DFD) which yields the worst result in terms of DSC and PE. For the metric HD, all the DIR algorithms behaved unsatisfactorily with no statistically significant performance difference (p= 0.273). These findings suggested that special care should be taken when utilizing the intensity-based DIR algorithms involved in this study to deform OAR contours between CT and CBCT, especially for those organs with low contrast.

Use of image registration and fusion algorithms and techniques in radiotherapy: Report of the AAPM Radiation Therapy Committee Task Group No. 132

Image registration and fusion algorithms exist in almost every software system that creates or uses images in radiotherapy. Most treatment planning systems support some form of image registration and fusion to allow the use of multimodality and time-series image data and even anatomical atlases to assist in target volume and normal tissue delineation. Treatment delivery systems perform registration and fusion between the planning images and the in-room images acquired during the treatment to assist patient positioning. Advanced applications are beginning to support daily dose assessment and enable adaptive radiotherapy using image registration and fusion to propagate contours and accumulate dose between image data taken over the course of therapy to provide up-to-date estimates of anatomical changes and delivered dose. This information aids in the detection of anatomical and functional changes that might elicit changes in the treatment plan or prescription. As the output of the image registration process is always used as the input of another process for planning or delivery, it is important to understand and communicate the uncertainty associated with the software in general and the result of a specific registration. Unfortunately, there is no standard mathematical formalism to perform this for real-world situations where noise, distortion, and complex anatomical variations can occur. Validation of the software systems performance is also complicated by the lack of documentation available from commercial systems leading to use of these systems in undesirable 'black-box' fashion. In view of this situation and the central role that image registration and fusion play in treatment planning and delivery, the Therapy Physics Committee of the American Association of Physicists in Medicine commissioned Task Group 132 to review current approaches and solutions for image registration (both rigid and deformable) in radiotherapy and to provide recommendations for quality assurance and quality control of these clinical processes.

Comprehensive evaluation of ten deformable image registration algorithms for contour propagation between CT and cone-beam CT images in adaptive head & neck radiotherapy

Deformable image registration (DIR) is a critical technic in adaptive radiotherapy (ART) for propagating contours between planning computerized tomography (CT) images and treatment CT/cone-beam CT (CBCT) images to account for organ deformation for treatment re-planning. To validate the ability and accuracy of DIR algorithms in organ at risk (OAR) contour mapping, ten intensity-based DIR strategies, which were classified into four categories—optical flow-based, demons-based, level-set-based and spline-based—were tested on planning CT and fractional CBCT images acquired from twenty-one head & neck (H&N) cancer patients who underwent 6~7-week intensity-modulated radiation therapy (IMRT). Three similarity metrics, i.e., the Dice similarity coefficient (DSC), the percentage error (PE) and the Hausdorff distance (HD), were employed to measure the agreement between the propagated contours and the physician-delineated ground truths of four OARs, including the vertebra (VTB), the vertebral foramen (VF), the parotid gland (PG) and the submandibular gland (SMG). It was found that the evaluated DIRs in this work did not necessarily outperform rigid registration. DIR performed better for bony structures than soft-tissue organs, and the DIR performance tended to vary for different ROIs with different degrees of deformation as the treatment proceeded. Generally, the optical flow-based DIR performed best, while the demons-based DIR usually ranked last except for a modified demons-based DISC used for CT-CBCT DIR. These experimental results suggest that the choice of a specific DIR algorithm depends on the image modality, anatomic site, magnitude of deformation and application. Therefore, careful examinations and modifications are required before accepting the auto-propagated contours, especially for automatic re-planning ART systems.

Evaluation of Deformable Image Registration-Based Contour Propagation From Planning CT to Cone-Beam CT

Purpose:

We evaluated the performance of organ contour propagation from a planning computed tomography to cone-beam computed tomography with deformable image registration by comparing contours to manual contouring.

Materials and Methods:

Sixteen patients were retrospectively identified based on showing considerable physical change throughout the course of treatment. Multiple organs in the 3 regions (head and neck, prostate, and pancreas) were evaluated. A cone-beam computed tomography from the end of treatment was registered to the planning computed tomography using rigid registration, followed by deformable image registration. The contours were copied on cone-beam computed tomography image sets using rigid registration and modified by 2 radiation oncologists. Contours were compared using Dice similarity coefficient, mean surface distance, and Hausdorff distance.

Results:

The mean physician-to-physician Dice similarity coefficient for all organs was 0.90. When compared to each physician’s contours, the overall mean for rigid was 0.76 (P < .001), and it was improved to 0.79 (P < .001) for deformable image registration. Comparing deformable image registration to physicians resulted in a mean Dice similarity coefficient of 0.77, 0.74, and 0.84 for head and neck, prostate, and pancreas groups, respectively; whereas, the physician-to-physician mean agreement for these sites was 0.87, 0.90, and 0.93 (P < .001, for all sites). The mean surface distance for physician-to-physician contours was 1.01 mm, compared to 2.58 mm for rigid-to-physician contours and 2.24 mm for deformable image registration-to-physician contours. The mean physician-to-physician Hausdorff distance was 11.32 mm, and when compared to any physician’s contours, the mean for rigid and deformable image registration was 12.1 mm and 12.0 mm (P < .001), respectively.

Conclusion:

The physicians had a high level of agreement via the 3 metrics; however, deformable image registration fell short of this level of agreement. The automatic workflows using deformable image registration to deform contours to cone-beam computed tomography to evaluate the changes during treatment should be used with caution.

Deformable registration of CT and cone-beam CT with local intensity matching

Cone-beam CT (CBCT) is a widely used intra-operative imaging modality in image-guided radiotherapy and surgery. A short scan followed by a filtered-backprojection is typically used for CBCT reconstruction. While data on the mid-plane (plane of source-detector rotation) is complete, off-mid-planes undergo different information deficiency and the computed reconstructions are approximate. This causes different reconstruction artifacts at off-mid-planes depending on slice locations, and therefore impedes accurate registration between CT and CBCT. In this paper, we propose a method to accurately register CT and CBCT by iteratively matching local CT and CBCT intensities. We correct CBCT intensities by matching local intensity histograms slice by slice in conjunction with intensity-based deformable registration. The correction-registration steps are repeated in an alternating way until the result image converges. We integrate the intensity matching into three different deformable registration methods, B-spline, demons, and optical flow that are widely used for CT-CBCT registration. All three registration methods were implemented on a graphics processing unit for efficient parallel computation. We tested the proposed methods on twenty five head and neck cancer cases and compared the performance with state-of-the-art registration methods. Normalized cross correlation (NCC), structural similarity index (SSIM), and target registration error (TRE) were computed to evaluate the registration performance. Our method produced overall NCC of 0.96, SSIM of 0.94, and TRE of 2.26 → 2.27 mm, outperforming existing methods by 9%, 12%, and 27%, respectively. Experimental results also show that our method performs consistently and is more accurate than existing algorithms, and also computationally efficient.

Accuracy of software-assisted contour propagation from planning CT to cone beam CT in head and neck radiotherapy

BACKGROUND

Autocontouring improves workflow in computed tomography (CT)-based dose planning, but could also potentially play a role for optimal use of daily cone beam CT (CBCT) in adaptive radiotherapy. This study aims to determine the accuracy of a deformable image registration (DIR) algorithm for organs at risk (OAR) in the neck region, when applied to CBCT.

MATERIAL AND METHODS

For 30 head and neck cancer (HNC) patients 14 OARs including parotid glands, swallowing structures and spinal cord were delineated. Contours were propagated by DIR from CT to the CBCTs of the first and last treatment fraction. An indirect approach, propagating contours to the first CBCT and from there to the last CBCT was also tested. Propagated contours were compared to manually corrected contours by Dice similarity coefficient (DSC) and Hausdorff distance (HD). Dose was recalculated on CBCTs and dosimetric consequences of uncertainties in DIR were reviewed.

RESULTS

Mean DSC values of ≥0.8 were considered adequate and were achieved in tongue base (0.91), esophagus (0.85), glottic (0.81) and supraglottic larynx (0.83), inferior pharyngeal constrictor muscle (0.84), spinal cord (0.89) and all salivary glands in the first CBCT. For the last CBCT by direct propagation, adequate DSC values were achieved for tongue base (0.85), esophagus (0.84), spinal cord (0.87) and all salivary glands. Using indirect propagation only tongue base (0.80) and parotid glands (0.87) were ≥0.8. Mean relative dose difference between automated and corrected contours was within ±2.5% of planed dose except for esophagus inlet (-4.5%) and esophagus (5.0%) for the last CBCT using indirect propagation.

CONCLUSION

Compared to manually corrected contours, the DIR algorithm was accurate for use in CBCT images of HNC patients and the minor inaccuracies had little consequence for mean dose in most clinically relevant OAR. The method can thus enable a more automated segmentation of CBCT for use in adaptive radiotherapy.

Feasibility of offline head & neck adaptive radiotherapy using deformed planning CT electron density mapping on weekly cone beam computed tomography

OBJECTIVE

The purpose of the study was to use deformable mapping of planning CT (pCT) electron density values on weekly cone-beam CT (CBCT) to quantify the anatomical changes and determine the dose-volume relationship in offline adaptive volumetric-modulated arc therapy.

METHODS

10 patients treated with RapidArc plans who had weekly CBCTs were selected retrospectively. The pCT was deformed to weekly CBCTs and the deformed contours were checked for any discrepancies. Clinical target volume 66 Gy and 60 Gy (CTV₆₆ and CTV₆₀), parotids and spinal cord were the structures selected for analysis. Volume reduction and dice similarity index (DSI) were determined. Hybrid RapidArc plans were created and the cumulative dose-volume histograms for selected structures were analyzed.

RESULTS

Results showed a mean volume reduction of 18.82 ± 6.08% and 18.22 ± 6.1% for Clinical target volume 66 Gy and 60 Gy (CTV₆₆ and CTV₆₀), respectively, and their corresponding DSI values were 0.94 ± 0.03 and 0.95 ± 0.01. Mean volume reductions of left and right parotids were 32.79 ± 10.28% and 29.46 ± 8.78%, respectively, and their corresponding mean DSI values were 0.90 ± 0.05 and 0.89 ± 0.05. The cumulative mean dose difference for Planning target volume 66 Gy (PTV₆₆) was -1.35 ± 1.71% and for Planning target volume 60 Gy (PTV₆₀), it was -0.69 ± 1.37%. Spinal cord doses varied for all patients over the course.

CONCLUSION

The results from the study showed that it is clinically feasible to estimate the dose-volume relationship using deformed pCT. Monitoring of patient anatomic changes and incorporating patient-specific replanning strategy are necessary to avoid critical structure complications. Advances in knowledge: Deformable mapping of pCT electron density values on weekly CBCTs has been performed to establish the volumetric and dosimetric changes. The anatomical changes differ among the patients and hence, the choice for adaptive radiotherapy should be strictly patient specific rather than time specific.

Technical Note: scuda: A software platform for cumulative dose assessment

PURPOSE

Accurate tracking of anatomical changes and computation of actually delivered dose to the patient are critical for successful adaptive radiation therapy (ART). Additionally, efficient data management and fast processing are practically important for the adoption in clinic as ART involves a large amount of image and treatment data. The purpose of this study was to develop an accurate and efficient Software platform for CUmulative Dose Assessment (scuda) that can be seamlessly integrated into the clinical workflow.

METHODS

scuda consists of deformable image registration (DIR), segmentation, dose computation modules, and a graphical user interface. It is connected to our image PACS and radiotherapy informatics databases from which it automatically queries/retrieves patient images, radiotherapy plan, beam data, and daily treatment information, thus providing an efficient and unified workflow. For accurate registration of the planning CT and daily CBCTs, the authors iteratively correct CBCT intensities by matching local intensity histograms during the DIR process. Contours of the target tumor and critical structures are then propagated from the planning CT to daily CBCTs using the computed deformations. The actual delivered daily dose is computed using the registered CT and patient setup information by a superposition/convolution algorithm, and accumulated using the computed deformation fields. Both DIR and dose computation modules are accelerated by a graphics processing unit.

RESULTS

The cumulative dose computation process has been validated on 30 head and neck (HN) cancer cases, showing 3.5 ± 5.0 Gy (mean±STD) absolute mean dose differences between the planned and the actually delivered doses in the parotid glands. On average, DIR, dose computation, and segmentation take 20 s/fraction and 17 min for a 35-fraction treatment including additional computation for dose accumulation.

CONCLUSIONS

The authors developed a unified software platform that provides accurate and efficient monitoring of anatomical changes and computation of actually delivered dose to the patient, thus realizing an efficient cumulative dose computation workflow. Evaluation on HN cases demonstrated the utility of our platform for monitoring the treatment quality and detecting significant dosimetric variations that are keys to successful ART.

Open source deformable image registration system for treatment planning and recurrence CT scans : Validation in the head and neck region

BACKGROUND

Clinical application of deformable registration (DIR) of medical images remains limited due to sparse validation of DIR methods in specific situations, e. g. in case of cancer recurrences. In this study the accuracy of DIR for registration of planning CT (pCT) and recurrence CT (rCT) images of head and neck squamous cell carcinoma (HNSCC) patients was evaluated.

PATIENTS AND MATERIALS

Twenty patients treated with definitive IMRT for HNSCC in 2010-2012 were included. For each patient, a pCT and an rCT scan were used. Median interval between the scans was 8.5 months. One observer manually contoured eight anatomical regions-of-interest (ROI) twice on pCT and once on rCT.

METHODS

pCT and rCT images were deformably registered using the open source software elastix. Mean surface distance (MSD) and Dice similarity coefficient (DSC) between contours were used for validation of DIR. A measure for delineation uncertainty was estimated by assessing MSD from the re-delineations of the same ROI on pCT. DIR and manual contouring uncertainties were correlated with tissue volume and rigidity.

RESULTS

MSD varied 1-3 mm for different ROIs for DIR and 1-1.5 mm for re-delineated ROIs performed on pCT. DSC for DIR varied between 0.58 and 0.79 for soft tissues and was 0.79 or higher for bony structures, and correlated with the volumes of ROIs (r = 0.5, p < 0.001) and tissue rigidity (r = 0.54, p < 0.001).

CONCLUSION

DIR using elastix in HNSCC on planning and recurrence CT scans is feasible; an uncertainty of the method is close to the voxel size length of the planning CT images.

Quantitative assessment of anatomical change using a virtual proton depth radiograph for adaptive head and neck proton therapy

The aim of this work is to demonstrate the feasibility of using water-equivalent thickness (WET) and virtual proton depth radiographs (PDRs) of intensity corrected cone-beam computed tomography (CBCT) to detect anatomical change and patient setup error to trigger adaptive head and neck proton therapy. The planning CT (pCT) and linear accelerator (linac) equipped CBCTs acquired weekly during treatment of a head and neck patient were used in this study. Deformable image registration (DIR) was used to register each CBCT with the pCT and map Hounsfield units (HUs) from the planning CT (pCT) onto the daily CBCT. The deformed pCT is referred as the corrected CBCT (cCBCT). Two dimensional virtual lateral PDRs were generated using a ray-tracing technique to project the cumulative WET from a virtual source through the cCBCT and the pCT onto a virtual plane. The PDRs were used to identify anatomic regions with large variations in the proton range between the cCBCT and pCT using a threshold of 3 mm relative difference of WET and 3 mm search radius criteria. The relationship between PDR differences and dose distribution is established. Due to weight change and tumor response during treatment, large variations in WETs were observed in the relative PDRs which corresponded spatially with an increase in the number of failing points within the GTV, especially in the pharynx area. Failing points were also evident near the posterior neck due to setup variations. Differences in PDRs correlated spatially to differences in the distal dose distribution in the beam's eye view. Virtual PDRs generated from volumetric data, such as pCTs or CBCTs, are potentially a useful quantitative tool in proton therapy. PDRs and WET analysis may be used to detect anatomical change from baseline during treatment and trigger further analysis in adaptive proton therapy.

The suitability of common metrics for assessing parotid and larynx autosegmentation accuracy

Contouring structures in the head and neck is time‐consuming, and automatic segmentation is an important part of an adaptive radiotherapy workflow. Geometric accuracy of automatic segmentation algorithms has been widely reported, but there is no consensus as to which metrics provide clinically meaningful results. This study investigated whether geometric accuracy (as quantified by several commonly used metrics) was associated with dosimetric differences for the parotid and larynx, comparing automatically generated contours against manually drawn ground truth contours. This enabled the suitability of different commonly used metrics to be assessed for measuring automatic segmentation accuracy of the parotid and larynx. Parotid and larynx structures for 10 head and neck patients were outlined by five clinicians to create ground truth structures. An automatic segmentation algorithm was used to create automatically generated normal structures, which were then used to create volumetric‐modulated arc therapy plans. The mean doses to the automatically generated structures were compared with those of the corresponding ground truth structures, and the relative difference in mean dose was calculated for each structure. It was found that this difference did not correlate with the geometric accuracy provided by several metrics, notably the Dice similarity coefficient, which is a commonly used measure of spatial overlap. Surface‐based metrics provided stronger correlation and are, therefore, more suitable for assessing automatic segmentation of the parotid and larynx.

PACS number(s): 87.57.nm, 87.55.D, 87.55.Qr

The Utility of Cloud Computing in Analyzing GPU-Accelerated Deformable Image Registration of CT and CBCT Images in Head and Neck Cancer Radiation Therapy

The images generated during radiation oncology treatments provide a valuable resource to conduct analysis for personalized therapy, outcomes prediction, and treatment margin optimization. Deformable image registration (DIR) is an essential tool in analyzing these images. We are enhancing and examining DIR with the contributions of this paper: 1) implementing and investigating a cloud and graphic processing unit (GPU) accelerated DIR solution and 2) assessing the accuracy and flexibility of that solution on planning computed tomography (CT) with cone-beam CT (CBCT). Registering planning CTs and CBCTs aids in monitoring tumors, tracking body changes, and assuring that the treatment is executed as planned. This provides significant information not only on the level of a single patient, but also for an oncology department. However, traditional methods for DIR are usually time-consuming, and manual intervention is sometimes required even for a single registration. In this paper, we present a cloud-based solution in order to increase the data analysis throughput, so that treatment tracking results may be delivered at the time of care. We assess our solution in terms of accuracy and flexibility compared with a commercial tool registering CT with CBCT. The latency of a previously reported mutual information-based DIR algorithm was improved with GPUs for a single registration. This registration consists of rigid registration followed by volume subdivision-based nonrigid registration. In this paper, the throughput of the system was accelerated on the cloud for hundreds of data analysis pairs. Nine clinical cases of head and neck cancer patients were utilized to quantitatively evaluate the accuracy and throughput. Target registration error (TRE) and structural similarity index were utilized as evaluation metrics for registration accuracy. The total computation time consisting of preprocessing the data, running the registration, and analyzing the results was used to evaluate the system throughput. Evaluation showed that the average TRE for GPU-accelerated DIR for each of the nine patients was from 1.99 to 3.39 mm, which is lower than the voxel dimension. The total processing time for 282 pairs on an Amazon Web Services cloud consisting of 20 GPU enabled nodes took less than an hour. Beyond the original registration, the cloud resources also included automatic registration quality checks with minimal impact to timing. Clinical data were utilized in quantitative evaluations, and the results showed that the presented method holds great potential for many high-impact clinical applications in radiation oncology, including adaptive radio therapy, patient outcomes prediction, and treatment margin optimization.

A finite element head and neck model as a supportive tool for deformable image registration

PURPOSE

A finite element (FE) head and neck model was developed as a tool to aid investigations and development of deformable image registration and patient modeling in radiation oncology. Useful aspects of a FE model for these purposes include ability to produce realistic deformations (similar to those seen in patients over the course of treatment) and a rational means of generating new configurations, e.g., via the application of force and/or displacement boundary conditions.

METHODS

The model was constructed based on a cone-beam computed tomography image of a head and neck cancer patient. The three-node triangular surface meshes created for the bony elements (skull, mandible, and cervical spine) and joint elements were integrated into a skeletal system and combined with the exterior surface. Nodes were additionally created inside the surface structures which were composed of the three-node triangular surface meshes, so that four-node tetrahedral FE elements were created over the whole region of the model. The bony elements were modeled as a homogeneous linear elastic material connected by intervertebral disks. The surrounding tissues were modeled as a homogeneous linear elastic material. Under force or displacement boundary conditions, FE analysis on the model calculates approximate solutions of the displacement vector field.

RESULTS

A FE head and neck model was constructed that skull, mandible, and cervical vertebrae were mechanically connected by disks. The developed FE model is capable of generating realistic deformations that are strain-free for the bony elements and of creating new configurations of the skeletal system with the surrounding tissues reasonably deformed.

CONCLUSIONS

The FE model can generate realistic deformations for skeletal elements. In addition, the model provides a way of evaluating the accuracy of image alignment methods by producing a ground truth deformation and correspondingly simulated images. The ability to combine force and displacement conditions provides flexibility for simulating realistic anatomic configurations.

Cone Beam Computed Tomography: The Challenges and Strategies in Its Application for Dose Accumulation

Online image guidance using cone beam computed tomography (CBCT) has greatly improved the geometric precision of radiotherapy. Changes in anatomy are common during a course of fractionated treatment, resulting in dose deviation from the planned distribution. There is increased interest in performing dose accumulation to compute the actual delivered dose and to adapt the treatment when necessary. This can be achieved by delineating the volume of interest and by generating "dose of the day" through dose computation on the CBCT. However, the image quality and the accuracy of the CT number of CBCT are deemed to be inferior to fan beam CT, which increases the uncertainty associated in this process. A review of literature was conducted to assess the reliability of and to examine strategies for overcoming the challenges in using CBCT for volume delineation and dose computation. The review demonstrates that the uncertainty varies across body sites, and different strategies have been recommended to generate comparable results to images from CT simulators. This facilitates a better understanding of the potential and the limitation of using CBCT for dose accumulation.

Automated planning of breast radiotherapy using cone beam CT imaging

PURPOSE

Develop and clinically validate a methodology for using cone beam computed tomography (CBCT) imaging in an automated treatment planning framework for breast IMRT.

METHODS

A technique for intensity correction of CBCT images was developed and evaluated. The technique is based on histogram matching of CBCT image sets, using information from "similar" planning CT image sets from a database of paired CBCT and CT image sets (n = 38). Automated treatment plans were generated for a testing subset (n = 15) on the planning CT and the corrected CBCT. The plans generated on the corrected CBCT were compared to the CT-based plans in terms of beam parameters, dosimetric indices, and dose distributions.

RESULTS

The corrected CBCT images showed considerable similarity to their corresponding planning CTs (average mutual information 1.0±0.1, average sum of absolute differences 185 ± 38). The automated CBCT-based plans were clinically acceptable, as well as equivalent to the CT-based plans with average gantry angle difference of 0.99°±1.1°, target volume overlap index (Dice) of 0.89±0.04 although with slightly higher maximum target doses (4482±90 vs 4560±84, P < 0.05). Gamma index analysis (3%, 3 mm) showed that the CBCT-based plans had the same dose distribution as plans calculated with the same beams on the registered planning CTs (average gamma index 0.12±0.04, gamma <1 in 99.4%±0.3%).

CONCLUSIONS

The proposed method demonstrates the potential for a clinically feasible and efficient online adaptive breast IMRT planning method based on CBCT imaging, integrating automation.