Error sensitivity of a log file analysis tool compared with a helical diode array dosimeter for VMAT delivery quality assurance

Monte Carlo simulation-based patient-specific QA using machine log files for line-scanning proton radiation therapy

BACKGROUND

Quality assurance (QA) is a prerequisite for safe and accurate pencil-beam proton therapy. Conventional measurement-based patient-specific QA (pQA) can only verify limited aspects of patient treatment and is labor-intensive. Thus, a better method is needed to ensure the integrity of the treatment plan.

PURPOSE

Line scanning, which involves continuous and rapid delivery of pencil beams, is a state-of-the-art proton therapy technique. Machine performance in delivering scanning protons is dependent on the complexity of the beam modulations. Moreover, it contributes to patient treatment accuracy. A Monte Carlo (MC) simulation-based QA method that reflects the uncertainty related to the machine during scanning beam delivery was developed and verified for clinical applications to pQA.

METHODS

Herein, a tool for particle simulation (TOPAS) for nozzle modeling was used, and the code was commissioned against the measurements. To acquire the beam delivery uncertainty for each plan, patient plans were delivered. Furthermore, log files recorded every 60 μs by the monitors downstream of the nozzle were exported from the treatment control system. The spot positions and monitor unit (MU) counts in the log files were converted to dipole magnet strengths and number of particles, respectively, and entered into the TOPAS. For the 68 clinical cases, MC simulations were performed in a solid water phantom, and two-dimensional (2D) absolute dose distributions at 20-mm depth were measured using an ionization chamber array (Octavius 1500, PTW, Freiburg, Germany). Consequently, the MC-simulated 2D dose distributions were compared with the measured data, and the dose distributions in the pre-treatment QA plan created with RayStation (RaySearch Laboratories, Stockholm, Sweden). Absolute dose comparisons were made using gamma analysis with 3%/3 mm and 2%/2 mm criteria for 47 clinical cases without considering daily machine output variation in the MC simulation and 21 cases with daily output variation, respectively. All cases were analyzed with 90% or 95% of passing rate thresholds.

RESULTS

For 47 clinical cases not considering daily output variations, the absolute gamma passing rates compared with the pre-treatment QA plan were 99.71% and 96.97%, and the standard deviations (SD) were 0.70% and 3.78% with the 3%/3 mm or 2%/2 mm criteria, respectively. Compared with the measurements, the passing rate of 2%/2 mm gamma criterion was 96.76% with 3.99% of SD. For the 21 clinical cases compared with pre-treatment QA plan data and measurements considering daily output variations, the 2%/2 mm absolute gamma analysis result was 98.52% with 1.43% of SD and 97.67% with 2.72% of SD, respectively. With a 95% passing rate threshold of 2%/2 mm criterion, the false-positive and false-negative were 21.8% and 8.3% for without and with considering output variation, respectively. With a 90% threshold, the false-positive and false-negative reduced to 11.4% and 0% for without and with considering output variation, respectively.

CONCLUSIONS

A log-file-based MC simulation method for patient QA of line-scanning proton therapy was successfully developed. The proposed method exhibited clinically acceptable accuracy, thereby exhibiting a potential to replace the measurement-based dosimetry QA method with a 90% gamma passing rate threshold when applying the 2%/2 mm criterion.

Sensitivity and specificity of Monte Carlo based independent secondary dose computation for detecting modulation-related dose errors in intensity modulated radiotherapy

BACKGROUND

The recent availability of Monte Carlo based independent secondary dose calculation (ISDC) for patient-specific quality assurance (QA) of modulated radiotherapy requires the definition of appropriate, more sensitive action levels, since contemporary recommendations were defined for less accurate ISDC dose algorithms.

PURPOSE

The objective is to establish an optimum action level and measure the efficacy of a Monte Carlo ISDC software for pre-treatment QA of intensity modulated radiotherapy treatments.

METHODS

The treatment planning system and the ISDC were commissioned by their vendors from independent base data sets, replicating a typical real-world scenario. In order to apply Receiver-Operator-Characteristics (ROC), a set of treatment plans for various case classes was created that consisted of 190 clinical treatment plans and 190 manipulated treatment plans with dose errors in the range of 1.5-2.5%. All 380 treatment plans were evaluated with ISDC in the patient geometry. ROC analysis was performed for a number of Gamma (dose-difference/distance-to-agreement) criteria. QA methods were ranked according to Area under the ROC curve (AUC) and optimum action levels were derived via Youden's J statistics.

RESULTS

Overall, for original treatment plans, the mean Gamma pass rate (GPR) for Gamma(1%, 1 mm) was close to 90%, although with some variation across case classes. The best QA criterion was Gamma(2%, 1 mm) with GPR > 90% and an AUC of 0.928. Gamma criteria with small distance-to-agreement had consistently higher AUC. GPR of original treatment plans depended on their modulation degree. An action level in terms of Gamma(1%, 1 mm) GPR that decreases with modulation degree was the most efficient criterion with sensitivity = 0.91 and specificity = 0.95, compared with Gamma(3%, 3 mm) GPR > 99%, sensitivity = 0.73 and specificity = 0.91 as a commonly used action level.

CONCLUSIONS

ISDC with Monte Carlo proves highly efficient to catch errors in the treatment planning process. For a Monte Carlo based TPS, dose-difference criteria of 2% or less, and distance-to-agreement criteria of 1 mm, achieve the largest AUC in ROC analysis.

Clinical implementation of a log file-based machine and patient QA system for IMRT and VMAT treatment plans

PURPOSE

To determine the error detection sensitivity of a commercial log file-based system (LINACWatch®, LW) for integration into clinical routine and to compare it with a measurement device (OCTAVIUS 4D, Oct4D) for IMRT and VMAT delivery QA.

MATERIALS AND METHODS

76 VMAT/IMRT plans (H&N, prostate, rectum and breast) preliminarily classified according to their Modulation Complexity Score (MCS) calculated by LW, were considered. Receiver Operating Characteristic (ROC) Curves were used to establish gamma criteria for LW. 12 plans (3 for each site) were intentionally modified in order to introduce delivery errors regarding MLC, jaws, collimator, gantry and MU (for a total set of 168 incorrect plans) and irradiated on Oct4D; the corresponding log files were analysed by LW. Each incorrect plan was compared to the error-free plan using γ-index analysis for MLC, jaws and MU errors investigation and Root-Mean-Square (RMS) values for gantry and collimator errors investigation.

RESULTS

MCS ranges values were: 0.10-0.20 for H&N, 0.21-0.40 for prostate and rectum, 0.41-1.00 for breast. From ROC curves, the Gamma Passing Rate (GPR) thresholds were: 87%, 92%, 99% for H&N, prostate and rectum, and breast, respectively. The 1.5%/1.5 mm/local criteria were adopted for the γ-analysis. LW sensitivity in detecting the introduced errors was higher when compared to Oct4D: 48.5% vs 30.4% respectively.

CONCLUSIONS

LW can be considered useful complement to phantom-based delivery QA of IMRT/VMAT plans. The MCS tool is effective in detecting over or under modulated plans prior to pre-treatment QA. However, rigorous and routinely machine QCs are recommended.

A method for patient-specific DVH verification using a high-sampling-rate log file in an Elekta linac

We have proposed a method for patient-specific dose-volume histogram (DVH) verification using a 40-ms high-sampling-rate log file (HLF) available in an Elekta linac. Ten prostate volumetric-modulated arc therapy plans were randomly selected, and systematic leaf position errors of ±0.2, ±0.4, or ±0.8 mm were added to the 10 plans, thereby producing a total of 70 plans. An RTP file was created by interpolating each leaf position in the HLF to obtain values at each control point, which is subsequently exported to a treatment planning system. The isocenter dose calculated by the HLF-based plan to a phantom (Delta4 Phantom+) was compared to that measured by the diode in the phantom in order to evaluate the accuracy of the HLF-based dose calculation. The D95 of the planning target volume (PTV) was also compared between the HLF-based plans and the original plans with the systematic leaf position errors, the latter being referred to as theory-based plans. Sensitivities of the DVH parameters in the target, the rectum, and the bladder were also calculated with the varied systematic leaf position errors. The relative differences in the isocenter doses between the HLF-based calculations and the measurements among the 70 plans were 0.21% ± 0.67% (SD). The maximum relative differences in PTV D95 between the HLF-based and the theory-based plans among the 70 cases were 0.11%. The patient-specific DVH verification method detected a change in the target DVH parameters of less than 1% when the systematic leaf position error was ±0.2 mm. It is therefore suggested that the proposed DVH verification method may simplify patient-specific dose quality assurance procedures without compromising accuracy and sensitivity.

Prediction of portal dosimetry quality assurance results using log files-derived errors and machine learning techniques

Objective

This work aims to use machine learning models to predict gamma passing rate of portal dosimetry quality assurance with log file derived features. This allows daily treatment monitoring for patients and reduce wear and tear on EPID detectors to save cost and prevent downtime.

Methods

578 VMAT trajectory log files selected from prostate, lung and spine SBRT were used in this work. Four machine learning models were explored to identify the best performing regression model for predicting gamma passing rate within each sub-site and the entire unstratified data. Predictors used in these models comprised of hand-crafted log file-derived features as well as modulation complexity score. Cross validation was used to evaluate the model performance in terms of R² and RMSE.

Result

Using gamma passing rate of 1%/1mm criteria and entire dataset, LASSO regression has a R² of 0.121 ± 0.005 and RMSE of 4.794 ± 0.013%, SVM regression has a R² of 0.605 ± 0.036 and RMSE of 3.210 ± 0.145%, Random Forest regression has a R² of 0.940 ± 0.019 and RMSE of 1.233 ± 0.197%. XGBoost regression has the best performance with a R² and RMSE value of 0.981 ± 0.015 and 0.652 ± 0.276%, respectively.

Conclusion

Log file-derived features can predict gamma passing rate of portal dosimetry with an average error of less than 2% using the 1%/1mm criteria. This model can potentially be applied to predict the patient specific QA results for every treatment fraction.

Evaluation of 4-Hz log files and secondary Monte Carlo dose calculation as patient-specific quality assurance for VMAT prostate plans

Purpose

In this study, 4‐Hz log files were evaluated with an independent secondary Monte Carlo dose calculation algorithm to reduce the workload for patient‐specific quality assurance (QA) in clinical routine.

Materials and Methods

A total of 30 randomly selected clinical prostate VMAT plans were included. The used treatment planning system (TPS) was Monaco (Elekta, Crawley), and the secondary dose calculation software was SciMoCa (Scientific‐RT, Munich). Monaco and SciMoCa work with a Monte Carlo algorithm. A plausibility check of Monaco and SciMoCa was performed using an ionization chamber in the BodyPhantom (BP). First, the original Monaco RT plans were verified with SciMoCa (pretreatment QA). Second, the corresponding 4‐Hz log files were converted into RT log file plans and sent to SciMoCa as on‐treatment QA. MLC shift errors were introduced for one prostate plan to determine the sensitivity of on‐treatment QA. For pretreatment and on‐treatment QA, a gamma analysis (2%/1mm/20%) was performed and dosimetric values of PTV and OARs were ascertained in SciMoCa.

Results

Plausibility check of TPS Monaco vs. BP measurement and SciMoCa vs. BP measurement showed valid accuracy for clinical VMAT QA. Using SciMoCa, there was no significant difference in PTV Dmean between RT plan and RT log file plan. Between pretreatment and on‐treatment QA, PTV metrics, femur right and left showed no significant dosimetric differences as opposed to OARs rectum and bladder. The overall gamma passing rate (GPR) ranged from 96.10% to 100% in pretreatment QA and from 93.50% to 99.80% in on‐treatment QA. MLC shift errors were identified for deviations larger than −0.50 mm and +0.75 mm using overall gamma criterion and PTV Dmean.

Conclusion

SciMoCa calculations of Monaco RT plans and RT log file plans are in excellent agreement to each other. Therefore, 4‐Hz log files and SciMoCa can replace labor‐intensive phantom‐based measurements as patient‐specific QA.

A patient-specific QA comparison between 2D and 3D diode arrays for single-lesion SRS and SBRT treatments

The purpose of this study is to compare patient-specific quality assurance (PSQA) results between two dimensional (2D) diode (SRS MapCHECK^®) and 3D diode (ArcCHECK^®) arrays. Twenty-eight intracranial stereotactic radiosurgery (SRS) and 26 lung stereotactic body radiation therapy (SBRT) clinical plans with a single lesion were selected and categorized into 4 groups: 20 SRS dynamic conformal arc therapy (DCAT) plans (Group A), 8 SRS volumetric modulated arc therapy (VMAT) plans (Group B), 6 SBRT DCAT plans (Group C) and 20 SBRT VMAT plans (Group D). An individual field of each plan was delivered on SRS MapCHECK and ArcCHECK and QA analysis was performed using 4 gamma criteria of dose difference/distance-to-agreement of 3%/3 mm, 3%/2 mm, 2%/2 mm and 2%/1 mm. Statistical analysis was performed to compare PSQA results between the 2 QA devices. For all 4 groups and all 4 gamma criteria, average gamma passing rates were higher with SRS MapCHECK.