Patient-specific IMRT QA verification using machine learning and gamma radiomics

A feasibility study to predict 3D dose delivery accuracy for IMRT using DenseNet with log files

OBJECTIVE

This study aims to explore the feasibility of DenseNet in the establishment of a three-dimensional (3D) gamma prediction model of IMRT based on the actual parameters recorded in the log files during delivery.

METHODS

A total of 55 IMRT plans (including 367 fields) were randomly selected. The gamma analysis was performed using gamma criteria of 3% /3 mm (Dose Difference/Distance to Agreement), 3% /2 mm, 2% /3 mm, and 2% /2 mm with a 10% dose threshold. In addition, the log files that recorded the gantry angle, monitor units (MU), multi-leaf collimator (MLC), and jaws position during delivery were collected. These log files were then converted to MU-weighted fluence maps as the input of DenseNet, gamma passing rates (GPRs) under four different gamma criteria as the output, and mean square errors (MSEs) as the loss function of this model.

RESULTS

Under different gamma criteria, the accuracy of a 3D GPR prediction model decreased with the implementation of stricter gamma criteria. In the test set, the mean absolute error (MAE) of the prediction model under the gamma criteria of 3% /3 mm, 2% /3 mm, 3% /2 mm, and 2% /2 mm was 1.41, 1.44, 3.29, and 3.54, respectively; the root mean square error (RMSE) was 1.91, 1.85, 4.27, and 4.40, respectively; the Sr was 0.487, 0.554, 0.573, and 0.506, respectively. There was a correlation between predicted and measured GPRs (P <  0.01). Additionally, there was no significant difference in the accuracy between the validation set and the test set. The accuracy in the high GPR group was high, and the MAE in the high GPR group was smaller than that in the low GPR group under four different gamma criteria.

CONCLUSIONS

In this study, a 3D GPR prediction model of patient-specific QA using DenseNet was established based on log files. As an auxiliary tool for 3D dose verification in IMRT, this model is expected to improve the accuracy and efficiency of dose validation.

Plan complexity metrics for head and neck VMAT competition plans

Previous plan competitions have largely focused on dose metric assessments. However, whether the submitted plans were realistic and reasonable from a quality assurance (QA) perspective remains unclear. This study aimed to investigate the relationship between aperture-based plan complexity metrics (PCM) in volumetric modulated arc therapy (VMAT) competition plans and clinical treatment plans verified through patient-specific QA (PSQA). In addition, the association of PCMs with plan quality was examined. A head and neck (HN) plan competition was held for Japanese institutions from June 2019 to July 2019, in which 210 competition plans were submitted. Dose distribution quality was quantified based on dose-volume histogram (DVH) metrics by calculating the dose distribution plan score (DDPS). Differences in PCMs between the two VMAT treatment plan groups (HN plan competitions held in Japan and clinically accepted HN VMAT plans through PSQA) were investigated. The mean (± standard deviation) DDPS for the 98 HN competition plans was 158.5 ± 20.6 (maximum DDPS: 200). DDPS showed a weak correlation with PCMs with a maximum r of 0.45 for monitor unit (MU); its correlation with some PCMs was "very weak." Significant differences were found in some PCMs between plans with the highest 20% DDPSs and the remaining plans. The clinical VMAT and competition plans revealed similar distributions for some PCMs. Deviations in PCMs for the two groups were comparable, indicating considerable variability among planners regarding planning skills. The plan complexity for HN VMAT competition plans increased for high-quality plans, as shown by the dose distribution. Direct comparison of PCMs between competition plans and clinically accepted plans showed that the submitted HN VMAT competition plans were realistic and reasonable from the QA perspective. This evaluation may provide a set of criteria for evaluating plan quality in plan competitions.

Prediction of patient-specific quality assurance for volumetric modulated arc therapy using radiomics-based machine learning with dose distribution

PURPOSE

We sought to develop machine learning models to predict the results of patient-specific quality assurance (QA) for volumetric modulated arc therapy (VMAT), which were represented by several dose-evaluation metrics-including the gamma passing rates (GPRs)-and criteria based on the radiomic features of 3D dose distribution in a phantom.

METHODS

A total of 4,250 radiomic features of 3D dose distribution in a cylindrical dummy phantom for 140 arcs from 106 clinical VMAT plans were extracted. We obtained the following dose-evaluation metrics: GPRs with global and local normalization, the dose difference (DD) in 1% and 2% passing rates (DD1% and DD2%) for 10% and 50% dose threshold, and the distance-to-agreement in 1-mm and 2-mm passing rates (DTA1 mm and DTA2 mm) for 0.5%/mm and 1.0%.mm dose gradient threshold determined by measurement using a diode array in patient-specific QA. The machine learning regression models for predicting the values of the dose-evaluation metrics using the radiomic features were developed based on the elastic net (EN) and extra trees (ET) models. The feature selection and tuning of hyperparameters were performed with nested cross-validation in which four-fold cross-validation is used within the inner loop, and the performance of each model was evaluated in terms of the root mean square error (RMSE), the mean absolute error (MAE), and Spearman's rank correlation coefficient.

RESULTS

The RMSE and MAE for the developed machine learning models ranged from <1% to nearly <10% depending on the dose-evaluation metric, the criteria, and dose and dose gradient thresholds used for both machine learning models. It was advantageous to focus on high dose region for predicating global GPR, DDs, and DTAs. For certain metrics and criteria, it was possible to create models applicable for patients' heterogeneity by training only with dose distributions in phantom.

CONCLUSIONS

The developed machine learning models showed high performance for predicting dose-evaluation metrics especially for high dose region depending on the metric and criteria. Our results demonstrate that the radiomic features of dose distribution can be considered good indicators of the plan complexity and useful in predicting measured dose evaluation metrics.

Application of error classification model using indices based on dose distribution for characteristics evaluation of multileaf collimator position errors

This study aims to evaluate the specific characteristics of various multileaf collimator (MLC) position errors that are correlated with the indices using dose distribution. The dose distribution was investigated using the gamma, structural similarity, and dosiomics indices. Cases from the American Association of Physicists in Medicine Task Group 119 were planned, and systematic and random MLC position errors were simulated. The indices were obtained from distribution maps and statistically significant indices were selected. The final model was determined when all values of the area under the curve, accuracy, precision, sensitivity, and specificity were higher than 0.8 (p < 0.05). The dose-volume histogram (DVH) relative percentage difference between the error-free and error datasets was examined to investigate clinical relations. Seven multivariate predictive models were finalized. The common significant dosiomics indices (GLCM Energy and GLRLM_LRHGE) can characterize the MLC position error. In addition, the finalized logistic regression model for MLC position error prediction showed excellent performance with AUC > 0.9. Furthermore, the results of the DVH were related to dosiomics analysis in that it reflects the characteristics of the MLC position error. It was also shown that dosiomics analysis could provide important information on localized dose-distribution differences in addition to DVH information.

Image-based features in machine learning to identify delivery errors and predict error magnitude for patient-specific IMRT quality assurance

OBJECTIVE

To identify delivery error type and predict associated error magnitude by image-based features using machine learning (ML).

METHODS

In this study, a total of 40 thoracic plans (including 208 beams) were selected, and four error types with different magnitudes were introduced into the original plans, including 1) collimator misalignment (COLL), 2) monitor unit (MU) variation, 3) systematic multileaf collimator misalignment (MLCS), and 4) random MLC misalignment (MLCR). These dose distributions of portal dose predictions for the original plans were defined as the reference dose distributions (RDD), while those for the error-introduced plans were defined as the error-introduced dose distributions (EDD). Both distributions were calculated for all beams with portal dose image prediction (PDIP). Besides, 14 image-based features were extracted from RDD and EDD of portal dose predictions to obtain the feature vectors. In addition, a random forest was adopted for the multiclass classification task, and regression prediction for error magnitude.

RESULTS

The top five features extracted with the highest weight included 1) the relative displacement in the x direction, 2) the ratio of the absolute minimum residual error to the maximal RDD value, 3) the product of the maximum and minimum residuals, 4) the ratio of the absolute maximum residual error to the maximal RDD value, and 5) the ratio of the absolute mean residual value to the maximal RDD value. The relative displacement in the x direction had the highest weight. The overall accuracy of the five-class classification model was 99.85% for the validation set and 99.30% for the testing set. This model could be applied to the classification of the error-free plan, COLL, MU, MLCS, and MLCR with an accuracy of 100%, 98.4%, 99.9%, 98.0%, and 98.3%, respectively. MLCR had the worst performance in error magnitude prediction (70.1-96.6%), while others had better performance in error magnitude prediction (higher than 93%). In the error magnitude prediction, the mean absolute error (MAE) between predicted error magnitude and actual error ranged from 0.03 to 0.33, with the root mean squared error (RMSE) varying from 0.17 to 0.56 for the validation set. The MAE and RMSE ranged from 0.03 to 0.50 and 0.44 to 0.59 for the test set, respectively.

CONCLUSION

It could be demonstrated in this study that the image-based features extracted from RDD and EDD can be employed to identify different types of delivery errors and accurately predict error magnitude with the assistance of ML techniques. They can be used to associate traditional gamma analysis with clinically based analysis for error classification and magnitude prediction in patient-specific IMRT quality assurance.

Radiomics-guided radiation therapy: opportunities and challenges

Radiomics is an advanced image-processing framework, which extracts image features and considers them as biomarkers towards personalized medicine. Applications include disease detection, diagnosis, prognosis, and therapy response assessment/prediction. As radiation therapy aims for further individualized treatments, radiomics could play a critical role in various steps before, during and after treatment. Elucidation of the concept of radiomics-guided radiation therapy (RGRT) is the aim of this review, attempting to highlight opportunities and challenges underlying the use of radiomics to guide clinicians and physicists towards more effective radiation treatments. This work identifies the value of RGRT in various steps of radiotherapy from patient selection to follow-up, and subsequently provides recommendations to improve future radiotherapy using quantitative imaging features.

Prospective clinical validation of virtual patient-specific quality assurance of VMAT radiation therapy plans

PURPOSE

Performing measurement-based patient-specific quality assurance (PSQA) is recognized as a resource-intensive and time inefficient task in the radiotherapy treatment workflow. Paired with technological refinements in modern radiotherapy, research towards measurement-free PSQA has seen increased interest over the last five years. However, these efforts have not been clinically implemented or prospectively validated in the U.S. We propose a virtual QA (VQA) system and workflow to assess the safety and workload reduction of measurement-free PSQA.

METHODS

An XGBoost machine learning model was designed to predict PSQA outcomes of VMAT plans, represented as percent differences between the measured ion chamber point dose in a phantom and the corresponding planned dose. The final model was deployed within a web application to predict PSQA outcomes of clinical plans within an existing clinical workflow. The application also displays relevant feature importance and plan-specific distribution analyses relative to database plans for documentation and to aid physicist interpretation and evaluation. VQA predictions were prospectively validated over three months of measurements at our clinic to assess safety and efficiency gains.

RESULTS

Over three months, VQA predictions for 445 VMAT plans were prospectively validated at our institution. VQA predictions for these plans had a mean absolute error of 1.08 +/- 0.77%, with a maximum absolute error of 2.98%. Employing a 1% prediction threshold (i.e. plans predicted to have an absolute error of less than 1% would not require a measurement) would yield a 69.2% reduction in QA workload - saving 32.5 hours per month on average - with 81.5%/72.4%/0.81 sensitivity/specificity/AUC at a 3% clinical threshold and 100%/70%/0.93 sensitivity/specificity/AUC at a 4% clinical threshold.

CONCLUSION

This is the first prospective clinical implementation and validation of VQA in the U.S., which we observed to be efficient. Using a conservative threshold, VQA can substantially reduce the number of required measurements for PSQA, leading to more effective allocation of clinical resources.