Contextual Atlas Regression Forests: Multiple-Atlas-Based Automated Dose Prediction in Radiation Therapy

A feasibility study of automating radiotherapy planning with large language model agents

Objective.Radiotherapy planning requires significant expertise to balance tumor control and organ-at-risk (OAR) sparing. Automated planning can improve both efficiency and quality. This study introduces GPT-Plan, a novel multi-agent system powered by the GPT-4 family of large language models (LLMs), for automating the iterative radiotherapy plan optimization.Approach.GPT-Plan uses LLM-driven agents, mimicking the collaborative clinical workflow of a dosimetrist and physicist, to iteratively generate and evaluate text-based radiotherapy plans based on predefined criteria. Supporting tools assist the agents by leveraging historical plans, mitigating LLM hallucinations, and balancing exploration and exploitation. Performance was evaluated on 12 lung (IMRT) and 5 cervical (VMAT) cancer cases, benchmarked against the ECHO auto-planning method and manual plans. The impact of historical plan retrieval on efficiency was also assessed.Results.For IMRT lung cancer cases, GPT-Plan generated high-quality plans, demonstrating superior target coverage and homogeneity compared to ECHO while maintaining comparable or better OAR sparing. For VMAT cervical cancer cases, plan quality was comparable to a senior physicist and consistently superior to a junior physicist, particularly for OAR sparing. Retrieving historical plans significantly reduced the number of required optimization iterations for lung cases (p < 0.01) and yielded iteration counts comparable to those of the senior physicist for cervical cases (p = 0.313). Occasional LLM hallucinations have been mitigated by self-reflection mechanisms. One limitation was the inaccuracy of vision-based LLMs in interpreting dose images.Significance.This pioneering study demonstrates the feasibility of automating radiotherapy planning using LLM-powered agents for complex treatment decision-making tasks. While challenges remain in addressing LLM limitations, ongoing advancements hold potential for further refining and expanding GPT-Plan's capabilities.

Artificial Intelligence for Radiation Treatment Planning: Bridging Gaps From Retrospective Promise to Clinical Reality

Artificial intelligence (AI) radiation therapy (RT) planning holds promise for enhancing the consistency and efficiency of the RT planning process. Despite technical advancements, the widespread integration of AI into RT treatment planning faces challenges. The transition from controlled retrospective environments to real-world clinical settings introduces heightened scrutiny from clinical end users, potentially leading to decreased clinical acceptance. Key considerations for implementing AI RT planning include ensuring the AI model performance aligns with clinical standards, using high-quality training data, and incorporating sufficient data variation through meticulous curation by clinical experts. Beyond technical aspects, factors such as potential biases and the level of trust clinical end users place in AI may present unforeseen obstacles for real-world clinical use. Addressing these challenges requires bridging education and expertise gaps among clinical end users, enabling them to confidently embrace and utilize AI for routine RT planning. By fostering a better understanding of AI capabilities, building trust, and providing comprehensive training, the promises of AI RT planning can be a reality in the clinical setting. This article assesses the current clinical use of AI RT planning and explores challenges and considerations for bridging gaps in knowledge and expertise for AI operationalization, with focus on training data curation, workflow integration, explainability, bias, and domain knowledge. Remaining challenges in clinical implementation of AI RT treatment planning are examined in the context of trust building approaches.

Deep learning architecture with shunted transformer and 3D deformable convolution for voxel-level dose prediction of head and neck tumors

Intensity-modulated radiation therapy (IMRT) has been widely used in treating head and neck tumors. However, due to the complex anatomical structures in the head and neck region, it is challenging for the plan optimizer to rapidly generate clinically acceptable IMRT treatment plans. A novel deep learning multi-scale Transformer (MST) model was developed in the current study aiming to accelerate the IMRT planning for head and neck tumors while generating more precise prediction of the voxel-level dose distribution. The proposed end-to-end MST model employs the shunted Transformer to capture multi-scale features and learn a global dependency, and utilizes 3D deformable convolution bottleneck blocks to extract shape-aware feature and compensate the loss of spatial information in the patch merging layers. Moreover, data augmentation and self-knowledge distillation are used to further improve the prediction performance of the model. The MST model was trained and evaluated on the OpenKBP Challenge dataset. Its prediction accuracy was compared with three previous dose prediction models: C3D, TrDosePred, and TSNet. The predicted dose distributions of our proposed MST model in the tumor region are closest to the original clinical dose distribution. The MST model achieves the dose score of 2.23 Gy and the DVH score of 1.34 Gy on the test dataset, outperforming the other three models by 8%-17%. For clinical-related DVH dosimetric metrics, the prediction accuracy in terms of mean absolute error (MAE) is 2.04% for D 99 , 1.54% for D 95 , 1.87% for D 1 , 1.87% for D mean , 1.89% for D 0.1 c c , respectively, superior to the other three models. The quantitative results demonstrated that the proposed MST model achieved more accurate voxel-level dose prediction than the previous models for head and neck tumors. The MST model has a great potential to be applied to other disease sites to further improve the quality and efficiency of radiotherapy planning.

DoseDiff: Distance-Aware Diffusion Model for Dose Prediction in Radiotherapy

Treatment planning, which is a critical component of the radiotherapy workflow, is typically carried out by a medical physicist in a time-consuming trial-and-error manner. Previous studies have proposed knowledge-based or deep-learning-based methods for predicting dose distribution maps to assist medical physicists in improving the efficiency of treatment planning. However, these dose prediction methods usually fail to effectively utilize distance information between surrounding tissues and targets or organs-at-risk (OARs). Moreover, they are poor at maintaining the distribution characteristics of ray paths in the predicted dose distribution maps, resulting in a loss of valuable information. In this paper, we propose a distance-aware diffusion model (DoseDiff) for precise prediction of dose distribution. We define dose prediction as a sequence of denoising steps, wherein the predicted dose distribution map is generated with the conditions of the computed tomography (CT) image and signed distance maps (SDMs). The SDMs are obtained by distance transformation from the masks of targets or OARs, which provide the distance from each pixel in the image to the outline of the targets or OARs. We further propose a multi-encoder and multi-scale fusion network (MMFNet) that incorporates multi-scale and transformer-based fusion modules to enhance information fusion between the CT image and SDMs at the feature level. We evaluate our model on two in-house datasets and a public dataset, respectively. The results demonstrate that our DoseDiff method outperforms state-of-the-art dose prediction methods in terms of both quantitative performance and visual quality.

Machine learning automated treatment planning for online magnetic resonance guided adaptive radiotherapy of prostate cancer

Background and purpose

No best practices currently exist for achieving high quality radiation therapy (RT) treatment plan adaptation during magnetic resonance (MR) guided RT of prostate cancer. This study validates the use of machine learning (ML) automated RT treatment plan adaptation and benchmarks it against current clinical RT plan adaptation methods.

Materials and methods

We trained an atlas-based ML automated treatment planning model using reference MR RT treatment plans (42.7 Gy in 7 fractions) from 46 patients with prostate cancer previously treated at our institution. For a held-out test set of 38 patients, retrospectively generated ML RT plans were compared to clinical human-generated adaptive RT plans for all 266 fractions. Differences in dose-volume metrics and clinical objective pass rates were evaluated using Wilcoxon tests (p < 0.05) and Exact McNemar tests (p < 0.05), respectively.

Results

Compared to clinical RT plans, ML RT plans significantly increased sparing and objective pass rates of the rectum, bladder, and left femur. The mean ± standard deviation of rectum D20 and D50 in ML RT plans were 2.5 ± 2.2 Gy and 1.6 ± 1.3 Gy lower than clinical RT plans, respectively, with 14 % higher pass rates; bladder D40 was 4.6 ± 2.9 Gy lower with a 20 % higher pass rate; and the left femur D5 was 0.8 ± 1.8 Gy lower with a 7 % higher pass rate.

Conclusions

ML automated RT treatment plan adaptation increases robustness to interfractional anatomical changes compared to current clinical adaptive RT practices by increasing compliance to treatment objectives.

Artificial intelligence-based predictive model for guidance on treatment strategy selection in oral and maxillofacial surgery

Application of deep learning (DL) and machine learning (ML) is rapidly increasing in the medical field. DL is gaining significance for medical image analysis, particularly, in oral and maxillofacial surgeries. Owing to the ability to accurately identify and categorize both diseased and normal soft- and hard-tissue structures, DL has high application potential in the diagnosis and treatment of tumors and in orthognathic surgeries. Moreover, DL and ML can be used to develop prediction models that can aid surgeons to assess prognosis by analyzing the patient's medical history, imaging data, and surgical records, develop more effective treatment strategies, select appropriate surgical modalities, and evaluate the risk of postoperative complications. Such prediction models can play a crucial role in the selection of treatment strategies for oral and maxillofacial surgeries. Their practical application can improve the utilization of medical staff, increase the treatment accuracy and efficiency, reduce surgical risks, and provide an enhanced treatment experience to patients. However, DL and ML face limitations, such as data drift, unstable model results, and vulnerable social trust. With the advancement of social concepts and technologies, the use of these models in oral and maxillofacial surgery is anticipated to become more comprehensive and extensive.

A Prospective Study of Machine Learning-Assisted Radiation Therapy Planning for Patients Receiving 54 Gy to the Brain

PURPOSE

The capacity for machine learning (ML) to facilitate radiation therapy (RT) planning for primary brain tumors has not been described. We evaluated ML-assisted RT planning with regard to clinical acceptability, dosimetric outcomes, and planning efficiency for adults and children with primary brain tumors.

METHODS AND MATERIALS

In this prospective study, children and adults receiving 54 Gy fractionated RT for a primary brain tumor were enrolled. For each patient, one ML-assisted RT plan was created and compared with 1 or 2 plans created using standard ("manual") planning procedures. Plans were evaluated by the treating oncologist, who was blinded to the method of plan creation. The primary endpoint was the proportion of ML plans that were clinically acceptable for treatment. Secondary endpoints included the frequency with which ML plans were selected as preferable for treatment, and dosimetric differences between ML and manual plans.

RESULTS

A total of 116 manual plans and 61 ML plans were evaluated across 61 patients. Ninety-four percent of ML plans and 93% of manual plans were judged to be clinically acceptable (P = 1.0). Overall, the quality of ML plans was similar to manual plans. ML plans comprised 34.5% of all plans evaluated and were selected for treatment in 36.1% of cases (P = .82). Similar tumor target coverage was achieved between both planning methods. Normal brain (brain minus planning target volume) received an average of 1 Gy less mean dose with ML plans (compared with manual plans, P < .001). ML plans required an average of 45.8 minutes less time to create, compared with manual plans (P < .001).

CONCLUSIONS

ML-assisted automated planning creates high-quality plans for patients with brain tumors, including children. Plans created with ML assistance delivered slightly less dose to normal brain tissues and can be designed in less time.

Deep-neural network approaches for predicting 3D dose distribution in intensity-modulated radiotherapy of the brain tumors

PURPOSE

The aim of this study is to reduce treatment planning time by predicting the intensity-modulated radiotherapy 3D dose distribution using deep learning for brain cancer patients. "For this purpose, two different approaches in dose prediction, i.e., first only planning target volume (PTV) and second PTV with organs at risk (OARs) as input of the U-net model, are employed and their results are compared."

METHODS AND MATERIALS

The data of 99 patients with glioma tumors referred for IMRT treatment were used so that the images of 90 patients were regarded as training datasets and the others were for the test. All patients were manually planned and treated with sixth-field IMRT; the photon energy was 6MV. The treatment plans were done with the Collapsed Cone Convolution algorithm to deliver 60 Gy in 30 fractions.

RESULTS

The obtained accuracy and similarity for the proposed methods in dose prediction when compared to the clinical dose distributions on test patients according to MSE, dice metric and SSIM for the Only-PTV and PTV-OARs methods are on average (0.05, 0.851, 0.83) and (0.056, 0.842, 0.82) respectively. Also, dose prediction is done in an extremely short time.

CONCLUSION

The same results of the two proposed methods prove that the presence of OARs in addition to PTV does not provide new knowledge to the network and only by defining the PTV and its location in the imaging slices, does the dose distribution become predictable. Therefore, the Only-PTV method by eliminating the process of introducing OARs can reduce the overall designing time of treatment by IMRT in patients with glioma tumors.

Deep-learning Method for the Prediction of Three-Dimensional Dose Distribution for Left Breast Cancer Conformal Radiation Therapy

AIMS

An increase in the demand of a new generation of radiotherapy planning systems based on learning approaches has been reported. At this stage, the new approach is able to improve the planning speed while saving a reasonable level of plan quality, compared with available planning systems. We believe that new achievements, such as deep-learning models, will be able to review the issue from a different point of view.

MATERIALS AND METHODS

The data of 120 breast cancer patients were used to train and test the three-dimensional U-Res-Net model. The network input was computed tomography images and patients' contouring, while the patients' dose distribution was addressed as the output of the model proposed. The predicted dose distributions, created by the model for 10 test patients, were then compared with corresponding dose distributions calculated by a reliable treatment planning system. In particular, the dice similarity coefficients for different isodose volumes, dose difference and mean absolute errors (MAE) for all voxels inside the body, Dmean, D98%, D50%, D2%, V95% for planning target volume and organs at risk were calculated and were statistically analysed with the paired-samples t-test.

RESULTS

The average dose difference for all patients and voxels in body was 0.60 ± 2.81%. The MAE varied from 3.85 ± 6.65% to 8.06 ± 10.00%. The average MAE for test cases was 5.71 ± 1.19%. The average dice similarity coefficients for isodose volumes was 0.91 ± 0.03. The three-dimensional gamma passing rates with 3 mm/3% criteria varied from 78.99% to 97.58% for planning target volume and organs at risk, respectively.

CONCLUSIONS

The investigation showed that a deep-learning model can be applied to predict the three-dimensional dose distribution with optimal accuracy and precision for patients with left breast cancer. As further study, the model can be extended to predict dose distribution in other cancers.

Artificial general intelligence for radiation oncology

The emergence of artificial general intelligence (AGI) is transforming radiation oncology. As prominent vanguards of AGI, large language models (LLMs) such as GPT-4 and PaLM 2 can process extensive texts and large vision models (LVMs) such as the Segment Anything Model (SAM) can process extensive imaging data to enhance the efficiency and precision of radiation therapy. This paper explores full-spectrum applications of AGI across radiation oncology including initial consultation, simulation, treatment planning, treatment delivery, treatment verification, and patient follow-up. The fusion of vision data with LLMs also creates powerful multimodal models that elucidate nuanced clinical patterns. Together, AGI promises to catalyze a shift towards data-driven, personalized radiation therapy. However, these models should complement human expertise and care. This paper provides an overview of how AGI can transform radiation oncology to elevate the standard of patient care in radiation oncology, with the key insight being AGI's ability to exploit multimodal clinical data at scale.

Enhancing head and neck tumor management with artificial intelligence: Integration and perspectives

Head and neck tumors (HNTs) constitute a multifaceted ensemble of pathologies that primarily involve regions such as the oral cavity, pharynx, and nasal cavity. The intricate anatomical structure of these regions poses considerable challenges to efficacious treatment strategies. Despite the availability of myriad treatment modalities, the overall therapeutic efficacy for HNTs continues to remain subdued. In recent years, the deployment of artificial intelligence (AI) in healthcare practices has garnered noteworthy attention. AI modalities, inclusive of machine learning (ML), neural networks (NNs), and deep learning (DL), when amalgamated into the holistic management of HNTs, promise to augment the precision, safety, and efficacy of treatment regimens. The integration of AI within HNT management is intricately intertwined with domains such as medical imaging, bioinformatics, and medical robotics. This article intends to scrutinize the cutting-edge advancements and prospective applications of AI in the realm of HNTs, elucidating AI's indispensable role in prevention, diagnosis, treatment, prognostication, research, and inter-sectoral integration. The overarching objective is to stimulate scholarly discourse and invigorate insights among medical practitioners and researchers to propel further exploration, thereby facilitating superior therapeutic alternatives for patients.

Automated treatment planning for proton pencil beam scanning using deep learning dose prediction and dose-mimicking optimization

PURPOSE

The purpose of this study is to investigate the use of a deep learning architecture for automated treatment planning for proton pencil beam scanning (PBS).

METHODS

A 3-dimensional (3D) U-Net model has been implemented in a commercial treatment planning system (TPS) that uses contoured regions of interest (ROI) binary masks as model inputs with a predicted dose distribution as the model output. Predicted dose distributions were converted to deliverable PBS treatment plans using a voxel-wise robust dose mimicking optimization algorithm. This model was leveraged to generate machine learning (ML) optimized plans for patients receiving proton PBS irradiation of the chest wall. Model training was carried out on a retrospective set of 48 previously-treated chest wall patient treatment plans. Model evaluation was carried out by generating ML-optimized plans on a hold-out set of 12 contoured chest wall patient CT datasets from previously treated patients. Clinical goal criteria and gamma analysis were used to compare dose distributions of the ML-optimized plans against the clinically approved plans across the test patients.

RESULTS

Statistical analysis of mean clinical goal criteria indicates that compared to the clinical plans, the ML optimization workflow generated robust plans with similar dose to the heart, lungs, and esophagus while achieving superior dosimetric coverage to the PTV chest wall (clinical mean V95 = 97.6% vs. ML mean V95 = 99.1%, p < 0.001) across the 12 test patients.

CONCLUSIONS

ML-based automated treatment plan optimization using the 3D U-Net model can generate treatment plans of similar clinical quality compared to human-driven optimization.

TransDose: Transformer-based radiotherapy dose prediction from CT images guided by super-pixel-level GCN classification

Radiotherapy is a mainstay treatment for cancer in clinic. An excellent radiotherapy treatment plan is always based on a high-quality dose distribution map which is produced by repeated manual trial-and-errors of experienced experts. To accelerate the radiotherapy planning process, many automatic dose distribution prediction methods have been proposed recently and achieved considerable fruits. Nevertheless, these methods require certain auxiliary inputs besides CT images, such as segmentation masks of the tumor and organs at risk (OARs), which limits their prediction efficiency and application potential. To address this issue, we design a novel approach named as TransDose for dose distribution prediction that treats CT images as the unique input in this paper. Specifically, instead of inputting the segmentation masks to provide the prior anatomical information, we utilize a super-pixel-based graph convolutional network (GCN) to extract category-specific features, thereby compensating the network for the necessary anatomical knowledge. Besides, considering the strong continuous dependency between adjacent CT slices as well as adjacent dose maps, we embed the Transformer into the backbone, and make use of its superior ability of long-range sequence modeling to endow input features with inter-slice continuity message. To our knowledge, this is the first network that specially designed for the task of dose prediction from only CT images without ignoring necessary anatomical structure. Finally, we evaluate our model on two real datasets, and extensive experiments demonstrate the generalizability and advantages of our method.

Deep-Learning-Based Dose Predictor for Glioblastoma-Assessing the Sensitivity and Robustness for Dose Awareness in Contouring

External beam radiation therapy requires a sophisticated and laborious planning procedure. To improve the efficiency and quality of this procedure, machine-learning models that predict these dose distributions were introduced. The most recent dose prediction models are based on deep-learning architectures called 3D U-Nets that give good approximations of the dose in 3D almost instantly. Our purpose was to train such a 3D dose prediction model for glioblastoma VMAT treatment and test its robustness and sensitivity for the purpose of quality assurance of automatic contouring. From a cohort of 125 glioblastoma (GBM) patients, VMAT plans were created according to a clinical protocol. The initial model was trained on a cascaded 3D U-Net. A total of 60 cases were used for training, 15 for validation and 20 for testing. The prediction model was tested for sensitivity to dose changes when subject to realistic contour variations. Additionally, the model was tested for robustness by exposing it to a worst-case test set containing out-of-distribution cases. The initially trained prediction model had a dose score of 0.94 Gy and a mean DVH (dose volume histograms) score for all structures of 1.95 Gy. In terms of sensitivity, the model was able to predict the dose changes that occurred due to the contour variations with a mean error of 1.38 Gy. We obtained a 3D VMAT dose prediction model for GBM with limited data, providing good sensitivity to realistic contour variations. We tested and improved the model's robustness by targeted updates to the training set, making it a useful technique for introducing dose awareness in the contouring evaluation and quality assurance process.

TrDosePred: A deep learning dose prediction algorithm based on transformers for head and neck cancer radiotherapy

BACKGROUND

Intensity-Modulated Radiation Therapy (IMRT) has been the standard of care for many types of tumors. However, treatment planning for IMRT is a time-consuming and labor-intensive process.

PURPOSE

To alleviate this tedious planning process, a novel deep learning based dose prediction algorithm (TrDosePred) was developed for head and neck cancers.

METHODS

The proposed TrDosePred, which generated the dose distribution from a contoured CT image, was a U-shape network constructed with a convolutional patch embedding and several local self-attention based transformers. Data augmentation and ensemble approach were used for further improvement. It was trained based on the dataset from Open Knowledge-Based Planning Challenge (OpenKBP). The performance of TrDosePred was evaluated with two mean absolute error (MAE) based scores utilized by OpenKBP challenge (i.e., Dose score and DVH score) and compared to the top three approaches of the challenge. In addition, several state-of-the-art methods were implemented and compared to TrDosePred.

RESULTS

The TrDosePred ensemble achieved the dose score of 2.426 Gy and the DVH score of 1.592 Gy on the test dataset, ranking at 3rd and 9th respectively in the leaderboard on CodaLab as of writing. In terms of DVH metrics, on average, the relative MAE against the clinical plans was 2.25% for targets and 2.17% for organs at risk.

CONCLUSIONS

A transformer-based framework TrDosePred was developed for dose prediction. The results showed a comparable or superior performance as compared to the previous state-of-the-art approaches, demonstrating the potential of transformer to boost the treatment planning procedures.

Toward real-time automatic treatment planning (RTTP) with a one-step 3D fluence map prediction method and (nonorthogonal) convolution technique

PURPOSE

To establish and evaluate a (quasi) real-time automated treatment planning (RTTP) strategy utilizing a one-step full 3D fluence map prediction model based on a nonorthogonal convolution operation for rectal cancer radiotherapy.

METHODS

The RTTP approach directly extracts 3D projections from volumetric CT and anatomical data according to the beam incident direction. A 3D deep learning model with a nonorthogonal convolution operation was established that takes projections in cone beam space as input, extracts the features along and around the ray-trace path, and outputs a predicted fluence map (PFM) for each beam. The PFM is then converted to the MLC sequence with deliverable MUs to generate the final treatment plan. A total of 314 rectal adenocarcinoma patients with 2198 projection data samples were used in model training and validation. An extra 20 patients were used to test the feasibility of the RTTP method by comparing the plan quality, efficiency, deliverability performance, and physician blinded review results with the manual plans.

RESULTS

Overall, the RTTP plans met the clinical dose criteria for target coverage, conformity, homogeneity, and organ-at-risk dose sparing. Compared to manual plans, the RTTP plans showed increases in PTV D_1% by only 2.33% (p < 0.001) and a decrease in PTV D_99% by 0.45% (p < 0.05). The RTTP plans showed a dose increase in the bladder, with a V₅₀ of 14.01 ± 11.75% vs. 10.74 ± 8.51%, respectively, and no significant increases in the femoral head with the mean dose. The planning efficiency was improved in RTTP planning, with 39 s vs. 944 s in fluence map generation; the deliverability performance was saved by 1.91% (p < 0.001) in total MU. According to the blinded plan review by our physician, 55% of RTTP plans can be directly used in clinical radiotherapy treatment.

CONCLUSION

The quasi RTTP method improves the planning efficiency and deliverability performance while maintaining a plan quality close to that of the optimized manual plans in rectal radiotherapy.

Automated Robust Planning for IMPT in Oropharyngeal Cancer Patients Using Machine Learning

PURPOSE

The aim of this study was to evaluate an automated treatment planning method for robustly optimized intensity modulated proton therapy (IMPT) plans for oropharyngeal carcinoma patients and to compare the results with manually optimized robust IMPT plans.

METHODS AND MATERIALS

An atlas regression forest-based machine learning (ML) model for dose prediction was trained on CT scans, contours, and dose distributions of robust IMPT plans of 88 oropharyngeal cancer (OPC) patients. The ML model was combined with a robust voxel and dose volume histogram-based dose mimicking optimization algorithm for 21 perturbed scenarios to generate a machine-deliverable plan from the predicted dose distribution. Machine learning optimization (MLO) configuration was performed using a cross-validation approach with 3 × 8 tuning patients and comprised of adjustments to the mimicking optimization, to generate higher-quality MLO plans. Independent testing of the MLO algorithm was performed with another 25 patients. Plan quality of clinical and MLO plans was evaluated by the clinical target volume (D98% voxel-wise minimum dose >94%), organ at risk (OAR) doses, and the normal tissue complication probability (NTCP) (sum (Σ) of grade-2 and grade-3 dysphagia and xerostomia).

RESULTS

Adequate robust target coverage was achieved in 24/25 clinical plans and in 23/25 MLO plans in the primary clinical target volume (CTV). In the elective CTV, 22/25 clinical plans and 24/25 MLO plans passed the robust target coverage evaluation threshold. The MLO average Σgrade 2 and Σgrade 3 NTCPs were comparable to the clinical plans (Σgrade 2 NTCPs: clinical 47.5% vs MLO 48.4%, Σgrade 3 NTCPs: clinical 11.9% vs MLO 12.3%). Significant increases in OAR average doses in MLO plans were found in the pharynx constrictor muscles (163 cGy, P = .002) and cervical esophagus (265 cGy, P = .002). The MLO plans were created within 45 minutes.

CONCLUSION

This study showed that automated MLO planning can generate robustly optimized MLO plans with quality comparable to clinical plans in OPC patients.

Clinical application of a template-guided automated planning routine

PURPOSE

Determine the dosimetric quality and the planning time reduction when utilizing a template-based automated planning application.

METHODS

A software application integrated through the treatment planning system application programing interface, QuickPlan, was developed to facilitate automated planning using configurable templates for contouring, knowledge-based planning structure matching, field design, and algorithm settings. Validations are performed at various levels of the planning procedure and assist in the evaluation of readiness of the CT image, structure set, and plan layout for automated planning. QuickPlan is evaluated dosimetrically against 22 hippocampal-avoidance whole brain radiotherapy patients. The required times to treatment plan generation are compared for the validations set as well as 10 prospective patients whose plans have been automated by QuickPlan.

RESULTS

The generations of 22 automated treatment plans are compared against a manual replanning using an identical process, resulting in dosimetric differences of minor clinical significance. The target dose to 2% volume and homogeneity index result in significantly decreased values for automated plans, whereas other dose metric evaluations are nonsignificant. The time to generate the treatment plans is reduced for all automated plans with a median difference of 9' 50″ ± 4' 33″.

CONCLUSIONS

Template-based automated planning allows for reduced treatment planning time with consistent optimization structure creation, treatment field creation, plan optimization, and dose calculation with similar dosimetric quality. This process has potential expansion to numerous disease sites.

ALEC: Active learning with ensemble of classifiers for clinical diagnosis of coronary artery disease

Invasive angiography is the reference standard for coronary artery disease (CAD) diagnosis but is expensive and associated with certain risks. Machine learning (ML) using clinical and noninvasive imaging parameters can be used for CAD diagnosis to avoid the side effects and cost of angiography. However, ML methods require labeled samples for efficient training. The labeled data scarcity and high labeling costs can be mitigated by active learning. This is achieved through selective query of challenging samples for labeling. To the best of our knowledge, active learning has not been used for CAD diagnosis yet. An Active Learning with Ensemble of Classifiers (ALEC) method is proposed for CAD diagnosis, consisting of four classifiers. Three of these classifiers determine whether a patient's three main coronary arteries are stenotic or not. The fourth classifier predicts whether the patient has CAD or not. ALEC is first trained using labeled samples. For each unlabeled sample, if the outputs of the classifiers are consistent, the sample along with its predicted label is added to the pool of labeled samples. Inconsistent samples are manually labeled by medical experts before being added to the pool. The training is performed once more using the samples labeled so far. The interleaved phases of labeling and training are repeated until all samples are labeled. Compared with 19 other active learning algorithms, ALEC combined with a support vector machine classifier attained superior performance with 97.01% accuracy. Our method is justified mathematically as well. We also comprehensively analyze the CAD dataset used in this paper. As part of dataset analysis, features pairwise correlation is computed. The top 15 features contributing to CAD and stenosis of the three main coronary arteries are determined. The relationship between stenosis of the main arteries is presented using conditional probabilities. The effect of considering the number of stenotic arteries on sample discrimination is investigated. The discrimination power over dataset samples is visualized, assuming each of the three main coronary arteries as a sample label and considering the two remaining arteries as sample features.

Domain adaptation of automated treatment planning from computed tomography to magnetic resonance

Objective. Machine learning (ML) based radiation treatment planning addresses the iterative and time-consuming nature of conventional inverse planning. Given the rising importance of magnetic resonance (MR) only treatment planning workflows, we sought to determine if an ML based treatment planning model, trained on computed tomography (CT) imaging, could be applied to MR through domain adaptation. Methods. In this study, MR and CT imaging was collected from 55 prostate cancer patients treated on an MR linear accelerator. ML based plans were generated for each patient on both CT and MR imaging using a commercially available model in RayStation 8B. The dose distributions and acceptance rates of MR and CT based plans were compared using institutional dose-volume evaluation criteria. The dosimetric differences between MR and CT plans were further decomposed into setup, cohort, and imaging domain components. Results. MR plans were highly acceptable, meeting 93.1% of all evaluation criteria compared to 96.3% of CT plans, with dose equivalence for all evaluation criteria except for the bladder wall, penile bulb, small and large bowel, and one rectum wall criteria (p < 0.05). Changing the input imaging modality (domain component) only accounted for about half of the dosimetric differences observed between MR and CT plans. Anatomical differences between the ML training set and the MR linac cohort (cohort component) were also a significant contributor. Significance. We were able to create highly acceptable MR based treatment plans using a CT-trained ML model for treatment planning, although clinically significant dose deviations from the CT based plans were observed. Future work should focus on combining this framework with atlas selection metrics to create an interpretable quality assurance QA framework for ML based treatment planning.

Towards a safe and efficient clinical implementation of machine learning in radiation oncology by exploring model interpretability, explainability and data-model dependency

The interest in machine learning (ML) has grown tremendously in recent years, partly due to the performance leap that occurred with new techniques of deep learning, convolutional neural networks for images, increased computational power, and wider availability of large datasets. Most fields of medicine follow that popular trend and, notably, radiation oncology is one of those that are at the forefront, with already a long tradition in using digital images and fully computerized workflows. ML models are driven by data, and in contrast with many statistical or physical models, they can be very large and complex, with countless generic parameters. This inevitably raises two questions, namely, the tight dependence between the models and the datasets that feed them, and the interpretability of the models, which scales with its complexity. Any problems in the data used to train the model will be later reflected in their performance. This, together with the low interpretability of ML models, makes their implementation into the clinical workflow particularly difficult. Building tools for risk assessment and quality assurance of ML models must involve then two main points: interpretability and data-model dependency. After a joint introduction of both radiation oncology and ML, this paper reviews the main risks and current solutions when applying the latter to workflows in the former. Risks associated with data and models, as well as their interaction, are detailed. Next, the core concepts of interpretability, explainability, and data-model dependency are formally defined and illustrated with examples. Afterwards, a broad discussion goes through key applications of ML in workflows of radiation oncology as well as vendors' perspectives for the clinical implementation of ML.

Knowledge-based three-dimensional dose prediction for tandem-and-ovoid brachytherapy

PURPOSE

The purpose of this work was to develop a knowledge-based dose prediction system using a convolution neural network (CNN) for cervical brachytherapy treatments with a tandem-and-ovoid applicator.

METHODS

A 3D U-NET CNN was utilized to make voxel-wise dose predictions based on organ-at-risk (OAR), high-risk clinical target volume (HRCTV), and possible source location geometry. The model comprised 395 previously treated cases: training (273), validation (61), test (61). To assess voxel prediction accuracy, we evaluated dose differences in all cohorts across the dose range of 20-130% of prescription, mean(SD) and standard deviation (σ), as well as isodose dice similarity coefficients for clinical and/or predicted dose distributions. We examined discrete Dose-Volume Histogram(DVH) metrics utilized for brachytherapy plan quality assessment (HRCTV D90%, and bladder and/or rectum and/or sigmoid D2cc) with ΔD_x=D_x,actual-D_x,predicted Pearson correlation coefficient, standard deviation, and mean further quantifying model performance.

RESULTS

Ranges of voxel-wise dose difference accuracy (δD¯±σ) for 20-130% dose interval in training (test) sets ranged from [-0.5% ± 2.0% to +2.0% ± 14.0%] ([-0.1% ± 4.0% to +4.0% ± 26.0%]) in all voxels, [-1.7% ± 5.1% to -3.5% ± 12.8%] ([-2.9% ± 4.8% to -2.6% ± 18.9%]) in HRCTV, [-0.02% ± 2.40% to +3.2% ± 12.0%] ([-2.5% ± 3.6% to +0.8% ± 12.7%]) in bladder, [-0.7% ± 2.4% to +15.5% ± 11.0%] ([-0.9% ± 3.2% to +27.8% ± 11.6%]) in rectum, and [-0.7% ± 2.3% to +10.7% ± 15.0%] ([-0.4% ± 3.0% to +18.4% ± 11.4%]) in sigmoid. Isodose dice similarity coefficients ranged from [0.96,0.91] for training and [0.94,0.87] for test cohorts. Relative DVH metric prediction in the training (test) set were HRCTV ΔD¯₉₀±σ_ΔD=-0.19 ± 0.55Gy(-0.09 ± 0.67 Gy), bladder ΔD¯_2cc±σ_ΔD= -0.06 ± 0.54Gy(-0.17 ± 0.67 Gy), rectum ΔD¯_2cc±σ_ΔD= -0.03 ± 0.36Gy(-0.04 ± 0.46 Gy), and sigmoid ΔD¯_2cc±σ_ΔD= -0.01 ± 0.34Gy(0.00 ± 0.44 Gy).

CONCLUSIONS

A 3D knowledge-based dose predictions provide voxel-level and DVH metric estimates that could be used for treatment plan quality control and data-driven plan guidance.

Artificial Intelligence in Radiation Therapy

Artificial intelligence (AI) has great potential to transform the clinical workflow of radiotherapy. Since the introduction of deep neural networks, many AI-based methods have been proposed to address challenges in different aspects of radiotherapy. Commercial vendors have started to release AI-based tools that can be readily integrated to the established clinical workflow. To show the recent progress in AI-aided radiotherapy, we have reviewed AI-based studies in five major aspects of radiotherapy including image reconstruction, image registration, image segmentation, image synthesis, and automatic treatment planning. In each section, we summarized and categorized the recently published methods, followed by a discussion of the challenges, concerns, and future development. Given the rapid development of AI-aided radiotherapy, the efficiency and effectiveness of radiotherapy in the future could be substantially improved through intelligent automation of various aspects of radiotherapy.

A pilot study of machine-learning based automated planning for primary brain tumours

Purpose

High-quality radiotherapy (RT) planning for children and young adults with primary brain tumours is essential to minimize the risk of late treatment effects. The feasibility of using automated machine-learning (ML) to aid RT planning in this population has not previously been studied.

Methods and materials

We developed a ML model that identifies learned relationships between image features and expected dose in a training set of 95 patients with a primary brain tumour treated with focal radiotherapy to a dose of 54 Gy in 30 fractions. This ML method was then used to create predicted dose distributions for 15 previously-treated brain tumour patients across two institutions, as a testing set. Dosimetry to target volumes and organs-at-risk (OARs) were compared between the clinically-delivered (human-generated) plans versus the ML plans.

Results

The ML method was able to create deliverable plans in all 15 patients in the testing set. All ML plans were generated within 30 min of initiating planning. Planning target volume coverage with 95% of the prescription dose was attained in all plans. OAR doses were similar across most structures evaluated; mean doses to brain and left temporal lobe were lower in ML plans than manual plans (mean difference to left temporal, – 2.3 Gy, p = 0.006; mean differences to brain, – 1.3 Gy, p = 0.017), whereas mean doses to right cochlea and lenses were higher in ML plans (+ 1.6–2.2 Gy, p < 0.05 for each).

Conclusions

Use of an automated ML method to aid RT planning for children and young adults with primary brain tumours is dosimetrically feasible and can be successfully used to create high-quality 54 Gy RT plans. Further evaluation after clinical implementation is planned.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13014-021-01967-3.

Development and Implementation of an Open Source Template Interpretation Class Library for Automated Treatment Planning

PURPOSE

Widespread implementation of automated treatment planning in radiation therapy remains elusive due to variability in clinic and physician preferences making it difficult to ensure consistent plan parameters. We have developed an open-source class library with the aim to improve efficiency and consistency for automated treatment planning in radiation therapy.

METHODS AND MATERIALS

An open source class library has been developed that interprets clinical templates within a commercial treatment planning system into a treatment plan for automated planning. This code was leveraged for the automated planning of 39 patients and retrospectively compared to the 78 clinically approved manual plans.

RESULTS

From the initial 39 patients, 74 of 78 plans were successfully generated without manual intervention. Target dose was more homogenous for automated plans, with an average homogeneity index of 3.30 vs 3.11 for manual and automated plans, respectively (p = 0.107). Generalized equivalent uniform dose decreased in the femurs and rectum for automated plans, with mean gEUD of 3746 cGy vs 3338 cGy (p ≤ 0.001) and 5761 cGy vs 5634 cGy (p ≤ 0.001) for femurs and rectum, respectively. Dose metrics for bladder and rectum (V6500 cGy and V4000 cGy) show recognizable but insignificant improvements. All automated plans delivered for quality assurance passed a gamma analysis (>95%) with an average composite pass rate of 99.3% and 98.8% for pelvis and prostate plans, respectively. Deliverability parameters such as total monitor units and aperture complexity indicate deliverable plans.

CONCLUSIONS

Prostate cancer and pelvic node radiotherapy can be automated using VMAT planning and clinical templates based on a standardized clinical workflow. The class library developed in this study conveniently interfaces between the plan template and the treatment planning system to automatically generate high quality plans on customizable templates.

Sharp loss: a new loss function for radiotherapy dose prediction based on fully convolutional networks

Background

Neural-network methods have been widely used for the prediction of dose distributions in radiotherapy. However, the prediction accuracy of existing methods may be degraded by the problem of dose imbalance. In this work, a new loss function is proposed to alleviate the dose imbalance and achieve more accurate prediction results. The U-Net architecture was employed to build a prediction model. Our study involved a total of 110 patients with left-breast cancer, who were previously treated by volumetric-modulated arc radiotherapy. The patient dataset was divided into training and test subsets of 100 and 10 cases, respectively. We proposed a novel ‘sharp loss’ function, and a parameter γ was used to adjust the loss properties. The mean square error (MSE) loss and the sharp loss with different γ values were tested and compared using the Wilcoxon signed-rank test.

Results

The sharp loss achieved superior dose prediction results compared to those of the MSE loss. The best performance with the MSE loss and the sharp loss was obtained when the parameter γ was set to 100. Specifically, the mean absolute difference values for the planning target volume were 318.87 ± 30.23 for the MSE loss versus 144.15 ± 16.27 for the sharp loss with γ = 100 (p < 0.05). The corresponding values for the ipsilateral lung, the heart, the contralateral lung, and the spinal cord were 278.99 ± 51.68 versus 198.75 ± 61.38 (p < 0.05), 216.99 ± 44.13 versus 144.86 ± 43.98 (p < 0.05), 125.96 ± 66.76 versus 111.86 ± 47.19 (p > 0.05), and 194.30 ± 14.51 versus 168.58 ± 25.97 (p < 0.05), respectively.

Conclusions

The sharp loss function could significantly improve the accuracy of radiotherapy dose prediction.

Knowledge-based radiation treatment planning: A data-driven method survey

This paper surveys the data-driven dose prediction methods investigated for knowledge-based planning (KBP) in the last decade. These methods were classified into two major categories-traditional KBP methods and deep-learning (DL) methods-according to their techniques of utilizing previous knowledge. Traditional KBP methods include studies that require geometric or anatomical features to either find the best-matched case(s) from a repository of prior treatment plans or to build dose prediction models. DL methods include studies that train neural networks to make dose predictions. A comprehensive review of each category is presented, highlighting key features, methods, and their advancements over the years. We separated the cited works according to the framework and cancer site in each category. Finally, we briefly discuss the performance of both traditional KBP methods and DL methods, then discuss future trends of both data-driven KBP methods to dose prediction.

An atlas-guided automatic planning approach for rectal cancer intensity-modulated radiotherapy

We try to develop an atlas-guided automatic planning (AGAP) approach and evaluate its feasibility and performance in rectal cancer intensity-modulated radiotherapy. The developed AGAP approach consisted of four independent modules: patient atlas, similar patient retrieval, beam morphing (BM), and plan fine-tuning (PFT) modules. The atlas was setup using anatomy and plan data from Pinnacle auto-planning (P-auto) plans. Given a new patient, the retrieval function searched the top similar patient by a generic Fourier descriptor algorithm and retrieved its plan information. The BM function generated an initial plan for the new patient by morphing the beam aperture from the top similar patient plan. The beam aperture and calculated dose of the initial plan were used to guide the new plan optimization in the PFT function. The AGAP approach was tested on 96 patients by the leave-one-out validation and plan quality was compared with the P-auto plans. The AGAP and P-auto plans had no statistical difference for target coverage and dose homogeneity in terms ofV_100%(p = 0.76) and homogeneity index (p = 0.073), respectively. The CI index showed they had a statistically significant difference. But the ΔCI was both 0.02 compared to the perfect CI index of 1. The AGAP approach reduced the bladder mean dose by 152.1 cGy (p < 0.05) andV₅₀by 0.9% (p < 0.05), and slightly increased the left and right femoral head mean dose by 70.1 cGy (p < 0.05) and 69.7 cGy (p < 0.05), respectively. This work developed an efficient and automatic approach that could fully automate the IMRT planning process in rectal cancer radiotherapy. It reduced the plan quality dependence on the planner experience and maintained the comparable plan quality with P-auto plans.

Comparing of two dimensional and three dimensional fully convolutional networks for radiotherapy dose prediction in left-sided breast cancer

Fully convolutional networks were developed for predicting optimal dose distributions for patients with left-sided breast cancer and compared the prediction accuracy between two-dimensional and three-dimensional networks. Sixty cases treated with volumetric modulated arc radiotherapy were analyzed. Among them, 50 cases were randomly chosen to conform the training set, and the remaining 10 were to construct the test set. Two U-Net fully convolutional networks predicted the dose distributions, with two-dimensional and three-dimensional convolution kernels, respectively. Computed tomography images, delineated regions of interest, or their combination were considered as input data. The accuracy of predicted results was evaluated against the clinical dose. Most types of input data retrieved a similar dose to the ground truth for organs at risk (p > 0.05). Overall, the two-dimensional model had higher performance than the three-dimensional model (p < 0.05). Moreover, the two-dimensional region of interest input provided the best prediction results regarding the planning target volume mean percentage difference (2.40 ± 0.18%), heart mean percentage difference (4.28 ± 2.02%), and the gamma index at 80% of the prescription dose are with tolerances of 3 mm and 3% (0.85 ± 0.03), whereas the two-dimensional combined input provided the best prediction regarding ipsilateral lung mean percentage difference (4.16 ± 1.48%), lung mean percentage difference (2.41 ± 0.95%), spinal cord mean percentage difference (0.67 ± 0.40%), and 80% Dice similarity coefficient (0.94 ± 0.01). Statistically, the two-dimensional combined inputs achieved higher prediction accuracy regarding 80% Dice similarity coefficient than the two-dimensional region of interest input (0.94 ± 0.01 vs 0.92 ± 0.01, p < 0.05). The two-dimensional data model retrieves higher performance than its three-dimensional counterpart for dose prediction, especially when using region of interest and combined inputs.

Volumetric modulated arc therapy dose prediction and deliverable treatment plan generation for prostate cancer patients using a densely connected deep learning model

BACKGROUND AND PURPOSE

Radiation therapy treatment planning is a manual, time-consuming task that might be accelerated using machine learning algorithms. In this study, we aimed to evaluate if a triplet-based deep learning model can predict volumetric modulated arc therapy (VMAT) dose distributions for prostate cancer patients.

MATERIALS AND METHODS

A modified U-Net was trained on triplets, a combination of three consecutive image slices and corresponding segmentations, from 160 patients, and compared to a baseline U-Net. Dose predictions from 17 test patients were transformed into deliverable treatment plans using a novel planning workflow.

RESULTS

The model achieved a mean absolute dose error of 1.3%, 1.9%, 1.0% and ≤ 2.6% for clinical target volume (CTV) CTV_D100%, planning target volume (PTV) PTV_D98%, PTV_D95% and organs at risk (OAR) respectively, when compared to the clinical ground truth (GT) dose distributions. All predicted distributions were successfully transformed into deliverable treatment plans and tested on a phantom, resulting in a passing rate of 100% (global gamma, 3%, 2 mm, 15% cutoff). The dose difference between deliverable treatment plans and GT dose distributions was within 4.4%. The difference between the baseline model and our improved model was statistically significant (p < 0.05) for CVT_D100%, PTV_D98% and PTV_D95%.

CONCLUSION

Triplet-based training improved VMAT dose distribution predictions when compared to 2D. Dose predictions were successfully transformed into deliverable treatment plans using our proposed treatment planning procedure. Our method may automate parts of the workflow for external beam prostate radiation therapy and improve the overall treatment speed and plan quality.

OpenKBP: The open-access knowledge-based planning grand challenge and dataset

PURPOSE

To advance fair and consistent comparisons of dose prediction methods for knowledge-based planning (KBP) in radiation therapy research.

METHODS

We hosted OpenKBP, a 2020 AAPM Grand Challenge, and challenged participants to develop the best method for predicting the dose of contoured computed tomography (CT) images. The models were evaluated according to two separate scores: (a) dose score, which evaluates the full three-dimensional (3D) dose distributions, and (b) dose-volume histogram (DVH) score, which evaluates a set DVH metrics. We used these scores to quantify the quality of the models based on their out-of-sample predictions. To develop and test their models, participants were given the data of 340 patients who were treated for head-and-neck cancer with radiation therapy. The data were partitioned into training ( n = 200 ), validation ( n = 40 ), and testing ( n = 100 ) datasets. All participants performed training and validation with the corresponding datasets during the first (validation) phase of the Challenge. In the second (testing) phase, the participants used their model on the testing data to quantify the out-of-sample performance, which was hidden from participants and used to determine the final competition ranking. Participants also responded to a survey to summarize their models.

RESULTS

The Challenge attracted 195 participants from 28 countries, and 73 of those participants formed 44 teams in the validation phase, which received a total of 1750 submissions. The testing phase garnered submissions from 28 of those teams, which represents 28 unique prediction methods. On average, over the course of the validation phase, participants improved the dose and DVH scores of their models by a factor of 2.7 and 5.7, respectively. In the testing phase one model achieved the best dose score (2.429) and DVH score (1.478), which were both significantly better than the dose score (2.564) and the DVH score (1.529) that was achieved by the runner-up models. Lastly, many of the top performing teams reported that they used generalizable techniques (e.g., ensembles) to achieve higher performance than their competition.

CONCLUSION

OpenKBP is the first competition for knowledge-based planning research. The Challenge helped launch the first platform that enables researchers to compare KBP prediction methods fairly and consistently using a large open-source dataset and standardized metrics. OpenKBP has also democratized KBP research by making it accessible to everyone, which should help accelerate the progress of KBP research. The OpenKBP datasets are available publicly to help benchmark future KBP research.

Performance stability evaluation of atlas-based machine learning radiation therapy treatment planning in prostate cancer

Atlas-based machine learning (ML) for radiation therapy (RT) treatment planning is effective at tailoring dose distributions to account for unique patient anatomies by selecting the most appropriate patients from the training database (atlases) to inform dose prediction for new patients. However, variations in clinical practice between the training dataset and a new patient to be planned may impact ML performance by confounding atlas selection. In this study, we simulated various contouring practices in prostate cancer RT to investigate the impact of changing input data on atlas-based ML treatment planning. We generated 225 ML plans for nine bespoke contouring protocol scenarios (reduced target margins, modified organ-at-risk (OAR) definitions, and inclusion of optional OARs less represented in the training database) on 25 patient datasets by applying a single, previously trained and validated ML model for prostate cancer followed by dose mimicking to create a final deliverable plan. ML treatment plans for each scenario were compared to base ML treatment plans that followed a contouring protocol consistent with the model training data. ML performance was evaluated based on atlas distance metrics that are calculated during ML dose prediction. There were significant changes between atlases selected for the base ML treatment plans and treatment plans when planning target volume margins were reduced and/or optional OARs were included. The deliverability of ML predicted dose distributions based on gamma analysis between predicted and mimicked final deliverable dose showed significant differences for seven out of eight scenarios compared with the base ML treatment plans. Overall, there were small but statistically significant dosimetric changes in predicted and mimicked dose with addition of optional OAR contours. This work presents a framework for benchmarking and performance monitoring of ML treatment planning algorithms in the context of evolving clinical practices.

Clinical integration of machine learning for curative-intent radiation treatment of patients with prostate cancer

Machine learning (ML) holds great promise for impacting healthcare delivery; however, to date most methods are tested in 'simulated' environments that cannot recapitulate factors influencing real-world clinical practice. We prospectively deployed and evaluated a random forest algorithm for therapeutic curative-intent radiation therapy (RT) treatment planning for prostate cancer in a blinded, head-to-head study with full integration into the clinical workflow. ML- and human-generated RT treatment plans were directly compared in a retrospective simulation with retesting (n = 50) and a prospective clinical deployment (n = 50) phase. Consistently throughout the study phases, treating physicians assessed ML- and human-generated RT treatment plans in a blinded manner following a priori defined standardized criteria and peer review processes, with the selected RT plan in the prospective phase delivered for patient treatment. Overall, 89% of ML-generated RT plans were considered clinically acceptable and 72% were selected over human-generated RT plans in head-to-head comparisons. RT planning using ML reduced the median time required for the entire RT planning process by 60.1% (118 to 47 h). While ML RT plan acceptability remained stable between the simulation and deployment phases (92 versus 86%), the number of ML RT plans selected for treatment was significantly reduced (83 versus 61%, respectively). These findings highlight that retrospective or simulated evaluation of ML methods, even under expert blinded review, may not be representative of algorithm acceptance in a real-world clinical setting when patient care is at stake.

Towards the Interpretability of Machine Learning Predictions for Medical Applications Targeting Personalised Therapies: A Cancer Case Survey

Artificial Intelligence is providing astonishing results, with medicine being one of its favourite playgrounds. Machine Learning and, in particular, Deep Neural Networks are behind this revolution. Among the most challenging targets of interest in medicine are cancer diagnosis and therapies but, to start this revolution, software tools need to be adapted to cover the new requirements. In this sense, learning tools are becoming a commodity but, to be able to assist doctors on a daily basis, it is essential to fully understand how models can be interpreted. In this survey, we analyse current machine learning models and other in-silico tools as applied to medicine-specifically, to cancer research-and we discuss their interpretability, performance and the input data they are fed with. Artificial neural networks (ANN), logistic regression (LR) and support vector machines (SVM) have been observed to be the preferred models. In addition, convolutional neural networks (CNNs), supported by the rapid development of graphic processing units (GPUs) and high-performance computing (HPC) infrastructures, are gaining importance when image processing is feasible. However, the interpretability of machine learning predictions so that doctors can understand them, trust them and gain useful insights for the clinical practice is still rarely considered, which is a factor that needs to be improved to enhance doctors' predictive capacity and achieve individualised therapies in the near future.

Artificial intelligence and machine learning for medical imaging: A technology review

Artificial intelligence (AI) has recently become a very popular buzzword, as a consequence of disruptive technical advances and impressive experimental results, notably in the field of image analysis and processing. In medicine, specialties where images are central, like radiology, pathology or oncology, have seized the opportunity and considerable efforts in research and development have been deployed to transfer the potential of AI to clinical applications. With AI becoming a more mainstream tool for typical medical imaging analysis tasks, such as diagnosis, segmentation, or classification, the key for a safe and efficient use of clinical AI applications relies, in part, on informed practitioners. The aim of this review is to present the basic technological pillars of AI, together with the state-of-the-art machine learning methods and their application to medical imaging. In addition, we discuss the new trends and future research directions. This will help the reader to understand how AI methods are now becoming an ubiquitous tool in any medical image analysis workflow and pave the way for the clinical implementation of AI-based solutions.

Development and evaluation of radiotherapy deep learning dose prediction models for breast cancer

Background and purpose

Treatment planning of radiotherapy is a time-consuming and planner dependent process that can be automated by dose prediction models. The purpose of this study was to evaluate the performance of two machine learning models for breast cancer radiotherapy before possible clinical implementation.

Materials and methods

An in-house developed model, based on U-net architecture, and a contextual atlas regression forest (cARF) model integrated in the treatment planning software were trained. Obtained dose distributions were mimicked to create clinically deliverable plans. For training and validation, 90 patients were used, 15 patients were used for testing. Treatment plans were scored on predefined evaluation criteria and percent errors with respect to clinical dose were calculated for doses to planning target volume (PTV) and organs at risk (OARs).

Results

The U-net plans before mimicking met all criteria for all patients, both models failed one evaluation criterion in three patients after mimicking. No significant differences (p < 0.05) were found between clinical and predicted U-net plans before mimicking. Doses to OARs in plans of both models differed significantly from clinical plans, but no clinically relevant differences were found. After mimicking, both models had a mean percent error within 1.5% for the average dose to PTV and OARs. The mean errors for maximum doses were higher, within 6.6%.

Conclusions

Differences between predicted doses to OARs of the models were small when compared to clinical plans, and not found to be clinically relevant. Both models show potential in automated treatment planning for breast cancer.

Fluence Map Prediction Using Deep Learning Models - Direct Plan Generation for Pancreas Stereotactic Body Radiation Therapy

Purpose: Treatment planning for pancreas stereotactic body radiation therapy (SBRT) is a difficult and time-consuming task. In this study, we aim to develop a novel deep learning framework to generate clinical-quality plans by direct prediction of fluence maps from patient anatomy using convolutional neural networks (CNNs).

Materials and Methods: Our proposed framework utilizes two CNNs to predict intensity-modulated radiation therapy fluence maps and generate deliverable plans: (1) Field-dose CNN predicts field-dose distributions in the region of interest using planning images and structure contours; (2) a fluence map CNN predicts the final fluence map per beam using the predicted field dose projected onto the beam's eye view. The predicted fluence maps were subsequently imported into the treatment planning system for leaf sequencing and final dose calculation (model-predicted plans). One hundred patients previously treated with pancreas SBRT were included in this retrospective study, and they were split into 85 training cases and 15 test cases. For each network, 10% of training data were randomly selected for model validation. Nine-beam benchmark plans with standardized target prescription and organ-at-risk constraints were planned by experienced clinical physicists and used as the gold standard to train the model. Model-predicted plans were compared with benchmark plans in terms of dosimetric endpoints, fluence map deliverability, and total monitor units.

Results: The average time for fluence-map prediction per patient was 7.1 s. Comparing model-predicted plans with benchmark plans, target mean dose, maximum dose (0.1 cc), and D_95% absolute differences in percentages of prescription were 0.1, 3.9, and 2.1%, respectively; organ-at-risk mean dose and maximum dose (0.1 cc) absolute differences were 0.2 and 4.4%, respectively. The predicted plans had fluence map gamma indices (97.69 ± 0.96% vs. 98.14 ± 0.74%) and total monitor units (2,122 ± 281 vs. 2,265 ± 373) that were comparable to the benchmark plans.

Conclusions: We develop a novel deep learning framework for pancreas SBRT planning, which predicts a fluence map for each beam and can, therefore, bypass the lengthy inverse optimization process. The proposed framework could potentially change the paradigm of treatment planning by harnessing the power of deep learning to generate clinically deliverable plans in seconds.

An Innovative Learning Tool for Radiation Therapy Treatment Plan Evaluation: Implementation and Evaluation

PURPOSE

To design, develop, and evaluate an interactive simulation-based learning tool for treatment plan evaluation for radiation oncology and medical physics residents to address gaps in learning.

METHODS AND MATERIALS

We first conducted a needs assessment for optimal learning tool design and case selection. Next, we generated a curated database of cases with clinically unacceptable treatment plans accessible through an in-house developed interactive web-based digital imaging and communications in medicine-radiation therapy viewer. We then developed an interactive user module that allows case selection, learner participation, and immediate feedback, including the final clinically acceptable plan. We pilot tested this case bank learning tool with current radiation oncology and medical physics residents within our institution. Afterward, residents completed an evaluation of tool design, content, and perceived impact on learning and provided suggestions for improvement.

RESULTS

We generated 70 cases and learning modules for the case bank, encompassing various clinical sites, levels of difficulty, and classified errors. Residents positively endorsed the learning tool, including design, content, and perceived impact on learning. The learning tool's interactivity was perceived to provide increased educational value compared with other current learning methods.

CONCLUSIONS

We created a high-fidelity simulation platform for treatment plan evaluation linked to a curated case bank. Evaluation of the pilot deployment demonstrated a benefit for resident learning and competency attainment. Future directions include external validation and expansion.

Development and evaluation of machine learning models for voxel dose predictions in online adaptive magnetic resonance guided radiation therapy

PURPOSE

Daily online adaptive plan quality in magnetic resonance imaging guided radiation therapy (MRgRT) is difficult to assess in relation to the fully optimized, high quality plans traditionally established offline. Machine learning prediction models developed in this work are capable of predicting 3D dose distributions, enabling the evaluation of online adaptive plan quality to better inform adaptive decision-making in MRgRT.

METHODS

Artificial neural networks predicted 3D dose distributions from input variables related to patient anatomy, geometry, and target/organ-at-risk relationships in over 300 treatment plans from 53 patients receiving adaptive, linac-based MRgRT for abdominal cancers. The models do not include any beam related variables such as beam angles or fluence and were optimized to balance errors related to raw dose and specific plan quality metrics used to guide daily online adaptive decisions.

RESULTS

Averaged over all plans, the dose prediction error and the absolute error were 0.1 ± 3.4 Gy (0.1 ± 6.2%) and 3.5 ± 2.4 Gy (6.4 ± 4.3%) respectively. Plan metric prediction errors were -0.1 ± 1.5%, -0.5 ± 2.1%, -0.9 ± 2.2 Gy, and 0.1 ± 2.7 Gy for V95, V100, D95, and Dmean respectively. Plan metric prediction absolute errors were 1.1 ± 1.1%, 1.5 ± 1.5%, 1.9 ± 1.4 Gy, and 2.2 ± 1.6 Gy. Approximately 10% (25) of the plans studied were clearly identified by the prediction models as inferior quality plans needing further optimization and refinement.

CONCLUSION

Machine learning prediction models for treatment plan 3D dose distributions in online adaptive MRgRT were developed and tested. Clinical integration of the models requires minimal effort, producing 3D dose predictions for a new patient's plan using only target and OAR structures as inputs. These models can enable improved workflows for MRgRT through more informed plan optimization and plan quality assessment in real time.

Radiotherapy dose distribution prediction for breast cancer using deformable image registration

BACKGROUND

Radiotherapy treatment planning dose prediction can be used to ensure plan quality and guide automatic plan. One of the dose prediction methods is incorporating historical treatment planning data into algorithms to estimate the dose-volume histogram (DVH) of organ for new patients. Although DVH is used extensively in treatment plan quality and radiotherapy prognosis evaluation, three-dimensional dose distribution can describe the radiation effects more explicitly. The purpose of this retrospective study was to predict the dose distribution of breast cancer radiotherapy by means of deformable registration into atlas images with historical treatment planning data that were considered to achieve expert level. The atlas cohort comprised 20 patients with left-sided breast cancer, previously treated by volumetric-modulated arc radiotherapy. The registration-based prediction technique was applied to 20 patients outside the atlas cohort. This study evaluated and compared three different approaches: registration to the most similar image from a dataset of individual atlas images (SIM), registration to all images from a database of individual atlas images with the average method (WEI_A), and the weighted method (WEI_F). The dose prediction performance of each strategy was quantified using nine metrics, including the region of interest dose error, 80% and 100% prescription area dice similarity coefficients (DSCs), and γ metrics. A Friedman test and a nonparametric exact Wilcoxon signed rank test were performed to compare the differences among groups. The clinical doses of all cases served as the gold standard.

RESULTS

The WEI_F method could achieve superior dose prediction results to those of WEI_A. WEI_F outperformed SIM in the organ-at-risk mean absolute difference (MAD). When using the WEI_F method, the MAD values for the ipsilateral lung, heart, and whole lung were 197.9 ± 42.9, 166 ± 55.1, 122.3 ± 25.5, and 55.3 ± 42.2 cGy, respectively. Moreover, SIM exhibited superior prediction in the DSC and γ metrics. When using the SIM method, the means of the 80% and 100% prescription area DSCs, 33γ metric, and 55γ metric were 0.85 ± 0.05, 0.84 ± 0.05, 0.64 ± 0.13, and 0.84 ± 0.10, respectively. The plan target volume and spinal cord MAD when using SIM and WEI were 235.6 ± 158.4 cGy versus 227.4 ± 144.0 cGy ([Formula: see text]) and 61.4 ± 44.9 cGy versus 55.3 ± 42.2 cGy ([Formula: see text]), respectively.

CONCLUSIONS

A predicted dose distribution that approximated the clinical plan could be generated using the methods presented in this study.

A method of using deep learning to predict three-dimensional dose distributions for intensity-modulated radiotherapy of rectal cancer

Purpose

To develop and test a three‐dimensional (3D) deep learning model for predicting 3D voxel‐wise dose distributions for intensity‐modulated radiotherapy (IMRT).

Methods

A total of 122 postoperative rectal cancer cases treated by IMRT were considered in the study, of which 100 cases were randomly selected as the training–validating set and the remaining as the testing set. A 3D deep learning model named 3D U‐Res‐Net_B was constructed to predict 3D dose distributions. Eight types of 3D matrices from CT images, contoured structures, and beam configurations were fed into the independent input channel, respectively, and the 3D matrix of dose distributions was taken as the output to train the 3D model. The obtained 3D model was used to predict new 3D dose distributions. The predicted accuracy was evaluated in two aspects: (a) The dice similarity coefficients (DSCs) of different isodose volumes, the average dose difference of all voxels within the body, and 3%/5 mm global gamma passing rates of organs at risks (OARs) and planned target volume (PTV) were used to address the spatial correspondence between predicted and clinical delivered 3D dose distributions; (b) The dosimetric index (DI) including homogeneity index, conformity index, V₅₀, V₄₅ for PTV and OARs between predicted and clinical truth were statistically analyzed with the paired‐samples t test. The model was also compared with 3D U‐Net and the same architecture model without beam configurations input (named as 3D U‐Res‐Net_O).

Results

The 3D U‐Res‐Net_B model predicted 3D dose distributions accurately. For the 22 testing cases, the average prediction bias ranged from −1.94% to 1.58%, and the overall mean absolute errors (MAEs) was 3.92 ± 4.16%; there was no statistically significant difference for nearly all DIs. The model had a DSCs value above 0.9 for most isodose volumes, and global 3D gamma passing rates varying from 0.81 to 0.90 for PTV and OARs, clearly outperforming 3D U‐Res‐Net_O and being slightly superior to 3D U‐Net.

Conclusions

This study developed a more general deep learning model by considering beam configurations input and achieved an accurate 3D voxel‐wise dose prediction for rectal cancer treated by IMRT, a potentially easier clinical implementation for more comprehensive automatic planning.

Fluence-map generation for prostate intensity-modulated radiotherapy planning using a deep-neural-network

A deep-neural-network (DNN) was successfully used to predict clinically-acceptable dose distributions from organ contours for intensity-modulated radiotherapy (IMRT). To provide the next step in the DNN-based plan automation, we propose a DNN that directly generates beam fluence maps from the organ contours and volumetric dose distributions, without inverse planning. We collected 240 prostate IMRT plans and used to train a DNN using organ contours and dose distributions. After training was done, we made 45 synthetic plans (SPs) using the generated fluence-maps and compared them with clinical plans (CP) using various plan quality metrics including homogeneity and conformity indices for the target and dose constraints for organs at risk, including rectum, bladder, and bowel. The network was able to generate fluence maps with small errors. The qualities of the SPs were comparable to the corresponding CPs. The homogeneity index of the target was slightly worse in the SPs, but there was no difference in conformity index of the target, V_60Gy of rectum, the V_60Gy of bladder and the V_45Gy of bowel. The time taken for generating fluence maps and qualities of SPs demonstrated the proposed method will improve efficiency of the treatment planning and help maintain the quality of plans.

Knowledge-based planning for intensity-modulated radiation therapy: A review of data-driven approaches

Purpose

Intensity‐Modulated Radiation Therapy (IMRT), including its variations (including IMRT, Volumetric Arc Therapy (VMAT), and Tomotherapy), is a widely used and critically important technology for cancer treatment. It is a knowledge‐intensive technology due not only to its own technical complexity, but also to the inherently conflicting nature of maximizing tumor control while minimizing normal organ damage. As IMRT experience and especially the carefully designed clinical plan data are accumulated during the past two decades, a new set of methods commonly termed knowledge‐based planning (KBP) have been developed that aim to improve the quality and efficiency of IMRT planning by learning from the database of past clinical plans. Some of this development has led to commercial products recently that allowed the investigation of KBP in numerous clinical applications. In this literature review, we will attempt to present a summary of published methods of knowledge‐based approaches in IMRT and recent clinical validation results.

Methods

In March 2018, a literature search was conducted in the NIH Medline database using the PubMed interface to identify publications that describe methods and validations related to KBP in IMRT including variations such as VMAT and Tomotherapy. The search criteria were designed to have a broad scope to capture relevant results with high sensitivity. The authors filtered down the search results according to a predefined selection criteria by reviewing the titles and abstracts first and then by reviewing the full text. A few papers were added to the list based on the references of the reviewed papers. The final set of papers was reviewed and summarized here.

Results

The initial search yielded a total of 740 articles. A careful review of the titles, abstracts, and eventually the full text and then adding relevant articles from reviewing the references resulted in a final list of 73 articles published between 2011 and early 2018. These articles described methods for developing knowledge models for predicting such parameters as dosimetric and dose‐volume points, voxel‐level doses, and objective function weights that improve or automate IMRT planning for various cancer sites, addressing different clinical and quality assurance needs, and using a variety of machine learning approaches. A number of articles reported carefully designed clinical studies that assessed the performance of KBP models in realistic clinical applications. Overwhelming majority of the studies demonstrated the benefits of KBP in achieving comparable and often improved quality of IMRT planning while reducing planning time and plan quality variation.

Conclusions

The number of KBP‐related studies has been steadily increasing since 2011 indicating a growing interest in applying this approach to clinical applications. Validation studies have generally shown KBP to produce plans with quality comparable to expert planners while reducing the time and efforts to generate plans. However, current studies are mostly retrospective and leverage relatively small datasets. Larger datasets collected through multi‐institutional collaboration will enable the development of more advanced models to further improve the performance of KBP in complex clinical cases. Prospective studies will be an important next step toward widespread adoption of this exciting technology.

An atlas-based method to predict three-dimensional dose distributions for cancer patients who receive radiotherapy

Due to the complexity of advanced radiotherapy techniques, treatment planning process is usually time consuming and plan quality can vary considerably among planners and institutions. It is also impractical to generate all possible treatment plans based on available radiotherapy techniques and select the best option for a specific patient. Automatic dose prediction will be very helpful in these situations, while there were a few studies of three-dimensional (3D) dose prediction for patients who received radiotherapy. The purpose of this work was to develop a novel atlas-based method to predict 3D dose prediction and to evaluate its performance. Previously treated nineteen left-sided post-mastectomy breast cancer patients and sixteen prostate cancer patients were included in this study. One patient was arbitrarily chosen as the reference for each type of cancer and all the remaining patients' computed tomography (CT) images and contours were aligned to it using deformable image registration (DIR). Deformable vector field (DVF) for each patient i (DVF _i-ref) was used to deform the original 3D dose matrix of that patient. CT scan of a test patient was also registered with the same reference patient using DIR and both direct DVF (DVF_test-ref) and inverse DVF ([Formula: see text]) were derived. Similarity of atlas patients to the test patient was determined based on the similarity of DVF_test-ref to atlas DVFs (DVF _i-ref) and appropriate weighting factors were calculated. Patients' doses in the atlas were deformed again using [Formula: see text] to transform them from the reference patient's coordinates to the test patient's coordinates and the final 3D dose distribution for the test patient was predicted by summing the weighted individual 3D dose distributions. Performance of our method was evaluated and the results revealed that the proposed method was able to predict the 3D dose distributions accurately. The mean dose difference between clinical and predicted 3D dose distributions were 0.9  ±  1.1 Gy and 1.9  ±  1.2 Gy for breast and prostate plans. The proposed dose prediction method can be used to improve planning quality and facilitate plan comparisons.

A deep learning method for prediction of three-dimensional dose distribution of helical tomotherapy

PURPOSE

To develop a deep learning method for prediction of three-dimensional (3D) voxel-by-voxel dose distributions of helical tomotherapy (HT).

METHODS

Using previously treated HT plans as training data, a deep learning model named U-ResNet-D was trained to predict a 3D dose distribution. First, the contoured structures and dose volumes were converted from plan database to 3D matrix with a program based on a developed visualization toolkit (VTK), then transferred to U-ResNet-D for correlating anatomical features and dose distributions at voxel-level. One hundred and ninety nasopharyngeal cancer (NPC) patients treated by HT with multiple planning target volumes (PTVs) in different prescription patterns were studied. The model was typically trained from scratch with weights randomly initialized rather than using transfer-learning method, and used to predict new patient's 3D dose distributions. The predictive accuracy was evaluated with three methods: (a) The dose difference at the position r, δ(r, r) = D_c (r) - D_p (r), was calculated for each voxel. The mean (μ_δ(r,r) ) and standard deviation (σ_δ(r,r) ) of δ(r, r) were calculated to assess the prediction bias and precision; (b) The mean absolute differences of dosimetric indexes (DIs) including maximum and mean dose, homogeneity index, conformity index, and dose spillage for PTVs and organ at risks (OARs) were calculated and statistically analyzed with the paired-samples t test; (c) Dice similarity coefficients (DSC) between predicted and clinical isodose volumes were calculated.

RESULTS

The U-ResNet-D model predicted 3D dose distribution accurately. For twenty tested patients, the prediction bias ranged from -2.0% to 2.3% and prediction error varied from 1.5% to 4.5% (relative to prescription) for 3D dose differences. The mean absolute dose differences for PTVs and OARs are within 2.0% and 4.2%, and nearly all the DIs for PTVs and OARs had no significant differences. The averaged DSC ranged from 0.95 to 1 for different isodose volumes.

CONCLUSIONS

The study developed a new deep learning method for 3D voxel-by-voxel dose prediction, and shown to be able to produce accurately dose predictions for nasopharyngeal patients treated by HT. The predicted 3D dose map can be useful for improving radiotherapy planning design, ensuring plan quality and consistency, making clinical technique comparison, and guiding automatic treatment planning.