A deep learning-based radiomics approach to predict head and neck tumor regression for adaptive radiotherapy

Robust vs. Non-robust radiomic features: the quest for optimal machine learning models using phantom and clinical studies

PURPOSE

This study aimed to select robust features against lung motion in a phantom study and use them as input to feature selection algorithms and machine learning classifiers in a clinical study to predict the lymphovascular invasion (LVI) of non-small cell lung cancer (NSCLC). The results of robust features were also compared with conventional techniques without considering the robustness of radiomic features.

METHODS

An in-house developed lung phantom was developed with two 22mm lesion sizes based on a clinical study. A specific motor was built to simulate motion in two orthogonal directions. Lesions of both clinical and phantom studies were segmented using a Fuzzy C-means-based segmentation algorithm. After inducing motion and extracting 105 radiomic features in 4 feature sets, including shape, first-, second-, and higher-order statistics features from each region of interest (ROI) of the phantom image, statistical analyses were performed to select robust features against motion. Subsequently, these robust features and a total of 105 radiomic features were extracted from 126 clinical data. Various feature selection (FS) and multiple machine learning (ML) classifiers were implemented to predict the LVI of NSCLC, followed by comparing the results of predicting LVI using robust features with common conventional techniques not considering the robustness of radiomic features.

RESULTS

Our results demonstrated that selecting robust features as input to FS algorithms and ML classifiers surges the sensitivity, which has a gentle negative effect on the accuracy and the area under the curve (AUC) of predictions compared with commonly used methods in 12 of 15 outcomes. The top performance of the LVI prediction was achieved by the NB classifier and RFE FS without considering the robustness of radiomic features with 95% area under the curve of AUC, 67% accuracy, and 100% sensitivity. Moreover, the top performance of the LVI prediction using robust features belonged to the NB classifier and Boruta feature selection with 92% AUC, 86% accuracy, and 100% sensitivity.

CONCLUSION

Robustness over various influential factors is critical and should be considered in a radiomic study. Selecting robust features is a solution to overcome the low reproducibility of radiomic features. Although setting robust features against motion in a phantom study has a minor negative impact on the accuracy and AUC of LVI prediction, it boosts the sensitivity of prediction to a large extent.

Radiomics and dosiomics approaches to estimate lung function after stereotactic body radiation therapy in patients with lung tumors

Lung function assessment is essential for determining the optimal treatment strategy for radiation therapy in patients with lung tumors. This study aimed to develop radiomics and dosiomics approaches to estimate pulmonary function test (PFT) results in post-stereotactic body radiation therapy (SBRT). Sixty-four patients with lung tumors who underwent SBRT were included. Models were created to estimate the PFT results at 0-6 months (Cohort 1) and 6-24 months (Cohort 2) after SBRT. Radiomics and dosiomics features were extracted from the computed tomography (CT) images and dose distributions, respectively. To estimate the PFT results, Models A (dose-volume histogram [DVH] + radiomics features) and B (DVH + radiomics + dosiomics features) were created. In the PFT results, the forced expiratory volume in one second (FEV1) and forced vital capacity (FVC) were estimated using each model, and the ratio of FEV1 to FVC (FEV1/FVC) was calculated. The Pearson's correlation coefficient (Pearson r) and area under the curve (AUC) for FEV1/FVC (< 70%) were calculated. The models were evaluated by comparing them with the conventional calculation formulae (Conventional). The Pearson r (FEV1/FVC) values were 0.30, 0.64, and 0.69 for Conventional and Models A and B (Cohort 2), respectively, and the AUC (FEV1/FVC < 70%) values were 0.63, 0.80, and 0.78, respectively. This study demonstrates the possibility of estimating lung function after SBRT using radiomics and dosiomics features based on planning CT images and dose distributions.

Development and validation of CNN-MLP models for predicting anti-VEGF therapy outcomes in diabetic macular edema

Diabetic macular edema (DME) is a common complication of diabetes that can lead to vision loss, and anti-vascular endothelial growth factor (anti-VEGF) therapy is the standard of care for DME, but the treatment outcomes vary widely among patients. This study collected optical coherence tomography (OCT) images and clinical data from DME patients who received anti-VEGF treatment to develop and validate deep learning (DL) models for predicting the anti-VEGF outcomes in DME patients based on convolutional neural network (CNN) and multilayer perceptron (MLP) combined architecture by using multimodal data. An Xception-MLP architecture was utilized to predict best-corrected visual acuity (BCVA), central subfield thickness (CST), cube volume (CV), and cube average thickness (CAT). Mean absolute error (MAE), mean squared error (MSE) and mean squared logarithmic error (MSLE) were employed to evaluate the model performance. In this study, both the training set and the validation set exhibited a consistent decreasing trend in MAE, MSE, and MSLE. No statistical difference was found between the actual and predicted values in all clinical indicators. This study demonstrated that the improved CNN-MLP regression models using multimodal data can accurately predict outcomes in BCVA, CST, CV, and CAT after anti-VEGF therapy in DME patients, which is valuable for ophthalmic clinical decisions and reduces the economic burden on patients.

Development of deep learning-based novel auto-segmentation for the prostatic urethra on planning CT images for prostate cancer radiotherapy

Urinary toxicities are one of the serious complications of radiotherapy for prostate cancer, and dose-volume histogram of prostatic urethra has been associated with such toxicities in previous reports. Previous research has focused on estimating the prostatic urethra, which is difficult to delineate in CT images; however, these studies, which are limited in number, mainly focused on cases undergoing brachytherapy uses low-dose-rate sources and do not involve external beam radiation therapy (EBRT). In this study, we aimed to develop a deep learning-based method of determining the position of the prostatic urethra in patients eligible for EBRT. We used contour data from 430 patients with localized prostate cancer. In all cases, a urethral catheter was placed when planning CT to identify the prostatic urethra. We used 2D and 3D U-Net segmentation models. The input images included the bladder and prostate, while the output images focused on the prostatic urethra. The 2D model determined the prostate's position based on results from both coronal and sagittal directions. Evaluation metrics included the average distance between centerlines. The average centerline distances for the 2D and 3D models were 2.07 ± 0.87 mm and 2.05 ± 0.92 mm, respectively. Increasing the number of cases while maintaining equivalent accuracy as we did in this study suggests the potential for high generalization performance and the feasibility of using deep learning technology for estimating the position of the prostatic urethra.

Evaluation of the performance of both machine learning models using PET and CT radiomics for predicting recurrence following lung stereotactic body radiation therapy: A single-institutional study

PURPOSE

Predicting recurrence following stereotactic body radiotherapy (SBRT) for non-small cell lung cancer provides important information for the feasibility of the individualized radiotherapy and allows to select the appropriate treatment strategy based on the risk of recurrence. In this study, we evaluated the performance of both machine learning models using positron emission tomography (PET) and computed tomography (CT) radiomic features for predicting recurrence after SBRT.

METHODS

Planning CT and PET images of 82 non-small cell lung cancer patients who performed SBRT at our hospital were used. First, tumors were delineated on each CT and PET of each patient, and 111 unique radiomic features were extracted, respectively. Next, the 10 features were selected using three different feature selection algorithms, respectively. Recurrence prediction models based on the selected features and four different machine learning algorithms were developed, respectively. Finally, we compared the predictive performance of each model for each recurrence pattern using the mean area under the curve (AUC) calculated following the 0.632+ bootstrap method.

RESULTS

The highest performance for local recurrence, regional lymph node metastasis, and distant metastasis were observed in models using Support vector machine with PET features (mean AUC = 0.646), Naive Bayes with PET features (mean AUC = 0.611), and Support vector machine with CT features (mean AUC = 0.645), respectively.

CONCLUSIONS

We comprehensively evaluated the performance of prediction model developed for recurrence following SBRT. The model in this study would provide information to predict the recurrence pattern and assist in making treatment strategies.

Prediction of Obliteration After the Gamma Knife Radiosurgery of Arteriovenous Malformations Using Hand-Crafted Radiomics and Deep-Learning Methods

INTRODUCTION

Brain arteriovenous malformations (bAVMs) are vascular abnormalities that can be treated with embolization or radiotherapy to prevent the risk of future rupture. In this study, we use hand-crafted radiomics and deep learning techniques to predict favorable vs. unfavorable outcomes following Gamma Knife radiosurgery (GKRS) of bAVMs and compare their prediction performances.

METHODS

One hundred twenty-six patients seen at one academic medical center for GKRS obliteration of bAVMs over 15 years were retrospectively reviewed. Forty-two patients met the inclusion criteria. Favorable outcomes were defined as complete nidus obliteration demonstrated on cerebral angiogram and asymptomatic recovery. Unfavorable outcomes were defined as incomplete obliteration or complications relating to the AVM that developed after GKRS. Outcome predictions were made using a random forest model with hand-crafted radiomic features and a fine-tuned ResNet-34 convolutional neural network (CNN) model. The performance was evaluated by using a ten-fold cross-validation technique.

RESULTS

The average accuracy and area-under-curve (AUC) values of the Random Forest Classifier (RFC) with radiomics features were 68.5 ±9.80% and 0.705 ±0.086, whereas those of the ResNet-34 model were 60.0 ±11.9% and 0.694 ±0.124. Four radiomics features used with RFC discriminated unfavorable response cases from favorable response cases with statistical significance. When cropped images were used with ResNet-34, the accuracy and AUC decreased to 59.3 ± 14.2% and 55.4 ±10.4%, respectively.

CONCLUSIONS

A hand-crafted radiomics model and a pre-trained CNN model can be fine-tuned on pre-treatment MRI scans to predict clinical outcomes of AVM patients undergoing GKRS with equivalent prediction performance. The outcome predictions are promising but require further external validation on more patients.

A CT-based Deep Learning Radiomics Nomogram for the Prediction of EGFR Mutation Status in Head and Neck Squamous Cell Carcinoma

RATIONALE AND OBJECTIVES

Accurately assessing epidermal growth factor receptor (EGFR) mutation status in head and neck squamous cell carcinoma (HNSCC) patients is crucial for prognosis and treatment selection. This study aimed to construct and validate a contrast-enhanced computed tomography (CECT)-based deep learning radiomics nomogram (DLRN) to predict EGFR mutation status of HNSCC.

MATERIALS AND METHODS

A total of 300 HNSCC patients who underwent CECT scans were enrolled in this study. Participants from two hospitals were separated into a training set (n = 200, 56 EGFR-negative and 144 EGFR-positive) from one hospital and an external test set from the other hospital (n = 100, 37 EGFR-negative and 63 EGFR-positive). The least absolute shrinkage and selection operator method was used to select the key features from CECT-based manually extracted radiomics (MER) features and features automatically extracted using a deep learning model (DL, extracted using a GoogLeNet model). The selected independent clinical factors, MER features, and DL features were then combined to construct a DLRN. The DLRN's performance was evaluated using receiver operating characteristics curves.

RESULTS

Five MER and six DL features were finally chosen. The DLRN, which includes "gender" and "necrotic areas," along with the selected features, predicted EGFR mutation status of HNSCC (EGFR-negative vs. positive) well in both the training (area under the curve [AUC], 0.901) and test (AUC, 0.875) sets.

CONCLUSION

A DLRN using CECT was built to predict EGFR mutation in HNSCC. The model showed high predictive ability and may aid in treatment selection and patient prognosis.

A hybrid few-shot multiple-instance learning model predicting the aggressiveness of lymphoma in PET/CT images

BACKGROUND AND OBJECTIVE

Patients with aggressive non-Hodgkin lymphoma (NHL) undergo distinct therapy strategies compared with indolent NHL patients. However, it is challenging to estimate NHL aggressiveness based on visual inspection of positron emission tomography (PET) or computed tomography (CT) images. Since diffuse large B-cell lymphoma (DLBCL) and Follicular lymphoma (FL) are the most typical and dominant aggressive and indolent NHL, respectively, this study aims to develop an artificial-intelligence-enabled model to distinguish DLBCL from FL in PET/CT images as the first step to tackle this challenge.

METHODS

We propose a hybrid few-shot multiple-instance learning model to predict the aggressiveness of the NHL. First, rotation-based self-supervision learning (SSL) has been employed to train the encoder on a large-scale, publicly available CT image dataset. Second, hybrid instance-level features are obtained for each NHL lesion by combining deep features with the radiomics features from both PET and CT modalities. Third, instance-level features are transformed into bag-level (or patient-level) representations. Finally, bag-level representations are fed into a distance-based classifier through few-shot learning to predict NHL aggressiveness.

RESULTS

Our model achieves an accuracy of 0.751 ± 0.008, a sensitivity of 0.787 ± 0.012, a specificity of 0.715 ± 0.013, an F1-score of 0.753 ± 0.009, and an area under the curve (AUC) of 0.795 ± 0.009 at the bag level. It outperforms the typical counterparts that use the radiomic features, random forest for feature selection, and support vector machines (SVMs) as classifiers. The three counterparts yield accuracies of 0.714 ± 0.023, 0.705 ± 0.008, and 0.698 ± 0.008, respectively. Moreover, settings of the SSL training dataset (Deep lesion) and task (rotation), hybrid CT and radiomic PET features, the pool-layer strategy of maximum, and distance-based classifier generate the best model.

CONCLUSIONS

A hybrid few-shot multiple-instance learning model can predict lymphoma aggressiveness in PET/CT images and could be a potential tool for determining therapy strategies. Hybrid features and the combination of SSL, few-shot learning, and weakly supervised learning are the two powerful pillars of the model, and these can be expanded to other medical applications with limited samples and incomplete annotations.

Development of a prediction model for head and neck volume reduction by clinical factors, dose-volume histogram parameters and radiomics in head and neck cancer†

In external radiotherapy of head and neck (HN) cancers, the reduction of irradiation accuracy due to HN volume reduction often causes a problem. Adaptive radiotherapy (ART) can effectively solve this problem; however, its application to all cases is impractical because of cost and time. Therefore, finding priority cases is essential. This study aimed to predict patients with HN cancers are more likely to need ART based on a quantitative measure of large HN volume reduction and evaluate model accuracy. The study included 172 cases of patients with HN cancer who received external irradiation. The HN volume was calculated using cone-beam computed tomography (CT) for irradiation-guided radiotherapy for all treatment fractions and classified into two groups: cases with a large reduction in the HN volume and cases without a large reduction. Radiomic features were extracted from the primary gross tumor volume (GTV) and nodal GTV of the planning CT. To develop the prediction model, four feature selection methods and two machine-learning algorithms were tested. Predictive performance was evaluated by the area under the curve (AUC), accuracy, sensitivity and specificity. Predictive performance was the highest for the random forest, with an AUC of 0.662. Furthermore, its accuracy, sensitivity and specificity were 0.692, 0.700 and 0.813, respectively. Selected features included radiomic features of the primary GTV, human papillomavirus in oropharyngeal cancer and the implementation of chemotherapy; thus, these features might be related to HN volume change. Our model suggested the potential to predict ART requirements based on HN volume reduction .

Image Quality Improvement in Deep Learning Image Reconstruction of Head Computed Tomography Examination

In this study, we assess image quality in computed tomography scans reconstructed via DLIR (Deep Learning Image Reconstruction) and compare it with iterative reconstruction ASIR-V (Adaptive Statistical Iterative Reconstruction) in CT (computed tomography) scans of the head. The CT scans of 109 patients were subjected to both objective and subjective evaluation of image quality. The objective evaluation was based on the SNR (signal-to-noise ratio) and CNR (contrast-to-noise ratio) of the brain's gray and white matter. The regions of interest for our study were set in the BGA (basal ganglia area) and PCF (posterior cranial fossa). Simultaneously, a subjective assessment of image quality, based on brain structure visibility, was conducted by experienced radiologists. In the assessed scans, we obtained up to a 54% increase in SNR for gray matter and a 60% increase for white matter using DLIR in comparison to ASIR-V. Moreover, we achieved a CNR increment of 58% in the BGA structures and 50% in the PCF. In the subjective assessment of the obtained images, DLIR had a mean rating score of 2.8, compared to the mean score of 2.6 for ASIR-V images. In conclusion, DLIR shows improved image quality compared to the standard iterative reconstruction of CT images of the head.

A CT-Based Deep Learning Radiomics Nomogram to Predict Histological Grades of Head and Neck Squamous Cell Carcinoma

RATIONALE AND OBJECTIVES

Accurate pretreatment assessment of histological differentiation grade of head and neck squamous cell carcinoma (HNSCC) is crucial for prognosis evaluation. This study aimed to construct and validate a contrast-enhanced computed tomography (CECT)-based deep learning radiomics nomogram (DLRN) to predict histological differentiation grades of HNSCC.

MATERIALS AND METHODS

A total of 204 patients with HNSCC who underwent CECT scans were enrolled in this study. The participants recruited from two hospitals were split into a training set (n=124, 74 well/moderately differentiated and 50 poorly differentiated) of patients from one hospital and an external test set of patients from the other hospital (n=80, 49 well/moderately differentiated and 31 poorly differentiated). CECT-based manually-extracted radiomics (MER) features and deep learning (DL) features were extracted and selected. The selected MER features and DL features were then combined to construct a DLRN via multivariate logistic regression. The predictive performance of the DLRN was assessed using ROCs and decision curve analysis (DCA).

RESULTS

Three MER features and seven DL features were finally selected. The DLRN incorporating the selected MER and DL features showed good predictive value for the histological differentiation grades of HNSCC (well/moderately differentiated vs. poorly differentiated) in both the training (AUC, 0.878) and test (AUC, 0.822) sets. DCA demonstrated that the DLRN was clinically useful for predicting histological differentiation grades of HNSCC.

CONCLUSION

A CECT-based DLRN was constructed to predict histological differentiation grades of HNSCC. The DLRN showed good predictive efficacy and might be useful for prognostic evaluation of patients with HNSCC.

The Promise and Future of Radiomics for Personalized Radiotherapy Dosing and Adaptation

Quantitative image analysis, also known as radiomics, aims to analyze large-scale quantitative features extracted from acquired medical images using hand-crafted or machine-engineered feature extraction approaches. Radiomics has great potential for a variety of clinical applications in radiation oncology, an image-rich treatment modality that utilizes computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET) for treatment planning, dose calculation, and image guidance. A promising application of radiomics is in predicting treatment outcomes after radiotherapy such as local control and treatment-related toxicity using features extracted from pretreatment and on-treatment images. Based on these individualized predictions of treatment outcomes, radiotherapy dose can be sculpted to meet the specific needs and preferences of each patient. Radiomics can aid in tumor characterization for personalized targeting, especially for identifying high-risk regions within a tumor that cannot be easily discerned based on size or intensity alone. Radiomics-based treatment response prediction can aid in developing personalized fractionation and dose adjustments. In order to make radiomics models more applicable across different institutions with varying scanners and patient populations, further efforts are needed to harmonize and standardize the acquisition protocols by minimizing uncertainties within the imaging data.

Artificial intelligence-driven radiomics study in cancer: the role of feature engineering and modeling

Modern medicine is reliant on various medical imaging technologies for non-invasively observing patients' anatomy. However, the interpretation of medical images can be highly subjective and dependent on the expertise of clinicians. Moreover, some potentially useful quantitative information in medical images, especially that which is not visible to the naked eye, is often ignored during clinical practice. In contrast, radiomics performs high-throughput feature extraction from medical images, which enables quantitative analysis of medical images and prediction of various clinical endpoints. Studies have reported that radiomics exhibits promising performance in diagnosis and predicting treatment responses and prognosis, demonstrating its potential to be a non-invasive auxiliary tool for personalized medicine. However, radiomics remains in a developmental phase as numerous technical challenges have yet to be solved, especially in feature engineering and statistical modeling. In this review, we introduce the current utility of radiomics by summarizing research on its application in the diagnosis, prognosis, and prediction of treatment responses in patients with cancer. We focus on machine learning approaches, for feature extraction and selection during feature engineering and for imbalanced datasets and multi-modality fusion during statistical modeling. Furthermore, we introduce the stability, reproducibility, and interpretability of features, and the generalizability and interpretability of models. Finally, we offer possible solutions to current challenges in radiomics research.

Artificial intelligence in radiotherapy

Radiotherapy is a discipline closely integrated with computer science. Artificial intelligence (AI) has developed rapidly over the past few years. With the explosive growth of medical big data, AI promises to revolutionize the field of radiotherapy through highly automated workflow, enhanced quality assurance, improved regional balances of expert experiences, and individualized treatment guided by multi-omics. In addition to independent researchers, the increasing number of large databases, biobanks, and open challenges significantly facilitated AI studies on radiation oncology. This article reviews the latest research, clinical applications, and challenges of AI in each part of radiotherapy including image processing, contouring, planning, quality assurance, motion management, and outcome prediction. By summarizing cutting-edge findings and challenges, we aim to inspire researchers to explore more future possibilities and accelerate the arrival of AI radiotherapy.

Early Prediction of Planning Adaptation Requirement Indication Due to Volumetric Alterations in Head and Neck Cancer Radiotherapy: A Machine Learning Approach

Background: During RT cycles, the tumor response pattern could affect tumor coverage and may lead to organs at risk of overdose. As such, early prediction of significant volumetric changes could therefore reduce potential radiation-related adverse effects. Nevertheless, effective machine learning approaches based on the radiomic features of the clinically used CBCT images to determine the tumor volume variations due to RT not having been implemented so far. Methods: CBCT images from 40 HN cancer patients were collected weekly during RT treatment. From the obtained images, the Clinical Target Volume (CTV) and Parotid Glands (PG) regions of interest were utilized to calculate 104 delta-radiomics features. These features were fed on a feature selection and classification procedure for the early prediction of significant volumetric alterations. Results: The proposed framework was able to achieve 0.90 classification performance accuracy while detecting a small subset of discriminative characteristics from the 1st week of RT. The selected features were further analyzed regarding their effects on temporal changes in anatomy and tumor response modeling. Conclusion: The use of machine learning algorithms offers promising perspectives for fast and reliable early prediction of large volumetric deviations as a result of RT treatment, exploiting hidden patterns in the overall anatomical characteristics.