Testing the Ability of Convolutional Neural Networks to Learn Radiomic Features

Prognostic Value of a Combined Nomogram Model Integrating 3-Dimensional Deep Learning and Radiomics for Head and Neck Cancer

OBJECTIVE

The preoperative prediction of the overall survival (OS) status of patients with head and neck cancer (HNC) is significant value for their individualized treatment and prognosis. This study aims to evaluate the impact of adding 3D deep learning features to radiomics models for predicting 5-year OS status.

METHODS

Two hundred twenty cases from The Cancer Imaging Archive public dataset were included in this study; 2212 radiomics features and 304 deep features were extracted from each case. The features were selected by univariate analysis and the least absolute shrinkage and selection operator, and then grouped into a radiomics model containing Positron Emission Tomography /Computed Tomography (PET/CT) radiomics features score, a deep model containing deep features score, and a combined model containing PET/CT radiomics features score +3D deep features score. TumorStage model was also constructed using initial patient tumor node metastasis stage to compare the performance of the combined model. A nomogram was constructed to analyze the influence of deep features on the performance of the model. The 10-fold cross-validation of the average area under the receiver operating characteristic curve and calibration curve were used to evaluate performance, and Shapley Additive exPlanations (SHAP) was developed for interpretation.

RESULTS

The TumorStage model, radiomics model, deep model, and the combined model achieved areas under the receiver operating characteristic curve of 0.604, 0.851, 0.840, and 0.895 on the train set and 0.571, 0.849, 0.832, and 0.900 on the test set. The combined model showed better performance of predicting the 5-year OS status of HNC patients than the radiomics model and deep model. The combined model was shown to provide a favorable fit in calibration curves and be clinically useful in decision curve analysis. SHAP summary plot and SHAP The SHAP summary plot and SHAP force plot visually interpreted the influence of deep features and radiomics features on the model results.

CONCLUSIONS

In predicting 5-year OS status in patients with HNC, 3D deep features could provide richer features for combined model, which showed outperformance compared with the radiomics model and deep model.

Multitask Learning with Convolutional Neural Networks and Vision Transformers Can Improve Outcome Prediction for Head and Neck Cancer Patients

Neural-network-based outcome predictions may enable further treatment personalization of patients with head and neck cancer. The development of neural networks can prove challenging when a limited number of cases is available. Therefore, we investigated whether multitask learning strategies, implemented through the simultaneous optimization of two distinct outcome objectives (multi-outcome) and combined with a tumor segmentation task, can lead to improved performance of convolutional neural networks (CNNs) and vision transformers (ViTs). Model training was conducted on two distinct multicenter datasets for the endpoints loco-regional control (LRC) and progression-free survival (PFS), respectively. The first dataset consisted of pre-treatment computed tomography (CT) imaging for 290 patients and the second dataset contained combined positron emission tomography (PET)/CT data of 224 patients. Discriminative performance was assessed by the concordance index (C-index). Risk stratification was evaluated using log-rank tests. Across both datasets, CNN and ViT model ensembles achieved similar results. Multitask approaches showed favorable performance in most investigations. Multi-outcome CNN models trained with segmentation loss were identified as the optimal strategy across cohorts. On the PET/CT dataset, an ensemble of multi-outcome CNNs trained with segmentation loss achieved the best discrimination (C-index: 0.29, 95% confidence interval (CI): 0.22-0.36) and successfully stratified patients into groups with low and high risk of disease progression (p=0.003). On the CT dataset, ensembles of multi-outcome CNNs and of single-outcome ViTs trained with segmentation loss performed best (C-index: 0.26 and 0.26, CI: 0.18-0.34 and 0.18-0.35, respectively), both with significant risk stratification for LRC in independent validation (p=0.002 and p=0.011). Further validation of the developed multitask-learning models is planned based on a prospective validation study, which has recently completed recruitment.

Deep Radiomics Features of Median Nerves for Automated Diagnosis of Carpal Tunnel Syndrome With Ultrasound Images: A Multi-Center Study

OBJECTIVES

Ultrasound is widely used in diagnosing carpal tunnel syndrome (CTS). However, the limitations of ultrasound in CTS detection are the lack of objective measures in the detection of nerve abnormality and the operator-dependent nature of ultrasound imaging. Therefore, in this study, we developed and proposed externally validated artificial intelligence (AI) models based on deep-radiomics features.

METHODS

We have used 416 median nerves from 2 countries (Iran and Colombia) for the development (112 entrapped and 112 normal nerves from Iran) and validation (26 entrapped and 26 normal nerves from Iran, and 70 entrapped and 70 normal nerves from Columbia) of our models. Ultrasound images were fed to the SqueezNet architecture to extract deep-radiomics features. Then a ReliefF method was used to select the clinically significant features. The selected deep-radiomics features were fed to 9 common machine-learning algorithms to choose the best-performing classifier. The 2 best-performing AI models were then externally validated.

RESULTS

Our developed model achieved an area under the receiver operating characteristic (ROC) curve (AUC) of 0.910 (88.46% sensitivity, 88.46% specificity) and 0.908 (84.62% sensitivity, 88.46% specificity) with support vector machine and stochastic gradient descent (SGD), respectively using the internal validation dataset. Furthermore, both models consistently performed well in the external validation dataset, and achieved an AUC of 0.890 (85.71% sensitivity, 82.86% specificity) and 0.890 (84.29% sensitivity and 82.86% specificity), with SVM and SGD models, respectively.

CONCLUSION

Our proposed AI models fed with deep-radiomics features performed consistently with internal and external datasets. This justifies that our proposed system can be employed for clinical use in hospitals and polyclinics.

Predictive performance of radiomic models based on features extracted from pretrained deep networks

OBJECTIVES

In radiomics, generic texture and morphological features are often used for modeling. Recently, features extracted from pretrained deep networks have been used as an alternative. However, extracting deep features involves several decisions, and it is unclear how these affect the resulting models. Therefore, in this study, we considered the influence of such choices on the predictive performance.

METHODS

On ten publicly available radiomic datasets, models were trained using feature sets that differed in terms of the utilized network architecture, the layer of feature extraction, the used set of slices, the use of segmentation, and the aggregation method. The influence of these choices on the predictive performance was measured using a linear mixed model. In addition, models with generic features were trained and compared in terms of predictive performance and correlation.

RESULTS

No single choice consistently led to the best-performing models. In the mixed model, the choice of architecture (AUC + 0.016; p < 0.001), the level of feature extraction (AUC + 0.016; p < 0.001), and using all slices (AUC + 0.023; p < 0.001) were highly significant; using the segmentation had a lower influence (AUC + 0.011; p = 0.023), while the aggregation method was insignificant (p = 0.774). Models based on deep features were not significantly better than those based on generic features (p > 0.05 on all datasets). Deep feature sets correlated moderately with each other (r = 0.4), in contrast to generic feature sets (r = 0.89).

CONCLUSIONS

Different choices have a significant effect on the predictive performance of the resulting models; however, for the highest performance, these choices should be optimized during cross-validation.

The transformational potential of molecular radiomics

Conventional radiomics in nuclear medicine involve hand-crafted and computer-assisted regions of interest. Recent developments in artificial intelligence (AI) have seen the emergence of AI-augmented segmentation and extraction of lower order traditional radiomic features. Deep learning (DL) affords the opportunity to extract abstract radiomic features directly from input tensors (images) without the need for segmentation. These fourth-order, high dimensional radiomics produce deep radiomics and are well suited to the data density associated with the molecular nature of hybrid imaging. Molecular radiomics and deep molecular radiomics provide insights beyond images and quantitation typical of semantic reporting. While the application of molecular radiomics using hand-crafted and computer-generated features is integrated into decision-making in nuclear medicine, the acceptance of deep molecular radiomics is less universal. This manuscript aims to provide an understanding of the language and principles associated with radiomics and deep radiomics in nuclear medicine.

Radiomics-guided radiation therapy: opportunities and challenges

Radiomics is an advanced image-processing framework, which extracts image features and considers them as biomarkers towards personalized medicine. Applications include disease detection, diagnosis, prognosis, and therapy response assessment/prediction. As radiation therapy aims for further individualized treatments, radiomics could play a critical role in various steps before, during and after treatment. Elucidation of the concept of radiomics-guided radiation therapy (RGRT) is the aim of this review, attempting to highlight opportunities and challenges underlying the use of radiomics to guide clinicians and physicists towards more effective radiation treatments. This work identifies the value of RGRT in various steps of radiotherapy from patient selection to follow-up, and subsequently provides recommendations to improve future radiotherapy using quantitative imaging features.