Robustness and performance of radiomic features in diagnosing cystic renal masses

Multiparametric MRI-Based Machine Learning Models for the Characterization of Cystic Renal Masses Compared to the Bosniak Classification, Version 2019: A Multicenter Study

RATIONALE AND OBJECTIVE

Accurate differentiation between benign and malignant cystic renal masses (CRMs) is challenging in clinical practice. This study aimed to develop MRI-based machine learning models for differentiating between benign and malignant CRMs and compare the best-performing model with the Bosniak classification, version 2019 (BC, version 2019).

METHODS

Between 2009 and 2021, consecutive surgery-proven CRM patients with renal MRI were enrolled in this multicenter study. Models were constructed to differentiate between benign and malignant CRMs using logistic regression (LR), random forest (RF), and support vector machine (SVM) algorithms, respectively. Meanwhile, two radiologists classified CRMs into I-IV categories according to the BC, version 2019 in consensus in the test set. A subgroup analysis was conducted to investigate the performance of the best-performing model in complicated CRMs (II-IV lesions in the test set). The performances of models and BC, version 2019 were evaluated using the area under the receiver operating characteristic curve (AUC). Performance was statistically compared between the best-performing model and the BC, version 2019.

RESULTS

278 and 48 patients were assigned to the training and test sets, respectively. In the test set, the AUC and accuracy of the LR model, the RF model, the SVM model, and the BC, version 2019 were 0.884 and 75.0%, 0.907 and 83.3%, 0.814 and 72.9%, and 0.893 and 81.2%, respectively. Neither the AUC nor the accuracy of the RF model that performed best were significantly different from the BC, version 2019 (P = 0.780, P = 0.065). The RF model achieved an AUC and accuracy of 0.880 and 81.0% in complicated CRMs.

CONCLUSIONS

The MRI-based RF model can accurately differentiate between benign and malignant CRMs with comparable performance to the BC, version 2019, and has good performance in complicated CRMs, which may facilitate treatment decision-making and is less affected by interobserver disagreements.

Enhanced and unenhanced: Radiomics models for discriminating between benign and malignant cystic renal masses on CT images: A multi-center study

BACKGROUND

Machine learning algorithms used to classify cystic renal masses (CRMs) nave not been applied to unenhanced CT images, and their diagnostic accuracy had not been compared against radiologists.

METHOD

This retrospective study aimed to develop radiomics models that discriminate between benign and malignant CRMs in a triple phase computed tomography (CT) protocol and compare the diagnostic accuracy of the radiomics approach with experienced radiologists. Predictive models were established using a training set and validation set of unenhanced and enhanced (arterial phase [AP] and venous phase [VP]) CT images of benign and malignant CRMs. The diagnostic capabilities of the models and experienced radiologists were compared using Receiver Operating Characteristic (ROC) curves.

RESULTS

On unenhanced, AP and VP CT images in the validation set, the AUC, specificity, sensitivity and accuracy for discriminating between benign and malignant CRMs were 90.0 (95%CI: 81-98%), 90.0%, 90.5% and 90.2%; 93.0% (95%CI: 86-99%), 86.7%, 95.2% and 88.3%; and 95.0% (95%CI: 90%-100%), 93.3%, 90.5% and 92.1%, respectively, for the radiomics models. Diagnostic accuracy of the radiomics models differed significantly on unenhanced images in the training set vs. each radiologist (p = 0.001 and 0.003) but not in the validation set (p = 0.230 and 0.590); differed significantly on AP images in the validation set vs. each radiologist (p = 0.007 and 0.007) but not in the training set (p = 0.663 and 0.663); and there were no differences on VP images in the training or validation sets vs. each radiologist (training set: p = 0.453 and 0.051, validation set: p = 0.236 and 0.786).

CONCLUSIONS

Radiomics models may have clinical utility for discriminating between benign and malignant CRMs on unenhanced and enhanced CT images. The performance of the radiomics model on unenhanced CT images was similar to experienced radiologists, implying it has potential as a screening and diagnostic tool for CRMs.

Differentiating Benign From Malignant Cystic Renal Masses: A Feasibility Study of Computed Tomography Texture-Based Machine Learning Algorithms

OBJECTIVE

The Bosniak classification attempts to predict the likelihood of renal cell carcinoma (RCC) among cystic renal masses but is subject to interobserver variability and often requires multiphase imaging. Artificial intelligence may provide a more objective assessment. We applied computed tomography texture-based machine learning algorithms to differentiate benign from malignant cystic renal masses.

METHODS

This is an institutional review board-approved, Health Insurance Portability and Accountability Act-compliant retrospective study of 147 patients (mean age, 62.4 years; range, 28-89 years; 94 men) with 144 cystic renal masses (93 benign, 51 RCC); 69 were pathology proven (51 RCC, 18 benign), and 75 were considered benign based on more than 4 years of stability at follow-up imaging. Using a single image from a contrast-enhanced abdominal computed tomography scan, mean, SD, mean value of positive pixels, entropy, skewness, and kurtosis radiomics features were extracted. Random forest, multivariate logistic regression, and support vector machine models were used to classify each mass as benign or malignant with 10-fold cross validation. Receiver operating characteristic curves assessed algorithm performance in the aggregated test data.

RESULTS

For the detection of malignancy, sensitivity, specificity, positive predictive value, negative predictive value, and area under the curve were 0.61, 0.87, 0.72, 0.80, and 0.79 for the random forest model; 0.59, 0.87, 0.71, 0.79, and 0.80 for the logistic regression model; and 0.55, 0.86, 0.68, 0.78, and 0.76 for the support vector machine model.

CONCLUSION

Computed tomography texture-based machine learning algorithms show promise in differentiating benign from malignant cystic renal masses. Once validated, these may serve as an adjunct to radiologists' assessments.