Facilitating standardized COVID-19 suspicion prediction based on computed tomography radiomics in a multi-demographic setting

Prediction study of surrounding tissue invasion in clear cell renal cell carcinoma based on multi-phase enhanced CT radiomics

OBJECTIVE

To examine the effectiveness of a nomogram model that combines clinical-image features and CT radiomics in predicting surrounding tissue invasion (STI) in clear cell renal cell carcinoma (ccRCC) patients before surgery.

METHODS

Postoperative pathological data of 248 ccRCC patients from two centers were retrospectively collected. Univariate and multivariate regression analyses were used to identify clinical and image features of ccRCC patients to construct a clinical model. Radiomics features were extracted from three CT scans, including tumoral, intratumor, and peritumoral regions. A nomogram was developed by integrating clinical model with optimal radiomics signature. The Shapley Additive Explanations (SHAP) method was used for interpretation.

RESULTS

This study included 65 ccRCC patients with STI and 183 patients without STI. The AUC of the clinical model was 0.766, 0.765, and 0.698 in the training cohort, internal validation cohort, and external validation cohort, respectively. The AUCs were higher in the radiomics signature based on ROI4 in NP than other radiomics (training cohort: 0.837 vs. 0.775-0.847; internal validation cohort: 0.831 vs. 0.695-0.811; external validation cohort: 0.762 vs. 0.623-0.731). Integrating the optimal radiomics signature with the clinical model to construct a combined model resulted in an AUC of 0.890, 0.886, and 0.826 in the training cohort, internal validation cohort, external validation cohort, respectively. SHAP values analysis revealed the top three radiomics features to be Small Dependence Low Gray Level Emphasis, Maximum 3D Diameter, and Maximum Probability.

CONCLUSION

A nomogram based on preoperative CT and clinical image features is a reliable tool for predicting STI in ccRCC patients. The use of SHAP values can help popularize this tool.

Tracheal computed tomography radiomics model for prediction of the Omicron variant of severe acute respiratory syndrome coronavirus 2

OBJECTIVES

The Omicron variant of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is highly contagious, fast-spreading, and insidious. Most patients present with normal findings on lung computed tomography (CT). The current study aimed to develop and validate a tracheal CT radiomics model to predict Omicron variant infection.

MATERIALS AND METHODS

In this retrospective study, a radiomics model was developed based on a training set consisting of 157 patients with an Omicron variant infection and 239 healthy controls between 1 January and 30 April 2022. A set of morphological expansions, with dilations of 1, 3, 5, 7, and 9 voxels, was applied to the trachea, and radiomic features were extracted from different dilation voxels of the trachea. Logistic regression (LR), support vector machines (SVM), and random forests (RF) were developed and evaluated; the models were validated on 67 patients with the Omicron variant and on 103 healthy controls between 1 May and 30 July 2022.

RESULTS

Logistic regression with 12 radiomic features extracted from the tracheal wall with dilation of 5 voxels achieved the highest classification performance compared with the other models. The LR model achieved an area under the curve of 0.993 (95% confidence interval [CI]: 0.987-0.998) in the training set and 0.989 (95% CI: 0.979-0.999) in the validation set. Sensitivity, specificity, and accuracy of the model for the training set were 0.994, 0.946, and 0.965, respectively, whereas those for the validation set were 0.970, 0.952, and 0.959, respectively.

CONCLUSION

The tracheal CT radiomics model reliably identified the Omicron variant of SARS-CoV‑2, and may help in clinical decision-making in future, especially in cases of normal lung CT findings.

Reproducibility of radiomics quality score: an intra- and inter-rater reliability study

OBJECTIVES

To investigate the intra- and inter-rater reliability of the total radiomics quality score (RQS) and the reproducibility of individual RQS items' score in a large multireader study.

METHODS

Nine raters with different backgrounds were randomly assigned to three groups based on their proficiency with RQS utilization: Groups 1 and 2 represented the inter-rater reliability groups with or without prior training in RQS, respectively; group 3 represented the intra-rater reliability group. Thirty-three original research papers on radiomics were evaluated by raters of groups 1 and 2. Of the 33 papers, 17 were evaluated twice with an interval of 1 month by raters of group 3. Intraclass coefficient (ICC) for continuous variables, and Fleiss' and Cohen's kappa (k) statistics for categorical variables were used.

RESULTS

The inter-rater reliability was poor to moderate for total RQS (ICC 0.30-055, p < 0.001) and very low to good for item's reproducibility (k - 0.12 to 0.75) within groups 1 and 2 for both inexperienced and experienced raters. The intra-rater reliability for total RQS was moderate for the less experienced rater (ICC 0.522, p = 0.009), whereas experienced raters showed excellent intra-rater reliability (ICC 0.91-0.99, p < 0.001) between the first and second read. Intra-rater reliability on RQS items' score reproducibility was higher and most of the items had moderate to good intra-rater reliability (k - 0.40 to 1).

CONCLUSIONS

Reproducibility of the total RQS and the score of individual RQS items is low. There is a need for a robust and reproducible assessment method to assess the quality of radiomics research.

CLINICAL RELEVANCE STATEMENT

There is a need for reproducible scoring systems to improve quality of radiomics research and consecutively close the translational gap between research and clinical implementation.

KEY POINTS

• Radiomics quality score has been widely used for the evaluation of radiomics studies. • Although the intra-rater reliability was moderate to excellent, intra- and inter-rater reliability of total score and point-by-point scores were low with radiomics quality score. • A robust, easy-to-use scoring system is needed for the evaluation of radiomics research.

Data drift in medical machine learning: implications and potential remedies

Data drift refers to differences between the data used in training a machine learning (ML) model and that applied to the model in real-world operation. Medical ML systems can be exposed to various forms of data drift, including differences between the data sampled for training and used in clinical operation, differences between medical practices or context of use between training and clinical use, and time-related changes in patient populations, disease patterns, and data acquisition, to name a few. In this article, we first review the terminology used in ML literature related to data drift, define distinct types of drift, and discuss in detail potential causes within the context of medical applications with an emphasis on medical imaging. We then review the recent literature regarding the effects of data drift on medical ML systems, which overwhelmingly show that data drift can be a major cause for performance deterioration. We then discuss methods for monitoring data drift and mitigating its effects with an emphasis on pre- and post-deployment techniques. Some of the potential methods for drift detection and issues around model retraining when drift is detected are included. Based on our review, we find that data drift is a major concern in medical ML deployment and that more research is needed so that ML models can identify drift early, incorporate effective mitigation strategies and resist performance decay.