Stability analysis of CT radiomic features with respect to segmentation variation in oropharyngeal cancer

Radiomics-driven spectral profiling of six kidney stone types with monoenergetic CT reconstructions in photon-counting CT

OBJECTIVES

Urolithiasis, a common and painful urological condition, is influenced by factors such as lifestyle, genetics, and medication. Differentiating between different types of kidney stones is crucial for personalized therapy. The purpose of this study is to investigate the use of photon-counting computed tomography (PCCT) in combination with radiomics and machine learning to develop a method for automated and detailed characterization of kidney stones. This approach aims to enhance the accuracy and detail of stone classification beyond what is achievable with conventional computed tomography (CT) and dual-energy CT (DECT).

MATERIALS AND METHODS

In this ex vivo study, 135 kidney stones were first classified using infrared spectroscopy. All stones were then scanned in a PCCT embedded in a phantom. Various monoenergetic reconstructions were generated, and radiomics features were extracted. Statistical analysis was performed using Random Forest (RF) classifiers for both individual reconstructions and a combined model.

RESULTS

The combined model, using radiomics features from all monoenergetic reconstructions, significantly outperformed individual reconstructions and SPP parameters, with an AUC of 0.95 and test accuracy of 0.81 for differentiating all six stone types. Feature importance analysis identified key parameters, including NGTDM_Strength and wavelet-LLH_firstorder_Variance.

CONCLUSION

This ex vivo study demonstrates that radiomics-driven PCCT analysis can improve differentiation between kidney stone subtypes. The combined model outperformed individual monoenergetic levels, highlighting the potential of spectral profiling in PCCT to optimize treatment through image-based strategies.

KEY POINTS

Question How can photon-counting computed tomography (PCCT) combined with radiomics improve the differentiation of kidney stone types beyond conventional CT and dual-energy CT, enhancing personalized therapy? Findings Our ex vivo study demonstrates that a combined spectral-driven radiomics model achieved 95% AUC and 81% test accuracy in differentiating six kidney stone types. Clinical relevance Implementing PCCT-based spectral-driven radiomics allows for precise non-invasive differentiation of kidney stone types, leading to improved diagnostic accuracy and more personalized, effective treatment strategies, potentially reducing the need for invasive procedures and recurrence.

Developing a Reproducible Radiomics Model for Diagnosis of Active Crohn's Disease on CT Enterography Across Annotation Variations and Acquisition Differences

To systematically identify radiomics features on CT enterography (CTE) scans which can accurately diagnose active Crohn's disease across multiple sources of variation. Retrospective study of CTE scans curated between 2013 and 2015, comprising 164 subjects (65 male, 99 female; all patients were over the age of 18) with endoscopic confirmation for the presence or absence of active Crohn's disease. All patients had three distinct sets of scans available (full and reduced dose, where the latter had been reconstructed via two different methods), acquired on a single scanner at a single institution. Radiomics descriptors from annotated terminal ileum regions were individually and systematically evaluated for resilience to different imaging variations (changes in dose/reconstruction, batch effects, and simulated annotation differences) via multiple reproducibility measures. Multiple radiomics models (by accounting for each source of variation) were evaluated in terms of classifier area under the ROC curve (AUC) for identifying patients with active Crohn's disease, across separate discovery and hold-out validation cohorts. Radiomics descriptors selected based on resiliency to multiple sources of imaging variation yielded the highest overall classification performance in the discovery cohort (AUC = 0.79 ± 0.04) which also best generalized in hold-out validation (AUC = 0.81). Performance was maintained across multiple doses and reconstructions while also being significantly better (p < 0.001) than non-resilient descriptors or descriptors only resilient to a single source of variation. Radiomics features can accurately diagnose active Crohn's disease on CTE scans across multiple sources of imaging variation via systematic analysis of reproducibility measures. Clinical utility and translatability of radiomics features for diagnosis and characterization of Crohn's disease on CTE scans will be contingent on their reproducibility across multiple types and sources of imaging variation.

ESR Essentials: a step-by-step guide of segmentation for radiologists-practice recommendations by the European Society of Medical Imaging Informatics

High-quality segmentation is important for AI-driven radiological research and clinical practice, with the potential to play an even more prominent role in the future. As medical imaging advances, accurately segmenting anatomical and pathological structures is increasingly used to obtain quantitative data and valuable insights. Segmentation and volumetric analysis could enable more precise diagnosis, treatment planning, and patient monitoring. These guidelines aim to improve segmentation accuracy and consistency, allowing for better decision-making in both research and clinical environments. Practical advice on planning and organization is provided, focusing on quality, precision, and communication among clinical teams. Additionally, tips and strategies for improving segmentation practices in radiology and radiation oncology are discussed, as are potential pitfalls to avoid. KEY POINTS: As AI continues to advance, volumetry will become more integrated into clinical practice, making it essential for radiologists to stay informed about its applications in diagnosis and treatment planning. There is a significant lack of practical guidelines and resources tailored specifically for radiologists on technical topics like segmentation and volumetric analysis. Establishing clear rules and best practices for segmentation can streamline volumetric assessment in clinical settings, making it easier to manage and leading to more accurate decision-making for patient care.

PET radiomics in lung cancer: advances and translational challenges

Radiomics is an emerging field of medical imaging that aims at improving the accuracy of diagnosis, prognosis, treatment planning and monitoring non-invasively through the automated or semi-automated quantitative analysis of high-dimensional image features. Specifically in the field of nuclear medicine, radiomics utilizes imaging methods such as positron emission tomography (PET) and single photon emission computed tomography (SPECT) to evaluate biomarkers related to metabolism, blood flow, cellular activity and some biological pathways. Lung cancer ranks among the leading causes of cancer-related deaths globally, and radiomics analysis has shown great potential in guiding individualized therapy, assessing treatment response, and predicting clinical outcomes. In this review, we summarize the current state-of-the-art radiomics progress in lung cancer, highlighting the potential benefits and existing limitations of this approach. The radiomics workflow was introduced first including image acquisition, segmentation, feature extraction, and model building. Then the published literatures were described about radiomics-based prediction models for lung cancer diagnosis, differentiation, prognosis and efficacy evaluation. Finally, we discuss current challenges and provide insights into future directions and potential opportunities for integrating radiomics into routine clinical practice.

Stability of Radiomic Features against Variations in Lesion Segmentations Computed on Apparent Diffusion Coefficient Maps of Breast Lesions

Diffusion-weighted imaging (DWI) combined with radiomics can aid in the differentiation of breast lesions. Segmentation characteristics, however, might influence radiomic features. To evaluate feature stability, we implemented a standardized pipeline featuring shifts and shape variations of the underlying segmentations. A total of 103 patients were retrospectively included in this IRB-approved study after multiparametric diagnostic breast 3T MRI with a spin-echo diffusion-weighted sequence with echoplanar readout (b-values: 50, 750 and 1500 s/mm2). Lesion segmentations underwent shifts and shape variations, with >100 radiomic features extracted from apparent diffusion coefficient (ADC) maps for each variation. These features were then compared and ranked based on their stability, measured by the Overall Concordance Correlation Coefficient (OCCC) and Dynamic Range (DR). Results showed variation in feature robustness to segmentation changes. The most stable features, excluding shape-related features, were FO (Mean, Median, RootMeanSquared), GLDM (DependenceNonUniformity), GLRLM (RunLengthNonUniformity), and GLSZM (SizeZoneNonUniformity), which all had OCCC and DR > 0.95 for both shifting and resizing the segmentation. Perimeter, MajorAxisLength, MaximumDiameter, PixelSurface, MeshSurface, and MinorAxisLength were the most stable features in the Shape category with OCCC and DR > 0.95 for resizing. Considering the variability in radiomic feature stability against segmentation variations is relevant when interpreting radiomic analysis of breast DWI data.

Comparing quantitative image parameters between animal and clinical CT-scanners: a translational phantom study analysis

Purpose

This study compares phantom-based variability of extracted radiomics features from scans on a photon counting CT (PCCT) and an experimental animal PET/CT-scanner (Albira II) to investigate the potential of radiomics for translation from animal models to human scans. While oncological basic research in animal PET/CT has allowed an intrinsic comparison between PET and CT, but no 1:1 translation to a human CT scanner due to resolution and noise limitations, Radiomics as a statistical and thus scale-independent method can potentially close the critical gap.

Methods

Two phantoms were scanned on a PCCT and animal PET/CT-scanner with different scan parameters and then the radiomics parameters were extracted. A Principal Component Analysis (PCA) was conducted. To overcome the limitation of a small dataset, a data augmentation technique was applied. A Ridge Classifier was trained and a Feature Importance- and Cluster analysis was performed.

Results

PCA and Cluster Analysis shows a clear differentiation between phantom types while emphasizing the comparability of both scanners. The Ridge Classifier exhibited a strong training performance with 93% accuracy, but faced challenges in generalization with a test accuracy of 62%.

Conclusion

These results show that radiomics has great potential as a translational tool between animal models and human routine diagnostics, especially using the novel photon counting technique. This is another crucial step towards integration of radiomics analysis into clinical practice.

Interobserver delineation variability of computed tomography-based radiomic features of the parotid gland

PURPOSE

To assess the interobserver delineation variability of radiomic features of the parotid gland from computed tomography (CT) images and evaluate the correlation of these features for head and neck cancer (HNC) radiotherapy patients.

MATERIALS AND METHODS

Contrast-enhanced CT images of 20 HNC patients were utilized. The parotid glands were delineated by treating radiation oncologists (ROs), a selected RO and AccuContour auto-segmentation software. Dice similarity coefficients (DSCs) between each pair of observers were calculated. A total of 107 radiomic features were extracted, whose robustness to interobserver delineation was assessed using the intraclass correlation coefficient (ICC). Pearson correlation coefficients (r) were calculated to determine the relationship between the features. The influence of excluding unrobust features from normal tissue complication probability (NTCP) modeling was investigated for severe oral mucositis (grade ≥3).

RESULTS

The average DSC was 0.84 (95% confidence interval, 0.83-0.86). Most of the shape features demonstrated robustness (ICC ≥0.75), while the first-order and texture features were influenced by delineation variability. Among the three observers investigated, 42 features were sufficiently robust, out of which 36 features exhibited weak correlation (|r|<0.8). No significant difference in the robustness level was found when comparing manual segmentation by a single RO or automated segmentation with the actual clinical contour data made by treating ROs. Excluding unrobust features from the NTCP model for severe oral mucositis did not deteriorate the model performance.

CONCLUSION

Interobserver delineation variability had substantial impact on radiomic features of the parotid gland. Both manual and automated segmentation methods contributed similarly to this variation.

Radiomics in CT and MR imaging of the liver and pancreas: tools with potential for clinical application

Radiomics allows the extraction of quantitative imaging features from clinical magnetic resonance imaging (MRI) and computerized tomography (CT) studies. The advantages of radiomics have primarily been exploited in oncological applications, including better characterization and staging of oncological lesions and prediction of patient outcomes and treatment response. The potential introduction of radiomics in the clinical setting requires the establishment of a standardized radiomics pipeline and a quality assurance program. Radiomics and texture analysis of the liver have improved the differentiation of hypervascular lesions such as adenomas, focal nodular hyperplasia, and hepatocellular carcinoma (HCC) during the arterial phase, and in the pretreatment determination of HCC prognostic factors (e.g., tumor grade, microvascular invasion, Ki-67 proliferation index). Radiomics of pancreatic CT and MR images has enhanced pancreatic ductal adenocarcinoma detection and its differentiation from pancreatic neuroendocrine tumors, mass-forming chronic pancreatitis, or autoimmune pancreatitis. Radiomics can further help to better characterize incidental pancreatic cystic lesions, accurately discriminating benign from malignant intrapancreatic mucinous neoplasms. Nonetheless, despite their encouraging results and exciting potential, these tools have yet to be implemented in the clinical setting. This non-systematic review will describe the essential steps in the implementation of the radiomics and feature extraction workflow from liver and pancreas CT and MRI studies for their potential clinical application. A succinct overview of reported radiomics applications in the liver and pancreas and the challenges and limitations of their implementation in the clinical setting is also discussed, concluding with a brief exploration of the future perspectives of radiomics in the gastroenterology field.

Spatially-aware clustering improves AJCC-8 risk stratification performance in oropharyngeal carcinomas

OBJECTIVE

Evaluate the effectiveness of machine learning tools that incorporate spatial information such as disease location and lymph node metastatic patterns-of-spread, for prediction of survival and toxicity in HPV+ oropharyngeal cancer (OPC).

MATERIALS & METHODS

675 HPV+ OPC patients that were treated at MD Anderson Cancer Center between 2005 and 2013 with curative intent IMRT were retrospectively collected under IRB approval. Risk stratifications incorporating patient radiometric data and lymph node metastasis patterns via an anatomically-adjacent representation with hierarchical clustering were identified. These clusterings were combined into a 3-level patient stratification and included along with other known clinical features in a Cox model for predicting survival outcomes, and logistic regression for toxicity, using independent subsets for training and validation.

RESULTS

Four groups were identified and combined into a 3-level stratification. The inclusion of patient stratifications in predictive models for 5-yr Overall survival (OS), 5-year recurrence free survival, (RFS) and Radiation-associated dysphagia (RAD) consistently improved model performance measured using the area under the curve (AUC). Test set AUC improvements over models with clinical covariates, was 9 % for predicting OS, and 18 % for predicting RFS, and 7 % for predicting RAD. For models with both clinical and AJCC covariates, AUC improvement was 7 %, 9 %, and 2 % for OS, RFS, and RAD, respectively.

CONCLUSION

Including data-driven patient stratifications considerably improve prognosis for survival and toxicity outcomes over the performance achieved by clinical staging and clinical covariates alone. These stratifications generalize well to across cohorts, and sufficient information for reproducing these clusters is included.

Phantom-based radiomics feature test-retest stability analysis on photon-counting detector CT

OBJECTIVES

Radiomics image data analysis offers promising approaches in research but has not been implemented in clinical practice yet, partly due to the instability of many parameters. The aim of this study is to evaluate the stability of radiomics analysis on phantom scans with photon-counting detector CT (PCCT).

METHODS

Photon-counting CT scans of organic phantoms consisting of 4 apples, kiwis, limes, and onions each were performed at 10 mAs, 50 mAs, and 100 mAs with 120-kV tube current. The phantoms were segmented semi-automatically and original radiomics parameters were extracted. This was followed by statistical analysis including concordance correlation coefficients (CCC), intraclass correlation coefficients (ICC), as well as random forest (RF) analysis, and cluster analysis to determine the stable and important parameters.

RESULTS

Seventy-three of the 104 (70%) extracted features showed excellent stability with a CCC value > 0.9 when compared in a test and retest analysis, and 68 features (65.4%) were stable compared to the original in a rescan after repositioning. Between the test scans with different mAs values, 78 (75%) features were rated with excellent stability. Eight radiomics features were identified that had an ICC value greater than 0.75 in at least 3 of 4 groups when comparing the different phantoms in a phantom group. In addition, the RF analysis identified many features that are important for distinguishing the phantom groups.

CONCLUSION

Radiomics analysis using PCCT data provides high feature stability on organic phantoms, which may facilitate the implementation of radiomics analysis likewise in clinical routine.

KEY POINTS

• Radiomics analysis using photon-counting computed tomography provides high feature stability. • Photon-counting computed tomography may pave the way for implementation of radiomics analysis in clinical routine.

Radiomics in Cardiac Computed Tomography

In recent years, there has been an increasing recognition of coronary computed tomographic angiography (CCTA) and gated non-contrast cardiac CT in the workup of coronary artery disease in patients with low and intermediate pretest probability, through the readjustment guidelines by medical societies. However, in routine clinical practice, these CT data sets are usually evaluated dominantly regarding relevant coronary artery stenosis and calcification. The implementation of radiomics analysis, which provides visually elusive quantitative information from digital images, has the potential to open a new era for cardiac CT that goes far beyond mere stenosis or calcification grade estimation. This review offers an overview of the results obtained from radiomics analyses in cardiac CT, including the evaluation of coronary plaques, pericoronary adipose tissue, and the myocardium itself. It also highlights the advantages and disadvantages of use in routine clinical practice.

Evaluation of radiomics feature stability in abdominal monoenergetic photon counting CT reconstructions

Feature stability and standardization remain challenges that impede the clinical implementation of radiomics. This study investigates the potential of spectral reconstructions from photon-counting computed tomography (PCCT) regarding organ-specific radiomics feature stability. Abdominal portal-venous phase PCCT scans of 10 patients in virtual monoenergetic (VM) (keV 40-120 in steps of 10), polyenergetic, virtual non-contrast (VNC), and iodine maps were acquired. Two 2D and 3D segmentations measuring 1 and 2 cm in diameter of the liver, lung, spleen, psoas muscle, subcutaneous fat, and air were obtained for spectral reconstructions. Radiomics features were extracted with pyradiomics. The calculation of feature-specific intraclass correlation coefficients (ICC) was performed by comparing all segmentation approaches and organs. Feature-wise and organ-wise correlations were evaluated. Segmentation-resegmentation stability was evaluated by concordance correlation coefficient (CCC). Compared to non-VM, VM-reconstruction features tended to be more stable. For VM reconstructions, 3D 2 cm segmentation showed the highest average ICC with 0.63. Based on a criterion of ≥ 3 stable organs and an ICC of ≥ 0.75, 12-mainly non-first-order features-are shown to be stable between the VM reconstructions. In a segmentation-resegmentation analysis in 3D 2 cm, three features were identified as stable based on a CCC of > 0.6 in ≥ 3 organs in ≥ 6 VM reconstructions. Certain radiomics features vary between monoenergetic reconstructions and depend on the ROI size. Feature stability was also shown to differ between different organs. Yet, glcm_JointEntropy, gldm_GrayLevelNonUniformity, and firstorder_Entropy could be identified as features that could be interpreted as energy-independent and segmentation-resegmentation stable in this PCCT collective. PCCT may support radiomics feature standardization and comparability between sites.

Feasibility of using CT radiomic signatures for predicting CD8-T cell infiltration and PD-L1 expression in renal cell carcinoma

Objectives

To identify computed tomography (CT)-based radiomic signatures of cluster of differentiation 8 (CD8)-T cell infiltration and programmed cell death ligand 1 (PD-L1) expression levels in patients with clear-cell renal cell carcinoma (ccRCC).

Methods

Seventy-eight patients with pathologically confirmed localized ccRCC, preoperative multiphase CT and tumor resection specimens were enrolled in this retrospective study. Regions of interest (ROI) of the ccRCC volume were manually segmented from the CT images and processed using a radiomics panel comprising of 1708 metrics. The extracted metrics were used as inputs to three machine learning classifiers: Random Forest, AdaBoost, and ElasticNet to create radiomic signatures for CD8-T cell infiltration and PD-L1 expression, respectively.

Results

Using a cut-off of 80 lymphocytes per high power field, 59 % were classified to CD8 highly infiltrated tumors and 41 % were CD8 non highly infiltrated tumors, respectively. An ElasticNet classifier discriminated between these two groups of CD8-T cells with an AUC of 0.68 (95 % CI, 0.55-0.80). In addition, based on tumor proportion score with a cut-off of > 1 % tumor cells expressing PD-L1, 76 % were PD-L1 positive and 24 % were PD-L1 negative. An Adaboost classifier discriminated between PD-L1 positive and PD-L1 negative tumors with an AUC of 0.8 95 % CI: (0.66, 0.95). 3D radiomics metrics of graylevel co-occurrence matrix (GLCM) and graylevel run-length matrix (GLRLM) metrics drove the performance for CD8-Tcell and PD-L1 classification, respectively.

Conclusions

CT-radiomic signatures can differentiate tumors with high CD8-T cell infiltration with moderate accuracy and positive PD-L1 expression with good accuracy in ccRCC.

Largest diameter delineations can substitute 3D tumor volume delineations for radiomics prediction of human papillomavirus status on MRI's of oropharyngeal cancer

PURPOSE

Laborious and time-consuming tumor segmentations are one of the factors that impede adoption of radiomics in the clinical routine. This study investigates model performance using alternative tumor delineation strategies in models predictive of human papillomavirus (HPV) in oropharyngeal squamous cell carcinoma (OPSCC).

METHODS

Of 153 OPSCC patients, HPV status was determined using p16/p53 immunohistochemistry. MR-based radiomic features were extracted within 3D delineations by an inexperienced observer, experienced radiologist or radiation oncologist, and within a 2D delineation of the largest axial tumor diameter and 3D spheres within the tumor. First, logistic regression prediction models were constructed and tested separately for each of these six delineation strategies. Secondly, the model trained on experienced delineations was tested using these delineation strategies. The latter methodology was repeated with the omission of shape features. Model performance was evaluated using area under the curve (AUC), sensitivity and specificity.

RESULTS

Models constructed and tested using single-slice delineations (AUC/Sensitivity/Specificity: 0.84/0.75/0.84) perform better compared to 3D experienced observer delineations (AUC/Sensitivity/Specificity: 0.76/0.76/0.71), where models based on 4 mm sphere delineations (AUC/Sensitivity/Specificity: 0.77/0.59/0.71) show similar performance. Similar performance was found when experienced and largest diameter delineations (AUC/Sens/Spec: 0.76/0.75/0.65 vs 0.76/0.69/0.69) was used to test the model constructed using experienced delineations without shape features.

CONCLUSION

Alternative delineations can substitute labor and time intensive full tumor delineations in a model that predicts HPV status in OPSCC. These faster delineations may improve adoption of radiomics in the clinical setting. Future research should evaluate whether these alternative delineations are valid in other radiomics models.

MRI-based radiomics to predict response in locally advanced rectal cancer: comparison of manual and automatic segmentation on external validation in a multicentre study

Background

Pathological complete response after neoadjuvant chemoradiotherapy in locally advanced rectal cancer (LARC) is achieved in 15–30% of cases. Our aim was to implement and externally validate a magnetic resonance imaging (MRI)-based radiomics pipeline to predict response to treatment and to investigate the impact of manual and automatic segmentations on the radiomics models.

Methods

Ninety-five patients with stage II/III LARC who underwent multiparametric MRI before chemoradiotherapy and surgical treatment were enrolled from three institutions. Patients were classified as responders if tumour regression grade was 1 or 2 and nonresponders otherwise. Sixty-seven patients composed the construction dataset, while 28 the external validation. Tumour volumes were manually and automatically segmented using a U-net algorithm. Three approaches for feature selection were tested and combined with four machine learning classifiers.

Results

Using manual segmentation, the best result reached an accuracy of 68% on the validation set, with sensitivity 60%, specificity 77%, negative predictive value (NPV) 63%, and positive predictive value (PPV) 75%. The automatic segmentation achieved an accuracy of 75% on the validation set, with sensitivity 80%, specificity 69%, and both NPV and PPV 75%. Sensitivity and NPV on the validation set were significantly higher (p = 0.047) for the automatic versus manual segmentation.

Conclusion

Our study showed that radiomics models can pave the way to help clinicians in the prediction of tumour response to chemoradiotherapy of LARC and to personalise per-patient treatment. The results from the external validation dataset are promising for further research into radiomics approaches using both manual and automatic segmentations.

Supplementary Information

The online version contains supplementary material available at 10.1186/s41747-022-00272-2.

Feature Robustness and Diagnostic Capabilities of Convolutional Neural Networks Against Radiomics Features in Computed Tomography Imaging

MATERIALS AND METHODS

Imaging phantoms were scanned twice on 3 computed tomography scanners from 2 different manufactures with varying tube voltages and currents. Phantoms were segmented, and features were extracted using PyRadiomics and a pretrained CNN. After standardization the concordance correlation coefficient (CCC), mean feature variance, feature range, and the coefficient of variant were calculated to assess feature robustness. In addition, the cosine similarity was calculated for the vectorized activation maps for an exemplary phantom. For the in vivo comparison, the radiomics and CNN features of 30 patients with hepatocellular carcinoma (HCC) and 30 patients with hepatic colon carcinoma metastasis were compared.

RESULTS

In total, 851 radiomics features and 256 CNN features were extracted for each phantom. For all phantoms, the global CCC of the CNN features was above 98%, whereas the highest CCC for the radiomics features was 36%. The mean feature variance and feature range was significantly lower for the CNN features. Using a coefficient of variant ≤0.2 as a threshold to define robust features and averaging across all phantoms 346 of 851 (41%) radiomics features and 196 of 256 (77%) CNN features were found to be robust. The cosine similarity was greater than 0.98 for all scanner and parameter variations. In the retrospective analysis, 122 of the 256 CNN (49%) features showed significant differences between HCC and hepatic colon metastasis.

DISCUSSION

Convolutional neural network features were more stable compared with radiomics features against technical variations. Moreover, the possibility of tumor entity differentiation based on CNN features was shown. Combined with visualization methods, CNN features are expected to increase reproducibility of quantitative image representations. Further studies are warranted to investigate the impact of feature stability on radiological image-based prediction of clinical outcomes.

Role of hepatic metastatic lesion size on inter-reader reproducibility of CT-based radiomics features

OBJECTIVES

To evaluate the effect of hepatic metastatic lesion size on inter-reader reproducibility of CT-based 2D radiomics imaging features.

METHODS

Computerized tomography (CT) scans of 59 liver metastases from 34 patients with colorectal cancer were evaluated. Image segmentation was performed manually by three readers blinded to each other's results. For each radiomics feature, we created two datasets by sorting measurements according to size, i.e., (i) from the smallest to the largest lesion and (ii) from the largest to the smallest lesion. The Lin concordance correlation coefficient (CCC) was employed to analyze the reproducibility of radiomics features. In particular, the CCC was computed as a function of a number of elements in the dataset, by gradually adding lesions from each sorted dataset. To evaluate the effect of lesion size, we analyzed the difference between these two functions thus assessing the contribution of small and large lesions into the reproducibility of radiomics features.

RESULTS

Inter-reader reproducibility of CT-based 2D radiomics features assessed using Lin's CCC demonstrates tumor-size dependence. For example, the Lin CCC for GLCM contrast equals 0.88 (95% C.I. 0.84 to 0.92, p < 0.003) and could change by an additional + / - 0.06 depending on the presence of large or small lesions.

CONCLUSIONS

Groups of "large" and "small" lesions show different inter-reader reproducibility. The inter-reader reproducibility from the mixed group consisting of "large" and "small" lesions depends on the lesion-size distribution and can vary widely. This finding could partially explain variability in reproducibility of radiomics features in the literature.

KEY POINTS

• Groups of "large" and "small" lesions show different inter-reader reproducibility. • The inter-reader reproducibility from the mixed group consisting of "large" and "small" lesions depends on the lesion-size distribution.

Robustness and performance of radiomic features in diagnosing cystic renal masses

PURPOSE

We study the inter-reader variability in manual delineation of cystic renal masses (CRMs) presented in computerized tomography (CT) images and its effect on the classification performance of a machine learning algorithm in distinguishing benign from potentially malignant CRMs. In addition, we assessed whether the inclusion of higher-order robust radiomic features improves the classification performance over the use of first-order features.

METHODS

230 CRMs were independently delineated by two radiologists. Through a combination of random fluctuations, dilation, and erosion operations over the original region of interests (ROIs), we generated four additional sets of synthetic ROIs to capture the inter-reader variability realistically, as confirmed by dice coefficient measurements and visual assessment. We then identified the robust features based on the intra-class coefficient (ICC > 0.85) across these datasets. We applied a tenfold stratified cross-validation (CV) to train and test the performance of the random forest model for the classification of CRMs into benign and potentially malignant.

RESULTS

The mean area under the curve (AUC), sensitivity, specificity, positive predictive value, and negative predictive value were 0.87, 0.82, 0.90, 0.85, and 0.93, respectively. With the usage of first-order features alone, the corresponding values were nearly identical.

CONCLUSION

AUC ranged for the robust and uncorrelated features from 0.83 ± 0.09 to 0.93 ± 0.04 and for the first-order features from 0.84 ± 0.09 to 0.91 ± 0.04. Our study indicates that the first-order features alone are sufficient for the classification of CRMs, and that inclusion of higher-order features does not necessarily improve performance.

Radiomics feature reliability assessed by intraclass correlation coefficient: a systematic review

Radiomics research is rapidly growing in recent years, but more concerns on radiomics reliability are also raised. This review attempts to update and overview the current status of radiomics reliability research in the ever expanding medical literature from the perspective of a single reliability metric of intraclass correlation coefficient (ICC). To conduct this systematic review, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed. After literature search and selection, a total of 481 radiomics studies using CT, PET, or MRI, covering a wide range of subject and disease types, were included for review. In these highly heterogeneous studies, feature reliability to image segmentation was much more investigated than reliability to other factors, such as image acquisition, reconstruction, post-processing, and feature quantification. The reported ICCs also suggested high radiomics feature reliability to image segmentation. Image acquisition was found to introduce much more feature variability than image segmentation, in particular for MRI, based on the reported ICC values. Image post-processing and feature quantification yielded different levels of radiomics reliability and might be used to mitigate image acquisition-induced variability. Some common flaws and pitfalls in ICC use were identified, and suggestions on better ICC use were given. Due to the extremely high study heterogeneities and possible risks of bias, the degree of radiomics feature reliability that has been achieved could not yet be safely synthesized or derived in this review. More future researches on radiomics reliability are warranted.

Reproducibility analysis of multi-institutional paired expert annotations and radiomic features of the Ivy Glioblastoma Atlas Project (Ivy GAP) dataset

PURPOSE

The availability of radiographic magnetic resonance imaging (MRI) scans for the Ivy Glioblastoma Atlas Project (Ivy GAP) has opened up opportunities for development of radiomic markers for prognostic/predictive applications in glioblastoma (GBM). In this work, we address two critical challenges with regard to developing robust radiomic approaches: (a) the lack of availability of reliable segmentation labels for glioblastoma tumor sub-compartments (i.e., enhancing tumor, non-enhancing tumor core, peritumoral edematous/infiltrated tissue) and (b) identifying "reproducible" radiomic features that are robust to segmentation variability across readers/sites.

ACQUISITION AND VALIDATION METHODS

From TCIA's Ivy GAP cohort, we obtained a paired set (n = 31) of expert annotations approved by two board-certified neuroradiologists at the Hospital of the University of Pennsylvania (UPenn) and at Case Western Reserve University (CWRU). For these studies, we performed a reproducibility study that assessed the variability in (a) segmentation labels and (b) radiomic features, between these paired annotations. The radiomic variability was assessed on a comprehensive panel of 11 700 radiomic features including intensity, volumetric, morphologic, histogram-based, and textural parameters, extracted for each of the paired sets of annotations. Our results demonstrated (a) a high level of inter-rater agreement (median value of DICE ≥0.8 for all sub-compartments), and (b) ≈24% of the extracted radiomic features being highly correlated (based on Spearman's rank correlation coefficient) to annotation variations. These robust features largely belonged to morphology (describing shape characteristics), intensity (capturing intensity profile statistics), and COLLAGE (capturing heterogeneity in gradient orientations) feature families.

DATA FORMAT AND USAGE NOTES

We make publicly available on TCIA's Analysis Results Directory (https://doi.org/10.7937/9j41-7d44), the complete set of (a) multi-institutional expert annotations for the tumor sub-compartments, (b) 11 700 radiomic features, and (c) the associated reproducibility meta-analysis.

POTENTIAL APPLICATIONS

The annotations and the associated meta-data for Ivy GAP are released with the purpose of enabling researchers toward developing image-based biomarkers for prognostic/predictive applications in GBM.

Impact of inter-reader contouring variability on textural radiomics of colorectal liver metastases

BACKGROUND

Radiomics is expected to improve the management of metastatic colorectal cancer (CRC). We aimed at evaluating the impact of liver lesion contouring as a source of variability on radiomic features (RFs).

METHODS

After Ethics Committee approval, 70 liver metastases in 17 CRC patients were segmented on contrast-enhanced computed tomography scans by two residents and checked by experienced radiologists. RFs from grey level co-occurrence and run length matrices were extracted from three-dimensional (3D) regions of interest (ROIs) and the largest two-dimensional (2D) ROIs. Inter-reader variability was evaluated with Dice coefficient and Hausdorff distance, whilst its impact on RFs was assessed using mean relative change (MRC) and intraclass correlation coefficient (ICC). For the main lesion of each patient, one reader also segmented a circular ROI on the same image used for the 2D ROI.

RESULTS

The best inter-reader contouring agreement was observed for 2D ROIs according to both Dice coefficient (median 0.85, interquartile range 0.78-0.89) and Hausdorff distance (0.21 mm, 0.14-0.31 mm). Comparing RF values, MRC ranged 0-752% for 2D and 0-1567% for 3D. For 24/32 RFs (75%), MRC was lower for 2D than for 3D. An ICC > 0.90 was observed for more RFs for 2D (53%) than for 3D (34%). Only 2/32 RFs (6%) showed a variability between 2D and circular ROIs higher than inter-reader variability.

CONCLUSIONS

A 2D contouring approach may help mitigate overall inter-reader variability, albeit stable RFs can be extracted from both 3D and 2D segmentations of CRC liver metastases.

Development of a Radiomics Prediction Model for Histological Type Diagnosis in Solitary Pulmonary Nodules: The Combination of CT and FDG PET

PURPOSE

To develop a diagnostic model for histological subtypes in lung cancer combined CT and FDG PET.

METHODS

Machine learning binary and four class classification of a cohort of 445 lung cancer patients who have CT and PET simultaneously. The outcomes to be predicted were primary, metastases (Mts), adenocarcinoma (Adc), and squamous cell carcinoma (Sqc). The classification method is a combination of machine learning and feature selection that is a Partition-Membership. The performance metrics include accuracy (Acc), precision (Pre), area under curve (AUC) and kappa statistics.

RESULTS

The combination of CT and PET radiomics (CPR) binary model showed more than 98% Acc and AUC on predicting Adc, Sqc, primary, and metastases, CPR four-class classification model showed 91% Acc and 0.89 Kappa.

CONCLUSION

The proposed CPR models can be used to obtain valid predictions of histological subtypes in lung cancer patients, assisting in diagnosis and shortening the time to diagnostic.