Reproducibility and Repeatability of CBCT-Derived Radiomics Features

Mastering CT-based radiomic research in lung cancer: a practical guide from study design to critical appraisal

Radiomics is a health technology that has the potential to extract clinically meaningful biomarkers from standard of care imaging. Despite a wealth of exploratory analysis performed on scans acquired from patients with lung cancer and existing guidelines describing some of the key steps, no radiomic-based biomarker has been widely accepted. This is primarily due to limitations with methodology, data analysis, and interpretation of the available studies. There is currently a lack of guidance relating to the entire radiomic workflow from study design to critical appraisal. This guide, written with early career lung cancer researchers, describes a more complete radiomic workflow. Lung cancer image analysis is the focus due to some of the unique challenges encountered such as patient movement from breathing. The guide will focus on CT imaging as these are the most common scans performed on patients with lung cancer. The aim of this article is to support the production of high-quality research that has the potential to positively impact outcome of patients with lung cancer.

Can Machine Learning Models Based on Computed Tomography Radiomics and Clinical Characteristics Provide Diagnostic Value for Epstein-Barr Virus-Associated Gastric Cancer?

OBJECTIVE

The aim of this study was to explore whether machine learning model based on computed tomography (CT) radiomics and clinical characteristics can differentiate Epstein-Barr virus-associated gastric cancer (EBVaGC) from non-EBVaGC.

METHODS

Contrast-enhanced CT images were collected from 158 patients with GC (46 EBV-positive, 112 EBV-negative) between April 2018 and February 2023. Radiomics features were extracted from the volumes of interest. A radiomics signature was built based on radiomics features by the least absolute shrinkage and selection operator logistic regression algorithm. Multivariate analyses were used to identify significant clinicoradiological variables. We developed 6 ML models for EBVaGC, including logistic regression, Extreme Gradient Boosting, random forest (RF), support vector machine, Gaussian Naive Bayes, and K-nearest neighbor algorithm. The area under the receiver operating characteristic curve (AUC), the area under the precision-recall curves (AP), calibration plots, and decision curve analysis were applied to assess the effectiveness of each model.

RESULTS

Six ML models achieved AUC of 0.706-0.854 and AP of 0.480-0.793 for predicting EBV status in GC. With an AUC of 0.854 and an AP of 0.793, the RF model performed the best. The forest plot of the AUC score revealed that the RF model had the most stable performance, with a standard deviation of 0.003 for AUC score. RF also performed well in the testing dataset, with an AUC of 0.832 (95% confidence interval: 0.679-0.951), accuracy of 0.833, sensitivity of 0.857, and specificity of 0.824, respectively.

CONCLUSIONS

The RF model based on clinical variables and Rad_score can serve as a noninvasive tool to evaluate the EBV status of gastric cancer.

Cone beam computed tomography (CBCT) and megavoltage computed tomography (MVCT)-based radiomics in head and neck cancers: a systematic review and radiomics quality score assessment

Background

Cone beam computed tomography (CBCT) and megavoltage computed tomography (MVCT)-based images demonstrate measurable radiomics features that are potentially prognostic. This study aims to systematically synthesize the current research applying radiomics in head and neck cancers for outcome prediction and to assess the radiomics quality score (RQS) of the studies.

Methods

A systematic search was performed to identify available studies on PubMed, Web of Science, and Scopus databases. Studies related to radiomics in oncology/radiotherapy fields and based on predefined Patient, Intervention, Comparator, Outcome, and Study design (PICOS) criteria were included. The methodological quality of the included study was evaluated independently by two reviewers according to the RQS. The Mann-Whitney U test was performed according to subgroups. The P values <0.05 were considered statistically significant. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 reporting guidelines were adhered to.

Results

From a total of 743 identified studies, six original studies were eligible for inclusion in the systematic review (median =97 patients). The intraclass correlation coefficient (ICC) for inter-reviewer on total RQS was excellent with 0.99 [95% confidence interval (CI) of 0.946< ICC <0.999]. There were no significant differences in the analyses between each RQS domain and subgroup components (P always >0.05). Numerically higher RQS domains score for publication year ≤2022 than 2023 and number of patients > median than ≤ median but not statistically significant.

Conclusions

The number of radiomics studies involving CBCT and MVCT is still very limited. Self-reported RQS assessments should be encouraged for all radiomics studies.

Stability of Radiomic Features against Variations in Lesion Segmentations Computed on Apparent Diffusion Coefficient Maps of Breast Lesions

Diffusion-weighted imaging (DWI) combined with radiomics can aid in the differentiation of breast lesions. Segmentation characteristics, however, might influence radiomic features. To evaluate feature stability, we implemented a standardized pipeline featuring shifts and shape variations of the underlying segmentations. A total of 103 patients were retrospectively included in this IRB-approved study after multiparametric diagnostic breast 3T MRI with a spin-echo diffusion-weighted sequence with echoplanar readout (b-values: 50, 750 and 1500 s/mm2). Lesion segmentations underwent shifts and shape variations, with >100 radiomic features extracted from apparent diffusion coefficient (ADC) maps for each variation. These features were then compared and ranked based on their stability, measured by the Overall Concordance Correlation Coefficient (OCCC) and Dynamic Range (DR). Results showed variation in feature robustness to segmentation changes. The most stable features, excluding shape-related features, were FO (Mean, Median, RootMeanSquared), GLDM (DependenceNonUniformity), GLRLM (RunLengthNonUniformity), and GLSZM (SizeZoneNonUniformity), which all had OCCC and DR > 0.95 for both shifting and resizing the segmentation. Perimeter, MajorAxisLength, MaximumDiameter, PixelSurface, MeshSurface, and MinorAxisLength were the most stable features in the Shape category with OCCC and DR > 0.95 for resizing. Considering the variability in radiomic feature stability against segmentation variations is relevant when interpreting radiomic analysis of breast DWI data.

Reproducibility Analysis of Radiomic Features on T2-weighted MR Images after Processing and Segmentation Alterations in Neuroblastoma Tumors

Purpose To evaluate the reproducibility of radiomics features extracted from T2-weighted MR images in patients with neuroblastoma. Materials and Methods A retrospective study included 419 patients (mean age, 29 months ± 34 [SD]; 220 male, 199 female) with neuroblastic tumors diagnosed between 2002 and 2023, within the scope of the PRedictive In-silico Multiscale Analytics to support cancer personalized diaGnosis and prognosis, Empowered by imaging biomarkers (ie, PRIMAGE) project, involving 746 T2/T2*-weighted MRI sequences at diagnosis and/or after initial chemotherapy. Images underwent processing steps (denoising, inhomogeneity bias field correction, normalization, and resampling). Tumors were automatically segmented, and 107 shape, first-order, and second-order radiomics features were extracted, considered as the reference standard. Subsequently, the previous image processing settings were modified, and volumetric masks were applied. New radiomics features were extracted and compared with the reference standard. Reproducibility was assessed using the concordance correlation coefficient (CCC); intrasubject repeatability was measured using the coefficient of variation (CoV). Results When normalization was omitted, only 5% of the radiomics features demonstrated high reproducibility. Statistical analysis revealed significant changes in the normalization and resampling processes (P < .001). Inhomogeneities removal had the least impact on radiomics (83% of parameters remained stable). Shape features remained stable after mask modifications, with a CCC greater than 0.90. Mask modifications were the most favorable changes for achieving high CCC values, with a radiomics features stability of 70%. Only 7% of second-order radiomics features showed an excellent CoV of less than 0.10. Conclusion Modifications in the T2-weighted MRI preparation process in patients with neuroblastoma resulted in changes in radiomics features, with normalization identified as the most influential factor for reproducibility. Inhomogeneities removal had the least impact on radiomics features. Keywords: Pediatrics, MR Imaging, Oncology, Radiomics, Reproducibility, Repeatability, Neuroblastic Tumors Supplemental material is available for this article. © RSNA, 2024 See also the commentary by Safdar and Galaria in this issue.

A comparative analysis of preclinical computed tomography radiomics using cone-beam and micro-computed tomography scanners

Background and purpose

Radiomics analysis extracts quantitative data (features) from medical images. These features could potentially reflect biological characteristics and act as imaging biomarkers within precision medicine. However, there is a lack of cross-comparison and validation of radiomics outputs which is paramount for clinical implementation. In this study, we compared radiomics outputs across two computed tomography (CT)-based preclinical scanners.

Materials and methods

Cone beam CT (CBCT) and µCT scans were acquired using different preclinical CT imaging platforms. The reproducibility of radiomics features on each scanner was assessed using a phantom across imaging energies (40 & 60 kVp) and segmentation volumes (44-238 mm3). Retrospective mouse scans were used to compare feature reliability across varying tissue densities (lung, heart, bone), scanners and after voxel size harmonisation. Reliable features had an intraclass correlation coefficient (ICC) > 0.8.

Results

First order and GLCM features were the most reliable on both scanners across different volumes. There was an inverse relationship between tissue density and feature reliability, with the highest number of features in lung (CBCT=580, µCT=734) and lowest in bone (CBCT=110, µCT=560). Comparable features for lung and heart tissues increased when voxel sizes were harmonised. We have identified tissue-specific preclinical radiomics signatures in mice for the lung (133), heart (35), and bone (15).

Conclusions

Preclinical CBCT and µCT scans can be used for radiomics analysis to support the development of meaningful radiomics signatures. This study demonstrates the importance of standardisation and emphasises the need for multi-centre studies.

Accuracy of cone-beam breast computed tomography for assessing breast cancer tumor size-comparison with breast magnetic resonance imaging

Background

Accurate preoperative assessment of tumor size is important in developing a surgical plan for breast cancer. The purpose of this study was to evaluate the accuracy of cone-beam breast computed tomography (CBBCT) and magnetic resonance imaging (MRI) in the assessment of tumor size and to analyze the factors influencing the discordance.

Methods

In this retrospective study, patients with breast cancer who underwent preoperative contrast-enhanced CBBCT (CE-CBBCT) and dynamic contrast-enhanced MRI (DCE-MRI) and received a complete pathologic diagnosis from August 2020 to December 2021 were included, using the pathological result as the gold standard. Two radiologists assessed the CBBCT and MRI features and measured the tumor size with a 2-week washout period. Intraclass correlation coefficient (ICC) and Bland-Altman analyses were used to assess inter-observer reproducibility and agreement based on CBBCT, MRI and pathology. Univariate analyses of differences in clinical, pathological and CBBCT/MRI features between the concordant and discordant groups was performed using the t-test, Mann-Whitney U-test, Chi-squared test and Fisher's exact test. Multivariate analyses were used to identify factors associated with discordance of CBBCT/MRI with pathology.

Results

A total of 115 female breast cancer patients (115 lesions) were included. All patients had a single malignant tumor of the unilateral breast. The reproducibility and the agreement ranged from moderate to excellent (ICC =0.607-0.983). Receiver operating characteristic (ROC) analyses showed that the cut-off values of CBBCT-pathology and MRI-pathology discordance were 2.25 and 2.65 cm, respectively. CBBCT/MRI-pathology concordance was significantly associated with the extent of pathology, lesion type, presence of calcification, human epidermal growth factor receptor 2 (HER2) status and fatty infiltration (P<0.05). In lesions containing calcification, the difference of CBBCT-pathology was significantly smaller than MRI-pathology (P=0.021). Non-mass enhancement (NME) was the main predictor of CBBCT- or MRI-pathology discordance [odds ratio (OR) =3.293-6.469, P<0.05], and HER2 positivity was a predictor of CBBCT-pathology discordance (OR =3.514, P=0.019).

Conclusions

CBBCT and MRI have comparable accuracy in measurement of tumor size, and CBBCT is advantageous in assessing the size of calcified lesions. NME and HER2 positivity are significant predictors of CBBCT-pathology discordance. This suggests that CBBCT might serve as an alternative imaging technique to assess tumor size when patients do not tolerate MRI.

Artificial intelligence applied to image-guided radiation therapy (IGRT): a systematic review by the Young Group of the Italian Association of Radiotherapy and Clinical Oncology (yAIRO)

INTRODUCTION

The advent of image-guided radiation therapy (IGRT) has recently changed the workflow of radiation treatments by ensuring highly collimated treatments. Artificial intelligence (AI) and radiomics are tools that have shown promising results for diagnosis, treatment optimization and outcome prediction. This review aims to assess the impact of AI and radiomics on modern IGRT modalities in RT.

METHODS

A PubMed/MEDLINE and Embase systematic review was conducted to investigate the impact of radiomics and AI to modern IGRT modalities. The search strategy was "Radiomics" AND "Cone Beam Computed Tomography"; "Radiomics" AND "Magnetic Resonance guided Radiotherapy"; "Radiomics" AND "on board Magnetic Resonance Radiotherapy"; "Artificial Intelligence" AND "Cone Beam Computed Tomography"; "Artificial Intelligence" AND "Magnetic Resonance guided Radiotherapy"; "Artificial Intelligence" AND "on board Magnetic Resonance Radiotherapy" and only original articles up to 01.11.2022 were considered.

RESULTS

A total of 402 studies were obtained using the previously mentioned search strategy on PubMed and Embase. The analysis was performed on a total of 84 papers obtained following the complete selection process. Radiomics application to IGRT was analyzed in 23 papers, while a total 61 papers were focused on the impact of AI on IGRT techniques.

DISCUSSION

AI and radiomics seem to significantly impact IGRT in all the phases of RT workflow, even if the evidence in the literature is based on retrospective data. Further studies are needed to confirm these tools' potential and provide a stronger correlation with clinical outcomes and gold-standard treatment strategies.

Reproducibility and location-stability of radiomic features derived from cone-beam computed tomography: a phantom study

OBJECTIVES

This study aims to determine the reproducibility and location-stability of cone-beam computed tomography (CBCT) radiomic features.

METHODS

Centrifugal tubes with six concentrations of K2HPO4 solutions (50, 100, 200, 400, 600, and 800 mg ml-1) were imaged within a customized phantom. For each concentration, images were captured twice as test and retest sets. Totally, 69 radiomic features were extracted by LIFEx. The reproducibility was assessed between the test and retest sets. We used the concordance correlation coefficient (CCC) to screen qualified features and then compared the differences in the numbers of them under 24 series (four locations groups * six concentrations). The location-stability was assessed using the Kruskal-Wallis test under different concentration sets; likewise, the numbers of qualified features under six test sets were analyzed.

RESULTS

There were 20 and 23 qualified features in the reproducibility and location-stability experiments, respectively. In the reproducibility experiment, the performance of the peripheral groups and high-concentration sets was significantly better than the center groups and low-concentration sets. The effect of concentration on the location-stability of features was not monotonic, and the number of qualified features in the low-concentration sets was greater than that in the high-concentration sets. No features were qualified in both experiments.

CONCLUSIONS

The density and location of the target object can affect the number of reproducible radiomic features, and its density can also affect the number of location-stable radiomic features. The problem of feature reliability should be treated cautiously in radiomic research on CBCT.

Identification of reproducible radiomic features from on-board volumetric images: A multi-institutional phantom study

BACKGROUND

Radiomics analysis using on-board volumetric images has attracted research attention as a method for predicting prognosis during treatment; however, the lack of standardization is still one of the main concerns.

PURPOSE

This study investigated the factors that influence the reproducibility of radiomic features extracted from on-board volumetric images using an anthropomorphic radiomics phantom. Furthermore, a phantom experiment was conducted with different treatment machines from multiple institutions as external validation to identify reproducible radiomic features.

METHODS

The phantom was designed to be 35 × 20 × 20 cm with eight types of heterogeneous spheres (⌀ = 1, 2, and 3 cm). On-board volumetric images were acquired using 15 treatment machines from eight institutions. Of these, kilovoltage cone-beam computed tomography (kV-CBCT) image data acquired from four treatment machines at one institution were used as an internal evaluation dataset to explore the reproducibility of radiomic features. The remaining image data, including kV-CBCT, megavoltage-CBCT (MV-CBCT), and megavoltage computed tomography (MV-CT) provided by seven different institutions (11 treatment machines), were used as an external validation dataset. A total of 1,302 radiomic features, including 18 first-order, 75 texture, 465 (i.e., 93 × 5) Laplacian of Gaussian (LoG) filter-based, and 744 (i.e., 93 × 8) wavelet filter-based features, were extracted within the spheres. The intraclass correlation coefficient (ICC) was calculated to explore feature repeatability and reproducibility using an internal evaluation dataset. Subsequently, the coefficient of variation (COV) was calculated to validate the feature variability of external institutions. An absolute ICC exceeding 0.85 or COV under 5% was considered indicative of a highly reproducible feature.

RESULTS

For internal evaluation, ICC analysis showed that the median percentage of radiomic features with high repeatability was 95.2%. The ICC analysis indicated that the median percentages of highly reproducible features for inter-tube current, reconstruction algorithm, and treatment machine were decreased by 20.8%, 29.2%, and 33.3%, respectively. For external validation, the COV analysis showed that the median percentage of reproducible features was 31.5%. A total of 16 features, including nine LoG filter-based and seven wavelet filter-based features, were indicated as highly reproducible features. The gray-level run-length matrix (GLRLM) was classified as containing the most frequent features (N = 8), followed by the gray-level dependence matrix (N = 7) and gray-level co-occurrence matrix (N = 1) features.

CONCLUSIONS

We developed the standard phantom for radiomics analysis of kV-CBCT, MV-CBCT, and MV-CT images. With this phantom, we revealed that the differences in the treatment machine and image reconstruction algorithm reduce the reproducibility of radiomic features from on-board volumetric images. Specifically, the most reproducible features for external validation were LoG or wavelet filter-based GLRLM features. However, the acceptability of the identified features should be examined in advance at each institution before applying the findings to prognosis prediction.

Development and optimisation of a preclinical cone beam computed tomography-based radiomics workflow for radiation oncology research

Background and purpose

Radiomics features derived from medical images have the potential to act as imaging biomarkers to improve diagnosis and predict treatment response in oncology. However, the complex relationships between radiomics features and the biological characteristics of tumours are yet to be fully determined. In this study, we developed a preclinical cone beam computed tomography (CBCT) radiomics workflow with the aim to use in vivo models to further develop radiomics signatures.

Materials and methods

CBCT scans of a mouse phantom were acquired using onboard imaging from a small animal radiotherapy research platform (SARRP, Xstrahl). The repeatability and reproducibility of radiomics outputs were compared across different imaging protocols, segmentation sizes, pre-processing parameters and materials. Robust features were identified and used to compare scans of two xenograft mouse tumour models (A549 and H460).

Results

Changes to the radiomics workflow significantly impact feature robustness. Preclinical CBCT radiomics analysis is feasible with 119 stable features identified from scans imaged at 60 kV, 25 bin width and 0.26 mm slice thickness. Large variation in segmentation volumes reduced the number of reliable radiomics features for analysis. Standardization in imaging and analysis parameters is essential in preclinical radiomics analysis to improve accuracy of outputs, leading to more consistent and reproducible findings.

Conclusions

We present the first optimised workflow for preclinical CBCT radiomics to identify imaging biomarkers. Preclinical radiomics has the potential to maximise the quantity of data captured in in vivo experiments and could provide key information supporting the wider application of radiomics.

Current Applications of Deep Learning and Radiomics on CT and CBCT for Maxillofacial Diseases

The increasing use of computed tomography (CT) and cone beam computed tomography (CBCT) in oral and maxillofacial imaging has driven the development of deep learning and radiomics applications to assist clinicians in early diagnosis, accurate prognosis prediction, and efficient treatment planning of maxillofacial diseases. This narrative review aimed to provide an up-to-date overview of the current applications of deep learning and radiomics on CT and CBCT for the diagnosis and management of maxillofacial diseases. Based on current evidence, a wide range of deep learning models on CT/CBCT images have been developed for automatic diagnosis, segmentation, and classification of jaw cysts and tumors, cervical lymph node metastasis, salivary gland diseases, temporomandibular (TMJ) disorders, maxillary sinus pathologies, mandibular fractures, and dentomaxillofacial deformities, while CT-/CBCT-derived radiomics applications mainly focused on occult lymph node metastasis in patients with oral cancer, malignant salivary gland tumors, and TMJ osteoarthritis. Most of these models showed high performance, and some of them even outperformed human experts. The models with performance on par with human experts have the potential to serve as clinically practicable tools to achieve the earliest possible diagnosis and treatment, leading to a more precise and personalized approach for the management of maxillofacial diseases. Challenges and issues, including the lack of the generalizability and explainability of deep learning models and the uncertainty in the reproducibility and stability of radiomic features, should be overcome to gain the trust of patients, providers, and healthcare organizers for daily clinical use of these models.

Enhancing the stability of CT radiomics across different volume of interest sizes using parametric feature maps: a phantom study

BACKGROUND

In radiomics studies, differences in the volume of interest (VOI) are often inevitable and may confound the extracted features. We aimed to correct this confounding effect of VOI variability by applying parametric maps with a fixed voxel size.

METHODS

Ten scans of a cup filled with sodium chloride solution were scanned using a multislice computed tomography (CT) unit. Sphere-shaped VOIs with different diameters (4, 8, or 16 mm) were drawn centrally into the phantom. A total of 93 features were extracted conventionally from the original images using PyRadiomics. Using a self-designed and pretested software tool, parametric maps for the same 93 features with a fixed voxel size of 4 mm3 were created. To retrieve the feature values from the maps, VOIs were copied from the original images to preserve the position. Differences in feature quantities between the VOI sizes were tested with the Mann-Whitney U-test and agreement with overall concordance correlation coefficients (OCCC).

RESULTS

Fifty-five conventionally extracted features were significantly different between the VOI sizes, and none of the features showed excellent agreement in terms of OCCCs. When read from the parametric maps, only 8 features showed significant differences, and 3 features showed an excellent OCCC (≥ 0.85). The OCCCs for 89 features substantially increased using the parametric maps.

CONCLUSIONS

This phantom study shows that converting CT images into parametric maps resolves the confounding effect of VOI variability and increases feature reproducibility across VOI sizes.