Effects of alterations in positron emission tomography imaging parameters on radiomics features

Impact of Preprocessing Parameters in Medical Imaging-Based Radiomic Studies: A Systematic Review

BACKGROUND

Lately, radiomic studies featuring the development of a signature to use in prediction models in diagnosis or prognosis outcomes have been increasingly published. While the results are shown to be promising, these studies still have many pitfalls and limitations. One of the main issues of these studies is that radiomic features depend on how the images are preprocessed before their computation. Since, in widely known and used software for radiomic features calculation, it is possible to set these preprocessing parameters before the calculation of the radiomic feature, there are ongoing studies assessing the stability and repeatability of radiomic features to find the most suitable preprocessing parameters for every used imaging modality.

MATERIALS AND METHODS

We performed a comprehensive literature search using four electronic databases: PubMed, Cochrane Library, Embase, and Scopus. Mesh terms and free text were modeled in search strategies for databases. The inclusion criteria were studies where preprocessing parameters' influence on feature values and model predictions was addressed. Records lacking information on image acquisition parameters were excluded, and any eligible studies with full-text versions were included in the review process, while conference proceedings and monographs were disregarded. We used the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies 2) tool to investigate the risk of bias. We synthesized our data in a table divided by the imaging modalities subgroups.

RESULTS

After applying the inclusion and exclusion criteria, we selected 43 works. This review examines the impact of preprocessing parameters on the reproducibility and reliability of radiomic features extracted from multimodality imaging (CT, MRI, CBCT, and PET/CT). Standardized preprocessing is crucial for consistent radiomic feature extraction. Key preprocessing steps include voxel resampling, normalization, and discretization, which influence feature robustness and reproducibility. In total, 44% of the included works studied the effects of an isotropic voxel resampling, and most studies opted to employ a discretization strategy. From 2021, several studies started selecting the best set of preprocessing parameters based on models' best performance. As for comparison metrics, ICC was the most used in MRI studies in 58% of the screened works.

CONCLUSIONS

From our work, we highlighted the need to harmonize the use of preprocessing parameters and their values, especially in light of future studies of prospective studies, which are still lacking in the current literature.

Impact of 18F-FDG PET Intensity Normalization on Radiomic Features of Oropharyngeal Squamous Cell Carcinomas and Machine Learning-Generated Biomarkers

We aimed to investigate the effects of 18F-FDG PET voxel intensity normalization on radiomic features of oropharyngeal squamous cell carcinoma (OPSCC) and machine learning-generated radiomic biomarkers. Methods: We extracted 1,037 18F-FDG PET radiomic features quantifying the shape, intensity, and texture of 430 OPSCC primary tumors. The reproducibility of individual features across 3 intensity-normalized images (body-weight SUV, reference tissue activity ratio to lentiform nucleus of brain and cerebellum) and the raw PET data was assessed using an intraclass correlation coefficient (ICC). We investigated the effects of intensity normalization on the features' utility in predicting the human papillomavirus (HPV) status of OPSCCs in univariate logistic regression, receiver-operating-characteristic analysis, and extreme-gradient-boosting (XGBoost) machine-learning classifiers. Results: Of 1,037 features, a high (ICC ≥ 0.90), medium (0.90 > ICC ≥ 0.75), and low (ICC < 0.75) degree of reproducibility across normalization methods was attained in 356 (34.3%), 608 (58.6%), and 73 (7%) features, respectively. In univariate analysis, features from the PET normalized to the lentiform nucleus had the strongest association with HPV status, with 865 of 1,037 (83.4%) significant features after multiple testing corrections and a median area under the receiver-operating-characteristic curve (AUC) of 0.65 (interquartile range, 0.62-0.68). Similar tendencies were observed in XGBoost models, with the lentiform nucleus-normalized model achieving the numerically highest average AUC of 0.72 (SD, 0.07) in the cross validation within the training cohort. The model generalized well to the validation cohorts, attaining an AUC of 0.73 (95% CI, 0.60-0.85) in independent validation and 0.76 (95% CI, 0.58-0.95) in external validation. The AUCs of the XGBoost models were not significantly different. Conclusion: Only one third of the features demonstrated a high degree of reproducibility across intensity-normalization techniques, making uniform normalization a prerequisite for interindividual comparability of radiomic markers. The choice of normalization technique may affect the radiomic features' predictive value with respect to HPV. Our results show trends that normalization to the lentiform nucleus may improve model performance, although more evidence is needed to draw a firm conclusion.

Brain metastases from NSCLC treated with stereotactic radiotherapy: prediction mismatch between two different radiomic platforms

BACKGROUND AND PURPOSE

Radiomics enables the mining of quantitative features from medical images. The influence of the radiomic feature extraction software on the final performance of models is still a poorly understood topic. This study aimed to investigate the ability of radiomic features extracted by two different radiomic platforms to predict clinical outcomes in patients treated with radiosurgery for brain metastases from non-small cell lung cancer. We developed models integrating pre-treatment magnetic resonance imaging (MRI)-derived radiomic features and clinical data.

MATERIALS AND METHODS

Pre-radiotherapy gadolinium enhanced axial T1-weighted MRI scans were used. MRI images were re-sampled, intensity-shifted, and histogram-matched before radiomic extraction by means of two different platforms (PyRadiomics and SOPHiA Radiomics). We adopted LASSO Cox regression models for multivariable analyses by creating radiomic, clinical, and combined models using three survival clinical endpoints (local control, distant progression, and overall survival). The statistical analysis was repeated 50 times with different random seeds and the median concordance index was used as performance metric of the models.

RESULTS

We analysed 276 metastases from 148 patients. The use of the two platforms resulted in differences in both the quality and the number of extractable features. That led to mismatches in terms of end-to-end performance, statistical significance of radiomic scores, and clinical covariates found significant in combined models.

CONCLUSION

This study shed new light on how extracting radiomic features from the same images using two different platforms could yield several discrepancies. That may lead to acute consequences on drawing conclusions, comparing results across the literature, and translating radiomics into clinical practice.

Impact of image quality on radiomics applications

Radiomics features extracted from medical images have been widely reported to be useful in the patient specific outcome modeling for variety of assessment and prediction purposes. Successful application of radiomics features as imaging biomarkers, however, is dependent on the robustness of the approach to the variation in each step of the modeling workflow. Variation in the input image quality is one of the main sources that impacts the reproducibility of radiomics analysis when a model is applied to broader range of medical imaging data. The quality of medical image is generally affected by both the scanner related factors such as image acquisition/reconstruction settings and the patient related factors such as patient motion. This article aimed to review the published literatures in this field that reported the impact of various imaging factors on the radiomics features through the change in image quality. The literatures were categorized by different imaging modalities and also tabulated based on the imaging parameters and the class of radiomics features included in the study. Strategies for image quality standardization were discussed based on the relevant literatures and recommendations for reducing the impact of image quality variation on the radiomics in multi-institutional clinical trial were summarized at the end of this article.

[18F]FDG-PET/CT radiomics and artificial intelligence in lung cancer: Technical aspects and potential clinical applications

Lung cancer is the second most common cancer and the leading cause of cancer-related death worldwide. Molecular imaging using [¹⁸F]fluorodeoxyglucose Positron Emission Tomography and/or Computed Tomography ([¹⁸F]FDG-PET/CT) plays an essential role in the diagnosis, evaluation of response to treatment, and prediction of outcomes. The images are evaluated using qualitative and conventional quantitative indices. However, there is far more information embedded in the images, which can be extracted by sophisticated algorithms. Recently, the concept of uncovering and analyzing the invisible data extracted from medical images, called radiomics, is gaining more attention. Currently, [¹⁸F]FDG-PET/CT radiomics is growingly evaluated in lung cancer to discover if it enhances the diagnostic performance or implication of [¹⁸F]FDG-PET/CT in the management of lung cancer. In this review, we provide a short overview of the technical aspects, as they are discussed in different articles of this special issue. We mainly focus on the diagnostic performance of the [¹⁸F]FDG-PET/CT-based radiomics and the role of artificial intelligence in non-small cell lung cancer, impacting the early detection, staging, prediction of tumor subtypes, biomarkers, and patient's outcomes.

A systematic review and quality of reporting checklist for repeatability and reproducibility of radiomic features

•

Main factors impacting feature stability: Image acquisition, reconstruction, tumor segmentation, and interpolation.

•

Textural features are less robust than morphological or statistical features.

•

A checklist is provided including items that should be reported in a radiomic study.

Purpose

Although quantitative image biomarkers (radiomics) show promising value for cancer diagnosis, prognosis, and treatment assessment, these biomarkers still lack reproducibility. In this systematic review, we aimed to assess the progress in radiomics reproducibility and repeatability in the recent years.

Methods and materials

Four hundred fifty-one abstracts were retrieved according to the original PubMed search pattern with the publication dates ranging from 2017/05/01 to 2020/12/01. Each abstract including the keywords was independently screened by four observers. Forty-two full-text articles were selected for further analysis. Patient population data, radiomic feature classes, feature extraction software, image preprocessing, and reproducibility results were extracted from each article. To support the community with a standardized reporting strategy, we propose a specific reporting checklist to evaluate the feasibility to reproduce each study.

Results

Many studies continue to under-report essential reproducibility information: all but one clinical and all but two phantom studies missed to report at least one important item reporting image acquisition. The studies included in this review indicate that all radiomic features are sensitive to image acquisition, reconstruction, tumor segmentation, and interpolation. However, the amount of sensitivity is feature dependent, for instance, textural features were, in general, less robust than statistical features.

Conclusions

Radiomics repeatability, reproducibility, and reporting quality can substantially be improved regarding feature extraction software and settings, image preprocessing and acquisition, cutoff values for stable feature selection. Our proposed radiomics reporting checklist can serve to simplify and improve the reporting and, eventually, guarantee the possibility to fully replicate and validate radiomic studies.

Radiomics feature reliability assessed by intraclass correlation coefficient: a systematic review

Radiomics research is rapidly growing in recent years, but more concerns on radiomics reliability are also raised. This review attempts to update and overview the current status of radiomics reliability research in the ever expanding medical literature from the perspective of a single reliability metric of intraclass correlation coefficient (ICC). To conduct this systematic review, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed. After literature search and selection, a total of 481 radiomics studies using CT, PET, or MRI, covering a wide range of subject and disease types, were included for review. In these highly heterogeneous studies, feature reliability to image segmentation was much more investigated than reliability to other factors, such as image acquisition, reconstruction, post-processing, and feature quantification. The reported ICCs also suggested high radiomics feature reliability to image segmentation. Image acquisition was found to introduce much more feature variability than image segmentation, in particular for MRI, based on the reported ICC values. Image post-processing and feature quantification yielded different levels of radiomics reliability and might be used to mitigate image acquisition-induced variability. Some common flaws and pitfalls in ICC use were identified, and suggestions on better ICC use were given. Due to the extremely high study heterogeneities and possible risks of bias, the degree of radiomics feature reliability that has been achieved could not yet be safely synthesized or derived in this review. More future researches on radiomics reliability are warranted.

Making Radiomics More Reproducible across Scanner and Imaging Protocol Variations: A Review of Harmonization Methods

Radiomics converts medical images into mineable data via a high-throughput extraction of quantitative features used for clinical decision support. However, these radiomic features are susceptible to variation across scanners, acquisition protocols, and reconstruction settings. Various investigations have assessed the reproducibility and validation of radiomic features across these discrepancies. In this narrative review, we combine systematic keyword searches with prior domain knowledge to discuss various harmonization solutions to make the radiomic features more reproducible across various scanners and protocol settings. Different harmonization solutions are discussed and divided into two main categories: image domain and feature domain. The image domain category comprises methods such as the standardization of image acquisition, post-processing of raw sensor-level image data, data augmentation techniques, and style transfer. The feature domain category consists of methods such as the identification of reproducible features and normalization techniques such as statistical normalization, intensity harmonization, ComBat and its derivatives, and normalization using deep learning. We also reflect upon the importance of deep learning solutions for addressing variability across multi-centric radiomic studies especially using generative adversarial networks (GANs), neural style transfer (NST) techniques, or a combination of both. We cover a broader range of methods especially GANs and NST methods in more detail than previous reviews.

Preselection of robust radiomic features does not improve outcome modelling in non-small cell lung cancer based on clinical routine FDG-PET imaging

Background

Radiomics is a promising tool for identifying imaging-based biomarkers. Radiomics-based models are often trained on single-institution datasets; however, multi-centre imaging datasets are preferred for external generalizability owing to the influence of inter-institutional scanning differences and acquisition settings. The study aim was to determine the value of preselection of robust radiomic features in routine clinical positron emission tomography (PET) images to predict clinical outcomes in locally advanced non-small cell lung cancer (NSCLC).

Methods

A total of 1404 primary tumour radiomic features were extracted from pre-treatment [¹⁸F]fluorodeoxyglucose (FDG)-PET scans of stage IIIA/N2 or IIIB NSCLC patients using a training cohort (n = 79; prospective Swiss multi-centre randomized phase III trial SAKK 16/00; 16 centres) and an internal validation cohort (n = 31; single centre). Robustness studies investigating delineation variation, attenuation correction and motion were performed (intraclass correlation coefficient threshold > 0.9). Two 12-/24-month event-free survival (EFS) and overall survival (OS) logistic regression models were trained using standardized imaging: (1) with robust features alone and (2) with all available features. Models were then validated using fivefold cross-validation, and validation on a separate single-centre dataset. Model performance was assessed using area under the receiver operating characteristic curve (AUC).

Results

Robustness studies identified 179 stable features (13%), with 25% stable features for 3D versus 4D acquisition, 31% for attenuation correction and 78% for delineation. Univariable analysis found no significant robust features predicting 12-/24-month EFS and 12-month OS (p value > 0.076). Prognostic models without robust preselection performed well for 12-month EFS in training (AUC = 0.73) and validation (AUC = 0.74). Patient stratification into two risk groups based on 12-month EFS was significant for training (p value = 0.02) and validation cohorts (p value = 0.03).

Conclusions

A PET-based radiomics model using a standardized, multi-centre dataset to predict EFS in locally advanced NSCLC was successfully established and validated with good performance. Prediction models with robust feature preselection were unsuccessful, indicating the need for a standardized imaging protocol.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13550-021-00809-3.

Radiogenomics in brain, breast, and lung cancer: opportunities and challenges

The field of radiogenomics largely focuses on developing imaging surrogates for genomic signatures and integrating imaging, genomic, and molecular data to develop combined personalized biomarkers for characterizing various diseases. Our study aims to highlight the current state-of-the-art and the role of radiogenomics in cancer research, focusing mainly on solid tumors, and is broadly divided into four sections. The first section reviews representative studies that establish the biologic basis of radiomic signatures using gene expression and molecular profiling information. The second section includes studies that aim to non-invasively predict molecular subtypes of tumors using radiomic signatures. The third section reviews studies that evaluate the potential to augment the performance of established prognostic signatures by combining complementary information encoded by radiomic and genomic signatures derived from cancer tumors. The fourth section includes studies that focus on ascertaining the biological significance of radiomic phenotypes. We conclude by discussing current challenges and opportunities in the field, such as the importance of coordination between imaging device manufacturers, regulatory organizations, health care providers, pharmaceutical companies, academic institutions, and physicians for the effective standardization of the results from radiogenomic signatures and for the potential use of these findings to improve precision care for cancer patients.

Standardization of imaging methods for machine learning in neuro-oncology

Radiomics is a novel technique in which quantitative phenotypes or features are extracted from medical images. Machine learning enables analysis of large quantities of medical imaging data generated by radiomic feature extraction. A growing number of studies based on these methods have developed tools for neuro-oncology applications. Despite the initial promises, many of these imaging tools remain far from clinical implementation. One major limitation hindering the use of these models is their lack of reproducibility when applied across different institutions and clinical settings. In this article, we discuss the importance of standardization of methodology and reporting in our effort to improve reproducibility. Ongoing efforts of standardization for neuro-oncological imaging are reviewed. Challenges related to standardization and potential disadvantages in over-standardization are also described. Ultimately, greater multi-institutional collaborative effort is needed to provide and implement standards for data acquisition and analysis methods to facilitate research results to be interoperable and reliable for integration into different practice environments.

Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform

OBJECTIVE

To investigate the effects of Image Biomarker Standardisation Initiative (IBSI) compliance, harmonisation of calculation settings and platform version on the statistical reliability of radiomic features and their corresponding ability to predict clinical outcome.

METHODS

The statistical reliability of radiomic features was assessed retrospectively in three clinical datasets (patient numbers: 108 head and neck cancer, 37 small-cell lung cancer, 47 non-small-cell lung cancer). Features were calculated using four platforms (PyRadiomics, LIFEx, CERR and IBEX). PyRadiomics, LIFEx and CERR are IBSI-compliant, whereas IBEX is not. The effects of IBSI compliance, user-defined calculation settings and platform version were assessed by calculating intraclass correlation coefficients and confidence intervals. The influence of platform choice on the relationship between radiomic biomarkers and survival was evaluated using univariable cox regression in the largest dataset.

RESULTS

The reliability of radiomic features calculated by the different software platforms was only excellent (ICC > 0.9) for 4/17 radiomic features when comparing all four platforms. Reliability improved to ICC > 0.9 for 15/17 radiomic features when analysis was restricted to the three IBSI-compliant platforms. Failure to harmonise calculation settings resulted in poor reliability, even across the IBSI-compliant platforms. Software platform version also had a marked effect on feature reliability in CERR and LIFEx. Features identified as having significant relationship to survival varied between platforms, as did the direction of hazard ratios.

CONCLUSION

IBSI compliance, user-defined calculation settings and choice of platform version all influence the statistical reliability and corresponding performance of prognostic models in radiomics.

KEY POINTS

• Reliability of radiomic features varies between feature calculation platforms and with choice of software version. • Image Biomarker Standardisation Initiative (IBSI) compliance improves reliability of radiomic features across platforms, but only when calculation settings are harmonised. • IBSI compliance, user-defined calculation settings and choice of platform version collectively affect the prognostic value of features.