Radiomics for the prediction of a postoperative pancreatic fistula following a pancreatoduodenectomy: A systematic review and radiomic score quality assessment

BACKGROUND

Postoperative pancreatic fistula (POPF) is a severe complication following a pancreatoduodenectomy. An accurate prediction of POPF could assist the surgeon in offering tailor-made treatment decisions. The use of radiomic features has been introduced to predict POPF. A systematic review was conducted to evaluate the performance of models predicting POPF using radiomic features and to systematically evaluate the methodological quality.

METHODS

Studies with patients undergoing a pancreatoduodenectomy and radiomics analysis on computed tomography or magnetic resonance imaging were included. Methodological quality was assessed using the Radiomics Quality Score (RQS) and Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement.

RESULTS

Seven studies were included in this systematic review, comprising 1300 patients, of whom 364 patients (28 %) developed POPF. The area under the curve (AUC) of the included studies ranged from 0.76 to 0.95. Only one study externally validated the model, showing an AUC of 0.89 on this dataset. Overall adherence to the RQS (31 %) and TRIPOD guidelines (54 %) was poor.

CONCLUSION

This systematic review showed that high predictive power was reported of studies using radiomic features to predict POPF. However, the quality of most studies was poor. Future studies need to standardize the methodology.

REGISTRATION

not registered.

Current status of magnetic resonance imaging radiomics in hepatocellular carcinoma: A quantitative review with Radiomics Quality Score

BACKGROUND

Radiomics is a promising tool that may increase the value of magnetic resonance imaging (MRI) for different tasks related to the management of patients with hepatocellular carcinoma (HCC). However, its implementation in clinical practice is still far, with many issues related to the methodological quality of radiomic studies.

AIM

To systematically review the current status of MRI radiomic studies concerning HCC using the Radiomics Quality Score (RQS).

METHODS

A systematic literature search of PubMed, Google Scholar, and Web of Science databases was performed to identify original articles focusing on the use of MRI radiomics for HCC management published between 2017 and 2023. The methodological quality of radiomic studies was assessed using the RQS tool. Spearman's correlation (ρ) analysis was performed to explore if RQS was correlated with journal metrics and characteristics of the studies. The level of statistical signi-ficance was set at P < 0.05.

RESULTS

One hundred and twenty-seven articles were included, of which 43 focused on HCC prognosis, 39 on prediction of pathological findings, 16 on prediction of the expression of molecular markers outcomes, 18 had a diagnostic purpose, and 11 had multiple purposes. The mean RQS was 8 ± 6.22, and the corresponding percentage was 24.15% ± 15.25% (ranging from 0.0% to 58.33%). RQS was positively correlated with journal impact factor (IF; ρ = 0.36, P = 2.98 × 10-5), 5-years IF (ρ = 0.33, P = 1.56 × 10-4), number of patients included in the study (ρ = 0.51, P < 9.37 × 10-10) and number of radiomics features extracted in the study (ρ = 0.59, P < 4.59 × 10-13), and time of publication (ρ = -0.23, P < 0.0072).

CONCLUSION

Although MRI radiomics in HCC represents a promising tool to develop adequate personalized treatment as a noninvasive approach in HCC patients, our study revealed that studies in this field still lack the quality required to allow its introduction into clinical practice.

Delta-radiomics in cancer immunotherapy response prediction: A systematic review

Background

The new immunotherapies have not only changed the oncological therapeutic approach but have also made it necessary to develop new imaging methods for assessing the response to treatment. Delta radiomics consists of the analysis of radiomic features variation between different medical images, usually before and after therapy.

Purpose

This review aims to evaluate the role of delta radiomics in the immunotherapy response assessment.

Methods

A systematic search was performed in PubMed, Scopus, and Web Of Science using "delta radiomics AND immunotherapy" as search terms. The included articles' methodological quality was measured using the Radiomics Quality Score (RQS) tool.

Results

Thirteen articles were finally included in the systematic review. Overall, the RQS of the included studies ranged from 4 to 17, with a mean RQS total of 11,15 ± 4,18 with a corresponding percentage of 30.98 ± 11.61 %. Eleven articles out of 13 performed imaging at multiple time points. All the included articles performed feature reduction. No study carried out prospective validation, decision curve analysis, or cost-effectiveness analysis.

Conclusions

Delta radiomics has been demonstrated useful in evaluating the response in oncologic patients undergoing immunotherapy. The overall quality was found law, due to the lack of prospective design and external validation. Thus, further efforts are needed to bring delta radiomics a step closer to clinical implementation.

Radiomics in Carotid Plaque: A Systematic Review and Radiomics Quality Score Assessment

Imaging modalities provide information on plaque morphology and vulnerability; however, they are operator dependent and miss a great deal of microscopic information. Recently, many radiomics models for carotid plaque that identify unstable plaques and predict cardiovascular outcomes have been proposed. This systematic review was aimed at assessing whether radiomics is a reliable and reproducible method for the clinical prediction of carotid plaque. A systematic search was conducted to identify studies published in PubMed and Cochrane library from January 1, 2001, to September 30, 2022. Both retrospective and prospective studies that developed and/or validated machine learning models based on radiomics data to classify or predict carotid plaques were included. The general characteristics of each included study were selected, and the methodological quality of radiomics reports and risk of bias were evaluated using the radiomics quality score (RQS) tool and Quality Assessment of Diagnostic Accuracy Studies-2, respectively. Two investigators independently reviewed each study, and the consensus data were used for analysis. A total of 2429 patients from 16 studies were included. The mean area under the curve of radiomics models for diagnostic or predictive performance of the included studies was 0.88 ± 0.02, with a range of 0.741-0.989. The mean RQS was 9.25 (standard deviation: 6.04), representing 25.7% of the possible maximum value of 36, whereas the lowest point was -2, and the highest score was 22. Radiomics models have revealed additional information on patients with carotid plaque, but with respect to methodological quality, radiomics reports are still in their infancy, and many hurdles need to be overcome.

What benefit can be obtained from magnetic resonance imaging diagnosis with artificial intelligence in prostate cancer compared with clinical assessments?

The present study aimed to explore the potential of artificial intelligence (AI) methodology based on magnetic resonance (MR) images to aid in the management of prostate cancer (PCa). To this end, we reviewed and summarized the studies comparing the diagnostic and predictive performance for PCa between AI and common clinical assessment methods based on MR images and/or clinical characteristics, thereby investigating whether AI methods are generally superior to common clinical assessment methods for the diagnosis and prediction fields of PCa. First, we found that, in the included studies of the present study, AI methods were generally equal to or better than the clinical assessment methods for the risk assessment of PCa, such as risk stratification of prostate lesions and the prediction of therapeutic outcomes or PCa progression. In particular, for the diagnosis of clinically significant PCa, the AI methods achieved a higher summary receiver operator characteristic curve (SROC-AUC) than that of the clinical assessment methods (0.87 vs. 0.82). For the prediction of adverse pathology, the AI methods also achieved a higher SROC-AUC than that of the clinical assessment methods (0.86 vs. 0.75). Second, as revealed by the radiomics quality score (RQS), the studies included in the present study presented a relatively high total average RQS of 15.2 (11.0-20.0). Further, the scores of the individual RQS elements implied that the AI models in these studies were constructed with relatively perfect and standard radiomics processes, but the exact generalizability and clinical practicality of the AI models should be further validated using higher levels of evidence, such as prospective studies and open-testing datasets.

Quality assessment of stroke radiomics studies: Promoting clinical application

PURPOSE

To evaluate the quality of radiomics studies on stroke using a radiomics quality score (RQS), Minimum Information for Medial AI reporting (MINIMAR) and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) to promote clinical application.

METHODS

PubMed MEDLINE and Embase were searched to identify radiomics studies on stroke. Of 464 articles, 52 relevant original research articles were included. The RQS, MINIMAR and TRIPOD were scored to evaluate the quality of the studies by neuroradiologists.

RESULTS

Only four studies (7.7 %) performed external validation. The mean RQS was 3.2 of 36 (8.9 %), and the basic adherence rate was 24.9 %. The adherence rate was low for conducting phantom study (1.9 %), stating comparison to 'gold standard' (1.9 %), offering potential clinical utility (13.5 %) and performing cost-effectiveness analysis (1.9 %). None of the studies performed a test-retest, stated biologic correlation, conducted prospective studies, or opened codes and data to the public, resulting in low RQS. The total MINIMAR adherence rate was 47.4 %. The overall adherence rate for TRIPOD was 54.6 %, with low scores for reporting the title (2.0 %), key elements of the study setting (6.1 %), and explaining the sample size (2.0 %).

CONCLUSIONS

The overall radiomics reporting quality and reporting of published radiomics studies on stoke was suboptimal. More thorough validation and open data are needed to increase clinical applicability of radiomics studies.

Systematic review of the radiomics quality score applications: an EuSoMII Radiomics Auditing Group Initiative

OBJECTIVE

The main aim of the present systematic review was a comprehensive overview of the Radiomics Quality Score (RQS)-based systematic reviews to highlight common issues and challenges of radiomics research application and evaluate the relationship between RQS and review features.

METHODS

The literature search was performed on multiple medical literature archives according to PRISMA guidelines for systematic reviews that reported radiomic quality assessment through the RQS. Reported scores were converted to a 0-100% scale. The Mann-Whitney and Kruskal-Wallis tests were used to compare RQS scores and review features.

RESULTS

The literature research yielded 345 articles, from which 44 systematic reviews were finally included in the analysis. Overall, the median of RQS was 21.00% (IQR = 11.50). No significant differences of RQS were observed in subgroup analyses according to targets (oncological/not oncological target, neuroradiology/body imaging focus and one imaging technique/more than one imaging technique, characterization/prognosis/detection/other).

CONCLUSIONS

Our review did not reveal a significant difference of quality of radiomic articles reported in systematic reviews, divided in different subgroups. Furthermore, low overall methodological quality of radiomics research was found independent of specific application domains. While the RQS can serve as a reference tool to improve future study designs, future research should also be aimed at improving its reliability and developing new tools to meet an ever-evolving research space.

KEY POINTS

• Radiomics is a promising high-throughput method that may generate novel imaging biomarkers to improve clinical decision-making process, but it is an inherently complex analysis and often lacks reproducibility and generalizability. • The Radiomics Quality Score serves a necessary role as the de facto reference tool for assessing radiomics studies. • External auditing of radiomics studies, in addition to the standard peer-review process, is valuable to highlight common limitations and provide insights to improve future study designs and practical applicability of the radiomics models.

Quality of Radiomics Research on Brain Metastasis: A Roadmap to Promote Clinical Translation

Objective

Our study aimed to evaluate the quality of radiomics studies on brain metastases based on the radiomics quality score (RQS), Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) checklist, and the Image Biomarker Standardization Initiative (IBSI) guidelines.

Materials and Methods

PubMed MEDLINE, and EMBASE were searched for articles on radiomics for evaluating brain metastases, published until February 2021. Of the 572 articles, 29 relevant original research articles were included and evaluated according to the RQS, TRIPOD checklist, and IBSI guidelines.

Results

External validation was performed in only three studies (10.3%). The median RQS was 3.0 (range, -6 to 12), with a low basic adherence rate of 50.0%. The adherence rate was low in comparison to the “gold standard” (10.3%), stating the potential clinical utility (10.3%), performing the cut-off analysis (3.4%), reporting calibration statistics (6.9%), and providing open science and data (3.4%). None of the studies involved test-retest or phantom studies, prospective studies, or cost-effectiveness analyses. The overall rate of adherence to the TRIPOD checklist was 60.3% and low for reporting title (3.4%), blind assessment of outcome (0%), description of the handling of missing data (0%), and presentation of the full prediction model (0%). The majority of studies lacked pre-processing steps, with bias-field correction, isovoxel resampling, skull stripping, and gray-level discretization performed in only six (20.7%), nine (31.0%), four (3.8%), and four (13.8%) studies, respectively.

Conclusion

The overall scientific and reporting quality of radiomics studies on brain metastases published during the study period was insufficient. Radiomics studies should adhere to the RQS, TRIPOD, and IBSI guidelines to facilitate the translation of radiomics into the clinical field.

Quality assessment of meningioma radiomics studies: Bridging the gap between exploratory research and clinical applications

PURPOSE

To evaluate the quality of radiomics studies on meningiomas, using a radiomics quality score (RQS), Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD), and the Image Biomarker Standardization Initiative (IBSI).

METHODS

PubMed MEDLINE and Embase were searched to identify radiomics studies on meningiomas. Of 138 identified articles, 25 relevant original research articles were included. Studies were scored according to the RQS, TRIPOD guidelines, and items in IBSI.

RESULTS

Only four studies (16 %) performed external validation. The mean RQS was 5.6 out of 36 (15.4 %), and the basic adherence rate was 26.8 %. The adherence rate was low for stating biological correlation (4%), conducting calibration statistics (12 %), multiple segmentation (16 %), and stating potential clinical utility (16 %). None of the studies conducted a test‒retest or phantom study, stated a comparison to a 'gold standard', conducted prospective studies or cost-effectivity analysis, or opened code and data to the public, resulting in low RQS. The overall adherence rate for TRIPOD was 54.1 %, with low scores for reporting the title (4%), abstract (0%), blind assessment of the outcome (8%), and explaining the sample size (0%). According to IBSI items, only 6 (24 %), 6 (24 %), and 3 (12 %) studies performed N4 bias-field correction, isovoxel resampling, and grey-level discretization, respectively. No study performed skull stripping.

CONCLUSION

The quality of radiomics studies for meningioma is insufficient. Acknowledgement of RQS, TRIPOD, and IBSI reporting guidelines may improve the quality of meningioma radiomics studies and enable their clinical application.

Prostate MRI radiomics: A systematic review and radiomic quality score assessment

BACKGROUND

Radiomics have the potential to further increase the value of MRI in prostate cancer management. However, implementation in clinical practice is still far and concerns have been raised regarding the methodological quality of radiomic studies. Therefore, we aimed to systematically review the literature to assess the quality of prostate MRI radiomic studies using the radiomics quality score (RQS).

METHODS

Multiple medical literature archives (PubMed, Web of Science and EMBASE) were searched to retrieve original investigations focused on prostate MRI radiomic approaches up to the end of June 2019. Three researchers independently assessed each paper using the RQS. Data from the most experienced researcher were used for descriptive analysis. Inter-rater reproducibility was assessed using the intraclass correlation coefficient (ICC) on the total RQS score.

RESULTS

73 studies were included in the analysis. Overall, the average RQS total score was 7.93 ± 5.13 on a maximum of 36 points, with a final average percentage of 23 ± 13%. Among the most critical items, the lack of feature robustness testing strategies and external validation datasets. The ICC resulted poor to moderate, with an average value of 0.57 and 95% Confidence Intervals between 0.44 and 0.69.

CONCLUSIONS

Current studies on prostate MRI radiomics still lack the quality required to allow their introduction in clinical practice.