Quality Reporting of Radiomics Analysis in Mild Cognitive Impairment and Alzheimer's Disease: A Roadmap for Moving Forward

Noninvasive Assessment of Vascular Endothelial Growth Factor and Prognosis in Gastric Cancer Through Radiomic Features

INTRODUCTION

Gastric cancer (GC) is a leading cause of cancer-related deaths worldwide, with delayed diagnosis often limiting effective treatment options. This study introduces a novel, noninvasive radiomics-based approach using [18F] FDG PET/CT (fluorodeoxyglucose positron emission tomography/computed tomography) to predict vascular endothelial growth factor (VEGF) status and survival in patients with GC. The ability to noninvasively assess these parameters can significantly influence therapeutic decisions and outcomes.

METHODS

We conducted a retrospective study involving patients diagnosed with GC, stratified into training, validation, and test groups. Each patient underwent a [18F] FDG PET/CT scan, and radiomic features were extracted using dedicated software. A Radiomics Score (RS) was calculated, serving as a predictor for VEGF status. Statistical analyses included logistic regression and Cox proportional hazards models to evaluate the predictive power of RS on survival outcomes.

RESULTS

The developed radiomics model demonstrated high predictive accuracy, with the RS formula achieving an area under the receiver operating characteristic curve of 0.861 in the training cohort and 0.857 in the validation cohort for predicting VEGF status. The model also identified RS as an independent prognostic factor for survival, where higher RS values correlated with poorer survival rates.

DISCUSSION

The findings underscore the potential of [18F] FDG PET/CT radiomics in transforming the management of GC by providing a noninvasive means to assess tumor aggressiveness and prognosis through VEGF status. This model could facilitate earlier and more tailored therapeutic interventions, potentially improving survival outcomes in a disease marked by typically late diagnosis and limited treatment success.

Support Vector Machine for Stratification of Cognitive Impairment Using 3D T1WI in Patients with Type 2 Diabetes Mellitus

Purpose

To explore the potential of MRI-based radiomics in predicting cognitive dysfunction in patients with diagnosed type 2 diabetes mellitus (T2DM).

Patients and Methods

In this study, data on 158 patients with T2DM were retrospectively collected between September 2019 and December 2020. The participants were categorized into a normal cognitive function (N) group (n=30), a mild cognitive impairment (MCI) group (n=90), and a dementia (DM) group (n=38) according to the Chinese version of the Montréal Cognitive Assessment Scale-B (MoCA-B). Radiomics features were extracted from the brain tissue except ventricles and sulci in the 3D T1WI images, support vector machine (SVM) model was then established to identify the CI and N groups, and the MCI and DM groups, respectively. The models were evaluated based on their area under the receiver operating characteristic curve (AUC), Precision (P), Recall rate (Recall, R), F1-score, and Support. Finally, ROC curves were plotted for each model.

Results

The study consisted of 68 cases in the N and CI group, with 54 cases in the training set and 14 in the verification set, and 128 cases were included in the MCI and DM groups, with 90 training sets and 38 verification sets. The consistency for inter-group and intra-group of radiomics features in two physicians were 0.86 and 0.90, respectively. After features selection, there were 11 optimal features to distinguish N and CI and 12 optimal features to MCI and DM. In the test set, the AUC for the SVM classifier was 0.857 and the accuracy was 0.830 in distinguishing CI and N, while AUC was 0.821 and the accuracy was 0.830 in distinguishing MCI and DM.

Conclusion

The SVM model based on MRI radiomics exhibits high efficacy in the diagnosis of cognitive dysfunction and evaluation of its severity among patients with T2DM.

Quality assessment of the MRI-radiomics studies for MGMT promoter methylation prediction in glioma: a systematic review and meta-analysis

OBJECTIVES

To evaluate the methodological quality and diagnostic accuracy of MRI-based radiomic studies predicting O6-methylguanine-DNA methyltransferase (MGMT) promoter methylation status in gliomas.

METHODS

PubMed Medline, EMBASE, and Web of Science were searched to identify MRI-based radiomic studies on MGMT methylation in gliomas published until December 31, 2022. Three raters evaluated the study methodological quality with Radiomics Quality Score (RQS, 16 components) and Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis Or Diagnosis (TRIPOD, 22 items) scales. Risk of bias and applicability concerns were assessed with QUADAS-2 tool. A meta-analysis was performed to estimate the pooled area under the curve (AUC) and to assess inter-study heterogeneity.

RESULTS

We included 26 studies, published from 2016. The median RQS total score was 8 out of 36 (22%, range 8-44%). Thirteen studies performed external validation. All studies reported AUC or accuracy, but only 4 (15%) performed calibration and decision curve analysis. No studies performed phantom analysis, cost-effectiveness analysis, and prospective validation. The overall TRIPOD adherence score was between 50% and 70% in 16 studies and below 50% in 10 studies. The pooled AUC was 0.78 (95% CI, 0.73-0.83, I2 = 94.1%) with a high inter-study heterogeneity. Studies with external validation and including only WHO-grade IV gliomas had significantly lower AUC values (0.65; 95% CI, 0.57-0.73, p < 0.01).

CONCLUSIONS

Study RQS and adherence to TRIPOD guidelines was generally low. Radiomic prediction of MGMT methylation status showed great heterogeneity of results and lower performances in grade IV gliomas, which hinders its current implementation in clinical practice.

CLINICAL RELEVANCE STATEMENT

MGMT promoter methylation status appears to be variably correlated with MRI radiomic features; radiomic models are not sufficiently robust to be integrated into clinical practice to accurately predict MGMT promoter methylation status in patients with glioma before surgery.

KEY POINTS

• Adherence to the indications of TRIPOD guidelines was generally low, as was RQS total score. • MGMT promoter methylation status prediction with MRI radiomic features provided heterogeneous diagnostic accuracy results across studies. • Studies that included grade IV glioma only and performed external validation had significantly lower diagnostic accuracy than others.

Development and validation of an automatic classification algorithm for the diagnosis of Alzheimer's disease using a high-performance interpretable deep learning network

OBJECTIVES

To develop and validate an automatic classification algorithm for diagnosing Alzheimer's disease (AD) or mild cognitive impairment (MCI).

METHODS AND MATERIALS

This study evaluated a high-performance interpretable network algorithm (TabNet) and compared its performance with that of XGBoost, a widely used classifier. Brain segmentation was performed using a commercially approved software. TabNet and XGBoost were trained on the volumes or radiomics features of 102 segmented regions for classifying subjects into AD, MCI, or cognitively normal (CN) groups. The diagnostic performances of the two algorithms were compared using areas under the curves (AUCs). Additionally, 20 deep learning-based AD signature areas were investigated.

RESULTS

Between December 2014 and March 2017, 161 AD, 153 MCI, and 306 CN cases were enrolled. Another 120 AD, 90 MCI, and 141 CN cases were included for the internal validation. Public datasets were used for external validation. TabNet with volume features had an AUC of 0.951 (95% confidence interval [CI], 0.947-0.955) for AD vs CN, which was similar to that of XGBoost (0.953 [95% CI, 0.951-0.955], p = 0.41). External validation revealed the similar performances of two classifiers using volume features (0.871 vs. 0.871, p = 0.86). Likewise, two algorithms showed similar performances with one another in classifying MCI. The addition of radiomics data did not improve the performance of TabNet. TabNet and XGBoost focused on the same 13/20 regions of interest, including the hippocampus, inferior lateral ventricle, and entorhinal cortex.

CONCLUSIONS

TabNet shows high performance in AD classification and detailed interpretation of the selected regions.

CLINICAL RELEVANCE STATEMENT

Using a high-performance interpretable deep learning network, the automatic classification algorithm assisted in accurate Alzheimer's disease detection using 3D T1-weighted brain MRI and detailed interpretation of the selected regions.

KEY POINTS

• MR volumetry data revealed that TabNet had a high diagnostic performance in differentiating Alzheimer's disease (AD) from cognitive normal cases, which was comparable with that of XGBoost. • The addition of radiomics data to the volume data did not improve the diagnostic performance of TabNet. • Both TabNet and XGBoost selected the clinically meaningful regions of interest in AD, including the hippocampus, inferior lateral ventricle, and entorhinal cortex.

18F-FDG-PET Radiomics Based on White Matter Predicts The Progression of Mild Cognitive Impairment to Alzheimer Disease: A Machine Learning Study

RATIONALE AND OBJECTIVES

To build a model using white-matter radiomics features on positron-emission tomography (PET) and machine learning methods to predict progression from mild cognitive impairment (MCI) to Alzheimer disease (AD).

MATERIALS AND METHODS

We analyzed the data of 341 MCI patients from the Alzheimer's Disease Neuroimaging Initiative, of whom 102 progressed to AD during an 8-year follow-up. The patients were divided into the training (238 patients) and test groups (103 patients). PET-based radiomics features were extracted from the white matter in the training group, and dimensionally reduced to construct a psychoradiomics signature (PS), which was combined with multimodal data using machine learning methods to construct an integrated model. Model performance was evaluated using receiver operating characteristic curves in the test group.

RESULTS

Clinical Dementia Rating (CDR) scores, Alzheimer's Disease Assessment Scale (ADAS) scores, and PS independently predicted MCI progression to AD on multivariate logistic regression. The areas under the curve (AUCs) of the CDR, ADAS and PS in the training and test groups were 0.683, 0.755, 0.747 and 0.737, 0.743, 0.719 respectively, and were combined using a support vector machine to construct an integrated model. The AUC of the integrated model in the training and test groups was 0.868 and 0.865, respectively (sensitivity, 0.873 and 0.839, respectively; specificity, 0.784 and 0.806, respectively). The AUCs of the integrated model significantly differed from those of other predictors in both groups (p < 0.05, Delong test).

CONCLUSION

Our psych radiomics signature based on white-matter PET data predicted MCI progression to AD. The integrated model built using multimodal data and machine learning identified MCI patients at a high risk of progression to AD.

Longitudinal changes in hippocampal texture from healthy aging to Alzheimer's disease

Early detection of Alzheimer's disease is essential to develop preventive treatment strategies. Detectible change in brain volume emerges relatively late in the pathogenic progression of disease, but microstructural changes caused by early neuropathology may cause subtle changes in the MR signal, quantifiable using texture analysis. Texture analysis quantifies spatial patterns in an image, such as smoothness, randomness and heterogeneity. We investigated whether the MRI texture of the hippocampus, an early site of Alzheimer's disease pathology, is sensitive to changes in brain microstructure before the onset of cognitive impairment. We also explored the longitudinal trajectories of hippocampal texture across the Alzheimer's continuum in relation to hippocampal volume and other biomarkers. Finally, we assessed the ability of texture to predict future cognitive decline, over and above hippocampal volume. Data were acquired from the Alzheimer's Disease Neuroimaging Initiative. Texture was calculated for bilateral hippocampi on 3T T1-weighted MRI scans. Two hundred and ninety-three texture features were reduced to five principal components that described 88% of total variance within cognitively unimpaired participants. We assessed cross-sectional differences in these texture components and hippocampal volume between four diagnostic groups: cognitively unimpaired amyloid-β- (n = 406); cognitively unimpaired amyloid-β+ (n = 213); mild cognitive impairment amyloid-β+ (n = 347); and Alzheimer's disease dementia amyloid-β+ (n = 202). To assess longitudinal texture change across the Alzheimer's continuum, we used a multivariate mixed-effects spline model to calculate a 'disease time' for all timepoints based on amyloid PET and cognitive scores. This was used as a scale on which to compare the trajectories of biomarkers, including volume and texture of the hippocampus. The trajectories were modelled in a subset of the data: cognitively unimpaired amyloid-β- (n = 345); cognitively unimpaired amyloid-β+ (n = 173); mild cognitive impairment amyloid-β+ (n = 301); and Alzheimer's disease dementia amyloid-β+ (n = 161). We identified a difference in texture component 4 at the earliest stage of Alzheimer's disease, between cognitively unimpaired amyloid-β- and cognitively unimpaired amyloid-β+ older adults (Cohen's d = 0.23, Padj = 0.014). Differences in additional texture components and hippocampal volume emerged later in the disease continuum alongside the onset of cognitive impairment (d = 0.30-1.22, Padj < 0.002). Longitudinal modelling of the texture trajectories revealed that, while most elements of texture developed over the course of the disease, noise reduced sensitivity for tracking individual textural change over time. Critically, however, texture provided additional information than was provided by volume alone to more accurately predict future cognitive change (d = 0.32-0.63, Padj < 0.0001). Our results support the use of texture as a measure of brain health, sensitive to Alzheimer's disease pathology, at a time when therapeutic intervention may be most effective.

Systematic review of the radiomics quality score applications: an EuSoMII Radiomics Auditing Group Initiative

OBJECTIVE

The main aim of the present systematic review was a comprehensive overview of the Radiomics Quality Score (RQS)-based systematic reviews to highlight common issues and challenges of radiomics research application and evaluate the relationship between RQS and review features.

METHODS

The literature search was performed on multiple medical literature archives according to PRISMA guidelines for systematic reviews that reported radiomic quality assessment through the RQS. Reported scores were converted to a 0-100% scale. The Mann-Whitney and Kruskal-Wallis tests were used to compare RQS scores and review features.

RESULTS

The literature research yielded 345 articles, from which 44 systematic reviews were finally included in the analysis. Overall, the median of RQS was 21.00% (IQR = 11.50). No significant differences of RQS were observed in subgroup analyses according to targets (oncological/not oncological target, neuroradiology/body imaging focus and one imaging technique/more than one imaging technique, characterization/prognosis/detection/other).

CONCLUSIONS

Our review did not reveal a significant difference of quality of radiomic articles reported in systematic reviews, divided in different subgroups. Furthermore, low overall methodological quality of radiomics research was found independent of specific application domains. While the RQS can serve as a reference tool to improve future study designs, future research should also be aimed at improving its reliability and developing new tools to meet an ever-evolving research space.

KEY POINTS

• Radiomics is a promising high-throughput method that may generate novel imaging biomarkers to improve clinical decision-making process, but it is an inherently complex analysis and often lacks reproducibility and generalizability. • The Radiomics Quality Score serves a necessary role as the de facto reference tool for assessing radiomics studies. • External auditing of radiomics studies, in addition to the standard peer-review process, is valuable to highlight common limitations and provide insights to improve future study designs and practical applicability of the radiomics models.

Based on Tau PET Radiomics Analysis for the Classification of Alzheimer's Disease and Mild Cognitive Impairment

Alzheimer's Disease (AD) and Mild Cognitive Impairment (MCI) are closely associated with Tau proteins accumulation. In this study, we aimed to implement radiomics analysis to discover high-order features from pathological biomarker and improve the classification accuracy based on Tau PET images. Two cross-racial independent cohorts from the ADNI database (121 AD patients, 197 MCI patients and 211 normal control (NC) subjects) and Huashan hospital (44 AD patients, 33 MCI patients and 36 NC subjects) were enrolled. The radiomics features of Tau PET imaging of AD related brain regions were computed for classification using a support vector machine (SVM) model. The radiomics model was trained and validated in the ADNI cohort and tested in the Huashan hospital cohort. The standard uptake value ratio (SUVR) and clinical scores model were also performed to compared with radiomics analysis. Additionally, we explored the possibility of using Tau PET radiomics features as a good biomarker to make binary identification of Tau-negative MCI versus Tau-positive MCI or apolipoprotein E (ApoE) ε4 carrier versus ApoE ε4 non-carrier. We found that the radiomics model demonstrated best classification performance in differentiating AD/MCI patients and NC in comparison to SUVR and clinical scores models, with an accuracy of 84.8 ± 4.5%, 73.1 ± 3.6% in the ANDI cohort. Moreover, the radiomics model also demonstrated greater performance in diagnosing AD than other methods in the Huashan hospital cohort, with an accuracy of 81.9 ± 6.1%. In addition, the radiomics model also showed the satisfactory classification performance in the MCI-tau subgroup experiment (72.3 ± 3.5%, 71.9 ± 3.6% and 63.7 ± 5.9%) and in the MCI-ApoE subgroup experiment (73.5 ± 4.3%, 70.1 ± 3.9% and 62.5 ± 5.4%). In conclusion, our study showed that based on Tau PET radiomics analysis has the potential to guide and facilitate clinical diagnosis, further providing evidence for identifying the risk factors in MCI patients.

The effect of hippocampal radiomic features and functional connectivity on the relationship between hippocampal volume and cognitive function in Alzheimer's disease

Hippocampal volume is associated with cognitive function in Alzheimer's disease (AD). Hippocampal radiomic features and resting-state functional connectivity (rs-FC) are promising biomarkers and correlate with AD pathology. However, few studies have been conducted on how hippocampal biomarkers affect the cognition-structure relationship. Therefore, we aimed to investigate the effects of hippocampal radiomic features and resting-state functional connectivity (rs-FC) on this relationship in AD. We enrolled 70 AD patients and 65 healthy controls (HCs). The FreeSurfer software was used to measure hippocampal volume. We selected hippocampal radiomic features to build a model to distinguish AD patients from HCs and used a seed-based approach to calculate the hippocampal rs-FC. Furthermore, we conducted mediation and moderation analyses to investigate the effect of hippocampal radiomic features and rs-FC on the relationship between hippocampal volume and cognition in AD. The results suggested that hippocampal radiomic features mediated the association between bilateral hippocampal volume and cognition in AD. Additionally, patients with AD showed weaker rs-FC between the bilateral hippocampus and right ventral posterior cingulate cortex and stronger rs-FC between the left hippocampus and left insula than HCs. The rs-FC between the hippocampus and insula moderated the relationship between hippocampal volume and cognition in AD, suggesting that this rs-FC could exacerbate or ameliorate the effects of hippocampal volume on cognition and may be essential in improving cognitive function in AD. Our findings may not only expand existing biological knowledge of the interrelationships among hippocampal biomarkers and cognition but also provide potential targets for treatment strategies for AD.

Quality reporting of radiomics analysis in pituitary adenomas: promoting clinical translation

OBJECTIVE

To evaluate the quality of radiomics studies on pituitary adenoma according to the radiomics quality score (RQS) and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD).

METHODS

PubMed MEDLINE and EMBASE were searched to identify radiomics studies on pituitary adenomas. From 138 articles, 20 relevant original research articles were included. Studies were scored based on RQS and TRIPOD guidelines.

RESULTS

Most included studies did not perform pre-processing; isovoxel resampling, signal intensity normalization, and N4 bias field correction were performed in only five (25%), eight (40%), and four (20%) studies, respectively. Only two (10%) studies performed external validation. The mean RQS and basic adherence rate were 2.8 (7.6%) and 26.6%, respectively. There was a low adherence rate for conducting comparison to "gold-standard" (20%), multiple segmentation (25%), and stating potential clinical utility (25%). No study stated the biological correlation, conducted a test-retest or phantom study, was a prospective study, conducted cost-effectiveness analysis, or provided open-source code and data, which resulted in low-level evidence. The overall adherence rate for TRIPOD was 54.6%, and it was low for reporting the title (5%), abstract (0%), explaining the sample size (10%), and suggesting a full prediction model (5%).

CONCLUSION

The radiomics reporting quality for pituitary adenoma is insufficient. Pre-processing is required for feature reproducibility and external validation is necessary. Feature reproducibility, clinical utility demonstration, higher evidence levels, and open science are required. Titles, abstracts, and full prediction model suggestions should be improved for transparent reporting.

ADVANCES IN KNOWLEDGE

Despite the rapidly increasing number of radiomics researches on pituitary adenoma, the quality of science in these researches is unknown. Our study indicates that the overall quality needs to be significantly improved in radiomics studies on pituitary adenoma, and since the concept of RQS and IBSI is still unfamiliar to clinicians and radiologist researchers, our study may help to reach higher technical and clinical impact in the future study.

Gray and white matter structural examination for diagnosis of major depressive disorder and subthreshold depression in adolescents and young adults: a preliminary radiomics analysis

BACKGROUND

Radiomics is an emerging image analysis framework that provides more details than conventional methods. In present study, we aimed to identify structural radiomics features of gray matter (GM) and white matter (WM), and to develop and validate the classification model for major depressive disorder (MDD) and subthreshold depression (StD) diagnosis using radiomics analysis.

METHODS

A consecutive cohort of 142 adolescents and young adults, including 43 cases with MDD, 49 cases with StD and 50 healthy controls (HC), were recruited and underwent the three-dimensional T1 weighted imaging (3D-T1WI) and diffusion tensor imaging (DTI). We extracted radiomics features representing the shape and diffusion properties of GM and WM from all participants. Then, an all-relevant feature selection process embedded in a 10-fold cross-validation framework was used to identify features with significant power for discrimination. Random forest classifiers (RFC) were established and evaluated successively using identified features.

RESULTS

The results showed that a total of 3030 features were extracted after preprocessing, including 2262 shape-related features from each T1-weighted image representing GM morphometry and 768 features from each DTI representing the diffusion properties of WM. 25 features were selected ultimately, including ten features for MDD versus HC, eight features for StD versus HC, and seven features for MDD versus StD. The accuracies and area under curve (AUC) the RFC achieved were 86.75%, 0.93 for distinguishing MDD from HC with significant radiomics features located in the left medial orbitofrontal cortex, right superior and middle temporal regions, right anterior cingulate, left cuneus and hippocampus, 70.51%, 0.69 for discriminating StD from HC within left cuneus, medial orbitofrontal cortex, cerebellar vermis, hippocampus, anterior cingulate and amygdala, right superior and middle temporal regions, and 59.15%, 0.66 for differentiating MDD from StD within left medial orbitofrontal cortex, middle temporal and cuneus, right superior frontal, superior temporal regions and hippocampus, anterior cingulate, respectively.

CONCLUSION

These findings provide preliminary evidence that radiomics features of brain structure are valid for discriminating MDD and StD subjects from healthy controls. The MRI-based radiomics approach, with further improvement and validation, might be a potential facilitating method to clinical diagnosis of MDD or StD.

Radiomics quality score in renal masses: a systematic assessment on current literature

OBJECTIVE

To perform a systematic assessment and analyze the quality of radiomics methodology in current literature in the evaluation of renal masses using the Radiomics Quality Score (RQS) approach.

METHODS

We systematically reviewed recent radiomics literature in renal masses published in PubMed, EMBASE, Elsevier, and Web of Science. Two reviewers blinded by each other's scores evaluated the quality of radiomics methodology in studies published from 2015 to August 2021 using the RQS approach. Owing to the diversity in the imaging modalities and radiomics applications, a meta-analysis could not be performed.

RESULTS

Based on our inclusion/exclusion criteria, a total of 87 published studies were included in our study. The highest RQS was noted in three categories: reporting of clinical utility, gold standard, and feature reduction. The average RQS of the two reviewers ranged from 5 ≤ RQS≤19, with the maximum attainable RQS being 36. Very few (7/87 i.e., 8%) studies received an average RQS that ranged from 17 < RQS≤19, which represents studies with the highest RQS in our study. Many (39/87 i.e., 45%) studies received an average RQS that ranged from 13 < RQS≤15. No significant interreviewer scoring differences were observed.

CONCLUSIONS

We report that the overall scientific quality and reporting of radiomics studies in renal masses is suboptimal, and subsequent studies should bolster current deficiencies to improve reporting of radiomics methodologies.

ADVANCES IN KNOWLEDGE

The RQS approach is a meaningful quantitative scoring system to assess radiomics methodology quality and supports a comprehensive evaluation of the radiomics approach before its incorporation into clinical practice.

Placenta Accreta Spectrum Disorders and Radiomics: Systematic review and quality appraisal

PURPOSE

Ultrasound and magnetic resonance imaging are the imaging modalities of choice for placenta accrete spectrum (PAS) disorders assessment. Radiomics could further increase the value of medical images and allow to overcome the limitations linked to their visual assessment. Aim of this systematic review was to identify and appraise the methodological quality of radiomics studies focused PAS disorders applications.

METHOD

Three online databases (PubMed, Scopus and Web of Science) were searched to identify original research articles on human subjects published in English. For the qualitative synthesis of results, data regarding study design (e.g., retrospective or prospective), purpose, patient population (e.g., sample size), imaging modalities and radiomics pipelines (e.g., segmentation and feature extraction strategy) were collected. The appraisal of methodological quality was performed using the Radiomics Quality Score (RQS).

RESULTS

10 articles were finally included and analyzed. All were retrospective and MRI-powered. The majority included more than 100 patients (6/10). Four were prognostic (focused on either the prediction of bleeding volume or the prediction of needed management) while six diagnostic (PAS vs not PAS classification) studies. The median RQS was 8, with maximum and minimum respectively equal to 17/36 and - 6/36. Major methodological concerns were the lack of feature stability to multiple segmentation testing and poor data openness.

CONCLUSIONS

Radiomics studies focused on PAS disorders showed a heterogeneous methodological quality, overall lower than desirable. Furthermore, many relevant research questions remain unexplored. More robust investigations are needed to foster advancements in the field and possibly clinical translation.

An updated systematic review of radiomics in osteosarcoma: utilizing CLAIM to adapt the increasing trend of deep learning application in radiomics

Objective

To update the systematic review of radiomics in osteosarcoma.

Methods

PubMed, Embase, Web of Science, China National Knowledge Infrastructure, and Wanfang Data were searched to identify articles on osteosarcoma radiomics until May 15, 2022. The studies were assessed by Radiomics Quality Score (RQS), Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement, Checklist for Artificial Intelligence in Medical Imaging (CLAIM), and modified Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool. The evidence supporting radiomics application for osteosarcoma was rated according to meta-analysis results.

Results

Twenty-nine articles were included. The average of the ideal percentage of RQS, the TRIPOD adherence rate and the CLAIM adherence rate were 29.2%, 59.2%, and 63.7%, respectively. RQS identified a radiomics-specific issue of phantom study. TRIPOD addressed deficiency in blindness of assessment. CLAIM and TRIPOD both pointed out shortness in missing data handling and sample size or power calculation. CLAIM identified extra disadvantages in data de-identification and failure analysis. External validation and open science were emphasized by all the above three tools. The risk of bias and applicability concerns were mainly related to the index test. The meta-analysis of radiomics predicting neoadjuvant chemotherapy response by MRI presented a diagnostic odds ratio (95% confidence interval) of 28.83 (10.27–80.95) on testing datasets and was rated as weak evidence.

Conclusions

The quality of osteosarcoma radiomics studies is insufficient. More investigation is needed before using radiomics to optimize osteosarcoma treatment. CLAIM is recommended to guide the design and reporting of radiomics research.

A systematic review of radiomics in pancreatitis: applying the evidence level rating tool for promoting clinical transferability

Background

Multiple tools have been applied to radiomics evaluation, while evidence rating tools for this field are still lacking. This study aims to assess the quality of pancreatitis radiomics research and test the feasibility of the evidence level rating tool.

Results

Thirty studies were included after a systematic search of pancreatitis radiomics studies until February 28, 2022, via five databases. Twenty-four studies employed radiomics for diagnostic purposes. The mean ± standard deviation of the adherence rate was 38.3 ± 13.3%, 61.3 ± 11.9%, and 37.1 ± 27.2% for the Radiomics Quality Score (RQS), the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) checklist, and the Image Biomarker Standardization Initiative (IBSI) guideline for preprocessing steps, respectively. The median (range) of RQS was 7.0 (− 3.0 to 18.0). The risk of bias and application concerns were mainly related to the index test according to the modified Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool. The meta-analysis on differential diagnosis of autoimmune pancreatitis versus pancreatic cancer by CT and mass-forming pancreatitis versus pancreatic cancer by MRI showed diagnostic odds ratios (95% confidence intervals) of, respectively, 189.63 (79.65–451.48) and 135.70 (36.17–509.13), both rated as weak evidence mainly due to the insufficient sample size.

Conclusions

More research on prognosis of acute pancreatitis is encouraged. The current pancreatitis radiomics studies have insufficient quality and share common scientific disadvantages. The evidence level rating is feasible and necessary for bringing the field of radiomics from preclinical research area to clinical stage.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13244-022-01279-4.

A Pipeline for the Implementation and Visualization of Explainable Machine Learning for Medical Imaging Using Radiomics Features

Machine learning (ML) models have been shown to predict the presence of clinical factors from medical imaging with remarkable accuracy. However, these complex models can be difficult to interpret and are often criticized as “black boxes”. Prediction models that provide no insight into how their predictions are obtained are difficult to trust for making important clinical decisions, such as medical diagnoses or treatment. Explainable machine learning (XML) methods, such as Shapley values, have made it possible to explain the behavior of ML algorithms and to identify which predictors contribute most to a prediction. Incorporating XML methods into medical software tools has the potential to increase trust in ML-powered predictions and aid physicians in making medical decisions. Specifically, in the field of medical imaging analysis the most used methods for explaining deep learning-based model predictions are saliency maps that highlight important areas of an image. However, they do not provide a straightforward interpretation of which qualities of an image area are important. Here, we describe a novel pipeline for XML imaging that uses radiomics data and Shapley values as tools to explain outcome predictions from complex prediction models built with medical imaging with well-defined predictors. We present a visualization of XML imaging results in a clinician-focused dashboard that can be generalized to various settings. We demonstrate the use of this workflow for developing and explaining a prediction model using MRI data from glioma patients to predict a genetic mutation.

A predictive model using the mesoscopic architecture of the living brain to detect Alzheimer's disease

Background

Alzheimer's disease, the most common cause of dementia, causes a progressive and irreversible deterioration of cognition that can sometimes be difficult to diagnose, leading to suboptimal patient care.

Methods

We developed a predictive model that computes multi-regional statistical morpho-functional mesoscopic traits from T1-weighted MRI scans, with or without cognitive scores. For each patient, a biomarker called "Alzheimer's Predictive Vector" (ApV) was derived using a two-stage least absolute shrinkage and selection operator (LASSO).

Results

The ApV reliably discriminates between people with (ADrp) and without (nADrp) Alzheimer's related pathologies (98% and 81% accuracy between ADrp - including the early form, mild cognitive impairment - and nADrp in internal and external hold-out test sets, respectively), without any a priori assumptions or need for neuroradiology reads. The new test is superior to standard hippocampal atrophy (26% accuracy) and cerebrospinal fluid beta amyloid measure (62% accuracy). A multiparametric analysis compared DTI-MRI derived fractional anisotropy, whose readout of neuronal loss agrees with ADrp phenotype, and SNPrs2075650 is significantly altered in patients with ADrp-like phenotype.

Conclusions

This new data analytic method demonstrates potential for increasing accuracy of Alzheimer diagnosis.

Quality assessment of radiomics research in cardiac CT: a systematic review

OBJECTIVES

To assess the quality of current radiomics research on cardiac CT using radiomics quality score (RQS) and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) systems.

METHODS

Systematic searches of PubMed and EMBASE were performed to identify all potentially relevant original research articles about cardiac CT radiomics. Fifteen original research articles were selected. Two cardiac radiologists assessed the quality of the methodology adopted in those studies according to the RQS and TRIPOD guidelines. Basic adherence rates for the following six key domains were evaluated: image protocol and reproducibility, feature reduction and validation, biologic/clinical utility, performance index, high level of evidence, and open science.

RESULTS

Among the 15 included articles, six (40%) were about coronary artery disease and six (40%) were about myocardial infarction. The mean RQS was 9.9 ± 7.3 (27.4% of the ideal score of 36), and the basic adherence rate was 44.6%. Fourteen (93.3%) and nine (60%) studies performed feature selection and validation, but only two (13.3%) of them performed external validation. Two studies (13.3%) were prospective, and only one study (6.7%) conducted calibration analysis and stated the potential clinical utility. None of the studies conducted phantom study and cost-effective analysis. The overall adherence rate for TRIPOD was 63%.

CONCLUSION

The quality of radiomics studies in cardiac CT is currently insufficient. A higher level of evidence is required, and analysis of clinical utility and calibration of model performance need to be improved.

KEY POINTS

• The quality of science of radiomics studies in cardiac CT is currently insufficient. • No study conducted a phantom study or cost-effective analysis, with further limitations being demonstrated in a high level of evidence for radiomics studies. • Analysis of clinical utility and calibration of model performance need to be improved, and a higher level of evidence is required.

Quality of science and reporting for radiomics in cardiac magnetic resonance imaging studies: a systematic review

OBJECTIVES

To evaluate the quality of radiomics studies using cardiac magnetic resonance imaging (CMR) according to the radiomics quality score (RQS), Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines, and the standards defined by the Image Biomarker Standardization Initiative (IBSI) and identify areas needing improvement.

MATERIALS AND METHODS

PubMed and Embase were searched to identify radiomics studies using CMR until March 10, 2021. Of the 259 identified articles, 32 relevant original research articles were included. Studies were scored according to the RQS, TRIPOD guidelines, and IBSI standards by two cardiac radiologists.

RESULTS

The mean RQS was 14.3% of the maximum (5.16 out of 36). RQS were low for the demonstration of validation (-60.6%), calibration statistics (1.6%), potential clinical utility (3.1%), and open science (3.1%) items. No study conducted a phantom study or cost-effectiveness analysis. The adherence to TRIPOD guidelines was 55.9%. Studies were deficient in reporting title (3.1%), stating objective in abstract and introduction (6.3% and 9.4%), missing data (0%), discrimination/calibration (3.1%), and how to use the prediction model (3.1%). According to the IBSI standards, non-uniformity correction, image interpolation, grey-level discretization, and signal intensity normalization were performed in two (6.3%), four (12.5%), six (18.8%), and twelve (37.5%) studies, respectively.

CONCLUSION

The quality of radiomics studies using CMR is suboptimal. Improvements are needed in the areas of validation, calibration, clinical utility, and open science. Complete reporting of study objectives, missing data, discrimination/calibration, how to use the prediction model, and preprocessing steps are necessary.

KEY POINTS

• The quality of science in radiomics studies using CMR is currently inadequate. • RQS were low for validation, calibration, clinical utility, and open science; no study conducted a phantom study or cost-effectiveness analysis. • In stating the study objective, missing data, discrimination/calibration, how to use the prediction model, and preprocessing steps, improvements are needed.

Quality of Radiomics Research on Brain Metastasis: A Roadmap to Promote Clinical Translation

Objective

Our study aimed to evaluate the quality of radiomics studies on brain metastases based on the radiomics quality score (RQS), Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) checklist, and the Image Biomarker Standardization Initiative (IBSI) guidelines.

Materials and Methods

PubMed MEDLINE, and EMBASE were searched for articles on radiomics for evaluating brain metastases, published until February 2021. Of the 572 articles, 29 relevant original research articles were included and evaluated according to the RQS, TRIPOD checklist, and IBSI guidelines.

Results

External validation was performed in only three studies (10.3%). The median RQS was 3.0 (range, -6 to 12), with a low basic adherence rate of 50.0%. The adherence rate was low in comparison to the “gold standard” (10.3%), stating the potential clinical utility (10.3%), performing the cut-off analysis (3.4%), reporting calibration statistics (6.9%), and providing open science and data (3.4%). None of the studies involved test-retest or phantom studies, prospective studies, or cost-effectiveness analyses. The overall rate of adherence to the TRIPOD checklist was 60.3% and low for reporting title (3.4%), blind assessment of outcome (0%), description of the handling of missing data (0%), and presentation of the full prediction model (0%). The majority of studies lacked pre-processing steps, with bias-field correction, isovoxel resampling, skull stripping, and gray-level discretization performed in only six (20.7%), nine (31.0%), four (3.8%), and four (13.8%) studies, respectively.

Conclusion

The overall scientific and reporting quality of radiomics studies on brain metastases published during the study period was insufficient. Radiomics studies should adhere to the RQS, TRIPOD, and IBSI guidelines to facilitate the translation of radiomics into the clinical field.

An interpretable multiparametric radiomics model for the diagnosis of schizophrenia using magnetic resonance imaging of the corpus callosum

There is a growing need to develop novel strategies for the diagnosis of schizophrenia using neuroimaging biomarkers. We investigated the robustness of the diagnostic model for schizophrenia using radiomic features from T1-weighted and diffusion tensor images of the corpus callosum (CC). A total of 165 participants [86 schizophrenia and 79 healthy controls (HCs)] were allocated to training (N = 115) and test (N = 50) sets. Radiomic features of the CC subregions were extracted from T1-weighted, apparent diffusion coefficient (ADC), and fractional anisotropy (FA) images (N = 1605). Following feature selection, various combinations of classifiers were trained, and Bayesian optimization was adopted in the best performing classifier. Discrimination, calibration, and clinical utility of the model were assessed. An online calculator was constructed to offer the probability of having schizophrenia. SHapley Additive exPlanations (SHAP) was applied to explore the interpretability of the model. We identified 30 radiomic features to differentiate participants with schizophrenia from HCs. The Bayesian optimized model achieved the highest performance, with an area under the curve (AUC), accuracy, sensitivity, and specificity of 0.89 (95% confidence interval: 0.81-0.98), 80.0, 83.3, and 76.9%, respectively, in the test set. The final model offers clinical probability in an online calculator. The model explanation by SHAP suggested that second-order features from the posterior CC were highly associated with the risk of schizophrenia. The multiparametric radiomics model focusing on the CC shows its robustness for the diagnosis of schizophrenia. Radiomic features could be a potential source of biomarkers that support the biomarker-based diagnosis of schizophrenia and improve the understanding of its neurobiology.

Radiomics machine learning study with a small sample size: Single random training-test set split may lead to unreliable results

This study aims to determine how randomly splitting a dataset into training and test sets affects the estimated performance of a machine learning model and its gap from the test performance under different conditions, using real-world brain tumor radiomics data. We conducted two classification tasks of different difficulty levels with magnetic resonance imaging (MRI) radiomics features: (1) "Simple" task, glioblastomas [n = 109] vs. brain metastasis [n = 58] and (2) "difficult" task, low- [n = 163] vs. high-grade [n = 95] meningiomas. Additionally, two undersampled datasets were created by randomly sampling 50% from these datasets. We performed random training-test set splitting for each dataset repeatedly to create 1,000 different training-test set pairs. For each dataset pair, the least absolute shrinkage and selection operator model was trained and evaluated using various validation methods in the training set, and tested in the test set, using the area under the curve (AUC) as an evaluation metric. The AUCs in training and testing varied among different training-test set pairs, especially with the undersampled datasets and the difficult task. The mean (±standard deviation) AUC difference between training and testing was 0.039 (±0.032) for the simple task without undersampling and 0.092 (±0.071) for the difficult task with undersampling. In a training-test set pair with the difficult task without undersampling, for example, the AUC was high in training but much lower in testing (0.882 and 0.667, respectively); in another dataset pair with the same task, however, the AUC was low in training but much higher in testing (0.709 and 0.911, respectively). When the AUC discrepancy between training and test, or generalization gap, was large, none of the validation methods helped sufficiently reduce the generalization gap. Our results suggest that machine learning after a single random training-test set split may lead to unreliable results in radiomics studies especially with small sample sizes.

Quality assessment of meningioma radiomics studies: Bridging the gap between exploratory research and clinical applications

PURPOSE

To evaluate the quality of radiomics studies on meningiomas, using a radiomics quality score (RQS), Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD), and the Image Biomarker Standardization Initiative (IBSI).

METHODS

PubMed MEDLINE and Embase were searched to identify radiomics studies on meningiomas. Of 138 identified articles, 25 relevant original research articles were included. Studies were scored according to the RQS, TRIPOD guidelines, and items in IBSI.

RESULTS

Only four studies (16 %) performed external validation. The mean RQS was 5.6 out of 36 (15.4 %), and the basic adherence rate was 26.8 %. The adherence rate was low for stating biological correlation (4%), conducting calibration statistics (12 %), multiple segmentation (16 %), and stating potential clinical utility (16 %). None of the studies conducted a test‒retest or phantom study, stated a comparison to a 'gold standard', conducted prospective studies or cost-effectivity analysis, or opened code and data to the public, resulting in low RQS. The overall adherence rate for TRIPOD was 54.1 %, with low scores for reporting the title (4%), abstract (0%), blind assessment of the outcome (8%), and explaining the sample size (0%). According to IBSI items, only 6 (24 %), 6 (24 %), and 3 (12 %) studies performed N4 bias-field correction, isovoxel resampling, and grey-level discretization, respectively. No study performed skull stripping.

CONCLUSION

The quality of radiomics studies for meningioma is insufficient. Acknowledgement of RQS, TRIPOD, and IBSI reporting guidelines may improve the quality of meningioma radiomics studies and enable their clinical application.