A systematic review of radiomics in pancreatitis: applying the evidence level rating tool for promoting clinical transferability

Predictive value of machine learning for the severity of acute pancreatitis: A systematic review and meta-analysis

Background

Predicting the severity of acute pancreatitis (AP) early poses a challenge in clinical practice. While there are well-established clinical scoring tools, their actual predictive performance remains uncertain. Various studies have explored the application of machine-learning methods for early AP prediction. However, a more comprehensive evidence-based assessment is needed to determine their predictive accuracy. Hence, this systematic review and meta-analysis aimed to evaluate the predictive accuracy of machine learning in assessing the severity of AP.

Methods

PubMed, EMBASE, Cochrane Library, and Web of Science were systematically searched until December 5, 2023. The risk of bias in eligible studies was assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST). Subgroup analyses, based on different machine learning types, were performed. Additionally, the predictive accuracy of mainstream scoring tools was summarized.

Results

This systematic review ultimately included 33 original studies. The pooled c-index in both the training and validation sets was 0.87 (95 % CI: 0.84-0.89) and 0.88 (95 % CI: 0.86-0.90), respectively. The sensitivity in the training set was 0.81 (95 % CI: 0.77-0.84), and in the validation set, it was 0.79 (95 % CI: 0.71-0.85). The specificity in the training set was 0.84 (95 % CI: 0.78-0.89), and in the validation set, it was 0.90 (95 % CI: 0.86-0.93). The primary model incorporated was logistic regression; however, its predictive accuracy was found to be inferior to that of neural networks, random forests, and xgboost. The pooled c-index of the APACHE II, BISAP, and Ranson were 0.74 (95 % CI: 0.68-0.80), 0.77 (95 % CI: 0.70-0.85), and 0.74 (95 % CI: 0.68-0.79), respectively.

Conclusions

Machine learning demonstrates excellent accuracy in predicting the severity of AP, providing a reference for updating or developing a straightforward clinical prediction tool.

Development and validation of a multimodal model in predicting severe acute pancreatitis based on radiomics and deep learning

OBJECTIVE

Aim to establish a multimodal model for predicting severe acute pancreatitis (SAP) using machine learning (ML) and deep learning (DL).

METHODS

In this multicentre retrospective study, patients diagnosed with acute pancreatitis at admission were enrolled from January 2017 to December 2021. Clinical information within 24 h and CT scans within 72 h of admission were collected. First, we trained Model α based on clinical features selected by least absolute shrinkage and selection operator analysis. Second, radiomics features were extracted from 3D-CT scans and Model β was developed on the features after dimensionality reduction using principal component analysis. Third, Model γ was trained on 2D-CT images. Lastly, a multimodal model, namely PrismSAP, was constructed based on aforementioned features in the training set. The predictive accuracy of PrismSAP was verified in the validation and internal test sets and further validated in the external test set. Model performance was evaluated using area under the curve (AUC), accuracy, sensitivity, specificity, recall, precision and F1-score.

RESULTS

A total of 1,221 eligible patients were randomly split into a training set (n = 864), a validation set (n = 209) and an internal test set (n = 148). Data of 266 patients were for external testing. In the external test set, PrismSAP performed best with the highest AUC of 0.916 (0.873-0.960) among all models [Model α: 0.709 (0.618-0.800); Model β: 0.749 (0.675-0.824); Model γ: 0.687 (0.592-0.782); MCTSI: 0.778 (0.698-0.857); RANSON: 0.642 (0.559-0.725); BISAP: 0.751 (0.668-0.833); SABP: 0.710 (0.621-0.798)].

CONCLUSION

The proposed multimodal model outperformed any single-modality models and traditional scoring systems.

Radiomics-Based Machine Learning Model for Diagnosis of Acute Pancreatitis Using Computed Tomography

In the early diagnostic workup of acute pancreatitis (AP), the role of contrast-enhanced CT is to establish the diagnosis in uncertain cases, assess severity, and detect potential complications like necrosis, fluid collections, bleeding or portal vein thrombosis. The value of texture analysis/radiomics of medical images has rapidly increased during the past decade, and the main focus has been on oncological imaging and tumor classification. Previous studies assessed the value of radiomics for differentiating between malignancies and inflammatory diseases of the pancreas as well as for prediction of AP severity. The aim of our study was to evaluate an automatic machine learning model for AP detection using radiomics analysis. Patients with abdominal pain and contrast-enhanced CT of the abdomen in an emergency setting were retrospectively included in this single-center study. The pancreas was automatically segmented using TotalSegmentator and radiomics features were extracted using PyRadiomics. We performed unsupervised hierarchical clustering and applied the random-forest based Boruta model to select the most important radiomics features. Important features and lipase levels were included in a logistic regression model with AP as the dependent variable. The model was established in a training cohort using fivefold cross-validation and applied to the test cohort (80/20 split). From a total of 1012 patients, 137 patients with AP and 138 patients without AP were included in the final study cohort. Feature selection confirmed 28 important features (mainly shape and first-order features) for the differentiation between AP and controls. The logistic regression model showed excellent diagnostic accuracy of radiomics features for the detection of AP, with an area under the curve (AUC) of 0.932. Using lipase levels only, an AUC of 0.946 was observed. Using both radiomics features and lipase levels, we showed an excellent AUC of 0.933 for the detection of AP. Automated segmentation of the pancreas and consecutive radiomics analysis almost achieved the high diagnostic accuracy of lipase levels, a well-established predictor of AP, and might be considered an additional diagnostic tool in unclear cases. This study provides scientific evidence that automated image analysis of the pancreas achieves comparable diagnostic accuracy to lipase levels and might therefore be used in the future in the rapidly growing era of AI-based image analysis.

Radiomics for the prediction of a postoperative pancreatic fistula following a pancreatoduodenectomy: A systematic review and radiomic score quality assessment

BACKGROUND

Postoperative pancreatic fistula (POPF) is a severe complication following a pancreatoduodenectomy. An accurate prediction of POPF could assist the surgeon in offering tailor-made treatment decisions. The use of radiomic features has been introduced to predict POPF. A systematic review was conducted to evaluate the performance of models predicting POPF using radiomic features and to systematically evaluate the methodological quality.

METHODS

Studies with patients undergoing a pancreatoduodenectomy and radiomics analysis on computed tomography or magnetic resonance imaging were included. Methodological quality was assessed using the Radiomics Quality Score (RQS) and Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement.

RESULTS

Seven studies were included in this systematic review, comprising 1300 patients, of whom 364 patients (28 %) developed POPF. The area under the curve (AUC) of the included studies ranged from 0.76 to 0.95. Only one study externally validated the model, showing an AUC of 0.89 on this dataset. Overall adherence to the RQS (31 %) and TRIPOD guidelines (54 %) was poor.

CONCLUSION

This systematic review showed that high predictive power was reported of studies using radiomic features to predict POPF. However, the quality of most studies was poor. Future studies need to standardize the methodology.

REGISTRATION

not registered.

The endorsement of general and artificial intelligence reporting guidelines in radiological journals: a meta-research study

BACKGROUND

Complete reporting is essential for clinical research. However, the endorsement of reporting guidelines in radiological journals is still unclear. Further, as a field extensively utilizing artificial intelligence (AI), the adoption of both general and AI reporting guidelines would be necessary for enhancing quality and transparency of radiological research. This study aims to investigate the endorsement of general reporting guidelines and those for AI applications in medical imaging in radiological journals, and explore associated journal characteristic variables.

METHODS

This meta-research study screened journals from the Radiology, Nuclear Medicine & Medical Imaging category, Science Citation Index Expanded of the 2022 Journal Citation Reports, and excluded journals not publishing original research, in non-English languages, and instructions for authors unavailable. The endorsement of fifteen general reporting guidelines and ten AI reporting guidelines was rated using a five-level tool: "active strong", "active weak", "passive moderate", "passive weak", and "none". The association between endorsement and journal characteristic variables was evaluated by logistic regression analysis.

RESULTS

We included 117 journals. The top-five endorsed reporting guidelines were CONSORT (Consolidated Standards of Reporting Trials, 58.1%, 68/117), PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses, 54.7%, 64/117), STROBE (STrengthening the Reporting of Observational Studies in Epidemiology, 51.3%, 60/117), STARD (Standards for Reporting of Diagnostic Accuracy, 50.4%, 59/117), and ARRIVE (Animal Research Reporting of In Vivo Experiments, 35.9%, 42/117). The most implemented AI reporting guideline was CLAIM (Checklist for Artificial Intelligence in Medical Imaging, 1.7%, 2/117), while other nine AI reporting guidelines were not mentioned. The Journal Impact Factor quartile and publisher were associated with endorsement of reporting guidelines in radiological journals.

CONCLUSIONS

The general reporting guideline endorsement was suboptimal in radiological journals. The implementation of reporting guidelines for AI applications in medical imaging was extremely low. Their adoption should be strengthened to facilitate quality and transparency of radiological study reporting.

Predictive role of radiomics features extracted from preoperative cross-sectional imaging of pancreatic ductal adenocarcinoma in detecting lymph node metastasis: a systemic review and meta-analysis

Lymph node metastases are associated with poor clinical outcomes in pancreatic ductal adenocarcinoma (PDAC). In preoperative imaging, conventional diagnostic modalities do not provide the desired accuracy in diagnosing lymph node metastasis. The current review aims to determine the pooled diagnostic profile of studies examining the role of radiomics features in detecting lymph node metastasis in PDAC. PubMed, Google Scholar, and Embase databases were searched for relevant articles. The quality of the studies was examined using the Radiomics Quality Score and Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tools. Pooled results for sensitivity, specificity, likelihood, and odds ratios with the corresponding 95% confidence intervals (CIs) were calculated using a random-effect model (DerSimonian-Liard method). No significant publication bias was detected among the studies included in this meta-analysis. The pooled sensitivity of the validation datasets included in the study was 77.4% (72.7%, 81.5%) and pooled specificity was 72.4% (63.8, 79.6%). The diagnostic odds ratio of the validation datasets was 9.6 (6.0, 15.2). No statistically significant heterogeneity was detected for sensitivity and odds ratio (P values of 0.3 and 0.08, respectively). However, there was significant heterogeneity concerning specificity (P = 0.003). The pretest probability of having lymph node metastasis in the pooled databases was 52% and a positive post-test probability was 76% after the radiomics features were used, showing a net benefit of 24%. Classifiers trained on radiomics features extracted from preoperative images can improve the sensitivity and specificity of conventional cross-sectional imaging in detecting lymph node metastasis in PDAC.

An overview of meta-analyses on radiomics: more evidence is needed to support clinical translation

OBJECTIVE

To conduct an overview of meta-analyses of radiomics studies assessing their study quality and evidence level.

METHODS

A systematical search was updated via peer-reviewed electronic databases, preprint servers, and systematic review protocol registers until 15 November 2022. Systematic reviews with meta-analysis of primary radiomics studies were included. Their reporting transparency, methodological quality, and risk of bias were assessed by PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) 2020 checklist, AMSTAR-2 (A MeaSurement Tool to Assess systematic Reviews, version 2) tool, and ROBIS (Risk Of Bias In Systematic reviews) tool, respectively. The evidence level supporting the radiomics for clinical use was rated.

RESULTS

We identified 44 systematic reviews with meta-analyses on radiomics research. The mean ± standard deviation of PRISMA adherence rate was 65 ± 9%. The AMSTAR-2 tool rated 5 and 39 systematic reviews as low and critically low confidence, respectively. The ROBIS assessment resulted low, unclear and high risk in 5, 11, and 28 systematic reviews, respectively. We reperformed 53 meta-analyses in 38 included systematic reviews. There were 3, 7, and 43 meta-analyses rated as convincing, highly suggestive, and weak levels of evidence, respectively. The convincing level of evidence was rated in (1) T2-FLAIR radiomics for IDH-mutant vs IDH-wide type differentiation in low-grade glioma, (2) CT radiomics for COVID-19 vs other viral pneumonia differentiation, and (3) MRI radiomics for high-grade glioma vs brain metastasis differentiation.

CONCLUSIONS

The systematic reviews on radiomics were with suboptimal quality. A limited number of radiomics approaches were supported by convincing level of evidence.

CLINICAL RELEVANCE STATEMENT

The evidence supporting the clinical application of radiomics are insufficient, calling for researches translating radiomics from an academic tool to a practicable adjunct towards clinical deployment.

A systematic review of radiomics in giant cell tumor of bone (GCTB): the potential of analysis on individual radiomics feature for identifying genuine promising imaging biomarkers

PURPOSE

To systematically assess the quality of radiomics research in giant cell tumor of bone (GCTB) and to test the feasibility of analysis at the level of radiomics feature.

METHODS

We searched PubMed, Embase, Web of Science, China National Knowledge Infrastructure, and Wanfang Data to identify articles of GCTB radiomics until 31 July 2022. The studies were assessed by radiomics quality score (RQS), transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement, checklist for artificial intelligence in medical imaging (CLAIM), and modified quality assessment of diagnostic accuracy studies (QUADAS-2) tool. The radiomic features selected for model development were documented.

RESULTS

Nine articles were included. The average of the ideal percentage of RQS, the TRIPOD adherence rate and the CLAIM adherence rate were 26%, 56%, and 57%, respectively. The risk of bias and applicability concerns were mainly related to the index test. The shortness in external validation and open science were repeatedly emphasized. In GCTB radiomics models, the gray level co-occurrence matrix features (40%), first order features (28%), and gray-level run-length matrix features (18%) were most selected features out of all reported features. However, none of the individual feature has appeared repeatably in multiple studies. It is not possible to meta-analyze radiomics features at present.

CONCLUSION

The quality of GCTB radiomics studies is suboptimal. The reporting of individual radiomics feature data is encouraged. The analysis at the level of radiomics feature has potential to generate more practicable evidence for translating radiomics into clinical application.

Role of radiomics in the diagnosis and treatment of gastrointestinal cancer

Gastrointestinal cancer (GIC) has high morbidity and mortality as one of the main causes of cancer death. Preoperative risk stratification is critical to guide patient management, but traditional imaging studies have difficulty predicting its biological behavior. The emerging field of radiomics allows the conversion of potential pathophysiological information in existing medical images that cannot be visually recognized into high-dimensional quantitative image features. Tumor lesion characterization, therapeutic response evaluation, and survival prediction can be achieved by analyzing the relationships between these features and clinical and genetic data. In recent years, the clinical application of radiomics to GIC has increased dramatically. In this editorial, we describe the latest progress in the application of radiomics to GIC and discuss the value of its potential clinical applications, as well as its limitations and future directions.