Data Analysis Strategies in Medical Imaging

Automated ASPECTS Segmentation and Scoring Tool: a Method Tailored for a Colombian Telestroke Network

To evaluate our two non-machine learning (non-ML)-based algorithmic approaches for detecting early ischemic infarcts on brain CT images of patients with acute ischemic stroke symptoms, tailored to our local population, to be incorporated in our telestroke software. One-hundred and thirteen acute stroke patients, excluding hemorrhagic, subacute, and chronic patients, with accessible brain CT images were divided into calibration and test sets. The gold standard was determined through consensus among three neuroradiologist. Four neuroradiologist independently reported Alberta Stroke Program Early CT Scores (ASPECTSs). ASPECTSs were also obtained using a commercial ML solution (CMLS), and our two methods, namely the Mean Hounsfield Unit (HU) relative difference (RELDIF) and the density distribution equivalence test (DDET), which used statistical analyze the of the HUs of each region and its contralateral side. Automated segmentation was perfect for cortical regions, while minimal adjustment was required for basal ganglia regions. For dichotomized-ASPECTSs (ASPECTS < 6) in the test set, the area under the receiver operating characteristic curve (AUC) was 0.85 for the DDET method, 0.84 for the RELDIF approach, 0.64 for the CMLS, and ranged from 0.71-0.89 for the neuroradiologist. The accuracy was 0.85 for the DDET method, 0.88 for the RELDIF approach, and was ranged from 0.83 - 0.96 for the neuroradiologist. Equivalence at a margin of 5% was documented among the DDET, RELDIF, and gold standard on mean ASPECTSs. Noninferiority tests of the AUC and accuracy of infarct detection revealed similarities between both DDET and RELDIF, and the CMLS, and with at least one neuroradiologist. The alignment of our methods with the evaluations of neuroradiologist and the CMLS indicates the potential of our methods to serve as supportive tools in clinical settings, facilitating prompt and accurate stroke diagnosis, especially in health care settings, such as Colombia, where neuroradiologist are limited.

Confirming SPSS Results With ChatGPT-4 and o3-mini Models

Background This research compared the simple and advanced statistical results of SPSS (IBM Corp., Armonk, NY, USA) with ChatGPT-4 and ChatGPT o3-mini (OpenAI, San Francisco, CA, USA) in statistical data output and interpretation with behavioral healthcare data. It evaluated their methodological approaches, quantitative performance, interpretability, adaptability, ethical considerations, and future trends.  Methods  Fourteen statistical analyses were conducted from two real datasets that produced peer-reviewed, published scientific articles in 2024. Descriptive statistics, Pearson r, multiple correlation with Pearson r, Spearman's rho, simple linear regression, one-sample t-test, paired t-test, two-independent sample t-test, multiple linear regression, one-way analysis of variance (ANOVA), repeated measures ANOVA, two-way (factorial) ANOVA, and multivariate ANOVA were computed. The two datasets adhered to a systematically structured timeframe, March 19, 2023, through June 11, 2023, and June 7, 2023, through July 7, 2023, thereby ensuring the integrity and temporal representativeness of the data gathering. The analyses were conducted by inputting the verbal (text) commands into ChatGPT-4 and ChatGPT o3-mini along with the relevant SPSS variables, which were copied and pasted from the SPSS datasets.  Results  The study found high concordance between SPSS and ChatGPT-4 in fundamental statistical analyses, such as measures of central tendency, variability, and simple Pearson and Spearman correlation analyses, where the results were nearly identical. ChatGPT-4 also closely matched SPSS in the three t-tests and simple linear regression, with minimal effect size variations. Discrepancies emerged in complex analyses. ChatGPT o3-mini showed inflated correlation values and significant results where none were expected, indicating computational deviations. ChatGPT o3-mini produced inflated coefficients in the multiple correlation and R-squared values in two-way ANOVA and multiple regression, suggesting differing assumptions. ChatGPT-4 and ChatGPT o3-mini produced identical F-statistics with repeated measures ANOVA but reported incorrect degrees of freedom (df) values. While ChatGPT-4 performed well in one-way ANOVA, it miscalculated degrees of freedom in multivariate ANOVA (MANOVA), leading to significant discrepancies. ChatGPT o3-mini also generated erroneous F-statistics in factorial ANOVA, highlighting the need for further optimization in multivariate statistical modeling. Conclusions This study underscored the rapid advancements in artificial intelligence (AI)-driven statistical analyses while highlighting areas that require further refinement. ChatGPT-4 accurately executed fundamental statistical tests, closely matching SPSS. However, its reliability diminished in more advanced statistical procedures, requiring further validation. ChatGPT o3-mini, while optimized for Science, Technology, Engineering, and Mathematics (STEM) applications, produced inconsistencies in correlation and multivariate analyses, limiting its dependability for complex research applications. Ensuring its alignment with established statistical methodologies will be essential for widespread scientific research adoption as AI evolves.

Predicting coronavirus disease 2019 severity using explainable artificial intelligence techniques

Predictive models for determining coronavirus disease 2019 (COVID-19) severity have been established; however, the complexity of the interactions among factors limits the use of conventional statistical methods. This study aimed to establish a simple and accurate predictive model for COVID-19 severity using an explainable machine learning approach. A total of 3,301 patients ≥ 18 years diagnosed with COVID-19 between February 2020 and October 2022 were included. The discovery cohort comprised patients whose disease onset fell before October 1, 2020 (N = 1,023), and the validation cohort comprised the remaining patients (N = 2,278). Pointwise linear and logistic regression models were used to extract 41 features. Reinforcement learning was used to generate a simple model with high predictive accuracy. The primary evaluation was the area under the receiver operating characteristic curve (AUC). The predictive model achieved an AUC of ≥ 0.905 using four features: serum albumin levels, lactate dehydrogenase levels, age, and neutrophil count. The highest AUC value was 0.906 (sensitivity, 0.842; specificity, 0.811) in the discovery cohort and 0.861 (sensitivity, 0.804; specificity, 0.675) in the validation cohort. Simple and well-structured predictive models were established, which may aid in patient management and the selection of therapeutic interventions.

MRI-based deep learning radiomics to differentiate dual-phenotype hepatocellular carcinoma from HCC and intrahepatic cholangiocarcinoma: a multicenter study

OBJECTIVES

To develop and validate radiomics and deep learning models based on contrast-enhanced MRI (CE-MRI) for differentiating dual-phenotype hepatocellular carcinoma (DPHCC) from HCC and intrahepatic cholangiocarcinoma (ICC).

METHODS

Our study consisted of 381 patients from four centers with 138 HCCs, 122 DPHCCs, and 121 ICCs (244 for training and 62 for internal tests, centers 1 and 2; 75 for external tests, centers 3 and 4). Radiomics, deep transfer learning (DTL), and fusion models based on CE-MRI were established for differential diagnosis, respectively, and their diagnostic performances were compared using the confusion matrix and area under the receiver operating characteristic (ROC) curve (AUC).

RESULTS

The radiomics model demonstrated competent diagnostic performance, with a macro-AUC exceeding 0.9, and both accuracy and F1-score above 0.75 in the internal and external validation sets. Notably, the vgg19-combined model outperformed the radiomics and other DTL models. The fusion model based on vgg19 further improved diagnostic performance, achieving a macro-AUC of 0.990 (95% CI: 0.965-1.000), an accuracy of 0.935, and an F1-score of 0.937 in the internal test set. In the external test set, it similarly performed well, with a macro-AUC of 0.988 (95% CI: 0.964-1.000), accuracy of 0.875, and an F1-score of 0.885.

CONCLUSIONS

Both the radiomics and the DTL models were able to differentiate DPHCC from HCC and ICC before surgery. The fusion models showed better diagnostic accuracy, which has important value in clinical application.

CRITICAL RELEVANCE STATEMENT

MRI-based deep learning radiomics were able to differentiate DPHCC from HCC and ICC preoperatively, aiding clinicians in the identification and targeted treatment of these malignant hepatic tumors.

KEY POINTS

Fusion models may yield an incremental value over radiomics models in differential diagnosis. Radiomics and deep learning effectively differentiate the three types of malignant hepatic tumors. The fusion models may enhance clinical decision-making for malignant hepatic tumors.

A Comprehensive Review on the Application of Artificial Intelligence for Predicting Postsurgical Recurrence Risk in Early-Stage Non-Small Cell Lung Cancer Using Computed Tomography, Positron Emission Tomography, and Clinical Data

INTRODUCTION

Non-small cell lung cancer (NSCLC) is the leading cause of cancer-related mortality worldwide. Despite advancements in early detection and treatment, postsurgical recurrence remains a significant challenge, occurring in 30%-55% of patients within 5 years after surgery. This review analysed existing studies on the utilisation of artificial intelligence (AI), incorporating CT, PET, and clinical data, for predicting recurrence risk in early-stage NSCLCs.

METHODS

A literature search was conducted across multiple databases, focusing on studies published between 2018 and 2024 that employed radiomics, machine learning, and deep learning based on preoperative positron emission tomography (PET), computed tomography (CT), and PET/CT, with or without clinical data integration. Sixteen studies met the inclusion criteria and were assessed for methodological quality using the METhodological RadiomICs Score (METRICS).

RESULTS

The reviewed studies demonstrated the potential of radiomics and AI models in predicting postoperative recurrence risk. Various approaches showed promising results, including handcrafted radiomics features, deep learning models, and multimodal models combining different imaging modalities with clinical data. However, several challenges and limitations were identified, such as small sample sizes, lack of external validation, interpretability issues, and the need for effective multimodal imaging techniques.

CONCLUSIONS

Future research should focus on conducting larger, prospective, multicentre studies, improving data integration and interpretability, enhancing the fusion of imaging modalities, assessing clinical utility, standardising methodologies, and fostering collaboration among researchers and institutions. Addressing these aspects will advance the development of robust and generalizable AI models for predicting postsurgical recurrence risk in early-stage NSCLC, ultimately improving patient care and outcomes.

Highly multiplexed imaging reveals prognostic immune and stromal spatial biomarkers in breast cancer

Spatial profiling of tissues promises to elucidate tumor-microenvironment interactions and generate prognostic and predictive biomarkers. We analyzed single-cell spatial data from 3 multiplex imaging technologies: cyclic immunofluorescence (CycIF) data we generated from 102 patients with breast cancer with clinical follow-up as well as publicly available mass cytometry and multiplex ion-beam imaging datasets. Similar single-cell phenotyping results across imaging platforms enabled combined analysis of epithelial phenotypes to delineate prognostic subtypes among patients who are estrogen-receptor+ (ER+). We utilized discovery and validation cohorts to identify biomarkers with prognostic value. Increased lymphocyte infiltration was independently associated with longer survival in triple-negative (TN) and high-proliferation ER+ breast tumors. An assessment of 10 spatial analysis methods revealed robust spatial biomarkers. In ER+ disease, quiescent stromal cells close to tumor were abundant in tumors with good prognoses, while tumor cell neighborhoods containing mixed fibroblast phenotypes were enriched in poor-prognosis tumors. In TN disease, macrophage/tumor and B/T lymphocyte neighbors were enriched, and lymphocytes were dispersed in good-prognosis tumors, while tumor cell neighborhoods containing vimentin+ fibroblasts were enriched in poor-prognosis tumors. In conclusion, we generated comparable single-cell spatial proteomic data from several clinical cohorts to enable prognostic spatial biomarker identification and validation.

Diagnostic Value of 18F-FDG PET/CT Radiomics in Lymphoma: A Systematic Review and Meta-Analysis

IntroductionVarious machine learning models and features have been proposed for lymphoma diagnosis using 18F-fluorodeoxyglucose (18F-FDG) PET/CT radiomics. This research aimed to systematically evaluate the diagnostic value of 18F-FDG PET/CT radiomics in lymphoma by conducting a meta-analysis.MethodsData from published studies regarding the diagnosis of lymphoma using 18F-FDG PET/CT radiomics, from January 2010 to July 2024, were gathered from PubMed, Web of Science, and the Cochrane Library. Following their separate searches and screenings of the literature, two researchers extracted data and assessed the caliber of all the included studies. The quality assessment involved the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2), the Radiomics Quality Score (RQS), and the METhodological RadiomICs Score (METRICS). The meta-analysis was conducted by using RevMan 5.4.1, R 4.4.0, and Stata 17.0 software. Six meta-regressions were conducted on study performance, considering sample size, image modality, region of interest (ROI) selection, ROI segmentation, radiomics mode, and algorithms.ResultsIn total, 20 studies classified as type 2a or above according to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement were included for this systematic review and meta-analysis. The studies achieved an average RQS of 13 (ranging from 10 to 17), accounting for 36.1% of the total points. The average METRICS score was 69.3% (ranging from 54.8% to 80.9%). The quality category of the studies is mainly "good". The results of our meta-analysis showed that the pooled sensitivity (SEN), specificity (SPE), positive likelihood ratio, negative likelihood ratio and diagnostic odds ratio with 95% confidence interval (CI) were 0.82 (0.78, 0.88), 0.83 (0.76, 0.87), 4.7 (3.4, 6.6), 0.20 (0.15, 0.28) and 23 (13, 42), respectively. The area under the curve of the summary receiver operating characteristic curve was 0.90 (0.87, 0.92). The results of Spearman correlation analysis revealed no threshold effect among the studies (P = .423). Significant heterogeneity was observed among the studies (overall I2 = 83.7%; 95% CI: 76.0, 88.9; P < .01). Meta-regressions indicated that sample size and ROI selection contributed to the heterogeneity in SEN, while algorithms affected the heterogeneity in SPE (P < .05). Deeks' test confirmed there was no significant publication bias in all the included studies. The Fagan nomogram showed an absolute increase of 34% in the post-test probability following a positive test result.ConclusionThe results supported that 18F-FDG PET/CT radiomics has high diagnostic value for lymphoma. However, there is high heterogeneity among different studies. In the future, clinical practicality needs to be substantiated by more prospective studies with rigorous adherence to existing guidelines and multicentric validation.

Assessing the accuracy and efficiency of Chat GPT-4 Omni (GPT-4o) in biomedical statistics: Comparative study with traditional tools

OBJECTIVES

To assess the accuracy of ChatGPT-4 Omni (GPT-4o) in biomedical statistics. The recent novel inauguration of Artificial Intelligence ChatGPT-Omni (GPT-4o), has emerged with the potential to analyze sophisticated and extensive data sets, challenging the expertise of statisticians using traditional statistical tools for data analysis.

METHODS

This study was performed in the Department of Physiology, College of Medicine, King Saud University, Riyadh, Saudi Arabia, in May 2024. Three datasets in a raw Excel file format were imported onto Statistical Package for the Social Sciences (SPSS) version 29 for data analysis. Based on this analysis, a script of 9 questions was prepared to command GPT-4 Omni, which was used for data analysis for all 3 datasets on Omni. The score and the time were recorded for each result and verified after being compared to the original analysis results performed on SPSS.

RESULTS

GPT-4 Omni scored 73 (85.88%) out of 85 points for all 3 datasets. All datasets took a total of 38.43 minutes to be fully analyzed. Individually, Omni scored 21/25 (84%) for the small dataset in 487.4 seconds, 20/25 (80%) for the middle dataset in 747.02 seconds and 32/35 (91.42%) for the large dataset in 1071 seconds. GPT-4 Omni produced accurate graphs and charts.

CONCLUSION

ChatGPT-4 Omni scored better over 80% in all 3 statistical datasets in a short period. GPT-4 Omni also produced accurate graphs and charts as commanded however it required explicit commands with clear instructions to avoid errors and omission of results to achieve appropriate results in biomedical data analysis.

Deep learning radiomics based on contrast enhanced MRI for preoperatively predicting early recurrence in hepatocellular carcinoma after curative resection

Purpose

To explore the role of deep learning (DL) and radiomics-based integrated approach based on contrast enhanced magnetic resonance imaging (CEMRI) for predicting early recurrence (ER) in hepatocellular carcinoma (HCC) patients after curative resection.

Methods

Total 165 HCC patients (ER, n = 96 vs. non-early recurrence (NER), n = 69) were retrospectively collected and divided into a training cohort (n = 132) and a validation cohort (n = 33). From pretreatment CEMR images, a total of 3111 radiomics features were extracted, and radiomics models were constructed using five machine learning classifiers (logistic regression, support vector machine, k-nearest neighbor, extreme gradient Boosting, and multilayer perceptron). DL models were established via three variations of ResNet architecture. The clinical-radiological (CR), radiomics combined with clinical-radiological (RCR), and deep learning combined with RCR (DLRCR) models were constructed. Model discrimination, calibration, and clinical utilities were evaluated by receiver operating characteristic curve, calibration curve, and decision curve analysis, respectively. The best-performing model was compared with the widely used staging systems and preoperative prognostic indexes.

Results

The RCR model (area under the curve (AUC): 0.841 and 0.811) and the optimal radiomics model (AUC: 0.839 and 0.804) achieved better performance than the CR model (AUC: 0.662 and 0.752) in the training and validation cohorts, respectively. The optimal DL model (AUC: 0.870 and 0.826) outperformed the radiomics model in the both cohorts. The DL, radiomics, and CR predictors (aspartate aminotransferase (AST) and tumor diameter) were combined to construct the DLRCR model. The DLRCR model presented the best performance over any model, yielding an AUC, an accuracy, a sensitivity, a specificity of 0.917, 0.886, 0.889, and 0.882 in the training cohort and of 0.844, 0.818, 0.800, and 0.846 in the validation cohort, respectively. The DLRCR model achieved better clinical utility compared to the clinical staging systems and prognostic indexes.

Conclusion

Both radiomics and DL models derived from CEMRI can predict HCC recurrence, and DL and radiomics-based integrated approach can provide a more effective tool for the precise prediction of ER for HCC patients undergoing resection.

A Guideline for Open-Source Tools to Make Medical Imaging Data Ready for Artificial Intelligence Applications: A Society of Imaging Informatics in Medicine (SIIM) Survey

In recent years, the role of Artificial Intelligence (AI) in medical imaging has become increasingly prominent, with the majority of AI applications approved by the FDA being in imaging and radiology in 2023. The surge in AI model development to tackle clinical challenges underscores the necessity for preparing high-quality medical imaging data. Proper data preparation is crucial as it fosters the creation of standardized and reproducible AI models while minimizing biases. Data curation transforms raw data into a valuable, organized, and dependable resource and is a fundamental process to the success of machine learning and analytical projects. Considering the plethora of available tools for data curation in different stages, it is crucial to stay informed about the most relevant tools within specific research areas. In the current work, we propose a descriptive outline for different steps of data curation while we furnish compilations of tools collected from a survey applied among members of the Society of Imaging Informatics (SIIM) for each of these stages. This collection has the potential to enhance the decision-making process for researchers as they select the most appropriate tool for their specific tasks.

The use of artificial intelligence in musculoskeletal ultrasound: a systematic review of the literature

PURPOSE

To systematically review the use of artificial intelligence (AI) in musculoskeletal (MSK) ultrasound (US) with an emphasis on AI algorithm categories and validation strategies.

MATERIAL AND METHODS

An electronic literature search was conducted for articles published up to January 2024. Inclusion criteria were the use of AI in MSK US, involvement of humans, English language, and ethics committee approval.

RESULTS

Out of 269 identified papers, 16 studies published between 2020 and 2023 were included. The research was aimed at predicting diagnosis and/or segmentation in a total of 11 (69%) out of 16 studies. A total of 11 (69%) studies used deep learning (DL)-based algorithms, three (19%) studies employed conventional machine learning (ML)-based algorithms, and two (12%) studies employed both conventional ML- and DL-based algorithms. Six (38%) studies used cross-validation techniques with K-fold cross-validation being the most frequently employed (n = 4, 25%). Clinical validation with separate internal test datasets was reported in nine (56%) papers. No external clinical validation was reported.

CONCLUSION

AI is a topic of increasing interest in MSK US research. In future studies, attention should be paid to the use of validation strategies, particularly regarding independent clinical validation performed on external datasets.

Hybrid deep spatial and statistical feature fusion for accurate MRI brain tumor classification

The classification of medical images is crucial in the biomedical field, and despite attempts to address the issue, significant challenges persist. To effectively categorize medical images, collecting and integrating statistical information that accurately describes the image is essential. This study proposes a unique method for feature extraction that combines deep spatial characteristics with handmade statistical features. The approach involves extracting statistical radiomics features using advanced techniques, followed by a novel handcrafted feature fusion method inspired by the ResNet deep learning model. A new feature fusion framework (FusionNet) is then used to reduce image dimensionality and simplify computation. The proposed approach is tested on MRI images of brain tumors from the BraTS dataset, and the results show that it outperforms existing methods regarding classification accuracy. The study presents three models, including a handcrafted-based model and two CNN models, which completed the binary classification task. The recommended hybrid approach achieved a high F1 score of 96.12 ± 0.41, precision of 97.77 ± 0.32, and accuracy of 97.53 ± 0.24, indicating that it has the potential to serve as a valuable tool for pathologists.

A clinical-radiomic-pathomic model for prognosis prediction in patients with hepatocellular carcinoma after radical resection

PURPOSE

Radical surgery, the first-line treatment for patients with hepatocellular cancer (HCC), faces the dilemma of high early recurrence rates and the inability to predict effectively. We aim to develop and validate a multimodal model combining clinical, radiomics, and pathomics features to predict the risk of early recurrence.

MATERIALS AND METHODS

We recruited HCC patients who underwent radical surgery and collected their preoperative clinical information, enhanced computed tomography (CT) images, and whole slide images (WSI) of hematoxylin and eosin (H & E) stained biopsy sections. After feature screening analysis, independent clinical, radiomics, and pathomics features closely associated with early recurrence were identified. Next, we built 16 models using four combination data composed of three type features, four machine learning algorithms, and 5-fold cross-validation to assess the performance and predictive power of the comparative models.

RESULTS

Between January 2016 and December 2020, we recruited 107 HCC patients, of whom 45.8% (49/107) experienced early recurrence. After analysis, we identified two clinical features, two radiomics features, and three pathomics features associated with early recurrence. Multimodal machine learning models showed better predictive performance than bimodal models. Moreover, the SVM algorithm showed the best prediction results among the multimodal models. The average area under the curve (AUC), accuracy (ACC), sensitivity, and specificity were 0.863, 0.784, 0.731, and 0.826, respectively. Finally, we constructed a comprehensive nomogram using clinical features, a radiomics score and a pathomics score to provide a reference for predicting the risk of early recurrence.

CONCLUSIONS

The multimodal models can be used as a primary tool for oncologists to predict the risk of early recurrence after radical HCC surgery, which will help optimize and personalize treatment strategies.

MRI-Based Radiomics and Deep Learning in Biological Characteristics and Prognosis of Hepatocellular Carcinoma: Opportunities and Challenges

Hepatocellular carcinoma (HCC) is the fifth most common malignancy and the third leading cause of cancer-related death worldwide. HCC exhibits strong inter-tumor heterogeneity, with different biological characteristics closely associated with prognosis. In addition, patients with HCC often distribute at different stages and require diverse treatment options at each stage. Due to the variability in tumor sensitivity to different therapies, determining the optimal treatment approach can be challenging for clinicians prior to treatment. Artificial intelligence (AI) technology, including radiomics and deep learning approaches, has emerged as a unique opportunity to improve the spectrum of HCC clinical care by predicting biological characteristics and prognosis in the medical imaging field. The radiomics approach utilizes handcrafted features derived from specific mathematical formulas to construct various machine-learning models for medical applications. In terms of the deep learning approach, convolutional neural network models are developed to achieve high classification performance based on automatic feature extraction from images. Magnetic resonance imaging offers the advantage of superior tissue resolution and functional information. This comprehensive evaluation plays a vital role in the accurate assessment and effective treatment planning for HCC patients. Recent studies have applied radiomics and deep learning approaches to develop AI-enabled models to improve accuracy in predicting biological characteristics and prognosis, such as microvascular invasion and tumor recurrence. Although AI-enabled models have demonstrated promising potential in HCC with biological characteristics and prognosis prediction with high performance, one of the biggest challenges, interpretability, has hindered their implementation in clinical practice. In the future, continued research is needed to improve the interpretability of AI-enabled models, including aspects such as domain knowledge, novel algorithms, and multi-dimension data sources. Overcoming these challenges would allow AI-enabled models to significantly impact the care provided to HCC patients, ultimately leading to their deployment for clinical use. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 2.

CT and MRI radiomics of bone and soft-tissue sarcomas: an updated systematic review of reproducibility and validation strategies

OBJECTIVE

To systematically review radiomic feature reproducibility and model validation strategies in recent studies dealing with CT and MRI radiomics of bone and soft-tissue sarcomas, thus updating a previous version of this review which included studies published up to 2020.

METHODS

A literature search was conducted on EMBASE and PubMed databases for papers published between January 2021 and March 2023. Data regarding radiomic feature reproducibility and model validation strategies were extracted and analyzed.

RESULTS

Out of 201 identified papers, 55 were included. They dealt with radiomics of bone (n = 23) or soft-tissue (n = 32) tumors. Thirty-two (out of 54 employing manual or semiautomatic segmentation, 59%) studies included a feature reproducibility analysis. Reproducibility was assessed based on intra/interobserver segmentation variability in 30 (55%) and geometrical transformations of the region of interest in 2 (4%) studies. At least one machine learning validation technique was used for model development in 34 (62%) papers, and K-fold cross-validation was employed most frequently. A clinical validation of the model was reported in 38 (69%) papers. It was performed using a separate dataset from the primary institution (internal test) in 22 (40%), an independent dataset from another institution (external test) in 14 (25%) and both in 2 (4%) studies.

CONCLUSIONS

Compared to papers published up to 2020, a clear improvement was noted with almost double publications reporting methodological aspects related to reproducibility and validation. Larger multicenter investigations including external clinical validation and the publication of databases in open-access repositories could further improve methodology and bring radiomics from a research area to the clinical stage.

CRITICAL RELEVANCE STATEMENT

An improvement in feature reproducibility and model validation strategies has been shown in this updated systematic review on radiomics of bone and soft-tissue sarcomas, highlighting efforts to enhance methodology and bring radiomics from a research area to the clinical stage.

KEY POINTS

• 2021-2023 radiomic studies on CT and MRI of musculoskeletal sarcomas were reviewed. • Feature reproducibility was assessed in more than half (59%) of the studies. • Model clinical validation was performed in 69% of the studies. • Internal (44%) and/or external (29%) test datasets were employed for clinical validation.

Biomedical Big Data Technologies, Applications, and Challenges for Precision Medicine: A Review

The explosive growth of biomedical Big Data presents both significant opportunities and challenges in the realm of knowledge discovery and translational applications within precision medicine. Efficient management, analysis, and interpretation of big data can pave the way for groundbreaking advancements in precision medicine. However, the unprecedented strides in the automated collection of large-scale molecular and clinical data have also introduced formidable challenges in terms of data analysis and interpretation, necessitating the development of novel computational approaches. Some potential challenges include the curse of dimensionality, data heterogeneity, missing data, class imbalance, and scalability issues. This overview article focuses on the recent progress and breakthroughs in the application of big data within precision medicine. Key aspects are summarized, including content, data sources, technologies, tools, challenges, and existing gaps. Nine fields-Datawarehouse and data management, electronic medical record, biomedical imaging informatics, Artificial intelligence-aided surgical design and surgery optimization, omics data, health monitoring data, knowledge graph, public health informatics, and security and privacy-are discussed.

Preoperative Prediction of New Vertebral Fractures after Vertebral Augmentation with a Radiomics Nomogram

The occurrence of new vertebral fractures (NVFs) after vertebral augmentation (VA) procedures is common in patients with osteoporotic vertebral compression fractures (OVCFs), leading to painful experiences and financial burdens. We aim to develop a radiomics nomogram for the preoperative prediction of NVFs after VA. Data from center 1 (training set: n = 153; internal validation set: n = 66) and center 2 (external validation set: n = 44) were retrospectively collected. Radiomics features were extracted from MRI images and radiomics scores (radscores) were constructed for each level-specific vertebra based on least absolute shrinkage and selection operator (LASSO). The radiomics nomogram, integrating radiomics signature with presence of intravertebral cleft and number of previous vertebral fractures, was developed by multivariable logistic regression analysis. The predictive performance of the vertebrae was level-specific based on radscores and was generally superior to clinical variables. RadscoreL2 had the optimal discrimination (AUC ≥ 0.751). The nomogram provided good predictive performance (AUC ≥ 0.834), favorable calibration, and large clinical net benefits in each set. It was used successfully to categorize patients into high- or low-risk subgroups. As a noninvasive preoperative prediction tool, the MRI-based radiomics nomogram holds great promise for individualized prediction of NVFs following VA.

Dual adversarial deconfounding autoencoder for joint batch-effects removal from multi-center and multi-scanner radiomics data

Medical imaging represents the primary tool for investigating and monitoring several diseases, including cancer. The advances in quantitative image analysis have developed towards the extraction of biomarkers able to support clinical decisions. To produce robust results, multi-center studies are often set up. However, the imaging information must be denoised from confounding factors-known as batch-effect-like scanner-specific and center-specific influences. Moreover, in non-solid cancers, like lymphomas, effective biomarkers require an imaging-based representation of the disease that accounts for its multi-site spreading over the patient's body. In this work, we address the dual-factor deconfusion problem and we propose a deconfusion algorithm to harmonize the imaging information of patients affected by Hodgkin Lymphoma in a multi-center setting. We show that the proposed model successfully denoises data from domain-specific variability (p-value < 0.001) while it coherently preserves the spatial relationship between imaging descriptions of peer lesions (p-value = 0), which is a strong prognostic biomarker for tumor heterogeneity assessment. This harmonization step allows to significantly improve the performance in prognostic models with respect to state-of-the-art methods, enabling building exhaustive patient representations and delivering more accurate analyses (p-values < 0.001 in training, p-values < 0.05 in testing). This work lays the groundwork for performing large-scale and reproducible analyses on multi-center data that are urgently needed to convey the translation of imaging-based biomarkers into the clinical practice as effective prognostic tools. The code is available on GitHub at this https://github.com/LaraCavinato/Dual-ADAE .

Multiparametric Oncologic Hybrid Imaging: Machine Learning Challenges and Opportunities

BACKGROUND

Machine learning (ML) is considered an important technology for future data analysis in health care.

METHODS

The inherently technology-driven fields of diagnostic radiology and nuclear medicine will both benefit from ML in terms of image acquisition and reconstruction. Within the next few years, this will lead to accelerated image acquisition, improved image quality, a reduction of motion artifacts and - for PET imaging - reduced radiation exposure and new approaches for attenuation correction. Furthermore, ML has the potential to support decision making by a combined analysis of data derived from different modalities, especially in oncology. In this context, we see great potential for ML in multiparametric hybrid imaging and the development of imaging biomarkers.

RESULTS AND CONCLUSION

In this review, we will describe the basics of ML, present approaches in hybrid imaging of MRI, CT, and PET, and discuss the specific challenges associated with it and the steps ahead to make ML a diagnostic and clinical tool in the future.

KEY POINTS

· ML provides a viable clinical solution for the reconstruction, processing, and analysis of hybrid imaging obtained from MRI, CT, and PET..

Utilizing heat maps as explainable artificial intelligence for detecting abnormalities on wrist and elbow radiographs

INTRODUCTION

Wrist and elbow radiographs, which plays a key role in diagnosing both fractures and degenerative conditions, present a diagnostic challenge due to intricate structures and subtle pathological signs. Artificial intelligence (AI) through deep learning models, has transformed diagnostic imaging, achieving accuracy rates, with explainable AI (XAI) and Gradient-weighted Class Activation Mapping (Grad-CAM) enhancing transparency of AI-driven diagnosis.

METHOD

The MURA-dataset, a comprehensive collection of musculoskeletal radiographs, specifically focuses on wrist and elbow images, ensuring a spectrum of normal and abnormal conditions. An ensemble of transfer-learning models, including VGG16, VGG19, ResNet, DenseNet, InceptionV3 and Xception, was applied, with implemented Grad-CAM techniques, providing interpretable heat maps. The Dice Similarity Coefficient (DSC) evaluated the algorithm's efficiency in recognizing regions of interest.

RESULTS

The average test accuracy of the 20 models were 0.81 (0.72-0.84), and 0.60 (0.49-0.73) for the wrist and elbow radiographs, respectively. The highest performing models were VGG16 with a test accuracy of 0.84, and DenseNet169 with a test accuracy of 0.73. The DSC were calculated for the six highest performing models, and agreements between algorithms were found on radiographs with metal, and only minimal agreement for radiographs with fractures.

CONCLUSION

The study employed twenty transfer-learning models on wrist and elbow radiographs presenting accuracy and partial agreement with Grad-CAM technique evaluation. This study enables comprehension of model performance and avenues for potential enhancement.

IMPLICATION FOR PRACTICE

The utilization of artificial intelligence, specifically transfer-learning models, could greatly enhance the accuracy and efficiency of diagnosing conditions from wrist and elbow radiographs. Additionally, the application of explainable AI techniques such as Grad-CAM can provide visual validation and transparency, thereby strengthening trust and adoption in clinical settings.

Transparency in Artificial Intelligence Research: a Systematic Review of Availability Items Related to Open Science in Radiology and Nuclear Medicine

RATIONALE AND OBJECTIVES

Reproducibility of artificial intelligence (AI) research has become a growing concern. One of the fundamental reasons is the lack of transparency in data, code, and model. In this work, we aimed to systematically review the radiology and nuclear medicine papers on AI in terms of transparency and open science.

MATERIALS AND METHODS

A systematic literature search was performed in PubMed to identify original research studies on AI. The search was restricted to studies published in Q1 and Q2 journals that are also indexed on the Web of Science. A random sampling of the literature was performed. Besides six baseline study characteristics, a total of five availability items were evaluated. Two groups of independent readers including eight readers participated in the study. Inter-rater agreement was analyzed. Disagreements were resolved with consensus.

RESULTS

Following eligibility criteria, we included a final set of 194 papers. The raw data was available in about one-fifth of the papers (34/194; 18%). However, the authors made their private data available only in one paper (1/161; 1%). About one-tenth of the papers made their pre-modeling (25/194; 13%), modeling (28/194; 14%), or post-modeling files (15/194; 8%) available. Most of the papers (189/194; 97%) did not attempt to create a ready-to-use system for real-world usage. Data origin, use of deep learning, and external validation had statistically significantly different distributions. The use of private data alone was negatively associated with the availability of at least one item (p<0.001).

CONCLUSION

Overall rates of availability for items were poor, leaving room for substantial improvement.

Radiomics in gastrointestinal stromal tumours: an up-to-date review

Gastrointestinal stromal tumours are rare mesenchymal neoplasms originating from the Cajal cells and represent the most common sarcomas in the gastroenteric tract. Symptoms may be absent or non-specific, ranging from fatigue and weight loss to acute abdomen. Nowadays endoscopy, echoendoscopy, contrast-enhanced computed tomography, magnetic resonance imaging and positron emission tomography are the main methods for diagnosis. Because of their rarity, these neoplasms may not be included immediately in the differential diagnosis of a solitary abdominal mass. Radiomics is an emerging technique that can extract medical imaging information, not visible to the human eye, transforming it into quantitative data. The purpose of this review is to demonstrate how radiomics can improve the already known imaging techniques by providing useful tools for the diagnosis, treatment, and prognosis of these tumours.

MRI-based radiomics model for distinguishing Stage I endometrial carcinoma from endometrial polyp: a multicenter study

BACKGROUND

Patients with early endometrial carcinoma (EC) have a good prognosis, but it is difficult to distinguish from endometrial polyps (EPs).

PURPOSE

To develop and assess magnetic resonance imaging (MRI)-based radiomics models for discriminating Stage I EC from EP in a multicenter setting.

MATERIAL AND METHODS

Patients with Stage I EC (n = 202) and EP (n = 99) who underwent preoperative MRI scans were collected in three centers (seven devices). The images from devices 1-3 were utilized for training and validation, and the images from devices 4-7 were utilized for testing, leading to three models. They were evaluated by the area under the receiver operating characteristic curve (AUC) and metrics including accuracy, sensitivity, and specificity. Two radiologists evaluated the endometrial lesions and compared them with the three models.

RESULTS

The AUCs of device 1, 2_ada, device 1, 3_ada, and device 2, 3_ada for discriminating Stage I EC from EP were 0.951, 0.912, and 0.896 for the training set, 0.755, 0.928, and 1.000 for the validation set, and 0.883, 0.956, and 0.878 for the external validation set, respectively. The specificity of the three models was higher, but the accuracy and sensitivity were lower than those of radiologists.

CONCLUSION

Our MRI-based models showed good potential in differentiating Stage I EC from EP and had been validated in multiple centers. Their specificity was higher than that of radiologists and may be used for computer-aided diagnosis in the future to assist clinical diagnosis.

Snake-Efficient Feature Selection-Based Framework for Precise Early Detection of Chronic Kidney Disease

Chronic kidney disease (CKD) refers to impairment of the kidneys that may worsen over time. Early detection of CKD is crucial for saving millions of lives. As a result, several studies are currently focused on developing computer-aided systems to detect CKD in its early stages. Manual screening is time-consuming and subject to personal judgment. Therefore, methods based on machine learning (ML) and automatic feature selection are used to support graders. The goal of feature selection is to identify the most relevant and informative subset of features in a given dataset. This approach helps mitigate the curse of dimensionality, reduce dimensionality, and enhance model performance. The use of natural-inspired optimization algorithms has been widely adopted to develop appropriate representations of complex problems by conducting a blackbox optimization process without explicitly formulating mathematical formulations. Recently, snake optimization algorithms have been developed to identify optimal or near-optimal solutions to difficult problems by mimicking the behavior of snakes during hunting. The objective of this paper is to develop a novel snake-optimized framework named CKD-SO for CKD data analysis. To select and classify the most suitable medical data, five machine learning algorithms are deployed, along with the snake optimization (SO) algorithm, to create an extremely accurate prediction of kidney and liver disease. The end result is a model that can detect CKD with 99.7% accuracy. These results contribute to our understanding of the medical data preparation pipeline. Furthermore, implementing this method will enable health systems to achieve effective CKD prevention by providing early interventions that reduce the high burden of CKD-related diseases and mortality.

Ensemble Deep Learning Derived from Transfer Learning for Classification of COVID-19 Patients on Hybrid Deep-Learning-Based Lung Segmentation: A Data Augmentation and Balancing Framework

BACKGROUND AND MOTIVATION

Lung computed tomography (CT) techniques are high-resolution and are well adopted in the intensive care unit (ICU) for COVID-19 disease control classification. Most artificial intelligence (AI) systems do not undergo generalization and are typically overfitted. Such trained AI systems are not practical for clinical settings and therefore do not give accurate results when executed on unseen data sets. We hypothesize that ensemble deep learning (EDL) is superior to deep transfer learning (TL) in both non-augmented and augmented frameworks.

METHODOLOGY

The system consists of a cascade of quality control, ResNet-UNet-based hybrid deep learning for lung segmentation, and seven models using TL-based classification followed by five types of EDL's. To prove our hypothesis, five different kinds of data combinations (DC) were designed using a combination of two multicenter cohorts-Croatia (80 COVID) and Italy (72 COVID and 30 controls)-leading to 12,000 CT slices. As part of generalization, the system was tested on unseen data and statistically tested for reliability/stability.

RESULTS

Using the K5 (80:20) cross-validation protocol on the balanced and augmented dataset, the five DC datasets improved TL mean accuracy by 3.32%, 6.56%, 12.96%, 47.1%, and 2.78%, respectively. The five EDL systems showed improvements in accuracy of 2.12%, 5.78%, 6.72%, 32.05%, and 2.40%, thus validating our hypothesis. All statistical tests proved positive for reliability and stability.

CONCLUSION

EDL showed superior performance to TL systems for both (a) unbalanced and unaugmented and (b) balanced and augmented datasets for both (i) seen and (ii) unseen paradigms, validating both our hypotheses.

Robust biomarker discovery through multiplatform multiplex image analysis of breast cancer clinical cohorts

Spatial profiling of tissues promises to elucidate tumor-microenvironment interactions and enable development of spatial biomarkers to predict patient response to immunotherapy and other therapeutics. However, spatial biomarker discovery is often carried out on a single patient cohort or imaging technology, limiting statistical power and increasing the likelihood of technical artifacts. In order to analyze multiple patient cohorts profiled on different platforms, we developed methods for comparative data analysis from three disparate multiplex imaging technologies: 1) cyclic immunofluorescence data we generated from 102 breast cancer patients with clinical follow-up, in addition to publicly available 2) imaging mass cytometry and 3) multiplex ion-beam imaging data. We demonstrate similar single-cell phenotyping results across breast cancer patient cohorts imaged with these three technologies and identify cellular abundance and proximity-based biomarkers with prognostic value across platforms. In multiple platforms, we identified lymphocyte infiltration as independently associated with longer survival in triple negative and high-proliferation breast tumors. Then, a comparison of nine spatial analysis methods revealed robust spatial biomarkers. In estrogen receptor-positive disease, quiescent stromal cells close to tumor were more abundant in good prognosis tumors while tumor neighborhoods of mixed fibroblast phenotypes were enriched in poor prognosis tumors. In triple-negative breast cancer (TNBC), macrophage proximity to tumor and B cell proximity to T cells were greater in good prognosis tumors, while tumor neighborhoods of vimentin-positive fibroblasts were enriched in poor prognosis tumors. We also tested previously published spatial biomarkers in our ensemble cohort, reproducing the positive prognostic value of isolated lymphocytes and lymphocyte occupancy and failing to reproduce the prognostic value of tumor-immune mixing score in TNBC. In conclusion, we demonstrate assembly of larger clinical cohorts from diverse platforms to aid in prognostic spatial biomarker identification and validation.

Predicting 18F-FDG SUVs of metastatic pulmonary nodes from CT images in patients with differentiated thyroid cancer by using a convolutional neural network

Purpose

The aim of this study was to predict standard uptake values (SUVs) from computed tomography (CT) images of patients with lung metastases from differentiated thyroid cancer (DTC-LM).

Methods

We proposed a novel SUVs prediction model using 18-layer Residual Network for generating SUVmax, SUVmean, SUVmin of metastatic pulmonary nodes from CT images of patients with DTC-LM. Nuclear medicine specialists outlined the metastatic pulmonary as primary set. The best model parameters were obtained after five-fold cross-validation on the training and validation set, further evaluated in independent test set. Mean absolute error (MAE), mean squared error (MSE), and mean relative error (MRE) were used to assess the performance of regression task. Specificity, sensitivity, F1 score, positive predictive value, negative predictive value and accuracy were used for classification task. The correlation between predicted and actual SUVs was analyzed.

Results

A total of 3407 nodes from 74 patients with DTC-LM were collected in this study. On the independent test set, the average MAE, MSE and MRE was 0.3843, 1.0133, 0.3491 respectively, and the accuracy was 88.26%. Our proposed model achieved high metric scores (MAE=0.3843, MSE=1.0113, MRE=34.91%) compared with other backbones. The predicted SUVmax (R^{2 =} 0.8987), SUVmean (R^{2 =} 0.8346), SUVmin (R^{2 =} 0.7373) were all significantly correlated with actual SUVs.

Conclusion

The novel approach proposed in this study provides new ideas for the application of predicting SUVs for metastatic pulmonary nodes in DTC patients.

CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII

Even though radiomics can hold great potential for supporting clinical decision-making, its current use is mostly limited to academic research, without applications in routine clinical practice. The workflow of radiomics is complex due to several methodological steps and nuances, which often leads to inadequate reporting and evaluation, and poor reproducibility. Available reporting guidelines and checklists for artificial intelligence and predictive modeling include relevant good practices, but they are not tailored to radiomic research. There is a clear need for a complete radiomics checklist for study planning, manuscript writing, and evaluation during the review process to facilitate the repeatability and reproducibility of studies. We here present a documentation standard for radiomic research that can guide authors and reviewers. Our motivation is to improve the quality and reliability and, in turn, the reproducibility of radiomic research. We name the checklist CLEAR (CheckList for EvaluAtion of Radiomics research), to convey the idea of being more transparent. With its 58 items, the CLEAR checklist should be considered a standardization tool providing the minimum requirements for presenting clinical radiomics research. In addition to a dynamic online version of the checklist, a public repository has also been set up to allow the radiomics community to comment on the checklist items and adapt the checklist for future versions. Prepared and revised by an international group of experts using a modified Delphi method, we hope the CLEAR checklist will serve well as a single and complete scientific documentation tool for authors and reviewers to improve the radiomics literature.

Efficacy of Artificial Intelligence in the Categorisation of Paediatric Pneumonia on Chest Radiographs: A Systematic Review

This study aimed to systematically review the literature to synthesise and summarise the evidence surrounding the efficacy of artificial intelligence (AI) in classifying paediatric pneumonia on chest radiographs (CXRs). Following the initial search of studies that matched the pre-set criteria, their data were extracted using a data extraction tool, and the included studies were assessed via critical appraisal tools and risk of bias. Results were accumulated, and outcome measures analysed included sensitivity, specificity, accuracy, and area under the curve (AUC). Five studies met the inclusion criteria. The highest sensitivity was by an ensemble AI algorithm (96.3%). DenseNet201 obtained the highest level of specificity and accuracy (94%, 95%). The most outstanding AUC value was achieved by the VGG16 algorithm (96.2%). Some of the AI models achieved close to 100% diagnostic accuracy. To assess the efficacy of AI in a clinical setting, these AI models should be compared to that of radiologists. The included and evaluated AI algorithms showed promising results. These algorithms can potentially ease and speed up diagnosis once the studies are replicated and their performances are assessed in clinical settings, potentially saving millions of lives.

Multiparametric Data-driven Imaging Markers: Guidelines for Development, Application and Reporting of Model Outputs in Radiomics

This paper is the fifth in a five-part series on statistical methodology for performance assessment of multi-parametric quantitative imaging biomarkers (mpQIBs) for radiomic analysis. Radiomics is the process of extracting visually imperceptible features from radiographic medical images using data-driven algorithms. We refer to the radiomic features as data-driven imaging markers (DIMs), which are quantitative measures discovered under a data-driven framework from images beyond visual recognition but evident as patterns of disease processes irrespective of whether or not ground truth exists for the true value of the DIM. This paper aims to set guidelines on how to build machine learning models using DIMs in radiomics and to apply and report them appropriately. We provide a list of recommendations, named RANDAM (an abbreviation of "Radiomic ANalysis and DAta Modeling"), for analysis, modeling, and reporting in a radiomic study to make machine learning analyses in radiomics more reproducible. RANDAM contains five main components to use in reporting radiomics studies: design, data preparation, data analysis and modeling, reporting, and material availability. Real case studies in lung cancer research are presented along with simulation studies to compare different feature selection methods and several validation strategies.

Must-have Qualities of Clinical Research on Artificial Intelligence and Machine Learning

In the field of computer science, known as artificial intelligence, algorithms imitate reasoning tasks that are typically performed by humans. The techniques that allow machines to learn and get better at tasks such as recognition and prediction, which form the basis of clinical practice, are referred to as machine learning, which is a subfield of artificial intelligence. The number of artificial intelligence-and machine learnings-related publications in clinical journals has grown exponentially, driven by recent developments in computation and the accessibility of simple tools. However, clinicians are often not included in data science teams, which may limit the clinical relevance, explanability, workflow compatibility, and quality improvement of artificial intelligence solutions. Thus, this results in the language barrier between clinicians and artificial intelligence developers. Healthcare practitioners sometimes lack a basic understanding of artificial intelligence research because the approach is difficult for non-specialists to understand. Furthermore, many editors and reviewers of medical publications might not be familiar with the fundamental ideas behind these technologies, which may prevent journals from publishing high-quality artificial intelligence studies or, worse still, could allow for the publication of low-quality works. In this review, we aim to improve readers’ artificial intelligence literacy and critical thinking. As a result, we concentrated on what we consider the 10 most important qualities of artificial intelligence research: valid scientific purpose, high-quality data set, robust reference standard, robust input, no information leakage, optimal bias-variance tradeoff, proper model evaluation, proven clinical utility, transparent reporting, and open science. Before designing a study, one should have defined a sound scientific purpose. Then, it should be backed by a high-quality data set, robust input, and a solid reference standard. The artificial intelligence development pipeline should prevent information leakage. For the models, optimal bias-variance tradeoff should be achieved, and generalizability assessment must be adequately performed. The clinical value of the final models must also be established. After the study, thought should be given to transparency in publishing the process and results as well as open science for sharing data, code, and models. We hope this work may improve the artificial intelligence literacy and mindset of the readers.

Vitreoretinal Surgical Instrument Tracking in Three Dimensions Using Deep Learning

Purpose

To evaluate the potential for artificial intelligence-based video analysis to determine surgical instrument characteristics when moving in the three-dimensional vitreous space.

Methods

We designed and manufactured a model eye in which we recorded choreographed videos of many surgical instruments moving throughout the eye. We labeled each frame of the videos to describe the surgical tool characteristics: tool type, location, depth, and insertional laterality. We trained two different deep learning models to predict each of the tool characteristics and evaluated model performances on a subset of images.

Results

The accuracy of the classification model on the training set is 84% for the x-y region, 97% for depth, 100% for instrument type, and 100% for laterality of insertion. The accuracy of the classification model on the validation dataset is 83% for the x-y region, 96% for depth, 100% for instrument type, and 100% for laterality of insertion. The close-up detection model performs at 67 frames per second, with precision for most instruments higher than 75%, achieving a mean average precision of 79.3%.

Conclusions

We demonstrated that trained models can track surgical instrument movement in three-dimensional space and determine instrument depth, tip location, instrument insertional laterality, and instrument type. Model performance is nearly instantaneous and justifies further investigation into application to real-world surgical videos.

Translational Relevance

Deep learning offers the potential for software-based safety feedback mechanisms during surgery or the ability to extract metrics of surgical technique that can direct research to optimize surgical outcomes.

Non-invasive prediction of microsatellite instability in colorectal cancer by a genetic algorithm-enhanced artificial neural network-based CT radiomics signature

OBJECTIVE

The stratification of microsatellite instability (MSI) status assists clinicians in making treatment decisions for colorectal cancer (CRC) patients. This study aimed to establish a CT-based radiomics signature to predict MSI status in patients with CRC.

METHODS

A total of 837 CRC patients who underwent preoperative enhanced CT and had available MSI status data were recruited from two hospitals. Radiomics features were extracted from segmented tumours, and a series of data balancing and feature selection strategies were used to select MSI-related features. Finally, an MSI-related radiomics signature was constructed using a genetic algorithm-enhanced artificial neural network model. Combined and clinical models were constructed using multivariate logistic regression analyses by integrating the clinical factors with or without the signature. A Kaplan-Meier survival analysis was conducted to explore the prognostic information of the signature in patients with CRC.

RESULTS

Ten features were selected to construct a signature which showed robust performance in both the internal and external validation cohorts, with areas under the curves (AUC) of 0.788 and 0.775, respectively. The performance of the signature was comparable to that of the combined model (AUCs of 0.777 and 0.767, respectively) and it outperformed the clinical model constituting age and tumour location (AUCs of 0.768 and 0.623, respectively). Survival analysis demonstrated that the signature could stratify patients with stage II CRC according to prognosis (HR: 0.402, p = 0.029).

CONCLUSIONS

This study built a robust radiomics signature for identifying the MSI status of CRC patients, which may assist individualised treatment decisions.

KEY POINTS

• Our well-designed modelling strategies helped overcome the problem of data imbalance caused by the low incidence of MSI. • Genetic algorithm-enhanced artificial neural network-based CT radiomics signature can effectively distinguish the MSI status of CRC patients. • Kaplan-Meier survival analysis demonstrated that our signature could significantly stratify stage II CRC patients into high- and low-risk groups.

A novel multimodal deep learning model for preoperative prediction of microvascular invasion and outcome in hepatocellular carcinoma

BACKGROUND

Accurate preoperative identification of the microvascular invasion (MVI) can relieve the pressure from personalized treatment adaptation and improve the poor prognosis for hepatocellular carcinoma (HCC). This study aimed to develop and validate a novel multimodal deep learning (DL) model for predicting MVI based on multi-parameter magnetic resonance imaging (MRI) and contrast-enhanced computed tomography (CT).

METHODS

A total of 397 HCC patients underwent both CT and MRI examinations before surgery. We established the radiological models (R^CT, R^MRI) by support vector machine (SVM), DL models (DL^CT_ALL, DL^MRI_ALL, DL^CT + MRI) by ResNet18. The comprehensive model (C^ALL) involving multi-modality DL features and clinical and radiological features was constructed using SVM. Model performance was quantified by the area under the receiver operating characteristic curve (AUC) and compared by net reclassification index (NRI) and integrated discrimination improvement (IDI).

RESULTS

The DL^CT + MRI model exhibited superior predicted efficiency over single-modality models, especially over the DL^CT_ALL model (AUC: 0.819 vs. 0.742, NRI > 0, IDI > 0). The DL^MRI_ALL model improved the performance over the R^MRI model (AUC: 0.794 vs. 0.766, NRI > 0, IDI < 0), but no such difference was found between the DL^CT_ALL model and R^CT model (AUC: 0.742 vs. 0.710, NRI < 0, IDI < 0). Furthermore, both the DL^CT + MRI and C^ALL models revealed the prognostic power in recurrence-free survival stratification (P < 0.001).

CONCLUSION

The proposed DL^CT + MRI model showed robust capability in predicting MVI and outcomes for HCC. Besides, the identification ability of the multi-modality DL model was better than any single modality, especially for CT.

Joint EANM/SNMMI guideline on radiomics in nuclear medicine : Jointly supported by the EANM Physics Committee and the SNMMI Physics, Instrumentation and Data Sciences Council

PURPOSE

The purpose of this guideline is to provide comprehensive information on best practices for robust radiomics analyses for both hand-crafted and deep learning-based approaches.

METHODS

In a cooperative effort between the EANM and SNMMI, we agreed upon current best practices and recommendations for relevant aspects of radiomics analyses, including study design, quality assurance, data collection, impact of acquisition and reconstruction, detection and segmentation, feature standardization and implementation, as well as appropriate modelling schemes, model evaluation, and interpretation. We also offer an outlook for future perspectives.

CONCLUSION

Radiomics is a very quickly evolving field of research. The present guideline focused on established findings as well as recommendations based on the state of the art. Though this guideline recognizes both hand-crafted and deep learning-based radiomics approaches, it primarily focuses on the former as this field is more mature. This guideline will be updated once more studies and results have contributed to improved consensus regarding the application of deep learning methods for radiomics. Although methodological recommendations in the present document are valid for most medical image modalities, we focus here on nuclear medicine, and specific recommendations when necessary are made for PET/CT, PET/MR, and quantitative SPECT.

A Comparison of Techniques for Class Imbalance in Deep Learning Classification of Breast Cancer

Tools based on deep learning models have been created in recent years to aid radiologists in the diagnosis of breast cancer from mammograms. However, the datasets used to train these models may suffer from class imbalance, i.e., there are often fewer malignant samples than benign or healthy cases, which can bias the model towards the healthy class. In this study, we systematically evaluate several popular techniques to deal with this class imbalance, namely, class weighting, over-sampling, and under-sampling, as well as a synthetic lesion generation approach to increase the number of malignant samples. These techniques are applied when training on three diverse Full-Field Digital Mammography datasets, and tested on in-distribution and out-of-distribution samples. The experiments show that a greater imbalance is associated with a greater bias towards the majority class, which can be counteracted by any of the standard class imbalance techniques. On the other hand, these methods provide no benefit to model performance with respect to Area Under the Curve of the Recall Operating Characteristic (AUC-ROC), and indeed under-sampling leads to a reduction of 0.066 in AUC in the case of a 19:1 benign to malignant imbalance. Our synthetic lesion methodology leads to better performance in most cases, with increases of up to 0.07 in AUC on out-of-distribution test sets over the next best experiment.

A Framework of Analysis to Facilitate the Harmonization of Multicenter Radiomic Features in Prostate Cancer

Pooling radiomic features coming from different centers in a statistical framework is challenging due to the variability in scanner models, acquisition protocols, and reconstruction settings. To remove technical variability, commonly called batch effects, different statistical harmonization strategies have been widely used in genomics but less considered in radiomics. The aim of this work was to develop a framework of analysis to facilitate the harmonization of multicenter radiomic features extracted from prostate T2-weighted magnetic resonance imaging (MRI) and to improve the power of radiomics for prostate cancer (PCa) management in order to develop robust non-invasive biomarkers translating into clinical practice. To remove technical variability and correct for batch effects, we investigated four different statistical methods (ComBat, SVA, Arsynseq, and mixed effect). The proposed approaches were evaluated using a dataset of 210 prostate cancer (PCa) patients from two centers. The impacts of the different statistical approaches were evaluated by principal component analysis and classification methods (LogitBoost, random forest, K-nearest neighbors, and decision tree). The ComBat method outperformed all other methods by achieving 70% accuracy and 78% AUC with the random forest method to automatically classify patients affected by PCa. The proposed statistical framework enabled us to define and develop a standardized pipeline of analysis to harmonize multicenter T2W radiomic features, yielding great promise to support PCa clinical practice.

Deep multiple instance learning for predicting chemotherapy response in non-small cell lung cancer using pretreatment CT images

The individual prognosis of chemotherapy is quite different in non-small cell lung cancer (NSCLC). There is an urgent need to precisely predict and assess the treatment response. To develop a deep multiple-instance learning (DMIL) based model for predicting chemotherapy response in NSCLC in pretreatment CT images. Two datasets of NSCLC patients treated with chemotherapy as the first-line treatment were collected from two hospitals. Dataset 1 (163 response and 138 nonresponse) was used to train, validate, and test the DMIL model and dataset 2 (22 response and 20 nonresponse) was used as the external validation cohort. Five backbone networks in the feature extraction module and three pooling methods were compared. The DMIL with a pre-trained VGG16 backbone and an attention mechanism pooling performed the best, with an accuracy of 0.883 and area under the curve (AUC) of 0.982 on Dataset 1. While using max pooling and convolutional pooling, the AUC was 0.958 and 0.931, respectively. In Dataset 2, the best DMIL model produced an accuracy of 0.833 and AUC of 0.940. Deep learning models based on the MIL can predict chemotherapy response in NSCLC using pretreatment CT images and the pre-trained VGG16 with attention mechanism pooling yielded better predictions.

Development and verification of radiomics framework for computed tomography image segmentation

BACKGROUND

Radiomics has been considered an imaging marker for capturing quantitative image information (QII). The introduction of radiomics to image segmentation is desirable but challenging.

PURPOSE

This study aims to develop and validate a radiomics-based framework for image segmentation (RFIS).

METHODS

RFIS is designed using features extracted from volume (svfeatures) created by sliding window (swvolume). The 53 svfeatures are extracted from 11 phantom series. Outliers in the svfeature datasets are detected by isolation forest (iForest) and specified as the mean value. The percentage coefficient of variation (%COV) is calculated to evaluate the reproducibility of svfeatures. RFIS is constructed and applied to the gross target volume (GTV) segmentation from the peritumoral region (GTV with a 10 mm margin) to assess its feasibility. The 127 lung cancer images are enrolled. The test-retest method, correlation matrix, and Mann-Whitney U test (p < 0.05) are used to select non-redundant svfeatures of statistical significance from the reproducible svfeatures. The synthetic minority over-sampling technique is utilized to balance the minority group in the training sets. The support vector machine is employed for RFIS construction, which is tuned in the training set using 10-fold stratified cross-validation and then evaluated in the test sets. The swvolumes with the consistent classification results are grouped and merged. Mode filtering is performed to remove very small subvolumes and create relatively large regions of completely uniform character. In addition, RFIS performance is evaluated by the area under the receiver operating characteristic (ROC) curve (AUC), accuracy, sensitivity, specificity, and Dice similarity coefficient (DSC).

RESULTS

30249 phantom and 145008 patient image swvolumes were analyzed. Forty-nine (92.45% of 53) svfeatures represented excellent reproducibility(%COV<15). Forty-five features (91.84% of 49) included five categories that passed test-retest analysis. Thirteen svfeatures (28.89% of 45) svfeatures were selected for RFIS construction. RFIS showed an average (95% confidence interval) sensitivity of 0.848 (95% CI:0.844-0.883), a specificity of 0.821 (95% CI: 0.818-0.825), an accuracy of 83.48% (95% CI: 83.27%-83.70%), and an AUC of 0.906 (95% CI: 0.904-0.908) with cross-validation. The sensitivity, specificity, accuracy, and AUC were equal to 0.762 (95% CI: 0.754-0.770), 0.840 (95% CI: 0.837-0.844), 82.29% (95% CI: 81.90%-82.60%), and 0.877 (95% CI: 0.873-0.881) in the test set, respectively. GTV was segmented by grouping and merging swvolume with identical classification results. The mean DSC after mode filtering was 0.707 ± 0.093 in the training sets and 0.688 ± 0.072 in the test sets.

CONCLUSION

Reproducible svfeatures can capture the differences in QII among swvolumes. RFIS can be applied to swvolume classification, which achieves image segmentation by grouping and merging the swvolume with similar QII.

Potential progression biomarkers of diabetic kidney disease determined using comprehensive machine learning analysis of non-targeted metabolomics

Diabetic kidney disease is the main cause of end-stage renal disease worldwide. The prediction of the clinical course of patients with diabetic kidney disease remains difficult, despite the identification of potential biomarkers; therefore, novel biomarkers are needed to predict the progression of the disease. We conducted non-targeted metabolomics using plasma and urine of patients with diabetic kidney disease whose estimated glomerular filtration rate was between 30 and 60 mL/min/1.73 m². We analyzed how the estimated glomerular filtration rate changed over time (up to 30 months) to detect rapid decliners of kidney function. Conventional logistic analysis suggested that only one metabolite, urinary 1-methylpyridin-1-ium (NMP), was a promising biomarker. We then applied a deep learning method to identify potential biomarkers and physiological parameters to predict the progression of diabetic kidney disease in an explainable manner. We narrowed down 3388 variables to 50 using the deep learning method and conducted two regression models, piecewise linear and handcrafted linear regression, both of which examined the utility of biomarker combinations. Our analysis, based on the deep learning method, identified systolic blood pressure and urinary albumin-to-creatinine ratio, six identified metabolites, and three unidentified metabolites including urinary NMP, as potential biomarkers. This research suggests that the machine learning method can detect potential biomarkers that could otherwise escape identification using the conventional statistical method.

Predicting Local Failure after Partial Prostate Re-Irradiation Using a Dosiomic-Based Machine Learning Model

The aim of this study is to predict local failure after partial prostate re-irradiation for the treatment of isolated locally recurrent prostate cancer by using a machine learning classifier based on radiomic features from pre-treatment computed tomography (CT), positron-emission tomography (PET) and biological effective dose distribution (BED) of the radiotherapy plan. The analysis was conducted on a monocentric dataset of 43 patients with evidence of isolated intraprostatic recurrence of prostate cancer after primary external beam radiotherapy. All patients received partial prostate re-irradiation delivered by volumetric modulated arc therapy. The gross tumor volume (GTV) of each patient was manually contoured from planning CT, choline-PET and dose maps. An ensemble machine learning pipeline including unbalanced data correction and feature selection was trained using the radiomic and dosiomic features as input for predicting occurrence of local failure. The model performance was assessed using sensitivity, specificity, accuracy and area under receiver operating characteristic curves of the score function in 10-fold cross validation repeated 100 times. Local failure was observed in 13 patients (30%), with a median time to recurrence of 36.7 months (range = 6.1-102.4 months). A four variables ensemble machine learning model resulted in accuracy of 0.62 and AUC 0.65. According to our results, a dosiomic machine learning classifier can predict local failure after partial prostate re-irradiation.

A role for artificial intelligence in molecular imaging of infection and inflammation

The detection of occult infections and low-grade inflammation in clinical practice remains challenging and much depending on readers’ expertise. Although molecular imaging, like [¹⁸F]FDG PET or radiolabeled leukocyte scintigraphy, offers quantitative and reproducible whole body data on inflammatory responses its interpretation is limited to visual analysis. This often leads to delayed diagnosis and treatment, as well as untapped areas of potential application. Artificial intelligence (AI) offers innovative approaches to mine the wealth of imaging data and has led to disruptive breakthroughs in other medical domains already. Here, we discuss how AI-based tools can improve the detection sensitivity of molecular imaging in infection and inflammation but also how AI might push the data analysis beyond current application toward predicting outcome and long-term risk assessment.

Development of a Hybrid-Imaging-Based Prognostic Index for Metastasized-Melanoma Patients in Whole-Body 18F-FDG PET/CT and PET/MRI Data

Besides tremendous treatment success in advanced melanoma patients, the rapid development of oncologic treatment options comes with increasingly high costs and can cause severe life-threatening side effects. For this purpose, predictive baseline biomarkers are becoming increasingly important for risk stratification and personalized treatment planning. Thus, the aim of this pilot study was the development of a prognostic tool for the risk stratification of the treatment response and mortality based on PET/MRI and PET/CT, including a convolutional neural network (CNN) for metastasized-melanoma patients before systemic-treatment initiation. The evaluation was based on 37 patients (19 f, 62 ± 13 y/o) with unresectable metastasized melanomas who underwent whole-body 18F-FDG PET/MRI and PET/CT scans on the same day before the initiation of therapy with checkpoint inhibitors and/or BRAF/MEK inhibitors. The overall survival (OS), therapy response, metastatically involved organs, number of lesions, total lesion glycolysis, total metabolic tumor volume (TMTV), peak standardized uptake value (SULpeak), diameter (Dmlesion) and mean apparent diffusion coefficient (ADCmean) were assessed. For each marker, a Kaplan−Meier analysis and the statistical significance (Wilcoxon test, paired t-test and Bonferroni correction) were assessed. Patients were divided into high- and low-risk groups depending on the OS and treatment response. The CNN segmentation and prediction utilized multimodality imaging data for a complementary in-depth risk analysis per patient. The following parameters correlated with longer OS: a TMTV < 50 mL; no metastases in the brain, bone, liver, spleen or pleura; ≤4 affected organ regions; no metastases; a Dmlesion > 37 mm or SULpeak < 1.3; a range of the ADCmean < 600 mm2/s. However, none of the parameters correlated significantly with the stratification of the patients into the high- or low-risk groups. For the CNN, the sensitivity, specificity, PPV and accuracy were 92%, 96%, 92% and 95%, respectively. Imaging biomarkers such as the metastatic involvement of specific organs, a high tumor burden, the presence of at least one large lesion or a high range of intermetastatic diffusivity were negative predictors for the OS, but the identification of high-risk patients was not feasible with the handcrafted parameters. In contrast, the proposed CNN supplied risk stratification with high specificity and sensitivity.

Artificial intelligence in lung cancer: Application and future thinking

The availability of medical big data and the rapid development of computer software and hardware have greatly promoted the advancement of intelligent medical healthcare. Artificial intelligence (AI) has been successfully applied in many fields of medicine, especially in lung cancer. The performance of AI in some specific tasks has surpassed that of humans. Several AI software has been deeply used in clinical practice to help decision-making, which is producing a profound influence on clinicians. At present, the application of AI in the field of lung cancer mainly includes detection, segmentation, classification, prognosis prediction, efficacy evaluation, and so on. AI faces certain challenges and opportunities in the era of big data in terms of data acquisition, annotation and interpretability. Researchers have conducted deep and extensive studies using AI in the field of lung cancer, and AI is expected to become a powerful assistant in the prevention and treatment of lung cancer. AI is bringing an unprecedented revolution to radiologists, but the role of radiologists is crucial in the development of AI.

Framework for Testing Robustness of Machine Learning-Based Classifiers

There has been a rapid increase in the number of artificial intelligence (AI)/machine learning (ML)-based biomarker diagnostic classifiers in recent years. However, relatively little work has focused on assessing the robustness of these biomarkers, i.e., investigating the uncertainty of the AI/ML models that these biomarkers are based upon. This paper addresses this issue by proposing a framework to evaluate the already-developed classifiers with regard to their robustness by focusing on the variability of the classifiers' performance and changes in the classifiers' parameter values using factor analysis and Monte Carlo simulations. Specifically, this work evaluates (1) the importance of a classifier's input features and (2) the variability of a classifier's output and model parameter values in response to data perturbations. Additionally, it was found that one can estimate a priori how much replacement noise a classifier can tolerate while still meeting accuracy goals. To illustrate the evaluation framework, six different AI/ML-based biomarkers are developed using commonly used techniques (linear discriminant analysis, support vector machines, random forest, partial-least squares discriminant analysis, logistic regression, and multilayer perceptron) for a metabolomics dataset involving 24 measured metabolites taken from 159 study participants. The framework was able to correctly predict which of the classifiers should be less robust than others without recomputing the classifiers itself, and this prediction was then validated in a detailed analysis.

Epidemiological challenges in pandemic coronavirus disease (COVID-19): Role of artificial intelligence

World is now experiencing a major health calamity due to the coronavirus disease (COVID-19) pandemic, caused by the severe acute respiratory syndrome coronavirus clade 2. The foremost challenge facing the scientific community is to explore the growth and transmission capability of the virus. Use of artificial intelligence (AI), such as deep learning, in (i) rapid disease detection from x-ray or computed tomography (CT) or high-resolution CT (HRCT) images, (ii) accurate prediction of the epidemic patterns and their saturation throughout the globe, (iii) forecasting the disease and psychological impact on the population from social networking data, and (iv) prediction of drug-protein interactions for repurposing the drugs, has attracted much attention. In the present study, we describe the role of various AI-based technologies for rapid and efficient detection from CT images complementing quantitative real-time polymerase chain reaction and immunodiagnostic assays. AI-based technologies to anticipate the current pandemic pattern, prevent the spread of disease, and face mask detection are also discussed. We inspect how the virus transmits depending on different factors. We investigate the deep learning technique to assess the affinity of the most probable drugs to treat COVID-19. This article is categorized under:Application Areas > Health CareAlgorithmic Development > Biological Data MiningTechnologies > Machine Learning.

Quality of Multicenter Studies Using MRI Radiomics for Diagnosing Clinically Significant Prostate Cancer: A Systematic Review

Background: Reproducibility and generalization are major challenges for clinically significant prostate cancer modeling using MRI radiomics. Multicenter data seem indispensable to deal with these challenges, but the quality of such studies is currently unknown. The aim of this study was to systematically review the quality of multicenter studies on MRI radiomics for diagnosing clinically significant PCa. Methods: This systematic review followed the 2020 Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. Multicenter studies investigating the value of MRI radiomics for the diagnosis of clinically significant prostate cancer were included. Quality was assessed using the checklist for artificial intelligence in medical imaging (CLAIM) and the radiomics quality score (RQS). CLAIM consisted of 42 equally important items referencing different elements of good practice AI in medical imaging. RQS consisted of 36 points awarded over 16 items related to good practice radiomics. Final CLAIM and RQS scores were percentage-based, allowing for a total quality score consisting of the average of CLAIM and RQS. Results: Four studies were included. The average total CLAIM score was 74.6% and the average RQS was 52.8%. The corresponding average total quality score (CLAIM + RQS) was 63.7%. Conclusions: A very small number of multicenter radiomics PCa classification studies have been performed with the existing studies being of bad or average quality. Good multicenter studies might increase by encouraging preferably prospective data sharing and paying extra care to documentation in regards to reproducibility and clinical utility.

Prediction of future healthcare expenses of patients from chest radiographs using deep learning: a pilot study

Our objective was to develop deep learning models with chest radiograph data to predict healthcare costs and classify top-50% spenders. 21,872 frontal chest radiographs were retrospectively collected from 19,524 patients with at least 1-year spending data. Among the patients, 11,003 patients had 3 years of cost data, and 1678 patients had 5 years of cost data. Model performances were measured with area under the receiver operating characteristic curve (ROC-AUC) for classification of top-50% spenders and Spearman ρ for prediction of healthcare cost. The best model predicting 1-year (N = 21,872) expenditure achieved ROC-AUC of 0.806 [95% CI 0.793–0.819] for top-50% spender classification and ρ of 0.561 [0.536–0.586] for regression. Similarly, for predicting 3-year (N = 12,395) expenditure, ROC-AUC of 0.771 [0.750–0.794] and ρ of 0.524 [0.489–0.559]; for predicting 5-year (N = 1779) expenditure ROC-AUC of 0.729 [0.667–0.729] and ρ of 0.424 [0.324–0.529]. Our deep learning model demonstrated the feasibility of predicting health care expenditure as well as classifying top 50% healthcare spenders at 1, 3, and 5 year(s), implying the feasibility of combining deep learning with information-rich imaging data to uncover hidden associations that may allude to physicians. Such a model can be a starting point of making an accurate budget in reimbursement models in healthcare industries.