Deep learning and radiomics in precision medicine

A Radiomic-Clinical Model of Contrast-Enhanced Mammography for Breast Cancer Biopsy Outcome Prediction

RATIONALE AND OBJECTIVES

In the USA over 1 million breast biopsies are performed annually. Approximately 9.6% diagnostic exams were given Breast Imaging Reporting and Data System (BI-RADS) ≥4A, most of which are 4A/4B. Contrast-enhanced mammography (CEM) may improve biopsy outcome prediction for this subpopulation, but machine learning-based analysis of CEM is largely unexplored. We aim to develop a machine learning-based analysis of CEM using computer-extracted radiomics and radiologist-assessed descriptors to predict breast biopsy outcomes of BI-RADS 4A/4B/4C or 5 lesions.

MATERIALS AND METHODS

This HIPPA-compliant, IRB-approved study included women in a single institution who had BI-RADS 4A/4B/4C or 5 lesions and underwent CEM imaging prior to biopsy. Logistic regression models were built to predict biopsy outcomes using radiomics features and four radiologist-assessed qualitative descriptors. A cohort of 201 patients was used for model development/training, and an independent group of 86 patients were used as an internal test set. AUC was used to measure model's performance. Positive predictive value (PPV) was assessed on subgroups of BI-RADS 4A or 4B lesions.

RESULTS

Model AUC was 0.90 for radiomics, 0.81 for clinical descriptors and 0.88 for their combination. On patients with an initial BI-RADS 4A or 4B scores, model combining radiomics and clinical descriptors of pre-biopsy CEM increased PPV3 to 18% from the radiologist's 6% for 4A patients, and to 25% from the radiologist's 17% for 4B patients.

CONCLUSION

Machine learning models combining radiomics features and clinical descriptors on CEM can predict breast biopsy outcomes on women with BI-RADS 4A/4B/4C or 5 lesions.

Predicting PD-L1 status in NSCLC patients using deep learning radiomics based on CT images

Radiomics refers to the utilization of automated or semi-automated techniques to extract and analyze numerous quantitative features from medical images, such as computerized tomography (CT) or magnetic resonance imaging (MRI) scans. This study aims to develop a deep learning radiomics (DLR)-based approach for predicting programmed death-ligand 1 (PD-L1) expression in patients with non-small cell lung cancer (NSCLC). Data from 352 NSCLC patients with known PD-L1 expression were collected, of which 48.29% (170/352) were tested positive for PD-L1 expression. Tumor regions of interest (ROI) were semi-automatically segmented based on CT images, and DL features were extracted using Residual Network 50. The least absolute shrinkage and selection operator (LASSO) algorithm was used for feature selection and dimensionality reduction. Seven algorithms were used to build models, and the most optimal ones were identified. A combined model integrating DLR with clinical data was also developed. The predictive performance of each model was evaluated using the area under the curve (AUC) of the receiver operating characteristic (ROC) curve analysis. The DLR model, based on CT images, demonstrated an AUC of 0.85 (95% confidence interval (CI), 0.82-0.88), sensitivity of 0.80 (0.74-0.85), and specificity of 0.73 (0.70-0.77) for predicting PD-L1 status. The integrated model exhibited superior performance, with an AUC of 0.91 (0.87-0.95), sensitivity of 0.85 (0.82-0.89), and specificity of 0.75 (0.72-0.80). Our findings indicate that the DLR model holds promise as a valuable tool for predicting the PD-L1 status in patients with NSCLC, which can greatly assist in clinical decision-making and the selection of personalized treatment strategies.

MRI-based radiomics for predicting pathological complete response after neoadjuvant chemoradiotherapy in locally advanced rectal cancer: a systematic review and meta-analysis

Purpose

To evaluate the value of MRI-based radiomics for predicting pathological complete response (pCR) after neoadjuvant chemoradiotherapy (NCRT) in patients with locally advanced rectal cancer (LARC) through a systematic review and meta-analysis.

Methods

A systematic literature search was conducted in PubMed, Embase, Proquest, Cochrane Library, and Web of Science databases, covering studies up to July 1st, 2024, on the diagnostic accuracy of MRI radiomics for predicting pCR in LARC patients following NCRT. Two researchers independently evaluated and selected studies using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool and the Radiomics Quality Score (RQS) tool. A random-effects model was employed to calculate the pooled sensitivity, specificity, and diagnostic odds ratio (DOR) for MRI radiomics in predicting pCR. Meta-regression and subgroup analyses were performed to explore potential sources of heterogeneity. Statistical analyses were performed using RevMan 5.4, Stata 17.0, and Meta-Disc 1.4.

Results

A total of 35 studies involving 9,696 LARC patients were included in this meta-analysis. The average RQS score of the included studies was 13.91 (range 9.00-24.00), accounting for 38.64% of the total score. According to QUADAS-2, there were risks of bias in patient selection and flow and timing domain, though the overall quality of the studies was acceptable. MRI-based radiomics showed no significant threshold effect in predicting pCR (Spearman correlation coefficient=0.119, P=0.498) but exhibited high heterogeneity (I2≥50%). The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio and DOR were 0.83, 0.82, 5.1, 0.23 and 27.22 respectively, with an area under the summary receiver operating characteristic (sROC) curve of 0.91. According to joint model analysis, publication year, country, multi-magnetic field strength, multi-MRI sequence, ROI structure, contour consistency, feature extraction software, and feature quantity after feature dimensionality reduction were potential sources of heterogeneity. Deeks' funnel plot suggested no significant publication bias (P=0.69).

Conclusions

MRI-based radiomics demonstrates high efficacy for predicting pCR in LARC patients following NCRT, holding significant promise for informing clinical decision-making processes and advancing individualized treatment in rectal cancer patients.

Systematic review registration

https://www.crd.york.ac.uk/prospero/, identifier CRD42024611733.

An interpretable ensemble model combining handcrafted radiomics and deep learning for predicting the overall survival of hepatocellular carcinoma patients after stereotactic body radiation therapy

PURPOSE

Hepatocellular carcinoma (HCC) remains a global health concern, marked by increasing incidence rates and poor outcomes. This study seeks to develop a robust predictive model by integrating radiomics and deep learning features with clinical data to predict 2-year survival in HCC patients treated with stereotactic body radiation therapy (SBRT).

METHODS

This study analyzed a cohort of 186 HCC patients who underwent SBRT. Radiomics features were extracted from CT scans, complemented by collection of clinical data. Training and validation of machine learning models were conducted using nested cross-validation techniques. Deep learning models, leveraging various convolutional neural networks (CNNs), were employed to effectively integrate both image and clinical data. Post-hoc explainability techniques were applied to elucidate the contribution of imaging data to predictive outcomes.

RESULTS

Handcrafted radiomics features demonstrated moderate predictive performance, with area under the receiver operating characteristic curve (AUC) values ranging from 0.59 to 0.72. Deep learning models, harnessing the fusion of image and clinical data, exhibited improved predictive accuracy, with AUC values ranging from 0.71 to 0.81. Notably, the ensemble model, amalgamating handcrafted radiomics and deep learning features with clinical data, demonstrated the most robust predictive capability, achieving an AUC of 0.86 (95% CI: 0.80-0.93).

CONCLUSION

The ensemble model represents a significant advancement, providing a comprehensive tool for predicting survival outcomes in HCC patients undergoing SBRT. The inclusion of interpretability methods such as Grad-CAM enhances transparency and understanding of these complex predictive models.

Radiomics for Predicting the Development of Brain Edema from Normal-Appearing Early Brain-CT After Cardiac Arrest and Return of Spontaneous Circulation

Background: Hypoxic-ischemic brain injury (HIBI) is a feared complication post-cardiac arrest (CA). The timing of brain imaging remains a topic of ongoing debate. Early computed tomography (CT) scans can reveal acute intracranial pathologies but may have limited predictive value due to delayed manifestation of HIBI-related changes. Radiomics analyses present a promising approach to identifying subtle imaging markers, potentially aiding early HIBI detection. Methods: This study retrospectively assessed post-CA patients between 2016 and 2023 who received immediate brain CTs. Patients without acute intracranial pathology on initial scans and who underwent follow-up brain CTs within 14 days post-return of spontaneous circulation (ROSC) were included. Image segmentation involved manual basalganglia segmentation and automated whole-brain segmentation. Radiomics features were calculated using Pyradiomics (v3.0.1) in 3DSlicer (v5.2.2). Feature selection involved reproducibility analysis via ICC and LASSO regression, retaining five features per segmentation method. A logistic regression model for each segmentation method underwent 5-fold cross-validation. Results were summarized with ROC analyses and average sensitivity and specificity. Results: A total of 83 patients (average age: 65 ± 13.3 years, 19 women) with CA and ROSC were included. Follow-up CT scans after 5.2 ± 2.9 days revealed brain edema in 47 patients. The model using manual segmentation achieved an average AUC of 0.76, sensitivity of 0.59, and specificity of 0.78. The automated segmentation model showed an average AUC of 0.66, sensitivity of 0.49, and specificity of 0.68. Conclusions: Radiomics, particularly focused on the basalganglia area in normal-appearing brain CTs after CA and ROSC, may enhance predictive insights for HIBI and the development of brain edema.

AI-powered innovations in pancreatitis imaging: a comprehensive literature synthesis

Early identification of pancreatitis remains a significant clinical diagnostic challenge that impacts patient outcomes. The evolution of quantitative imaging followed by deep learning models has shown great promise in the non-invasive diagnosis of pancreatitis and its complications. We provide an overview of advancements in diagnostic imaging and quantitative imaging methods along with the evolution of artificial intelligence (AI). In this article, we review the current and future states of methodology and limitations of AI in improving clinical support in the context of early detection and management of pancreatitis.

A Systematic Review of the Applications of Deep Learning for the Interpretation of Positron Emission Tomography Images of Patients with Lymphoma

Background: Positron emission tomography (PET) is a valuable tool for the assessment of lymphoma, while artificial intelligence (AI) holds promise as a reliable resource for the analysis of medical images. In this context, we systematically reviewed the applications of deep learning (DL) for the interpretation of lymphoma PET images. Methods: We searched PubMed until 11 September 2024 for studies developing DL models for the evaluation of PET images of patients with lymphoma. The risk of bias and applicability concerns were assessed using the prediction model risk of bias assessment tool (PROBAST). The articles included were categorized and presented based on the task performed by the proposed models. Our study was registered with the international prospective register of systematic reviews, PROSPERO, as CRD42024600026. Results: From 71 papers initially retrieved, 21 studies with a total of 9402 participants were ultimately included in our review. The proposed models achieved a promising performance in diverse medical tasks, namely, the detection and histological classification of lesions, the differential diagnosis of lymphoma from other conditions, the quantification of metabolic tumor volume, and the prediction of treatment response and survival with areas under the curve, F1-scores, and R2 values of up to 0.963, 87.49%, and 0.94, respectively. Discussion: The primary limitations of several studies were the small number of participants and the absence of external validation. In conclusion, the interpretation of lymphoma PET images can reliably be aided by DL models, which are not designed to replace physicians but to assist them in managing large volumes of scans through rapid and accurate calculations, alleviate their workload, and provide them with decision support tools for precise care and improved outcomes.

Assessing Patient Health Dynamics by Comparative CT Analysis: An Automatic Approach to Organ and Body Feature Evaluation

Background/Objectives: The integration of machine learning into the domain of radiomics has revolutionized the approach to personalized medicine, particularly in oncology. Our research presents RadTA (RADiomics Trend Analysis), a novel framework developed to facilitate the automatic analysis of quantitative imaging biomarkers (QIBs) from time-series CT volumes. Methods: RadTA is designed to bridge a technical gap for medical experts and enable sophisticated radiomic analyses without deep learning expertise. The core of RadTA includes an automated command line interface, streamlined image segmentation, comprehensive feature extraction, and robust evaluation mechanisms. RadTA utilizes advanced segmentation models, specifically TotalSegmentator and Body Composition Analysis (BCA), to accurately delineate anatomical structures from CT scans. These models enable the extraction of a wide variety of radiomic features, which are subsequently processed and compared to assess health dynamics across timely corresponding CT series. Results: The effectiveness of RadTA was tested using the HNSCC-3DCT-RT dataset, which includes CT scans from oncological patients undergoing radiation therapy. The results demonstrate significant changes in tissue composition and provide insights into the physical effects of the treatment. Conclusions: RadTA demonstrates a step of clinical adoption in the field of radiomics, offering a user-friendly, robust, and effective tool for the analysis of patient health dynamics. It can potentially also be used for other medical specialties.

Impact of Emerging Deep Learning-Based MR Image Reconstruction Algorithms on Abdominal MRI Radiomic Features

OBJECTIVE

This study aims to evaluate, on one MRI vendor's platform, the impact of deep learning (DL)-based reconstruction techniques on MRI radiomic features compared to conventional image reconstruction techniques.

METHODS

Under IRB approval and informed consent, we prospectively collected undersampled coronal T2-weighted MR images of the abdomen (1.5 T; Philips Healthcare) from 17 pediatric and adult subjects and reconstructed them using a conventional image reconstruction technique (compressed sensitivity encoding [C-SENSE]) and two DL-based reconstruction techniques (SmartSpeed [Philips Healthcare, US FDA cleared] and SmartSpeed with Super Resolution [SmartSpeed-SuperRes, not US FDA cleared to date]). Eight regions of interest (ROIs) across organs/tissues (liver, spleen, kidney, pancreas, fat, and muscle) were manually placed. Eighty-six MRI radiomic features were then extracted. Pearson's correlation coefficients (PCCs) and intraclass correlation coefficients (ICCs) were calculated between (A) C-SENSE versus SmartSpeed, and (B) C-SENSE versus SmartSpeed-SuperRes. To quantify the impact from the perspective of the whole MR image, cross-ROI mean PCCs and ICCs were calculated for individual radiomic features. The impact of image reconstruction on individual radiomic features in different organs/tissues was evaluated using ANOVA analyses.

RESULTS

According to cross-ROI mean PCCs, 50 out of 86 radiomic features were highly correlated (PCC, ≥0.8) between SmartSpeed and C-SENSE, whereas only 15 radiomic features were highly correlated between SmartSpeed-SuperRes and C-SENSE reconstructions. According to cross-ROI mean ICCs, 58 out of 86 radiomic features had high agreements (ICC ≥0.75) between SmartSpeed and C-SENSE, whereas only 9 radiomic features had high agreements between SmartSpeed-SuperRes and C-SENSE reconstructions. For SmartSpeed reconstruction, the psoas muscle ROI appeared to be impacted most with the lowest median (IQR) correlation of 0.57 (0.25). The circular liver ROI was impacted most by SmartSpeed-SuperRes (PCC, 0.60 [0.22]). ANOVA analyses suggest that the impact of DL reconstruction algorithms on radiomic features varies significantly among different organs/tissues ( P < 0.001).

CONCLUSIONS

MRI radiomic features are significantly altered by DL-based reconstruction compared to a conventional reconstruction technique. The impact of DL reconstruction algorithms on radiomic features varies significantly between different organs/tissues.

Deep learning-based automatic pipeline system for predicting lateral cervical lymph node metastasis in patients with papillary thyroid carcinoma using computed tomography: A multi-center study

Objective

The assessment of lateral lymph node metastasis (LLNM) in patients with papillary thyroid carcinoma (PTC) holds great significance. This study aims to develop and evaluate a deep learning-based automatic pipeline system (DLAPS) for diagnosing LLNM in PTC using computed tomography (CT).

Methods

A total of 1,266 lateral lymph nodes (LLNs) from 519 PTC patients who underwent CT examinations from January 2019 to November 2022 were included and divided into training and validation set, internal test set, pooled external test set, and prospective test set. The DLAPS consists of an auto-segmentation network based on RefineNet model and a classification network based on ensemble model (ResNet, Xception, and DenseNet). The performance of the DLAPS was compared with that of manually segmented DL models, the clinical model, and Node Reporting and Data System (Node-RADS). The improvement of radiologists' diagnostic performance under the DLAPS-assisted strategy was explored. In addition, bulk RNA-sequencing was conducted based on 12 LLNs to reveal the underlying biological basis of the DLAPS.

Results

The DLAPS yielded good performance with area under the receiver operating characteristic curve (AUC) of 0.872, 0.910, and 0.822 in the internal, pooled external, and prospective test sets, respectively. The DLAPS significantly outperformed clinical models (AUC 0.731, P<0.001) and Node-RADS (AUC 0.602, P<0.001) in the internal test set. Moreover, the performance of the DLAPS was comparable to that of the manually segmented deep learning (DL) model with AUCs ranging 0.814-0.901 in three test sets. Furthermore, the DLAPS-assisted strategy improved the performance of radiologists and enhanced inter-observer consistency. In clinical situations, the rate of unnecessary LLN dissection decreased from 33.33% to 7.32%. Furthermore, the DLAPS was associated with the cell-cell conjunction in the microenvironment.

Conclusions

Using CT images from PTC patients, the DLAPS could effectively segment and classify LLNs non-invasively, and this system had a good generalization ability and clinical applicability.

Deep-learning features based on F18 fluorodeoxyglucose positron emission tomography/computed tomography (18F-FDG PET/CT) to predict preoperative colorectal cancer lymph node metastasis

AIM

The objective of this study was to create and authenticate a prognostic model for lymph node metastasis (LNM) in colorectal cancer (CRC) that integrates clinical, radiomics, and deep transfer learning features.

MATERIALS AND METHODS

In this study, we analyzed data from 119 CRC patients who underwent F18 fluorodeoxyglucose positron emission tomography/computed tomography (18F-FDG PET/CT) scanning. The patient cohort was divided into training and validation subsets in an 8:2 ratio, with an additional 33 external data points for testing. Initially, we conducted univariate analysis to screen clinical parameters. Radiomics features were extracted from manually drawn images using pyradiomics, and deep-learning features, radiomics features, and clinical features were selected using Least Absolute Shrinkage and Selection Operator (LASSO) regression and Spearman correlation coefficient. We then constructed a model by training a support vector machine (SVM), and evaluated the performance of the prediction model by comparing the area under the curve (AUC), sensitivity, and specificity. Finally, we developed nomograms combining clinical and radiological features for interpretation and analysis.

RESULTS

The deep learning radiomics (DLR) nomogram model, which was developed by integrating deep learning, radiomics, and clinical features, exhibited excellent performance. The area under the curve was (AUC = 0.934, 95% confidence interval [CI]: 0.884-0.983) in the training cohort, (AUC = 0.902, 95% CI: 0.769-1.000) in the validation cohort, and (AUC = 0.836, 95% CI: 0.673-0.998) in the test cohort.

CONCLUSION

We developed a preoperative predictive machine-learning model using deep transfer learning, radiomics, and clinical features to differentiate LNM status in CRC, aiding in treatment decision-making for patients.

Multi-objective Bayesian optimization with enhanced features for adaptively improved glioblastoma partitioning and survival prediction

Glioblastoma, an aggressive brain tumor prevalent in adults, exhibits heterogeneity in its microstructures and vascular patterns. The delineation of its subregions could facilitate the development of region-targeted therapies. However, current unsupervised learning techniques for this task face challenges in reliability due to fluctuations of clustering algorithms, particularly when processing data from diverse patient cohorts. Furthermore, stable clustering results do not guarantee clinical meaningfulness. To establish the clinical relevance of these subregions, we will perform survival predictions using radiomic features extracted from them. Following this, achieving a balance between outcome stability and clinical relevance presents a significant challenge, further exacerbated by the extensive time required for hyper-parameter tuning. In this study, we introduce a multi-objective Bayesian optimization (MOBO) framework, which leverages a Feature-enhanced Auto-Encoder (FAE) and customized losses to assess both the reproducibility of clustering algorithms and the clinical relevance of their outcomes. Specifically, we embed the entirety of these processes within the MOBO framework, modeling both using distinct Gaussian Processes (GPs). The proposed MOBO framework can automatically balance the trade-off between the two criteria by employing bespoke stability and clinical significance losses. Our approach efficiently optimizes all hyper-parameters, including the FAE architecture and clustering parameters, within a few steps. This not only accelerates the process but also consistently yields robust MRI subregion delineations and provides survival predictions with strong statistical validation.

Radiomics analysis of gadoxetic acid-enhanced MRI for evaluating vessels encapsulating tumour clusters in hepatocellular carcinoma

Purpose

The aim of this study was to develop an integrated model that combines clinical-radiologic and radiomics features based on gadoxetic acid-enhanced MRI for preoperative evaluating of vessels encapsulating tumour clusters (VETC) patterns in hepatocellular carcinoma (HCC).

Methods

This retrospective study encompassed 234 patients who underwent surgical resection. Among them, 101 patients exhibited VETC-positive HCC, while 133 patients displayed VETC-negative HCC. Volumes of interest were manually delineated for entire tumour regions in the arterial phase (AP), portal phase (PP), and hepatobiliary phase (HBP) images. Independent predictors for VETC were identified through least absolute shrinkage and selection operator (LASSO) regression and multivariable logistic regression analysis, utilising radiomics-AP, PP, HBP, along with 24 imaging features and 19 clinical characteristics. Subsequently, the clinico-radiologic model, radiomics model, and integrated model were established, with a nomogram visualising the integrated model. The performance for VETC prediction was evaluated using a receiver operating characteristic curve.

Results

The integrated model, composed of 3 selected traditional imaging features (necrosis or severe ischemia [OR=2.457], peripheral washout [OR=1.678], LLR_AP (Lesion to liver ratio_AP) [OR=0.433] and radiomics-AP [OR=2.870], radiomics-HBP [OR=2.023], radiomics-PP [OR=1.546]), showcased good accuracy in predicting VETC patterns in both the training (AUC=0.873, 95% confidence interval [CI]: 0.821-0.925)) and validation (AUC=0.869, 95% CI:0.789-0.950) cohorts.

Conclusion

This study established an integrated model that combines traditional imaging features and radiomic features from gadoxetic acid-enhanced MRI, demonstrating good performance in predicting VETC patterns.

Differentiation of granulomatous nodules with lobulation and spiculation signs from solid lung adenocarcinomas using a CT deep learning model

BACKGROUND

The diagnosis of solitary pulmonary nodules has always been a difficult and important point in clinical research, especially granulomatous nodules (GNs) with lobulation and spiculation signs, which are easily misdiagnosed as malignant tumors. Therefore, in this study, we utilised a CT deep learning (DL) model to distinguish GNs with lobulation and spiculation signs from solid lung adenocarcinomas (LADCs), to improve the diagnostic accuracy of preoperative diagnosis.

METHODS

420 patients with pathologically confirmed GNs and LADCs from three medical institutions were retrospectively enrolled. The regions of interest in non-enhanced CT (NECT) and venous contrast-enhanced CT (VECT) were identified and labeled, and self-supervised labels were constructed. Cases from institution 1 were randomly divided into a training set (TS) and an internal validation set (IVS), and cases from institutions 2 and 3 were treated as an external validation set (EVS). Training and validation were performed using self-supervised transfer learning, and the results were compared with the radiologists' diagnoses.

RESULTS

The DL model achieved good performance in distinguishing GNs and LADCs, with area under curve (AUC) values of 0.917, 0.876, and 0.896 in the IVS and 0.889, 0.879, and 0.881 in the EVS for NECT, VECT, and non-enhanced with venous contrast-enhanced CT (NEVECT) images, respectively. The AUCs of radiologists 1, 2, 3, and 4 were, respectively, 0.739, 0.783, 0.883, and 0.901 in the (IVS) and 0.760, 0.760, 0.841, and 0.844 in the EVS.

CONCLUSIONS

A CT DL model showed great value for preoperative differentiation of GNs with lobulation and spiculation signs from solid LADCs, and its predictive performance was higher than that of radiologists.

MRI-based prostate cancer classification using 3D efficient capsule network

BACKGROUND

Prostate cancer (PCa) is the most common cancer in men and the second leading cause of male cancer-related death. Gleason score (GS) is the primary driver of PCa risk-stratification and medical decision-making, but can only be assessed at present via biopsy under anesthesia. Magnetic resonance imaging (MRI) is a promising non-invasive method to further characterize PCa, providing additional anatomical and functional information. Meanwhile, the diagnostic power of MRI is limited by qualitative or, at best, semi-quantitative interpretation criteria, leading to inter-reader variability.

PURPOSES

Computer-aided diagnosis employing quantitative MRI analysis has yielded promising results in non-invasive prediction of GS. However, convolutional neural networks (CNNs) do not implicitly impose a frame of reference to the objects. Thus, CNNs do not encode the positional information properly, limiting method robustness against simple image variations such as flipping, scaling, or rotation. Capsule network (CapsNet) has been proposed to address this limitation and achieves promising results in this domain. In this study, we develop a 3D Efficient CapsNet to stratify GS-derived PCa risk using T2-weighted (T2W) MRI images.

METHODS

In our method, we used 3D CNN modules to extract spatial features and primary capsule layers to encode vector features. We then propose to integrate fully-connected capsule layers (FC Caps) to create a deeper hierarchy for PCa grading prediction. FC Caps comprises a secondary capsule layer which routes active primary capsules and a final capsule layer which outputs PCa risk. To account for data imbalance, we propose a novel dynamic weighted margin loss. We evaluate our method on a public PCa T2W MRI dataset from the Cancer Imaging Archive containing data from 976 patients.

RESULTS

Two groups of experiments were performed: (1) we first identified high-risk disease by classifying low + medium risk versus high risk; (2) we then stratified disease in one-versus-one fashion: low versus high risk, medium versus high risk, and low versus medium risk. Five-fold cross validation was performed. Our model achieved an area under receiver operating characteristic curve (AUC) of 0.83 and 0.64 F1-score for low versus high grade, 0.79 AUC and 0.75 F1-score for low + medium versus high grade, 0.75 AUC and 0.69 F1-score for medium versus high grade and 0.59 AUC and 0.57 F1-score for low versus medium grade. Our method outperformed state-of-the-art radiomics-based classification and deep learning methods with the highest metrics for each experiment. Our divide-and-conquer strategy achieved weighted Cohen's Kappa score of 0.41, suggesting moderate agreement with ground truth PCa risks.

CONCLUSIONS

In this study, we proposed a novel 3D Efficient CapsNet for PCa risk stratification and demonstrated its feasibility. This developed tool provided a non-invasive approach to assess PCa risk from T2W MR images, which might have potential to personalize the treatment of PCa and reduce the number of unnecessary biopsies.

Integration of deep learning and habitat radiomics for predicting the response to immunotherapy in NSCLC patients

BACKGROUND

The non-invasive biomarkers for predicting immunotherapy response are urgently needed to prevent both premature cessation of treatment and ineffective extension. This study aimed to construct a non-invasive model for predicting immunotherapy response, based on the integration of deep learning and habitat radiomics in patients with advanced non-small cell lung cancer (NSCLC).

METHODS

Independent patient cohorts from three medical centers were enrolled for training (n = 164) and test (n = 82). Habitat imaging radiomics features were derived from sub-regions clustered from individual's tumor by K-means method. The deep learning features were extracted based on 3D ResNet algorithm. Pearson correlation coefficient, T test and least absolute shrinkage and selection operator regression were used to select features. Support vector machine was applied to implement deep learning and habitat radiomics, respectively. Then, a combination model was developed integrating both sources of data.

RESULTS

The combination model obtained a strong well-performance, achieving area under receiver operating characteristics curve of 0.865 (95% CI 0.772-0.931). The model significantly discerned high and low-risk patients, and exhibited a significant benefit in the clinical use.

CONCLUSION

The integration of deep-leaning and habitat radiomics contributed to predicting response to immunotherapy in patients with NSCLC. The developed integration model may be used as potential tool for individual immunotherapy management.

Multitask deep learning for prediction of microvascular invasion and recurrence-free survival in hepatocellular carcinoma based on MRI images

BACKGROUND AND AIMS

Accurate preoperative prediction of microvascular invasion (MVI) and recurrence-free survival (RFS) is vital for personalised hepatocellular carcinoma (HCC) management. We developed a multitask deep learning model to predict MVI and RFS using preoperative MRI scans.

METHODS

Utilising a retrospective dataset of 725 HCC patients from seven institutions, we developed and validated a multitask deep learning model focused on predicting MVI and RFS. The model employs a transformer architecture to extract critical features from preoperative MRI scans. It was trained on a set of 234 patients and internally validated on a set of 58 patients. External validation was performed using three independent sets (n = 212, 111, 110).

RESULTS

The multitask deep learning model yielded high MVI prediction accuracy, with AUC values of 0.918 for the training set and 0.800 for the internal test set. In external test sets, AUC values were 0.837, 0.815 and 0.800. Radiologists' sensitivity and inter-rater agreement for MVI prediction improved significantly when integrated with the model. For RFS, the model achieved C-index values of 0.763 in the training set and ranged between 0.628 and 0.728 in external test sets. Notably, PA-TACE improved RFS only in patients predicted to have high MVI risk and low survival scores (p < .001).

CONCLUSIONS

Our deep learning model allows accurate MVI and survival prediction in HCC patients. Prospective studies are warranted to assess the clinical utility of this model in guiding personalised treatment in conjunction with clinical criteria.

Preoperative prediction of microsatellite instability status: development and validation of a pan-cancer PET/CT-based radiomics model

OBJECTIVE

The purpose of this study is to verify the feasibility of preoperative prediction of patients' microsatellite instability status by applying a PET/CT-based radiation model.

METHODS

This retrospective study ultimately included 142 patients. Three prediction models have been developed. The predictive performance of all models was evaluated by the receiver operating characteristic curve and area under the curve values. The PET/CT radiological histology score (Radscore) was calculated to evaluate the microsatellite instability status, and the corresponding nomogram was established. The correlation between clinical factors and radiological characteristics was analyzed to verify the value of radiological characteristics in predicting microsatellite instability status.

RESULTS

Twelve features were retained to establish a comprehensive prediction model of radiological and clinical features. M phase of the tumor has been proven to be an independent predictor of microsatellite instability status. The receiver operating characteristic results showed that the area under the curve values of the training set and the validation set of the radiomics model were 0.82 and 0.75, respectively. The sensitivity, specificity, positive predictive value and negative predictive value of the training set were 0.72, 0.78, 0.83 and 0.66, respectively. The sensitivity, specificity, positive predictive value and negative predictive value of the validation set were 1.00, 0.50, 0.76 and 1.00, respectively. The risk of patients with microsatellite instability was calculated by Radscore and nomograph, and the cutoff value was -0.4385. The validity of the results was confirmed by the decision and calibration curves.

CONCLUSION

Radiological models based on PET/CT can provide clinical and practical noninvasive prediction of microsatellite instability status of several different cancer types, reducing or avoiding unnecessary biopsy to a certain extent.

A Multiplanar Radiomics Model Based on Cranial Ultrasound to Predict the White Matter Injury in Premature Infants and an Analysis of its Correlation With Neurodevelopment

OBJECTIVES

To develop and evaluate a multiplanar radiomics model based on cranial ultrasound (CUS) to predict white matter injury (WMI) in premature infants and explore its correlation with neurodevelopment.

METHODS

We retrospectively reviewed 267 premature infants. The radiomics features were extracted from five standard sections of CUS. The Spearman's correlation coefficient combined with the least absolute shrinkage and selection operator (LASSO) was applied to select features and build radiomics signature, and a multiplanar radiomics model was constructed based on the radiomics signature of five planes. The performance of the model was evaluated using the area under the receiver operating characteristic curve (AUC). Infants with WMI were re-examined by ultrasound at 2 and 4 weeks after birth, and the recovery degree of WMI was evaluated using multiplanar radiomics. The relationship between WMI and the recovery degree and neurodevelopment was analyzed.

RESULTS

The AUC of the multiplanar radiomics in the training and validation sets were 0.94 and 0.91, respectively. The neurodevelopmental function scores in infants with WMI were significantly lower than those in healthy preterm infants and full-term newborns (P < .001). There were statistically significant differences in the neurodevelopmental function scores of infants between the 2- and 4-week lesion disappearance and 4-week lesion persistence (P < .001).

CONCLUSIONS

The multiplanar radiomics model showed a good performance in predicting the WMI of premature infants. It can not only provide objective and accurate results but also dynamically monitor the degree of recovery of WMI to predict the prognosis of premature infants.

A Monte Carlo simulation-based decision support system for radiation oncologists in the treatment of glioblastoma multiforme

In the present research, we have developed a model-based crisp logic function statistical classifier decision support system supplemented with treatment planning systems for radiation oncologists in the treatment of glioblastoma multiforme (GBM). This system is based on Monte Carlo radiation transport simulation and it recreates visualization of treatment environments on mathematical anthropomorphic brain (MAB) phantoms. Energy deposition within tumour tissue and normal tissues are graded by quality audit factors which ensure planned dose delivery to tumour site thereby minimising damages to healthy tissues. The proposed novel methodology predicts tumour growth response to radiation therapy from a patient-specific medicine quality audit perspective. Validation of the study was achieved by recreating thirty-eight patient-specific mathematical anthropomorphic brain phantoms of treatment environments by taking into consideration density variation and composition of brain tissues. Dose computations accomplished through water phantom, tissue-equivalent head phantoms are neither cost-effective, nor patient-specific customized and is often less accurate. The above-highlighted drawbacks can be overcome by using open-source Electron Gamma Shower (EGSnrc) software and clinical case reports for MAB phantom synthesis which would result in accurate dosimetry with due consideration to the time factors. Considerable dose deviations occur at the tumour site for environments with intraventricular glioblastoma, haematoma, abscess, trapped air and cranial flaps leading to quality factors with a lower logic value of 0. Logic value of 1 depicts higher dose deposition within healthy tissues and also leptomeninges for majority of the environments which results in radiation-induced laceration.

Application of artificial intelligence in the prediction of immunotherapy efficacy in hepatocellular carcinoma: Current status and prospects

Artificial Intelligence (AI) has increased as a potent tool in medicine, with promising oncology applications. The emergence of immunotherapy has transformed the treatment terrain for hepatocellular carcinoma (HCC), offering new hope to patients with this challenging malignancy. This article examines the role and future of AI in forecasting the effectiveness of immunotherapy in HCC. We highlight the potential of AI to revolutionize the prediction of therapy response, thus improving patient selection and clinical outcomes. The article further outlines the challenges and future research directions in this emerging field.

Non-invasive prediction for pathologic complete response to neoadjuvant chemoimmunotherapy in lung cancer using CT-based deep learning: a multicenter study

Neoadjuvant chemoimmunotherapy has revolutionized the therapeutic strategy for non-small cell lung cancer (NSCLC), and identifying candidates likely responding to this advanced treatment is of important clinical significance. The current multi-institutional study aims to develop a deep learning model to predict pathologic complete response (pCR) to neoadjuvant immunotherapy in NSCLC based on computed tomography (CT) imaging and further prob the biologic foundation of the proposed deep learning signature. A total of 248 participants administrated with neoadjuvant immunotherapy followed by surgery for NSCLC at Ruijin Hospital, Ningbo Hwamei Hospital, and Affiliated Hospital of Zunyi Medical University from January 2019 to September 2023 were enrolled. The imaging data within 2 weeks prior to neoadjuvant chemoimmunotherapy were retrospectively extracted. Patients from Ruijin Hospital were grouped as the training set (n = 104) and the validation set (n = 69) at the 6:4 ratio, and other participants from Ningbo Hwamei Hospital and Affiliated Hospital of Zunyi Medical University served as an external cohort (n = 75). For the entire population, pCR was obtained in 29.4% (n = 73) of cases. The areas under the curve (AUCs) of our deep learning signature for pCR prediction were 0.775 (95% confidence interval [CI]: 0.649 - 0.901) and 0.743 (95% CI: 0.618 - 0.869) in the validation set and the external cohort, significantly superior than 0.579 (95% CI: 0.468 - 0.689) and 0.569 (95% CI: 0.454 - 0.683) of the clinical model. Furthermore, higher deep learning scores correlated to the upregulation for pathways of cell metabolism and more antitumor immune infiltration in microenvironment. Our developed deep learning model is capable of predicting pCR to neoadjuvant chemoimmunotherapy in patients with NSCLC.

Practical Applications of Artificial Intelligence in Spine Imaging: A Review

Artificial intelligence (AI), a transformative technology with unprecedented potential in medical imaging, can be applied to various spinal pathologies. AI-based approaches may improve imaging efficiency, diagnostic accuracy, and interpretation, which is essential for positive patient outcomes. This review explores AI algorithms, techniques, and applications in spine imaging, highlighting diagnostic impact and challenges with future directions for integrating AI into spine imaging workflow.

Dual-Energy Computed Tomography in Detecting and Predicting Lymph Node Metastasis in Malignant Tumor Patients: A Comprehensive Review

The accurate and timely assessment of lymph node involvement is paramount in the management of patients with malignant tumors, owing to its direct correlation with cancer staging, therapeutic strategy formulation, and prognostication. Dual-energy computed tomography (DECT), as a burgeoning imaging modality, has shown promising results in the diagnosis and prediction of preoperative metastatic lymph nodes in recent years. This article aims to explore the application of DECT in identifying metastatic lymph nodes (LNs) across various cancer types, including but not limited to thyroid carcinoma (focusing on papillary thyroid carcinoma), lung cancer, and colorectal cancer. Through this narrative review, we aim to elucidate the clinical relevance and utility of DECT in the detection and predictive assessment of lymph node metastasis in malignant tumors, thereby contributing to the broader academic discourse in oncologic radiology and diagnostic precision.

How artificial intelligence is reshaping the autonomy and boundary work of radiologists. A qualitative study

The application of artificial intelligence (AI) in medical practice is spreading, especially in technologically dense fields such as radiology, which could consequently undergo profound transformations in the near future. This article aims to qualitatively explore the potential influence of AI technologies on the professional identity of radiologists. Drawing on 12 in-depth interviews with a subgroup of radiologists who participated in a larger study, this article investigated (1) whether radiologists perceived AI as a threat to their decision-making autonomy; and (2) how radiologists perceived the future of their profession compared to other health-care professions. The findings revealed that while AI did not generally affect radiologists' decision-making autonomy, it threatened their professional and epistemic authority. Two discursive strategies were identified to explain these findings. The first strategy emphasised radiologists' specific expertise and knowledge that extends beyond interpreting images, a task performed with high accuracy by AI machines. The second strategy underscored the fostering of radiologists' professional prestige through developing expertise in using AI technologies, a skill that would distinguish them from other clinicians who did not pose this knowledge. This study identifies AI machines as status objects and useful tools in performing boundary work in and around the radiological profession.

Convolutional Neural Networks for Glioma Segmentation and Prognosis: A Systematic Review

Deep learning (DL) is poised to redefine the way medical images are processed and analyzed. Convolutional neural networks (CNNs), a specific type of DL architecture, are exceptional for high-throughput processing, allowing for the effective extraction of relevant diagnostic patterns from large volumes of complex visual data. This technology has garnered substantial interest in the field of neuro-oncology as a promising tool to enhance medical imaging throughput and analysis. A multitude of methods harnessing MRI-based CNNs have been proposed for brain tumor segmentation, classification, and prognosis prediction. They are often applied to gliomas, the most common primary brain cancer, to classify subtypes with the goal of guiding therapy decisions. Additionally, the difficulty of repeating brain biopsies to evaluate treatment response in the setting of often confusing imaging findings provides a unique niche for CNNs to help distinguish the treatment response to gliomas. For example, glioblastoma, the most aggressive type of brain cancer, can grow due to poor treatment response, can appear to grow acutely due to treatment-related inflammation as the tumor dies (pseudo-progression), or falsely appear to be regrowing after treatment as a result of brain damage from radiation (radiation necrosis). CNNs are being applied to separate this diagnostic dilemma. This review provides a detailed synthesis of recent DL methods and applications for intratumor segmentation, glioma classification, and prognosis prediction. Furthermore, this review discusses the future direction of MRI-based CNN in the field of neuro-oncology and challenges in model interpretability, data availability, and computation efficiency.

Radiologic imaging biomarkers in triple-negative breast cancer: a literature review about the role of artificial intelligence and the way forward

Breast cancer is one of the most common and deadly cancers in women. Triple-negative breast cancer (TNBC) accounts for approximately 10%-15% of breast cancer diagnoses and is an aggressive molecular breast cancer subtype associated with important challenges in its diagnosis, treatment, and prognostication. This poses an urgent need for developing more effective and personalized imaging biomarkers for TNBC. Towards this direction, artificial intelligence (AI) for radiologic imaging holds a prominent role, leveraging unique advantages of radiologic breast images, being used routinely for TNBC diagnosis, staging, and treatment planning, and offering high-resolution whole-tumour visualization, combined with the immense potential of AI to elucidate anatomical and functional properties of tumours that may not be easily perceived by the human eye. In this review, we synthesize the current state-of-the-art radiologic imaging applications of AI in assisting TNBC diagnosis, treatment, and prognostication. Our goal is to provide a comprehensive overview of radiomic and deep learning-based AI developments and their impact on advancing TNBC management over the last decade (2013-2024). For completeness of the review, we start with a brief introduction of AI, radiomics, and deep learning. Next, we focus on clinically relevant AI-based diagnostic, predictive, and prognostic models for radiologic breast images evaluated in TNBC. We conclude with opportunities and future directions for AI towards advancing diagnosis, treatment response predictions, and prognostic evaluations for TNBC.

Use of a radiomics-clinical model based on magnetic diffusion-weighted imaging for preoperative prediction of lymph node metastasis in rectal cancer patients

Rectal cancer is the eighth most prevalent malignancy worldwide with a 3.2% mortality rate and 3.9% incidence rate. Radiologists still have difficulty in correctly diagnosing lymph node metastases that have been suspected preoperatively. To assess the effectiveness of a model combining clinical and radiomics features for the preoperative prediction of lymph node metastasis in rectal cancer. We retrospectively analyzed data from 104 patients with rectal cancer. All patients were selected as samples for the training (n = 72) and validation cohorts (n = 32). Lymph nodes (LNs) in diffusion-weighted images were analyzed to obtain 842 radiomic characteristics, which were then used to draw the region of interest. Logistic regression, least absolute shrinkage and selection operator, and between-group and within-group correlation analyses were combined to establish the radiomic score (rad-score). Receiver operating characteristic curves were used to estimate the prediction accuracy of the model. A calibration curve was constructed to test the predictive ability of the model. A decision curve analysis was performed to analyze the model's value in clinical application. The area under the curve for the radiomics-clinical, clinical, and radiomics models was 0.856, 0.810, and 0.781, respectively, in the training cohort and 0.880, 0.849, and 0.827, respectively, in the validation cohort. The calibration curve and DCA showed that the radiomics-clinical prediction model had good prediction accuracy, which was higher than that of the other models. The radiomics-clinical model showed a favorable predictive performance for the preoperative prediction of LN metastasis in patients with rectal cancer.

Convolutional neural network-based classification and monitoring models for lung cancer detection: 3D perspective approach

Recent developments in technology and research have offered a wide variety of new techniques for image and data analysis within the medical field. Medical research helps doctors and researchers acquire not only knowledge about health and new diseases, but also techniques of prevention and treatment. In particular, radiomic analysis is mainly used to extract quantitative data from medical images and to build a model strong enough to diagnose focal diseases. However, finding a model capable to fit all patient situations is not an easy task. In this paper frame prediction models and classification models are reported in order to predict the evolution of a given data series and determine whether an anomaly exists or not. This article also shows how to build and make use of a convolutional neural network-based architecture aiming to accomplish prediction task for medical images, not only using common computer tomography scans, but also 3D volumes.

The use of deep learning models to predict progression-free survival in patients with neuroendocrine tumors

Aim: The RAISE project assessed whether deep learning could improve early progression-free survival (PFS) prediction in patients with neuroendocrine tumors. Patients & methods: Deep learning models extracted features from CT scans from patients in CLARINET (NCT00353496) (n = 138/204). A Cox model assessed PFS prediction when combining deep learning with the sum of longest diameter ratio (SLDr) and logarithmically transformed CgA concentration (logCgA), versus SLDr and logCgA alone. Results: Deep learning models extracted features other than lesion shape to predict PFS at week 72. No increase in performance was achieved with deep learning versus SLDr and logCgA models alone. Conclusion: Deep learning models extracted relevant features to predict PFS, but did not improve early prediction based on SLDr and logCgA.

Acute coronary event (ACE) prediction following breast radiotherapy by features extracted from 3D CT, dose, and cardiac structures

PURPOSE

Heart toxicity, such as major acute coronary events (ACE), following breast radiation therapy (RT) is of utmost concern. Thus, many studies have been investigating the effect of mean heart dose (MHD) and dose received in heart sub-structures on toxicity. Most studies focused on the dose thresholds in the heart and its sub-structures, while few studies adopted such computational methods as deep neural networks (DNN) and radiomics. This work aims to construct a feature-driven predictive model for ACE after breast RT.

METHODS

A recently proposed two-step predictive model that extracts a number of features from a deep auto-segmentation network and processes the selected features for prediction was adopted. This work refined the auto-segmenting network and feature processing algorithms to enhance performance in cardiac toxicity prediction. In the predictive model, the deep convolutional neural network (CNN) extracted features from 3D computed tomography (CT) images and dose distributions in three automatically segmented heart sub-structures, including the left anterior descending artery (LAD), right coronary artery (RCA), and left ventricle (LV). The optimal feature processing workflow for the extracted features was explored to enhance the prediction accuracy. The regions associated with toxicity were visualized using a class activation map (CAM)-based technique. Our proposed model was validated against a conventional DNN (convolutional and fully connected layers) and radiomics with a patient cohort of 84 cases, including 29 and 55 patient cases with and without ACE. Of the entire 84 cases, 12 randomly chosen cases (5 toxicity and 7 non-toxicity cases) were set aside for independent test, and the remaining 72 cases were applied to 4-fold stratified cross-validation.

RESULTS

Our predictive model outperformed the conventional DNN by 38% and 10% and radiomics-based predictive models by 9% and 10% in AUC for 4-fold cross-validations and independent test, respectively. The degree of enhancement was greater when incorporating dose information and heart sub-structures into feature extraction. The model whose inputs were CT, dose, and three sub-structures (LV, LAD, and RCA) reached 96% prediction accuracy on average and 0.94 area under the curve (AUC) on average in the cross-validation, and also achieved prediction accuracy of 83% and AUC of 0.83 in the independent test. On 10 correctly predicted cases out of 12 for the independent test, the activation maps implied that for cases of ACE toxicity, the higher intensity was more likely to be observed inside the LV.

CONCLUSIONS

The proposed model characterized by modifications in model input with dose distributions and cardiac sub-structures, and serial processing of feature extraction and feature selection techniques can improve the predictive performance in ACE following breast RT.

Deep radiomic model based on the sphere-shell partition for predicting treatment response to chemotherapy in lung cancer

BACKGROUND

The prognosis of chemotherapy is important in clinical decision-making for non-small cell lung cancer (NSCLC) patients.

OBJECTIVES

To develop a model for predicting treatment response to chemotherapy in NSCLC patients from pre-chemotherapy CT images.

MATERIALS AND METHODS

This retrospective multicenter study enrolled 485 patients with NSCLC who received chemotherapy alone as a first-line treatment. Two integrated models were developed using radiomic and deep-learning-based features. First, we partitioned pre-chemotherapy CT images into spheres and shells with different radii around the tumor (0-3, 3-6, 6-9, 9-12, 12-15 mm) containing intratumoral and peritumoral regions. Second, we extracted radiomic and deep-learning-based features from each partition. Third, using radiomic features, five sphere-shell models, one feature fusion model, and one image fusion model were developed. Finally, the model with the best performance was validated in two cohorts.

RESULTS

Among the five partitions, the model of 9-12 mm achieved the highest area under the curve (AUC) of 0.87 (95% confidence interval: 0.77-0.94). The AUC was 0.94 (0.85-0.98) for the feature fusion model and 0.91 (0.82-0.97) for the image fusion model. For the model integrating radiomic and deep-learning-based features, the AUC was 0.96 (0.88-0.99) for the feature fusion method and 0.94 (0.85-0.98) for the image fusion method. The best-performing model had an AUC of 0.91 (0.81-0.97) and 0.89 (0.79-0.93) in two validation sets, respectively.

CONCLUSIONS

This integrated model can predict the response to chemotherapy in NSCLC patients and assist physicians in clinical decision-making.

Adversarial Attack and Defense in Breast Cancer Deep Learning Systems

Deep-learning-assisted medical diagnosis has brought revolutionary innovations to medicine. Breast cancer is a great threat to women's health, and deep-learning-assisted diagnosis of breast cancer pathology images can save manpower and improve diagnostic accuracy. However, researchers have found that deep learning systems based on natural images are vulnerable to attacks that can lead to errors in recognition and classification, raising security concerns about deep systems based on medical images. We used the adversarial attack algorithm FGSM to reveal that breast cancer deep learning systems are vulnerable to attacks and thus misclassify breast cancer pathology images. To address this problem, we built a deep learning system for breast cancer pathology image recognition with better defense performance. Accurate diagnosis of medical images is related to the health status of patients. Therefore, it is very important and meaningful to improve the security and reliability of medical deep learning systems before they are actually deployed.

Advancements in MRI-Based Radiomics and Artificial Intelligence for Prostate Cancer: A Comprehensive Review and Future Prospects

The use of multiparametric magnetic resonance imaging (mpMRI) has become a common technique used in guiding biopsy and developing treatment plans for prostate lesions. While this technique is effective, non-invasive methods such as radiomics have gained popularity for extracting imaging features to develop predictive models for clinical tasks. The aim is to minimize invasive processes for improved management of prostate cancer (PCa). This study reviews recent research progress in MRI-based radiomics for PCa, including the radiomics pipeline and potential factors affecting personalized diagnosis. The integration of artificial intelligence (AI) with medical imaging is also discussed, in line with the development trend of radiogenomics and multi-omics. The survey highlights the need for more data from multiple institutions to avoid bias and generalize the predictive model. The AI-based radiomics model is considered a promising clinical tool with good prospects for application.

Radiomics using computed tomography to predict CD73 expression and prognosis of colorectal cancer liver metastases

BACKGROUND

Finding a noninvasive radiomic surrogate of tumor immune features could help identify patients more likely to respond to novel immune checkpoint inhibitors. Particularly, CD73 is an ectonucleotidase that catalyzes the breakdown of extracellular AMP into immunosuppressive adenosine, which can be blocked by therapeutic antibodies. High CD73 expression in colorectal cancer liver metastasis (CRLM) resected with curative intent is associated with early recurrence and shorter patient survival. The aim of this study was hence to evaluate whether machine learning analysis of preoperative liver CT-scan could estimate high vs low CD73 expression in CRLM and whether such radiomic score would have a prognostic significance.

METHODS

We trained an Attentive Interpretable Tabular Learning (TabNet) model to predict, from preoperative CT images, stratified expression levels of CD73 (CD73^High vs. CD73^Low) assessed by immunofluorescence (IF) on tissue microarrays. Radiomic features were extracted from 160 segmented CRLM of 122 patients with matched IF data, preprocessed and used to train the predictive model. We applied a five-fold cross-validation and validated the performance on a hold-out test set.

RESULTS

TabNet provided areas under the receiver operating characteristic curve of 0.95 (95% CI 0.87 to 1.0) and 0.79 (0.65 to 0.92) on the training and hold-out test sets respectively, and outperformed other machine learning models. The TabNet-derived score, termed rad-CD73, was positively correlated with CD73 histological expression in matched CRLM (Spearman's ρ = 0.6004; P < 0.0001). The median time to recurrence (TTR) and disease-specific survival (DSS) after CRLM resection in rad-CD73^High vs rad-CD73^Low patients was 13.0 vs 23.6 months (P = 0.0098) and 53.4 vs 126.0 months (P = 0.0222), respectively. The prognostic value of rad-CD73 was independent of the standard clinical risk score, for both TTR (HR = 2.11, 95% CI 1.30 to 3.45, P < 0.005) and DSS (HR = 1.88, 95% CI 1.11 to 3.18, P = 0.020).

CONCLUSIONS

Our findings reveal promising results for non-invasive CT-scan-based prediction of CD73 expression in CRLM and warrant further validation as to whether rad-CD73 could assist oncologists as a biomarker of prognosis and response to immunotherapies targeting the adenosine pathway.

Within-Modality Synthesis and Novel Radiomic Evaluation of Brain MRI Scans

One of the most common challenges in brain MRI scans is to perform different MRI sequences depending on the type and properties of tissues. In this paper, we propose a generative method to translate T2-Weighted (T2W) Magnetic Resonance Imaging (MRI) volume from T2-weight-Fluid-attenuated-Inversion-Recovery (FLAIR) and vice versa using Generative Adversarial Networks (GAN). To evaluate the proposed method, we propose a novel evaluation schema for generative and synthetic approaches based on radiomic features. For the evaluation purpose, we consider 510 pair-slices from 102 patients to train two different GAN-based architectures Cycle GAN and Dual Cycle-Consistent Adversarial network (DC2Anet). The results indicate that generative methods can produce similar results to the original sequence without significant change in the radiometric feature. Therefore, such a method can assist clinics to make decisions based on the generated image when different sequences are not available or there is not enough time to re-perform the MRI scans.

Application of radiomics to meningiomas: A systematic review

BACKGROUND

Quantitative imaging analysis through radiomics is a powerful technology to non-invasively assess molecular correlates and guide clinical decision-making. There has been growing interest in image-based phenotyping for meningiomas given the complexities in management.

METHODS

We systematically reviewed meningioma radiomics analyses published in PubMed, Embase, and Web of Science until December 20, 2021. We compiled performance data and assessed publication quality using the radiomics quality score (RQS).

RESULTS

A total of 170 publications were grouped into 5 categories of radiomics applications to meningiomas: Tumor detection and segmentation (21%), classification across neurologic diseases (54%), grading (14%), feature correlation (3%), and prognostication (8%). A majority focused on technical model development (73%) versus clinical applications (27%), with increasing adoption of deep learning. Studies utilized either private institutional (50%) or public (49%) datasets, with only 68% using a validation dataset. For detection and segmentation, radiomic models had a mean accuracy of 93.1 ± 8.1% and a dice coefficient of 88.8 ± 7.9%. Meningioma classification had a mean accuracy of 95.2 ± 4.0%. Tumor grading had a mean area-under-the-curve (AUC) of 0.85 ± 0.08. Correlation with meningioma biological features had a mean AUC of 0.89 ± 0.07. Prognostication of the clinical course had a mean AUC of 0.83 ± 0.08. While clinical studies had a higher mean RQS compared to technical studies, quality was low overall with a mean RQS of 6.7 ± 5.9 (possible range -8 to 36).

CONCLUSIONS

There has been global growth in meningioma radiomics, driven by data accessibility and novel computational methodology. Translatability toward complex tasks such as prognostication requires studies that improve quality, develop comprehensive patient datasets, and engage in prospective trials.

Prediction of Cognitive Decline in Parkinson's Disease Using Clinical and DAT SPECT Imaging Features, and Hybrid Machine Learning Systems

BACKGROUND

We aimed to predict Montreal Cognitive Assessment (MoCA) scores in Parkinson's disease patients at year 4 using handcrafted radiomics (RF), deep (DF), and clinical (CF) features at year 0 (baseline) applied to hybrid machine learning systems (HMLSs).

METHODS

297 patients were selected from the Parkinson's Progressive Marker Initiative (PPMI) database. The standardized SERA radiomics software and a 3D encoder were employed to extract RFs and DFs from single-photon emission computed tomography (DAT-SPECT) images, respectively. The patients with MoCA scores over 26 were indicated as normal; otherwise, scores under 26 were indicated as abnormal. Moreover, we applied different combinations of feature sets to HMLSs, including the Analysis of Variance (ANOVA) feature selection, which was linked with eight classifiers, including Multi-Layer Perceptron (MLP), K-Neighbors Classifier (KNN), Extra Trees Classifier (ETC), and others. We employed 80% of the patients to select the best model in a 5-fold cross-validation process, and the remaining 20% were employed for hold-out testing.

RESULTS

For the sole usage of RFs and DFs, ANOVA and MLP resulted in averaged accuracies of 59 ± 3% and 65 ± 4% for 5-fold cross-validation, respectively, with hold-out testing accuracies of 59 ± 1% and 56 ± 2%, respectively. For sole CFs, a higher performance of 77 ± 8% for 5-fold cross-validation and a hold-out testing performance of 82 + 2% were obtained from ANOVA and ETC. RF+DF obtained a performance of 64 ± 7%, with a hold-out testing performance of 59 ± 2% through ANOVA and XGBC. Usage of CF+RF, CF+DF, and RF+DF+CF enabled the highest averaged accuracies of 78 ± 7%, 78 ± 9%, and 76 ± 8% for 5-fold cross-validation, and hold-out testing accuracies of 81 ± 2%, 82 ± 2%, and 83 ± 4%, respectively.

CONCLUSIONS

We demonstrated that CFs vitally contribute to predictive performance, and combining them with appropriate imaging features and HMLSs can result in the best prediction performance.

Are deep models in radiomics performing better than generic models? A systematic review

BACKGROUND

Application of radiomics proceeds by extracting and analysing imaging features based on generic morphological, textural, and statistical features defined by formulas. Recently, deep learning methods were applied. It is unclear whether deep models (DMs) can outperform generic models (GMs).

METHODS

We identified publications on PubMed and Embase to determine differences between DMs and GMs in terms of receiver operating area under the curve (AUC).

RESULTS

Of 1,229 records (between 2017 and 2021), 69 studies were included, 61 (88%) on tumours, 68 (99%) retrospective, and 39 (56%) single centre; 30 (43%) used an internal validation cohort; and 18 (26%) applied cross-validation. Studies with independent internal cohort had a median training sample of 196 (range 41-1,455); those with cross-validation had only 133 (43-1,426). Median size of validation cohorts was 73 (18-535) for internal and 94 (18-388) for external. Considering the internal validation, in 74% (49/66), the DMs performed better than the GMs, vice versa in 20% (13/66); no difference in 6% (4/66); and median difference in AUC 0.045. On the external validation, DMs were better in 65% (13/20), GMs in 20% (4/20) cases; no difference in 3 (15%); and median difference in AUC 0.025. On internal validation, fused models outperformed GMs and DMs in 72% (20/28), while they were worse in 14% (4/28) and equal in 14% (4/28); median gain in AUC was + 0.02. On external validation, fused model performed better in 63% (5/8), worse in 25% (2/8), and equal in 13% (1/8); median gain in AUC was + 0.025.

CONCLUSIONS

Overall, DMs outperformed GMs but in 26% of the studies, DMs did not outperform GMs.

Radiomics approach with deep learning for predicting T4 obstructive colorectal cancer using CT image

OBJECTIVES

Patients with T4 obstructive colorectal cancer (OCC) have a high mortality rate. Therefore, an accurate distinction between T4 and T1-T3 (NT4) in OCC is an important part of preoperative evaluation, especially in the emergency setting. This paper introduces three models of radiomics, deep learning, and deep learning-based radiomics to identify T4 OCC.

METHODS

We established a dataset of computed tomography (CT) images of 164 patients with pathologically confirmed OCC, from which 2537 slides were extracted. First, since T4 tumors penetrate the bowel wall and involve adjacent organs, we explored whether the peritumoral region contributes to the assessment of T4 OCC. Furthermore, we visualized the radiomics and deep learning features using the t-distributed stochastic neighbor embedding technique (t-SNE). Finally, we built a merged model by fusing radiomic features with deep learning features. In this experiment, the performance of each model was evaluated by the area under the receiver operating characteristic curve (AUC).

RESULTS

In the test cohort, the AUC values predicted by the radiomics model in the dilated region of interest (dROI) was 0.770. And the AUC value of the deep learning model with the patches extended 20-pixel reached 0.936. Combining the characteristics of radiomics and deep learning, our method achieved an AUC value of 0.947 in the T4 and non-T4 (NT4) classification, and increased the AUC value to 0.950 after the addition of clinical features.

CONCLUSION

The prediction results of our merged model of deep learning radiomics outperformed the deep learning model and significantly outperformed the radiomics model. The experimental results demonstrate that combining the peritumoral region improves the prediction performance of the radiomics model and the deep learning model.

Preoperative prediction of cytokeratin-19 expression for hepatocellular carcinoma using T1 mapping on gadoxetic acid-enhanced MRI combined with diffusion-weighted imaging and clinical indicators

Objectives

To explore the value of T1 mapping on gadoxetic acid-enhanced magnetic resonance imaging (MRI) in preoperative predicting cytokeratin 19 (CK19) expression for hepatocellular carcinoma (HCC).

Methods

This retrospective study included 158 patients from two institutions with surgically resected treatment-native solitary HCC who underwent preoperative T1 mapping on gadoxetic acid-enhanced MRI. Patients from institution I (n = 102) and institution II (n = 56) were assigned to training and test sets, respectively. univariable and multivariable logistic regression analyses were performed to investigate the association of clinicoradiological variables with CK19. The receiver operating characteristic (ROC) curve and precision-recall (PR) curve were used to evaluate the performance for CK19 prediction. Then, a prediction nomogram was developed for CK19 expression. The performance of the prediction nomogram was evaluated by its discrimination, calibration, and clinical utility.

Results

Multivariable logistic regression analysis showed that AFP>400ng/ml (OR=4.607, 95%CI: 1.098-19.326; p=0.037), relative apparent diffusion coefficient (rADC)≤0.71 (OR=3.450, 95%CI: 1.126-10.567; p=0.030), T1 relaxation time in the 20-minute hepatobiliary phase (T1rt-HBP)>797msec (OR=4.509, 95%CI: 1.301-15.626; p=0.018) were significant independent predictors of CK19 expression. The clinical-quantitative model (CQ-Model) constructed based on these significant variables had the best predictive performance with an area under the ROC curve of 0.844, an area under the PR curve of 0.785 and an F1 score of 0.778. The nomogram constructed based on CQ-Model demonstrated satisfactory performance with C index of 0.844 (95%CI: 0.759-0.908) and 0.818 (95%CI: 0.693-0.902) in the training and test sets, respectively.

Conclusions

T1 mapping on gadoxetic acid-enhanced MRI has good predictive efficacy for preoperative prediction of CK19 expression in HCC, which can promote the individualized risk stratification and further treatment decision of HCC patients.

Current status and quality of radiomic studies for predicting KRAS mutations in colorectal cancer patients: A systematic review and meta‑analysis

PURPOSE

The purpose of this study was to evaluate the methodological quality of radiomics-based studies for noninvasive, preoperative prediction of Kirsten rat sarcoma (KRAS) mutations in patients with colorectal cancer; furthermore, we systematically evaluate the diagnostic accuracy of predicting models.

METHODS

We systematically searched PubMed, Embase, Cochrane Library and Web of Science databases up to 20 April 2022 for eligible studies. The methodological quality of included studies was assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) and Radiomics Quality Score (RQS) tools. A meta-analysis of studies on the prediction of KRAS status in colorectal cancer patients was performed.

RESULT

Twenty-nine studies were identified in the systematic review, including three studies on the prediction of KRAS status in colorectal cancer liver metastases. All studies had an average RQS score of 9.55 (26.5% of the total score), ranging from 3 to 17. Most studies demonstrated a low or unclear risk of bias in the domains of QUADAS-2. Nineteen studies were included in the meta-analysis, mostly imaged with magnetic resonance imaging (MRI), followed by computed tomography (CT), positron emission tomography-CT (PET/CT). With pooled sensitivity, specificity and area under the curve (AUC) of the training cohorts were 0.80(95% confidence interval(CI), 0.75-0.84), 0.80(95% CI, 0.74-0.85) and 0.87(95% CI, 0.84-0.90),respectively. The pooled sensitivity, specificity, and AUC for the validation cohorts (13 studies) were 0.78(95% CI, 0.71-0.84), 0.84(95% CI, 0.74-0.90), and 0.86(95% CI, 0.83-0.89), respectively.

CONCLUSION

Radiomics is a potential noninvasive technology that has a moderate preoperative diagnosis and prediction effect on KRAS mutations. However, it has not been implemented as a clinical decision-making tool. Future researchers should pay more attention to the methodological quality of the study and further externally validate the model using multicenter datasets.

Artificial Intelligence in Emergency Radiology: Where Are We Going?

Emergency Radiology is a unique branch of imaging, as rapidity in the diagnosis and management of different pathologies is essential to saving patients' lives. Artificial Intelligence (AI) has many potential applications in emergency radiology: firstly, image acquisition can be facilitated by reducing acquisition times through automatic positioning and minimizing artifacts with AI-based reconstruction systems to optimize image quality, even in critical patients; secondly, it enables an efficient workflow (AI algorithms integrated with RIS-PACS workflow), by analyzing the characteristics and images of patients, detecting high-priority examinations and patients with emergent critical findings. Different machine and deep learning algorithms have been trained for the automated detection of different types of emergency disorders (e.g., intracranial hemorrhage, bone fractures, pneumonia), to help radiologists to detect relevant findings. AI-based smart reporting, summarizing patients' clinical data, and analyzing the grading of the imaging abnormalities, can provide an objective indicator of the disease's severity, resulting in quick and optimized treatment planning. In this review, we provide an overview of the different AI tools available in emergency radiology, to keep radiologists up to date on the current technological evolution in this field.

Predictive performance of radiomic models based on features extracted from pretrained deep networks

OBJECTIVES

In radiomics, generic texture and morphological features are often used for modeling. Recently, features extracted from pretrained deep networks have been used as an alternative. However, extracting deep features involves several decisions, and it is unclear how these affect the resulting models. Therefore, in this study, we considered the influence of such choices on the predictive performance.

METHODS

On ten publicly available radiomic datasets, models were trained using feature sets that differed in terms of the utilized network architecture, the layer of feature extraction, the used set of slices, the use of segmentation, and the aggregation method. The influence of these choices on the predictive performance was measured using a linear mixed model. In addition, models with generic features were trained and compared in terms of predictive performance and correlation.

RESULTS

No single choice consistently led to the best-performing models. In the mixed model, the choice of architecture (AUC + 0.016; p < 0.001), the level of feature extraction (AUC + 0.016; p < 0.001), and using all slices (AUC + 0.023; p < 0.001) were highly significant; using the segmentation had a lower influence (AUC + 0.011; p = 0.023), while the aggregation method was insignificant (p = 0.774). Models based on deep features were not significantly better than those based on generic features (p > 0.05 on all datasets). Deep feature sets correlated moderately with each other (r = 0.4), in contrast to generic feature sets (r = 0.89).

CONCLUSIONS

Different choices have a significant effect on the predictive performance of the resulting models; however, for the highest performance, these choices should be optimized during cross-validation.

The deep radiomic analytics pipeline

Radiomics refers to the process of extracting useful imaging features from radiological data. Conventional radiomics like standard uptake value, intensity histograms, or phase images involve hand-crafted (manual) or automated regions of interest (computer generated), however, artificial intelligence (AI) segmentation (AI-augmented radiomics) has recently emerged. Radiomic feature extraction extends image insights beyond simply data quantitation and provides additional insights to aid semantic reporting. Deeper layers of a convolutional neural network produce more abstract radiomic features that are referred to as deep radiomics. The application of radiomics in veterinary radiology is already firmly entrenched using hand-crafted and automated computer-generated radiomic features in X-ray, nuclear medicine, CT, ultrasound, and MRI. There is an opportunity for veterinary radiology to capitalize on advances in AI, machine learning, and deep learning to enrich imaging interpretation using deep radiomic feature extraction. This manuscript aims to provide a general understanding of radiomics and deep radiomics, and to arm readers with the vernacular to progress discussion and development of deep radiomics in veterinary imaging.

Deep learning for predicting major pathological response to neoadjuvant chemoimmunotherapy in non-small cell lung cancer: A multicentre study

Background

This study, based on multicentre cohorts, aims to utilize computed tomography (CT) images to construct a deep learning model for predicting major pathological response (MPR) to neoadjuvant chemoimmunotherapy in non-small cell lung cancer (NSCLC) and further explore the biological basis under its prediction.

Methods

274 patients undergoing curative surgery after neoadjuvant chemoimmunotherapy for NSCLC at 4 centres from January 2019 to December 2021 were included and divided into a training cohort, an internal validation cohort, and an external validation cohort. ShuffleNetV2x05-based features of the primary tumour on the CT scans within the 2 weeks preceding neoadjuvant administration were employed to develop a deep learning score for distinguishing MPR and non-MPR. To reveal the underlying biological basis of the deep learning score, a genetic analysis was conducted based on 25 patients with RNA-sequencing data.

Findings

MPR was achieved in 54.0% (n = 148) patients. The area under the curve (AUC) of the deep learning score to predict MPR was 0.73 (95% confidence interval [CI]: 0.58–0.86) and 0.72 (95% CI: 0.58–0.85) in the internal validation and external validation cohorts, respectively. After integrating the clinical characteristic into the deep learning score, the combined model achieved satisfactory performance in the internal validation (AUC: 0.77, 95% CI: 0.64–0.89) and external validation cohorts (AUC: 0.75, 95% CI: 0.62–0.87). In the biological basis exploration for the deep learning score, a high deep learning score was associated with the downregulation of pathways mediating tumour proliferation and the promotion of antitumour immune cell infiltration in the microenvironment.

Interpretation

The proposed deep learning model could effectively predict MPR in NSCLC patients treated with neoadjuvant chemoimmunotherapy.

Funding

This study was supported by National Key Research and Development Program of China, China (2017YFA0205200); National Natural Science Foundation of China, China (91959126, 82022036, 91959130, 81971776, 81771924, 6202790004, 81930053, 9195910169, 62176013, 8210071009); Beijing Natural Science Foundation, China (L182061); Strategic Priority Research Program of Chinese Academy of Sciences, China (XDB38040200); Chinese Academy of Sciences, China (GJJSTD20170004, QYZDJ-SSW-JSC005); Shanghai Hospital Development Center, China (SHDC2020CR3047B); and Science and Technology Commission of Shanghai Municipality, China (21YF1438200).

Radiomics in veterinary medicine: Overview, methods, and applications

Radiomics, or quantitative image analysis from radiographic image data, borrows the suffix from other emerging -omics fields of study, such as genomics, proteomics, and metabolomics. This report provides an overview of the general principles of how radiomic features are computed, describes major types of morphological, first order, and texture features, and the applications, challenges, and opportunities of radiomics as applied in veterinary medicine. Some advantages radiomics has over traditional semantic radiological features include standardized methodology in computing semantic features, the ability to compute features in multi-dimensional images, their newfound associations with genomic and pathological abnormalities, and the number of perceptible and imperceptible features available for regression or classification modeling. Some challenges in deploying radiomics in a clinical setting include sensitivity to image acquisition settings and image artifacts, pre- and post-image reconstruction and calculation settings, variability in feature estimates stemming from inter- and intra-observer contouring errors, and challenges with software and data harmonization and generalizability of findings given the challenges of small sample size and patient selection bias in veterinary medicine. Despite this, radiomics has enormous potential in patient-centric diagnostics, prognosis, and theragnostics. Fully leveraging the utility of radiomics in veterinary medicine will require inter-institutional collaborations, data harmonization, and data sharing strategies amongst institutions, transparent and robust model development, and multi-disciplinary efforts within and outside the veterinary medical imaging community.

Predicting survival of glioblastoma from automatic whole-brain and tumor segmentation of MR images

Survival prediction models can potentially be used to guide treatment of glioblastoma patients. However, currently available MR imaging biomarkers holding prognostic information are often challenging to interpret, have difficulties generalizing across data acquisitions, or are only applicable to pre-operative MR data. In this paper we aim to address these issues by introducing novel imaging features that can be automatically computed from MR images and fed into machine learning models to predict patient survival. The features we propose have a direct anatomical-functional interpretation: They measure the deformation caused by the tumor on the surrounding brain structures, comparing the shape of various structures in the patient's brain to their expected shape in healthy individuals. To obtain the required segmentations, we use an automatic method that is contrast-adaptive and robust to missing modalities, making the features generalizable across scanners and imaging protocols. Since the features we propose do not depend on characteristics of the tumor region itself, they are also applicable to post-operative images, which have been much less studied in the context of survival prediction. Using experiments involving both pre- and post-operative data, we show that the proposed features carry prognostic value in terms of overall- and progression-free survival, over and above that of conventional non-imaging features.

Gross tumour volume radiomics for prognostication of recurrence & death following radical radiotherapy for NSCLC

Recurrence occurs in up to 36% of patients treated with curative-intent radiotherapy for NSCLC. Identifying patients at higher risk of recurrence for more intensive surveillance may facilitate the earlier introduction of the next line of treatment. We aimed to use radiotherapy planning CT scans to develop radiomic classification models that predict overall survival (OS), recurrence-free survival (RFS) and recurrence two years post-treatment for risk-stratification. A retrospective multi-centre study of >900 patients receiving curative-intent radiotherapy for stage I-III NSCLC was undertaken. Models using radiomic and/or clinical features were developed, compared with 10-fold cross-validation and an external test set, and benchmarked against TNM-stage. Respective validation and test set AUCs (with 95% confidence intervals) for the radiomic-only models were: (1) OS: 0.712 (0.592–0.832) and 0.685 (0.585–0.784), (2) RFS: 0.825 (0.733–0.916) and 0.750 (0.665–0.835), (3) Recurrence: 0.678 (0.554–0.801) and 0.673 (0.577–0.77). For the combined models: (1) OS: 0.702 (0.583–0.822) and 0.683 (0.586–0.78), (2) RFS: 0.805 (0.707–0.903) and 0·755 (0.672–0.838), (3) Recurrence: 0·637 (0.51–0.·765) and 0·738 (0.649–0.826). Kaplan-Meier analyses demonstrate OS and RFS difference of >300 and >400 days respectively between low and high-risk groups. We have developed validated and externally tested radiomic-based prediction models. Such models could be integrated into the routine radiotherapy workflow, thus informing a personalised surveillance strategy at the point of treatment. Our work lays the foundations for future prospective clinical trials for quantitative personalised risk-stratification for surveillance following curative-intent radiotherapy for NSCLC.

The transformational potential of molecular radiomics

Conventional radiomics in nuclear medicine involve hand-crafted and computer-assisted regions of interest. Recent developments in artificial intelligence (AI) have seen the emergence of AI-augmented segmentation and extraction of lower order traditional radiomic features. Deep learning (DL) affords the opportunity to extract abstract radiomic features directly from input tensors (images) without the need for segmentation. These fourth-order, high dimensional radiomics produce deep radiomics and are well suited to the data density associated with the molecular nature of hybrid imaging. Molecular radiomics and deep molecular radiomics provide insights beyond images and quantitation typical of semantic reporting. While the application of molecular radiomics using hand-crafted and computer-generated features is integrated into decision-making in nuclear medicine, the acceptance of deep molecular radiomics is less universal. This manuscript aims to provide an understanding of the language and principles associated with radiomics and deep radiomics in nuclear medicine.