A novel multimodal deep learning model for preoperative prediction of microvascular invasion and outcome in hepatocellular carcinoma

Preliminary study on detection and diagnosis of focal liver lesions based on a deep learning model using multimodal PET/CT images

Objectives

To develop and validate a deep learning model using multimodal PET/CT imaging for detecting and classifying focal liver lesions (FLL).

Methods

This study included 185 patients who underwent 18F-FDG PET/CT imaging at our institution from March 2022 to February 2023. We analyzed serological data and imaging. Liver lesions were segmented on PET and CT, serving as the "reference standard". Deep learning models were trained using PET and CT images to generate predicted segmentations and classify lesion nature. Model performance was evaluated by comparing the predicted segmentations with the reference segmentations, using metrics such as Dice, Precision, Recall, F1-score, ROC, and AUC, and compared it with physician diagnoses.

Results

This study finally included 150 patients, comprising 46 patients with benign liver nodules, 51 patients with malignant liver nodules, and 53 patients with no FLLs. Significant differences were observed among groups for age, AST, ALP, GGT, AFP, CA19-9and CEA. On the validation set, the Dice coefficient of the model was 0.740. For the normal group, the recall was 0.918, precision was 0.904, F1-score was 0.909, and AUC was 0.976. For the benign group, the recall was 0.869, precision was 0.862, F1-score was 0.863, and AUC was 0.928. For the malignant group, the recall was 0.858, precision was 0.914, F1-score was 0.883, and AUC was 0.979. The model's overall diagnostic performance was between that of junior and senior physician.

Conclusion

This deep learning model demonstrated high sensitivity in detecting FLLs and effectively differentiated between benign and malignant lesions.

An Integrated Model Combined Conventional Radiomics and Deep Learning Features to Predict Early Recurrence of Hepatocellular Carcinoma Eligible for Curative Ablation: A Multicenter Cohort Study

OBJECTIVE

Hepatocellular carcinoma (HCC) is the most common primary liver malignancy. Ablation therapy is one of the first-line treatments for early HCC. Accurately predicting early recurrence (ER) is crucial for making precise treatment plans and improving prognosis. This study aimed to develop and validate a model (DLRR) that incorporates deep learning radiomics and traditional radiomics features to predict ER following curative ablation for HCC.

METHODS

We retrospectively analysed the data of 288 eligible patients from 3 hospitals-1 primary cohort (center 1, n=222) and 2 external test cohorts (center 2, n=32 and center 3, n=34)-from April 2008 to March 2022. 3D ResNet-18 and PyRadiomics were applied to extract features from contrast-enhanced computed tomography (CECT) images. The 3-step (ICC-LASSO-RFE) method was used for feature selection, and 6 machine learning methods were used to construct models. Performance was compared through the area under the receiver operating characteristic curve (AUC), net reclassification improvement (NRI) and integrated discrimination improvement (IDI) indices. Calibration and clinical applicability were assessed through calibration curves and decision curve analysis (DCA), respectively. Kaplan-Meier (K-M) curves were generated to stratify patients based on progression-free survival (PFS) and overall survival (OS).

RESULTS

The DLRR model had the best performance, with AUCs of 0.981, 0.910, and 0.851 in the training, internal validation, and external validation sets, respectively. In addition, the calibration curve and DCA curve revealed that the DLRR model had good calibration ability and clinical applicability. The K-M curve indicated that the DLRR model provided risk stratification for progression-free survival (PFS) and overall survival (OS) in HCC patients.

CONCLUSIONS

The DLRR model noninvasively and efficiently predicts ER after curative ablation in HCC patients, which helps to categorize the risk in patients to formulate precise diagnosis and treatment plans and management strategies for patients and to improve the prognosis.

Preoperative Prediction of Microvascular Invasion in Hepatocellular Carcinoma From Multi-Sequence Magnetic Resonance Imaging Based on Deep Fusion Representation Learning

Recent studies have identified microvascular invasion (MVI) as the most vital independent biomarker associated with early tumor recurrence. With advancements in medical technology, several computational methods have been developed to predict preoperative MVI using diverse medical images. These existing methods rely on human experience, attribute selection or clinical trial testing, which is often time-consuming and labor-intensive. Leveraging the advantages of deep learning, this study presents a novel end-to-end algorithm for predicting MVI prior to surgery. We devised a series of data preprocessing strategies to fully extract multi-view features from the data while preserving peritumoral information. Notably, a new multi-branch deep fused feature algorithm based on ResNet (DFFResNet) is introduced, which combines Magnetic Resonance Images (MRI) from different sequences to enhance information complementarity and integration. We conducted prediction experiments on a dataset from the Radiology Department of the First Hospital of Lanzhou University, comprising 117 individuals and seven MRI sequences. The model was trained on 80% of the data using 10-fold cross-validation, and the remaining 20% were used for testing. This evaluation was processed in two cases: CROI, containing samples with a complete region of interest (ROI), and PROI, containing samples with a partial ROI region. The robustness results from repeated experiments at both image and patient levels demonstrate the superior performance and improved generalization of the proposed method compared to alternative models. Our approach yields highly competitive prediction results even when the ROI region outline is incomplete, offering a novel and effective multi-sequence fused strategy for predicting preoperative MVI.

Enhancing Radiologist Productivity with Artificial Intelligence in Magnetic Resonance Imaging (MRI): A Narrative Review

Artificial intelligence (AI) shows promise in streamlining MRI workflows by reducing radiologists' workload and improving diagnostic accuracy. Despite MRI's extensive clinical use, systematic evaluation of AI-driven productivity gains in MRI remains limited. This review addresses that gap by synthesizing evidence on how AI can shorten scanning and reading times, optimize worklist triage, and automate segmentation. On 15 November 2024, we searched PubMed, EMBASE, MEDLINE, Web of Science, Google Scholar, and Cochrane Library for English-language studies published between 2000 and 15 November 2024, focusing on AI applications in MRI. Additional searches of grey literature were conducted. After screening for relevance and full-text review, 67 studies met inclusion criteria. Extracted data included study design, AI techniques, and productivity-related outcomes such as time savings and diagnostic accuracy. The included studies were categorized into five themes: reducing scan times, automating segmentation, optimizing workflow, decreasing reading times, and general time-saving or workload reduction. Convolutional neural networks (CNNs), especially architectures like ResNet and U-Net, were commonly used for tasks ranging from segmentation to automated reporting. A few studies also explored machine learning-based automation software and, more recently, large language models. Although most demonstrated gains in efficiency and accuracy, limited external validation and dataset heterogeneity could reduce broader adoption. AI applications in MRI offer potential to enhance radiologist productivity, mainly through accelerated scans, automated segmentation, and streamlined workflows. Further research, including prospective validation and standardized metrics, is needed to enable safe, efficient, and equitable deployment of AI tools in clinical MRI practice.

Gd-EOB-DTPA-enhanced MRI radiomics and deep learning models to predict microvascular invasion in hepatocellular carcinoma: a multicenter study

BACKGROUND

Microvascular invasion (MVI) is an important risk factor for early postoperative recurrence of hepatocellular carcinoma (HCC). Based on gadolinium-ethoxybenzyl-diethylenetriamine pentaacetic acid (Gd-EOB-DTPA)-enhanced magnetic resonance imaging (MRI) images, we developed a novel radiomics model. It combined bi-regional features and two machine learning algorithms. The aim of this study was to validate its potential value for preoperative prediction of MVI.

METHODS

This retrospective study included 304 HCC patients (training cohort, 216 patients; testing cohort, 88 patients) from three hospitals. Intratumoral and peritumoral volumes of interest were delineated in arterial phase, portal venous phase, and hepatobiliary phase images. Conventional radiomics (CR) and deep learning radiomics (DLR) features were extracted based on FeAture Explorer software and the 3D ResNet-18 extractor, respectively. Clinical variables were selected using univariate and multivariate analyses. Clinical, CR, DLR, CR-DLR, and clinical-radiomics (Clin-R) models were built using support vector machines. The predictive capacity of the models was assessed by the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity.

RESULTS

The bi-regional CR-DLR model showed more gains and gave better predictive performance than the single-regional models or single-machine learning models. Its AUC, accuracy, sensitivity, and specificity were 0.844, 76.9%, 87.8%, and 69.1% in the training cohort and 0.740, 73.9%, 50%, and 84.5% in the testing cohort. Alpha-fetoprotein (odds ratio was 0.32) and maximum tumor diameter (odds ratio was 1.270) were independent predictors. The AUCs of the clinical model and the Clin-R model were 0.655 and 0.672, respectively. There was no significant difference in the AUCs between all the models (P > 0.005).

CONCLUSION

Based on Gd-EOB-DTPA-enhanced MRI images, we focused on developing a radiomics model that combines bi-regional features and two machine learning algorithms (CR and DLR). The application of the new model will provide a more accurate and non-invasive diagnostic solution for medical imaging. It will provide valuable information for clinical personalized treatment, thereby improving patient prognosis.

CLINICAL TRIAL NUMBER

Not applicable.

Multimodal multiphasic preoperative image-based deep-learning predicts HCC outcomes after curative surgery

BACKGROUND AND AIMS

HCC recurrence frequently occurs after curative surgery. Histological microvascular invasion (MVI) predicts recurrence but cannot provide preoperative prognostication, whereas clinical prediction scores have variable performances.

APPROACH AND RESULTS

Recurr-NET, a multimodal multiphasic residual-network random survival forest deep-learning model incorporating preoperative CT and clinical parameters, was developed to predict HCC recurrence. Preoperative triphasic CT scans were retrieved from patients with resected histology-confirmed HCC from 4 centers in Hong Kong (internal cohort). The internal cohort was randomly divided in an 8:2 ratio into training and internal validation. External testing was performed in an independent cohort from Taiwan.Among 1231 patients (age 62.4y, 83.1% male, 86.8% viral hepatitis, and median follow-up 65.1mo), cumulative HCC recurrence rates at years 2 and 5 were 41.8% and 56.4%, respectively. Recurr-NET achieved excellent accuracy in predicting recurrence from years 1 to 5 (internal cohort AUROC 0.770-0.857; external AUROC 0.758-0.798), significantly outperforming MVI (internal AUROC 0.518-0.590; external AUROC 0.557-0.615) and multiple clinical risk scores (ERASL-PRE, ERASL-POST, DFT, and Shim scores) (internal AUROC 0.523-0.587, external AUROC: 0.524-0.620), respectively (all p < 0.001). Recurr-NET was superior to MVI in stratifying recurrence risks at year 2 (internal: 72.5% vs. 50.0% in MVI; external: 65.3% vs. 46.6% in MVI) and year 5 (internal: 86.4% vs. 62.5% in MVI; external: 81.4% vs. 63.8% in MVI) (all p < 0.001). Recurr-NET was also superior to MVI in stratifying liver-related and all-cause mortality (all p < 0.001). The performance of Recurr-NET remained robust in subgroup analyses.

CONCLUSIONS

Recurr-NET accurately predicted HCC recurrence, outperforming MVI and clinical prediction scores, highlighting its potential in preoperative prognostication.

Deep learning radiomics based on contrast enhanced MRI for preoperatively predicting early recurrence in hepatocellular carcinoma after curative resection

Purpose

To explore the role of deep learning (DL) and radiomics-based integrated approach based on contrast enhanced magnetic resonance imaging (CEMRI) for predicting early recurrence (ER) in hepatocellular carcinoma (HCC) patients after curative resection.

Methods

Total 165 HCC patients (ER, n = 96 vs. non-early recurrence (NER), n = 69) were retrospectively collected and divided into a training cohort (n = 132) and a validation cohort (n = 33). From pretreatment CEMR images, a total of 3111 radiomics features were extracted, and radiomics models were constructed using five machine learning classifiers (logistic regression, support vector machine, k-nearest neighbor, extreme gradient Boosting, and multilayer perceptron). DL models were established via three variations of ResNet architecture. The clinical-radiological (CR), radiomics combined with clinical-radiological (RCR), and deep learning combined with RCR (DLRCR) models were constructed. Model discrimination, calibration, and clinical utilities were evaluated by receiver operating characteristic curve, calibration curve, and decision curve analysis, respectively. The best-performing model was compared with the widely used staging systems and preoperative prognostic indexes.

Results

The RCR model (area under the curve (AUC): 0.841 and 0.811) and the optimal radiomics model (AUC: 0.839 and 0.804) achieved better performance than the CR model (AUC: 0.662 and 0.752) in the training and validation cohorts, respectively. The optimal DL model (AUC: 0.870 and 0.826) outperformed the radiomics model in the both cohorts. The DL, radiomics, and CR predictors (aspartate aminotransferase (AST) and tumor diameter) were combined to construct the DLRCR model. The DLRCR model presented the best performance over any model, yielding an AUC, an accuracy, a sensitivity, a specificity of 0.917, 0.886, 0.889, and 0.882 in the training cohort and of 0.844, 0.818, 0.800, and 0.846 in the validation cohort, respectively. The DLRCR model achieved better clinical utility compared to the clinical staging systems and prognostic indexes.

Conclusion

Both radiomics and DL models derived from CEMRI can predict HCC recurrence, and DL and radiomics-based integrated approach can provide a more effective tool for the precise prediction of ER for HCC patients undergoing resection.

Artificial intelligence techniques in liver cancer

Hepatocellular Carcinoma (HCC), the most common primary liver cancer, is a significant contributor to worldwide cancer-related deaths. Various medical imaging techniques, including computed tomography, magnetic resonance imaging, and ultrasound, play a crucial role in accurately evaluating HCC and formulating effective treatment plans. Artificial Intelligence (AI) technologies have demonstrated potential in supporting physicians by providing more accurate and consistent medical diagnoses. Recent advancements have led to the development of AI-based multi-modal prediction systems. These systems integrate medical imaging with other modalities, such as electronic health record reports and clinical parameters, to enhance the accuracy of predicting biological characteristics and prognosis, including those associated with HCC. These multi-modal prediction systems pave the way for predicting the response to transarterial chemoembolization and microvascular invasion treatments and can assist clinicians in identifying the optimal patients with HCC who could benefit from interventional therapy. This paper provides an overview of the latest AI-based medical imaging models developed for diagnosing and predicting HCC. It also explores the challenges and potential future directions related to the clinical application of AI techniques.

Multitask deep learning for prediction of microvascular invasion and recurrence-free survival in hepatocellular carcinoma based on MRI images

BACKGROUND AND AIMS

Accurate preoperative prediction of microvascular invasion (MVI) and recurrence-free survival (RFS) is vital for personalised hepatocellular carcinoma (HCC) management. We developed a multitask deep learning model to predict MVI and RFS using preoperative MRI scans.

METHODS

Utilising a retrospective dataset of 725 HCC patients from seven institutions, we developed and validated a multitask deep learning model focused on predicting MVI and RFS. The model employs a transformer architecture to extract critical features from preoperative MRI scans. It was trained on a set of 234 patients and internally validated on a set of 58 patients. External validation was performed using three independent sets (n = 212, 111, 110).

RESULTS

The multitask deep learning model yielded high MVI prediction accuracy, with AUC values of 0.918 for the training set and 0.800 for the internal test set. In external test sets, AUC values were 0.837, 0.815 and 0.800. Radiologists' sensitivity and inter-rater agreement for MVI prediction improved significantly when integrated with the model. For RFS, the model achieved C-index values of 0.763 in the training set and ranged between 0.628 and 0.728 in external test sets. Notably, PA-TACE improved RFS only in patients predicted to have high MVI risk and low survival scores (p < .001).

CONCLUSIONS

Our deep learning model allows accurate MVI and survival prediction in HCC patients. Prospective studies are warranted to assess the clinical utility of this model in guiding personalised treatment in conjunction with clinical criteria.

MRI-Based Radiomics and Deep Learning in Biological Characteristics and Prognosis of Hepatocellular Carcinoma: Opportunities and Challenges

Hepatocellular carcinoma (HCC) is the fifth most common malignancy and the third leading cause of cancer-related death worldwide. HCC exhibits strong inter-tumor heterogeneity, with different biological characteristics closely associated with prognosis. In addition, patients with HCC often distribute at different stages and require diverse treatment options at each stage. Due to the variability in tumor sensitivity to different therapies, determining the optimal treatment approach can be challenging for clinicians prior to treatment. Artificial intelligence (AI) technology, including radiomics and deep learning approaches, has emerged as a unique opportunity to improve the spectrum of HCC clinical care by predicting biological characteristics and prognosis in the medical imaging field. The radiomics approach utilizes handcrafted features derived from specific mathematical formulas to construct various machine-learning models for medical applications. In terms of the deep learning approach, convolutional neural network models are developed to achieve high classification performance based on automatic feature extraction from images. Magnetic resonance imaging offers the advantage of superior tissue resolution and functional information. This comprehensive evaluation plays a vital role in the accurate assessment and effective treatment planning for HCC patients. Recent studies have applied radiomics and deep learning approaches to develop AI-enabled models to improve accuracy in predicting biological characteristics and prognosis, such as microvascular invasion and tumor recurrence. Although AI-enabled models have demonstrated promising potential in HCC with biological characteristics and prognosis prediction with high performance, one of the biggest challenges, interpretability, has hindered their implementation in clinical practice. In the future, continued research is needed to improve the interpretability of AI-enabled models, including aspects such as domain knowledge, novel algorithms, and multi-dimension data sources. Overcoming these challenges would allow AI-enabled models to significantly impact the care provided to HCC patients, ultimately leading to their deployment for clinical use. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 2.

Vision-Guided Attention-Enhanced Network for Predicting Microvascular Invasion in Hepatocellular Carcinoma

As the most common malignant tumor worldwide, hepatocellular carcinoma (HCC) has a high rate of death and recurrence, and microvascular invasion (MVI) is considered to be an independent risk factor affecting its early recurrence and poor survival rate. Accurate preoperative prediction of MVI is of great significance for the formulation of individualized treatment plans and long-term prognosis assessment for HCC patients. However, as the mechanism of MVI is still unclear, existing studies use deep learning methods to directly train CT or MR images, with limited predictive performance and lack of explanation. We map the pathological "7-point" baseline sampling method used to confirm the diagnosis of MVI onto MR images, propose a vision-guided attention-enhanced network to improve the prediction performance of MVI, and validate the prediction on the corresponding pathological images reliability of the results. Specifically, we design a learnable online class activation map (CAM) to guide the network to focus on high-incidence regions of MVI guided by an extended tumor mask. Further, an attention-enhanced module is proposed to force the network to learn image regions that can explain the MVI results. The generated attention maps capture long-distance dependencies and can be used as spatial priors for MVI to promote the learning of vision-guided module. The experimental results on the constructed multi-center dataset show that the proposed algorithm achieves the state-of-the-art compared to other models.

MVI-TR: A Transformer-Based Deep Learning Model with Contrast-Enhanced CT for Preoperative Prediction of Microvascular Invasion in Hepatocellular Carcinoma

In this study, we considered preoperative prediction of microvascular invasion (MVI) status with deep learning (DL) models for patients with early-stage hepatocellular carcinoma (HCC) (tumor size ≤ 5 cm). Two types of DL models based only on venous phase (VP) of contrast-enhanced computed tomography (CECT) were constructed and validated. From our hospital (First Affiliated Hospital of Zhejiang University, Zhejiang, P.R. China), 559 patients, who had histopathological confirmed MVI status, participated in this study. All preoperative CECT were collected, and the patients were randomly divided into training and validation cohorts at a ratio of 4:1. We proposed a novel transformer-based end-to-end DL model, named MVI-TR, which is a supervised learning method. MVI-TR can capture features automatically from radiomics and perform MVI preoperative assessments. In addition, a popular self-supervised learning method, the contrastive learning model, and the widely used residual networks (ResNets family) were constructed for fair comparisons. With an accuracy of 99.1%, a precision of 99.3%, an area under the curve (AUC) of 0.98, a recalling rate of 98.8%, and an F1-score of 99.1% in the training cohort, MVI-TR achieved superior outcomes. Additionally, the validation cohort's MVI status prediction had the best accuracy (97.2%), precision (97.3%), AUC (0.935), recalling rate (93.1%), and F1-score (95.2%). MVI-TR outperformed other models for predicting MVI status, and showed great preoperative predictive value for early-stage HCC patients.

Preoperative and Prognostic Prediction of Microvascular Invasion in Hepatocellular Carcinoma: A Review Based on Artificial Intelligence

Microvascular invasion of hepatocellular carcinoma is an important factor affecting tumor recurrence after liver resection and liver transplantation. There are many ways to classify microvascular invasion, however, an international consensus is urgently needed. Recently, artificial intelligence has emerged as an important tool for improving the clinical management of hepatocellular carcinoma. Many studies about microvascular invasion currently focus on preoperative and prognosis prediction of microvascular invasion using artificial intelligence. In this paper, we review the definition and staging of microvascular invasion, especially the diagnosis of it by using artificial intelligence. In preoperative prediction, deep learning based on multimodal data modeling of radiomics-screened features, clinical features, and medical images is currently the most effective means. In prognostic prediction, pathology is the gold standard, and the techniques used should more effectively utilize the global features of the pathology images.