Hybrid network with difference degree and attention mechanism combined with radiomics (H-DARnet) for MVI prediction in HCC

Preoperative Prediction of Microvascular Invasion in Hepatocellular Carcinoma From Multi-Sequence Magnetic Resonance Imaging Based on Deep Fusion Representation Learning

Recent studies have identified microvascular invasion (MVI) as the most vital independent biomarker associated with early tumor recurrence. With advancements in medical technology, several computational methods have been developed to predict preoperative MVI using diverse medical images. These existing methods rely on human experience, attribute selection or clinical trial testing, which is often time-consuming and labor-intensive. Leveraging the advantages of deep learning, this study presents a novel end-to-end algorithm for predicting MVI prior to surgery. We devised a series of data preprocessing strategies to fully extract multi-view features from the data while preserving peritumoral information. Notably, a new multi-branch deep fused feature algorithm based on ResNet (DFFResNet) is introduced, which combines Magnetic Resonance Images (MRI) from different sequences to enhance information complementarity and integration. We conducted prediction experiments on a dataset from the Radiology Department of the First Hospital of Lanzhou University, comprising 117 individuals and seven MRI sequences. The model was trained on 80% of the data using 10-fold cross-validation, and the remaining 20% were used for testing. This evaluation was processed in two cases: CROI, containing samples with a complete region of interest (ROI), and PROI, containing samples with a partial ROI region. The robustness results from repeated experiments at both image and patient levels demonstrate the superior performance and improved generalization of the proposed method compared to alternative models. Our approach yields highly competitive prediction results even when the ROI region outline is incomplete, offering a novel and effective multi-sequence fused strategy for predicting preoperative MVI.

Artificial intelligence techniques in liver cancer

Hepatocellular Carcinoma (HCC), the most common primary liver cancer, is a significant contributor to worldwide cancer-related deaths. Various medical imaging techniques, including computed tomography, magnetic resonance imaging, and ultrasound, play a crucial role in accurately evaluating HCC and formulating effective treatment plans. Artificial Intelligence (AI) technologies have demonstrated potential in supporting physicians by providing more accurate and consistent medical diagnoses. Recent advancements have led to the development of AI-based multi-modal prediction systems. These systems integrate medical imaging with other modalities, such as electronic health record reports and clinical parameters, to enhance the accuracy of predicting biological characteristics and prognosis, including those associated with HCC. These multi-modal prediction systems pave the way for predicting the response to transarterial chemoembolization and microvascular invasion treatments and can assist clinicians in identifying the optimal patients with HCC who could benefit from interventional therapy. This paper provides an overview of the latest AI-based medical imaging models developed for diagnosing and predicting HCC. It also explores the challenges and potential future directions related to the clinical application of AI techniques.

Systematic review and meta-analysis on the classification metrics of machine learning algorithm based radiomics in hepatocellular carcinoma diagnosis

The aim of this systematic review and meta-analysis is to evaluate the performance of classification metrics of machine learning-driven radiomics in diagnosing hepatocellular carcinoma (HCC). Following the PRISMA guidelines, a comprehensive search was conducted across three major scientific databases-PubMed, ScienceDirect, and Scopus-from 2018 to 2022. The search yielded a total of 436 articles pertinent to the application of machine learning and deep learning for HCC prediction. These studies collectively reflect the burgeoning interest and rapid advancements in employing artificial intelligence (AI)-driven radiomics for enhanced HCC diagnostic capabilities. After the screening process, 34 of these articles were chosen for the study. The area under curve (AUC), accuracy, specificity, and sensitivity of the proposed and basic models were assessed in each of the studies. Jamovi (version 1.1.9.0) was utilised to carry out a meta-analysis of 12 cohort studies to evaluate the classification accuracy rate. The risk of bias was estimated, and Logistic Regression was found to be the most suitable classifier for binary problems, with least absolute shrinkage and selection operator (LASSO) as the feature selector. The pooled proportion for HCC prediction classification was high for all performance metrics, with an AUC value of 0.86 (95 % CI: 0.83-0.88), accuracy of 0.83 (95 % CI: 0.78-0.88), sensitivity of 0.80 (95 % CI: 0.75-0.84) and specificity of 0.84 (95 % CI: 0.80-0.88). The performance of feature selectors, classifiers, and input features in detecting HCC and related factors was evaluated and it was observed that radiomics features extracted from medical images were adequate for AI to accurately distinguish the condition. HCC based radiomics has favourable predictive performance especially with addition of clinical features that may serve as tool that support clinical decision-making.

MRI-Based Radiomics and Deep Learning in Biological Characteristics and Prognosis of Hepatocellular Carcinoma: Opportunities and Challenges

Hepatocellular carcinoma (HCC) is the fifth most common malignancy and the third leading cause of cancer-related death worldwide. HCC exhibits strong inter-tumor heterogeneity, with different biological characteristics closely associated with prognosis. In addition, patients with HCC often distribute at different stages and require diverse treatment options at each stage. Due to the variability in tumor sensitivity to different therapies, determining the optimal treatment approach can be challenging for clinicians prior to treatment. Artificial intelligence (AI) technology, including radiomics and deep learning approaches, has emerged as a unique opportunity to improve the spectrum of HCC clinical care by predicting biological characteristics and prognosis in the medical imaging field. The radiomics approach utilizes handcrafted features derived from specific mathematical formulas to construct various machine-learning models for medical applications. In terms of the deep learning approach, convolutional neural network models are developed to achieve high classification performance based on automatic feature extraction from images. Magnetic resonance imaging offers the advantage of superior tissue resolution and functional information. This comprehensive evaluation plays a vital role in the accurate assessment and effective treatment planning for HCC patients. Recent studies have applied radiomics and deep learning approaches to develop AI-enabled models to improve accuracy in predicting biological characteristics and prognosis, such as microvascular invasion and tumor recurrence. Although AI-enabled models have demonstrated promising potential in HCC with biological characteristics and prognosis prediction with high performance, one of the biggest challenges, interpretability, has hindered their implementation in clinical practice. In the future, continued research is needed to improve the interpretability of AI-enabled models, including aspects such as domain knowledge, novel algorithms, and multi-dimension data sources. Overcoming these challenges would allow AI-enabled models to significantly impact the care provided to HCC patients, ultimately leading to their deployment for clinical use. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 2.

Current status of magnetic resonance imaging radiomics in hepatocellular carcinoma: A quantitative review with Radiomics Quality Score

BACKGROUND

Radiomics is a promising tool that may increase the value of magnetic resonance imaging (MRI) for different tasks related to the management of patients with hepatocellular carcinoma (HCC). However, its implementation in clinical practice is still far, with many issues related to the methodological quality of radiomic studies.

AIM

To systematically review the current status of MRI radiomic studies concerning HCC using the Radiomics Quality Score (RQS).

METHODS

A systematic literature search of PubMed, Google Scholar, and Web of Science databases was performed to identify original articles focusing on the use of MRI radiomics for HCC management published between 2017 and 2023. The methodological quality of radiomic studies was assessed using the RQS tool. Spearman's correlation (ρ) analysis was performed to explore if RQS was correlated with journal metrics and characteristics of the studies. The level of statistical signi-ficance was set at P < 0.05.

RESULTS

One hundred and twenty-seven articles were included, of which 43 focused on HCC prognosis, 39 on prediction of pathological findings, 16 on prediction of the expression of molecular markers outcomes, 18 had a diagnostic purpose, and 11 had multiple purposes. The mean RQS was 8 ± 6.22, and the corresponding percentage was 24.15% ± 15.25% (ranging from 0.0% to 58.33%). RQS was positively correlated with journal impact factor (IF; ρ = 0.36, P = 2.98 × 10-5), 5-years IF (ρ = 0.33, P = 1.56 × 10-4), number of patients included in the study (ρ = 0.51, P < 9.37 × 10-10) and number of radiomics features extracted in the study (ρ = 0.59, P < 4.59 × 10-13), and time of publication (ρ = -0.23, P < 0.0072).

CONCLUSION

Although MRI radiomics in HCC represents a promising tool to develop adequate personalized treatment as a noninvasive approach in HCC patients, our study revealed that studies in this field still lack the quality required to allow its introduction into clinical practice.

Enhancing preoperative diagnosis of microvascular invasion in hepatocellular carcinoma: domain-adaptation fusion of multi-phase CT images

Objectives

In patients with hepatocellular carcinoma (HCC), accurately predicting the preoperative microvascular invasion (MVI) status is crucial for improving survival rates. This study proposes a multi-modal domain-adaptive fusion model based on deep learning methods to predict the preoperative MVI status in HCC.

Materials and methods

From January 2008 to May 2022, we collected 163 cases of HCC from our institution and 42 cases from another medical facility, with each case including Computed Tomography (CT) images from the pre-contrast phase (PCP), arterial phase (AP), and portal venous phase (PVP). We divided our institution's dataset (n=163) into training (n=119) and test sets (n=44) in an approximate 7:3 ratio. Additionally, we included cases from another institution (n=42) as an external validation set (test1 set). We constructed three single-modality models, a simple concatenated multi-modal model, two current state-of-the-art image fusion model and a multi-modal domain-adaptive fusion model (M-DAFM) based on deep learning methods. We evaluated and analyzed the performance of these constructed models in predicting preoperative MVI using the area under the receiver operating characteristic curve (AUC), decision curve analysis (DCA), and net reclassification improvement (NRI) methods.

Results

In comparison with all models, M-DAFM achieved the highest AUC values across the three datasets (0.8013 for the training set, 0.7839 for the test set, and 0.7454 for the test1 set). Notably, in the test set, M-DAFM's Decision Curve Analysis (DCA) curves consistently demonstrated favorable or optimal net benefits within the 0-0.65 threshold probability range. Additionally, the Net Reclassification Improvement (NRI) values between M-DAFM and the three single-modal models, as well as the simple concatenation model, were all greater than 0 (all p < 0.05). Similarly, the NRI values between M-DAFM and the two current state-of-the-art image fusion models were also greater than 0. These findings collectively indicate that M-DAFM effectively integrates valuable information from multi-phase CT images, thereby enhancing the model's preoperative predictive performance for MVI.

Conclusion

The M-DAFM proposed in this study presents an innovative approach to improve the preoperative predictive performance of MVI.

Modality-based attention and dual-stream multiple instance convolutional neural network for predicting microvascular invasion of hepatocellular carcinoma

BACKGROUND AND PURPOSE

The presence of microvascular invasion (MVI) is a crucial indicator of postoperative recurrence in patients with hepatocellular carcinoma (HCC). Detecting MVI before surgery can improve personalized surgical planning and enhance patient survival. However, existing automatic diagnosis methods for MVI have certain limitations. Some methods only analyze information from a single slice and overlook the context of the entire lesion, while others require high computational resources to process the entire tumor with a three-dimension (3D) convolutional neural network (CNN), which could be challenging to train. To address these limitations, this paper proposes a modality-based attention and dual-stream multiple instance learning(MIL) CNN.

MATERIALS AND METHODS

In this retrospective study, 283 patients with histologically confirmed HCC who underwent surgical resection between April 2017 and September 2019 were included. Five magnetic resonance (MR) modalities including T2-weighted, arterial phase, venous phase, delay phase and apparent diffusion coefficient images were used in image acquisition of each patient. Firstly, Each two-dimension (2D) slice of HCC magnetic resonance image (MRI) was converted into an instance embedding. Secondly, modality attention module was designed to emulates the decision-making process of doctors and helped the model to focus on the important MRI sequences. Thirdly, instance embeddings of 3D scans were aggregated into a bag embedding by a dual-stream MIL aggregator, in which the critical slices were given greater consideration. The dataset was split into a training set and a testing set in a 4:1 ratio, and model performance was evaluated using five-fold cross-validation.

RESULTS

Using the proposed method, the prediction of MVI achieved an accuracy of 76.43% and an AUC of 74.22%, significantly surpassing the performance of the baseline methods.

CONCLUSION

Our modality-based attention and dual-stream MIL CNN can achieve outstanding results for MVI prediction.

Are deep models in radiomics performing better than generic models? A systematic review

BACKGROUND

Application of radiomics proceeds by extracting and analysing imaging features based on generic morphological, textural, and statistical features defined by formulas. Recently, deep learning methods were applied. It is unclear whether deep models (DMs) can outperform generic models (GMs).

METHODS

We identified publications on PubMed and Embase to determine differences between DMs and GMs in terms of receiver operating area under the curve (AUC).

RESULTS

Of 1,229 records (between 2017 and 2021), 69 studies were included, 61 (88%) on tumours, 68 (99%) retrospective, and 39 (56%) single centre; 30 (43%) used an internal validation cohort; and 18 (26%) applied cross-validation. Studies with independent internal cohort had a median training sample of 196 (range 41-1,455); those with cross-validation had only 133 (43-1,426). Median size of validation cohorts was 73 (18-535) for internal and 94 (18-388) for external. Considering the internal validation, in 74% (49/66), the DMs performed better than the GMs, vice versa in 20% (13/66); no difference in 6% (4/66); and median difference in AUC 0.045. On the external validation, DMs were better in 65% (13/20), GMs in 20% (4/20) cases; no difference in 3 (15%); and median difference in AUC 0.025. On internal validation, fused models outperformed GMs and DMs in 72% (20/28), while they were worse in 14% (4/28) and equal in 14% (4/28); median gain in AUC was + 0.02. On external validation, fused model performed better in 63% (5/8), worse in 25% (2/8), and equal in 13% (1/8); median gain in AUC was + 0.025.

CONCLUSIONS

Overall, DMs outperformed GMs but in 26% of the studies, DMs did not outperform GMs.

Radiomics: A radiological evidence-based artificial intelligence technique to facilitate personalized precision medicine in hepatocellular carcinoma

The high postoperative recurrence rates in hepatocellular carcinoma (HCC) remain a major hurdle in its management. Appropriate staging and treatment selection may alleviate the extent of fatal recurrence. However, effective methods to preoperatively evaluate pathophysiologic and molecular characteristics of HCC are lacking. Imaging plays a central role in HCC diagnosis and stratification due to the non-invasive diagnostic criteria. Vast and crucial information is hidden within image data. Other than providing a morphological sketch for lesion diagnosis, imaging could provide new insights to describe the pathophysiological and genetic landscape of HCC. Radiomics aims to facilitate diagnosis and prognosis of HCC using artificial intelligence techniques to harness the immense information contained in medical images. Radiomics produces a set of archetypal and robust imaging features that are correlated to key pathological or molecular biomarkers to preoperatively risk-stratify HCC patients. Inferred with outcome data, comprehensive combination of radiomic, clinical and/or multi-omics data could also improve direct prediction of response to treatment and prognosis. The evolution of radiomics is changing our understanding of personalized precision medicine in HCC management. Herein, we review the key techniques and clinical applications in HCC radiomics and discuss current limitations and future opportunities to improve clinical decision making.

Data augmentation for medical imaging: A systematic literature review

Recent advances in Deep Learning have largely benefited from larger and more diverse training sets. However, collecting large datasets for medical imaging is still a challenge due to privacy concerns and labeling costs. Data augmentation makes it possible to greatly expand the amount and variety of data available for training without actually collecting new samples. Data augmentation techniques range from simple yet surprisingly effective transformations such as cropping, padding, and flipping, to complex generative models. Depending on the nature of the input and the visual task, different data augmentation strategies are likely to perform differently. For this reason, it is conceivable that medical imaging requires specific augmentation strategies that generate plausible data samples and enable effective regularization of deep neural networks. Data augmentation can also be used to augment specific classes that are underrepresented in the training set, e.g., to generate artificial lesions. The goal of this systematic literature review is to investigate which data augmentation strategies are used in the medical domain and how they affect the performance of clinical tasks such as classification, segmentation, and lesion detection. To this end, a comprehensive analysis of more than 300 articles published in recent years (2018-2022) was conducted. The results highlight the effectiveness of data augmentation across organs, modalities, tasks, and dataset sizes, and suggest potential avenues for future research.

MVI-Mind: A Novel Deep-Learning Strategy Using Computed Tomography (CT)-Based Radiomics for End-to-End High Efficiency Prediction of Microvascular Invasion in Hepatocellular Carcinoma

Simple Summary

Microvascular invasion is an important indicator for reflecting the prognosis of hepatocellular carcinoma, but the traditional diagnosis requires a postoperative pathological examination. This study is the first to propose an end-to-end deep learning architecture for predicting microvascular invasion in hepatocellular carcinoma by collecting retrospective data. This method can achieve noninvasive, accurate and efficient preoperative prediction only through the patient’s radiomic data, which is very beneficial to doctors for clinical decision making in HCC patients.

Abstract

Microvascular invasion (MVI) in hepatocellular carcinoma (HCC) directly affects a patient’s prognosis. The development of preoperative noninvasive diagnostic methods is significant for guiding optimal treatment plans. In this study, we investigated 138 patients with HCC and presented a novel end-to-end deep learning strategy based on computed tomography (CT) radiomics (MVI-Mind), which integrates data preprocessing, automatic segmentation of lesions and other regions, automatic feature extraction, and MVI prediction. A lightweight transformer and a convolutional neural network (CNN) were proposed for the segmentation and prediction modules, respectively. To demonstrate the superiority of MVI-Mind, we compared the framework’s performance with that of current, mainstream segmentation, and classification models. The test results showed that MVI-Mind returned the best performance in both segmentation and prediction. The mean intersection over union (mIoU) of the segmentation module was 0.9006, and the area under the receiver operating characteristic curve (AUC) of the prediction module reached 0.9223. Additionally, it only took approximately 1 min to output a prediction for each patient, end-to-end using our computing device, which indicated that MVI-Mind could noninvasively, efficiently, and accurately predict the presence of MVI in HCC patients before surgery. This result will be helpful for doctors to make rational clinical decisions.