Deep learning for differential diagnosis of malignant hepatic tumors based on multi-phase contrast-enhanced CT and clinical data

Preliminary study on detection and diagnosis of focal liver lesions based on a deep learning model using multimodal PET/CT images

Objectives

To develop and validate a deep learning model using multimodal PET/CT imaging for detecting and classifying focal liver lesions (FLL).

Methods

This study included 185 patients who underwent 18F-FDG PET/CT imaging at our institution from March 2022 to February 2023. We analyzed serological data and imaging. Liver lesions were segmented on PET and CT, serving as the "reference standard". Deep learning models were trained using PET and CT images to generate predicted segmentations and classify lesion nature. Model performance was evaluated by comparing the predicted segmentations with the reference segmentations, using metrics such as Dice, Precision, Recall, F1-score, ROC, and AUC, and compared it with physician diagnoses.

Results

This study finally included 150 patients, comprising 46 patients with benign liver nodules, 51 patients with malignant liver nodules, and 53 patients with no FLLs. Significant differences were observed among groups for age, AST, ALP, GGT, AFP, CA19-9and CEA. On the validation set, the Dice coefficient of the model was 0.740. For the normal group, the recall was 0.918, precision was 0.904, F1-score was 0.909, and AUC was 0.976. For the benign group, the recall was 0.869, precision was 0.862, F1-score was 0.863, and AUC was 0.928. For the malignant group, the recall was 0.858, precision was 0.914, F1-score was 0.883, and AUC was 0.979. The model's overall diagnostic performance was between that of junior and senior physician.

Conclusion

This deep learning model demonstrated high sensitivity in detecting FLLs and effectively differentiated between benign and malignant lesions.

Micro-Nano Convergence-Driven Radiotheranostic Revolution in Hepatocellular Carcinoma

Radiotherapy, as an important means of treating hepatocellular carcinoma (HCC), has shown unique therapeutic advantages, especially in patients who are unable to undergo surgery or transplantation. It mainly includes external radiotherapy, transarterial radioembolization and intratumoral radioactive particle implantation. However, under the influence of factors such as the hypoxic characteristics of the liver tumor microenvironment and the radioresistance of tumor cells, the effect of radiotherapy may be unstable and may cause side effects, affecting the quality of life of patients. In recent years, with the development of nanotechnology, drug delivery systems based on micro-nanomaterials have provided new solutions for improving the effect of radiotherapy for HCC. Despite this, the application of micro-nano drug delivery systems in the treatment of HCC still faces some challenges, mainly including the in vivo safety and in vivo metabolism of micro-nano materials. This article reviews the latest progress of micro-nano materials in the treatment of HCC, especially their application in radiosensitization and their clinical translation potential. This article systematically analyzes the role of micro-nanomaterials in external or internal radiotherapy sensitization and radioimmunotherapy and explores the advantages of micro-nanomaterials in improving the treatment effect of HCC.

An Ultrasound Image-Based Deep Learning Radiomics Nomogram for Differentiating Between Benign and Malignant Indeterminate Cytology (Bethesda III) Thyroid Nodules: A Retrospective Study

RATIONALE AND OBJECTIVES

Our objective is to develop and validate a deep learning radiomics nomogram (DLRN) based on preoperative ultrasound images and clinical features, for predicting the malignancy of thyroid nodules with indeterminate cytology (Bethesda III).

MATERIALS AND METHODS

Between June 2017 and June 2022, we conducted a retrospective study on 194 patients with surgically confirmed indeterminate cytology (Bethesda III) in our hospital. The training and internal validation cohorts were comprised of 155 and 39 patients, in a 7:3 ratio. To facilitate external validation, we selected an additional 80 patients from each of the remaining two medical centers. Utilizing preoperative ultrasound data, we obtained imaging markers that encompass both deep learning and manually radiomic features. After feature selection, we developed a comprehensive diagnostic model to evaluate the predictive value for Bethesda III benign and malignant cases. The model's diagnostic accuracy, calibration, and clinical applicability were systematically assessed.

RESULTS

The results showed that the prediction model, which integrated 512 DTL features extracted from the pre-trained Resnet34 network, ultrasound radiomics, and clinical features, exhibited superior stability in distinguishing between benign and malignant indeterminate thyroid nodules (Bethesda Class III). In the validation set, the AUC was 0.92 (95% CI: 0.831-1.000), and the accuracy, sensitivity, specificity, precision, and recall were 0.897, 0.882, 0.909, 0.882, and 0.882, respectively.

CONCLUSION

The comprehensive multidimensional data model based on deep transfer learning, ultrasound radiomics features, and clinical characteristics can effectively distinguish the benign and malignant indeterminate thyroid nodules (Bethesda Class III), providing valuable guidance for treatment selection in patients with indeterminate thyroid nodules (Bethesda Class III).

Comprehensive multi-phase 3D contrast-enhanced CT imaging for primary liver cancer

Primary liver cancer is a significant global health issue with high incidence and mortality rates worldwide. Accurate diagnosis and classification of its subtypes are crucial for choosing the right treatment options and improving patient outcomes. Contrast-enhanced computed tomography (CECT) is known for its high sensitivity and specificity in diagnosing liver cancer. However, publicly available datasets of liver cancer CECT scans are limited and often do not fully cover all subtypes or include complete CT scan phases. We hypothesize that using 3D CECT images with complete scan phases can help develop and validate diagnostic and segmentation models for primary liver cancer. Therefore, we created a CECT dataset with annotated liver and lesion areas. This dataset includes 278 cases of liver cancer, featuring hepatocellular carcinoma, intrahepatic cholangiocarcinoma, and combined hepatocellular-cholangiocarcinoma, along with CECT images from 83 non-liver cancer subjects. It contains over 50,000 layers of liver cancer lesion images. We believe this dataset can offer valuable support for developing and validating models for classifying and segmenting primary liver cancer.

Deep Learning to Enhance Diagnosis and Management of Intrahepatic Cholangiocarcinoma

Intrahepatic cholangiocarcinoma (iCCA) is associated with a poor prognosis and necessitates a multimodal, multidisciplinary approach from diagnosis to treatment to achieve optimal outcomes. A noninvasive preoperative diagnosis using abdominal imaging techniques can represent a clinical challenge. Given the differential response of iCCA to localized and systemic therapies compared with hepatocellular carcinoma and secondary hepatic malignancies, an accurate diagnosis is crucial. Deep learning (DL) models for image analysis have emerged as a promising adjunct for the abdominal radiologist, potentially enhancing the accurate detection and diagnosis of iCCA. Over the last five years, several reports have proposed robust DL models, which demonstrate a diagnostic accuracy that is either comparable to or surpasses that of radiologists with varying levels of experience. Recent studies have expanded DL applications into other aspects of iCCA management, including histopathologic diagnosis, the prediction of histopathological features, the preoperative prediction of survival, and the pretreatment prediction of responses to systemic therapy. We herein critically evaluate the expanding body of research on DL applications in the diagnosis and management of iCCA, providing insights into the current progress and future research directions. We comprehensively synthesize the performance and limitations of DL models in iCCA research, identifying key challenges that serve as a translational reference for clinicians.

SDR-Former: A Siamese Dual-Resolution Transformer for liver lesion classification using 3D multi-phase imaging

Automated classification of liver lesions in multi-phase CT and MR scans is of clinical significance but challenging. This study proposes a novel Siamese Dual-Resolution Transformer (SDR-Former) framework, specifically designed for liver lesion classification in 3D multi-phase CT and MR imaging with varying phase counts. The proposed SDR-Former utilizes a streamlined Siamese Neural Network (SNN) to process multi-phase imaging inputs, possessing robust feature representations while maintaining computational efficiency. The weight-sharing feature of the SNN is further enriched by a hybrid Dual-Resolution Transformer (DR-Former), comprising a 3D Convolutional Neural Network (CNN) and a tailored 3D Transformer for processing high- and low-resolution images, respectively. This hybrid sub-architecture excels in capturing detailed local features and understanding global contextual information, thereby, boosting the SNN's feature extraction capabilities. Additionally, a novel Adaptive Phase Selection Module (APSM) is introduced, promoting phase-specific intercommunication and dynamically adjusting each phase's influence on the diagnostic outcome. The proposed SDR-Former framework has been validated through comprehensive experiments on two clinically collected datasets: a 3-phase CT dataset and an 8-phase MR dataset. The experimental results affirm the efficacy of the proposed framework. To support the scientific community, we are releasing our extensive multi-phase MR dataset for liver lesion analysis to the public. This pioneering dataset, being the first publicly available multi-phase MR dataset in this field, also underpins the MICCAI LLD-MMRI Challenge. The dataset is publicly available at: https://github.com/LMMMEng/LLD-MMRI-Dataset.

Prognostic impact of extracellular volume fraction derived from equilibrium contrast-enhanced CT in HCC patients receiving immune checkpoint inhibitors

This study aimed to investigate whether extracellular volume (ECV) fraction derived from equilibration contrast-enhanced computed tomography (CECT) affects prognosis in HCC patients receiving ICIs. This retrospective study ultimately included 211 HCC patients undergoing ICIs, of whom 60 were included in an internal validation to assess the reproducibility of the results. Baseline unenhanced and equilibrated CECT were used to measure CT values of the tumor, liver and aorta, which were combined with hematocrit to calculate the ECV fraction. Correlation analysis was used to investigate the association between tumor ECV and liver ECV fractions. The effects of clinical variables and ECV fraction on progression-free survival (PFS) and overall survival (OS) were evaluated using Cox proportional hazards models and Kaplan-Meier curves. Of these 151 patients, tumor ECV fraction positively correlated with liver ECV fraction. In the Lower tumor ECV group, PFS (5.6 vs. 7.6 months) and OS (10.5 vs. 15.5 months) were notably shorter than in the Higher tumor ECV group, while no significant differences were found between the Higher and Lower liver ECV groups. Furthermore, the multivariable Cox regression model demonstrated that higher tumor ECV fraction level was an independent protective factor for PFS and OS (all P < 0.001). Internal validation cohort preliminary demonstrated reproducibility of results. The tumor ECV fraction is expected to become a routine indicator before ICIs therapy for HCC patients in contrast to liver ECV fraction, contributing to their subsequent management.

Use of deep learning-based high-resolution magnetic resonance to identify intracranial and extracranial symptom-related plaques

This study aims to develop a deep learning model using high-resolution vessel wall imaging (HR-VWI) to differentiate symptom-related intracranial and extracranial plaques, which is crucial for stroke treatment and prevention. We retrospectively analyzed HR-VWI data from 235 patients, dividing them into a training set (n = 156) and a testing set (n = 79). Using T1-weighted and contrast-enhanced T1WI images, we constructed five deep learning models and selected the best-performing DenseNet 201 model to extract features. Traditional and radiomics features were also obtained to build both single and combined models. The models were evaluated using receiver operating characteristic curves and the area under the curve (AUC). The deep learning model showed the highest diagnostic performance, while combined models, particularly T + D, performed well, though not better than the single deep learning model. The deep learning model based on HR-VWI is superior in discriminating symptom-related plaques and offers valuable guidance for plaque management.

Predicting treatment response and prognosis of immune checkpoint inhibitors-based combination therapy in advanced hepatocellular carcinoma using a longitudinal CT-based radiomics model: a multicenter study

BACKGROUND

Identifying effective predictive strategies to assess the response of immune checkpoint inhibitors (ICIs)-based combination therapy in advanced hepatocellular carcinoma (HCC) is crucial. This study presents a new longitudinal CT-based radiomics model to predict treatment response and prognosis in advanced HCC patients undergoing ICIs-based combination therapy.

METHODS

Longitudinal CT images were collected before and during the treatment for HCC patients across three institutions from January 2019 to April 2022. A total of 1316 radiomic features were extracted from arterial and portal venous phase abdominal CT images for each patient. A model called Longitudinal Whole-liver CT-based Radiomics (LWCTR) was developed to categorize patients into responders or non-responders using radiomic features and clinical information through support vector machine (SVM) classifiers. The area under the curve (AUC) was used as the performance metric and subsequently applied for risk stratification and prognostic assessment. The Shapley Additive explanations (SHAP) method was used to calculate the Shapley value, which explains the contribution of each feature in the SVM model to the prediction.

RESULTS

This study included 395 eligible participants, with a median age of 57 years (IQR 51-66), comprising 344 males and 51 females. The LWCTR model performed well in predicting treatment response, achieving an AUC of 0.883 (95% confidence interval [CI] 0.881-0.888) in the training cohort, 0.876 (0.858-0.895) in the internal validation cohort, and 0.875 (0.860-0.887) in the external test cohort. The Rad-Nomo model, integrating the LWCTR model's prediction score (Rad-score) with the modified Response Evaluation Criteria in Solid Tumors (mRECIST), demonstrated strong prognostic performance. It achieved time-dependent AUC values of 0.902, 0.823, and 0.850 at 1, 2, and 3 years in the internal validation cohort and 0.893, 0.848, and 0.762 at the same intervals in the external test cohort.

CONCLUSION

The proposed LWCTR model performs well in predicting treatment response and prognosis in patients with HCC receiving ICIs-based combination therapy, potentially contributing to personalized and timely treatment decisions.

Artificial intelligence in gastrointestinal cancer research: Image learning advances and applications

With the rapid advancement of artificial intelligence (AI) technologies, including deep learning, large language models, and neural networks, these methodologies are increasingly being developed and integrated into cancer research. Gastrointestinal tumors are characterized by complexity and heterogeneity, posing significant challenges for early detection, diagnostic accuracy, and the development of personalized treatment strategies. The application of AI in digestive oncology has demonstrated its transformative potential. AI not only alleviates the diagnostic burden on clinicians, but it improves tumor screening sensitivity, specificity, and accuracy. Additionally, AI aids the detection of biomarkers such as microsatellite instability and mismatch repair, supports intraoperative assessments of tumor invasion depth, predicts treatment responses, and facilitates the design of personalized treatment plans to potentially significantly enhance patient outcomes. Moreover, the integration of AI with multiomics analyses and imaging technologies has led to substantial advancements in foundational research on the tumor microenvironment. This review highlights the progress of AI in gastrointestinal oncology over the past 5 years with focus on early tumor screening, diagnosis, molecular marker identification, treatment planning, and prognosis predictions. We also explored the potential of AI to enhance medical imaging analyses to aid tumor detection and characterization as well as its role in automating and refining histopathological assessments.

Artificial intelligence for detection and characterization of focal hepatic lesions: a review

Focal liver lesions (FLL) are common incidental findings in abdominal imaging. While the majority of FLLs are benign and asymptomatic, some can be malignant or pre-malignant, and need accurate detection and classification. Current imaging techniques, such as computed tomography (CT) and magnetic resonance imaging (MRI), play a crucial role in assessing these lesions. Artificial intelligence (AI), particularly deep learning (DL), offers potential solutions by analyzing large data to identify patterns and extract clinical features that aid in the early detection and classification of FLLs. This manuscript reviews the diagnostic capacity of AI-based algorithms in processing CT and MRIs to detect benign and malignant FLLs, with an emphasis in the characterization and classification of these lesions and focusing on differentiating benign from pre-malignant and potentially malignant lesions. A comprehensive literature search from January 2010 to April 2024 identified 45 relevant studies. The majority of AI systems employed convolutional neural networks (CNNs), with expert radiologists providing reference standards through manual lesion delineation, and histology as the gold standard. The studies reviewed indicate that AI-based algorithms demonstrate high accuracy, sensitivity, specificity, and AUCs in detecting and characterizing FLLs. These algorithms excel in differentiating between benign and malignant lesions, optimizing diagnostic protocols, and reducing the needs of invasive procedures. Future research should concentrate on the expansion of data sets, the improvement of model explainability, and the validation of AI tools across a range of clinical setting to ensure the applicability and reliability of such tools.

The value of predicting breast cancer with a DBT 2.5D deep learning model

OBJECTIVE

To evaluate the accuracy and efficacy of a 2.5-dimensional deep learning (DL) model based on digital breast tomosynthesis (DBT) in predicting breast cancer.

METHODS

Through a retrospective analysis of data from 361 patients with breast tumor lesions treated at Shandong Provincial Hospital Affiliated to Shandong First Medical University between 2018 and 2020, this study utilized deep convolutional neural networks (DCNN) to automatically extract key features from DBT images. By applying dimensionality reduction and feature fusion selection, a variety of machine learning predictive models based on a 2.5-dimensional feature set were constructed. Additionally, a comprehensive predictive model was developed by combining univariate and multivariate logistic regression analyses with clinical data. The model's performance was assessed using receiver operating characteristic (ROC) curves, area under the curve (AUC) values, and accuracy rates.

RESULTS

In the test set, DBT 2.5D deep learning-based logistic regression, LightGBM, multilayer perceptron, and comprehensive models achieved accuracies of 72.2%, 75.0%, 79.2%, and 80.6%; AUCs of 0.826, 0.756, 0.859, and 0.871; sensitivities of 63.8%, 70.2%, 80.9%, and 87.2%; specificities of 88.0%, 84.0%, 76.0%, and 68.0%; PPVs of 90.9%, 89.2%, 86.4%, and 83.7%; NPVs of 56.4%, 60.0%, 67.9%, and 73.9%; and F1 scores of 75.0%, 78.6%, 83.5%, and 85.4%, respectively. These results underscore the high efficiency and potential of DBT 2.5D deep learning models in breast cancer diagnosis, particularly the comprehensive model's superior performance across key metrics.

CONCLUSION

The 2.5D deep learning model based on DBT shows good performance in preoperative breast cancer prediction, with its integration with clinical data further enhancing its effectiveness. The combination of deep learning and radiomics offers a viable approach for early breast cancer diagnosis, supporting the development of more accurate personalized diagnostic and treatment strategies.

Comparison of MRI and CT based deep learning radiomics analyses and their combination for diagnosing intrahepatic cholangiocarcinoma

Intrahepatic cholangiocarcinoma (iCCA) and other subtypes of primary liver cancer (PLC) have overlapping clinical manifestations and radiological characteristics. The objective of this study was to evaluate the efficacy of deep learning (DL) radiomics analysis, performed using computed tomography (CT) and magnetic resonance imaging (MRI), in diagnosing iCCA within PLC. 178 pathologically confirmed PLC patients (training cohort: test cohort = 124: 54) who underwent both CT and MRI examinations was enrolled. Univariate and multivariate analysis was used to identify the significant factors of radiological findings for diagnosing iCCA. DL radiomics analysis was applied to CT and MRI images, respectively. We constructed and evaluated six distinct models: CT DL radiomics (DLRSCT), CT radiological (RCT), CT DL radiomics-radiological (DLRRCT), MRI DL radiomics (DLRSMRI), MRI radiological (RMRI) and MRI DL radiomics-radiological (DLRRMRI). To further explore the diagnostic and predictive value of a cross-modal approach, we developed a fused model that combined DLRRCT and DLRRMRI. Receiver operating characteristic (ROC) curves, calibration curves, and decision curve analysis (DCA) were employed to compare the performance of different models. MRI-based models demonstrated a superior predictive performance than CT-based models in test cohort (AUCs of MRI vs. CT: DLRR, 0.923 vs. 0.880, P = 0.521; DLRS, 0.875 vs. 0.867, P = 0.922; R, 0.859 vs. 0.840, P = 0.808). The CT-MRI cross-modal model yielded the highest AUC of 0.994 and 0.937 in training and test cohorts, respectively. CT- and MRI-based DL radiomics analyses exhibited good performance in diagnosing iCCA, and the CT-MRI cross-modal model may have significant clinical implications on detection of liver malignancies.

Applications of artificial intelligence in interventional oncology: An up-to-date review of the literature

Interventional oncology provides image-guided therapies, including transarterial tumor embolization and percutaneous tumor ablation, for malignant tumors in a minimally invasive manner. As in other medical fields, the application of artificial intelligence (AI) in interventional oncology has garnered significant attention. This narrative review describes the current state of AI applications in interventional oncology based on recent literature. A literature search revealed a rapid increase in the number of studies relevant to this topic recently. Investigators have attempted to use AI for various tasks, including automatic segmentation of organs, tumors, and treatment areas; treatment simulation; improvement of intraprocedural image quality; prediction of treatment outcomes; and detection of post-treatment recurrence. Among these, the AI-based prediction of treatment outcomes has been the most studied. Various deep and conventional machine learning algorithms have been proposed for these tasks. Radiomics has often been incorporated into prediction and detection models. Current literature suggests that AI is potentially useful in various aspects of interventional oncology, from treatment planning to post-treatment follow-up. However, most AI-based methods discussed in this review are still at the research stage, and few have been implemented in clinical practice. To achieve widespread adoption of AI technologies in interventional oncology procedures, further research on their reliability and clinical utility is necessary. Nevertheless, considering the rapid research progress in this field, various AI technologies will be integrated into interventional oncology practices in the near future.

MRI-based deep learning radiomics to differentiate dual-phenotype hepatocellular carcinoma from HCC and intrahepatic cholangiocarcinoma: a multicenter study

OBJECTIVES

To develop and validate radiomics and deep learning models based on contrast-enhanced MRI (CE-MRI) for differentiating dual-phenotype hepatocellular carcinoma (DPHCC) from HCC and intrahepatic cholangiocarcinoma (ICC).

METHODS

Our study consisted of 381 patients from four centers with 138 HCCs, 122 DPHCCs, and 121 ICCs (244 for training and 62 for internal tests, centers 1 and 2; 75 for external tests, centers 3 and 4). Radiomics, deep transfer learning (DTL), and fusion models based on CE-MRI were established for differential diagnosis, respectively, and their diagnostic performances were compared using the confusion matrix and area under the receiver operating characteristic (ROC) curve (AUC).

RESULTS

The radiomics model demonstrated competent diagnostic performance, with a macro-AUC exceeding 0.9, and both accuracy and F1-score above 0.75 in the internal and external validation sets. Notably, the vgg19-combined model outperformed the radiomics and other DTL models. The fusion model based on vgg19 further improved diagnostic performance, achieving a macro-AUC of 0.990 (95% CI: 0.965-1.000), an accuracy of 0.935, and an F1-score of 0.937 in the internal test set. In the external test set, it similarly performed well, with a macro-AUC of 0.988 (95% CI: 0.964-1.000), accuracy of 0.875, and an F1-score of 0.885.

CONCLUSIONS

Both the radiomics and the DTL models were able to differentiate DPHCC from HCC and ICC before surgery. The fusion models showed better diagnostic accuracy, which has important value in clinical application.

CRITICAL RELEVANCE STATEMENT

MRI-based deep learning radiomics were able to differentiate DPHCC from HCC and ICC preoperatively, aiding clinicians in the identification and targeted treatment of these malignant hepatic tumors.

KEY POINTS

Fusion models may yield an incremental value over radiomics models in differential diagnosis. Radiomics and deep learning effectively differentiate the three types of malignant hepatic tumors. The fusion models may enhance clinical decision-making for malignant hepatic tumors.

18F-FAPI-42 PET/CT for preoperatively identifying intrahepatic cholangiocarcinoma and hepatocellular carcinoma

Background

Accurately differentiating hepatocellular carcinoma (HCC) from intrahepatic cholangiocarcinoma (ICC) is essential for therapeutic decision-making. This study aimed to explore the value of Fluor 18 (18F)-conjugated fibroblast-activation protein inhibitor (FAPI-42) positron emission tomography-computed tomography (PET/CT) in distinguishing HCC from ICC preoperatively.

Methods

Patients with suspected intrahepatic lesions who underwent 18F-FAPI-42 PET/CT were retrospectively assessed and placed into an HCC group and an ICC group based on postoperative pathology. Clinical indicators and PET/CT metabolic parameters were documented. Univariate and multivariate logistic regression analyses identified the independent predictive factors. The receiver operating characteristic curve and the area under the receiver operating curve (AUC) were used to determine the diagnostic efficacy.

Results

A total of 48 patients (age 55.4±10.8 years; 35 males and 13 females), including 28 in the HCC and 20 in the ICC group, were included. Univariate analysis revealed significant differences in clinical indicators such as gender, platelet count, and hepatitis B surface antigen, carbohydrate antigen 19-9 (CA19-9), and alpha fetoprotein (AFP) levels between the groups (P<0.05). Metabolic parameters including the maximum standardized uptake value (SUVmax), the total lesion-FAPI (TL-FAPI), and the target-to-background ratio (TBR) were statistically different between groups (P<0.01), and patients with ICC had higher values than those with HCC, while the FAPI tumor volume (FAPI-TV) was not statistically different (P>0.05). Multivariate binary logistic regression analysis identified the SUVmax (P=0.003) as an independent predictor for ICC, and the AUC of the regression-based diagnostic model for predicting ICC was 0.914 at an optimal cutoff value of 12.13, with a sensitivity of 85.0% and a specificity of 89.3%.

Conclusions

SUVmax is an independent predictor for ICC. With the optimal cutoff value of 12.13, 18F-FAPI-42 PET/CT indicates good diagnostic efficacy in distinguishing between HCC and ICC.

Application of a deep learning algorithm for the diagnosis of HCC

Background & Aims

Hepatocellular carcinoma (HCC) is characterized by a high mortality rate. The Liver Imaging Reporting and Data System (LI-RADS) results in a considerable number of indeterminate observations, rendering an accurate diagnosis difficult.

Methods

We developed four deep learning models for diagnosing HCC on computed tomography (CT) via a training-validation-testing approach. Thin-slice triphasic CT liver images and relevant clinical information were collected and processed for deep learning. HCC was diagnosed and verified via a 12-month clinical composite reference standard. CT observations among at-risk patients were annotated using LI-RADS. Diagnostic performance was assessed by internal validation and independent external testing. We conducted sensitivity analyses of different subgroups, deep learning explainability evaluation, and misclassification analysis.

Results

From 2,832 patients and 4,305 CT observations, the best-performing model was Spatio-Temporal 3D Convolution Network (ST3DCN), achieving area under receiver-operating-characteristic curves (AUCs) of 0.919 (95% CI, 0.903-0.935) and 0.901 (95% CI, 0.879-0.924) at the observation (n = 1,077) and patient (n = 685) levels, respectively during internal validation, compared with 0.839 (95% CI, 0.814-0.864) and 0.822 (95% CI, 0.790-0.853), respectively for standard of care radiological interpretation. The negative predictive values of ST3DCN were 0.966 (95% CI, 0.954-0.979) and 0.951 (95% CI, 0.931-0.971), respectively. The observation-level AUCs among at-risk patients, 2-5-cm observations, and singular portovenous phase analysis of ST3DCN were 0.899 (95% CI, 0.874-0.924), 0.872 (95% CI, 0.838-0.909) and 0.912 (95% CI, 0.895-0.929), respectively. In external testing (551/717 patients/observations), the AUC of ST3DCN was 0.901 (95% CI, 0.877-0.924), which was non-inferior to radiological interpretation (AUC 0.900; 95% CI, 0.877--923).

Conclusions

ST3DCN achieved strong, robust performance for accurate HCC diagnosis on CT. Thus, deep learning can expedite and improve the process of diagnosing HCC.

Impact and implications

The clinical applicability of deep learning in HCC diagnosis is potentially huge, especially considering the expected increase in the incidence and mortality of HCC worldwide. Early diagnosis through deep learning can lead to earlier definitive management, particularly for at-risk patients. The model can be broadly deployed for patients undergoing a triphasic contrast CT scan of the liver to reduce the currently high mortality rate of HCC.

Diagnostic Performance of Deep Learning Applications in Hepatocellular Carcinoma Detection Using Computed Tomography Imaging

Hepatocellular carcinoma (HCC) is a prevalent cancer that significantly contributes to mortality globally, primarily due to its late diagnosis. Early detection is crucial yet challenging. This study leverages the potential of deep learning (DL) technologies, employing the You Only Look Once (YOLO) architecture, to enhance the detection of HCC in computed tomography (CT) images, aiming to improve early diagnosis and thereby patient outcomes. We used a dataset of 1290 CT images from 122 patients, segmented according to a standard 70:20:10 split for training, validation, and testing phases. The YOLO-based DL model was trained on these images, with subsequent phases for validation and testing to assess the model's diagnostic capabilities comprehensively. The model exhibited exceptional diagnostic accuracy, with a precision of 0.97216, recall of 0.919, and an overall accuracy of 95.35%, significantly surpassing traditional diagnostic approaches. It achieved a specificity of 95.83% and a sensitivity of 94.74%, evidencing its effectiveness in clinical settings and its potential to reduce the rate of missed diagnoses and unnecessary interventions. The implementation of the YOLO architecture for detecting HCC in CT scans has shown substantial promise, indicating that DL models could soon become a standard tool in oncological diagnostics. As artificial intelligence technology continues to evolve, its integration into healthcare systems is expected to advance the accuracy and efficiency of diagnostics in oncology, enhancing early detection and treatment strategies and potentially improving patient survival rates.

Advancements in Artificial Intelligence-Enhanced Imaging Diagnostics for the Management of Liver Disease-Applications and Challenges in Personalized Care

Liver disease can significantly impact life expectancy, making early diagnosis and therapeutic intervention critical challenges in medical care. Imaging diagnostics play a crucial role in diagnosing and managing liver diseases. Recently, the application of artificial intelligence (AI) in medical imaging analysis has become indispensable in healthcare. AI, trained on vast datasets of medical images, has sometimes demonstrated diagnostic accuracy that surpasses that of human experts. AI-assisted imaging diagnostics are expected to contribute significantly to the standardization of diagnostic quality. Furthermore, AI has the potential to identify image features that are imperceptible to humans, thereby playing an essential role in clinical decision-making. This capability enables physicians to make more accurate diagnoses and develop effective treatment strategies, ultimately improving patient outcomes. Additionally, AI is anticipated to become a powerful tool in personalized medicine. By integrating individual patient imaging data with clinical information, AI can propose optimal plans for treatment, making it an essential component in the provision of the most appropriate care for each patient. Current reports highlight the advantages of AI in managing liver diseases. As AI technology continues to evolve, it is expected to advance personalized diagnostics and treatments and contribute to overall improvements in healthcare quality.

Self-supervised multi-modal feature fusion for predicting early recurrence of hepatocellular carcinoma

Surgical resection stands as the primary treatment option for early-stage hepatocellular carcinoma (HCC) patients. Postoperative early recurrence (ER) is a significant factor contributing to the mortality of HCC patients. Therefore, accurately predicting the risk of ER after curative resection is crucial for clinical decision-making and improving patient prognosis. This study leverages a self-supervised multi-modal feature fusion approach, combining multi-phase MRI and clinical features, to predict ER of HCC. Specifically, we utilized attention mechanisms to suppress redundant features, enabling efficient extraction and fusion of multi-phase features. Through self-supervised learning (SSL), we pretrained an encoder on our dataset to extract more generalizable feature representations. Finally, we achieved effective multi-modal information fusion via attention modules. To enhance explainability, we employed Score-CAM to visualize the key regions influencing the model's predictions. We evaluated the effectiveness of the proposed method on our dataset and found that predictions based on multi-phase feature fusion outperformed those based on single-phase features. Additionally, predictions based on multi-modal feature fusion were superior to those based on single-modal features.

Multimodal deep learning approaches for precision oncology: a comprehensive review

The burgeoning accumulation of large-scale biomedical data in oncology, alongside significant strides in deep learning (DL) technologies, has established multimodal DL (MDL) as a cornerstone of precision oncology. This review provides an overview of MDL applications in this field, based on an extensive literature survey. In total, 651 articles published before September 2024 are included. We first outline publicly available multimodal datasets that support cancer research. Then, we discuss key DL training methods, data representation techniques, and fusion strategies for integrating multimodal data. The review also examines MDL applications in tumor segmentation, detection, diagnosis, prognosis, treatment selection, and therapy response monitoring. Finally, we critically assess the limitations of current approaches and propose directions for future research. By synthesizing current progress and identifying challenges, this review aims to guide future efforts in leveraging MDL to advance precision oncology.

Evolving and Novel Applications of Artificial Intelligence in Abdominal Imaging

Advancements in artificial intelligence (AI) have significantly transformed the field of abdominal radiology, leading to an improvement in diagnostic and disease management capabilities. This narrative review seeks to evaluate the current standing of AI in abdominal imaging, with a focus on recent literature contributions. This work explores the diagnosis and characterization of hepatobiliary, pancreatic, gastric, colonic, and other pathologies. In addition, the role of AI has been observed to help differentiate renal, adrenal, and splenic disorders. Furthermore, workflow optimization strategies and quantitative imaging techniques used for the measurement and characterization of tissue properties, including radiomics and deep learning, are highlighted. An assessment of how these advancements enable more precise diagnosis, tumor description, and body composition evaluation is presented, which ultimately advances the clinical effectiveness and productivity of radiology. Despite the advancements of AI in abdominal imaging, technical, ethical, and legal challenges persist, and these challenges, as well as opportunities for future development, are highlighted.

A new artificial intelligence system for both stomach and small-bowel capsule endoscopy

BACKGROUND AND AIMS

Despite the benefits of artificial intelligence in small-bowel (SB) capsule endoscopy (CE) image reading, information on its application in the stomach and SB CE is lacking.

METHODS

In this multicenter, retrospective diagnostic study, gastric imaging data were added to the deep learning-based SmartScan (SS), which has been described previously. A total of 1069 magnetically controlled GI CE examinations (comprising 2,672,542 gastric images) were used in the training phase for recognizing gastric pathologies, producing a new artificial intelligence algorithm named SS Plus. A total of 342 fully automated, magnetically controlled CE examinations were included in the validation phase. The performance of both senior and junior endoscopists with both the SS Plus-assisted reading (SSP-AR) and conventional reading (CR) modes was assessed.

RESULTS

SS Plus was designed to recognize 5 types of gastric lesions and 17 types of SB lesions. SS Plus reduced the number of CE images required for review to 873.90 (median, 1000; interquartile range [IQR], 814.50-1000) versus 44,322.73 (median, 42,393; IQR, 31,722.75-54,971.25) for CR. Furthermore, with SSP-AR, endoscopists took 9.54 minutes (median, 8.51; IQR, 6.05-13.13) to complete the CE video reading. In the 342 CE videos, SS Plus identified 411 gastric and 422 SB lesions, whereas 400 gastric and 368 intestinal lesions were detected with CR. Moreover, junior endoscopists remarkably improved their CE image reading ability with SSP-AR.

CONCLUSIONS

Our study shows that the newly upgraded deep learning-based algorithm SS Plus can detect GI lesions and help improve the diagnostic performance of junior endoscopists in interpreting CE videos.

Four-phase CT lesion recognition based on multi-phase information fusion framework and spatiotemporal prediction module

Multiphase information fusion and spatiotemporal feature modeling play a crucial role in the task of four-phase CT lesion recognition. In this paper, we propose a four-phase CT lesion recognition algorithm based on multiphase information fusion framework and spatiotemporal prediction module. Specifically, the multiphase information fusion framework uses the interactive perception mechanism to realize the channel-spatial information interactive weighting between multiphase features. In the spatiotemporal prediction module, we design a 1D deep residual network to integrate multiphase feature vectors, and use the GRU architecture to model the temporal enhancement information between CT slices. In addition, we employ CT image pseudo-color processing for data augmentation and train the whole network based on a multi-task learning framework. We verify the proposed network on a four-phase CT dataset. The experimental results show that the proposed network can effectively fuse the multi-phase information and model the temporal enhancement information between CT slices, showing excellent performance in lesion recognition.

Novel deep learning radiomics nomogram-based multiparametric MRI for predicting the lymph node metastasis in rectal cancer: A dual-center study

PURPOSE

To develop and evaluate a nomogram that integrates clinical parameters with deep learning radiomics (DLR) extracted from Magnetic Resonance Imaging (MRI) data to enhance the predictive accuracy for preoperative lymph node (LN) metastasis in rectal cancer.

METHODS

A retrospective analysis was conducted on 356 patients diagnosed with rectal cancer. Of these, 286 patients were allocated to the training set, and 70 patients comprised the external validation cohort. Preprocessed T2-weighted and diffusion-weighted imaging performed preoperatively facilitated the extraction of DLR features. Five machine learning algorithms-k-nearest neighbor, light gradient boosting machine, logistic regression, random forest, and support vector machine-were utilized to develop DLR models. The most effective algorithm was identified and used to establish a clinical DLR (CDLR) nomogram specifically designed to predict LN metastasis in rectal cancer. The performance of the nomogram was evaluated using receiver operating characteristic curve analysis.

RESULTS

The logistic regression classifier demonstrated significant predictive accuracy using the DLR signature, achieving an Area Under the Curve (AUC) of 0.919 in the training cohort and 0.778 in the external validation cohort. The integrated CDLR nomogram exhibited robust predictive performance across both datasets, with AUC values of 0.921 in the training cohort and 0.818 in the external validation cohort. Notably, it outperformed both the clinical model, which had AUC values of 0.770 and 0.723 in the training and external validation cohorts, respectively, and the stand-alone DLR model.

CONCLUSION

The nomogram derived from multiparametric MRI data, referred to as the CDLR model, demonstrates strong predictive efficacy in forecasting LN metastasis in rectal cancer.

A flexible deep learning framework for liver tumor diagnosis using variable multi-phase contrast-enhanced CT scans

BACKGROUND

Liver cancer is a significant cause of cancer-related mortality worldwide and requires tailored treatment strategies for different types. However, preoperative accurate diagnosis of the type presents a challenge. This study aims to develop an automatic diagnostic model based on multi-phase contrast-enhanced CT (CECT) images to distinguish between hepatocellular carcinoma (HCC), intrahepatic cholangiocarcinoma (ICC), and normal individuals.

METHODS

We designed a Hierarchical Long Short-Term Memory (H-LSTM) model, whose core components consist of a shared image feature extractor across phases, an internal LSTM for each phase, and an external LSTM across phases. The internal LSTM aggregates features from different layers of 2D CECT images, while the external LSTM aggregates features across different phases. H-LSTM can handle incomplete phases and varying numbers of CECT image layers, making it suitable for real-world decision support scenarios. Additionally, we applied phase augmentation techniques to process multi-phase CECT images, improving the model's robustness.

RESULTS

The H-LSTM model achieved an overall average AUROC of 0.93 (0.90, 1.00) on the test dataset, with AUROC for HCC classification reaching 0.97 (0.93, 1.00) and for ICC classification reaching 0.90 (0.78, 1.00). Comprehensive validation in scenarios with incomplete phases was performed, with the H-LSTM model consistently achieving AUROC values over 0.9.

CONCLUSION

The proposed H-LSTM model can be employed for classification tasks involving incomplete phases of CECT images in real-world scenarios, demonstrating high performance. This highlights the potential of AI-assisted systems in achieving accurate diagnosis and treatment of liver cancer. H-LSTM offers an effective solution for processing multi-phase data and provides practical value for clinical diagnostics.

Artificial intelligence techniques in liver cancer

Hepatocellular Carcinoma (HCC), the most common primary liver cancer, is a significant contributor to worldwide cancer-related deaths. Various medical imaging techniques, including computed tomography, magnetic resonance imaging, and ultrasound, play a crucial role in accurately evaluating HCC and formulating effective treatment plans. Artificial Intelligence (AI) technologies have demonstrated potential in supporting physicians by providing more accurate and consistent medical diagnoses. Recent advancements have led to the development of AI-based multi-modal prediction systems. These systems integrate medical imaging with other modalities, such as electronic health record reports and clinical parameters, to enhance the accuracy of predicting biological characteristics and prognosis, including those associated with HCC. These multi-modal prediction systems pave the way for predicting the response to transarterial chemoembolization and microvascular invasion treatments and can assist clinicians in identifying the optimal patients with HCC who could benefit from interventional therapy. This paper provides an overview of the latest AI-based medical imaging models developed for diagnosing and predicting HCC. It also explores the challenges and potential future directions related to the clinical application of AI techniques.

Diagnostic performance of AI-based models versus physicians among patients with hepatocellular carcinoma: a systematic review and meta-analysis

Background

Hepatocellular carcinoma (HCC) is a common primary liver cancer that requires early diagnosis due to its poor prognosis. Recent advances in artificial intelligence (AI) have facilitated hepatocellular carcinoma detection using multiple AI models; however, their performance is still uncertain.

Aim

This meta-analysis aimed to compare the diagnostic performance of different AI models with that of clinicians in the detection of hepatocellular carcinoma.

Methods

We searched the PubMed, Scopus, Cochrane Library, and Web of Science databases for eligible studies. The R package was used to synthesize the results. The outcomes of various studies were aggregated using fixed-effect and random-effects models. Statistical heterogeneity was evaluated using I-squared (I2) and chi-square statistics.

Results

We included seven studies in our meta-analysis;. Both physicians and AI-based models scored an average sensitivity of 93%. Great variation in sensitivity, accuracy, and specificity was observed depending on the model and diagnostic technique used. The region-based convolutional neural network (RCNN) model showed high sensitivity (96%). Physicians had the highest specificity in diagnosing hepatocellular carcinoma(100%); furthermore, models-based convolutional neural networks achieved high sensitivity. Models based on AI-assisted Contrast-enhanced ultrasound (CEUS) showed poor accuracy (69.9%) compared to physicians and other models. The leave-one-out sensitivity revealed high heterogeneity among studies, which represented true differences among the studies.

Conclusion

Models based on Faster R-CNN excel in image classification and data extraction, while both CNN-based models and models combining contrast-enhanced ultrasound (CEUS) with artificial intelligence (AI) had good sensitivity. Although AI models outperform physicians in diagnosing HCC, they should be utilized as supportive tools to help make more accurate and timely decisions.

Focal liver lesion diagnosis with deep learning and multistage CT imaging

Diagnosing liver lesions is crucial for treatment choices and patient outcomes. This study develops an automatic diagnosis system for liver lesions using multiphase enhanced computed tomography (CT). A total of 4039 patients from six data centers are enrolled to develop Liver Lesion Network (LiLNet). LiLNet identifies focal liver lesions, including hepatocellular carcinoma (HCC), intrahepatic cholangiocarcinoma (ICC), metastatic tumors (MET), focal nodular hyperplasia (FNH), hemangioma (HEM), and cysts (CYST). Validated in four external centers and clinically verified in two hospitals, LiLNet achieves an accuracy (ACC) of 94.7% and an area under the curve (AUC) of 97.2% for benign and malignant tumors. For HCC, ICC, and MET, the ACC is 88.7% with an AUC of 95.6%. For FNH, HEM, and CYST, the ACC is 88.6% with an AUC of 95.9%. LiLNet can aid in clinical diagnosis, especially in regions with a shortage of radiologists.

Integrating intratumoral and peritumoral radiomics with deep transfer learning for DCE-MRI breast lesion differentiation: A multicenter study comparing performance with radiologists

PURPOSE

To conduct the fusion of radiomics and deep transfer learning features from the intratumoral and peritumoral areas in breast DCE-MRI images to differentiate between benign and malignant breast tumors, and to compare the diagnostic accuracy of this fusion model against the assessments made by experienced radiologists.

MATERIALS AND METHODS

This multi-center study conducted a retrospective analysis of DCE-MRI images from 330 women diagnosed with breast cancer, with 138 cases categorized as benign and 192 as malignant. The training and internal testing sets comprised 270 patients from center 1, while the external testing cohort consisted of 60 patients from center 2. A fusion feature set consisting of radiomics features and deep transfer learning features was constructed from both intratumoral (ITR) and peritumoral (PTR) areas. The Least absolute shrinkage and selection operator (LASSO) based support vector machine was chosen as the classifier by comparing its performance with five other machine learning models. The diagnostic performance and clinical usefulness of fusion model were verified and assessed through the area under the receiver operating characteristics (ROC) and decision curve analysis. Additionally, the performance of the fusion model was compared with the diagnostic assessments of two experienced radiologists to evaluate its relative accuracy. The study strictly adhered to CLEAR and METRICS guidelines for standardization to ensure rigorous and reproducible methods.

RESULTS

The findings show that the fusion model, utilizing radiomics and deep transfer learning features from the ITR and PTR, exhibited exceptional performance in classifying breast tumors, achieving AUCs of 0.950 in the internal testing set and 0.921 in the external testing set. This performance significantly surpasses that of models relying on singular regional radiomics or deep transfer learning features alone. Moreover, the fusion model demonstrated superior diagnostic accuracy compared to the evaluations conducted by two experienced radiologists, thereby highlighting its potential to support and enhance clinical decision-making in the differentiation of benign and malignant breast tumors.

CONCLUSION

The fusion model, combining multi-regional radiomics with deep transfer learning features, not only accurately differentiates between benign and malignant breast tumors but also outperforms the diagnostic assessments made by experienced radiologists. This underscores the model's potential as a valuable tool for improving the accuracy and reliability of breast tumor diagnosis.

Pretreatment CT-based machine learning radiomics model predicts response in unresectable hepatocellular carcinoma treated with lenvatinib plus PD-1 inhibitors and interventional therapy

BACKGROUND

Lenvatinib plus PD-1 inhibitors and interventional (LPI) therapy have demonstrated promising treatment effects in unresectable hepatocellular carcinoma (HCC). However, biomarkers for predicting the response to LPI therapy remain to be further explored. We aimed to develop a radiomics model to noninvasively predict the efficacy of LPI therapy.

METHODS

Clinical data of patients with HCC receiving LPI therapy were collected in our institution. The clinical model was built with clinical information. Nine machine learning classifiers were tested and the multilayer perceptron classifier with optimal performance was used as the radiomics model. The clinical-radiomics model was constructed by integrating clinical and radiomics scores through logistic regression analysis.

RESULTS

151 patients were enrolled in this study (2:1 randomization, 101 and 50 in the training and validation cohorts), of which three achieved complete response, 69 showed partial response, 46 showed stable disease, and 33 showed progressive disease. The objective response rate, disease control rate, and conversion resection rates were 47.7, 78.1 and 23.2%. 14 features were selected from the initially extracted 1223 for radiomics model construction. The area under the curves of the radiomics model (0.900 for training and 0.893 for validation) were comparable to that of the clinical-radiomics model (0.912 for training and 0.892 for validation), and both were superior to the clinical model (0.669 for training and 0.585 for validation). Meanwhile, the radiomics model can categorize participants into high-risk and low-risk groups for progression-free survival (PFS) and overall survival (OS) in the training (HR 1.913, 95% CI 1.121 to 3.265, p=0.016 for PFS; HR 4.252, 95% CI 2.051 to 8.816, p=0.001 for OS) and validation sets (HR 2.347, 95% CI 1.095 to 5.031, p=0.012 for PFS; HR 2.592, 95% CI 1.050 to 6.394, p=0.019 for OS).

CONCLUSION

The promising machine learning radiomics model was developed and validated to predict the efficacy of LPI therapy for patients with HCC and perform risk stratification, with comparable performance to clinical-radiomics model.

Research Progress in Predicting Hepatocellular Carcinoma with Portal Vein Tumour Thrombus in the Era of Artificial Intelligence

Hepatocellular Carcinoma (HCC) is a condition associated with significant morbidity and mortality. The presence of Portal Vein Tumour Thrombus (PVTT) typically signifies advanced disease stages and poor prognosis. Artificial intelligence (AI), particularly Machine Learning (ML) and Deep Learning (DL), has emerged as a promising tool for extracting quantitative data from medical images. AI is increasingly integrated into the imaging omics workflow and has become integral to various medical disciplines. This paper provides a comprehensive review of the mechanisms underlying the formation and progression of PVTT, as well as its impact on clinical management and prognosis. Additionally, it outlines the advancements in AI for predicting the diagnosis of HCC and the development of PVTT. The limitations of existing studies are critically evaluated, and potential future research directions in the realm of imaging for the diagnostic prediction of HCC and PVTT are discussed, with the ultimate goal of enhancing survival outcomes for PVTT patients.

Predicting Survival Using Whole-Liver MRI Radiomics in Patients with Hepatocellular Carcinoma After TACE Refractoriness

PURPOSE

To develop a model based on whole-liver radiomics features of pre-treatment enhanced MRI for predicting the prognosis of hepatocellular carcinoma (HCC) patients undergoing continued transarterial chemoembolization (TACE) after TACE-resistance.

MATERIALS AND METHODS

Data from 111 TACE-resistant HCC patients between January 2014 and March 2018 were retrospectively collected. At a ratio of 7:3, patients were randomly assigned to developing and validation cohorts. The whole-liver were manually segmented, and the radiomics signature was extracted. The tumor and liver radiomics score (TLrad-score) was calculated. Models were trained by machine learning algorithms and their predictive efficacies were compared.

RESULTS

Tumor stage, tumor burden, body mass index, alpha-fetoprotein, and vascular invasion were revealed as independent risk factors for survival. The model trained by Random Forest algorithms based on tumor burden, whole-liver radiomics signature, and clinical features had the highest predictive efficacy, with c-index values of 0.85 and 0.80 and areas under the ROC curve of 0.96 and 0.83 in the developing cohort and validation cohort, respectively. In the high-rad-score group (TLrad-score > - 0.34), the median overall survival (mOS) was significantly shorter than in the low-rad-score group (17 m vs. 37 m, p < 0.001). A shorter mOS was observed in patients with high tumor burden compared to those with low tumor burden (14 m vs. 29 m, p = 0.007).

CONCLUSION

The combined radiomics model from whole-liver signatures may effectively predict survival for HCC patients continuing TACE after TACE refractoriness. The TLrad-score and tumor burden are potential prognostic markers for TACE therapy following TACE-resistance.

Artificial intelligence-based tools with automated segmentation and measurement on CT images to assist accurate and fast diagnosis in acute pancreatitis

OBJECTIVES

To develop an artificial intelligence (AI) tool with automated pancreas segmentation and measurement of pancreatic morphological information on CT images to assist improved and faster diagnosis in acute pancreatitis.

METHODS

This study retrospectively contained 1124 patients suspected for AP and received non-contrast and enhanced abdominal CT examination between September 2013 and September 2022. Patients were divided into training (N = 688), validation (N = 145), testing dataset [N = 291; N = 104 for normal pancreas, N = 98 for AP, N = 89 for AP complicated with PDAC (AP&PDAC)]. A model based on convolutional neural network (MSAnet) was developed. The pancreas segmentation and measurement were performed via eight open-source models and MSAnet based tools, and the efficacy was evaluated using dice similarity coefficient (DSC) and intersection over union (IoU). The DSC and IoU for patients with different ages were also compared. The outline of tumour and oedema in the AP and were segmented by clustering. The diagnostic efficacy for radiologists with or without the assistance of MSAnet tool in AP and AP&PDAC was evaluated using receiver operation curve and confusion matrix.

RESULTS

Among all models, MSAnet based tool showed best performance on the training and validation dataset, and had high efficacy on testing dataset. The performance was age-affected. With assistance of the AI tool, the diagnosis time was significantly shortened by 26.8% and 32.7% for junior and senior radiologists, respectively. The area under curve (AUC) in diagnosis of AP was improved from 0.91 to 0.96 for junior radiologist and 0.98 to 0.99 for senior radiologist. In AP&PDAC diagnosis, AUC was increased from 0.85 to 0.92 for junior and 0.97 to 0.99 for senior.

CONCLUSION

MSAnet based tools showed good pancreas segmentation and measurement performance, which help radiologists improve diagnosis efficacy and workflow in both AP and AP with PDAC conditions.

ADVANCES IN KNOWLEDGE

This study developed an AI tool with automated pancreas segmentation and measurement and provided evidence for AI tool assistance in improving the workflow and accuracy of AP diagnosis.

Using Vision Transformer for high robustness and generalization in predicting EGFR mutation status in lung adenocarcinoma

BACKGROUND

Lung adenocarcinoma is a common cause of cancer-related deaths worldwide, and accurate EGFR genotyping is crucial for optimal treatment outcomes. Conventional methods for identifying the EGFR genotype have several limitations. Therefore, we proposed a deep learning model using non-invasive CT images to predict EGFR mutation status with robustness and generalizability.

METHODS

A total of 525 patients were enrolled at the local hospital to serve as the internal data set for model training and validation. In addition, a cohort of 30 patients from the publicly available Cancer Imaging Archive Data Set was selected for external testing. All patients underwent plain chest CT, and their EGFR mutation status labels were categorized as either mutant or wild type. The CT images were analyzed using a self-attention-based ViT-B/16 model to predict the EGFR mutation status, and the model's performance was evaluated. To produce an attention map indicating the suspicious locations of EGFR mutations, Grad-CAM was utilized.

RESULTS

The ViT deep learning model achieved impressive results, with an accuracy of 0.848, an AUC of 0.868, a sensitivity of 0.924, and a specificity of 0.718 on the validation cohort. Furthermore, in the external test cohort, the model achieved comparable performances, with an accuracy of 0.833, an AUC of 0.885, a sensitivity of 0.900, and a specificity of 0.800.

CONCLUSIONS

The ViT model demonstrates a high level of accuracy in predicting the EGFR mutation status of lung adenocarcinoma patients. Moreover, with the aid of attention maps, the model can assist clinicians in making informed clinical decisions.

A landmark-supervised registration framework for multi-phase CT images with cross-distillation

Objective.Multi-phase computed tomography (CT) has become a leading modality for identifying hepatic tumors. Nevertheless, the presence of misalignment in the images of different phases poses a challenge in accurately identifying and analyzing the patient's anatomy. Conventional registration methods typically concentrate on either intensity-based features or landmark-based features in isolation, so imposing limitations on the accuracy of the registration process.Method.We establish a nonrigid cycle-registration network that leverages semi-supervised learning techniques, wherein a point distance term based on Euclidean distance between registered landmark points is introduced into the loss function. Additionally, a cross-distillation strategy is proposed in network training to further improve registration performance which incorporates response-based knowledge concerning the distances between feature points.Results.We conducted experiments using multi-centered liver CT datasets to evaluate the performance of the proposed method. The results demonstrate that our method outperforms baseline methods in terms of target registration error. Additionally, Dice scores of the warped tumor masks were calculated. Our method consistently achieved the highest scores among all the comparing methods. Specifically, it achieved scores of 82.9% and 82.5% in the hepatocellular carcinoma and the intrahepatic cholangiocarcinoma dataset, respectively.Significance.The superior registration performance indicates its potential to serve as an important tool in hepatic tumor identification and analysis.

Deep learning system for malignancy risk prediction in cystic renal lesions: a multicenter study

OBJECTIVES

To develop an interactive, non-invasive artificial intelligence (AI) system for malignancy risk prediction in cystic renal lesions (CRLs).

METHODS

In this retrospective, multicenter diagnostic study, we evaluated 715 patients. An interactive geodesic-based 3D segmentation model was created for CRLs segmentation. A CRLs classification model was developed using spatial encoder temporal decoder (SETD) architecture. The classification model combines a 3D-ResNet50 network for extracting spatial features and a gated recurrent unit (GRU) network for decoding temporal features from multi-phase CT images. We assessed the segmentation model using sensitivity (SEN), specificity (SPE), intersection over union (IOU), and dice similarity (Dice) metrics. The classification model's performance was evaluated using the area under the receiver operator characteristic curve (AUC), accuracy score (ACC), and decision curve analysis (DCA).

RESULTS

From 2012 to 2023, we included 477 CRLs (median age, 57 [IQR: 48-65]; 173 men) in the training cohort, 226 CRLs (median age, 60 [IQR: 52-69]; 77 men) in the validation cohort, and 239 CRLs (median age, 59 [IQR: 53-69]; 95 men) in the testing cohort (external validation cohort 1, cohort 2, and cohort 3). The segmentation model and SETD classifier exhibited excellent performance in both validation (AUC = 0.973, ACC = 0.916, Dice = 0.847, IOU = 0.743, SEN = 0.840, SPE = 1.000) and testing datasets (AUC = 0.998, ACC = 0.988, Dice = 0.861, IOU = 0.762, SEN = 0.876, SPE = 1.000).

CONCLUSION

The AI system demonstrated excellent benign-malignant discriminatory ability across both validation and testing datasets and illustrated improved clinical decision-making utility.

CRITICAL RELEVANCE STATEMENT

In this era when incidental CRLs are prevalent, this interactive, non-invasive AI system will facilitate accurate diagnosis of CRLs, reducing excessive follow-up and overtreatment.

KEY POINTS

The rising prevalence of CRLs necessitates better malignancy prediction strategies. The AI system demonstrated excellent diagnostic performance in identifying malignant CRL. The AI system illustrated improved clinical decision-making utility.

Artificial intelligence in liver imaging: methods and applications

Liver disease is regarded as one of the major health threats to humans. Radiographic assessments hold promise in terms of addressing the current demands for precisely diagnosing and treating liver diseases, and artificial intelligence (AI), which excels at automatically making quantitative assessments of complex medical image characteristics, has made great strides regarding the qualitative interpretation of medical imaging by clinicians. Here, we review the current state of medical-imaging-based AI methodologies and their applications concerning the management of liver diseases. We summarize the representative AI methodologies in liver imaging with focusing on deep learning, and illustrate their promising clinical applications across the spectrum of precise liver disease detection, diagnosis and treatment. We also address the current challenges and future perspectives of AI in liver imaging, with an emphasis on feature interpretability, multimodal data integration and multicenter study. Taken together, it is revealed that AI methodologies, together with the large volume of available medical image data, might impact the future of liver disease care.

Artificial Intelligence in Liver Diseases: Recent Advances

Liver diseases cause a significant burden on public health worldwide. In spite of great advances during recent years, there are still many challenges in the diagnosis and treatment of liver diseases. During recent years, artificial intelligence (AI) has been widely used for the diagnosis, risk stratification, and prognostic prediction of various diseases based on clinical datasets and medical images. Accumulative studies have shown its performance for diagnosing patients with nonalcoholic fatty liver disease and liver fibrosis and assessing their severity, and for predicting treatment response and recurrence of hepatocellular carcinoma, outcomes of liver transplantation recipients, and risk of drug-induced liver injury. Herein, we aim to comprehensively summarize the current evidence regarding diagnostic, prognostic, and/or therapeutic role of AI in these common liver diseases.

A New Automated Prognostic Prediction Method Based on Multi-Sequence Magnetic Resonance Imaging for Hepatic Resection of Colorectal Cancer Liver Metastases

Colorectal cancer is a prevalent and life-threatening disease, where colorectal cancer liver metastasis (CRLM) exhibits the highest mortality rate. Currently, surgery stands as the most effective curative option for eligible patients. However, due to the insufficient performance of traditional methods and the lack of multi-modality MRI feature complementarity in existing deep learning methods, the prognosis of CRLM surgical resection has not been fully explored. This paper proposes a new method, multi-modal guided complementary network (MGCNet), which employs multi-sequence MRI to predict 1-year recurrence and recurrence-free survival in patients after CRLM resection. In light of the complexity and redundancy of features in the liver region, we designed the multi-modal guided local feature fusion module to utilize the tumor features to guide the dynamic fusion of prognostically relevant local features within the liver. On the other hand, to solve the loss of spatial information during multi-sequence MRI fusion, the cross-modal complementary external attention module designed an external mask branch to establish inter-layer correlation. The results show that the model has accuracy (ACC) of 0.79, the area under the curve (AUC) of 0.84, C-Index of 0.73, and hazard ratio (HR) of 4.0, which is a significant improvement over state-of-the-art methods. Additionally, MGCNet exhibits good interpretability.

MRI-Based Radiomics and Deep Learning in Biological Characteristics and Prognosis of Hepatocellular Carcinoma: Opportunities and Challenges

Hepatocellular carcinoma (HCC) is the fifth most common malignancy and the third leading cause of cancer-related death worldwide. HCC exhibits strong inter-tumor heterogeneity, with different biological characteristics closely associated with prognosis. In addition, patients with HCC often distribute at different stages and require diverse treatment options at each stage. Due to the variability in tumor sensitivity to different therapies, determining the optimal treatment approach can be challenging for clinicians prior to treatment. Artificial intelligence (AI) technology, including radiomics and deep learning approaches, has emerged as a unique opportunity to improve the spectrum of HCC clinical care by predicting biological characteristics and prognosis in the medical imaging field. The radiomics approach utilizes handcrafted features derived from specific mathematical formulas to construct various machine-learning models for medical applications. In terms of the deep learning approach, convolutional neural network models are developed to achieve high classification performance based on automatic feature extraction from images. Magnetic resonance imaging offers the advantage of superior tissue resolution and functional information. This comprehensive evaluation plays a vital role in the accurate assessment and effective treatment planning for HCC patients. Recent studies have applied radiomics and deep learning approaches to develop AI-enabled models to improve accuracy in predicting biological characteristics and prognosis, such as microvascular invasion and tumor recurrence. Although AI-enabled models have demonstrated promising potential in HCC with biological characteristics and prognosis prediction with high performance, one of the biggest challenges, interpretability, has hindered their implementation in clinical practice. In the future, continued research is needed to improve the interpretability of AI-enabled models, including aspects such as domain knowledge, novel algorithms, and multi-dimension data sources. Overcoming these challenges would allow AI-enabled models to significantly impact the care provided to HCC patients, ultimately leading to their deployment for clinical use. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 2.

Automatic origin prediction of liver metastases via hierarchical artificial-intelligence system trained on multiphasic CT data: a retrospective, multicentre study

Background

Currently, the diagnostic testing for the primary origin of liver metastases (LMs) can be laborious, complicating clinical decision-making. Directly classifying the primary origin of LMs at computed tomography (CT) images has proven to be challenging, despite its potential to streamline the entire diagnostic workflow.

Methods

We developed ALMSS, an artificial intelligence (AI)-based LMs screening system, to provide automated liver contrast-enhanced CT analysis for distinguishing LMs from hepatocellular carcinoma (HCC) and intrahepatic cholangiocarcinoma (ICC), as well as subtyping primary origin of LMs as six organ systems. We processed a CECT dataset between January 1, 2013 and June 30, 2022 (n = 3105: 840 HCC, 354 ICC, and 1911 LMs) for training and internally testing ALMSS, and two additional cohorts (n = 622) for external validation of its diagnostic performance. The performance of radiologists with and without the assistance of ALMSS in diagnosing and subtyping LMs was assessed.

Findings

ALMSS achieved average area under the curve (AUC) of 0.917 (95% confidence interval [CI]: 0.899-0.931) and 0.923 (95% [CI]: 0.905-0.937) for differentiating LMs, HCC and ICC on both the internal testing set and external testing set, outperformed that of two radiologists. Moreover, ALMSS yielded average AUC of 0.815 (95% [CI]: 0.794-0.836) and 0.818 (95% [CI]: 0.790-0.842) for predicting six primary origins on both two testing sets. Interestingly, ALMSS assigned origin diagnoses for LMs with pathological phenotypes beyond the training categories with average AUC of 0.761 (95% [CI]: 0.657-0.842), which verify the model's diagnostic expandability.

Interpretation

Our study established an AI-based diagnostic system that effectively identifies and characterizes LMs directly from multiphasic CT images.

Funding

National Natural Science Foundation of China, Guangdong Provincial Key Laboratory of Medical Image Processing.

Imaging features and deep learning for prediction of pulmonary epithelioid hemangioendothelioma in CT images

BACKGROUND

Pulmonary epithelioid hemangioendothelioma (PEH) is a rare vascular tumour, and its early diagnosis remains challenging. This study aims to comprehensively analyse the imaging features of PEH and develop a model for predicting PEH.

METHODS

Retrospective and pooled analyses of imaging findings were performed in PEH patients at our center (n=25) and in published cases (n=71), respectively. Relevant computed tomography (CT) images were extracted and used to build a deep learning model for PEH identification and differentiation from other diseases.

RESULTS

In this study, bilateral multiple nodules/masses (n=19) appeared to be more common with most nodules less than 2 cm. In addition to the common types and features, the pattern of mixed type (n=4) and isolated nodules (n=4), punctate calcifications (5/25) and lymph node enlargement were also observed (10/25). The presence of pleural effusion is associated with a poor prognosis in PEH. The deep learning model, with an area under the receiver operating characteristic curve (AUC) of 0.71 [95% confidence interval (CI): 0.69-0.72], has a differentiation accuracy of 100% and 74% for the training and test sets respectively.

CONCLUSIONS

This study confirmed the heterogeneity of the imaging findings in PEH and showed several previously undescribed types and features. The current deep learning model based on CT has potential for clinical application and needs to be further explored in the future.

MTANet: Multi-Task Attention Network for Automatic Medical Image Segmentation and Classification

Medical image segmentation and classification are two of the most key steps in computer-aided clinical diagnosis. The region of interest were usually segmented in a proper manner to extract useful features for further disease classification. However, these methods are computationally complex and time-consuming. In this paper, we proposed a one-stage multi-task attention network (MTANet) which efficiently classifies objects in an image while generating a high-quality segmentation mask for each medical object. A reverse addition attention module was designed in the segmentation task to fusion areas in global map and boundary cues in high-resolution features, and an attention bottleneck module was used in the classification task for image feature and clinical feature fusion. We evaluated the performance of MTANet with CNN-based and transformer-based architectures across three imaging modalities for different tasks: CVC-ClinicDB dataset for polyp segmentation, ISIC-2018 dataset for skin lesion segmentation, and our private ultrasound dataset for liver tumor segmentation and classification. Our proposed model outperformed state-of-the-art models on all three datasets and was superior to all 25 radiologists for liver tumor diagnosis.

Non-invasively Discriminating the Pathological Subtypes of Non-small Cell Lung Cancer with Pretreatment 18F-FDG PET/CT Using Deep Learning

RATIONALE AND OBJECTIVES

To develop an end-to-end deep learning (DL) model for non-invasively predicting non-small cell lung cancer (NSCLC) pathological subtypes based on 18F-fluorodeoxyglucose (FDG) positron emission tomography/computed tomography (PET/CT) images, and to explore the potential value of DL technology.

MATERIALS AND METHODS

Preoperative 18F-FDG PET/CT images of 189 patients with NSCLC were retrospectively collected. The whole cohort was randomly divided into a training cohort, a validation cohort, and an internal/extended test cohort at the ratio of 6:2:2 after preprocessing the images. In the training and validation cohorts, seven DL models-Shufflenet, VGG16, Googlenet, Inception v3, Resnet50, Densenet201, and Mobilenet v2-were trained and optimized. The generalization ability and clinical utility of the optimal model were evaluated in the internal and extended test cohorts. Moreover, Spearman's correlation analysis was used to evaluate the correlation between DL features and traditional radiological features such as tumor size and maximum standardized uptake values (SUVmax).

RESULTS

Some DL features were significantly correlated with SUVmax and tumor size (P < 0.05). The Mobilenet v2 model achieved the best performance during the model development and validation phases. In the internal test group (area under the receiver operating characteristic curve [AUC]: 0.744, area under the precision-recall curve [AP]: 0.759) and extended test group (AUC: 0.767, AP: 0.768), the Mobilenet v2 model showed good generalization ability and reproducibility. Meanwhile, the decision curve analysis revealed that patients can benefit from the decisions made based on the Mobilenet v2 model.

CONCLUSION

DL models offer great potential for classifying NSCLC pathological subtypes. Specifically, the Mobilenet v2 model performs well at end-to-end non-invasive pathological subtype stratification of NSCLC.

Deep learning nomogram for preoperative distinction between Xanthogranulomatous cholecystitis and gallbladder carcinoma: A novel approach for surgical decision

The distinction between Xanthogranulomatous Cholecystitis (XGC) and Gallbladder Carcinoma (GBC) is challenging due to their similar imaging features. This study aimed to differentiate between XGC and GBC using a deep learning nomogram model built from contrast enhanced computed tomography (CT) scans. 297 patients were included with confirmed XGC (94) and GBC (203) as the training and internal validation cohort from 2017 to 2021. The deep learning model Resnet-18 with Fourier transformation named FCovResnet18, shows most impressive potential in distinguishing XGC from GBC using 3-phase merged images. The accuracy, precision and area under the curve (AUC) of the model were then calculated. An additional cohort of 74 patients consisting of 22 XGC and 52 GBC patients was enrolled from two subsidiary hospitals as the external validation cohort. The accuracy, precision and AUC achieve 0.98, 0.99, 1.00 in the internal validation cohort and 0.89, 0.92, 0.92 in external validation cohort. A nomogram model combining clinical characteristics and deep learning prediction score showed improved predicting value. Altogether, FCovResnet18 nomogram has demonstrated its ability to effectively differentiate XGC from GBC preoperatively, which significantly aid surgeons in making informed and accurate surgical decisions for XGC and GBC patients.

Deep Learning Methods in Medical Image-Based Hepatocellular Carcinoma Diagnosis: A Systematic Review and Meta-Analysis

(1) Background: The aim of our research was to systematically review papers specifically focused on the hepatocellular carcinoma (HCC) diagnostic performance of DL methods based on medical images. (2) Materials: To identify related studies, a comprehensive search was conducted in prominent databases, including Embase, IEEE, PubMed, Web of Science, and the Cochrane Library. The search was limited to studies published before 3 July 2023. The inclusion criteria consisted of studies that either developed or utilized DL methods to diagnose HCC using medical images. To extract data, binary information on diagnostic accuracy was collected to determine the outcomes of interest, namely, the sensitivity, specificity, and area under the curve (AUC). (3) Results: Among the forty-eight initially identified eligible studies, thirty studies were included in the meta-analysis. The pooled sensitivity was 89% (95% CI: 87-91), the specificity was 90% (95% CI: 87-92), and the AUC was 0.95 (95% CI: 0.93-0.97). Analyses of subgroups based on medical image methods (contrast-enhanced and non-contrast-enhanced images), imaging modalities (ultrasound, magnetic resonance imaging, and computed tomography), and comparisons between DL methods and clinicians consistently showed the acceptable diagnostic performance of DL models. The publication bias and high heterogeneity observed between studies and subgroups can potentially result in an overestimation of the diagnostic accuracy of DL methods in medical imaging. (4) Conclusions: To improve future studies, it would be advantageous to establish more rigorous reporting standards that specifically address the challenges associated with DL research in this particular field.

A multi-center performance assessment for automated histopathological classification and grading of glioma using whole slide images

Accurate pathological classification and grading of gliomas is crucial in clinical diagnosis and treatment. The application of deep learning techniques holds promise for automated histological pathology diagnosis. In this study, we collected 733 whole slide images from four medical centers, of which 456 were used for model training, 150 for internal validation, and 127 for multi-center testing. The study includes 5 types of common gliomas. A subtask-guided multi-instance learning image-to-label training pipeline was employed. The pipeline leveraged "patch prompting" for the model to converge with reasonable computational cost. Experiments showed that an overall accuracy of 0.79 in the internal validation dataset. The performance on the multi-center testing dataset showed an overall accuracy to 0.73. The findings suggest a minor yet acceptable performance decrease in multi-center data, demonstrating the model's strong generalizability and establishing a robust foundation for future clinical applications.

Differentiation of Hepatocellular Carcinoma from Intrahepatic Cholangiocarcinoma through MRI Radiomics

The purpose of this study was to investigate the efficacy of magnetic resonance imaging (MRI) radiomics in differentiating hepatocellular carcinoma (HCC) from intrahepatic cholangiocarcinoma (ICC). The clinical and MRI data of 129 pathologically confirmed HCC patients and 48 ICC patients treated at the Affiliated Hospital of North Sichuan Medical College between April 2016 and December 2021 were retrospectively analyzed. The patients were randomly divided at a ratio of 7:3 into a training group of 124 patients (90 with HCC and 34 with ICC) and a validation group of 53 patients (39 with HCC and 14 with ICC). Radiomic features were extracted from axial fat suppression T2-weighted imaging (FS-T2WI) and axial arterial-phase (AP) and portal-venous-phase (PVP) dynamic-contrast-enhanced MRI (DCE-MRI) sequences, and the corresponding datasets were generated. The least absolute shrinkage and selection operator (LASSO) method was used to select the best radiomic features. Logistic regression was used to establish radiomic models for each sequence (FS-T2WI, AP and PVP models), a clinical model for optimal clinical variables (C model) and a joint radiomics model (JR model) integrating the radiomics features of all the sequences as well as a radiomics-clinical model combining optimal radiomic features and clinical risk factors (RC model). The performance of each model was evaluated using the area under the receiver operating characteristic curve (AUC). The AUCs of the FS-T2WI, AP, PVP, JR, C and RC models for distinguishing HCC from ICC were 0.693, 0.863, 0.818, 0.914, 0.936 and 0.977 in the training group and 0.690, 0.784, 0.727, 0.802, 0.860 and 0.877 in the validation group, respectively. The results of this study suggest that MRI-based radiomics may help noninvasively differentiate HCC from ICC. The model integrating the radiomics features and clinical risk factors showed a further improvement in performance.

Multimodal deep learning for liver cancer applications: a scoping review

Background

Hepatocellular carcinoma is a malignant neoplasm of the liver and a leading cause of cancer-related deaths worldwide. The multimodal data combines several modalities, such as medical images, clinical parameters, and electronic health record (EHR) reports, from diverse sources to accomplish the diagnosis of liver cancer. The introduction of deep learning models with multimodal data can enhance the diagnosis and improve physicians' decision-making for cancer patients.

Objective

This scoping review explores the use of multimodal deep learning techniques (i.e., combining medical images and EHR data) in diagnosing and prognosis of hepatocellular carcinoma (HCC) and cholangiocarcinoma (CCA).

Methodology

A comprehensive literature search was conducted in six databases along with forward and backward references list checking of the included studies. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) extension for scoping review guidelines were followed for the study selection process. The data was extracted and synthesized from the included studies through thematic analysis.

Results

Ten studies were included in this review. These studies utilized multimodal deep learning to predict and diagnose hepatocellular carcinoma (HCC), but no studies examined cholangiocarcinoma (CCA). Four imaging modalities (CT, MRI, WSI, and DSA) and 51 unique EHR records (clinical parameters and biomarkers) were used in these studies. The most frequently used medical imaging modalities were CT scans followed by MRI, whereas the most common EHR parameters used were age, gender, alpha-fetoprotein AFP, albumin, coagulation factors, and bilirubin. Ten unique deep-learning techniques were applied to both EHR modalities and imaging modalities for two main purposes, prediction and diagnosis.

Conclusion

The use of multimodal data and deep learning techniques can help in the diagnosis and prediction of HCC. However, there is a limited number of works and available datasets for liver cancer, thus limiting the overall advancements of AI for liver cancer applications. Hence, more research should be undertaken to explore further the potential of multimodal deep learning in liver cancer applications.