Convolutional neural network for automated segmentation of the liver and its vessels on non-contrast T1 vibe Dixon acquisitions

Development and Validation of a Modality-Invariant 3D Swin U-Net Transformer for Liver and Spleen Segmentation on Multi-Site Clinical Bi-parametric MR Images

To develop and validate a modality-invariant Swin U-Net Transformer (UNETR) deep learning model for liver and spleen segmentation on abdominal T1-weighted (T1w) or T2-weighted (T2w) MR images from multiple institutions for pediatric and adult patients with known or suspected chronic liver diseases. In this IRB-approved retrospective study, clinical abdominal axial T1w and T2w MR images from pediatric and adult patients were retrieved from four study sites, including Cincinnati Children's Hospital Medical Center (CCHMC), New York University (NYU), University of Wisconsin (UW) and University of Michigan / Michigan Medicine (UM). The whole liver and spleen were manually delineated as the ground truth masks. We developed a modality-invariant 3D Swin UNETR using a modality-invariant training strategy, in which each patient's T1w and T2w MR images were treated as separate training samples. We conducted both internal and external validation experiments. A total of 241 T1w and 339 T2w MR sequences from 304 patients (age [mean ± standard deviation], 31.8 ± 20.3 years; 132 [43%] female) were included for model development. The Swin UNETR achieved a Dice similarity coefficient (DSC) of 0.95 ± 0.02 on T1w images and 0.93 ± 0.05 on T2w images for liver segmentation. This is significantly better than the U-Net model (0.90 ± 0.05, p < 0.001 and 0.90 ± 0.13, p < 0.001, respectively). The Swin UNETR achieved a DSC of 0.88 ± 0.12 on T1w images and 0.93 ± 0.10 on T2w images for spleen segmentation, and it significantly outperformed a modality-invariant U-Net model (0.80 ± 0.18, p = 0.001 and 0.88 ± 0.12, p = 0.002, respectively). Our study demonstrated that a modality-invariant Swin UNETR model can segment the liver and spleen on routinely collected clinical bi-parametric abdominal MR images from children and adult patients.

Artificial intelligence techniques in liver cancer

Hepatocellular Carcinoma (HCC), the most common primary liver cancer, is a significant contributor to worldwide cancer-related deaths. Various medical imaging techniques, including computed tomography, magnetic resonance imaging, and ultrasound, play a crucial role in accurately evaluating HCC and formulating effective treatment plans. Artificial Intelligence (AI) technologies have demonstrated potential in supporting physicians by providing more accurate and consistent medical diagnoses. Recent advancements have led to the development of AI-based multi-modal prediction systems. These systems integrate medical imaging with other modalities, such as electronic health record reports and clinical parameters, to enhance the accuracy of predicting biological characteristics and prognosis, including those associated with HCC. These multi-modal prediction systems pave the way for predicting the response to transarterial chemoembolization and microvascular invasion treatments and can assist clinicians in identifying the optimal patients with HCC who could benefit from interventional therapy. This paper provides an overview of the latest AI-based medical imaging models developed for diagnosing and predicting HCC. It also explores the challenges and potential future directions related to the clinical application of AI techniques.

Vascular liver segmentation: a narrative review on methods and new insights brought by artificial intelligence

Liver vessel segmentation from routinely performed medical imaging is a useful tool for diagnosis, treatment planning and delivery, and prognosis evaluation for many diseases, particularly liver cancer. A precise representation of liver anatomy is crucial to define the extent of the disease and, when suitable, the consequent resective or ablative procedure, in order to guarantee a radical treatment without sacrificing an excessive volume of healthy liver. Once mainly performed manually, with notable cost in terms of time and human energies, vessel segmentation is currently realized through the application of artificial intelligence (AI), which has gained increased interest and development of the field. Many different AI-driven models adopted for this aim have been described and can be grouped into different categories: thresholding methods, edge- and region-based methods, model-based methods, and machine learning models. The latter includes neural network and deep learning models that now represent the principal algorithms exploited for vessel segmentation. The present narrative review describes how liver vessel segmentation can be realized through AI models, with a summary of model results in terms of accuracy, and an overview on the future progress of this topic.

Fully automated assessment of the future liver remnant in a blood-free setting via CT before major hepatectomy via deep learning

OBJECTIVES

To develop and validate a deep learning (DL) model for automated segmentation of hepatic and portal veins, and apply the model in blood-free future liver remnant (FLR) assessments via CT before major hepatectomy.

METHODS

3-dimensional 3D U-Net models were developed for the automatic segmentation of hepatic veins and portal veins on contrast-enhanced CT images. A total of 170 patients treated from January 2018 to March 2019 were included. 3D U-Net models were trained and tested under various liver conditions. The Dice similarity coefficient (DSC) and volumetric similarity (VS) were used to evaluate the segmentation accuracy. The use of quantitative volumetry for evaluating resection was compared between blood-filled and blood-free settings and between manual and automated segmentation.

RESULTS

The DSC values in the test dataset for hepatic veins and portal veins were 0.66 ± 0.08 (95% CI: (0.65, 0.68)) and 0.67 ± 0.07 (95% CI: (0.66, 0.69)), the VS values were 0.80 ± 0.10 (95% CI: (0.79, 0.84)) and 0.74 ± 0.08 (95% CI: (0.73, 0.76)), respectively No significant differences in FLR, FLR% assessments, or the percentage of major hepatectomy patients were noted between the blood-filled and blood-free settings (p = 0.67, 0.59 and 0.99 for manual methods, p = 0.66, 0.99 and 0.99 for automated methods, respectively) according to the use of manual and automated segmentation methods.

CONCLUSION

Fully automated segmentation of hepatic veins and portal veins and FLR assessment via blood-free CT before major hepatectomy are accurate and applicable in clinical cases involving the use of DL.

CRITICAL RELEVANCE STATEMENT

Our fully automatic models could segment hepatic veins, portal veins, and future liver remnant in blood-free setting on CT images before major hepatectomy with reliable outcomes.

KEY POINTS

Fully automatic segmentation of hepatic veins and portal veins was feasible in clinical practice. Fully automatic volumetry of future liver remnant (FLR)% in a blood-free setting was robust. No significant differences in FLR% assessments were noted between the blood-filled and blood-free settings.

Signal Intensity Trajectories Clustering for Liver Vasculature Segmentation and Labeling (LiVaS) on Contrast-Enhanced MR Images: A Feasibility Pilot Study

This study aims to develop a semiautomated pipeline and user interface (LiVaS) for rapid segmentation and labeling of MRI liver vasculature and evaluate its time efficiency and accuracy against manual reference standard. Retrospective feasibility pilot study. Liver MR images from different scanners from 36 patients were included, and 4/36 patients were randomly selected for manual segmentation as referenced standard. The liver was segmented in each contrast phase and masks registered to the pre-contrast segmentation. Voxel-wise signal trajectories were clustered using the k-means algorithm. Voxel clusters that best segment the liver vessels were selected and labeled by three independent radiologists and a research scientist using LiVaS. Segmentation times were compared using a paired-sample t-test on log-transformed data. The agreement was analyzed qualitatively and quantitatively using DSC for hepatic and portal vein segmentations. The mean segmentation time among four readers was significantly shorter than manual (3.6 ± 1.4 vs. 70.0 ± 29.2 min; p < 0.001), even when using a higher number of clusters to enhance accuracy. The DSC for portal and hepatic veins reached up to 0.69 and 0.70, respectively. LiVaS segmentations were overall of good quality, with variations in performance related to the presence/severity of liver disease, acquisition timing, and image quality. Our semi-automated pipeline was robust to different MRI vendors in producing segmentation and labeling of liver vasculature in agreement with expert manual annotations, with significantly higher time efficiency. LiVaS could facilitate the creation of large, annotated datasets for training and validation of neural networks for automated MRI liver vascularity segmentation. HIGHLIGHTS: Key Finding: In this pilot feasibility study, our semiautomated pipeline for segmentation of liver vascularity (LiVaS) on MR images produced segmentations with simultaneous labeling of portal and hepatic veins in good agreement with the manual reference standard but at significantly shorter times (mean LiVaS 3.6 ± 1.4 vs. mean manual 70.0 ± 29.2 min; p < 0.001). Importance: LiVaS was robust in producing liver MRI vascular segmentations across images from different scanners in agreement with expert manual annotations, with significant ly higher time efficiency, and therefore potential scalability.

Progress and clinical translation in hepatocellular carcinoma of deep learning in hepatic vascular segmentation

This paper reviews the advancements in deep learning for hepatic vascular segmentation and its clinical implications in the holistic management of hepatocellular carcinoma (HCC). The key to the diagnosis and treatment of HCC lies in imaging examinations, with the challenge in liver surgery being the precise assessment of Hepatic vasculature. In this regard, deep learning methods, including convolutional neural networksamong various other approaches, have significantly improved accuracy and speed. The review synthesizes findings from 30 studies, covering aspects such as network architectures, applications, supervision techniques, evaluation metrics, and motivations. Furthermore, we also examine the challenges and future prospects of deep learning technologies in enhancing the comprehensive diagnosis and treatment of HCC, discussing anticipated breakthroughs that could transform patient management. By combining clinical needs with technological advancements, deep learning is expected to make greater breakthroughs in the field of hepatic vascular segmentation, thereby providing stronger support for the diagnosis and treatment of HCC.

Deep Learning Framework for Liver Segmentation from T1-Weighted MRI Images

The human liver exhibits variable characteristics and anatomical information, which is often ambiguous in radiological images. Machine learning can be of great assistance in automatically segmenting the liver in radiological images, which can be further processed for computer-aided diagnosis. Magnetic resonance imaging (MRI) is preferred by clinicians for liver pathology diagnosis over volumetric abdominal computerized tomography (CT) scans, due to their superior representation of soft tissues. The convenience of Hounsfield unit (HoU) based preprocessing in CT scans is not available in MRI, making automatic segmentation challenging for MR images. This study investigates multiple state-of-the-art segmentation networks for liver segmentation from volumetric MRI images. Here, T1-weighted (in-phase) scans are investigated using expert-labeled liver masks from a public dataset of 20 patients (647 MR slices) from the Combined Healthy Abdominal Organ Segmentation grant challenge (CHAOS). The reason for using T1-weighted images is that it demonstrates brighter fat content, thus providing enhanced images for the segmentation task. Twenty-four different state-of-the-art segmentation networks with varying depths of dense, residual, and inception encoder and decoder backbones were investigated for the task. A novel cascaded network is proposed to segment axial liver slices. The proposed framework outperforms existing approaches reported in the literature for the liver segmentation task (on the same test set) with a dice similarity coefficient (DSC) score and intersect over union (IoU) of 95.15% and 92.10%, respectively.

Automated 3D liver segmentation from hepatobiliary phase MRI for enhanced preoperative planning

Recent advancements in deep learning have facilitated significant progress in medical image analysis. However, there is lack of studies specifically addressing the needs of surgeons in terms of practicality and precision for surgical planning. Accurate understanding of anatomical structures, such as the liver and its intrahepatic structures, is crucial for preoperative planning from a surgeon's standpoint. This study proposes a deep learning model for automatic segmentation of liver parenchyma, vascular and biliary structures, and tumor mass in hepatobiliary phase liver MRI to improve preoperative planning and enhance patient outcomes. A total of 120 adult patients who underwent liver resection due to hepatic mass and had preoperative gadoxetic acid-enhanced MRI were included in the study. A 3D residual U-Net model was developed for automatic segmentation of liver parenchyma, tumor mass, hepatic vein (HV), portal vein (PV), and bile duct (BD). The model's performance was assessed using Dice similarity coefficient (DSC) by comparing the results with manually delineated structures. The model achieved high accuracy in segmenting liver parenchyma (DSC 0.92 ± 0.03), tumor mass (DSC 0.77 ± 0.21), hepatic vein (DSC 0.70 ± 0.05), portal vein (DSC 0.61 ± 0.03), and bile duct (DSC 0.58 ± 0.15). The study demonstrated the potential of the 3D residual U-Net model to provide a comprehensive understanding of liver anatomy and tumors for preoperative planning, potentially leading to improved surgical outcomes and increased patient safety.

Automated liver segmental volume ratio quantification on non-contrast T1-Vibe Dixon liver MRI using deep learning

PURPOSE

To evaluate the effectiveness of automated liver segmental volume quantification and calculation of the liver segmental volume ratio (LSVR) on a non-contrast T1-vibe Dixon liver MRI sequence using a deep learning segmentation pipeline.

METHOD

A dataset of 200 liver MRI with a non-contrast 3 mm T1-vibe Dixon sequence was manually labeledslice-by-sliceby an expert for Couinaud liver segments, while portal and hepatic veins were labeled separately. A convolutional neural networkwas trainedusing 170 liver MRI for training and 30 for evaluation. Liver segmental volumes without liver vessels were retrieved and LSVR was calculated as the liver segmental volumes I-III divided by the liver segmental volumes IV-VIII. LSVR was compared with the expert manual LSVR calculation and the LSVR calculated on CT scans in 30 patients with CT and MRI within 6 months.

RESULTS

Theconvolutional neural networkclassified the Couinaud segments I-VIII with an average Dice score of 0.770 ± 0.03, ranging between 0.726 ± 0.13 (segment IVb) and 0.810 ± 0.09 (segment V). The calculated mean LSVR with liver MRI unseen by the model was 0.32 ± 0.14, as compared with manually quantified LSVR of 0.33 ± 0.15, resulting in a mean absolute error (MAE) of 0.02. A comparable LSVR of 0.35 ± 0.14 with a MAE of 0.04 resulted with the LSRV retrieved from the CT scans. The automated LSVR showed significant correlation with the manual MRI LSVR (Spearman r = 0.97, p < 0.001) and CT LSVR (Spearman r = 0.95, p < 0.001).

CONCLUSIONS

A convolutional neural network allowed for accurate automated liver segmental volume quantification and calculation of LSVR based on a non-contrast T1-vibe Dixon sequence.

Large scale crowdsourced radiotherapy segmentations across a variety of cancer anatomic sites

Clinician generated segmentation of tumor and healthy tissue regions of interest (ROIs) on medical images is crucial for radiotherapy. However, interobserver segmentation variability has long been considered a significant detriment to the implementation of high-quality and consistent radiotherapy dose delivery. This has prompted the increasing development of automated segmentation approaches. However, extant segmentation datasets typically only provide segmentations generated by a limited number of annotators with varying, and often unspecified, levels of expertise. In this data descriptor, numerous clinician annotators manually generated segmentations for ROIs on computed tomography images across a variety of cancer sites (breast, sarcoma, head and neck, gynecologic, gastrointestinal; one patient per cancer site) for the Contouring Collaborative for Consensus in Radiation Oncology challenge. In total, over 200 annotators (experts and non-experts) contributed using a standardized annotation platform (ProKnow). Subsequently, we converted Digital Imaging and Communications in Medicine data into Neuroimaging Informatics Technology Initiative format with standardized nomenclature for ease of use. In addition, we generated consensus segmentations for experts and non-experts using the Simultaneous Truth and Performance Level Estimation method. These standardized, structured, and easily accessible data are a valuable resource for systematically studying variability in segmentation applications.