Detection of liver cirrhosis in standard T2-weighted MRI using deep transfer learning

Development of image-based decision support systems utilizing information extracted from radiological free-text report databases with text-based transformers

OBJECTIVES

To investigate the potential and limitations of utilizing transformer-based report annotation for on-site development of image-based diagnostic decision support systems (DDSS).

METHODS

The study included 88,353 chest X-rays from 19,581 intensive care unit (ICU) patients. To label the presence of six typical findings in 17,041 images, the corresponding free-text reports of the attending radiologists were assessed by medical research assistants ("gold labels"). Automatically generated "silver" labels were extracted for all reports by transformer models trained on gold labels. To investigate the benefit of such silver labels, the image-based models were trained using three approaches: with gold labels only (MG), with silver labels first, then with gold labels (MS/G), and with silver and gold labels together (MS+G). To investigate the influence of invested annotation effort, the experiments were repeated with different numbers (N) of gold-annotated reports for training the transformer and image-based models and tested on 2099 gold-annotated images. Significant differences in macro-averaged area under the receiver operating characteristic curve (AUC) were assessed by non-overlapping 95% confidence intervals.

RESULTS

Utilizing transformer-based silver labels showed significantly higher macro-averaged AUC than training solely with gold labels (N = 1000: MG 67.8 [66.0-69.6], MS/G 77.9 [76.2-79.6]; N = 14,580: MG 74.5 [72.8-76.2], MS/G 80.9 [79.4-82.4]). Training with silver and gold labels together was beneficial using only 500 gold labels (MS+G 76.4 [74.7-78.0], MS/G 75.3 [73.5-77.0]).

CONCLUSIONS

Transformer-based annotation has potential for unlocking free-text report databases for the development of image-based DDSS. However, on-site development of image-based DDSS could benefit from more sophisticated annotation pipelines including further information than a single radiological report.

CLINICAL RELEVANCE STATEMENT

Leveraging clinical databases for on-site development of artificial intelligence (AI)-based diagnostic decision support systems by text-based transformers could promote the application of AI in clinical practice by circumventing highly regulated data exchanges with third parties.

KEY POINTS

• The amount of data from a database that can be used to develop AI-assisted diagnostic decision systems is often limited by the need for time-consuming identification of pathologies by radiologists. • The transformer-based structuring of free-text radiological reports shows potential to unlock corresponding image databases for on-site development of image-based diagnostic decision support systems. • However, the quality of image annotations generated solely on the content of a single radiology report may be limited by potential inaccuracies and incompleteness of this report.

Artificial Intelligence in Liver Diseases: Recent Advances

Liver diseases cause a significant burden on public health worldwide. In spite of great advances during recent years, there are still many challenges in the diagnosis and treatment of liver diseases. During recent years, artificial intelligence (AI) has been widely used for the diagnosis, risk stratification, and prognostic prediction of various diseases based on clinical datasets and medical images. Accumulative studies have shown its performance for diagnosing patients with nonalcoholic fatty liver disease and liver fibrosis and assessing their severity, and for predicting treatment response and recurrence of hepatocellular carcinoma, outcomes of liver transplantation recipients, and risk of drug-induced liver injury. Herein, we aim to comprehensively summarize the current evidence regarding diagnostic, prognostic, and/or therapeutic role of AI in these common liver diseases.

The Use of Artificial Intelligence in the Liver Histopathology Field: A Systematic Review

Digital pathology (DP) has begun to play a key role in the evaluation of liver specimens. Recent studies have shown that a workflow that combines DP and artificial intelligence (AI) applied to histopathology has potential value in supporting the diagnosis, treatment evaluation, and prognosis prediction of liver diseases. Here, we provide a systematic review of the use of this workflow in the field of hepatology. Based on the PRISMA 2020 criteria, a search of the PubMed, SCOPUS, and Embase electronic databases was conducted, applying inclusion/exclusion filters. The articles were evaluated by two independent reviewers, who extracted the specifications and objectives of each study, the AI tools used, and the results obtained. From the 266 initial records identified, 25 eligible studies were selected, mainly conducted on human liver tissues. Most of the studies were performed using whole-slide imaging systems for imaging acquisition and applying different machine learning and deep learning methods for image pre-processing, segmentation, feature extractions, and classification. Of note, most of the studies selected demonstrated good performance as classifiers of liver histological images compared to pathologist annotations. Promising results to date bode well for the not-too-distant inclusion of these techniques in clinical practice.

Direct deep learning-based survival prediction from pre-interventional CT prior to transcatheter aortic valve replacement

PURPOSE

To investigate survival prediction in patients undergoing transcatheter aortic valve replacement (TAVR) using deep learning (DL) methods applied directly to pre-interventional CT images and to compare performance with survival models based on scalar markers of body composition.

METHOD

This retrospective single-center study included 760 patients undergoing TAVR (mean age 81 ± 6 years; 389 female). As a baseline, a Cox proportional hazards model (CPHM) was trained to predict survival on sex, age, and the CT body composition markers fatty muscle fraction (FMF), skeletal muscle radiodensity (SMRD), and skeletal muscle area (SMA) derived from paraspinal muscle segmentation of a single slice at L3/L4 level. The convolutional neural network (CNN) encoder of the DL model for survival prediction was pre-trained in an autoencoder setting with and without a focus on paraspinal muscles. Finally, a combination of DL and CPHM was evaluated. Performance was assessed by C-index and area under the receiver operating curve (AUC) for 1-year and 2-year survival. All methods were trained with five-fold cross-validation and were evaluated on 152 hold-out test cases.

RESULTS

The CNN for direct image-based survival prediction, pre-trained in a focussed autoencoder scenario, outperformed the baseline CPHM (CPHM: C-index = 0.608, 1Y-AUC = 0.606, 2Y-AUC = 0.594 vs. DL: C-index = 0.645, 1Y-AUC = 0.687, 2Y-AUC = 0.692). Combining DL and CPHM led to further improvement (C-index = 0.668, 1Y-AUC = 0.713, 2Y-AUC = 0.696).

CONCLUSIONS

Direct DL-based survival prediction shows potential to improve image feature extraction compared to segmentation-based scalar markers of body composition for risk assessment in TAVR patients.

Diagnosis of Liver Fibrosis Using Artificial Intelligence: A Systematic Review

Background and Objectives: The development of liver fibrosis as a consequence of continuous inflammation represents a turning point in the evolution of chronic liver diseases. The recent developments of artificial intelligence (AI) applications show a high potential for improving the accuracy of diagnosis, involving large sets of clinical data. For this reason, the aim of this systematic review is to provide a comprehensive overview of current AI applications and analyze the accuracy of these systems to perform an automated diagnosis of liver fibrosis. Materials and Methods: We searched PubMed, Cochrane Library, EMBASE, and WILEY databases using predefined keywords. Articles were screened for relevant publications about AI applications capable of diagnosing liver fibrosis. Exclusion criteria were animal studies, case reports, abstracts, letters to the editor, conference presentations, pediatric studies, studies written in languages other than English, and editorials. Results: Our search identified a total of 24 articles analyzing the automated imagistic diagnosis of liver fibrosis, out of which six studies analyze liver ultrasound images, seven studies analyze computer tomography images, five studies analyze magnetic resonance images, and six studies analyze liver biopsies. The studies included in our systematic review showed that AI-assisted non-invasive techniques performed as accurately as human experts in detecting and staging liver fibrosis. Nevertheless, the findings of these studies need to be confirmed through clinical trials to be implemented into clinical practice. Conclusions: The current systematic review provides a comprehensive analysis of the performance of AI systems in diagnosing liver fibrosis. Automatic diagnosis, staging, and risk stratification for liver fibrosis is currently possible considering the accuracy of the AI systems, which can overcome the limitations of non-invasive diagnosis methods.

Transformer-based structuring of free-text radiology report databases

OBJECTIVES

To provide insights for on-site development of transformer-based structuring of free-text report databases by investigating different labeling and pre-training strategies.

METHODS

A total of 93,368 German chest X-ray reports from 20,912 intensive care unit (ICU) patients were included. Two labeling strategies were investigated to tag six findings of the attending radiologist. First, a system based on human-defined rules was applied for annotation of all reports (termed "silver labels"). Second, 18,000 reports were manually annotated in 197 h (termed "gold labels") of which 10% were used for testing. An on-site pre-trained model (T_mlm) using masked-language modeling (MLM) was compared to a public, medically pre-trained model (T_med). Both models were fine-tuned on silver labels only, gold labels only, and first with silver and then gold labels (hybrid training) for text classification, using varying numbers (N: 500, 1000, 2000, 3500, 7000, 14,580) of gold labels. Macro-averaged F1-scores (MAF1) in percent were calculated with 95% confidence intervals (CI).

RESULTS

T_mlm,gold (95.5 [94.5-96.3]) showed significantly higher MAF1 than T_med,silver (75.0 [73.4-76.5]) and T_mlm,silver (75.2 [73.6-76.7]), but not significantly higher MAF1 than T_med,gold (94.7 [93.6-95.6]), T_med,hybrid (94.9 [93.9-95.8]), and T_mlm,hybrid (95.2 [94.3-96.0]). When using 7000 or less gold-labeled reports, T_mlm,gold (N: 7000, 94.7 [93.5-95.7]) showed significantly higher MAF1 than T_med,gold (N: 7000, 91.5 [90.0-92.8]). With at least 2000 gold-labeled reports, utilizing silver labels did not lead to significant improvement of T_mlm,hybrid (N: 2000, 91.8 [90.4-93.2]) over T_mlm,gold (N: 2000, 91.4 [89.9-92.8]).

CONCLUSIONS

Custom pre-training of transformers and fine-tuning on manual annotations promises to be an efficient strategy to unlock report databases for data-driven medicine.

KEY POINTS

• On-site development of natural language processing methods that retrospectively unlock free-text databases of radiology clinics for data-driven medicine is of great interest. • For clinics seeking to develop methods on-site for retrospective structuring of a report database of a certain department, it remains unclear which of previously proposed strategies for labeling reports and pre-training models is the most appropriate in context of, e.g., available annotator time. • Using a custom pre-trained transformer model, along with a little annotation effort, promises to be an efficient way to retrospectively structure radiological databases, even if not millions of reports are available for pre-training.

Deep Learning Staging of Liver Iron Content From Multiecho MR Images

BACKGROUND

MRI represents the most established liver iron content (LIC) evaluation approach by estimation of liver T2* value, but it is dependent on the choice of the measurement region and the software used for image analysis.

PURPOSE

To develop a deep-learning method for unsupervised classification of LIC from magnitude T2* multiecho MR images.

STUDY TYPE

Retrospective.

POPULATION/SUBJECTS

A total of 1069 thalassemia major patients enrolled in the core laboratory of the Myocardial Iron Overload in Thalassemia (MIOT) network, which were included in the training (80%) and test (20%) sets. Twenty patients from different MRI vendors included in the external test set.

FIELD STRENGTH/SEQUENCE

A5 T, T2* multiecho magnitude images.

ASSESSMENT

Four deep-learning convolutional neural networks (HippoNet-2D, HippoNet-3D, HippoNet-LSTM, and an ensemble network HippoNet-Ensemble) were used to achieve unsupervised staging of LIC using five classes (normal, borderline, middle, moderate, severe). The training set was employed to construct the deep-learning model. The performance of the LIC staging model was evaluated in the test set and in the external test set. The model's performances were assessed by evaluating the accuracy, sensitivity, and specificity with respect to the ground truth labels obtained by T2* measurements and by comparison with operator-induced variability originating from different region of interest (ROI) placements.

STATISTICAL TESTS

The network's performances were evaluated by single-class accuracy, specificity, and sensitivity and compared by one-way repeated measures analysis of variance (ANOVA) and one-way ANOVA.

RESULTS

HippoNet-Ensemble reached an accuracy significantly higher than the other networks, and a sensitivity and specificity higher than HippoNet-LSTM. Accuracy, sensitivity, and specificity values for the LIC stages were: normal: 0.96/0.93/0.97, borderline: 0.95/0.85/0.98, mild: 0.96/0.88/0.98, moderate: 0.95/0.89/0.97, severe: 0.97/0.95/0.98. Correctly staging of cases was in the range of 85%-95%, depending on the LIC class. Multiclass accuracy was 0.90 against 0.92 for the interobserver variability.

DATA CONCLUSION

The proposed HippoNet-Ensemble network can perform unsupervised LIC staging and achieves good prognostic performance.

EVIDENCE LEVEL

4 TECHNICAL EFFICACY: Stage 2.

Automatic detection of Gibbs artefact in MR images with transfer learning approach

BACKGROUND

Quality control of magnetic resonance imaging includes image validation, which covers also artefact detection. The daily manual review of magnetic resonance images for possible artefacts can be time-consuming, so automated methods for computer-assisted quality assessment of magnetic resonance imaging need to be developed.

OBJECTIVE

The aim of this study was to develop automatic detection of Gibbs artefacts in magnetic resonance imaging using a deep learning method called transfer learning, and to demonstrate the potential of this approach for the development of an automatic quality control tool for the detection of such artefacts in magnetic resonance imaging.

METHODS

The magnetic resonance image dataset of the scanned phantom for quality assurance was created using a turbo spin-echo pulse sequence in the transverse plane. Images were created to include Gibbs artefacts of varying intensities. The images were annotated by two independent reviewers. The annotated dataset was used to develop a method for Gibbs artefact detection using the transfer learning approach. The VGG-16, VGG-19, and ResNet-152 convolutional neural networks were used as pre-trained networks for transfer learning and compared using 5-fold cross-validation.

RESULTS

All accuracies of the classification models were above 97%, while the AUC values were all above 0.99, confirming the high quality of the constructed models.

CONCLUSION

We show that transfer learning can be successfully used to detect Gibbs artefacts on magnetic resonance images. The main advantages of transfer learning are that it can be applied on small training datasets, the procedures to build the models are not so complicated, and they do not require much computational power. This shows the potential of transfer learning for the more general task of detecting artefacts in magnetic resonance images of patients, which consequently can improve and speed up the process of quality assessment in medical imaging practice.

Deep learning enables automated MRI-based estimation of uterine volume also in patients with uterine fibroids undergoing high-intensity focused ultrasound therapy

BACKGROUND

High-intensity focused ultrasound (HIFU) is used for the treatment of symptomatic leiomyomas. We aim to automate uterine volumetry for tracking changes after therapy with a 3D deep learning approach.

METHODS

A 3D nnU-Net model in the default setting and in a modified version including convolutional block attention modules (CBAMs) was developed on 3D T2-weighted MRI scans. Uterine segmentation was performed in 44 patients with routine pelvic MRI (standard group) and 56 patients with uterine fibroids undergoing ultrasound-guided HIFU therapy (HIFU group). Here, preHIFU scans (n = 56), postHIFU imaging maximum one day after HIFU (n = 54), and the last available follow-up examination (n = 53, days after HIFU: 420 ± 377) were included. The training was performed on 80% of the data with fivefold cross-validation. The remaining data were used as a hold-out test set. Ground truth was generated by a board-certified radiologist and a radiology resident. For the assessment of inter-reader agreement, all preHIFU examinations were segmented independently by both.

RESULTS

High segmentation performance was already observed for the default 3D nnU-Net (mean Dice score = 0.95 ± 0.05) on the validation sets. Since the CBAM nnU-Net showed no significant benefit, the less complex default model was applied to the hold-out test set, which resulted in accurate uterus segmentation (Dice scores: standard group 0.92 ± 0.07; HIFU group 0.96 ± 0.02), which was comparable to the agreement between the two readers.

CONCLUSIONS

This study presents a method for automatic uterus segmentation which allows a fast and consistent assessment of uterine volume. Therefore, this method could be used in the clinical setting for objective assessment of therapeutic response to HIFU therapy.

Deep learning for standardized, MRI-based quantification of subcutaneous and subfascial tissue volume for patients with lipedema and lymphedema

OBJECTIVES

To contribute to a more in-depth assessment of shape, volume, and asymmetry of the lower extremities in patients with lipedema or lymphedema utilizing volume information from MR imaging.

METHODS

A deep learning (DL) pipeline was developed including (i) localization of anatomical landmarks (femoral heads, symphysis, knees, ankles) and (ii) quality-assured tissue segmentation to enable standardized quantification of subcutaneous (SCT) and subfascial tissue (SFT) volumes. The retrospectively derived dataset for method development consisted of 45 patients (42 female, 44.2 ± 14.8 years) who underwent clinical 3D DIXON MR-lymphangiography examinations of the lower extremities. Five-fold cross-validated training was performed on 16,573 axial slices from 40 patients and testing on 2187 axial slices from 5 patients. For landmark detection, two EfficientNet-B1 convolutional neural networks (CNNs) were applied in an ensemble. One determines the relative foot-head position of each axial slice with respect to the landmarks by regression, the other identifies all landmarks in coronal reconstructed slices using keypoint detection. After landmark detection, segmentation of SCT and SFT was performed on axial slices employing a U-Net architecture with EfficientNet-B1 as encoder. Finally, the determined landmarks were used for standardized analysis and visualization of tissue volume, distribution, and symmetry, independent of leg length, slice thickness, and patient position.

RESULTS

Excellent test results were observed for landmark detection (z-deviation = 4.5 ± 3.1 mm) and segmentation (Dice score: SCT = 0.989 ± 0.004, SFT = 0.994 ± 0.002).

CONCLUSIONS

The proposed DL pipeline allows for standardized analysis of tissue volume and distribution and may assist in diagnosis of lipedema and lymphedema or monitoring of conservative and surgical treatments.

KEY POINTS

• Efficient use of volume information that MRI inherently provides can be extracted automatically by deep learning and enables in-depth assessment of tissue volumes in lipedema and lymphedema. • The deep learning pipeline consisting of body part regression, keypoint detection, and quality-assured tissue segmentation provides detailed information about the volume, distribution, and asymmetry of lower extremity tissues, independent of leg length, slice thickness, and patient position.

Comprehensive Clinical Evaluation of a Deep Learning-Accelerated, Single-Breath-Hold Abdominal HASTE at 1.5 T and 3 T

To evaluate the clinical performance of a deep learning-accelerated single-breath-hold half-Fourier acquisition single-shot turbo spin echo (HASTE_DL)-sequence for T2-weighted fat-suppressed MRI of the abdomen at 1.5 T and 3 T in comparison to standard T2-weighted fat-suppressed multi-shot turbo spin echo-sequence. A total of 320 patients who underwent a clinically indicated liver MRI at 1.5 T and 3 T between August 2020 and February 2021 were enrolled in this single-center, retrospective study. HASTE_DL and standard sequences were assessed regarding overall and organ-based image quality, noise, contrast, sharpness, artifacts, diagnostic confidence, as well as lesion detectability using a Likert scale ranging from 1 to 4 (4 = best). The number of visible lesions of each organ was counted and the largest diameter of the major lesion was measured. HASTE_DL showed excellent image quality (median 4, interquartile range 3-4), although BLADE (median 4, interquartile range 4-4) was rated significantly higher for overall and organ-based image quality of the adrenal gland (P < .001), contrast (P < 0.001), sharpness (P < 0.001), artifacts (P < 0.001), as well as diagnostic confidence (P < .001). No significant differences were found concerning noise (P = 0.886), organ-based image quality of the liver, pancreas, spleen, and kidneys (P = 0.120-0.366), number and measured diameter of the detected lesions (ICC = 0.972-1.0). Reduction of the aquisition time (TA) was at least 89% for 1.5 T images and 86% for 3 T images. HASTE_DL provided excellent image quality, good diagnostic confidence and lesion detection compared to a standard T2-sequences, allowing an eminent reduction of the acquisition time.

Deep Learning Algorithms in the Automatic Segmentation of Liver Lesions in Ultrasound Investigations

BACKGROUND

The ultrasound is one of the most used medical imaging investigations worldwide. It is non-invasive and effective in assessing liver tumors or other types of parenchymal changes.

METHODS

The aim of the study was to build a deep learning model for image segmentation in ultrasound video investigations. The dataset used in the study was provided by the University of Medicine and Pharmacy Craiova, Romania and contained 50 video examinations from 49 patients. The mean age of the patients in the cohort was 69.57. Regarding presence of a subjacent liver disease, 36.73% had liver cirrhosis and 16.32% had chronic viral hepatitis (5 patients: chronic hepatitis C and 3 patients: chronic hepatitis B). Frames were extracted and cropped from each examination and an expert gastroenterologist labelled the lesions in each frame. After labelling, the labels were exported as binary images. A deep learning segmentation model (U-Net) was trained with focal Tversky loss as a loss function. Two models were obtained with two different sets of parameters for the loss function. The performance metrics observed were intersection over union and recall and precision.

RESULTS

Analyzing the intersection over union metric, the first segmentation model obtained performed better compared to the second model: 0.8392 (model 1) vs. 0.7990 (model 2). The inference time for both models was between 32.15 milliseconds and 77.59 milliseconds.

CONCLUSIONS

Two segmentation models were obtained in the study. The models performed similarly during training and validation. However, one model was trained to focus on hard-to-predict labels. The proposed segmentation models can represent a first step in automatically extracting time-intensity curves from CEUS examinations.

Deep learning supports the differentiation of alcoholic and other-than-alcoholic cirrhosis based on MRI

Although CT and MRI are standard procedures in cirrhosis diagnosis, differentiation of etiology based on imaging is not established. This proof-of-concept study explores the potential of deep learning (DL) to support imaging-based differentiation of the etiology of liver cirrhosis. This retrospective, monocentric study included 465 patients with confirmed diagnosis of (a) alcoholic (n = 221) and (b) other-than-alcoholic (n = 244) cirrhosis. Standard T2-weighted single-slice images at the caudate lobe level were randomly split for training with fivefold cross-validation (85%) and testing (15%), balanced for (a) and (b). After automated upstream liver segmentation, two different ImageNet pre-trained convolutional neural network (CNN) architectures (ResNet50, DenseNet121) were evaluated for classification of alcohol-related versus non-alcohol-related cirrhosis. The highest classification performance on test data was observed for ResNet50 with unfrozen pre-trained parameters, yielding an area under the receiver operating characteristic curve of 0.82 (95% confidence interval (CI) 0.71–0.91) and an accuracy of 0.75 (95% CI 0.64–0.85). An ensemble of both models did not lead to significant improvement in classification performance. This proof-of-principle study shows that deep-learning classifiers have the potential to aid in discriminating liver cirrhosis etiology based on standard MRI.

Assessment of Liver Function With MRI: Where Do We Stand?

Liver disease and hepatocellular carcinoma (HCC) have become a global health burden. For this reason, the determination of liver function plays a central role in the monitoring of patients with chronic liver disease or HCC. Furthermore, assessment of liver function is important, e.g., before surgery to prevent liver failure after hepatectomy or to monitor the course of treatment. Liver function and disease severity are usually assessed clinically based on clinical symptoms, biopsy, and blood parameters. These are rather static tests that reflect the current state of the liver without considering changes in liver function. With the development of liver-specific contrast agents for MRI, noninvasive dynamic determination of liver function based on signal intensity or using T1 relaxometry has become possible. The advantage of this imaging modality is that it provides additional information about the vascular structure, anatomy, and heterogeneous distribution of liver function. In this review, we summarized and discussed the results published in recent years on this technique. Indeed, recent data show that the T1 reduction rate seems to be the most appropriate value for determining liver function by MRI. Furthermore, attention has been paid to the development of automated tools for image analysis in order to uncover the steps necessary to obtain a complete process flow from image segmentation to image registration to image analysis. In conclusion, the published data show that liver function values obtained from contrast-enhanced MRI images correlate significantly with the global liver function parameters, making it possible to obtain both functional and anatomic information with a single modality.