Liver Fat Assessment in Multiview Sonography Using Transfer Learning With Convolutional Neural Networks

Development of a Deep Learning Model for Classification of Hepatic Steatosis from Clinical Standard Ultrasound

OBJECTIVE

Early detection and monitoring of hepatic steatosis can help establish appropriate preventative measures against progression to more advanced disease. We aimed to develop a deep learning (DL) program for classification of hepatic steatosis from standard-of-care grayscale ultrasound (US) images.

METHODS

In this single-center retrospective study, we utilized grayscale US images from January 1, 2010, to October 23, 2022, labeled with magnetic resonance imaging (MRI) proton density fat fraction (MRI-PDFF) to develop a DL multi-instance program for differentiating normal (S0) from steatotic liver (S1/2/3) and normal/mild steatosis (S0/1) from moderate/severe steatosis (S2/3). Diagnostic performances were assessed with area under the receiver operating characteristic curves (AUC), sensitivity, specificity and balanced accuracy with 95% confidence interval (CI).

RESULTS

A total of 403 patients with 403 US exams were included: 171 (42%) were normal (S0: MRI-PDFF <5%), 154 (38%) had mild steatosis (S1: MRI-PDFF 5-17.4%), 29 (7%) had moderate steatosis (S2: MRI-PDFF >17.4%-22.1%) and 49 (12%) had severe steatosis (S3: MRI-PDFF >22.1%). The dataset was split to include 322 patients in train/validation and 81 patients in a holdout test set (kept blind). The S0 versus S1/2/3 model achieved 81.3% (95% CI 72.1-90.5) AUC, 81.1% (70.6-91.6) sensitivity, 71.4% (54.7-88.2) specificity and 76.3% (66.4-86.2) balanced accuracy. The S0/1 versus S2/3 model achieved 95.9% (89-100) AUC, 87.5% (71.3-100) sensitivity, 96.9% (92.7-100) specificity and 92.2% (83.8-100) balanced accuracy. A multi-class model achieved a sensitivity of 71.4% (54.7-88.2) for S0, 67.6% (52.5-82.7) for S1 and 87.5% (71.3-100) for S2/3; specificity for the same model was 81.1% (70.6-91.6) for S0, 77.3% (64.9-89.7) for S1 and 96.9% (92.7-100) for S2/3.

CONCLUSION

Our DL program offered high sensitivity and accuracy in detecting and categorizing hepatic steatosis from standard-of-care ultrasound.

Non-invasive imaging biomarkers in chronic liver disease

Chronic liver disease (CLD) is a global and worldwide clinical challenge, considering that different underlying liver entities can lead to hepatic dysfunction. In the past, blood tests and clinical evaluation were the main noninvasive tools used to detect, diagnose and follow-up patients with CLD; in case of clinical suspicion of CLD or unclear diagnosis, liver biopsy has been considered as the reference standard to rule out different chronic liver conditions. Nowadays, noninvasive tests have gained a central role in the clinical pathway. Particularly, liver stiffness measurement (LSM) and cross-sectional imaging techniques can provide transversal information to clinicians, helping them to correctly manage, treat and follow patients during time. Cross-sectional imaging techniques, namely computed tomography (CT) and magnetic resonance imaging (MRI), have plenty of potential. Both techniques allow to compute the liver surface nodularity (LSN), associated with CLDs and risk of decompensation. MRI can also help quantify fatty liver infiltration, mainly with the proton density fat fraction (PDFF) sequences, and detect and quantify fibrosis, especially thanks to elastography (MRE). Advanced techniques, such as intravoxel incoherent motion (IVIM), T1- and T2- mapping are promising tools for detecting fibrosis deposition. Furthermore, the injection of hepatobiliary contrast agents has gained an important role not only in liver lesion characterization but also in assessing liver function, especially in CLDs. Finally, the broad development of radiomics signatures, applied to CT and MR, can be considered the next future approach to CLDs. The aim of this review is to provide a comprehensive overview of the current advancements and applications of both invasive and noninvasive imaging techniques in the evaluation and management of CLD.

Machine-to-Machine Transfer Function in Deep Learning-Based Quantitative Ultrasound

A transfer function approach was recently demonstrated to mitigate data mismatches at the acquisition level for a single ultrasound scanner in deep learning (DL)-based quantitative ultrasound (QUS). As a natural progression, we further investigate the transfer function approach and introduce a machine-to-machine (M2M) transfer function, which possesses the ability to mitigate data mismatches at a machine level. This ability opens the door to unprecedented opportunities for reducing DL model development costs, enabling the combination of data from multiple sources or scanners, or facilitating the transfer of DL models between machines. We tested the proposed method utilizing a SonixOne machine and a Verasonics machine with an L9-4 array and an L11-5 array. We conducted two types of acquisitions to obtain calibration data: stable and free-hand, using two different calibration phantoms. Without the proposed method, the mean classification accuracy when applying a model on data acquired from one system to data acquired from another system was 50%, and the mean average area under the receiver operator characteristic (ROC) curve (AUC) was 0.405. With the proposed method, mean accuracy increased to 99%, and the AUC rose to the 0.999. Additional observations include the choice of the calibration phantom led to statistically significant changes in the performance of the proposed method. Moreover, robust implementation inspired by Wiener filtering provided an effective method for transferring the domain from one machine to another machine, and it can succeed using just a single calibration view. Lastly, the proposed method proved effective when a different transducer was used in the test machine.

Application of artificial intelligence techniques for non-alcoholic fatty liver disease diagnosis: A systematic review (2005-2023)

BACKGROUND AND OBJECTIVES

Non-alcoholic fatty liver disease (NAFLD) is a common liver disease with a rapidly growing incidence worldwide. For prognostication and therapeutic decisions, it is important to distinguish the pathological stages of NAFLD: steatosis, steatohepatitis, and liver fibrosis, which are definitively diagnosed on invasive biopsy. Non-invasive ultrasound (US) imaging, including US elastography technique, and clinical parameters can be used to diagnose and grade NAFLD and its complications. Artificial intelligence (AI) is increasingly being harnessed for developing NAFLD diagnostic models based on clinical, biomarker, or imaging data. In this work, we systemically reviewed the literature for AI-enabled NAFLD diagnostic models based on US (including elastography) and clinical (including serological) data.

METHODS

We performed a comprehensive search on Google Scholar, Scopus, and PubMed search engines for articles published between January 2005 and June 2023 related to AI models for NAFLD diagnosis based on US and/or clinical parameters using the following search terms: "non-alcoholic fatty liver disease", "non-alcoholic steatohepatitis", "deep learning", "machine learning", "artificial intelligence", "ultrasound imaging", "sonography", "clinical information".

RESULTS

We reviewed 64 published models that used either US (including elastography) or clinical data input to detect the presence of NAFLD, non-alcoholic steatohepatitis, and/or fibrosis, and in some cases, the severity of steatosis, inflammation, and/or fibrosis as well. The performances of the published models were summarized, and stratified by data input and algorithms used, which could be broadly divided into machine and deep learning approaches.

CONCLUSION

AI models based on US imaging and clinical data can reliably detect NAFLD and its complications, thereby reducing diagnostic costs and the need for invasive liver biopsy. The models offer advantages of efficiency, accuracy, and accessibility, and serve as virtual assistants for specialists to accelerate disease diagnosis and reduce treatment costs for patients and healthcare systems.

Image-based AI diagnostic performance for fatty liver: a systematic review and meta-analysis

BACKGROUND

The gold standard to diagnose fatty liver is pathology. Recently, image-based artificial intelligence (AI) has been found to have high diagnostic performance. We systematically reviewed studies of image-based AI in the diagnosis of fatty liver.

METHODS

We searched the Cochrane Library, Pubmed, Embase and assessed the quality of included studies by QUADAS-AI. The pooled sensitivity, specificity, negative likelihood ratio (NLR), positive likelihood ratio (PLR), and diagnostic odds ratio (DOR) were calculated using a random effects model. Summary receiver operating characteristic curves (SROC) were generated to identify the diagnostic accuracy of AI models.

RESULTS

15 studies were selected in our meta-analysis. Pooled sensitivity and specificity were 92% (95% CI: 90-93%) and 94% (95% CI: 93-96%), PLR and NLR were 12.67 (95% CI: 7.65-20.98) and 0.09 (95% CI: 0.06-0.13), DOR was 182.36 (95% CI: 94.85-350.61). After subgroup analysis by AI algorithm (conventional machine learning/deep learning), region, reference (US, MRI or pathology), imaging techniques (MRI or US) and transfer learning, the model also demonstrated acceptable diagnostic efficacy.

CONCLUSION

AI has satisfactory performance in the diagnosis of fatty liver by medical imaging. The integration of AI into imaging devices may produce effective diagnostic tools, but more high-quality studies are needed for further evaluation.

The Future Is Beyond Bright: The Evolving Role of Quantitative US for Fatty Liver Disease

Nonalcoholic fatty liver disease (NAFLD) is a common cause of morbidity and mortality. Nonfocal liver biopsy is the historical reference standard for evaluating NAFLD, but it is limited by invasiveness, high cost, and sampling error. Imaging methods are ideally situated to provide quantifiable results and rule out other anatomic diseases of the liver. MRI and US have shown great promise for the noninvasive evaluation of NAFLD. US is particularly well suited to address the population-level problem of NAFLD because it is lower-cost, more available, and more tolerable to a broader range of patients than MRI. Noninvasive US methods to evaluate liver fibrosis are widely available, and US-based tools to evaluate steatosis and inflammation are gaining traction. US techniques including shear-wave elastography, Doppler spectral imaging, attenuation coefficient, hepatorenal index, speed of sound, and backscatter-based estimation have regulatory clearance and are in clinical use. New methods based on channel and radiofrequency data analysis approaches have shown promise but are mostly experimental. This review discusses the advantages and limitations of clinically available and experimental approaches to sonographic liver tissue characterization for NAFLD diagnosis as well as future applications and strategies to overcome current limitations.

Steatosis Quantification on Ultrasound Images by a Deep Learning Algorithm on Patients Undergoing Weight Changes

INTRODUCTION

A deep learning algorithm to quantify steatosis from ultrasound images may change a subjective diagnosis to objective quantification. We evaluate this algorithm in patients with weight changes.

MATERIALS AND METHODS

Patients (N = 101) who experienced weight changes ≥ 5% were selected for the study, using serial ultrasound studies retrospectively collected from 2013 to 2021. After applying our exclusion criteria, 74 patients from 239 studies were included. We classified images into four scanning views and applied the algorithm. Mean values from 3-5 images in each group were used for the results and correlated against weight changes.

RESULTS

Images from the left lobe (G1) in 45 patients, right intercostal view (G2) in 67 patients, and subcostal view (G4) in 46 patients were collected. In a head-to-head comparison, G1 versus G2 or G2 versus G4 views showed identical steatosis scores (R2 > 0.86, p < 0.001). The body weight and steatosis scores were significantly correlated (R2 = 0.62, p < 0.001). Significant differences in steatosis scores between the highest and lowest body weight timepoints were found (p < 0.001). Men showed a higher liver steatosis/BMI ratio than women (p = 0.026).

CONCLUSIONS

The best scanning conditions are 3-5 images from the right intercostal view. The algorithm objectively quantified liver steatosis, which correlated with body weight changes and gender.

Nonalcoholic fatty liver disease (NAFLD) detection and deep learning in a Chinese community-based population

OBJECTIVES

We aimed to develop and validate a deep learning system (DLS) by using an auxiliary section that extracts and outputs specific ultrasound diagnostic features to improve the explainable, clinical relevant utility of using DLS for detecting NAFLD.

METHODS

In a community-based study of 4144 participants with abdominal ultrasound scan in Hangzhou, China, we sampled 928 (617 [66.5%] females, mean age: 56 years ± 13 [standard deviation]) participants (2 images per participant) to develop and validate DLS, a two-section neural network (2S-NNet). Radiologists' consensus diagnosis classified hepatic steatosis as none steatosis, mild, moderate, and severe. We also explored the NAFLD detection performance of six one-section neural network models and five fatty liver indices on our data set. We further evaluated the influence of participants' characteristics on the correctness of 2S-NNet by logistic regression.

RESULTS

Area under the curve (AUROC) of 2S-NNet for hepatic steatosis was 0.90 for ≥ mild, 0.85 for ≥ moderate, and 0.93 for severe steatosis, and was 0.90 for NAFLD presence, 0.84 for moderate to severe NAFLD, and 0.93 for severe NAFLD. The AUROC of NAFLD severity was 0.88 for 2S-NNet, and 0.79-0.86 for one-section models. The AUROC of NAFLD presence was 0.90 for 2S-NNet, and 0.54-0.82 for fatty liver indices. Age, sex, body mass index, diabetes, fibrosis-4 index, android fat ratio, and skeletal muscle via dual-energy X-ray absorptiometry had no significant impact on the correctness of 2S-NNet (p > 0.05).

CONCLUSIONS

By using two-section design, 2S-NNet had improved the performance for detecting NAFLD with more explainable, clinical relevant utility than using one-section design.

KEY POINTS

• Based on the consensus review derived from radiologists, our DLS (2S-NNet) had an AUROC of 0.88 by using two-section design and yielded better performance for detecting NAFLD than using one-section design with more explainable, clinical relevant utility. • The 2S-NNet outperformed five fatty liver indices with the highest AUROCs (0.84-0.93 vs. 0.54-0.82) for different NAFLD severity screening, indicating screening utility of deep learning-based radiology may perform better than blood biomarker panels in epidemiology. • The correctness of 2S-NNet was not significantly influenced by individual's characteristics, including age, sex, body mass index, diabetes, fibrosis-4 index, android fat ratio, and skeletal muscle via dual-energy X-ray absorptiometry.

US Quantification of Liver Fat: Past, Present, and Future

Fatty liver disease has a high and increasing prevalence worldwide, is associated with adverse cardiovascular events and higher long-term medical costs, and may lead to liver-related morbidity and mortality. There is an urgent need for accurate, reproducible, accessible, and noninvasive techniques appropriate for detecting and quantifying liver fat in the general population and for monitoring treatment response in at-risk patients. CT may play a potential role in opportunistic screening, and MRI proton-density fat fraction provides high accuracy for liver fat quantification; however, these imaging modalities may not be suited for widespread screening and surveillance, given the high global prevalence. US, a safe and widely available modality, is well positioned as a screening and surveillance tool. Although well-established qualitative signs of liver fat perform well in moderate and severe steatosis, these signs are less reliable for grading mild steatosis and are likely unreliable for detecting subtle changes over time. New and emerging quantitative biomarkers of liver fat, such as those based on standardized measurements of attenuation, backscatter, and speed of sound, hold promise. Evolving techniques such as multiparametric modeling, radiofrequency envelope analysis, and artificial intelligence-based tools are also on the horizon. The authors discuss the societal impact of fatty liver disease, summarize the current state of liver fat quantification with CT and MRI, and describe past, currently available, and potential future US-based techniques for evaluating liver fat. For each US-based technique, they describe the concept, measurement method, advantages, and limitations. © RSNA, 2023 Online supplemental material is available for this article. Quiz questions for this article are available through the Online Learning Center.

A Data-Efficient Deep Learning Strategy for Tissue Characterization via Quantitative Ultrasound: Zone Training

Deep learning (DL) powered biomedical ultrasound imaging is an emerging research field where researchers adapt the image analysis capabilities of DL algorithms to biomedical ultrasound imaging settings. A major roadblock to wider adoption of DL powered biomedical ultrasound imaging is that acquisition of large and diverse datasets is expensive in clinical settings, which is a requirement for successful DL implementation. Hence, there is a constant need for developing data-efficient DL techniques to turn DL powered biomedical ultrasound imaging into reality. In this work, we develop a data-efficient DL training strategy for classifying tissues based on the ultrasonic backscattered RF data, i.e., quantitative ultrasound (QUS), which we named zone training. In zone training, we propose to divide the complete field of view of an ultrasound image into multiple zones associated with different regions of a diffraction pattern and then, train separate DL networks for each zone. The main advantage of zone training is that it requires less training data to achieve high accuracy. In this work, three different tissue-mimicking phantoms were classified by a DL network. The results demonstrated that zone training can require a factor of 2-3 less training data in low data regime to achieve similar classification accuracies compared to a conventional training strategy.

Automated fatty liver disease detection in point-of-care ultrasound B-mode images

Purpose

Non-alcoholic fatty liver disease (NAFLD) is an increasing global health concern, with a prevalence of 25% worldwide. The rising incidence of NAFLD, an asymptomatic condition, reinforces the need for systematic screening strategies in primary care. We present the use of non-expert acquired point-of-care ultrasound (POCUS) B-mode images for the development of an automated steatosis classification algorithm.

Approach

We obtained a Health Insurance Portability and Accountability Act compliant dataset consisting of 478 patients [body mass index 23.60 ± 3.55 , age 40.97 ± 10.61 ], imaged with POCUS by non-expert health care personnel. A U-Net deep learning (DL) model was used for liver segmentation in the POCUS B-mode images, followed by 224 × 224 patch extraction of liver parenchyma. Several DL models including VGG-16, ResNet-50, Inception V3, and DenseNet-121 were trained for binary classification of steatosis. All layers of each tested model were unfrozen, and the final layer was replaced with a custom classifier. Majority voting was applied for patient-level results.

Results

On a hold-out test set of 81 patients, the final DenseNet-121 model yielded an area under the receiver operator characteristic curve of 90.1%, sensitivity of 95.0%, and specificity of 85.2% for the detection of liver steatosis. Average cross-validation performance in models using patches of liver parenchyma as input outperformed methods using complete B-mode frames.

Conclusions

Despite minimal POCUS acquisition training, and low-quality B-mode images, it is possible to detect steatosis using DL algorithms. Implementation of this algorithm in POCUS software may offer an accessible, low-cost steatosis screening technology, for use by non-expert health care personnel.

Improving diagnostic accuracy of ultrasound texture features in detecting and quantifying hepatic steatosis using various beamforming sound speeds

Objective.While ultrasound image texture has been utilized to detect and quantify hepatic steatosis, the texture features extracted using a single (conventionally 1540 m s-1) beamforming speed of sound (SoS) failed to achieve reliable diagnostic performance. This study aimed to investigate if the texture features extracted using various beamforming SoSs can improve the accuracy of hepatic steatosis detection and quantification.Approach.Patients with suspected non-alcoholic fatty liver disease underwent liver biopsy or MRI proton density fat fraction (PDFF) as part of standard of care, were prospectively enrolled. The radio-frequency data from subjects' right and left liver lobes were collected using 6 beamforming SoSs: 1300, 1350, 1400, 1450, 1500 and 1540 m s-1and analyzed offline. The texture features, i.e. Contrast, Correlation, Energy and Homogeneity from gray-level co-occurrence matrix of normalized envelope were obtained from a region of interest in the liver parenchyma.Main results.Forty-three subjects (67.2%) were diagnosed with steatosis while 21 had no steatosis. Homogeneity showed the area under the curve (AUC) of 0.75-0.82 and 0.58-0.81 for left and right lobes, respectively with varying beamforming SoSs. The combined Homogeneity value over 1300-1540 m s-1from left and right lobes showed the AUC of 0.90 and 0.81, respectively. Furthermore, the combined Homogeneity values from left and right lobes over 1300-1540 m s-1improved the AUC to 0.94. The correlation between texture features and steatosis severity was improved by using the images from various beamforming SoSs. The combined Contrast values over 1300-1540 m s-1from left and right lobes demonstrated the highest correlation (r= 0.90) with the MRI PDFF while the combined Homogeneity values over 1300-1540 m s-1from left and right lobes showed the highest correlation with the biopsy grades (r= -0.81).Significance.The diagnostic accuracy of ultrasound texture features in detecting and quantifying hepatic steatosis was improved by combining its values extracted using various beamforming SoSs.

Advances in the Diagnosis and Treatment of Non-Alcoholic Fatty Liver Disease

Non-alcoholic fatty liver disease (NAFLD) is the most prevalent chronic liver disease that affects approximately one-quarter of the global adult population, posing a significant threat to human health with wide-ranging social and economic implications. The main characteristic of NAFLD is considered that the excessive fat is accumulated and deposited in hepatocytes without excess alcohol intake or some other pathological causes. NAFLD is a progressive disease, ranging from steatosis to non-alcoholic steatohepatitis (NASH), cirrhosis, hepatocellular carcinoma, liver transplantation, and death. Therefore, NAFLD will probably emerge as the leading cause of end-stage liver disease in the coming decades. Unlike other highly prevalent diseases, NAFLD has received little attention from the global public health community. Liver biopsy is currently considered the gold standard for the diagnosis and staging of NAFLD because of the absence of noninvasive and specific biomarkers. Due to the complex pathophysiological mechanisms of NAFLD and the heterogeneity of the disease phenotype, no specific pharmacological therapies have been approved for NAFLD at present, although several drugs are in advanced stages of development. This review summarizes the current evidence on the pathogenesis, diagnosis and treatment of NAFLD.

Liver fat analysis using optimized support vector machine with support vector regression

BACKGROUND

Fatty liver disease is a common condition caused by excess fat in the liver. It consists of two types: Alcoholic Fatty Liver Disease, also called alcoholic steatohepatitis, and Non-Alcoholic Fatty Liver Disease (NAFLD). As per epidemiological studies, fatty liver encompasses 9% to 32% of the general population in India and affects overweight people.

OBJECTIVE

An Optimized Support Vector Machine with Support Vector Regression model is proposed to evaluate the volume of liver fat by image analysis (LFA-OSVM-SVR).

METHOD

The input computed tomography (CT) liver images are collected from the Chennai liver foundation and Liver Segmentation (LiTS) datasets. Here, input datasets are pre-processed using Gaussian smoothing filter and bypass filter to reduce noise and improve image intensity. The proposed U-Net method is used to perform the liver segmentation. The Optimized Support Vector Machine is used to classify the liver images as fatty liver image and normal images. The support vector regression (SVR) is utilized for analyzing the fat in percentage.

RESULTS

The LFA-OSVM-SVR model effectively analyzed the liver fat from CT scan images. The proposed approach is activated in python and its efficiency is analyzed under certain performance metrics.

CONCLUSION

The proposed LFA-OSVM-SVR method attains 33.4%, 28.3%, 25.7% improved accuracy with 55%, 47.7%, 32.6% lower error rate for fatty image classification and 30%, 21%, 19.5% improved accuracy with 57.9%, 46.5%, 31.76% lower error rate for normal image classificationthan compared to existing methods such as Convolutional Neural Network (CNN) with Fractional Differential Enhancement (FDE) (CNN-FDE), Fully Convolutional Networks (FCN) and Non-negative Matrix Factorization (NMF) (FCN-NMF), and Deep Learning with Fully Convolutional Networks (FCN) (DL-FCN).

Ultrasound Radiomics for the Detection of Early-Stage Liver Fibrosis

Objective: The study evaluates quantitative ultrasound (QUS) texture features with machine learning (ML) to enhance the sensitivity of B-mode ultrasound (US) for the detection of fibrosis at an early stage and distinguish it from advanced fibrosis. Different ML methods were evaluated to determine the best diagnostic model. Methods: 233 B-mode images of liver lobes with early and advanced-stage fibrosis induced in a rat model were analyzed. Sixteen features describing liver texture were measured from regions of interest (ROIs) drawn on B-mode images. The texture features included a first-order statistics run length (RL) and gray-level co-occurrence matrix (GLCM). The features discriminating between early and advanced fibrosis were used to build diagnostic models with logistic regression (LR), naïve Bayes (nB), and multi-class perceptron (MLP). The diagnostic performances of the models were compared by ROC analysis using different train-test sampling approaches, including leave-one-out, 10-fold cross-validation, and varying percentage splits. METAVIR scoring was used for histological fibrosis staging of the liver. Results: 15 features showed a significant difference between the advanced and early liver fibrosis groups, p < 0.05. Among the individual features, first-order statics features led to the best classification with a sensitivity of 82.1−90.5% and a specificity of 87.1−89.8%. For the features combined, the diagnostic performances of nB and MLP were high, with the area under the ROC curve (AUC) approaching 0.95−0.96. LR also yielded high diagnostic performance (AUC = 0.91−0.92) but was lower than nB and MLP. The diagnostic variability between test-train trials, measured by the coefficient-of-variation (CV), was higher for LR (3−5%) than nB and MLP (1−2%). Conclusion: Quantitative ultrasound with machine learning differentiated early and advanced fibrosis. Ultrasound B-mode images contain a high level of information to enable accurate diagnosis with relatively straightforward machine learning methods like naïve Bayes and logistic regression. Implementing simple ML approaches with QUS features in clinical settings could reduce the user-dependent limitation of ultrasound in detecting early-stage liver fibrosis.

Comparative Study of Raw Ultrasound Data Representations in Deep Learning to Classify Hepatic Steatosis

Adiposity accumulation in the liver is an early-stage indicator of non-alcoholic fatty liver disease. Analysis of ultrasound (US) backscatter echoes from liver parenchyma with deep learning (DL) may offer an affordable alternative for hepatic steatosis staging. The aim of this work was to compare DL classification scores for liver steatosis using different data representations constructed from raw US data. Steatosis in N = 31 patients with confirmed or suspected non-alcoholic fatty liver disease was stratified based on fat-fraction cutoff values using magnetic resonance imaging as a reference standard. US radiofrequency (RF) frames (raw data) and clinical B-mode images were acquired. Intermediate image formation stages were modeled from RF data. Power spectrum representations and phase representations were also calculated. Co-registered patches were used to independently train 1-, 2- and 3-D convolutional neural networks (CNNs), and classifications scores were compared with cross-validation. There were 67,800 patches available for 2-D/3-D classification and 1,830,600 patches for 1-D classification. The results were also compared with radiologist B-mode annotations and quantitative ultrasound (QUS) metrics. Patch classification scores (area under the receiver operating characteristic curve [AUROC]) revealed significant reductions along successive stages of the image formation process (p &lt; 0.001). Patient AUROCs were 0.994 for RF data and 0.938 for clinical B-mode images. For all image formation stages, 2-D CNNs revealed higher patch and patient AUROCs than 1-D CNNs. CNNs trained with power spectrum representations converged faster than those trained with RF data. Phase information, which is usually discarded in the image formation process, provided a patient AUROC of 0.988. DL models trained with RF and power spectrum data (AUROC = 0.998) provided higher scores than conventional QUS metrics and multiparametric combinations thereof (AUROC = 0.986). Radiologist annotations indicated lower hepatic steatosis classification accuracies (Acc = 0.914) with respect to magnetic resonance imaging proton density fat fraction that DL models (Acc = 0.989). Access to raw ultrasound data combined with artificial intelligence techniques may offer superior opportunities for quantitative tissue diagnostics than conventional sonographic images.

Artificial intelligence in liver ultrasound

Artificial intelligence (AI) is playing an increasingly important role in medicine, especially in the field of medical imaging. It can be used to diagnose diseases and predict certain statuses and possible events that may happen. Recently, more and more studies have confirmed the value of AI based on ultrasound in the evaluation of diffuse liver diseases and focal liver lesions. It can assess the severity of liver fibrosis and nonalcoholic fatty liver, differentially diagnose benign and malignant liver lesions, distinguish primary from secondary liver cancers, predict the curative effect of liver cancer treatment and recurrence after treatment, and predict microvascular invasion in hepatocellular carcinoma. The findings from these studies have great clinical application potential in the near future. The purpose of this review is to comprehensively introduce the current status and future perspectives of AI in liver ultrasound.

Accurate and generalizable quantitative scoring of liver steatosis from ultrasound images via scalable deep learning

BACKGROUND

Hepatic steatosis is a major cause of chronic liver disease. Two-dimensional (2D) ultrasound is the most widely used non-invasive tool for screening and monitoring, but associated diagnoses are highly subjective.

AIM

To develop a scalable deep learning (DL) algorithm for quantitative scoring of liver steatosis from 2D ultrasound images.

METHODS

Using multi-view ultrasound data from 3310 patients, 19513 studies, and 228075 images from a retrospective cohort of patients received elastography, we trained a DL algorithm to diagnose steatosis stages (healthy, mild, moderate, or severe) from clinical ultrasound diagnoses. Performance was validated on two multi-scanner unblinded and blinded (initially to DL developer) histology-proven cohorts (147 and 112 patients) with histopathology fatty cell percentage diagnoses and a subset with FibroScan diagnoses. We also quantified reliability across scanners and viewpoints. Results were evaluated using Bland-Altman and receiver operating characteristic (ROC) analysis.

RESULTS

The DL algorithm demonstrated repeatable measurements with a moderate number of images (three for each viewpoint) and high agreement across three premium ultrasound scanners. High diagnostic performance was observed across all viewpoints: Areas under the curve of the ROC to classify mild, moderate, and severe steatosis grades were 0.85, 0.91, and 0.93, respectively. The DL algorithm outperformed or performed at least comparably to FibroScan control attenuation parameter (CAP) with statistically significant improvements for all levels on the unblinded histology-proven cohort and for “= severe” steatosis on the blinded histology-proven cohort.

CONCLUSION

The DL algorithm provides a reliable quantitative steatosis assessment across view and scanners on two multi-scanner cohorts. Diagnostic performance was high with comparable or better performance than the CAP.

Joint segmentation and classification of breast masses based on ultrasound radio-frequency data and convolutional neural networks

In this paper, we propose a novel deep learning method for joint classification and segmentation of breast masses based on radio-frequency (RF) ultrasound (US) data. In comparison to commonly used classification and segmentation techniques, utilizing B-mode US images, we train the network with RF data (data before envelope detection and dynamic compression), which are considered to include more information on tissue's physical properties than standard B-mode US images. Our multi-task network, based on the Y-Net architecture, can effectively process large matrices of RF data by mixing 1D and 2D convolutional filters. We use data collected from 273 breast masses to compare the performance of networks trained with RF data and US images. The multi-task model developed based on the RF data achieved good classification performance, with area under the receiver operating characteristic curve (AUC) of 0.90. The network based on the US images achieved AUC of 0.87. In the case of the segmentation, we obtained mean Dice scores of 0.64 and 0.60 for the approaches utilizing US images and RF data, respectively. Moreover, the interpretability of the networks was studied using class activation mapping technique and by filter weights visualizations.

A systematic review on application of deep learning in digestive system image processing

With the advent of the big data era, the application of artificial intelligence represented by deep learning in medicine has become a hot topic. In gastroenterology, deep learning has accomplished remarkable accomplishments in endoscopy, imageology, and pathology. Artificial intelligence has been applied to benign gastrointestinal tract lesions, early cancer, tumors, inflammatory bowel diseases, livers, pancreas, and other diseases. Computer-aided diagnosis significantly improve diagnostic accuracy and reduce physicians' workload and provide a shred of evidence for clinical diagnosis and treatment. In the near future, artificial intelligence will have high application value in the field of medicine. This paper mainly summarizes the latest research on artificial intelligence in diagnosing and treating digestive system diseases and discussing artificial intelligence's future in digestive system diseases. We sincerely hope that our work can become a stepping stone for gastroenterologists and computer experts in artificial intelligence research and facilitate the application and development of computer-aided image processing technology in gastroenterology.

Nonalcoholic fatty liver disease: use of diagnostic biomarkers and modalities in clinical practice

INTRODUCTION

The global burden of liver disease is increasing, and nonalcoholic fatty liver disease (NAFLD) is among the most common chronic liver diseases in Asia, Europe, North and South America. The field of noninvasive diagnostic and their role in staging, but also predicting outcome is evolving rapidly. There is a high-unmet need to stage patients with NAFLD and to identify the subset of patients at risk of progression to end-stage liver disease.

AREAS COVERED

The review covers all established diagnostic blood-based and imaging biomarkers to stage and grade NAFLD. Noninvasive surrogate scores are put into perspective of the available evidence and recommended use. The outlook includes genetics, combined algorithms, and artificial intelligence that will allow clinicians to guide and support the management in both early and later disease stages.

EXPERT OPINION

In the future, these diagnostics tests will help clinicians to establish patient care pathways and support the identification of relevant subgroups for monitoring and pharmacotherapy. In addition, researchers will be guided to better understand available scores and support the development of future prediction systems. These will likely include multiparametric aspects of the disease and machine learning algorithms will refine their use and integration with large datasets.