Clinical Value of Information Entropy Compared with Deep Learning for Ultrasound Grading of Hepatic Steatosis

Ultrasound k-nearest neighbor entropy imaging: Theory, algorithm, and applications

Ultrasound information entropy is a flexible approach for analyzing ultrasound backscattering. Shannon entropy imaging based on probability distribution histograms (PDHs) has been implemented as a promising method for tissue characterization and diagnosis. However, the bin number affects the stability of entropy estimation. In this study, we introduced the k-nearest neighbor (KNN) algorithm to estimate entropy values and proposed ultrasound KNN entropy imaging. The proposed KNN estimator leveraged the Euclidean distance between data samples, rather than the histogram bins by conventional PDH estimators. We also proposed cumulative relative entropy (CRE) imaging to analyze time-series radiofrequency signals and applied it to monitor thermal lesions induced by microwave ablation (MWA). Computer simulation phantom experiments were conducted to validate and compare the performance of the proposed KNN entropy imaging, the conventional PDH entropy imaging, and Nakagami-m parametric imaging in detecting the variations of scatterer densities and visualizing inclusions. Clinical data of breast lesions were analyzed, and porcine liver MWA experiments ex vivo were conducted to validate the performance of KNN entropy imaging in classifying benign and malignant breast tumors and monitoring thermal lesions, respectively. Compared with PDH, the entropy estimation based on KNN was less affected by the tuning parameters. KNN entropy imaging was more sensitive to changes in scatterer densities and performed better visualizable capability than typical Shannon entropy (TSE) and Nakagami-m parametric imaging. Among different imaging methods, KNN-based Shannon entropy (KSE) imaging achieved the higher accuracy in classification of benign and malignant breast tumors and KNN-based CRE imaging had larger lesion-to-normal contrast when monitoring the ablated areas during MWA at different powers and treatment durations. Ultrasound KNN entropy imaging is a potential quantitative ultrasound approach for tissue characterization.

Steatosis Quantification on Ultrasound Images by a Deep Learning Algorithm on Patients Undergoing Weight Changes

INTRODUCTION

A deep learning algorithm to quantify steatosis from ultrasound images may change a subjective diagnosis to objective quantification. We evaluate this algorithm in patients with weight changes.

MATERIALS AND METHODS

Patients (N = 101) who experienced weight changes ≥ 5% were selected for the study, using serial ultrasound studies retrospectively collected from 2013 to 2021. After applying our exclusion criteria, 74 patients from 239 studies were included. We classified images into four scanning views and applied the algorithm. Mean values from 3-5 images in each group were used for the results and correlated against weight changes.

RESULTS

Images from the left lobe (G1) in 45 patients, right intercostal view (G2) in 67 patients, and subcostal view (G4) in 46 patients were collected. In a head-to-head comparison, G1 versus G2 or G2 versus G4 views showed identical steatosis scores (R2 > 0.86, p < 0.001). The body weight and steatosis scores were significantly correlated (R2 = 0.62, p < 0.001). Significant differences in steatosis scores between the highest and lowest body weight timepoints were found (p < 0.001). Men showed a higher liver steatosis/BMI ratio than women (p = 0.026).

CONCLUSIONS

The best scanning conditions are 3-5 images from the right intercostal view. The algorithm objectively quantified liver steatosis, which correlated with body weight changes and gender.

Nonalcoholic fatty liver disease (NAFLD) detection and deep learning in a Chinese community-based population

OBJECTIVES

We aimed to develop and validate a deep learning system (DLS) by using an auxiliary section that extracts and outputs specific ultrasound diagnostic features to improve the explainable, clinical relevant utility of using DLS for detecting NAFLD.

METHODS

In a community-based study of 4144 participants with abdominal ultrasound scan in Hangzhou, China, we sampled 928 (617 [66.5%] females, mean age: 56 years ± 13 [standard deviation]) participants (2 images per participant) to develop and validate DLS, a two-section neural network (2S-NNet). Radiologists' consensus diagnosis classified hepatic steatosis as none steatosis, mild, moderate, and severe. We also explored the NAFLD detection performance of six one-section neural network models and five fatty liver indices on our data set. We further evaluated the influence of participants' characteristics on the correctness of 2S-NNet by logistic regression.

RESULTS

Area under the curve (AUROC) of 2S-NNet for hepatic steatosis was 0.90 for ≥ mild, 0.85 for ≥ moderate, and 0.93 for severe steatosis, and was 0.90 for NAFLD presence, 0.84 for moderate to severe NAFLD, and 0.93 for severe NAFLD. The AUROC of NAFLD severity was 0.88 for 2S-NNet, and 0.79-0.86 for one-section models. The AUROC of NAFLD presence was 0.90 for 2S-NNet, and 0.54-0.82 for fatty liver indices. Age, sex, body mass index, diabetes, fibrosis-4 index, android fat ratio, and skeletal muscle via dual-energy X-ray absorptiometry had no significant impact on the correctness of 2S-NNet (p > 0.05).

CONCLUSIONS

By using two-section design, 2S-NNet had improved the performance for detecting NAFLD with more explainable, clinical relevant utility than using one-section design.

KEY POINTS

• Based on the consensus review derived from radiologists, our DLS (2S-NNet) had an AUROC of 0.88 by using two-section design and yielded better performance for detecting NAFLD than using one-section design with more explainable, clinical relevant utility. • The 2S-NNet outperformed five fatty liver indices with the highest AUROCs (0.84-0.93 vs. 0.54-0.82) for different NAFLD severity screening, indicating screening utility of deep learning-based radiology may perform better than blood biomarker panels in epidemiology. • The correctness of 2S-NNet was not significantly influenced by individual's characteristics, including age, sex, body mass index, diabetes, fibrosis-4 index, android fat ratio, and skeletal muscle via dual-energy X-ray absorptiometry.

Tissue Characterization of Puborectalis Muscle From 3-D Ultrasound

Pelvic floor (PF) muscles have the role of preventing pelvic organ descent. The puborectalis muscle (PRM), which is one of the female PF muscles, can be damaged during child delivery. This damage can potentially cause irreversible muscle trauma and even lead to an avulsion, which is disconnection of the muscle from its insertion point, the pubic bone. Ultrasound imaging allows diagnosis of such trauma based on comparison of geometric features of a damaged muscle with the geometric features of a healthy muscle. Although avulsion, which is considered severe damage, can be diagnosed, microdamage within the muscle itself leading to structural changes cannot be diagnosed by visual inspection through imaging only. Therefore, we developed a quantitative ultrasound tissue characterization method to obtain information on the state of the tissue of the PRM and the presence of microdamage in avulsed PRMs. The muscle was segmented as the region of interest (ROI) and further subdivided into six regions of interest (sub-ROIs). Mean echogenicity, entropy and shape parameter of the statistical distribution of gray values were analyzed on two of these sub-ROIs nearest to the bone. The regions nearest to the bones are also the most likely regions to exhibit damage in case of disconnection or avulsion. This analysis was performed for both the muscle at rest and the muscle in contraction. We found that, for PRMs with unilateral avulsion compared with undamaged PRMs, the mean echogenicity (p = 0.02) and shape parameter (p < 0.01) were higher, whereas the entropy was lower (p < 0.01). This method might be applicable to quantification of PRM damage within the muscle.

A comprehensive survey of deep learning research on medical image analysis with focus on transfer learning

This survey aims to identify commonly used methods, datasets, future trends, knowledge gaps, constraints, and limitations in the field to provide an overview of current solutions used in medical image analysis in parallel with the rapid developments in transfer learning (TL). Unlike previous studies, this survey grouped the last five years of current studies for the period between January 2017 and February 2021 according to different anatomical regions and detailed the modality, medical task, TL method, source data, target data, and public or private datasets used in medical imaging. Also, it provides readers with detailed information on technical challenges, opportunities, and future research trends. In this way, an overview of recent developments is provided to help researchers to select the most effective and efficient methods and access widely used and publicly available medical datasets, research gaps, and limitations of the available literature.

Information Entropy and Its Applications

Ultrasound is a first-line diagnostic tool for imaging many disease states. A number of statistical distributions have been proposed to describe ultrasound backscattering measured from tissues having different disease states. As an example, in this chapter we use nonalcoholic fatty liver disease (NAFLD), which is a critical health issue on a global scale, to demonstrate the capabilities of ultrasound to diagnose disease. Ultrasound interaction with the liver is typically characterized by scattering, which is quantified for the purpose of determining the degree of liver steatosis and fibrosis. Information entropy provides an insight into signal uncertainty. This concept allows for the analysis of backscattered statistics without considering the distribution of data or the statistical properties of ultrasound signals. In this chapter, we examined the background of NAFLD and the sources of scattering in the liver. The fundamentals of information entropy and an algorithmic scheme for ultrasound entropy imaging are then presented. Lastly, some examples of using ultrasound entropy imaging to grade hepatic steatosis and evaluate the risk of liver fibrosis in patients with significant hepatic steatosis are presented to illustrate future opportunities for clinical use.

Artificial Intelligence for Detecting and Quantifying Fatty Liver in Ultrasound Images: A Systematic Review

BACKGROUND

Non-alcoholic Fatty Liver Disease (NAFLD) is growing more prevalent worldwide. Although non-invasive diagnostic approaches such as conventional ultrasonography and clinical scoring systems have been proposed as alternatives to liver biopsy, their efficacy has been called into doubt. Artificial Intelligence (AI) is now combined with traditional diagnostic processes to improve the performance of non-invasive approaches.

OBJECTIVE

This study explores how well various AI methods function and perform on ultrasound (US) images to diagnose and quantify non-alcoholic fatty liver disease.

METHODOLOGY

A systematic review was conducted to achieve this objective. Five science bibliographic databases were searched, including PubMed, Association for Computing Machinery ACM Digital Library, Institute of Electrical and Electronics Engineers IEEE Xplore, Scopus, and Google Scholar. Only peer-reviewed English articles, conferences, theses, and book chapters were included. Data from studies were synthesized using narrative methodologies per Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria.

RESULTS

Forty-nine studies were included in the systematic review. According to the qualitative analysis, AI significantly enhanced the diagnosis of NAFLD, Non-Alcoholic Steatohepatitis (NASH), and liver fibrosis. In addition, modalities, image acquisition, feature extraction and selection, data management, and classifiers were assessed and compared in terms of performance measures (i.e., accuracy, sensitivity, and specificity).

CONCLUSION

AI-supported systems show potential performance increases in detecting and quantifying steatosis, NASH, and liver fibrosis in NAFLD patients. Before real-world implementation, prospective studies with direct comparisons of AI-assisted modalities and conventional techniques are necessary.

Comparative Study of Raw Ultrasound Data Representations in Deep Learning to Classify Hepatic Steatosis

Adiposity accumulation in the liver is an early-stage indicator of non-alcoholic fatty liver disease. Analysis of ultrasound (US) backscatter echoes from liver parenchyma with deep learning (DL) may offer an affordable alternative for hepatic steatosis staging. The aim of this work was to compare DL classification scores for liver steatosis using different data representations constructed from raw US data. Steatosis in N = 31 patients with confirmed or suspected non-alcoholic fatty liver disease was stratified based on fat-fraction cutoff values using magnetic resonance imaging as a reference standard. US radiofrequency (RF) frames (raw data) and clinical B-mode images were acquired. Intermediate image formation stages were modeled from RF data. Power spectrum representations and phase representations were also calculated. Co-registered patches were used to independently train 1-, 2- and 3-D convolutional neural networks (CNNs), and classifications scores were compared with cross-validation. There were 67,800 patches available for 2-D/3-D classification and 1,830,600 patches for 1-D classification. The results were also compared with radiologist B-mode annotations and quantitative ultrasound (QUS) metrics. Patch classification scores (area under the receiver operating characteristic curve [AUROC]) revealed significant reductions along successive stages of the image formation process (p &lt; 0.001). Patient AUROCs were 0.994 for RF data and 0.938 for clinical B-mode images. For all image formation stages, 2-D CNNs revealed higher patch and patient AUROCs than 1-D CNNs. CNNs trained with power spectrum representations converged faster than those trained with RF data. Phase information, which is usually discarded in the image formation process, provided a patient AUROC of 0.988. DL models trained with RF and power spectrum data (AUROC = 0.998) provided higher scores than conventional QUS metrics and multiparametric combinations thereof (AUROC = 0.986). Radiologist annotations indicated lower hepatic steatosis classification accuracies (Acc = 0.914) with respect to magnetic resonance imaging proton density fat fraction that DL models (Acc = 0.989). Access to raw ultrasound data combined with artificial intelligence techniques may offer superior opportunities for quantitative tissue diagnostics than conventional sonographic images.

Intelligent Identification of Jute Pests Based on Transfer Learning and Deep Convolutional Neural Networks

Pest attacks pose a substantial threat to jute production and other significant crop plants. Jute farmers in Bangladesh generally distinguish between different pests that appear to be the same using their eyes and expertise, which isn't always accurate. We developed an intelligent model for jute pests identification based on transfer learning (TL) and deep convolutional neural networks (DCNN) to solve this practical problem. The proposed DCNN model can realize fast and accurate automatic identification of jute pests based on photographs. Specifically, the VGG19 CNN model was trained by TL on the ImageNet database. A well-structured image dataset of four dominant jute pests is also established. Our model shows a final accuracy of 95.86% on the four most vital jute pest classes. The model's performance is further demonstrated by the precision, recall, F1-score, and confusion matrix results. The proposed model is integrated into Android and IOS applications for practical uses.

Accurate and generalizable quantitative scoring of liver steatosis from ultrasound images via scalable deep learning

BACKGROUND

Hepatic steatosis is a major cause of chronic liver disease. Two-dimensional (2D) ultrasound is the most widely used non-invasive tool for screening and monitoring, but associated diagnoses are highly subjective.

AIM

To develop a scalable deep learning (DL) algorithm for quantitative scoring of liver steatosis from 2D ultrasound images.

METHODS

Using multi-view ultrasound data from 3310 patients, 19513 studies, and 228075 images from a retrospective cohort of patients received elastography, we trained a DL algorithm to diagnose steatosis stages (healthy, mild, moderate, or severe) from clinical ultrasound diagnoses. Performance was validated on two multi-scanner unblinded and blinded (initially to DL developer) histology-proven cohorts (147 and 112 patients) with histopathology fatty cell percentage diagnoses and a subset with FibroScan diagnoses. We also quantified reliability across scanners and viewpoints. Results were evaluated using Bland-Altman and receiver operating characteristic (ROC) analysis.

RESULTS

The DL algorithm demonstrated repeatable measurements with a moderate number of images (three for each viewpoint) and high agreement across three premium ultrasound scanners. High diagnostic performance was observed across all viewpoints: Areas under the curve of the ROC to classify mild, moderate, and severe steatosis grades were 0.85, 0.91, and 0.93, respectively. The DL algorithm outperformed or performed at least comparably to FibroScan control attenuation parameter (CAP) with statistically significant improvements for all levels on the unblinded histology-proven cohort and for “= severe” steatosis on the blinded histology-proven cohort.

CONCLUSION

The DL algorithm provides a reliable quantitative steatosis assessment across view and scanners on two multi-scanner cohorts. Diagnostic performance was high with comparable or better performance than the CAP.

Information Entropy augmented High Density Crowd Counting Network:

The research proposes an innovated structure of the density map-based crowd counting network augmented by information entropy. The network comprises of a front-end network to extract features and a back-end network to generate density maps. In order to validate the assumption that the entropy can boost the accuracy of density map generation, a multi-scale entropy map extraction process is imported into the front-end network along with a fine-tuned convolutional feature extraction process, In the back-end network, extracted features are decoded into the density map with a multi-column dilated convolution network. Finally, the decoded density map can be mapped as the estimated counting number. Experimental results indicate that the devised network is capable of accurately estimating the count in extremely high crowd density. Compared to similar structured networks which don’t adapt entropy feature, the proposed network exhibits higher performance. This result proves the feature of information entropy is capable of enhancing the efficiency of density map-based crowd counting approaches.

Ultrasound Sample Entropy Imaging: A New Approach for Evaluating Hepatic Steatosis and Fibrosis

Objective: Hepatic steatosis causes nonalcoholic fatty liver disease and may progress to fibrosis. Ultrasound is the first-line approach to examining hepatic steatosis. Fatty droplets in the liver parenchyma alter ultrasound radiofrequency (RF) signal statistical properties. This study proposes using sample entropy, a measure of irregularity in time-series data determined by the dimension [Formula: see text] and tolerance [Formula: see text], for ultrasound parametric imaging of hepatic steatosis and fibrosis. Methods: Liver donors and patients were enrolled, and their hepatic fat fraction (HFF) ([Formula: see text]), steatosis grade ([Formula: see text]), and fibrosis score ([Formula: see text]) were measured to verify the results of sample entropy imaging using sliding-window processing of ultrasound RF data. Results: The sample entropy calculated using [Formula: see text] 4 and [Formula: see text] was highly correlated with the HFF when a small window with a side length of one pulse was used. The areas under the receiver operating characteristic curve for detecting hepatic steatosis that was [Formula: see text]mild, [Formula: see text]moderate, and [Formula: see text]severe were 0.86, 0.90, and 0.88, respectively, and the area was 0.87 for detecting liver fibrosis in individuals with significant steatosis. Discussion/Conclusions: Ultrasound sample entropy imaging enables the identification of time-series patterns in RF signals received from the liver. The algorithmic scheme proposed in this study is compatible with general ultrasound pulse-echo systems, allowing clinical fibrosis risk evaluations of individuals with developing hepatic steatosis.

Deep Learning of Ultrasound Imaging for Evaluating Ambulatory Function of Individuals with Duchenne Muscular Dystrophy

Duchenne muscular dystrophy (DMD) results in loss of ambulation and premature death. Ultrasound provides real-time, safe, and cost-effective routine examinations. Deep learning allows the automatic generation of useful features for classification. This study utilized deep learning of ultrasound imaging for classifying patients with DMD based on their ambulatory function. A total of 85 individuals (including ambulatory and nonambulatory subjects) underwent ultrasound examinations of the gastrocnemius for deep learning of image data using LeNet, AlexNet, VGG-16, VGG-16_TL, VGG-19, and VGG-19_TL models (the notation TL indicates fine-tuning pretrained models). Gradient-weighted class activation mapping (Grad-CAM) was used to visualize features recognized by the models. The classification performance was evaluated using the confusion matrix and receiver operating characteristic (ROC) curve analysis. The results show that each deep learning model endows muscle ultrasound imaging with the ability to enable DMD evaluations. The Grad-CAMs indicated that boundary visibility, muscular texture clarity, and posterior shadowing are relevant sonographic features recognized by the models for evaluating ambulatory function. Of the proposed models, VGG-19 provided satisfying classification performance (the area under the ROC curve: 0.98; accuracy: 94.18%) and feature recognition in terms of physical characteristics. Deep learning of muscle ultrasound is a potential strategy for DMD characterization.

Utility of quantitative ultrasound in community screening for hepatic steatosis

Nonalcoholic fatty liver disease (NAFLD) is the most common chronic liver disease. Quantitative ultrasound facilitates clinical grading of hepatic steatosis (the early stage of NAFLD). However, the utility of quantitative ultrasound as a first-line method for community screening of hepatic steatosis remains unclear. Therefore, this study aimed to investigate the utility of quantitative ultrasound to screen for hepatic steatosis and for metabolic evaluation at the community level. In total, 278 participants enrolled from a community satisfied the study criteria. Each subject underwent anthropometric and biochemical examinations, and abdominal ultrasound imaging was performed to measure the controlled attenuation (CAP), integrated backscatter (IB), and information Shannon entropy (ISE). The assessment outcomes were compared with the fatty liver index (FLI), hepatic steatosis index (HSI), metabolic syndrome (MetS), and insulin resistance to evaluate the screening performance through the area under the receiver operating characteristic curve (AUROC) and Delong's test. Ultrasound ISE, CAP, and IB were effective in screening hepatic steatosis, MetS, and insulin resistance. In screening for hepatic steatosis, the AUROCs of ISE, CAP, and IB were 0.85, 0.83, and 0.80 (the cutoff FLI = 60), respectively, and 0.84, 0.75, 0.77 (the cutoff HSI = 36), respectively, and those for the evaluation of MetS and insulin resistance were 0.79, 0.75, 0.79, respectively, and 0.83, 0.76, 0.78, respectively. Delong's test revealed that ISE outperformed CAP and IB for the detection of hepatic steatosis and insulin resistance (P < .05). Based on the present results, ultrasound ISE is a potential imaging biomarker during first-line community screening of hepatic steatosis and insulin resistance.