Computer-aided image analysis of focal hepatic lesions in ultrasonography: preliminary results

Joint segmentation and classification of hepatic lesions in ultrasound images using deep learning

OBJECTIVES

To develop a convolutional neural network system to jointly segment and classify a hepatic lesion selected by user clicks in ultrasound images.

METHODS

In total, 4309 anonymized ultrasound images of 3873 patients with hepatic cyst (n = 1214), hemangioma (n = 1220), metastasis (n = 1001), or hepatocellular carcinoma (HCC) (n = 874) were collected and annotated. The images were divided into 3909 training and 400 test images. Our network is composed of one shared encoder and two inference branches used for segmentation and classification and takes the concatenation of an input image and two Euclidean distance maps of foreground and background clicks provided by a user as input. The performance of hepatic lesion segmentation was evaluated based on the Jaccard index (JI), and the performance of classification was based on accuracy, sensitivity, specificity, and the area under the receiver operating characteristic curve (AUROC).

RESULTS

We achieved performance improvements by jointly conducting segmentation and classification. In the segmentation only system, the mean JI was 68.5%. In the classification only system, the accuracy of classifying four types of hepatic lesions was 79.8%. The mean JI and classification accuracy were 68.5% and 82.2%, respectively, for the proposed joint system. The optimal sensitivity and specificity and the AUROC of classifying benign and malignant hepatic lesions of the joint system were 95.0%, 86.0%, and 0.970, respectively. The respective sensitivity, specificity, and the AUROC for classifying four hepatic lesions of the joint system were 86.7%, 89.7%, and 0.947.

CONCLUSIONS

The proposed joint system exhibited fair performance compared to segmentation only and classification only systems.

KEY POINTS

• The joint segmentation and classification system using deep learning accurately segmented and classified hepatic lesions selected by user clicks in US examination. • The joint segmentation and classification system for hepatic lesions in US images exhibited higher performance than segmentation only and classification only systems. • The joint segmentation and classification system could assist radiologists with minimal experience in US imaging by characterizing hepatic lesions.

Quantitative Ultrasound Image Analysis Helps in the Differentiation of Hepatocellular Carcinoma (HCC) From Borderline Lesions and Predicting the Histologic Grade of HCC and Microvascular Invasion

OBJECTIVES

Quantitative image analysis is one of the methods to overcome the lack of objectivity of ultrasound (US). The aim of this study was to clarify the correlation between the features from a US image analysis and the histologic grade and microvascular invasion (MVI) of hepatocellular carcinoma (HCC) and differentiation of HCC smaller than 2 cm from borderline lesions.

METHODS

We retrospectively analyzed grayscale US images with histopathologic evidence of HCC or a precancerous lesion using ImageJ version 1.47 software (National Institutes of Health, Bethesda, MD).

RESULTS

A total of 148 nodules were included (borderline lesion, n = 31; early HCC [eHCC], n = 3; well-differentiated HCC [wHCC], n = 16; moderately differentiated HCC [mHCC], n = 79; and poorly differentiated HCC [pHCC], n = 19). A multivariate analysis selected lower minimum gray values (odds ratio [OR], 0.431; P = .003) and a higher standard deviation (OR, 1.880; P = .019) as predictors of HCC smaller than 2 cm. Median (range) minimum gray values of borderline lesions, eHCC, wHCC, mHCC, and pHCC were 29 (0-103), 7 (0-47), 6 (0-60), 10 (0-53), and 2 (0-38), respectively, and gradually decreased from borderline lesions to pHCC (P < 0.001). The multivariate analysis showed a higher aspect ratio (OR, 2.170; P = .001) and lower minimum gray value (OR, 0.475; P = .043) as predictors of MVI. An anechoic area diagnosed by a subjective evaluation was correlated with the minimum gray value (P < .0001). The proportion of the anechoic area gradually increased from eHCC to pHCC (P = .031).

CONCLUSIONS

In a US image analysis, HCC smaller than 2 cm had features of greater heterogeneity and a lower minimum gray value than borderline lesions. Moderately differentiated HCC was smoother than borderline lesions, and the anechoic area correlated with histologic grading. Microvascular invasion was correlated with a slender shape and a lower minimum gray value.

Recognizing Focal Liver Lesions in CEUS With Dynamically Trained Latent Structured Models

This work investigates how to automatically classify Focal Liver Lesions (FLLs) into three specific benign or malignant types in Contrast-Enhanced Ultrasound (CEUS) videos, and aims at providing a computational framework to assist clinicians in FLL diagnosis. The main challenge for this task is that FLLs in CEUS videos often show diverse enhancement patterns at different temporal phases. To handle these diverse patterns, we propose a novel structured model, which detects a number of discriminative Regions of Interest (ROIs) for the FLL and recognize the FLL based on these ROIs. Our model incorporates an ensemble of local classifiers in the attempt to identify different enhancement patterns of ROIs, and in particular, we make the model reconfigurable by introducing switch variables to adaptively select appropriate classifiers during inference. We formulate the model learning as a non-convex optimization problem, and present a principled optimization method to solve it in a dynamic manner: the latent structures (e.g. the selections of local classifiers, and the sizes and locations of ROIs) are iteratively determined along with the parameter learning. Given the updated model parameters in each step, the data-driven inference is also proposed to efficiently determine the latent structures by using the sequential pruning and dynamic programming method. In the experiments, we demonstrate superior performances over the state-of-the-art approaches. We also release hundreds of CEUS FLLs videos used to quantitatively evaluate this work, which to the best of our knowledge forms the largest dataset in the literature. Please find more information at "http://vision.sysu.edu.cn/projects/fllrecog/".

Neural network ensemble based CAD system for focal liver lesions from B-mode ultrasound

A neural network ensemble (NNE) based computer-aided diagnostic (CAD) system to assist radiologists in differential diagnosis between focal liver lesions (FLLs), including (1) typical and atypical cases of Cyst, hemangioma (HEM) and metastatic carcinoma (MET) lesions, (2) small and large hepatocellular carcinoma (HCC) lesions, along with (3) normal (NOR) liver tissue is proposed in the present work. Expert radiologists, visualize the textural characteristics of regions inside and outside the lesions to differentiate between different FLLs, accordingly texture features computed from inside lesion regions of interest (IROIs) and texture ratio features computed from IROIs and surrounding lesion regions of interests (SROIs) are taken as input. Principal component analysis (PCA) is used for reducing the dimensionality of the feature space before classifier design. The first step of classification module consists of a five class PCA-NN based primary classifier which yields probability outputs for five liver image classes. The second step of classification module consists of ten binary PCA-NN based secondary classifiers for NOR/Cyst, NOR/HEM, NOR/HCC, NOR/MET, Cyst/HEM, Cyst/HCC, Cyst/MET, HEM/HCC, HEM/MET and HCC/MET classes. The probability outputs of five class PCA-NN based primary classifier is used to determine the first two most probable classes for a test instance, based on which it is directed to the corresponding binary PCA-NN based secondary classifier for crisp classification between two classes. By including the second step of the classification module, classification accuracy increases from 88.7 % to 95 %. The promising results obtained by the proposed system indicate its usefulness to assist radiologists in differential diagnosis of FLLs.

Characterization of primary and secondary malignant liver lesions from B-mode ultrasound

Characterization of hepatocellular carcinomas (HCCs) and metastatic carcinomas (METs) from B-mode ultrasound presents a daunting challenge for radiologists due to their highly overlapping appearances. The differential diagnosis between HCCs and METs is often carried out by observing the texture of regions inside the lesion and the texture of background liver on which the lesion has evolved. The present study investigates the contribution made by texture patterns of regions inside and outside of the lesions for binary classification between HCC and MET lesions. The study is performed on 51 real ultrasound liver images with 54 malignant lesions, i.e., 27 images with 27 solitary HCCs (13 small HCCs and 14 large HCCs) and 24 images with 27 MET lesions (12 typical cases and 15 atypical cases). A total of 120 within-lesion regions of interest and 54 surrounding lesion regions of interest are cropped from 54 lesions. Subsequently, 112 texture features (56 texture features and 56 texture ratio features) are computed by statistical, spectral, and spatial filtering based texture features extraction methods. A two-step methodology is used for feature set optimization, i.e., feature pruning by removal of nondiscriminatory features followed by feature selection by genetic algorithm-support vector machine (SVM) approach. The SVM classifier is designed based on optimum features. The proposed computer-aided diagnostic system achieved the overall classification accuracy of 91.6 % with sensitivity of 90 % and 93.3 % for HCCs and METs, respectively. The promising results obtained by the proposed system indicate its usefulness to assist radiologists in diagnosing liver malignancies.

A comparative study of computer-aided classification systems for focal hepatic lesions from B-mode ultrasound

A comparative study of three computer-aided classification (CAC) systems for characterization of focal hepatic lesions (FHLs), such as cyst, hemangioma (HEM), hepatocellular carcinoma (HCC) and metastatic carcinoma (MET), along with normal (NOR) liver tissue is carried out in the present work. In order to develop efficient CAC systems a comprehensive and representative dataset consisting of B-mode ultrasound images with (1) typical and atypical cases of cyst, HEM and MET lesions, (2) small and large HCC lesions and (3) NOR liver cases have been used for designing K-nearest neighbour (KNN), probabilistic neural network (PNN) and a back propagation neural network (BPNN) classifiers. For differential diagnosis between atypical FHLs, expert radiologists often visualize the textural characteristics of regions inside and outside the lesion. Accordingly in the present work, texture features and texture ratio features are computed from regions inside and outside the lesions. A feature set consisting of 208 texture features (i.e. 104 texture features and 104 texture ratio features) is subjected to principal component analysis (PCA) for dimensionality reduction; it is observed that maximum accuracy of 87.7% is obtained for a PCA-BPNN-based CAC system in comparison to 86.1% and 85% as obtained by PCA-PNN and PCA-KNN-based CAC systems. The sensitivity of the proposed PCA-BPNN based CAC system for NOR, Cyst, HEM, HCC and MET cases is 82.5%, 96%, 93.3%, 90% and 82.2%, respectively. The sensitivity values with respect to typical, atypical, small HCC and large HCC cases are 85.9%, 88.1%, 100% and 87%, respectively. Keeping in view the comprehensive and representative dataset used for designing the classifier, the results obtained by the proposed PCA-BPNN-based CAC system are quite encouraging and indicate its usefulness to assist experienced radiologists for interpretation and diagnosis of FHLs.

A comprehensive descriptor of shape: method and application to content-based retrieval of similar appearing lesions in medical images

We have developed a method to quantify the shape of liver lesions in CT images and to evaluate its performance for retrieval of images with similarly-shaped lesions. We employed a machine learning method to combine several shape descriptors and defined similarity measures for a pair of shapes as a weighted combination of distances calculated based on each feature. We created a dataset of 144 simulated shapes and established several reference standards for similarity and computed the optimal weights so that the retrieval result agrees best with the reference standard. Then we evaluated our method on a clinical database consisting of 79 portal-venous-phase CT liver images, where we derived a reference standard of similarity from radiologists' visual evaluation. Normalized Discounted Cumulative Gain (NDCG) was calculated to compare this ordering with the expected ordering based on the reference standard. For the simulated lesions, the mean NDCG values ranged from 91% to 100%, indicating that our methods for combining features were very accurate in representing true similarity. For the clinical images, the mean NDCG values were still around 90%, suggesting a strong correlation between the computed similarity and the independent similarity reference derived the radiologists.