Ultrasound RF time series for classification of breast lesions

Spectral Ultrasound Combined With Clinical Pathological Parameters in Prediction of Axillary Lymph Node Metastasis in Breast Cancer

OBJECTIVES

To explore the clinical value of the nomogram based on spectral Doppler ultrasound combined with clinical pathological parameter in predicting axillary lymph node metastasis in breast cancer.

METHODS

We prospectively gathered clinicopathologic and ultrasonic data from 240 patients confirmed breast cancer. The risk factors of axillary lymph node metastasis were analyzed by univariate and multivariate logistic regression, and the prediction model was established. The model calibration, predictive ability, and diagnostic efficiency in the training set and the testing set were analyzed by receiver operating characteristic curve and calibration curve analysis, respectively.

RESULTS

Univariate analysis showed that lymph node metastasis was related with tumor size, Ki-67, axillary ultrasound, ultrasound spectral quantitative parameter, internal echo, and calcification (P < .05). Multivariate logistic regression analysis showed that the Ki-67, axillary ultrasound, quantitative parameter (the mean of the mid-band fit in tumor and posterior tumor) were independent risk factors of axillary lymph node metastasis (P < .05). The models developed using Ki-67, axillary ultrasound, and quantitative parameters for predicting axillary lymph node metastasis demonstrated an area under the receiver operating characteristic curve of 0.83. Additionally, the prediction model exhibited outstanding predictability for axillary lymph node metastasis, as evidenced by a Harrell C-index of 0.83 (95% confidence interval 0.73-0.93).

CONCLUSION

Axillary ultrasound combined with Ki-67 and spectral ultrasound parameters has the potential to predict axillary lymph node metastasis in breast cancer, which is superior to axillary ultrasound alone.

Non-Invasive Monitoring of Cerebral Edema Using Ultrasonic Echo Signal Features and Machine Learning

OBJECTIVES

Cerebral edema, a prevalent consequence of brain injury, is associated with significant mortality and disability. Timely diagnosis and monitoring are crucial for patient prognosis. There is a pressing clinical demand for a real-time, non-invasive cerebral edema monitoring method. Ultrasound methods are prime candidates for such investigations due to their non-invasive nature.

METHODS

Acute cerebral edema was introduced in rats by permanently occluding the left middle cerebral artery (MCA). Ultrasonic echo signals were collected at nine time points over a 24 h period to extract features from both the time and frequency domains. Concurrently, histomorphological changes were examined. We utilized support vector machine (SVM), logistic regression (LogR), decision tree (DT), and random forest (RF) algorithms for classifying cerebral edema types, and SVM, RF, linear regression (LR), and feedforward neural network (FNNs) for predicting the cerebral infarction volume ratio.

RESULTS

The integration of 16 ultrasonic features associated with cerebral edema development with the RF model enabled effective classification of cerebral edema types, with a high accuracy rate of 97.9%. Additionally, it provided an accurate prediction of the cerebral infarction volume ratio, with an R2 value of 0.8814.

CONCLUSIONS

Our proposed strategy classifies cerebral edema and predicts the cerebral infarction volume ratio with satisfactory precision. The fusion of ultrasound echo features with machine learning presents a promising non-invasive approach for the monitoring of cerebral edema.

Computer-Aided Classification of Breast Lesions Based on US RF Time Series Using a Novel Machine Learning Approach

OBJECTIVES

One of the most promising adjuncts for screening breast cancer is ultrasound (US) radio-frequency (RF) time series. It has the superiority of not requiring any supplementary equipment over other methods. This research aimed to propound a machine learning (ML) approach for automatically classifying benign, probably benign, suspicious, and malignant breast lesions based on the features extracted from the accumulated US RF time series.

METHODS

In this article, 220 data of the aforementioned categories, recorded from 118 patients, were analyzed. The dataset, named RFTSBU, was registered by a SuperSonic Imagine Aixplorer medical/research system equipped with a linear transducer. The regions of interest (ROIs) of the B-mode images were manually selected by an expert radiologist before computing the suggested features. Regarding time, frequency, and time-frequency domains, 291 various features were extracted from each ROI. Finally, the features were classified by a pioneering technique named the reference classification method (RCM). Furthermore, the Lee filter was applied to evaluate the effectiveness of reducing speckle noise on the outcomes.

RESULTS

The accuracy of two-class, three-class, and four-class classifications were respectively calculated 98.59 ± 0.71%, 98.13 ± 0.69%, and 96.10 ± 0.66% (considering 10 repetitions) while support vector machine (SVM) and K-nearest neighbor (KNN) classifiers with 5-fold cross-validation were utilized.

CONCLUSIONS

This article represented the proposed approach, named CCRFML, to distinguish between breast lesions based on registered in vivo RF time series employing an ML framework. The proposed method's impressive level of classification accuracy attests to its capability of effectively assisting medical professionals in the noninvasive differentiation of breast lesions.

Evaluating the effect of tissue stimulation at different frequencies on breast lesion classification based on nonlinear features using a novel radio frequency time series approach

Objective

Radio Frequency Time Series (RF TS) is a cutting-edge ultrasound approach in tissue typing. The RF TS does not provide dynamic insights into the propagation medium; when the tissue and probe are fixed. We previously proposed the innovative RFTSDP method in which the RF data are recorded while stimulating the tissue. Applying stimulation can unveil the mechanical characteristics of the tissue in RF echo.

Materials and methods

In this study, an apparatus was developed to induce vibrations at different frequencies to the medium. Data were collected from four PVA phantoms simulating the nonlinear behaviors of healthy, fibroadenoma, cyst, and cancerous breast tissues. Raw focused, raw, and beamformed ultrafast data were collected under conditions of no stimulation, constant force, and various vibrational stimulations using the Supersonic Imagine Aixplorer clinical/research ultrasound imaging system. Time domain (TD), spectral, and nonlinear features were extracted from each RF TS. Support Vector Machine (SVM), Random Forest, and Decision Tree algorithms were employed for classification.

Results

The optimal outcome was achieved using the SVM classifier considering 19 features extracted from beamformed ultrafast data recorded while applying vibration at the frequency of 65 Hz. The classification accuracy, specificity, and precision were 98.44 ± 0.20 %, 99.49 ± 0.01 %, and 98.53 ± 0.04 %, respectively. Applying RFTSDP, a notable 24.45 % improvement in accuracy was observed compared to the case of fixed probe assessing the recorded raw focused data.

Conclusions

External vibration at an appropriate frequency, as applied in RFTSDP, incorporates beneficial information about the medium and its dynamic characteristics into the RF TS, which can improve tissue characterization.

A novel deep learning model for breast lesion classification using ultrasound Images: A multicenter data evaluation

PURPOSE

Breast cancer is one of the major reasons of death due to cancer in women. Early diagnosis is the most critical key for disease screening, control, and reducing mortality. A robust diagnosis relies on the correct classification of breast lesions. While breast biopsy is referred to as the "gold standard" in assessing both the activity and degree of breast cancer, it is an invasive and time-consuming approach.

METHOD

The current study's primary objective was to develop a novel deep-learning architecture based on the InceptionV3 network to classify ultrasound breast lesions. The main promotions of the proposed architecture were converting the InceptionV3 modules to residual inception ones, increasing their number, and altering the hyperparameters. In addition, we used a combination of five datasets (three public datasets and two prepared from different imaging centers) for training and evaluating the model.

RESULTS

The dataset was split into the train (80%) and test (20%) groups. The model achieved 0.83, 0.77, 0.8, 0.81, 0.81, 0.18, and 0.77 for the precision, recall, F1 score, accuracy, AUC, Root Mean Squared Error, and Cronbach's α in the test group, respectively.

CONCLUSIONS

This study illustrates that the improved InceptionV3 can robustly classify breast tumors, potentially reducing the need for biopsy in many cases.

Improving breast cancer diagnosis by incorporating raw ultrasound parameters into machine learning

The improved diagnostic accuracy of ultrasound breast examinations remains an important goal. In this study, we propose a biophysical feature-based machine learning method for breast cancer detection to improve the performance beyond a benchmark deep learning algorithm and to furthermore provide a color overlay visual map of the probability of malignancy within a lesion. This overall framework is termed disease-specific imaging. Previously, 150 breast lesions were segmented and classified utilizing a modified fully convolutional network and a modified GoogLeNet, respectively. In this study multiparametric analysis was performed within the contoured lesions. Features were extracted from ultrasound radiofrequency, envelope, and log-compressed data based on biophysical and morphological models. The support vector machine with a Gaussian kernel constructed a nonlinear hyperplane, and we calculated the distance between the hyperplane and each feature's data point in multiparametric space. The distance can quantitatively assess a lesion and suggest the probability of malignancy that is color-coded and overlaid onto B-mode images. Training and evaluation were performed on in vivo patient data. The overall accuracy for the most common types and sizes of breast lesions in our study exceeded 98.0% for classification and 0.98 for an area under the receiver operating characteristic curve, which is more precise than the performance of radiologists and a deep learning system. Further, the correlation between the probability and Breast Imaging Reporting and Data System enables a quantitative guideline to predict breast cancer. Therefore, we anticipate that the proposed framework can help radiologists achieve more accurate and convenient breast cancer classification and detection.

Evaluation of Hepatic Fibrosis Using Ultrasound Backscattered Radiofrequency Signals and One-Dimensional Convolutional Neural Networks

The early detection of hepatic fibrosis is of critical importance. Ultrasound backscattered radiofrequency signals from the liver contain abundant information about its microstructure. We proposed a method for characterizing human hepatic fibrosis using one-dimensional convolutional neural networks (CNNs) based on ultrasound backscattered signals. The proposed CNN model was composed of four one-dimensional convolutional layers, four one-dimensional max-pooling layers, and four fully connected layers. Ultrasound radiofrequency signals collected from 230 participants (F0: 23; F1: 46; F2: 51; F3: 49; F4: 61) with a 3-MHz transducer were analyzed. Liver regions of interest (ROIs) that contained most of the liver ultrasound backscattered signals were manually delineated using B-mode images reconstructed from the backscattered signals. ROI signals were normalized and augmented by using a sliding window technique. After data augmentation, the radiofrequency signal segments were divided into training sets, validation sets and test sets at a ratio of 80%:10%:10%. In the test sets, the proposed algorithm produced an area under the receive operating characteristic curve of 0.933 (accuracy: 91.30%; sensitivity: 92.00%; specificity: 90.48%), 0.997 (accuracy: 94.29%; sensitivity: 94.74%; specificity: 93.75%), 0.818 (accuracy: 75.00%; sensitivity: 69.23%; specificity: 81.82%), and 0.934 (accuracy: 91.67%; sensitivity: 88.89%; specificity: 94.44%) for diagnosis liver fibrosis stage ≥F1, ≥F2, ≥F3, and ≥F4, respectively. Experimental results indicated that the proposed deep learning algorithm based on ultrasound backscattered signals yields a satisfying performance when diagnosing hepatic fibrosis stages. The proposed method may be used as a new quantitative ultrasound approach to characterizing hepatic fibrosis.

Osteoporosis Diagnostic Model Using a Multichannel Convolutional Neural Network Based on Quantitative Ultrasound Radiofrequency Signal

Quantitative ultrasound (QUS) is a promising screening method for osteoporosis. In this study, a new method to improve the diagnostic accuracy of QUS was established in which a multichannel convolutional neural network (MCNN) processes the raw radiofrequency (RF) signal of QUS. The improvement in the diagnostic accuracy of osteoporosis using this new method was evaluated by comparison with the conventional speed of sound (SOS) method. Dual-energy X-ray absorptiometry was used as the diagnostic standard. After being trained, validated and tested in a data set consisting of 274 participants, the MCNN model could significantly raise the accuracy of osteoporosis diagnosis compared with the SOS method. The adjusted MCNN model performed even better when adjusted by age, height and weight data. The sensitivity, specificity and accuracy of the adjusted MCNN method for osteoporosis diagnosis were 80.86%, 84.23% and 83.05%, respectively; the corresponding values for SOS were 50.60%, 73.68% and 66.67%. The area under the receiver operating characteristic curve of the adjusted MCNN method was also higher than that of SOS (0.846 vs. 0.679). In conclusion, our study indicates that the MCNN method may be more accurate than the conventional SOS method. The MCNN tool and ultrasound RF signal analysis are promising future developmental directions for QUS in screening for osteoporosis.

Joint segmentation and classification of breast masses based on ultrasound radio-frequency data and convolutional neural networks

In this paper, we propose a novel deep learning method for joint classification and segmentation of breast masses based on radio-frequency (RF) ultrasound (US) data. In comparison to commonly used classification and segmentation techniques, utilizing B-mode US images, we train the network with RF data (data before envelope detection and dynamic compression), which are considered to include more information on tissue's physical properties than standard B-mode US images. Our multi-task network, based on the Y-Net architecture, can effectively process large matrices of RF data by mixing 1D and 2D convolutional filters. We use data collected from 273 breast masses to compare the performance of networks trained with RF data and US images. The multi-task model developed based on the RF data achieved good classification performance, with area under the receiver operating characteristic curve (AUC) of 0.90. The network based on the US images achieved AUC of 0.87. In the case of the segmentation, we obtained mean Dice scores of 0.64 and 0.60 for the approaches utilizing US images and RF data, respectively. Moreover, the interpretability of the networks was studied using class activation mapping technique and by filter weights visualizations.

A Multi-Task Learning Framework for Automated Segmentation and Classification of Breast Tumors From Ultrasound Images

Breast cancer is one of the most fatal diseases leading to the death of several women across the world. But early diagnosis of breast cancer can help to reduce the mortality rate. So an efficient multi-task learning approach is proposed in this work for the automatic segmentation and classification of breast tumors from ultrasound images. The proposed learning approach consists of an encoder, decoder, and bridge blocks for segmentation and a dense branch for the classification of tumors. For efficient classification, multi-scale features from different levels of the network are used. Experimental results show that the proposed approach is able to enhance the accuracy and recall of segmentation by 1.08%, 4.13%, and classification by 1.16%, 2.34%, respectively than the methods available in the literature.

Breast Cancer Detection using Multimodal Time Series Features from Ultrasound Shear Wave Absolute Vibro-Elastography

In shear wave absolute vibro-elastography (S-WAVE), a steady-state multi-frequency external mechanical excitation is applied to tissue, while a time-series of ultrasound radio-frequency (RF) data are acquired. Our objective is to determine the potential of S-WAVE to classify breast tissue lesions as malignant or benign. We present a new processing pipeline for feature-based classification of breast cancer using S-WAVE data, and we evaluate it on a new data set collected from 40 patients. Novel bi-spectral and Wigner spectrum features are computed directly from the RF time series and are combined with textural and spectral features from B-mode and elasticity images. The Random Forest permutation importance ranking and the Quadratic Mutual Information methods are used to reduce the number of features from 377 to 20. Support Vector Machines and Random Forest classifiers are used with leave-one-patient-out and Monte Carlo cross-validations. Classification results obtained for different feature sets are presented. Our best results (95% confidence interval, Area Under Curve = 95%1.45%, sensitivity = 95%, and specificity = 93%) outperform the state-of-the-art reported S-WAVE breast cancer classification performance. The effect of feature selection and the sensitivity of the above classification results to changes in breast lesion contours is also studied. We demonstrate that time-series analysis of externally vibrated tissue as an elastography technique, even if the elasticity is not explicitly computed, has promise and should be pursued with larger patient datasets. Our study proposes novel directions in the field of elasticity imaging for tissue classification.

Index of Reflectivity of Ultrasound Radio Frequency Signal from the Carotid Artery Wall Increases in Astronauts after a 6 mo Spaceflight

The objective was to quantify the index of reflectivity of the common carotid artery and surrounding structures, before and after 6 mo of microgravity. Our hypothesis was that structural changes in the insonated target would increase its index of reflectivity. The neck anterior muscle and common carotid artery (walls and lumen) were visualized by echography (17 MHz linear probe), and the radiofrequency signal along each vertical line was displayed. The limits of the radiofrequency data corresponding to each target (muscle, vessel wall) were determined from the B-mode image and radiofrequency trace. Each target's index of reflectivity was calculated as the proportion of backscattered energy to the whole backscattered energy along the line. After 6 mo in flight, the index of reflectivity increased significantly for both common carotid walls, while it remained unchanged for the neck muscle, carotid intima and lumen. The index of reflectivity provided additional information beyond traditional B-mode imaging.

Classification of red blood cell aggregation using empirical wavelet transform analysis of ultrasonic radiofrequency echo signals

Grading red blood cell (RBC) aggregation is important for the early diagnosis and prevention of related diseases such as ischemic cardio-cerebrovascular disease, type II diabetes, deep vein thrombosis, and sickle cell disease. In this study, a machine learning technique based on an adaptive analysis of ultrasonic radiofrequency (RF) echo signals in blood is proposed, and its feasibility for classifying RBC aggregation is explored. Using an adaptive empirical wavelet transform (EWT) analysis, the ultrasonic RF signals are decomposed into a series of empirical mode functions (EMFs); then, dominant empirical mode functions (DEMFs) are selected from the series. Six statistical characteristics, including the mean, variance, median, kurtosis, root mean square (RMS), and skewness are calculated for the locally normalized DEMFs, aiming to form primary feature vectors. Random forest (RDF) and support vector machine (SVM) classifiers are trained with the given feature vectors to obtain prediction models for RBC classification. Ultrasonic RF echo signals are acquired from five groups of six types of porcine blood samples with average numbers of aggregated RBCs of 1.04, 1.20, 1.83, 2.31, 2.72, and 4.28, respectively, to test the classification performance of the proposed method. The best subset with regard to the variance, kurtosis, and RMS is determined according to the maximum accuracy based on the RDF and SVM classifiers. The classification accuracies are 84.03 ± 3.13% for the RDF classifier, and 85.88 ± 2.99% for the SVM classifier. The mean classification accuracy of the SVM classifier is 1.85% better than that of the RDF classifier. In conclusion, the machine learning method is useful for the discrimination of varying degrees of RBC aggregation, and has potential for use in characterizing and monitoring the RBC aggregation in vessels.

Virtual Source Synthetic Aperture for Accurate Lateral Displacement Estimation in Ultrasound Elastography

Ultrasound elastography (USE) is an emerging noninvasive imaging technique in which pathological alterations can be visualized by revealing the mechanical properties of the tissue. Estimating tissue displacement in all directions is required to accurately estimate the mechanical properties. Despite capabilities of elastography techniques in estimating displacement in both axial and lateral directions, estimation of axial displacement is more accurate than lateral direction due to higher sampling frequency, higher resolution, and having a carrier signal propagating in the axial direction. Among different ultrasound imaging techniques, synthetic aperture (SA) has better lateral resolution than others, but it is not commonly used for USE due to its limitation in imaging depth of field. Virtual source synthetic aperture (VSSA) imaging is a technique to implement SA beamforming on the focused transmitted data to overcome the limitation of SA in depth of field while maintaining the same lateral resolution as SA. Besides lateral resolution, VSSA has the capability of increasing sampling frequency in the lateral direction without interpolation. In this article, we utilize VSSA to perform beamforming to enable higher resolution and sampling frequency in the lateral direction. The beamformed data are then processed using our recently published elastography technique, OVERWIND. Simulation and experimental results show substantial improvement in the estimation of lateral displacements.

Microcalcification-Based Tumor Malignancy Evaluation in Fresh Breast Biopsies with Hyperspectral Stimulated Raman Scattering

Precise evaluation of breast tumor malignancy based on tissue calcifications has important practical value in the disease diagnosis, as well as the understanding of tumor development. Traditional X-ray mammography provides the overall morphologies of the calcifications but lacks intrinsic chemical information. In contrast, spontaneous Raman spectroscopy offers detailed chemical analysis but lacks the spatial profiles. Here, we applied hyperspectral stimulated Raman scattering (SRS) microscopy to extract both the chemical and morphological features of the microcalcifications, based on the spectral and spatial domain analysis. A total of 211 calcification sites from 23 patients were imaged with SRS, and the results were analyzed with a support vector machine (SVM) based classification algorithm. With optimized combinations of chemical and geometrical features of microcalcifications, we were able to reach a precision of 98.21% and recall of 100.00% for classifying benign and malignant cases, significantly improved from the pure spectroscopy or imaging based methods. Our findings may provide a rapid means to accurately evaluate breast tumor malignancy based on fresh tissue biopsies.

Quantitative Evaluation of Rotator Cuff Tears Based on Non-linear Statistical Analysis of Ultrasound Radiofrequency Signals

There is increasing clinical requirement for early and accurate ultrasound diagnosis of rotator cuff tears (RCTs). A method based on non-linear statistical analysis was proposed for the detection of RCTs using ultrasound radiofrequency (RF) signals. One hundred fifty-two patients with shoulder pain were first examined with ultrasound and then diagnosed with magnetic resonance imaging (MRI) as the ground truth. By comparison of the region of interest (ROI) with a part of the supraspinatus with no pathologic change part in the same RF signal frame, the relative P_ks value (viz., rP_ks value) was evaluated to quantify the pathophysiologic changes. The results indicated that the rP_ks values of all RCTs are <0.7, and the accuracy, sensitivity and specificity of the proposed method can reach 97.5%, 100% and 91.4%, respectively. This computer-aided method was found to perform better diagnostic than the results reported by an experienced radiologist (accuracy = 75.7%, sensitivity = 72.6%, and specificity = 85.7%). The high sensitivity advantage of this method indicates that the prospects for its application in the computer-aided diagnosis of RCTs are good.

Quantitative evaluation of liver fibrosis based on ultrasound radio frequency signals: An animal experimental study

BACKGROUND

Chronic liver disease is an important cause of liver failure and death worldwide, and liver fibrosis is a common pathological process of most chronic liver diseases. There still lacks a useful tool for evaluating liver fibrosis progression precisely and non-invasively. The purpose of this study was to explore the use of ultrasound radio frequency (RF) signals combined with deep learning approach to evaluate the degree of liver fibrosis quantitatively.

METHODS

In this study, by extracting the output of deep learning models as a prediction value, a quantitative liver fibrosis prediction method was achieved based on the bidirectional long short-term memory (Bi-LSTM) network to analyze radio frequency (RF) signals. The dataset consisted of 160 sets of ultrasound RF signals of rat livers, including five fibrosis stages 0-4, upon pathological diagnosis. In total, 150 sets of RF signals were used to train four deep learning classification models, the output of which contained quantitative information. In each training stage of the four models, a large number of signal segments were extracted from the 150 sets and divided randomly into training and validation sets in a ratio of 80:20. Ten sets of RF data using the gold standard of quantitative fibrosis parameter (q-FP) of liver tissues were left for independent testing. To validate the proposed method, correlation analysis was carried out between q-FP and the quantitative prediction results based on the independent test data.

RESULTS

The accuracy of the four deep learning networks using the training and validation data was above 0.83 and 0.80, and the corresponding areas under the receiver operating characteristic curves were higher than 0.95 and 0.93, respectively. For the quantitative analysis in the independent test set, the determination coefficient, R², of the linear regression analysis between the quantitative prediction results and q-FP was above 0.93. liver fibrosis is a common pathological process of most chronic liver diseases.

CONCLUSIONS

This study indicates that a prediction system based on ultrasound RF signals and a deep learning approach is promising for realizing quantitative and visualized diagnosis of liver fibrosis, which would be of great value in monitoring liver fibrosis non-invasively.

Radial Motion Estimation of Myocardium in Rats with Myocardial Infarction: A Hybrid Method of FNCCGLAM and Polar Transformation

Ultrasound elastography is a novel approach of evaluating regional myocardial systolic function and detecting infarcted area. This study aims to evaluate the radial motion of myocardial infarction (MI) area in left ventricular parasternal short axis (PSAX) view using a hybrid method of fast normalized cross-correlation and global analytic minimization (FNCCGLAM) and polar transformation. Fifteen rats were randomly selected for sham group, MI group and ischemia-reperfusion (IR) group (N = 5 for each group). The ultrasound radiofrequency data of the PSAX view of rat heart were acquired. After polar transformation of the data, the infarcted myocardium with the change of mechanical property was tracked over one myocardial systolic phase by the proposed method in comparison with fast normalized cross-correlation (FNCC) and dynamic programming analytic minimization (DPAM). To obtain a clear visualization of the myocardium, the inverse polar transformation was performed. The results indicated that the use of FNCCGLAM refined the myocardial displacements to obtain high-quality myocardial elastographic map with a higher contrast-to-noise ratio and dynamically tracked the infarcted myocardial segment with a higher success rate in comparison with FNCC and DPAM. It was found that the radial systolic motion of the infarcted anterior segment in the MI group reduced significantly (p < 0.05) in comparison with the sham group, while the systolic function of that myocardial segment in the IR group recovered at some extent. The results in this study suggest that FNCCGLAM is superior to FNCC and DPAM with the improved accuracy and robustness of motion estimation and has potentials as displacement estimator in ultrasound elastography.

Multi-task learning for segmentation and classification of tumors in 3D automated breast ultrasound images

Tumor classification and segmentation are two important tasks for computer-aided diagnosis (CAD) using 3D automated breast ultrasound (ABUS) images. However, they are challenging due to the significant shape variation of breast tumors and the fuzzy nature of ultrasound images (e.g., low contrast and signal to noise ratio). Considering the correlation between tumor classification and segmentation, we argue that learning these two tasks jointly is able to improve the outcomes of both tasks. In this paper, we propose a novel multi-task learning framework for joint segmentation and classification of tumors in ABUS images. The proposed framework consists of two sub-networks: an encoder-decoder network for segmentation and a light-weight multi-scale network for classification. To account for the fuzzy boundaries of tumors in ABUS images, our framework uses an iterative training strategy to refine feature maps with the help of probability maps obtained from previous iterations. Experimental results based on a clinical dataset of 170 3D ABUS volumes collected from 107 patients indicate that the proposed multi-task framework improves tumor segmentation and classification over the single-task learning counterparts.

Synthetic aperture with high lateral sampling frequency for ultrasound elastography

Ultrasound elastography is a non-invasive technique for detecting pathological alterations in tissue. It is known that pathological alteration of tissue often has a direct impact on its elastic modulus, which can be revealed using elastography. For estimating elastic modulus, we need to estimate both axial and lateral displacement accurately. Current state of the art elastography techniques provide a substantially less accurate lateral displacement field as compared to the axial displacement field. One of the most important factors in poor lateral estimation is a low sampling frequency in the lateral direction. In this paper, we use synthetic aperture beamforming to benefit from its capability of high sampling frequency in the lateral direction. We compare highly sampled data and focused line per line beam formed data by feeding them to our recently published elastography method, OVERWIND [1]. According to simulation and phantom experiments, not only the lateral displacement estimation is substantially improved, but also the axial displacement estimation is improved.

Accurate and Precise Time-Delay Estimation for Ultrasound Elastography With Prebeamformed Channel Data

Free-hand palpation ultrasound elastography is a noninvasive approach for detecting pathological alteration in tissue. In this method, the tissue is compressed by a handheld probe and displacement of each sample is estimated, a process which is also known as time-delay estimation (TDE). Even with the simplifying assumption that ignores out of plane motion, TDE is an ill-posed problem requiring estimation of axial and lateral displacements for each sample from its intensity. A well-known class of methods for making elastography a well-posed problem is regularized optimization-based methods, which imposes smoothness regularization in the associated cost function. In this article, we propose to utilize channel data that have been compensated for time gain and time delay (introduced by transmission) instead of postbeamformed radio frequency (RF) data in the optimization problem. We name our proposed method Channel data for GLobal Ultrasound Elastography (CGLUE). We analytically derive bias and variances of TDE as functions of data noise for CGLUE and Global Ultrasound Elastography (GLUE) and use the Cauchy-Schwarz inequality to prove that CGLUE provides a TDE with lower bias and variance error. To further illustrate the improved performance of CGLUE, the results of simulation, experimental phantom, and ex-vivo experiments are presented.

Soft Tissue Characterization with Temporal Enhanced Ultrasound through Periodic Manipulation of Point Spread Function: A Feasibility Study

Temporal enhanced ultrasound (TeUS) is a tissue characterization approach based on analysis of a temporal series of US data. Previously we demonstrated that intrinsic or external micro-motions of scatterers in the tissue contribute towards the tissue classification properties of TeUS. This property is beneficial to detect early stage cancer, for example, where changes in nuclei configuration (scatteres) dominate tissue properties. In this study, we propose an analytical derivation and experiments to acquire TeUS through manipulation of US imaging parameters, which may be simpler to translate to clinical applications. The feasibility of the proposed method is demonstrated on tissue-mimicking phantoms. Using an autoencoder classifier, we are able to classify phantoms of varying elasticities and scattering sizes.

3D normalized cross-correlation for estimation of the displacement field in ultrasound elastography

This paper introduces a novel technique to estimate tissue displacement in quasi-static elastography. A major challenge in elastography is estimation of displacement (also referred to time-delay estimation) between pre-compressed and post-compressed ultrasound data. Maximizing normalized cross correlation (NCC) of ultrasound radio-frequency (RF) data of the pre- and post-compressed images is a popular technique for strain estimation due to its simplicity and computational efficiency. Several papers have been published to increase the accuracy and quality of displacement estimation based on NCC. All of these methods use 2D spatial windows in RF data to estimate NCC, wherein displacement is assumed to be constant within each window. In this work, we extend this assumption along the third dimension. Two approaches are proposed to get third dimension. In the first approach, we use temporal domain to exploit neighboring samples in both spatial and temporal directions. Considering temporal information is important since traditional and ultrafast ultrasound machines are, respectively, capable of imaging at more than 30 frame per second (fps) and 1000 fps. Another approach is to use time-delayed pre-beam formed data (channel data) instead of RF data. In this method information of all channels that are recorded as pre-beam formed data of each RF line will be considered as 3^rd dimension. We call these methods as spatial temporal normalized cross correlation (STNCC) and channel data normalized cross correlation (CNCC) and show that they substantially outperforms NCC using simulation, phantom and in-vivo experiments. Given substantial improvements of results in addition to the relative simplicity of implementing STNCC and CNCC, the proposed approaches can potentially have a large impact in both academic and commercial work on ultrasound elastography.

Breast tumor classification through learning from noisy labeled ultrasound images

PURPOSE

To train deep learning models to differentiate benign and malignant breast tumors in ultrasound images, we need to collect many training samples with clear labels. In general, biopsy results can be used as benign/malignant labels. However, most clinical samples generally do not have biopsy results. Previous works have proposed generating benign/malignant labels according to Breast Imaging, Reporting and Data System (BI-RADS) ratings. However, this approach will cause noisy labels, which means that the benign/malignant labels produced from BI-RADS diagnoses may be inconsistent with the ground truths. Consequently, deep models will overfit the noisy labels and hence obtain poor generalization performance. In this work, we mainly focus on how to reduce the negative effect of noisy labels when they are used to train breast tumor classification models.

METHODS

We propose an effective approach called noise filter network (NF-Net) to address the problem of noisy labels when training breast tumor classification models. Specifically, to prevent deep models from overfitting the noisy labels, we propose incorporating two softmax layers for classification. Additionally, to strengthen the effect of clean labels, we design a teacher-student module for distilling the knowledge of clean labels.

RESULTS

We conduct extensive comparisons with the existing works on addressing noisy labels. Our method achieves a classification accuracy of 73%, with a precision of 69%, recall of 80%, and F1-score of 0.74. This result is significantly better than those of the existing state-of-the-art works on addressing noisy labels.

CONCLUSIONS

This work provides a means to overcome the label shortage problem in training breast tumor classification models. Specifically, we can generate benign/malignant labels according to the BI-RADS ratings. Although this approach will cause noisy labels, the design of NF-Net can effectively reduce the negative effect of such labels.

Combining Total Variation Regularization with Window-Based Time Delay Estimation in Ultrasound Elastography

A major challenge of free-hand palpation ultrasound elastography (USE) is estimating the displacement of RF samples between pre- and post-compressed RF data. The problem of displacement estimation is ill-posed since the displacement of one sample by itself cannot be uniquely calculated. To resolve this problem, two categories of methods have emerged. The first category assumes that the displacement of samples within a small window surrounding the reference sample is constant. The second class imposes smoothness regularization and optimizes an energy function. Herein, we propose a novel method that combines both approaches, and as such, is more robust to noise. The second contribution of this work is the introduction of the L1 norm as the regularization term in our cost function, which is often referred to as the total variation (TV) regularization. Compared to previous work that used the L2 norm regularization, optimization of the new cost function is more challenging. However, the advantages of using the L1 norm are twofold. First, it leads to substantial improvement in the sharpness of displacement estimates. Second, to optimize the cost function with the L1 norm regularization, we use an iterative method that further increases the robustness. We name our proposed method tOtal Variation Regularization and WINDow-based time delay estimation (OVERWIND) and show that it is robust to signal decorrelation and generates sharp displacement and strain maps for simulated, experimental phantom and in-vivo data. In particular, OVERWIND improves strain contrast-to-noise ratio (CNR) by 27.26%, 144.05%, and 49.90% on average in simulation, phantom, and in-vivo data, respectively, compared to our recent Global Ultrasound Elastography (GLUE) method.

Classification and prediction of inertial cavitation activity induced by pulsed high-intensity focused ultrasound

Classification and prediction of ultrasound-induced microbubble inertial cavitation (IC) activity may play an important role in better design of ultrasound treatment strategy with improved efficiency and safety. Here, a new method was proposed by combining support vector machine (SVM) algorithm with passive cavitation detection (PCD) measurements to fulfill the tasks of IC event classification and IC dose prediction. By using the PCD system, IC thresholds and IC doses were firstly measured for various ultrasound contrast agent (UCA) solutions exposed to pulsed high-intensity focused ultrasound (pHIFU) at different driving pressures and pulse lengths. Then, after trained and tested by measured data, two SVM models (viz. C-SVC and ε-SVR) were established to classify the likelihood of IC event occurrence and predict IC dose, respectively, under different parameter conditions. The findings of this study indicate that the combination of SVM and PCD could be used as a useful tool to optimize the operation strategy of cavitation-facilitated pHIFU therapy.

Early differentiating between the chemotherapy responders and nonresponders: preliminary results with ultrasonic spectrum analysis of the RF time series in preclinical breast cancer models

Background

This study was aimed to assess whether ultrasonic spectrum analysis of radiofrequency (RF) time series using a clinical ultrasound system allows for early differentiating between the chemotherapy responders and nonresponders in human breast cancer xenografts that imitate clinical responding and nonresponding tumors.

Methods

Clinically responding (n = 20; MCF-7) and nonresponding (n = 20; MBA-MD-231) breast cancer xenografts were established in 40 nude mice. Ten mice from each group received either chemotherapy (adriamycin, 4 mg/kg) or saline as controls. Each tumor was imaged longitudinally with a clinical ultrasound scanner at baseline (day 0) and subsequently on days 2, 4, 6, 8 and 12 following treatment, and the corresponding RF time-series data were collected. Changes in six RF time-series parameters (slope, intercept, S1, S2, S3 and S4) were compared with the measurement of the tumor cell density, and their differential performances of the treatment response were analyzed.

Results

Adriamycin significantly inhibited tumor growth and decreased the cancer cell density in responders (P < 0.001) but not in nonresponders (P > 0.05). Fold changes of slope were significantly increased in responders two days after adriamycin treatment (P = 0.002), but not in nonresponders (P > 0.05). Early changes in slope on day 2 could differentiate the treatment response in 100% of both responders (95% CI, 62.9–100.0%) and nonresponders (95% CI, 88.4–100%).

Conclusions

Ultrasonic RF time series allowed for the monitoring of the tumor response to chemotherapy and could further serve as biomarkers for early differentiating between the treatment responders and nonresponders.

Electronic supplementary material

The online version of this article (10.1186/s40644-019-0248-y) contains supplementary material, which is available to authorized users.

Artificial intelligence in breast ultrasound

Artificial intelligence (AI) is gaining extensive attention for its excellent performance in image-recognition tasks and increasingly applied in breast ultrasound. AI can conduct a quantitative assessment by recognizing imaging information automatically and make more accurate and reproductive imaging diagnosis. Breast cancer is the most commonly diagnosed cancer in women, severely threatening women’s health, the early screening of which is closely related to the prognosis of patients. Therefore, utilization of AI in breast cancer screening and detection is of great significance, which can not only save time for radiologists, but also make up for experience and skill deficiency on some beginners. This article illustrates the basic technical knowledge regarding AI in breast ultrasound, including early machine learning algorithms and deep learning algorithms, and their application in the differential diagnosis of benign and malignant masses. At last, we talk about the future perspectives of AI in breast ultrasound.

Assessment of Rigid Registration Quality Measures in Ultrasound-Guided Radiotherapy

Image guidance has become the standard of care for patient positioning in radiotherapy, where image registration is often a critical step to help manage patient motion. However, in practice, verification of registration quality is often adversely affected by difficulty in manual inspection of 3-D images and time constraint, thus affecting the therapeutic outcome. Therefore, we proposed to employ both bootstrapping and the supervised learning methods of linear discriminant analysis and random forest to help robustly assess registration quality in ultrasound-guided radiotherapy. We validated both approaches using phantom and real clinical ultrasound images, and showed that both performed well for the task. While learning-based techniques offer better accuracy and shorter evaluation time, bootstrapping requires no prior training and has a higher sensitivity.

Classification of breast cancer in ultrasound imaging using a generic deep learning analysis software: a pilot study

OBJECTIVE

To train a generic deep learning software (DLS) to classify breast cancer on ultrasound images and to compare its performance to human readers with variable breast imaging experience.

METHODS

In this retrospective study, all breast ultrasound examinations from January 1, 2014 to December 31, 2014 at our institution were reviewed. Patients with post-surgical scars, initially indeterminate, or malignant lesions with histological diagnoses or 2-year follow-up were included. The DLS was trained with 70% of the images, and the remaining 30% were used to validate the performance. Three readers with variable expertise also evaluated the validation set (radiologist, resident, medical student). Diagnostic accuracy was assessed with a receiver operating characteristic analysis.

RESULTS

82 patients with malignant and 550 with benign lesions were included. Time needed for training was 7 min (DLS). Evaluation time for the test data set were 3.7 s (DLS) and 28, 22 and 25 min for human readers (decreasing experience). Receiver operating characteristic analysis revealed non-significant differences (p-values 0.45-0.47) in the area under the curve of 0.84 (DLS), 0.88 (experienced and intermediate readers) and 0.79 (inexperienced reader).

CONCLUSION

DLS may aid diagnosing cancer on breast ultrasound images with an accuracy comparable to radiologists, and learns better and faster than a human reader with no prior experience. Further clinical trials with dedicated algorithms are warranted. Advances in knowledge: DLS can be trained classify cancer on breast ultrasound images high accuracy even with comparably few training cases. The fast evaluation speed makes real-time image analysis feasible.

Ultrasonic RF time series for early assessment of the tumor response to chemotherapy

Ultrasound radio-frequency (RF) time series have been shown to carry tissue typing information. To evaluate the potential of RF time series for early prediction of tumor response to chemotherapy, 50MCF-7 breast cancer-bearing nude mice were randomized to receive cisplatin and paclitaxel (treatment group; n = 26) or sterile saline (control group; n = 24). Sequential ultrasound imaging was performed on days 0, 3, 6, and 8 of treatment to simultaneously collect B-mode images and RF data. Six RF time series features, slope, intercept, S1, S2, S3, and S4, were extracted during RF data analysis and contrasted with microstructural tumor changes on histopathology. Chemotherapy administration reduced tumor growth relative to control on days 6 and 8. Compared with day 0, intercept, S1, and S2 were increased while slope was decreased on days 3, 6, and 8 in the treatment group. Compared with the control group, intercept, S1, S2, S3, and S4 were increased, and slope was decreased, on days 3, 6, and 8 in the treatment group. Tumor cell density decreased significantly in the latter on day 3. We conclude that ultrasonic RF time series analysis provides a simple way to noninvasively assess the early tumor response to chemotherapy.

A nonlinear approach to identify pathological change of thyroid nodules based on statistical analysis of ultrasound RF signals

In order to reassure the majority of patients with benign nodules from unnecessary needle biopsy, there is an increasing clinical requirement to identify benign and malignant thyroid nodules during ultrasound diagnosis. A nonlinear approach based on statistical analysis of ultrasound radio-frequency (RF) signals was developed for differential diagnosing the thyroid nodules to improve the diagnostic accuracy. Data from 44 patients with solitary thyroid nodules were collected, following with the ultrasound-guided fine needle aspiration (FNA) as the ground truth. The relative P-value (rP-value) was estimated to quantify the pathophysiologic changes by comparing the region of interest (ROI) with the no pathological change part in the thyroid gland using only one frame of raw RF data. The malignant nodules were distinguished from benign ones with high accuracy and high credibility (sensitivity = 100%, specificity = 80%). Suspicious nodules (rP-value < 0.5) could be picked out for FNA with no additional instruments. This method shows promising in differentiating malignant from benign thyroid nodules, subsequently avoiding unnecessary biopsies.

Dual-modal computer-assisted evaluation of axillary lymph node metastasis in breast cancer patients on both real-time elastography and B-mode ultrasound

PURPOSE

To propose a computer-assisted method for quantifying the hardness of an axillary lymph node on real-time elastography (RTE) and its morphology on B-mode ultrasound; and to combine the dual-modal features for differentiation of metastatic and benign axillary lymph nodes in breast cancer patients.

MATERIALS AND METHODS

A total of 161 axillary lymph nodes (benign, n=69; metastatic, n=92) from 158 patients with breast cancer were examined with both B-mode ultrasound and RTE. With computer assistance, five morphological features describing the hilum, size, shape, and echogenic uniformity of a lymph node were extracted from B-mode, and three hardness features were extracted from RTE. Single-modal and dual-modal features were used to classify benign and metastatic nodes with two computerized classification approaches, i.e., a scoring approach and a support vector machine (SVM) approach. The computerized approaches were also compared with a visual evaluation approach.

RESULTS

All features exhibited significant differences between benign and metastatic nodes (p<0.001), with the highest area under the receiver operating characteristic curve (AUC) of 0.803 and the highest accuracy (ACC) of 75.2% for a single feature. The SVM on dual-modal features achieved the largest AUC (0.895) and ACC (85.7%) among all methods, exceeding the scoring (AUC=0.881; ACC=83.6%) and the visual evaluation methods (AUC=0.830; ACC=84.5%). With the leave-one-out cross validation, the SVM on dual-modal features still obtained an ACC as high as 84.5%.

CONCLUSION

Dual-modal features can be extracted from RTE and B-mode ultrasound with computer assistance, which are valuable for discrimination between benign and metastatic lymph nodes. The SVM on dual-modal features outperforms the scoring and visual evaluation methods, as well as all methods using single-modal features. The computer-assisted dual-modal evaluation of lymph nodes could be potentially used in daily clinical practice for assessing axillary metastasis in breast cancer patients.

Evaluating pathologic response of breast cancer to neoadjuvant chemotherapy with computer-extracted features from contrast-enhanced ultrasound videos

PURPOSE

To extract quantitative perfusion and texture features with computer assistance from contrast-enhanced ultrasound (CEUS) videos of breast cancer before and after neoadjuvant chemotherapy (NAC), and to evaluate pathologic response to NAC with these features.

METHODS

Forty-two CEUS videos with 140,484 images were acquired from 21 breast cancer patients pre- and post-NAC. Time-intensity curve (TIC) features were calculated including the difference between area under TIC within a tumor and that within a computer-detected reference region (AUT_T-R). Four texture features were extracted including Homogeneity and Contrast. All patients were identified as pathologic responders by Miller and Payne criteria. The features between pre- and post-treatment in these responders were statistically compared, and the discrimination between pre- and post-treatment cancers was assessed with a receiver operating characteristic (ROC) curve.

RESULTS

Compared with the pre-treatment cancers, the post-treatment cancers had significantly lower Homogeneity (p<0.001) and AUT_T-R (p=0.014), as well as higher Contrast (p<0.001), indicating the intratumoral contrast enhancement decreased and became more heterogeneous after NAC in responders. The combination of Homogeneity and AUT_T-R achieved an accuracy of 90.5% and area under ROC curve of 0.946 for discrimination between pre- and post-chemotherapy cancers without cross validation. The accuracy still reached as high as 85.7% under leave-one-out cross validation.

CONCLUSIONS

The computer-extracted CEUS features show reduced and more heterogeneous neovascularization of cancer after NAC. The features achieve high accuracy for discriminating between pre- and post-chemotherapy cancers in responders and thus are potentially valuable for tumor response evaluation in clinical practice.

Sonoelastomics for Breast Tumor Classification: A Radiomics Approach with Clustering-Based Feature Selection on Sonoelastography

A radiomics approach to sonoelastography, called "sonoelastomics," is proposed for classification of benign and malignant breast tumors. From sonoelastograms of breast tumors, a high-throughput 364-dimensional feature set was calculated consisting of shape features, intensity statistics, gray-level co-occurrence matrix texture features and contourlet texture features, which quantified the shape, hardness and hardness heterogeneity of a tumor. The high-throughput features were then selected for feature reduction using hierarchical clustering and three-feature selection metrics. For a data set containing 42 malignant and 75 benign tumors from 117 patients, seven selected sonoelastomic features achieved an area under the receiver operating characteristic curve of 0.917, an accuracy of 88.0%, a sensitivity of 85.7% and a specificity of 89.3% in a validation set via the leave-one-out cross-validation, revealing superiority over the principal component analysis, deep polynomial networks and manually selected features. The sonoelastomic features are valuable in breast tumor differentiation.

A Fusion-Based Approach for Breast Ultrasound Image Classification Using Multiple-ROI Texture and Morphological Analyses

Ultrasound imaging is commonly used for breast cancer diagnosis, but accurate interpretation of breast ultrasound (BUS) images is often challenging and operator-dependent. Computer-aided diagnosis (CAD) systems can be employed to provide the radiologists with a second opinion to improve the diagnosis accuracy. In this study, a new CAD system is developed to enable accurate BUS image classification. In particular, an improved texture analysis is introduced, in which the tumor is divided into a set of nonoverlapping regions of interest (ROIs). Each ROI is analyzed using gray-level cooccurrence matrix features and a support vector machine classifier to estimate its tumor class indicator. The tumor class indicators of all ROIs are combined using a voting mechanism to estimate the tumor class. In addition, morphological analysis is employed to classify the tumor. A probabilistic approach is used to fuse the classification results of the multiple-ROI texture analysis and morphological analysis. The proposed approach is applied to classify 110 BUS images that include 64 benign and 46 malignant tumors. The accuracy, specificity, and sensitivity obtained using the proposed approach are 98.2%, 98.4%, and 97.8%, respectively. These results demonstrate that the proposed approach can effectively be used to differentiate benign and malignant tumors.

Deep learning based classification of breast tumors with shear-wave elastography

This study aims to build a deep learning (DL) architecture for automated extraction of learned-from-data image features from the shear-wave elastography (SWE), and to evaluate the DL architecture in differentiation between benign and malignant breast tumors. We construct a two-layer DL architecture for SWE feature extraction, comprised of the point-wise gated Boltzmann machine (PGBM) and the restricted Boltzmann machine (RBM). The PGBM contains task-relevant and task-irrelevant hidden units, and the task-relevant units are connected to the RBM. Experimental evaluation was performed with five-fold cross validation on a set of 227 SWE images, 135 of benign tumors and 92 of malignant tumors, from 121 patients. The features learned with our DL architecture were compared with the statistical features quantifying image intensity and texture. Results showed that the DL features achieved better classification performance with an accuracy of 93.4%, a sensitivity of 88.6%, a specificity of 97.1%, and an area under the receiver operating characteristic curve of 0.947. The DL-based method integrates feature learning with feature selection on SWE. It may be potentially used in clinical computer-aided diagnosis of breast cancer.