An integrated system for the segmentation of atherosclerotic carotid plaque ultrasound video

Assessing the impact of ultrasound image standardization in deep learning-based segmentation of carotid plaque types

BACKGROUND AND OBJECTIVE

Carotid B-mode ultrasound (CBUS) imaging is often used to detect and assess atherosclerotic plaques. Doctors often need to segment plaques in the CBUS images to further examine them. Multiple studies have proposed two-dimensional CBUS plaque segmentation deep learning (DL)-based solutions, achieving promising results. In most of these studies, image standardization is not reported, while not all plaque types are represented. However, prior multiple studies have highlighted the importance of data standardization in computerized CBUS plaque classification or segmentation solutions. In this study, we propose and separately evaluate three progressive preprocessing schemes, to discover the most optimal to standardize CBUS images for DL-based carotid plaque segmentation, while we also assess the effect of each preprocessing in the segmentation performance per echodensity-based plaque type (I, II, III, IV and V).

METHODS

We included three CBUS image datasets (276 CBUS images, from three medical centres), with which we produced 3 data folds (with the best possible equal inclusion of images from all centers per fold), to perform 3-fold cross validation-based training and evaluation of the pre-released Channel-wise Feature Pyramid Network for Medicine (CFPNet-M) model, in carotid plaque type segmentation. We included the three data folds in their original version (O), generating also three preprocessed versions of them, namely, the resolution-normalized (R), the resolution- and intensity-normalized (RN), and the resolution- and intensity-normalized combined with despeckling (RND) versions. The samples were cropped to the plaque level, and the intersection over union (IoU) and the Dice Similarity Coefficient (DSC), along with other metrics, were used to measure the model's performance. In each training round, 12 % of the images in the 2 training folds was used for internal validation (last fold was used in evaluation). Two experienced ultrasonographers manually delineated plaques in the dataset, to provide us with ground truths, while the plaque types (I to V) were extracted according to the Gray-Weale and Geroulakos classification system. We measured the mean±standard deviation of DSC within and across the three evaluated folds, per preprocessing scheme and per plaque type.

RESULTS

CFPNet-M segmented the plaques in the CBUS images in all the data preprocessing versions, yielding progressively improved performances (mean DSC at 81.9 ± 9.1 %, 83.6 ± 9.0 %, 84.1 ± 8.3 %, and 84.4 ± 8.1 % for the O, R, RN and RND 3-fold cross validation processes, respectively), irrespective of the plaque type. Interestingly, CFPNet_M yielded improved performances, for all plaque types (I, II, III, IV and V), when trained and tested with the RND data versus the O version, achieving an 80.6 ± 11 % versus 77.6 ± 17 % DSC for type I, an 84.3 ± 8 % versus 81.2 ± 9 % DSC for type II, an 84.9 ± 7 % versus 82.6 ± 7 % for type III, an 85.3 ± 8 % versus 83.9 ± 7 % for type IV, and a 84.8 ± 8 % versus 81.8 ± 2 % for type V. The best increase in DSC, from the O to the RND CBUS images, was found for the plaque type I (3.86 % increase), with types II and V, following.

CONCLUSIONS

In this study, we investigated the impact of CBUS standardization in DL-based carotid plaque type segmentation and showed that indeed normalization of the image resolution and intensity, combined with speckle noise removal, prior to model training and testing, enhances the DL model's performance, across all plaque types. Based on the findings in this study, CBUS images should be standardized when destined for DL-based segmentation tasks, while all plaque types should be considered, as in a plethora of existing relevant studies, uniformly echolucent plaques or heavily calcified plaques with acoustic shadow are notably underrepresented.

Cardiovascular Disease Risk Stratification Using Hybrid Deep Learning Paradigm: First of Its Kind on Canadian Trial Data

BACKGROUND

The risk of cardiovascular disease (CVD) has traditionally been predicted via the assessment of carotid plaques. In the proposed study, AtheroEdge™ 3.0HDL (AtheroPoint™, Roseville, CA, USA) was designed to demonstrate how well the features obtained from carotid plaques determine the risk of CVD. We hypothesize that hybrid deep learning (HDL) will outperform unidirectional deep learning, bidirectional deep learning, and machine learning (ML) paradigms.

METHODOLOGY

500 people who had undergone targeted carotid B-mode ultrasonography and coronary angiography were included in the proposed study. ML feature selection was carried out using three different methods, namely principal component analysis (PCA) pooling, the chi-square test (CST), and the random forest regression (RFR) test. The unidirectional and bidirectional deep learning models were trained, and then six types of novel HDL-based models were designed for CVD risk stratification. The AtheroEdge™ 3.0HDL was scientifically validated using seen and unseen datasets while the reliability and statistical tests were conducted using CST along with p-value significance. The performance of AtheroEdge™ 3.0HDL was evaluated by measuring the p-value and area-under-the-curve for both seen and unseen data.

RESULTS

The HDL system showed an improvement of 30.20% (0.954 vs. 0.702) over the ML system using the seen datasets. The ML feature extraction analysis showed 70% of common features among all three methods. The generalization of AtheroEdge™ 3.0HDL showed less than 1% (p-value < 0.001) difference between seen and unseen data, complying with regulatory standards.

CONCLUSIONS

The hypothesis for AtheroEdge™ 3.0HDL was scientifically validated, and the model was tested for reliability and stability and is further adaptable clinically.

Ultrasound carotid plaque segmentation via image reconstruction-based self-supervised learning with limited training labels

Carotid total plaque area (TPA) is an important contributing measurement to the evaluation of stroke risk. Deep learning provides an efficient method for ultrasound carotid plaque segmentation and TPA quantification. However, high performance of deep learning requires datasets with many labeled images for training, which is very labor-intensive. Thus, we propose an image reconstruction-based self-supervised learning algorithm (IR-SSL) for carotid plaque segmentation when few labeled images are available. IR-SSL consists of pre-trained and downstream segmentation tasks. The pre-trained task learns region-wise representations with local consistency by reconstructing plaque images from randomly partitioned and disordered images. The pre-trained model is then transferred to the segmentation network as the initial parameters in the downstream task. IR-SSL was implemented with two networks, UNet++ and U-Net, and evaluated on two independent datasets of 510 carotid ultrasound images from 144 subjects at SPARC (London, Canada) and 638 images from 479 subjects at Zhongnan hospital (Wuhan, China). Compared to the baseline networks, IR-SSL improved the segmentation performance when trained on few labeled images (n = 10, 30, 50 and 100 subjects). For 44 SPARC subjects, IR-SSL yielded Dice-similarity-coefficients (DSC) of 80.14-88.84%, and algorithm TPAs were strongly correlated (r=0.962-0.993, p < 0.001) with manual results. The models trained on the SPARC images but applied to the Zhongnan dataset without retraining achieved DSCs of 80.61-88.18% and strong correlation with manual segmentation (r=0.852-0.978, p < 0.001). These results suggest that IR-SSL could improve deep learning when trained on small labeled datasets, making it useful for monitoring carotid plaque progression/regression in clinical use and trials.

Unsupervised domain adaptation method for segmenting cross-sectional CCA images

BACKGROUND AND OBJECTIVES

Automatic vessel segmentation in ultrasound is challenging due to the quality of the ultrasound images, which is affected by attenuation, high level of speckle noise and acoustic shadowing. Recently, deep convolutional neural networks are increasing in popularity due to their great performance on image segmentation problems, including vessel segmentation. Traditionally, large labeled datasets are required to train a network that achieves high performance, and is able to generalize well to different orientations, transducers and ultrasound scanners. However, these large datasets are rare, given that it is challenging and time-consuming to acquire and manually annotate in-vivo data.

METHODS

In this work, we present a model-based, unsupervised domain adaptation method that consists of two stages. In the first stage, the network is trained on simulated ultrasound images, which have an accurate ground truth. In the second stage, the network continues training on in-vivo data in an unsupervised way, therefore not requiring the data to be labelled. Rather than using an adversarial neural network, prior knowledge on the elliptical shape of the segmentation mask is used to detect unexpected outputs.

RESULTS

The segmentation performance was quantified using manually segmented images as ground truth. Due to the proposed domain adaptation method, the median Dice similarity coefficient increased from 0 to 0.951, outperforming a domain adversarial neural network (median Dice 0.922) and a state-of-the-art Star-Kalman algorithm that was specifically designed for this dataset (median Dice 0.942).

CONCLUSIONS

The results show that it is feasible to first train a neural network on simulated data, and then apply model-based domain adaptation to further improve segmentation performance by training on unlabeled in-vivo data. This overcomes the limitation of conventional deep learning approaches to require large amounts of manually labeled in-vivo data. Since the proposed domain adaptation method only requires prior knowledge on the shape of the segmentation mask, performance can be explored in various domains and applications in future research.

Interplay between Artificial Intelligence and Biomechanics Modeling in the Cardiovascular Disease Prediction

Cardiovascular disease (CVD) is the most common cause of morbidity and mortality worldwide, and early accurate diagnosis is the key point for improving and optimizing the prognosis of CVD. Recent progress in artificial intelligence (AI), especially machine learning (ML) technology, makes it possible to predict CVD. In this review, we first briefly introduced the overview development of artificial intelligence. Then we summarized some ML applications in cardiovascular diseases, including ML-based models to directly predict CVD based on risk factors or medical imaging findings and the ML-based hemodynamics with vascular geometries, equations, and methods for indirect assessment of CVD. We also discussed case studies where ML could be used as the surrogate for computational fluid dynamics in data-driven models and physics-driven models. ML models could be a surrogate for computational fluid dynamics, accelerate the process of disease prediction, and reduce manual intervention. Lastly, we briefly summarized the research difficulties and prospected the future development of AI technology in cardiovascular diseases.

FRDD-Net: Automated Carotid Plaque Ultrasound Images Segmentation Using Feature Remapping and Dense Decoding

Automated segmentation and evaluation of carotid plaques ultrasound images is of great significance for the diagnosis and early intervention of high-risk groups of cardiovascular and cerebrovascular diseases. However, it remains challenging to develop such solutions due to the relatively low quality of ultrasound images and heterogenous characteristics of carotid plaques. To address those problems, in this paper, we propose a novel deep convolutional neural network, FRDD-Net, with an encoder–decoder architecture to automatically segment carotid plaques. We propose the feature remapping modules (FRMs) and incorporate them into the encoding and decoding blocks to ameliorate the reliability of acquired features. We also propose a new dense decoding mechanism as part of the decoder, thus promoting the utilization efficiency of encoded features. Additionally, we construct a compound loss function to train our network to further enhance its robustness in the face of numerous cases. We train and test our network in multiple carotid plaque ultrasound datasets and our method yields the best performance compared to other state-of-the-art methods. Further ablation studies consistently show the advancement of our proposed architecture.

Deep Learning-Based Carotid Plaque Segmentation from B-Mode Ultrasound Images

Carotid ultrasound measurement of total plaque area (TPA) provides a method for quantifying carotid plaque burden and monitoring changes in carotid atherosclerosis in response to medical treatment. Plaque boundary segmentation is required to generate the TPA measurement; however, training of observers and manual delineation are time consuming. Thus, our objective was to develop an automated plaque segmentation method to generate TPA from longitudinal carotid ultrasound images. In this study, a deep learning-based method, modified U-Net, was used to train the segmentation model and generate TPA measurement. A total of 510 plaques from 144 patients were used in our study, where the Monte Carlo cross-validation was used by randomly splitting the data set into 2/3 and 1/3 for training and testing. Two observers were trained to manually delineate the 510 plaques separately, which were used as the ground-truth references. Two U-Net models (M1 and M2) were trained using the two different ground-truth data sets from the two observers to evaluate the accuracy, variability and sensitivity on the ground-truth data sets used for training our method. The results of the algorithm segmentations of the two models yielded strong agreement with the two manual segmentations with the Pearson correlation coefficient r = 0.989 (p < 0.0001) and r = 0.987 (p < 0.0001). Comparison of the U-Net and manual segmentations resulted in mean TPA differences of 0.05 ± 7.13 mm² (95% confidence interval: 14.02-13.02 mm²) and 0.8 ± 8.7 mm² (17.85-16.25 mm²) for the two models, which are small compared with the TPA range in our data set from 4.7 to 312.8 mm². Furthermore, the mean time to segment a plaque was only 8.3 ± 3.1 ms. The presented deep learning-based method described has sufficient accuracy with a short computation time and exhibits high agreement between the algorithm and manual TPA measurements, suggesting that the method could be used to measure TPA and to monitor the progression and regression of carotid atherosclerosis.

Deep Learning-Based Measurement of Total Plaque Area in B-Mode Ultrasound Images

Measurement of total-plaque-area (TPA) is important for determining long term risk for stroke and monitoring carotid plaque progression. Since delineation of carotid plaques is required, a deep learning method can provide automatic plaque segmentations and TPA measurements; however, it requires large datasets and manual annotations for training with unknown performance on new datasets. A UNet++ ensemble algorithm was proposed to segment plaques from 2D carotid ultrasound images, trained on three small datasets (n = 33, 33, 34 subjects) and tested on 44 subjects from the SPARC dataset (n = 144, London, Canada). The ensemble was also trained on the entire SPARC dataset and tested with a different dataset (n = 497, Zhongnan Hospital, China). Algorithm and manual segmentations were compared using Dice-similarity-coefficient (DSC), and TPAs were compared using the difference ( ∆TPA), Pearson correlation coefficient (r) and Bland-Altman analyses. Segmentation variability was determined using the intra-class correlation coefficient (ICC) and coefficient-of-variation (CoV). For 44 SPARC subjects, algorithm DSC was 83.3-85.7%, and algorithm TPAs were strongly correlated (r = 0.985-0.988; p < 0.001) with manual results with marginal biases (0.73-6.75) mm ² using the three training datasets. Algorithm ICC for TPAs (ICC = 0.996) was similar to intra- and inter-observer manual results (ICC = 0.977, 0.995). Algorithm CoV = 6.98% for plaque areas was smaller than the inter-observer manual CoV (7.54%). For the Zhongnan dataset, DSC was 88.6% algorithm and manual TPAs were strongly correlated (r = 0.972, p < 0.001) with ∆TPA = -0.44 ±4.05 mm ² and ICC = 0.985. The proposed algorithm trained on small datasets and segmented a different dataset without retraining with accuracy and precision that may be useful clinically and for research.

Ultrasound diaphragmatic manual and semi-automated motion measurements: Application in simulated and in vivo data of critically ill subjects

BACKGROUND AND OBJECTIVE

Ultrasound diaphragmatic muscle motion characteristics may provide useful information about normal or abnormal diaphragmatic function and indicate diaphragmatic weakness, or paralysis. In the present work we propose and evaluate an integrated semi-automated analysis system for the quantitative analysis of ultrasonic motion from ultrasound diaphragmatic videos.

METHODS

The proposed system was evaluated in simulated videos and in 13 patients, four of whom patients were mechanically ventilated. The major steps of the methodology were as follows: video normalization, despeckle filtering, generation of an M-Mode image, snakes segmentation, and motion measurements.

RESULTS

The following manual (-/) vs semi-automated (/-), (median±IQR) measurements, which are routinely carried out by the experts, for assessing the severity of the disease, were computed. For the simulated videos the diaphragmatic excursion was 1.80±0.00 cm / 1.76±0.03 cm. For all the real ultrasound videos investigated in this study the following measurements were computed: (i) diaphragmatic excursion: 0.84±0.15 cm / 0.83±0.14 cm, (ii) inspiration time (T_insp): 0.71±0.18 sec / 0.70±0.15 sec, (iii) total breathing time for one cycle (T_tot): 1.71±0.37 sec / 1.67±0.37 sec, (iv) diaphragmatic curve slope: 1.29±0.36 cm/sec / 1.27±0.36 cm/sec, and (v) relaxation rate (RR): 0.82±0.17 cm/sec / 0.82±0.18 cm/sec.

CONCLUSIONS

Manual and semi-automated measurements were very close with non-statistical significant differences and strong correlations between them. It is anticipated that the proposed system might be useful in the clinical practice in the assessment and follow up of patients with diaphragmatic weakness or paralysis and aid in the separation of normal and abnormal diaphragmatic motion. Further validation and additional experimentation in a larger sample of videos and different patient groups is required.

Research on Sensitivity of Speckle Center Coordinate Values by Contour and Background Noise and Elimination Method

The accuracy of measuring the target object displacement is greatly influenced by the offset of central coordinate value in a laser speckle contour (CCVLSC) due to the defects obtained in background or on measuring surface, as a measuring combination of both monocular vision and laser speckle is used. In this paper, the theoretical principle of displacement measurement is first presented by a combination of monocular vision and laser speckle. Then, a model between object displacement and CCVLSC (particularly, [Formula: see text] coordinate value) is derived. Finally, a denoising algorithm with competitive protection of contour effective points is proposed, on the basis of effects of noises coming from background and contour edge on CCVLSC. The algorithm includes ellipse fitting to laser speckle contour, calculating offsets between all contour points and the fitted eclipse, eliminating noise points with higher deviation (generally about 5% of all contour points) by using competitive strategy, ellipse refitting, and recalculating and re-eliminating until the deviation is below a specified threshold. It is shown that the algorithm can not only eliminate the fixed noise points in each round but also protect the number of effective points to the greatest extent. Finally, the feasibility of the algorithm is verified by two ways. One is an ideal data validation. It proves that the algorithm can guarantee the convergence towards the ideal center coordinate value. Another is an experimental verification. An experimental system is built up based on the relationship between object displacement and Y coordinate value of CCVLSC for obtaining relevant dada. It is shown by the comparison between predictions and experimental data that the algorithm has a better robustness and a higher accuracy of distance measurement than other typical algorithms.

Texture Feature Variability in Ultrasound Video of the Atherosclerotic Carotid Plaque

The objective of this paper was to investigate texture feature variability in ultrasound video of the carotid artery during the cardiac cycle in an attempt to define new discriminatory biomarkers of the vulnerable plaque. More specifically, in this paper, 120 longitudinal ultrasound videos, acquired from 40 normal (N) subjects from the common carotid artery and 40 asymptomatic (A) and 40 symptomatic (S) subjects from the proximal internal carotid artery were investigated. The videos were intensity normalized and despeckled, and the intima-media complex (IMC) (from the N subjects) and the atherosclerotic carotid plaques (from the A and S subjects) were segmented from each video, in order to extract the M-mode image, and the texture features associated with cardiac states of systole and diastole. The main results of this paper can be summarized as follows: 1) texture features varied significantly throughout the cardiac cycle with significant differences identified between the cardiac systolic and cardiac diastolic states; 2) gray scale median was significantly higher at cardiac systole versus diastole for the N, A, and S groups investigated; 3) plaque texture features extracted during the cardiac cycle at the systolic and diastolic states were statistically significantly different between A and S subjects (and can thus be used to discriminate between A and S subjects successfully). The combination of systolic and diastolic features yields better performance than those alone. It is anticipated that the proposed system may aid the physician in clinical practice in classifying between N, A, and S subjects using texture features extracted from ultrasound videos of IMC and carotid artery plaque. However, further evaluation has to be carried out with more videos and additional features.

Deep learning based classification of breast tumors with shear-wave elastography

This study aims to build a deep learning (DL) architecture for automated extraction of learned-from-data image features from the shear-wave elastography (SWE), and to evaluate the DL architecture in differentiation between benign and malignant breast tumors. We construct a two-layer DL architecture for SWE feature extraction, comprised of the point-wise gated Boltzmann machine (PGBM) and the restricted Boltzmann machine (RBM). The PGBM contains task-relevant and task-irrelevant hidden units, and the task-relevant units are connected to the RBM. Experimental evaluation was performed with five-fold cross validation on a set of 227 SWE images, 135 of benign tumors and 92 of malignant tumors, from 121 patients. The features learned with our DL architecture were compared with the statistical features quantifying image intensity and texture. Results showed that the DL features achieved better classification performance with an accuracy of 93.4%, a sensitivity of 88.6%, a specificity of 97.1%, and an area under the receiver operating characteristic curve of 0.947. The DL-based method integrates feature learning with feature selection on SWE. It may be potentially used in clinical computer-aided diagnosis of breast cancer.

A Review on Carotid Ultrasound Atherosclerotic Tissue Characterization and Stroke Risk Stratification in Machine Learning Framework

Cardiovascular diseases (including stroke and heart attack) are identified as the leading cause of death in today's world. However, very little is understood about the arterial mechanics of plaque buildup, arterial fibrous cap rupture, and the role of abnormalities of the vasa vasorum. Recently, ultrasonic echogenicity characteristics and morphological characterization of carotid plaque types have been shown to have clinical utility in classification of stroke risks. Furthermore, this characterization supports aggressive and intensive medical therapy as well as procedures, including endarterectomy and stenting. This is the first state-of-the-art review to provide a comprehensive understanding of the field of ultrasonic vascular morphology tissue characterization. This paper presents fundamental and advanced ultrasonic tissue characterization and feature extraction methods for analyzing plaque. Additionally, the paper shows how the risk stratification is achieved using machine learning paradigms. More advanced methods need to be developed which can segment the carotid artery walls into multiple regions such as the bulb region and areas both proximal and distal to the bulb. Furthermore, multimodality imaging is needed for validation of such advanced methods for stroke and cardiovascular risk stratification.

Measurement of ultrasonic diaphragmatic motion

The motion characteristics of the diaphragmatic muscle may provide useful information about normal and abnormal diaphragmatic function and indicate diaphragmatic weakness. The objective of this paper was to introduce a simple system for the quantitative analysis of ultrasonic diaphragmatic motion. The measurements routinely carried out by the experts were computed and these include: (i) excursion, (ii) inspiration time (Tinsp) and (iii) cycle duration (Ttot). The system was evaluated on four simulated videos and one real video. Manual and automated measurements were very close. Further work in a larger number of videos is needed for validating the proposed method.

Despeckle Filtering for Multiscale Amplitude-Modulation Frequency-Modulation (AM-FM) Texture Analysis of Ultrasound Images of the Intima-Media Complex

The intima-media thickness (IMT) of the common carotid artery (CCA) is widely used as an early indicator of cardiovascular disease (CVD). Typically, the IMT grows with age and this is used as a sign of increased risk of CVD. Beyond thickness, there is also clinical interest in identifying how the composition and texture of the intima-media complex (IMC) changed and how these textural changes grow into atherosclerotic plaques that can cause stroke. Clearly though texture analysis of ultrasound images can be greatly affected by speckle noise, our goal here is to develop effective despeckle noise methods that can recover image texture associated with increased rates of atherosclerosis disease. In this study, we perform a comparative evaluation of several despeckle filtering methods, on 100 ultrasound images of the CCA, based on the extracted multiscale Amplitude-Modulation Frequency-Modulation (AM-FM) texture features and visual image quality assessment by two clinical experts. Texture features were extracted from the automatically segmented IMC for three different age groups. The despeckle filters hybrid median and the homogeneous mask area filter showed the best performance by improving the class separation between the three age groups and also yielded significantly improved image quality.