Fully Automatic Myocardial Segmentation of Contrast Echocardiography Sequence Using Random Forests Guided by Shape Model

Structural uncertainty estimation for medical image segmentation

Precise segmentation and uncertainty estimation are crucial for error identification and correction in medical diagnostic assistance. Existing methods mainly rely on pixel-wise uncertainty estimations. They (1) neglect the global context, leading to erroneous uncertainty indications, and (2) bring attention interference, resulting in the waste of extensive details and potential understanding confusion. In this paper, we propose a novel structural uncertainty estimation method, based on Convolutional Neural Networks (CNN) and Active Shape Models (ASM), named SU-ASM, which incorporates global shape information for providing precise segmentation and uncertainty estimation. The SU-ASM consists of three components. Firstly, multi-task generation provides multiple outcomes to assist ASM initialization and shape optimization via a multi-task learning module. Secondly, information fusion involves the creation of a Combined Boundary Probability (CBP) and along with a rapid shape initialization algorithm, Key Landmark Template Matching (KLTM), to enhance boundary reliability and select appropriate shape templates. Finally, shape model fitting where multiple shape templates are matched to the CBP while maintaining their intrinsic shape characteristics. Fitted shapes generate segmentation results and structural uncertainty estimations. The SU-ASM has been validated on cardiac ultrasound dataset, ciliary muscle dataset of the anterior eye segment, and the chest X-ray dataset. It outperforms state-of-the-art methods in terms of segmentation and uncertainty estimation.

CMR-BENet: A confidence map refinement boundary enhancement network for left ventricular myocardium segmentation

BACKGROUND AND OBJECTIVE

Left ventricular myocardium segmentation is of great significance for clinical diagnosis, treatment, and prognosis. However, myocardium segmentation is challenging as the medical image quality is disturbed by various factors such as motion, artifacts, and noise. Its accuracy largely depends on the accurate identification of edges and structures. Most existing encoder-decoder based segmentation methods capture limited contextual information and ignore the awareness of myocardial shape and structure, often producing unsatisfactory boundary segmentation results in noisy scenes. Moreover, these methods fail to assess the reliability of the predictions, which is crucial for clinical decisions and applications in medical tasks. Therefore, this study explores how to effectively combine contextual information with myocardial edge structure and confidence maps to improve segmentation performance in an end-to-end network.

METHODS

In this paper, we propose an end-to-end confidence map refinement boundary enhancement network (CMR-BENet) for left ventricular myocardium segmentation. CMR-BENet has three components: a layer semantic-aware module (LSA), an edge information enhancement module (EIE), and a confidence map-based refinement module (CMR). Specifically, LSA first adaptively fuses high- and low-level semantic information across hierarchical layers to mitigate the bias of single-layer features affected by noise. EIE then improves the edge and structure recognition by designing the edge and mask guidance module (EMG) and the edge structure-aware module (ESA). Finally, CMR provides a simple and efficient way to estimate confidence maps and effectively combines the encoder features to refine the segmentation results.

RESULTS

Experiments on two echocardiography datasets and one cardiac MRI dataset show that the proposed CMR-BENet outperforms its rivals in the left ventricular myocardium segmentation task with Dice (DI) of 87.71%, 79.33%, and 89.11%, respectively.

CONCLUSION

This paper utilizes edge information to characterize the shape and structure of the myocardium and introduces learnable confidence maps to evaluate and refine the segmentation results. Our findings provide strong support and reference for physicians in diagnosis and treatment.

M4S-Net: a motion-enhanced shape-aware semi-supervised network for echocardiography sequence segmentation

Sequence segmentation of echocardiograms is of great significance for the diagnosis and treatment of cardiovascular diseases. However, the low quality of ultrasound imaging and the complexity of cardiac motion pose great challenges to it. In addition, the difficulty and cost of labeling echocardiography sequences limit the performance of supervised learning methods. In this paper, we proposed a Motion-enhanced Shape-aware Semi-supervised Sequence Segmentation Network named M4S-Net. First, multi-level shape priors are used to enhance the model's shape representation capabilities, overcoming the low image quality and improving single-frame segmentation. Then, a motion-enhanced optimization module utilizes optical flows to assist segmentation in a geometric sense, which robustly responds to the complex motions and ensures the temporal consistency of sequence segmentation. A hybrid loss function is devised to maximize the effectiveness of each module and further improve the temporal stability of predicted masks. Furthermore, the parameter-sharing strategy allows it to perform sequence segmentation in a semi-supervised manner. Massive experiments on both public and in-house datasets show that M4S-Net outperforms the state-of-the-art methods in both spatial and temporal segmentation performance. A downstream apical rocking recognition task based on M4S-Net also achieves an AUC of 0.944, which significantly exceeds specialized physicians.

Topology-sensitive weighting model for myocardial segmentation

Accurate myocardial segmentation is crucial for the diagnosis of various heart diseases. However, segmentation results often suffer from topology structural errors, such as broken connections and holes, especially in cases of poor image quality. These errors are unacceptable in clinical diagnosis. We proposed a Topology-Sensitive Weight (TSW) model to keep both pixel-wise accuracy and topological correctness. Specifically, the Position Weighting Update (PWU) strategy with the Boundary-Sensitive Topology (BST) module can guide the model to focus on positions where topological features are sensitive to pixel values. The Myocardial Integrity Topology (MIT) module can serve as a guide for maintaining myocardial integrity. We evaluate the TSW model on the CAMUS dataset and a private echocardiography myocardial segmentation dataset. The qualitative and quantitative experimental results show that the TSW model significantly enhances topological accuracy while maintaining pixel-wise precision.

Early diagnosis of coronary microvascular dysfunction by myocardial contrast stress echocardiography

Coronary microvascular dysfunction (CMD) is one of the basic mechanisms of myocardial ischemia. Myocardial contrast echocardiography (MCE) is a bedside technique that utilises microbubbles which remain entirely within the intravascular space and denotes the status of microvascular perfusion within that region. Some pilot studies suggested that MCE may be used to diagnose CMD, but without further validation. This study is aimed to investigate the diagnostic performance of MCE for the evaluation of CMD. MCE was performed at rest and during adenosine triphosphate stress. ECG triggered real-time frames were acquired in the apical 4-chamber, 3-chamber, 2-chamber, and long-axis imaging planes. These images were imported into Narnar for further processing. Eighty-two participants with suspicion of coronary disease and absence of significant epicardial lesions were prospectively investigated. Thermodilution was used as the gold standard to diagnose CMD. CMD was present in 23 (28%) patients. Myocardial blood flow reserve (MBF) was assessed using MCE. CMD was defined as MBF reserve < 2. The MCE method had a high sensitivity (88.1%) and specificity (95.7%) in the diagnosis of CMD. There was strong agreement with thermodilution (Kappa coefficient was 0.727; 95% CI: 0.57-0.88, p < 0.001). However, the correlation coefficient (r = 0.376; p < 0.001) was not high.

Semantic segmentation method for myocardial contrast echocardiogram based on DeepLabV3+ deep learning architecture

Myocardial contrast echocardiography (MCE) has been proposed as a method to assess myocardial perfusion for the detection of coronary artery diseases in a non-invasive way. As a critical step of automatic MCE perfusion quantification, myocardium segmentation from the MCE frames faces many challenges due to the low image quality and complex myocardial structure. In this paper, a deep learning semantic segmentation method is proposed based on a modified DeepLabV3+ structure with an atrous convolution and atrous spatial pyramid pooling module. The model was trained separately on three chamber views (apical two-chamber view, apical three-chamber view, and apical four-chamber view) on 100 patients' MCE sequences, divided by a proportion of 7:3 into training and testing datasets. The results evaluated by using the dice coefficient (0.84, 0.84, and 0.86 for three chamber views respectively) and Intersection over Union(0.74, 0.72 and 0.75 for three chamber views respectively) demonstrated the better performance of the proposed method compared to other state-of-the-art methods, including the original DeepLabV3+, PSPnet, and U-net. In addition, we conducted a trade-off comparison between model performance and complexity in different depths of the backbone convolution network, which illustrated model application feasibility.

Assessment and Treatment for Coronary Microvascular Dysfunction by Contrast Enhanced Ultrasound

With growing evidence in clinical practice, the understanding of coronary syndromes has gradually evolved out of focusing on the well-established link between stenosis of epicardial coronary artery and myocardial ischemia to the structural and functional abnormalities at the level of coronary microcirculation, known as coronary microvascular dysfunction (CMD). CMD encompasses several pathophysiological mechanisms of coronary microcirculation and is considered as an important cause of myocardial ischemia in patients with angina symptoms without obstructive coronary artery disease (CAD). As a result of growing knowledge of the understanding of CMD assessed by multiple non-invasive modalities, CMD has also been found to be involved in other cardiovascular diseases, including primary cardiomyopathies as well as heart failure with preserved ejection fraction (HFpEF). In the past 2 decades, almost all the imaging modalities have been used to non-invasively quantify myocardial blood flow (MBF) and promote a better understanding of CMD. Myocardial contrast echocardiography (MCE) is a breakthrough as a non-invasive technique, which enables assessment of myocardial perfusion and quantification of MBF, exhibiting promising diagnostic performances that were comparable to other non-invasive techniques. With unique advantages over other non-invasive techniques, MCE has gradually developed into a novel modality for assessment of the coronary microvasculature, which may provide novel insights into the pathophysiological role of CMD in different clinical conditions. Moreover, the sonothrombolysis and the application of artificial intelligence (AI) will offer the opportunity to extend the use of contrast ultrasound theragnostics.

Optimization algorithm of CT image edge segmentation using improved convolution neural network

To address the problem of high failure rate and low accuracy in computed tomography (CT) image edge segmentation, we proposed a CT sequence image edge segmentation optimization algorithm using improved convolution neural network. Firstly, the pattern clustering algorithm is applied to cluster the pixels with relationship in the CT sequence image space to extract the edge information of the real CT image; secondly, Euclidean distance is used to calculate similarity and measure similarity, according to the measurement results, convolution neural network (CNN) hierarchical optimization is carried out to improve the convergence ability of CNN; finally, the pixel classification of CT sequence images is carried out, and the edge segmentation of CT sequence images is optimized according to the classification results. The results show that the overall recognition rate of this method is at a high level. The training time is obviously reduced when the training times exceed 12 times, the recall rate is always about 90%, and the accuracy of image segmentation is high, which solves the problem of large failure rate and low accuracy.

Model-Agnostic Temporal Regularizer for Object Localization Using Motion Fields

Video analysis often requires locating and tracking target objects. In some applications, the localization system has access to the full video, which allows fine-grain motion information to be estimated. This paper proposes capturing this information through motion fields and using it to improve the localization results. The learned motion fields act as a model-agnostic temporal regularizer that can be used with any localization system based on keypoints. Unlike optical flow-based strategies, our motion fields are estimated from the model domain, based on the trajectories described by the object keypoints. Therefore, they are not affected by poor imaging conditions. The benefits of the proposed strategy are shown on three applications: 1) segmentation of cardiac magnetic resonance; 2) facial model alignment; and 3) vehicle tracking. In each case, combining popular localization methods with the proposed regularizer leads to improvement in overall accuracies and reduces gross errors.

Deep learning applications in myocardial perfusion imaging, a systematic review and meta-analysis

Background

Coronary artery disease (CAD) is a leading cause of death worldwide, and the diagnostic process comprises of invasive testing with coronary angiography and non-invasive imaging, in addition to history, clinical examination, and electrocardiography (ECG). A highly accurate assessment of CAD lies in perfusion imaging which is performed by myocardial perfusion scintigraphy (MPS) and magnetic resonance imaging (stress CMR). Recently deep learning has been increasingly applied on perfusion imaging for better understanding of the diagnosis, safety, and outcome of CAD.The aim of this review is to summarise the evidence behind deep learning applications in myocardial perfusion imaging.

Methods

A systematic search was performed on MEDLINE and EMBASE databases, from database inception until September 29, 2020. This included all clinical studies focusing on deep learning applications and myocardial perfusion imaging, and excluded competition conference papers, simulation and animal studies, and studies which used perfusion imaging as a variable with different focus. This was followed by review of abstracts and full texts. A meta-analysis was performed on a subgroup of studies which looked at perfusion images classification. A summary receiver-operating curve (SROC) was used to compare the performance of different models, and area under the curve (AUC) was reported. Effect size, risk of bias and heterogeneity were tested.

Results

46 studies in total were identified, the majority were MPS studies (76%). The most common neural network was convolutional neural network (CNN) (41%). 13 studies (28%) looked at perfusion imaging classification using MPS, the pooled diagnostic accuracy showed AUC = 0.859. The summary receiver operating curve (SROC) comparison showed superior performance of CNN (AUC = 0.894) compared to MLP (AUC = 0.848). The funnel plot was asymmetrical, and the effect size was significantly different with p value < 0.001, indicating small studies effect and possible publication bias. There was no significant heterogeneity amongst studies according to Q test (p = 0.2184).

Conclusion

Deep learning has shown promise to improve myocardial perfusion imaging diagnostic accuracy, prediction of patients' events and safety. More research is required in clinical applications, to achieve better care for patients with known or suspected CAD.

A deep learning approach with temporal consistency for automatic myocardial segmentation of quantitative myocardial contrast echocardiography

Quantitative myocardial contrast echocardiography (MCE) has been proved to be valuable in detecting myocardial ischemia. During quantitative MCE analysis, myocardial segmentation is a critical step in determining accurate region of interests (ROIs). However, traditional myocardial segmentation mainly relies on manual tracing of myocardial contours, which is time-consuming and laborious. To solve this problem, we propose a fully automatic myocardial segmentation framework that can segment myocardial regions in MCE accurately without human intervention. A total of 100 patients' MCE sequences were divided into a training set and a test set according to a 7: 3 proportion for analysis. We proposed a bi-directional training schema, which incorporated temporal information of forward and backward direction among frames in MCE sequences to ensure temporal consistency by combining convolutional neural network with recurrent neural network. Experiment results demonstrated that compared with a traditional segmentation model (U-net) and the model considering only forward temporal information (U-net + forward), our framework achieved the highest segmentation precision in Dice coefficient (U-net vs U-net + forward vs our framework: 0.78 ± 0.07 vs 0.79 ± 0.07 vs 0.81 ± 0.07, p < 0.01), Intersection over Union (0.65 ± 0.09 vs 0.66 ± 0.09 vs 0.68 ± 0.09, p < 0.01), and lowest Hausdorff Distance (32.68 ± 14.6 vs 28.69 ± 13.18 vs 27.59 ± 12.82 pixel point, p < 0.01). In the visual grading study, the performance of our framework was the best among these three models (52.47 ± 4.29 vs 54.53 ± 5.10 vs 57.30 ± 4.73, p < 0.01). A case report on a randomly selected subject for perfusion analysis showed that the perfusion parameters generated by using myocardial segmentation of our proposed framework were similar to that of the expert annotation. The proposed framework could generate more precise myocardial segmentation when compared with traditional methods. The perfusion parameters generated by these myocardial segmentations have a good similarity to that of manual annotation, suggesting that it has the potential to be utilized in routine clinical practice.

Shape prior model via dual subspace segment projection learning

BACKGROUND AND OBJECTIVE

Shape prior models play a vital role for segmentation in medical image analysis. These models are most effective when shape variations can be captured by a parametric distribution, and sufficient training data is available. However, in the absence of these conditions, results are invariably much poorer. In this paper, we propose a novel shape prior model, via dual subspace segment projection learning (DSSPL), to address these challenges.

METHODS

DSSPL serves to compose shapes from an ensemble of shape segments where each segment is formed using two subspaces: global shape subspace and segment-specific subspace, each necessary for extracting global shape patterns and local patterns, respectively. This ensures the proposed approach has general shape plausibility in regions of signal drop-out or missing boundary information, and also more localized flexibility. The learned projections are constrained with l_2,1 sparse norm terms to extract the most distinguishable features, while the reconstructive properties of DSSPL reduces information loss and leverages the subspaces to provide contiguous shapes without any post-processing.

RESULTS

Extensive analysis is performed on three databases from different medical imaging systems across X-Ray, MRI, and ultrasound. DSSPL outperforms all compared benchmarks in terms of shape generalization ability and segmentation performance.

CONCLUSIONS

We propose a new shape prior model for segmentation in medical image analysis to address the challenges of modelling complex organ shapes with low sample size training data.

Stress CMR in Known or Suspected CAD: Diagnostic and Prognostic Role

The recently published 2019 guidelines on chronic coronary syndromes (CCS) focus on the need for noninvasive imaging modalities to accurately establish the diagnosis of coronary artery disease (CAD) and assess the risk of clinical scenario occurrence. Appropriate patient management should rely on controlling symptoms, improving prognosis, and guiding each therapeutic strategy as well as monitoring disease progress. Among the noninvasive imaging modalities, cardiovascular magnetic resonance (CMR) has gained broad acceptance in past years due to its unique features in providing a complete assessment of CAD through data on cardiac anatomy and function and myocardial viability, with high spatial and temporal resolution and without ionizing radiation. In detail, evaluation of the presence and extent of myocardial ischemia through stress CMR (S-CMR) has shown a high rule-in power in detecting functionally significant coronary artery stenosis in patients suspected of CCS. Moreover, S-CMR technique may add significant prognostic value, as demonstrated by different studies which have progressively evidenced the valuable power of this multiparametric imaging modality in predicting adverse cardiac events. The latest scientific progress supports a greater expansion of S-CMR with improvement of quantitative myocardial perfusion analysis, myocardial strain, and native mapping within the same examination. Although further study is warranted, these techniques, which are currently mostly restricted to the research field, are likely to become increasingly prevalent in the clinical setting with the scope of increasing accuracy in the selection of patients to be sent to invasive revascularization. This review investigates the diagnostic and prognostic role of S-CMR in the context of CAD, by analysing a strong, long-standing, scientific evidence together with an appraisal of new advanced techniques which may potentially enrich CAD management in the next future.

Deep learning in spatiotemporal cardiac imaging: A review of methodologies and clinical usability

The use of different cardiac imaging modalities such as MRI, CT or ultrasound enables the visualization and interpretation of altered morphological structures and function of the heart. In recent years, there has been an increasing interest in AI and deep learning that take into account spatial and temporal information in medical image analysis. In particular, deep learning tools using temporal information in image processing have not yet found their way into daily clinical practice, despite its presumed high diagnostic and prognostic value. This review aims to synthesize the most relevant deep learning methods and discuss their clinical usability in dynamic cardiac imaging using for example the complete spatiotemporal image information of the heart cycle. Selected articles were categorized according to the following indicators: clinical applications, quality of datasets, preprocessing and annotation, learning methods and training strategy, and test performance. Clinical usability was evaluated based on these criteria by classifying the selected papers into (i) clinical level, (ii) robust candidate and (iii) proof of concept applications. Interestingly, not a single one of the reviewed papers was classified as a "clinical level" study. Almost 39% of the articles achieved a "robust candidate" and as many as 61% a "proof of concept" status. In summary, deep learning in spatiotemporal cardiac imaging is still strongly research-oriented and its implementation in clinical application still requires considerable efforts. Challenges that need to be addressed are the quality of datasets together with clinical verification and validation of the performance achieved by the used method.

Clinical quantitative cardiac imaging for the assessment of myocardial ischaemia

Cardiac imaging has a pivotal role in the prevention, diagnosis and treatment of ischaemic heart disease. SPECT is most commonly used for clinical myocardial perfusion imaging, whereas PET is the clinical reference standard for the quantification of myocardial perfusion. MRI does not involve exposure to ionizing radiation, similar to echocardiography, which can be performed at the bedside. CT perfusion imaging is not frequently used but CT offers coronary angiography data, and invasive catheter-based methods can measure coronary flow and pressure. Technical improvements to the quantification of pathophysiological parameters of myocardial ischaemia can be achieved. Clinical consensus recommendations on the appropriateness of each technique were derived following a European quantitative cardiac imaging meeting and using a real-time Delphi process. SPECT using new detectors allows the quantification of myocardial blood flow and is now also suited to patients with a high BMI. PET is well suited to patients with multivessel disease to confirm or exclude balanced ischaemia. MRI allows the evaluation of patients with complex disease who would benefit from imaging of function and fibrosis in addition to perfusion. Echocardiography remains the preferred technique for assessing ischaemia in bedside situations, whereas CT has the greatest value for combined quantification of stenosis and characterization of atherosclerosis in relation to myocardial ischaemia. In patients with a high probability of needing invasive treatment, invasive coronary flow and pressure measurement is well suited to guide treatment decisions. In this Consensus Statement, we summarize the strengths and weaknesses as well as the future technological potential of each imaging modality.

Vessel Segmentation of X-Ray Coronary Angiographic Image Sequence

OBJECTIVE

To facilitate the analysis and diagnosis of X-ray coronary angiography in interventional surgery, it is necessary to extract vessel from X-ray coronary angiography. However, vessel images of angiography suffer from low quality with large artefacts, which challenges the existing vascular technology.

METHODS

In this paper, we propose a ávessel framework to detect vessels and segment vessels in angiographic vessel data. In this framework, we develop a new matrix decomposition model with gradient sparse in the tensor representation. Then, the energy function with the input of the hierarchical vessel is used in vessel detection and vessel segmentation.

RESULTS

Through experiments conducted on angiographic data, we have demonstrated the good performance of the proposed method in removing background structure.

CONCLUSION

We evaluated our method for vessel detection and segmentation in different clinical settings, including LAO/RAO with cranial and caudal angulation, and showed its competitive results compared with eight state-of-the-art methods in terms of extensive qualitative and quantitative evaluation.

SIGNIFICANCE

Our method can remove a large number of background artefacts and obtain a better vascular structure, which has contributed to the clinical diagnosis of coronary artery diseases.

High-Frame-Rate Contrast Echocardiography Using Diverging Waves: Initial In Vitro and In Vivo Evaluation

Contrast echocardiography (CE) ultrasound with microbubble contrast agents has significantly advanced our capability for assessment of cardiac function, including myocardium perfusion quantification. However, in standard CE techniques obtained with line by line scanning, the frame rate and image quality are limited. Recent research has shown significant frame-rate improvement in noncontrast cardiac imaging. In this work, we present and initially evaluate, both in vitro and in vivo, a high-frame-rate (HFR) CE imaging system using diverging waves and pulse inversion sequence. An imaging frame rate of 5500 frames/s before and 250 frames/s after compounding is achieved. A destruction-replenishment sequence has also been developed. The developed HFR CE is compared with standard CE in vitro on a phantom and then in vivo on a sheep heart. The image signal-to-noise ratio and contrast between the myocardium and the chamber are evaluated. The results show up to 13.4-dB improvement in contrast for HFR CE over standard CE when compared at the same display frame rate even when the average spatial acoustic pressure in HFR CE is 36% lower than the standard CE. It is also found that when coherent compounding is used, the HFR CE image intensity can be significantly modulated by the flow motion in the chamber.