Ultrasound image segmentation: a survey

SSAT-Swin: Deep Learning-Based Spinal Ultrasound Feature Segmentation for Scoliosis Using Self-Supervised Swin Transformer

OBJECTIVE

Scoliosis, a 3-D spinal deformity, requires early detection and intervention. Ultrasound curve angle (UCA) measurement using ultrasound images has emerged as a promising diagnostic tool. However, calculating the UCA directly from ultrasound images remains challenging due to low contrast, high noise, and irregular target shapes. Accurate segmentation results are therefore crucial to enhance image clarity and precision prior to UCA calculation.

METHODS

We propose the SSAT-Swin model, a transformer-based multi-class segmentation framework designed for ultrasound image analysis in scoliosis diagnosis. The model integrates a boundary-enhancement module in the decoder and a channel attention module in the skip connections. Additionally, self-supervised proxy tasks are used during pre-training on 1,170 images, followed by fine-tuning on 109 image-label pairs.

RESULTS

The SSAT-Swin achieved Dice scores of 85.6% and Jaccard scores of 74.5%, with a 92.8% scoliosis bone feature detection rate, outperforming state-of-the-art models.

CONCLUSION

Self-supervised learning enhances the model's ability to capture global context information, making it well-suited for addressing the unique challenges of ultrasound images, ultimately advancing scoliosis assessment through more accurate segmentation.

A probabilistic reduced-order modeling framework for patient-specific cardio-mechanical analysis

Cardio-mechanical models can be used to support clinical decision-making. Unfortunately, the substantial computational effort involved in many cardiac models hinders their application in the clinic, despite the fact that they may provide valuable information. In this work, we present a probabilistic reduced-order modeling (ROM) framework to dramatically reduce the computational effort of such models while providing a credibility interval. In the online stage, a fast-to-evaluate generalized one-fiber model is considered. This generalized one-fiber model incorporates correction factors to emulate patient-specific attributes, such as local geometry variations. In the offline stage, Bayesian inference is used to calibrate these correction factors on training data generated using a full-order isogeometric cardiac model (FOM). A Gaussian process is used in the online stage to predict the correction factors for geometries that are not in the training data. The proposed framework is demonstrated using two examples. The first example considers idealized left-ventricle geometries, for which the behavior of the ROM framework can be studied in detail. In the second example, the ROM framework is applied to scan-based geometries, based on which the application of the ROM framework in the clinical setting is discussed. The results for the two examples convey that the ROM framework can provide accurate online predictions, provided that adequate FOM training data is available. The uncertainty bands provided by the ROM framework give insight into the trustworthiness of its results. Large uncertainty bands can be considered as an indicator for the further population of the training data set.

Quantitative Evaluation of Kidney and Gallbladder Stones by Texture Analysis Using Gray Level Co-Occurrence Matrix Based on Diagnostic Ultrasound Images

Background/Objectives: Accurate diagnosis during ultrasound examinations of patients with kidney and gallbladder stones is crucial. Although stone areas typically show posterior acoustic shadowing on ultrasound images, their accurate diagnosis can be challenging if the shaded areas are vague. This study proposes a method to improve the diagnostic accuracy of kidney and gallbladder stones through texture analysis of ultrasound images. Methods: Two doctors and three sonographers evaluated abdominal ultrasound images and categorized kidney and gallbladder stones into groups based on their predicted likelihood of being present: 50-60%, 60-80%, and ≥80%. The texture analysis method for the posterior acoustic shadows generated from ultrasound images of stones was modeled using a gray level co-occurrence matrix (GLCM). Average values and 95% confidence intervals were used to evaluate the method. Results: The three prediction classes were clearly distinguished when GLCMContrast was applied to the ultrasound images of patients with kidney and gallbladder stones. However, GLCMCorrelation, GLCMEnergy, and GLCMHomogeneity were found to be difficult for analyzing the texture of shadowed areas in ultrasound images because they did not clearly or completely distinguish between the three classes. Conclusions: Accurate diagnosis of kidney and gallbladder stones may be possible using the GLCM texture analysis method applied to ultrasound images.

Enhancing Left Ventricular Segmentation in Echocardiograms Through GAN-Based Synthetic Data Augmentation and MultiResUNet Architecture

Background: Accurate segmentation of the left ventricle in echocardiograms is crucial for the diagnosis and monitoring of cardiovascular diseases. However, this process is hindered by the limited availability of high-quality annotated datasets and the inherent complexities of echocardiogram images. Traditional methods often struggle to generalize across varying image qualities and conditions, necessitating a more robust solution. Objectives: This study aims to enhance left ventricular segmentation in echocardiograms by developing a framework that integrates Generative Adversarial Networks (GANs) for synthetic data augmentation with a MultiResUNet architecture, providing a more accurate and reliable segmentation method. Methods: We propose a GAN-based framework that generates synthetic echocardiogram images and their corresponding segmentation masks, augmenting the available training data. The synthetic data, along with real echocardiograms from the EchoNet-Dynamic dataset, were used to train the MultiResUNet architecture. MultiResUNet incorporates multi-resolution blocks, residual connections, and attention mechanisms to effectively capture fine details at multiple scales. Additional enhancements include atrous spatial pyramid pooling (ASPP) and scaled exponential linear units (SELUs) to further improve segmentation accuracy. Results: The proposed approach significantly outperforms existing methods, achieving a Dice Similarity Coefficient of 95.68% and an Intersection over Union (IoU) of 91.62%. This represents improvements of 2.58% in Dice and 4.84% in IoU over previous segmentation techniques, demonstrating the effectiveness of GAN-based augmentation in overcoming data scarcity and improving segmentation performance. Conclusions: The integration of GAN-generated synthetic data and the MultiResUNet architecture provides a robust and accurate solution for left ventricular segmentation in echocardiograms. This approach has the potential to enhance clinical decision-making in cardiovascular medicine by improving the accuracy of automated diagnostic tools, even in the presence of limited and complex training data.

Syn-Net: A Synchronous Frequency-Perception Fusion Network for Breast Tumor Segmentation in Ultrasound Images

Accurate breast tumor segmentation in ultrasound images is a crucial step in medical diagnosis and locating the tumor region. However, segmentation faces numerous challenges due to the complexity of ultrasound images, similar intensity distributions, variable tumor morphology, and speckle noise. To address these challenges and achieve precise segmentation of breast tumors in complex ultrasound images, we propose a Synchronous Frequency-perception Fusion Network (Syn-Net). Initially, we design a synchronous dual-branch encoder to extract local and global feature information simultaneously from complex ultrasound images. Secondly, we introduce a novel Frequency- perception Cross-Feature Fusion (FrCFusion) Block, which utilizes Discrete Cosine Transform (DCT) to learn all-frequency features and effectively fuse local and global features while mitigating issues arising from similar intensity distributions. In addition, we develop a Full-Scale Deep Supervision method that not only corrects the influence of speckle noise on segmentation but also effectively guides decoder features towards the ground truth. We conduct extensive experiments on three publicly available ultrasound breast tumor datasets. Comparison with 14 state-of-the-art deep learning segmentation methods demonstrates that our approach exhibits superior sensitivity to different ultrasound images, variations in tumor size and shape, speckle noise, and similarity in intensity distribution between surrounding tissues and tumors. On the BUSI and Dataset B datasets, our method achieves better Dice scores compared to state-of-the-art methods, indicating superior performance in ultrasound breast tumor segmentation.

A novel cross-modal data augmentation method based on contrastive unpaired translation network for kidney segmentation in ultrasound imaging

BACKGROUND

Kidney ultrasound (US) image segmentation is one of the key steps in computer-aided diagnosis and treatment planning of kidney diseases. Recently, deep learning (DL) technology has demonstrated promising prospects in automatic kidney US segmentation. However, due to the poor quality, particularly the weak boundaries in kidney US imaging, obtaining accurate annotations for DL-based segmentation methods remain a challenging and time-consuming task. This issue can hinder the application of data-hungry deep learning methods.

PURPOSE

In this paper, we explore a novel cross-modal data augmentation method aimed at enhancing the performance of DL-based segmentation networks on the limited labeled kidney US dataset.

METHODS

In particular, we adopt a novel method based on contrastive unpaired translation network (CUT) to obtain simulated labeled kidney US images at a low cost from labeled abdomen computed tomography (CT) data and unlabeled kidney US images. To effectively improve the segmentation network performance, we propose an instance-weighting training strategy that simultaneously captures useful information from both the simulated and real labeled kidney US images. We trained our generative networks on a dataset comprising 4418 labeled CT slices and 4594 unlabeled US images. For segmentation network, we used a dataset consisting of 4594 simulated and 100 real kidney US images for training, 20 images for validation, and 169 real images for testing. We compared the performance of our method to several state-of-the-art approaches using the Wilcoxon signed-rank test, and applied the Bonferroni method for multiple comparison correction.

RESULTS

The experimental results show that we can synthesize accurate labeled kidney US images with a Fréchet inception distance of 52.52. Moreover, the proposed method achieves a segmentation accuracy of 0.9360 ± 0.0398 for U-Net on normal kidney US images, and 0.7719 ± 0.2449 on the abnormal dataset, as measured by the dice similarity coefficient. When compared to other training strategies, the proposed method demonstrated statistically significant superiority, with all p-values being less than 0.01.

CONCLUSIONS

The proposed method can effectively improve the accuracy and generalization ability of kidney US image segmentation models with limited annotated training data.

Deep Learning-Based Glioma Segmentation of 2D Intraoperative Ultrasound Images: A Multicenter Study Using the Brain Tumor Intraoperative Ultrasound Database (BraTioUS)

Background: Intraoperative ultrasound (ioUS) provides real-time imaging during neurosurgical procedures, with advantages such as portability and cost-effectiveness. Accurate tumor segmentation has the potential to substantially enhance the interpretability of ioUS images; however, its implementation is limited by persistent challenges, including noise, artifacts, and anatomical variability. This study aims to develop a convolutional neural network (CNN) model for glioma segmentation in ioUS images via a multicenter dataset. Methods: We retrospectively collected data from the BraTioUS and ReMIND datasets, including histologically confirmed gliomas with high-quality B-mode images. For each patient, the tumor was manually segmented on the 2D slice with its largest diameter. A CNN was trained using the nnU-Net framework. The dataset was stratified by center and divided into training (70%) and testing (30%) subsets, with external validation performed on two independent cohorts: the RESECT-SEG database and the Imperial College NHS Trust London cohort. Performance was evaluated using metrics such as the Dice similarity coefficient (DSC), average symmetric surface distance (ASSD), and 95th percentile Hausdorff distance (HD95). Results: The training cohort consisted of 197 subjects, 56 of whom were in the hold-out testing set and 53 in the external validation cohort. In the hold-out testing set, the model achieved a median DSC of 0.90, ASSD of 8.51, and HD95 of 29.08. On external validation, the model achieved a DSC of 0.65, ASSD of 14.14, and HD95 of 44.02 on the RESECT-SEG database and a DSC of 0.93, ASSD of 8.58, and HD95 of 28.81 on the Imperial-NHS cohort. Conclusions: This study supports the feasibility of CNN-based glioma segmentation in ioUS across multiple centers. Future work should enhance segmentation detail and explore real-time clinical implementation, potentially expanding ioUS's role in neurosurgical resection.

Leveraging anatomical constraints with uncertainty for pneumothorax segmentation

Background

Pneumothorax is a medical emergency caused by the abnormal accumulation of air in the pleural space-the potential space between the lungs and chest wall. On 2D chest radiographs, pneumothorax occurs within the thoracic cavity and outside of the mediastinum, and we refer to this area as "lung + space." While deep learning (DL) has increasingly been utilized to segment pneumothorax lesions in chest radiographs, many existing DL models employ an end-to-end approach. These models directly map chest radiographs to clinician-annotated lesion areas, often neglecting the vital domain knowledge that pneumothorax is inherently location-sensitive.

Methods

We propose a novel approach that incorporates the lung + space as a constraint during DL model training for pneumothorax segmentation on 2D chest radiographs. To circumvent the need for additional annotations and to prevent potential label leakage on the target task, our method utilizes external datasets and an auxiliary task of lung segmentation. This approach generates a specific constraint of lung + space for each chest radiograph. Furthermore, we have incorporated a discriminator to eliminate unreliable constraints caused by the domain shift between the auxiliary and target datasets.

Results

Our results demonstrated considerable improvements, with average performance gains of 4.6%, 3.6%, and 3.3% regarding intersection over union, dice similarity coefficient, and Hausdorff distance. These results were consistent across six baseline models built on three architectures (U-Net, LinkNet, or PSPNet) and two backbones (VGG-11 or MobileOne-S0). We further conducted an ablation study to evaluate the contribution of each component in the proposed method and undertook several robustness studies on hyper-parameter selection to validate the stability of our method.

Conclusions

The integration of domain knowledge in DL models for medical applications has often been underemphasized. Our research underscores the significance of incorporating medical domain knowledge about the location-specific nature of pneumothorax to enhance DL-based lesion segmentation and further bolster clinicians' trust in DL tools. Beyond pneumothorax, our approach is promising for other thoracic conditions that possess location-relevant characteristics.

Segmentation of four-chamber view images in fetal ultrasound exams using a novel deep learning model ensemble method

Fetal echocardiography, a specialized ultrasound application commonly utilized for fetal heart assessment, can greatly benefit from automated segmentation of anatomical structures, aiding operators in their evaluations. We introduce a novel approach that combines various deep learning models for segmenting key anatomical structures in 2D ultrasound images of the fetal heart. Our ensemble method combines the raw predictions from the selected models, obtaining the optimal set of segmentation components that closely approximate the distribution of the fetal heart, resulting in improved segmentation outcomes. The selection of these components involves sequential and hierarchical geometry filtering, focusing on the analysis of shape and relative distances. Unlike other ensemble strategies that average predictions, our method works as a shape selector, ensuring that the final segmentation aligns more accurately with anatomical expectations. Considering a large private dataset for model training and evaluation, we present both numerical and visual experiments highlighting the advantages of our method in comparison to the segmentations produced by the individual models and a conventional average ensemble. Furthermore, we show some applications where our method proves instrumental in obtaining reliable estimations.

DetSegDiff: A joint periodontal landmark detection and segmentation in intraoral ultrasound using edge-enhanced diffusion-based network

Individuals with malocclusion require an orthodontic diagnosis and treatment plan based on the severity of their condition. Assessing and monitoring changes in periodontal structures before, during, and after orthodontic procedures is crucial, and intraoral ultrasound (US) imaging has been shown a promising diagnostic tool in imaging periodontium. However, accurately delineating and analyzing periodontal structures in US videos is a challenging task for clinicians, as it is time-consuming and subject to interpretation errors. This paper introduces DetSegDiff, an edge-enhanced diffusion-based network developed to simultaneously detect the cementoenamel junction (CEJ) and segment alveolar bone structure in intraoral US videos. An edge feature encoder is designed to enhance edge and texture information for precise delineation of periodontal structures. Additionally, we employed the spatial squeeze-attention module (SSAM) to extract more representative features to perform both detection and segmentation tasks at global and local levels. This study used 169 videos from 17 orthodontic patients for training purposes and was subsequently tested on 41 videos from 4 additional patients. The proposed method achieved a mean distance difference of 0.17 ± 0.19 mm for the CEJ and an average Dice score of 90.1% for alveolar bone structure. As there is a lack of multi-task benchmark networks, thorough experiments were undertaken to assess and benchmark the proposed method against state-of-the-art (SOTA) detection and segmentation individual networks. The experimental results demonstrated that DetSegDiff outperformed SOTA approaches, confirming the feasibility of using automated diagnostic systems for orthodontists.

ACL-DUNet: A tumor segmentation method based on multiple attention and densely connected breast ultrasound images

Breast cancer is the most common cancer in women. Breast masses are one of the distinctive signs for diagnosing breast cancer, and ultrasound is widely used for screening as a non-invasive and effective method for breast examination. In this study, we used the Mendeley and BUSI datasets, comprising 250 images (100 benign, 150 malignant) and 780 images (133 normal, 487 benign, 210 malignant), respectively. The datasets were split into 80% for training and 20% for validation. The accurate measurement and characterization of different breast tumors play a crucial role in guiding clinical decision-making. The area and shape of the different breast tumors detected are critical for clinicians to make accurate diagnostic decisions. In this study, a deep learning method for mass segmentation in breast ultrasound images is proposed, which uses densely connected U-net with attention gates (AGs) as well as channel attention modules and scale attention modules for accurate breast tumor segmentation.The densely connected network is employed in the encoding stage to enhance the network's feature extraction capabilities. Three attention modules are integrated in the decoding stage to better capture the most relevant features. After validation on the Mendeley and BUSI datasets, the experimental results demonstrate that our method achieves a Dice Similarity Coefficient (DSC) of 0.8764 and 0.8313, respectively, outperforming other deep learning approaches. The source code is located at github.com/zhanghaoCV/plos-one.

NeighborNet: Learning Intra- and Inter-Image Pixel Neighbor Representation for Breast Lesion Segmentation

Breast lesion segmentation from ultrasound images is essential in computer-aided breast cancer diagnosis. To alleviate the problems of blurry lesion boundaries and irregular morphologies, common practices combine CNN and attention to integrate global and local information. However, previous methods use two independent modules to extract global and local features separately, such feature-wise inflexible integration ignores the semantic gap between them, resulting in representation redundancy/insufficiency and undesirable restrictions in clinic practices. Moreover, medical images are highly similar to each other due to the imaging methods and human tissues, but the captured global information by transformer-based methods in the medical domain is limited within images, the semantic relations and common knowledge across images are largely ignored. To alleviate the above problems, in the neighbor view, this paper develops a pixel neighbor representation learning method (NeighborNet) to flexibly integrate global and local context within and across images for lesion morphology and boundary modeling. Concretely, we design two neighbor layers to investigate two properties (i.e., number and distribution) of neighbors. The neighbor number for each pixel is not fixed but determined by itself. The neighbor distribution is extended from one image to all images in the datasets. With the two properties, for each pixel at each feature level, the proposed NeighborNet can evolve into the transformer or degenerate into the CNN for adaptive context representation learning to cope with the irregular lesion morphologies and blurry boundaries. The state-of-the-art performances on three ultrasound datasets prove the effectiveness of the proposed NeighborNet.

Ultrasound segmentation analysis via distinct and completed anatomical borders

PURPOSE

Segmenting ultrasound images is important for precise area and/or volume calculations, ensuring reliable diagnosis and effective treatment evaluation for diseases. Recently, many segmentation methods have been proposed and shown impressive performance. However, currently, there is no deeper understanding of how networks segment target regions or how they define the boundaries. In this paper, we present a new approach that analyzes ultrasound segmentation networks in terms of learned borders because border delimitation is challenging in ultrasound.

METHODS

We propose a way to split the boundaries for ultrasound images into distinct and completed. By exploiting the Grad-CAM of the split borders, we analyze the areas each network pays attention to. Further, we calculate the ratio of correct predictions for distinct and completed borders. We conducted experiments on an in-house leg ultrasound dataset (LEG-3D-US) as well as on two additional public datasets of thyroid, nerves, and one private for prostate.

RESULTS

Quantitatively, the networks exhibit around 10% improvement in handling completed borders compared to distinct borders. Similar to doctors, the network struggles to define the borders in less visible areas. Additionally, the Seg-Grad-CAM analysis underscores how completion uses distinct borders and landmarks, while distinct focuses mainly on the shiny structures. We also observe variations depending on the attention mechanism of each architecture.

CONCLUSION

In this work, we highlight the importance of studying ultrasound borders differently than other modalities such as MRI or CT. We split the borders into distinct and completed, similar to clinicians, and show the quality of the network-learned information for these two types of borders. Additionally, we open-source a 3D leg ultrasound dataset to the community https://github.com/Al3xand1a/segmentation-border-analysis .

MSDE-Net: A Multi-Scale Dual-Encoding Network for Surgical Instrument Segmentation

Minimally invasive surgery, which relies on surgical robots and microscopes, demands precise image segmentation to ensure safe and efficient procedures. Nevertheless, achieving accurate segmentation of surgical instruments remains challenging due to the complexity of the surgical environment. To tackle this issue, this paper introduces a novel multiscale dual-encoding segmentation network, termed MSDE-Net, designed to automatically and precisely segment surgical instruments. The proposed MSDE-Net leverages a dual-branch encoder comprising a convolutional neural network (CNN) branch and a transformer branch to effectively extract both local and global features. Moreover, an attention fusion block (AFB) is introduced to ensure effective information complementarity between the dual-branch encoding paths. Additionally, a multilayer context fusion block (MCF) is proposed to enhance the network's capacity to simultaneously extract global and local features. Finally, to extend the scope of global feature information under larger receptive fields, a multi-receptive field fusion (MRF) block is incorporated. Through comprehensive experimental evaluations on two publicly available datasets for surgical instrument segmentation, the proposed MSDE-Net demonstrates superior performance compared to existing methods.

Deep learning for real-time multi-class segmentation of artefacts in lung ultrasound

Lung ultrasound (LUS) has emerged as a safe and cost-effective modality for assessing lung health, particularly during the COVID-19 pandemic. However, interpreting LUS images remains challenging due to its reliance on artefacts, leading to operator variability and limiting its practical uptake. To address this, we propose a deep learning pipeline for multi-class segmentation of objects (ribs, pleural line) and artefacts (A-lines, B-lines, B-line confluence) in ultrasound images of a lung training phantom. Lightweight models achieved a mean Dice Similarity Coefficient (DSC) of 0.74, requiring fewer than 500 training images. Applying this method in real-time, at up to 33.4 frames per second in inference, allows enhanced visualisation of these features in LUS images. This could be useful in providing LUS training and helping to address the skill gap. Moreover, the segmentation masks obtained from this model enable the development of explainable measures of disease severity, which have the potential to assist in the triage and management of patients. We suggest one such semi-quantitative measure called the B-line Artefact Score, which is related to the percentage of an intercostal space occupied by B-lines and in turn may be associated with the severity of a number of lung conditions. Moreover, we show how transfer learning could be used to train models for small datasets of clinical LUS images, identifying pathologies such as simple pleural effusions and lung consolidation with DSC values of 0.48 and 0.32 respectively. Finally, we demonstrate how such DL models could be translated into clinical practice, implementing the phantom model alongside a portable point-of-care ultrasound system, facilitating bedside assessment and improving the accessibility of LUS.

Segmentation of bone surface from ultrasound using a lightweight network UBS-Net

Objective. Ultrasound-assisted orthopaedic navigation held promise due to its non-ionizing feature, portability, low cost, and real-time performance. To facilitate the applications, it was critical to have accurate and real-time bone surface segmentation. Nevertheless, the imaging artifacts and low signal-to-noise ratios in the tomographical B-mode ultrasound (B-US) images created substantial challenges in bone surface detection. In this study, we presented an end-to-end lightweight US bone segmentation network (UBS-Net) for bone surface detection.Approach. We presented an end-to-end lightweight UBS-Net for bone surface detection, using the U-Net structure as the base framework and a level set loss function for improved sensitivity to bone surface detectability. A dual attention (DA) mechanism was introduced at the end of the encoder, which considered both position and channel information to obtain the correlation between the position and channel dimensions of the feature map, where axial attention (AA) replaced the traditional self-attention (SA) mechanism in the position attention module for better computational efficiency. The position attention and channel attention (CA) were combined with a two-class fusion module for the DA map. The decoding module finally completed the bone surface detection.Main Results. As a result, a frame rate of 21 frames per second (fps) in detection were achieved. It outperformed the state-of-the-art method with higher segmentation accuracy (Dice similarity coefficient: 88.76% versus 87.22%) when applied the retrospective ultrasound (US) data from 11 volunteers.Significance. The proposed UBS-Net for bone surface detection in ultrasound achieved outstanding accuracy and real-time performance. The new method out-performed the state-of-the-art methods. It had potential in US-guided orthopaedic surgery applications.

Breast ultrasound image despeckling using multi-filtering DFrFT and adaptive fast BM3D

Improving the quality of breast ultrasound images is of great significance for clinical diagnosis which can greatly boost the diagnostic accuracy of ultrasonography. However, due to the influence of ultrasound imaging principles and acquisition equipment, the collected ultrasound images naturally contain a large amount of speckle noise, which leads to a decrease in image quality and affects clinical diagnosis. To overcome this problem, we propose an improved denoising algorithm combining multi-filter DFrFT (Discrete Fractional Fourier Transform) and the adaptive fast BM3D (Block Matching and 3D collaborative filtering) method. Firstly, we provide the multi-filtering DFrFT method for preprocessing the original breast ultrasound image so as to remove some speckle noise early in fractional transformation domain. Based on the fractional frequency spectrum characteristics of breast ultrasound images, three types of filters are designed correspondingly in low, medium, and high frequency domains. And by integrating filtered images, the enhanced images are obtained which not only remove some speckle noise in background but also preserve the details of breast lesions. Secondly, for further enhancing the image quality on the basis of multi-filter DFrFT, we propose the adaptive fast BM3D method by introducing the DBSCAN-based super pixel segmentation to block matching process, which utilizes super pixel segmentation labels to provide a reference on how similar it is between target block and retrieval blocks. It reduces the number of blocks to be retrieved and make the matched blocks with more similar features. At last, the local noise parameter estimation is also adopted in the hard threshold filtering process of traditional BM3D algorithm to achieve local adaptive filtering and further improving the denoising effect. The synthetic data and real breast ultrasound data examples show that this combined method can improve the speckle suppression level and keep the fidelity of structure effectively without increasing time cost.

An efficient framework for lesion segmentation in ultrasound images using global adversarial learning and region-invariant loss

Lesion segmentation in ultrasound images is an essential yet challenging step for early evaluation and diagnosis of cancers. In recent years, many automatic CNN-based methods have been proposed to assist this task. However, most modern approaches often lack capturing long-range dependencies and prior information making it difficult to identify the lesions with unfixed shapes, sizes, locations, and textures. To address this, we present a novel lesion segmentation framework that guides the model to learn the global information about lesion characteristics and invariant features (e.g., morphological features) of lesions to improve the segmentation in ultrasound images. Specifically, the segmentation model is guided to learn the characteristics of lesions from the global maps using an adversarial learning scheme with a self-attention-based discriminator. We argue that under such a lesion characteristics-based guidance mechanism, the segmentation model gets more clues about the boundaries, shapes, sizes, and positions of lesions and can produce reliable predictions. In addition, as ultrasound lesions have different textures, we embed this prior knowledge into a novel region-invariant loss to constrain the model to focus on invariant features for robust segmentation. We demonstrate our method on one in-house breast ultrasound (BUS) dataset and two public datasets (i.e., breast lesion (BUS B) and thyroid nodule from TNSCUI2020). Experimental results show that our method is specifically suitable for lesion segmentation in ultrasound images and can outperform the state-of-the-art approaches with Dice of 0.931, 0.906, and 0.876, respectively. The proposed method demonstrates that it can provide more important information about the characteristics of lesions for lesion segmentation in ultrasound images, especially for lesions with irregular shapes and small sizes. It can assist the current lesion segmentation models to better suit clinical needs.

Weakly supervised learning-based 3D bladder reconstruction from 2D ultrasound images for bladder volume measurement

BACKGROUND

Accurate measurement of bladder volume is necessary to maintain the consistency of the patient's anatomy in radiation therapy for pelvic tumors. As the diversity of the bladder shape, traditional methods for bladder volume measurement from 2D ultrasound have been found to produce inaccurate results.

PURPOSE

To improve the accuracy of bladder volume measurement from 2D ultrasound images for patients with pelvic tumors.

METHODS

The bladder ultrasound images from 130 patients with pelvic cancer were collected retrospectively. All data were split into a training set (80 patients), a validation set (20 patients), and a test set (30 patients). A total of 12 transabdominal ultrasound images for one patient were captured by automatically rotating the ultrasonic probe with an angle step of 15°. An incomplete 3D ultrasound volume was synthesized by arranging these 2D ultrasound images in 3D space according to the acquisition angles. With this as input, a weakly supervised learning-based 3D bladder reconstruction neural network model was built to predict the complete 3D bladder. The key point is that we designed a novel loss function, including the supervised loss of bladder segmentation in the ultrasound images at known angles and the compactness loss of the 3D bladder. Bladder volume was calculated by counting the number of voxels belonging to the 3D bladder. The dice similarity coefficient (DSC) was used to evaluate the accuracy of bladder segmentation, and the relative standard deviation (RSD) was used to evaluate the calculation accuracy of bladder volume with that of computed tomography (CT) images as the gold standard.

RESULTS

The results showed that the mean DSC was up to 0.94 and the mean absolute RSD can be reduced to 6.3% when using 12 ultrasound images of one patient. Further, the mean DSC also was up to 0.90 and the mean absolute RSD can be reduced to 9.0% even if only two ultrasound images were used (i.e., the angle step is 90°). Compared with the commercial algorithm in bladder scanners, which has a mean absolute RSD of 13.6%, our proposed method showed a considerably huge improvement.

CONCLUSIONS

The proposed weakly supervised learning-based 3D bladder reconstruction method can greatly improve the accuracy of bladder volume measurement. It has great potential to be used in bladder volume measurement devices in the future.

MLAU-Net: Deep supervised attention and hybrid loss strategies for enhanced segmentation of low-resolution kidney ultrasound

Objective

The precise segmentation of kidneys from a 2D ultrasound (US) image is crucial for diagnosing and monitoring kidney diseases. However, achieving detailed segmentation is difficult due to US images' low signal-to-noise ratio and low-contrast object boundaries.

Methods

This paper presents an approach called deep supervised attention with multi-loss functions (MLAU-Net) for US segmentation. The MLAU-Net model combines the benefits of attention mechanisms and deep supervision to improve segmentation accuracy. The attention mechanism allows the model to selectively focus on relevant regions of the kidney and ignore irrelevant background information, while the deep supervision captures the high-dimensional structure of the kidney in US images.

Results

We conducted experiments on two datasets to evaluate the MLAU-Net model's performance. The Wuerzburg Dynamic Kidney Ultrasound (WD-KUS) dataset with annotation contained kidney US images from 176 patients split into training and testing sets totaling 44,880. The Open Kidney Dataset's second dataset has over 500 B-mode abdominal US images. The proposed approach achieved the highest dice, accuracy, specificity, Hausdorff distance (HD95), recall, and Average Symmetric Surface Distance (ASSD) scores of 90.2%, 98.26%, 98.93%, 8.90 mm, 91.78%, and 2.87 mm, respectively, upon testing and comparison with state-of-the-art U-Net series segmentation frameworks, which demonstrates the potential clinical value of our work.

Conclusion

The proposed MLAU-Net model has the potential to be applied to other medical image segmentation tasks that face similar challenges of low signal-to-noise ratios and low-contrast object boundaries.

Knowledge distillation for efficient standard scanplane detection of fetal ultrasound

In clinical practice, ultrasound standard planes (SPs) selection is experience-dependent and it suffers from inter-observer and intra-observer variability. Automatic recognition of SPs can help improve the quality of examinations and make the evaluations more objective. In this paper, we propose a method for the automatic identification of SPs, to be installed onboard a portable ultrasound system with limited computational power. The deep Learning methodology we design is based on the concept of Knowledge Distillation, transferring knowledge from a large and well-performing teacher to a smaller student architecture. To this purpose, we evaluate a set of different potential teachers and students, as well as alternative knowledge distillation techniques, to balance a trade-off between performances and architectural complexity. We report a thorough analysis of fetal ultrasound data, focusing on a benchmark dataset, to the best of our knowledge the only one available to date.

Radiomics Harmonization in Ultrasound Images for Cervical Cancer Lymph Node Metastasis Prediction Using Cycle-GAN

Background: Ultrasound (US) based radiomics is susceptible to variations in scanners, sonographers. Objective: To retrospectively investigate the feasibility of an adapted cycle generative adversarial networks (CycleGAN) in the style transfer to improve US based radiomics in the prediction of lymph node metastasis (LNM) with images from multiple scanners for patients with early cervical cancer (ECC). Methods: The CycleGAN was firstly trained to transfer paired US phantom images from one US device to another one; the model was then further trained and tested with clinical US images of ECC by transferring images from four US devices to one specific device; finally, the adapted model was tested with its effects on the radiomics feature harmonization and accuracy of LNM prediction in US based radiomics for ECC patients. Results: Phantom study demonstrated an increased radiomics harmonization using CycleGAN with an average Pearson correlation coefficient of 0.60 and 0.81 for radiomics features extracted from original and generated images in correlation with the target phantom images, respectively. Additionally, the image quality metric Peak Signal-to-Noise Ratio (PSNR) was increased from 11.18 for the original images to 15.45 for the generated image. Clinical US images of 169 ECC patients were enrolled for style transfer model training and validation. The area under curve (AUC) of LNM prediction radiomics models with features extracted from generated images of different style transfer models ranged from 0.73 to 0.85. The AUC was improved from 0.78 with features extracted from original images to 0.85 with style transferred images. Conclusions: The adapted CycleGAN network is able to increase the radiomics feature harmonization for images from different ultrasound equipment based on image domain and improve the LNM prediction accuracy for ECC.

MDF-Net: A Multi-Scale Dynamic Fusion Network for Breast Tumor Segmentation of Ultrasound Images

Breast tumor segmentation of ultrasound images provides valuable information of tumors for early detection and diagnosis. Accurate segmentation is challenging due to low image contrast between areas of interest; speckle noises, and large inter-subject variations in tumor shape and size. This paper proposes a novel Multi-scale Dynamic Fusion Network (MDF-Net) for breast ultrasound tumor segmentation. It employs a two-stage end-to-end architecture with a trunk sub-network for multiscale feature selection and a structurally optimized refinement sub-network for mitigating impairments such as noise and inter-subject variation via better feature exploration and fusion. The trunk network is extended from UNet++ with a simplified skip pathway structure to connect the features between adjacent scales. Moreover, deep supervision at all scales, instead of at the finest scale in UNet++, is proposed to extract more discriminative features and mitigate errors from speckle noise via a hybrid loss function. Unlike previous works, the first stage is linked to a loss function of the second stage so that both the preliminary segmentations and refinement subnetworks can be refined together at training. The refinement sub-network utilizes a structurally optimized MDF mechanism to integrate preliminary segmentation information (capturing general tumor shape and size) at coarse scales and explores inter-subject variation information at finer scales. Experimental results from two public datasets show that the proposed method achieves better Dice and other scores over state-of-the-art methods. Qualitative analysis also indicates that our proposed network is more robust to tumor size/shapes, speckle noise and heavy posterior shadows along tumor boundaries. An optional post-processing step is also proposed to facilitate users in mitigating segmentation artifacts. The efficiency of the proposed network is also illustrated on the "Electron Microscopy neural structures segmentation dataset". It outperforms a state-of-the-art algorithm based on UNet-2022 with simpler settings. This indicates the advantages of our MDF-Nets in other challenging image segmentation tasks with small to medium data sizes.

A Machine Learning Method for Automated Description and Workflow Analysis of First Trimester Ultrasound Scans

Obstetric ultrasound assessment of fetal anatomy in the first trimester of pregnancy is one of the less explored fields in obstetric sonography because of the paucity of guidelines on anatomical screening and availability of data. This paper, for the first time, examines imaging proficiency and practices of first trimester ultrasound scanning through analysis of full-length ultrasound video scans. Findings from this study provide insights to inform the development of more effective user-machine interfaces, of targeted assistive technologies, as well as improvements in workflow protocols for first trimester scanning. Specifically, this paper presents an automated framework to model operator clinical workflow from full-length routine first-trimester fetal ultrasound scan videos. The 2D+t convolutional neural network-based architecture proposed for video annotation incorporates transfer learning and spatio-temporal (2D+t) modelling to automatically partition an ultrasound video into semantically meaningful temporal segments based on the fetal anatomy detected in the video. The model results in a cross-validation A1 accuracy of 96.10% , F1=0.95 , precision =0.94 and recall =0.95 . Automated semantic partitioning of unlabelled video scans (n=250) achieves a high correlation with expert annotations ( ρ = 0.95, p=0.06 ). Clinical workflow patterns, operator skill and its variability can be derived from the resulting representation using the detected anatomy labels, order, and distribution. It is shown that nuchal translucency (NT) is the toughest standard plane to acquire and most operators struggle to localize high-quality frames. Furthermore, it is found that newly qualified operators spend 25.56% more time on key biometry tasks than experienced operators.

AAU-Net: An Adaptive Attention U-Net for Breast Lesions Segmentation in Ultrasound Images

Various deep learning methods have been proposed to segment breast lesions from ultrasound images. However, similar intensity distributions, variable tumor morphologies and blurred boundaries present challenges for breast lesions segmentation, especially for malignant tumors with irregular shapes. Considering the complexity of ultrasound images, we develop an adaptive attention U-net (AAU-net) to segment breast lesions automatically and stably from ultrasound images. Specifically, we introduce a hybrid adaptive attention module (HAAM), which mainly consists of a channel self-attention block and a spatial self-attention block, to replace the traditional convolution operation. Compared with the conventional convolution operation, the design of the hybrid adaptive attention module can help us capture more features under different receptive fields. Different from existing attention mechanisms, the HAAM module can guide the network to adaptively select more robust representation in channel and space dimensions to cope with more complex breast lesions segmentation. Extensive experiments with several state-of-the-art deep learning segmentation methods on three public breast ultrasound datasets show that our method has better performance on breast lesions segmentation. Furthermore, robustness analysis and external experiments demonstrate that our proposed AAU-net has better generalization performance in the breast lesion segmentation. Moreover, the HAAM module can be flexibly applied to existing network frameworks. The source code is available on https://github.com/CGPxy/AAU-net.

EchoEFNet: Multi-task deep learning network for automatic calculation of left ventricular ejection fraction in 2D echocardiography

Left ventricular ejection fraction (LVEF) is essential for evaluating left ventricular systolic function. However, its clinical calculation requires the physician to interactively segment the left ventricle and obtain the mitral annulus and apical landmarks. This process is poorly reproducible and error prone. In this study, we propose a multi-task deep learning network EchoEFNet. The network use ResNet50 with dilated convolution as the backbone to extract high-dimensional features while maintaining spatial features. The branching network used our designed multi-scale feature fusion decoder to segment the left ventricle and detect landmarks simultaneously. The LVEF was then calculated automatically and accurately using the biplane Simpson's method. The model was tested for performance on the public dataset CAMUS and private dataset CMUEcho. The experimental results showed that the geometrical metrics and percentage of correct keypoints of EchoEFNet outperformed other deep learning methods. The correlation between the predicted LVEF and true values on the CAMUS and CMUEcho datasets was 0.854 and 0.916, respectively.

SCCNet: Self-correction boundary preservation with a dynamic class prior filter for high-variability ultrasound image segmentation

The highly ambiguous nature of boundaries and similar objects is difficult to address in some ultrasound image segmentation tasks, such as neck muscle segmentation, leading to unsatisfactory performance. Thus, this paper proposes a two-stage network called SCCNet (self-correction context network) using a self-correction boundary preservation module and class-context filter to alleviate these problems. The proposed self-correction boundary preservation module uses a dynamic key boundary point (KBP) map to increase the capability of iteratively discriminating ambiguous boundary points segments, and the predicted segmentation map from one stage is used to obtain a dynamic class prior filter to improve the segmentation performance at Stage 2. Finally, three datasets, Neck Muscle, CAMUS and Thyroid, are used to demonstrate that our proposed SCCNet outperforms other state-of-the art methods, such as BPBnet, DSNnet, and RAGCnet. Our proposed network shows at least a 1.2-3.7% improvement on the three datasets, Neck Muscle, Thyroid, and CAMUS. The source code is available at https://github.com/lijixing0425/SCCNet.

A Signal-Domain Object Segmentation Method for Ultrasound and Photoacoustic Computed Tomography

Image segmentation is important in improving the diagnostic capability of ultrasound computed tomography (USCT) and photoacoustic computed tomography (PACT), as it can be included in the image reconstruction process to improve image quality and quantification abilities. Segmenting the imaged object out of the background using image domain methods is easily complicated by low contrast, noise, and artifacts in the reconstructed image. Here, we introduce a new signal domain object segmentation method for USCT and PACT which does not require image reconstruction beforehand and is automatic, robust, computationally efficient, accurate, and straightforward. We first establish the relationship between the time-of-flight (TOF) of the received first arrival waves and the object's boundary which is described by ellipse equations. Then, we show that the ellipses are tangent to the boundary. By looking for tangent points on the common tangent of neighboring ellipses, the boundary can be approximated with high fidelity. Imaging experiments of human fingers and mice cross sections showed that our method provided equivalent or better segmentations than the optimal ones by active contours. In summary, our method greatly reduces the overall complexity of object segmentation and shows great potential in eliminating user dependency without sacrificing segmentation accuracy. The method can be further seamlessly incorporated into algorithms for other processing purposes in USCT and PACT, such as high-quality image reconstruction.

Coherent Bistatic 3-D Ultrasound Imaging Using Two Sparse Matrix Arrays

In the last decade, many advances have been made in high frame rate 3-D ultrasound imaging, including more flexible acquisition systems, transmit (TX) sequences, and transducer arrays. Compounding multiangle transmits of diverging waves has shown to be fast and effective for 2-D matrix arrays, where heterogeneity between transmits is key in optimizing the image quality. However, the anisotropy in contrast and resolution remains a drawback that cannot be overcome with a single transducer. In this study, a bistatic imaging aperture is demonstrated that consists of two synchronized matrix ( 32×32 ) arrays, allowing for fast interleaved transmits with a simultaneous receive (RX). First, for a single array, the aperture efficiency for high volume rate imaging was evaluated between sparse random arrays and fully multiplexed arrays. Second, the performance of the bistatic acquisition scheme was analyzed for various positions on a wire phantom and was showcased in a dynamic setup mimicking the human abdomen and aorta. Sparse array volume images were equal in resolution and lower in contrast compared to fully multiplexed arrays but can efficiently minimize decorrelation during motion for multiaperture imaging. The dual-array imaging aperture improved the spatial resolution in the direction of the second transducer, reducing the average volumetric speckle size with 72% and the axial-lateral eccentricity with 8%. In the aorta phantom, the angular coverage increased by a factor of 3 in the axial-lateral plane, raising the wall-lumen contrast with 16% compared to single-array images, despite accumulation of thermal noise in the lumen.

Cross-Domain Echocardiography Segmentation with Multi-Space Joint Adaptation

The segmentation of the left ventricle endocardium (LVendo) and the left ventricle epicardium (LVepi) in echocardiography plays an important role in clinical diagnosis. Recently, deep neural networks have been the most commonly used approach for echocardiography segmentation. However, the performance of a well-trained segmentation network may degrade in unseen domain datasets due to the distribution shift of the data. Adaptation algorithms can improve the generalization of deep neural networks to different domains. In this paper, we present a multi-space adaptation-segmentation-joint framework, named MACS, for cross-domain echocardiography segmentation. It adopts a generative adversarial architecture; the generator fulfills the segmentation task and the multi-space discriminators align the two domains on both the feature space and output space. We evaluated the MACS method on two echocardiography datasets from different medical centers and vendors, the publicly available CAMUS dataset and our self-acquired dataset. The experimental results indicated that the MACS could handle unseen domain datasets well, without requirements for manual annotations, and improve the generalization performance by 2.2% in the Dice metric.

Left ventricle segmentation in transesophageal echocardiography images using a deep neural network

PURPOSE

There has been little progress in research on the best anatomical position for effective chest compressions and cardiac function during cardiopulmonary resuscitation (CPR). This study aimed to divide the left ventricle (LV) into segments to determine the best position for effective chest compressions using the LV systolic function seen during CPR.

METHODS

We used transesophageal echocardiography images acquired during CPR. A deep neural network with an attention mechanism and a residual feature aggregation module were applied to the images to segment the LV. The results were compared between the proposed model and U-Net.

RESULTS

The results of the proposed model showed higher performance in most metrics when compared to U-Net: dice coefficient (0.899±0.017 vs. 0.792±0.027, p<0.05); intersection of union (0.822±0.026 vs. 0.668±0.034, p<0.05); recall (0.904±0.023 vs. 0.757±0.037, p<0.05); precision (0.901±0.021 vs. 0.859±0.034, p>0.05). There was a significant difference between the proposed model and U-Net.

CONCLUSION

Compared to U-Net, the proposed model showed better performance for all metrics. This model would allow us to evaluate the systolic function of the heart during CPR in greater detail by segmenting the LV more accurately.

DPC-MSGATNet: dual-path chain multi-scale gated axial-transformer network for four-chamber view segmentation in fetal echocardiography

Echocardiography is essential in evaluating fetal cardiac anatomical structures and functions when clinicians conduct early treatment and screening for congenital heart defects, a common and intricate fetal malformation. Nevertheless, the prenatal detection rate of fetal CHD remains low since the peculiarities of fetal cardiac structures and the variousness of fetal CHD. Precisely segmenting four cardiac chambers can assist clinicians in analyzing cardiac morphology and further facilitate CHD diagnosis. Hence, we design a dual-path chain multi-scale gated axial-transformer network (DPC-MSGATNet) that simultaneously models global dependencies and local visual cues for fetal ultrasound (US) four-chamber (FC) views and further accurately segments four chambers. Our DPC-MSGATNet includes a global and a local branch that simultaneously operates on an entire FC view and image patches to learn multi-scale representations. We design a plug-and-play module, Interactive dual-path chain gated axial-transformer (IDPCGAT), to enhance the interactions between global and local branches. In IDPCGAT, the multi-scale representations from the two branches can complement each other, capturing the same region’s salient features and suppressing feature responses to maintain only the activations associated with specific targets. Extensive experiments demonstrate that the DPC-MSGATNet exceeds seven state-of-the-art convolution- and transformer-based methods by a large margin in terms of F1 and IoU scores on our fetal FC view dataset, achieving a F1 score of 96.87$$\%$$ % and an IoU score of 93.99$$\%$$ % . The codes and datasets can be available at https://github.comQiaoSiBo/DPC-MSGATNet.

A Computer-Aided Markov Random Field Segmentation Algorithm for Assessing Fetal Ventricular Chambers

Congenital heart disease (CHD) is the most widely occurring congenital defect and accounts to about 28% of the overall congenital defects. Analysis of the development of the fetal heart thus plays an important role for detection of abnormality in early stages and to take corrective measures. Cardiac chamber analysis is one of the important diagnosing methods. Segmentation of the cardiac chambers must be done appropriately to avoid false interpretations. Effective segmentation of fetal ventricular chambers is a challenging task as the speckle noise inherent in ultrasound images cause blurring of the boundaries of anatomical structures. Several segmentation techniques have been proposed for extracting the fetal cardiac chambers. This article discusses the performance evaluation of automated, probability based segmentation approach, and Markov random field (MRF) for segmenting the fetal ventricular chambers of ultrasonic cineloop sequences. 837 ultrasonic biometery sequences of various gestations were collected from local diagnostic center after due ethical clearance and used for the study. In order to assess the efficiency of the segmentation technique, four metrics such as dice coefficient, true positive ratio (TPR), false positive ratio (FPR), similarity ratio (SIR), and precision (PR) were used. In order to perform ground truth validation, 56% of the data used in this study were annotated by clinical experts. The automated segmentation yielded comparable results with manual annotation. The technique results in average value of 0.68 for Dice coefficient, 0.723 for TPR, 0.604 for SIR, and 0.632 for PR.

Comparison of auto-contouring and hand-contouring of ultrasound images of the tongue surface

Contours traced by trained phoneticians have been considered to be the most accurate way to identify the midsagittal tongue surface from ultrasound video frames. In this study, inter-measurer reliability was evaluated using measures that quantified both how closely human-placed contours approximated each other as well as how consistent measurers were in defining the start and end points of contours. High reliability across three measurers was found for all measures, consistent with treating contours placed by trained phoneticians as the 'gold standard.' However, due to the labour-intensive nature of hand-placing contours, automatic algorithms that detect the tongue surface are increasingly being used to extract tongue-surface data from ultrasound videos. Contours placed by six automatic algorithms (SLURP, EdgeTrak, EPCS, and three different configurations of the algorithm provided in Articulate Assistant Advanced) were compared to human-placed contours, with the same measures used to evaluate the consistency of the trained phoneticians. We found that contours defined by SLURP, EdgeTrak, and two of the AAA configurations closely matched the hand-placed contours along sections of the image where the algorithms and humans agreed that there was a discernible contour. All of the algorithms were much less reliable than humans in determining the anterior (tongue-tip) edge of tongue contours. Overall, the contours produced by SLURP, EdgeTrak, and AAA should be useable in a variety of clinical applications, subject to spot-checking. Additional practical considerations of these algorithms are also discussed.

SILICOFCM platform, multiscale modeling of left ventricle from echocardiographic images and drug influence for cardiomyopathy disease

BACKGROUND AND OBJECTIVE

In silico clinical trials are the future of medicine and virtual testing and simulation are the future of medical engineering. The use of a computational platform can reduce costs and time required for developing new models of medical devices and drugs. The computational platform, which is one of the main results of the SILICOFCM project, was developed using state-of-the-art finite element modeling for macro simulation of fluid-structure interaction with micro modeling at the molecular level for drug interaction with the cardiac cells. SILICOFCM platform is using for risk prediction and optimal drug therapy of familial cardiomyopathy in a specific patient.

METHODS

In order to obtain 3D image reconstruction, the U-net architecture was used to determine geometric parameters for the left ventricle which were extracted from the echocardiographic apical and M-mode views. A micro-mechanics cellular model which includes three kinetic processes of sarcomeric proteins interactions was developed. It allows simulation of the drugs which are divided into three major groups defined by the principal action of each drug. Fluid-solid coupling for the left ventricle was presented. A nonlinear material model of the heart wall that was developed by using constitutive curves which include the stress-strain relationship was used.

RESULTS

The results obtained with the parametric model of the left ventricle where pressure-volume (PV) diagrams depend on the change of Ca2+ were presented. It directly affects the ejection fraction. The presented approach with the variation of the left ventricle (LV) geometry and simulations which include the influence of different parameters on the PV diagrams are directly interlinked with drug effects on the heart function. It includes different drugs such as Entresto and Digoxin that directly affect the cardiac PV diagrams and ejection fraction.

CONCLUSIONS

Computational platforms such as the SILICOFCM platform are novel tools for risk prediction of cardiac disease in a specific patient that will certainly open a new avenue for in silico clinical trials in the future.

Left ventricle analysis in echocardiographic images using transfer learning

The segmentation of cardiac boundaries, specifically Left Ventricle (LV) segmentation in 2D echocardiographic images, is a critical step in LV segmentation and cardiac function assessment. These images are generally of poor quality and present low contrast, making daily clinical delineation difficult, time-consuming, and often inaccurate. Thus, it is necessary to design an intelligent automatic endocardium segmentation system. The present work aims to examine and assess the performance of some deep learning-based architectures such as U-Net1, U-Net2, LinkNet, Attention U-Net, and TransUNet using the public CAMUS (Cardiac Acquisitions for Multi-structure Ultrasound Segmentation) dataset. The adopted approach emphasizes the advantage of using transfer learning and resorting to pre-trained backbones in the encoder part of a segmentation network for echocardiographic image analysis. The experimental findings indicated that the proposed framework with the [Formula: see text]-[Formula: see text] is quite promising; it outperforms other more recent approaches with a Dice similarity coefficient of 93.30% and a Hausdorff Distance of 4.01 mm. In addition, a good agreement between the clinical indices calculated from the automatic segmentation and those calculated from the ground truth segmentation. For instance, the mean absolute errors for the left ventricular end-diastolic volume, end-systolic volume, and ejection fraction are equal to 7.9 ml, 5.4 ml, and 6.6%, respectively. These results are encouraging and point out additional perspectives for further improvement.

Tongue Contour Tracking and Segmentation in Lingual Ultrasound for Speech Recognition: A Review

Lingual ultrasound imaging is essential in linguistic research and speech recognition. It has been used widely in different applications as visual feedback to enhance language learning for non-native speakers, study speech-related disorders and remediation, articulation research and analysis, swallowing study, tongue 3D modelling, and silent speech interface. This article provides a comparative analysis and review based on quantitative and qualitative criteria of the two main streams of tongue contour segmentation from ultrasound images. The first stream utilizes traditional computer vision and image processing algorithms for tongue segmentation. The second stream uses machine and deep learning algorithms for tongue segmentation. The results show that tongue tracking using machine learning-based techniques is superior to traditional techniques, considering the performance and algorithm generalization ability. Meanwhile, traditional techniques are helpful for implementing interactive image segmentation to extract valuable features during training and postprocessing. We recommend using a hybrid approach to combine machine learning and traditional techniques to implement a real-time tongue segmentation tool.

C-Net: Cascaded convolutional neural network with global guidance and refinement residuals for breast ultrasound images segmentation

BACKGROUND AND OBJECTIVE

Breast lesions segmentation is an important step of computer-aided diagnosis system. However, speckle noise, heterogeneous structure, and similar intensity distributions bring challenges for breast lesion segmentation.

METHODS

In this paper, we presented a novel cascaded convolutional neural network integrating U-net, bidirectional attention guidance network (BAGNet) and refinement residual network (RFNet) for the lesion segmentation in breast ultrasound images. Specifically, we first use U-net to generate a set of saliency maps containing low-level and high-level image structures. Then, the bidirectional attention guidance network is used to capture the context between global (low-level) and local (high-level) features from the saliency map. The introduction of the global feature map can reduce the interference of surrounding tissue on the lesion regions. Furthermore, we developed a refinement residual network based on the core architecture of U-net to learn the difference between rough saliency feature maps and ground-truth masks. The learning of residuals can assist us to obtain a more complete lesion mask.

RESULTS

To evaluate the segmentation performance of the network, we compared with several state-of-the-art segmentation methods on the public breast ultrasound dataset (BUSIS) using six commonly used evaluation metrics. Our method achieves the highest scores on six metrics. Furthermore, p-values indicate significant differences between our method and the comparative methods.

CONCLUSIONS

Experimental results show that our method achieves the most competitive segmentation results. In addition, we apply the network on renal ultrasound images segmentation. In general, our method has good adaptability and robustness on ultrasound image segmentation.

Structure boundary-preserving U-Net for prostate ultrasound image segmentation

Prostate cancer diagnosis is performed under ultrasound-guided puncture for pathological cell extraction. However, determining accurate prostate location remains a challenge from two aspects: (1) prostate boundary in ultrasound images is always ambiguous; (2) the delineation of radiologists always occupies multiple pixels, leading to many disturbing points around the actual contour. We proposed a boundary structure-preserving U-Net (BSP U-Net) in this paper to achieve precise prostate contour. BSP U-Net incorporates prostate shape prior to traditional U-Net. The prior shape is built by the key point selection module, which is an active shape model-based method. Then, the module plugs into the traditional U-Net structure network to achieve prostate segmentation. The experiments were conducted on two datasets: PH2 + ISBI 2016 challenge and our private prostate ultrasound dataset. The results on PH2 + ISBI 2016 challenge achieved a Dice similarity coefficient (DSC) of 95.94% and a Jaccard coefficient (JC) of 88.58%. The results of prostate contour based on our method achieved a higher pixel accuracy of 97.05%, a mean intersection over union of 93.65%, a DSC of 92.54%, and a JC of 93.16%. The experimental results show that the proposed BSP U-Net has good performance on PH2 + ISBI 2016 challenge and prostate ultrasound image segmentation and outperforms other state-of-the-art methods.

MallesNet: A multi-object assistance based network for brachial plexus segmentation in ultrasound images

Ultrasound-guided injection is widely used to help anesthesiologists perform anesthesia in peripheral nerve blockade (PNB). However, it is a daunting task to accurately identify nerve structure in ultrasound images even for the experienced anesthesiologists. In this paper, a Multi-object assistance based Brachial Plexus Segmentation Network, named MallesNet, is proposed to improve the nerve segmentation performance in ultrasound image with the assistance of simultaneously segmenting its surrounding anatomical structures (e.g., muscle, vein, and artery). The MallesNet is designed by following the framework of Mask R-CNN to implement the multi object identification and segmentation. Moreover, a spatial local contrast feature (SLCF) extraction module is proposed to compute contrast features at different scales to effectively obtain useful features for small objects. And the self-attention gate (SAG) is also utilized to capture the spatial relationships in different channels and further re-weight the channels in feature maps by following the design of non-local operation and channel attention. Furthermore, the upsampling mechanism in original Feature Pyramid Network (FPN) is improved by adopting the transpose convolution and skip concatenation to fine-tune the feature maps. The Ultrasound Brachial Plexus Dataset (UBPD) is also proposed to support the research on brachial plexus segmentation, which consists of 1055 ultrasound images with four objects (i.e., nerve, artery, vein and muscle) and their corresponding label masks. Extensive experimental results using UBPD dataset demonstrate that MallesNet can achieve a better segmentation performance on nerves structure and also on surrounding structures in comparison to other competing approaches.

FCRB U-Net: A novel fully connected residual block U-Net for fetal cerebellum ultrasound image segmentation

In this paper, we propose a novel U-Net with fully connected residual blocks (FCRB U-Net) for the fetal cerebellum Ultrasound image segmentation task. FCRB U-Net, an improved convolutional neural network (CNN) based on U-Net, replaces the double convolution operation in the original model with the fully connected residual block and embeds an effective channel attention module to enhance the extraction of valid features. Moreover, in the decoding stage, a feature reuse module is employed to form a fully connected decoder to make full use of deep features. FCRB U-Net can effectively alleviate the problem of the loss of feature information during the convolution process and improve segmentation accuracy. Experimental results demonstrate that the proposed approach is effective and promising in the field of fetal cerebellar segmentation in actual Ultrasound images. The average IoU value and mean Dice index reach 86.72% and 90.45%, respectively, which are 3.07% and 5.25% higher than that of the basic U-Net.