Global guidance network for breast lesion segmentation in ultrasound images

Flip Learning: Weakly supervised erase to segment nodules in breast ultrasound

Accurate segmentation of nodules in both 2D breast ultrasound (BUS) and 3D automated breast ultrasound (ABUS) is crucial for clinical diagnosis and treatment planning. Therefore, developing an automated system for nodule segmentation can enhance user independence and expedite clinical analysis. Unlike fully-supervised learning, weakly-supervised segmentation (WSS) can streamline the laborious and intricate annotation process. However, current WSS methods face challenges in achieving precise nodule segmentation, as many of them depend on inaccurate activation maps or inefficient pseudo-mask generation algorithms. In this study, we introduce a novel multi-agent reinforcement learning-based WSS framework called Flip Learning, which relies solely on 2D/3D boxes for accurate segmentation. Specifically, multiple agents are employed to erase the target from the box to facilitate classification tag flipping, with the erased region serving as the predicted segmentation mask. The key contributions of this research are as follows: (1) Adoption of a superpixel/supervoxel-based approach to encode the standardized environment, capturing boundary priors and expediting the learning process. (2) Introduction of three meticulously designed rewards, comprising a classification score reward and two intensity distribution rewards, to steer the agents' erasing process precisely, thereby avoiding both under- and over-segmentation. (3) Implementation of a progressive curriculum learning strategy to enable agents to interact with the environment in a progressively challenging manner, thereby enhancing learning efficiency. Extensively validated on the large in-house BUS and ABUS datasets, our Flip Learning method outperforms state-of-the-art WSS methods and foundation models, and achieves comparable performance as fully-supervised learning algorithms.

FET-UNet: Merging CNN and transformer architectures for superior breast ultrasound image segmentation

PURPOSE

Breast cancer remains a significant cause of mortality among women globally, highlighting the critical need for accurate diagnosis. Although Convolutional Neural Networks (CNNs) have shown effectiveness in segmenting breast ultrasound images, they often face challenges in capturing long-range dependencies, particularly for lesions with similar intensity distributions, irregular shapes, and blurred boundaries. To overcome these limitations, we introduce FET-UNet, a novel hybrid framework that integrates CNNs and Swin Transformers within a UNet-like architecture.

METHODS

FET-UNet features parallel branches for feature extraction: one utilizes ResNet34 blocks, and the other employs Swin Transformer blocks. These branches are fused using an advanced feature aggregation module (AFAM), enabling the network to effectively combine local details and global context. Additionally, we include a multi-scale upsampling mechanism in the decoder to ensure precise segmentation outputs. This design enhances the capture of both local details and long-range dependencies.

RESULTS

Extensive evaluations on the BUSI, UDIAT, and BLUI datasets demonstrate the superior performance of FET-UNet compared to state-of-the-art methods. The model achieves Dice coefficients of 82.9% on BUSI, 88.9% on UDIAT, and 90.1% on BLUI.

CONCLUSION

FET-UNet shows great potential to advance breast ultrasound image segmentation and support more precise clinical diagnoses. Further research could explore the application of this framework to other medical imaging modalities and its integration into clinical workflows.

Multi-scale multi-object semi-supervised consistency learning for ultrasound image segmentation

Manual annotation of ultrasound images relies on expert knowledge and requires significant time and financial resources. Semi-supervised learning (SSL) exploits large amounts of unlabeled data to improve model performance under limited labeled data. However, it faces two challenges: fusion of contextual information at multiple scales and bias of spatial information between multiple objects. We propose a consistency learning-based multi-scale multi-object (MSMO) semi-supervised framework for ultrasound image segmentation. MSMO addresses these challenges by employing a contextual-aware encoder coupled with a multi-object semantic calibration and fusion decoder. First, the encoder extracts multi-scale multi-objects context-aware features, and introduces attention module to refine the feature map and enhance channel information interaction. Then, the decoder uses HConvLSTM to calibrate the output features of the current object by using the hidden state of the previous object, and recursively fuses multi-object semantics at different scales. Finally, MSMO further reduces variations among multiple decoders in different perturbations through consistency constraints, thereby producing consistent predictions for highly uncertain areas. Extensive experiments show that proposed MSMO outperforms the SSL baseline on four benchmark datasets, whether for single-object or multi-object ultrasound image segmentation. MSMO significantly reduces the burden of manual analysis of ultrasound images and holds great potential as a clinical tool. The source code is accessible to the public at: https://github.com/lol88/MSMO.

Liver lesion segmentation in ultrasound: A benchmark and a baseline network

Accurate liver lesion segmentation in ultrasound is a challenging task due to high speckle noise, ambiguous lesion boundaries, and inhomogeneous intensity distribution inside the lesion regions. This work first collected and annotated a dataset for liver lesion segmentation in ultrasound. In this paper, we propose a novel convolutional neural network to learn dual self-attentive transformer features for boosting liver lesion segmentation by leveraging the complementary information among non-local features encoded at different layers of the transformer architecture. To do so, we devise a dual self-attention refinement (DSR) module to synergistically utilize self-attention and reverse self-attention mechanisms to extract complementary lesion characteristics between cascaded multi-layer feature maps, assisting the model to produce more accurate segmentation results. Moreover, we propose a False-Positive-Negative loss to enable our network to further suppress the non-liver-lesion noise at shallow transformer layers and enhance more target liver lesion details into CNN features at deep transformer layers. Experimental results show that our network outperforms state-of-the-art methods quantitatively and qualitatively.

Multi-task interaction learning for accurate segmentation and classification of breast tumors in ultrasound images

Objective.In breast diagnostic imaging, the morphological variability of breast tumors and the inherent ambiguity of ultrasound images pose significant challenges. Moreover, multi-task computer-aided diagnosis systems in breast imaging may overlook inherent relationships between pixel-wise segmentation and categorical classification tasks.Approach.In this paper, we propose a multi-task learning network with deep inter-task interactions that exploits the inherently relations between two tasks. First, we fuse self-task attention and cross-task attention mechanisms to explore the two types of interaction information, location and semantic, between tasks. In addition, a feature aggregation block is developed based on the channel attention mechanism, which reduces the semantic differences between the decoder and the encoder. To exploit inter-task further, our network uses an circle training strategy to refine heterogeneous feature with the help of segmentation maps obtained from previous training.Main results.The experimental results show that our method achieved excellent performance on the BUSI and BUS-B datasets, with DSCs of 81.95% and 86.41% for segmentation tasks, and F1 scores of 82.13% and 69.01% for classification tasks, respectively.Significance.The proposed multi-task interaction learning not only enhances the performance of all tasks related to breast tumor segmentation and classification but also promotes research in multi-task learning, providing further insights for clinical applications.

Automated Segmentation of Breast Cancer Focal Lesions on Ultrasound Images

Ultrasound (US) remains the main modality for the differential diagnosis of changes revealed by mammography. However, the US images themselves are subject to various types of noise and artifacts from reflections, which can worsen the quality of their analysis. Deep learning methods have a number of disadvantages, including the often insufficient substantiation of the model, and the complexity of collecting a representative training database. Therefore, it is necessary to develop effective algorithms for the segmentation, classification, and analysis of US images. The aim of the work is to develop a method for the automated detection of pathological lesions in breast US images and their segmentation. A method is proposed that includes two stages of video image processing: (1) searching for a region of interest using a random forest classifier, which classifies normal tissues, (2) selecting the contour of the lesion based on the difference in brightness of image pixels. The test set included 52 ultrasound videos which contained histologically proven suspicious lesions. The average frequency of lesion detection per frame was 91.89%, and the average accuracy of contour selection according to the IoU metric was 0.871. The proposed method can be used to segment a suspicious lesion.

A multi-task framework for breast cancer segmentation and classification in ultrasound imaging

BACKGROUND

Ultrasound (US) is a medical imaging modality that plays a crucial role in the early detection of breast cancer. The emergence of numerous deep learning systems has offered promising avenues for the segmentation and classification of breast cancer tumors in US images. However, challenges such as the absence of data standardization, the exclusion of non-tumor images during training, and the narrow view of single-task methodologies have hindered the practical applicability of these systems, often resulting in biased outcomes. This study aims to explore the potential of multi-task systems in enhancing the detection of breast cancer lesions.

METHODS

To address these limitations, our research introduces an end-to-end multi-task framework designed to leverage the inherent correlations between breast cancer lesion classification and segmentation tasks. Additionally, a comprehensive analysis of a widely utilized public breast cancer ultrasound dataset named BUSI was carried out, identifying its irregularities and devising an algorithm tailored for detecting duplicated images in it.

RESULTS

Experiments are conducted utilizing the curated dataset to minimize potential biases in outcomes. Our multi-task framework exhibits superior performance in breast cancer respecting single-task approaches, achieving improvements close to 15% in segmentation and classification. Moreover, a comparative analysis against the state-of-the-art reveals statistically significant enhancements across both tasks.

CONCLUSION

The experimental findings underscore the efficacy of multi-task techniques, showcasing better generalization capabilities when considering all image types: benign, malignant, and non-tumor images. Consequently, our methodology represents an advance towards more general architectures with real clinical applications in the breast cancer field.

CMR-BENet: A confidence map refinement boundary enhancement network for left ventricular myocardium segmentation

BACKGROUND AND OBJECTIVE

Left ventricular myocardium segmentation is of great significance for clinical diagnosis, treatment, and prognosis. However, myocardium segmentation is challenging as the medical image quality is disturbed by various factors such as motion, artifacts, and noise. Its accuracy largely depends on the accurate identification of edges and structures. Most existing encoder-decoder based segmentation methods capture limited contextual information and ignore the awareness of myocardial shape and structure, often producing unsatisfactory boundary segmentation results in noisy scenes. Moreover, these methods fail to assess the reliability of the predictions, which is crucial for clinical decisions and applications in medical tasks. Therefore, this study explores how to effectively combine contextual information with myocardial edge structure and confidence maps to improve segmentation performance in an end-to-end network.

METHODS

In this paper, we propose an end-to-end confidence map refinement boundary enhancement network (CMR-BENet) for left ventricular myocardium segmentation. CMR-BENet has three components: a layer semantic-aware module (LSA), an edge information enhancement module (EIE), and a confidence map-based refinement module (CMR). Specifically, LSA first adaptively fuses high- and low-level semantic information across hierarchical layers to mitigate the bias of single-layer features affected by noise. EIE then improves the edge and structure recognition by designing the edge and mask guidance module (EMG) and the edge structure-aware module (ESA). Finally, CMR provides a simple and efficient way to estimate confidence maps and effectively combines the encoder features to refine the segmentation results.

RESULTS

Experiments on two echocardiography datasets and one cardiac MRI dataset show that the proposed CMR-BENet outperforms its rivals in the left ventricular myocardium segmentation task with Dice (DI) of 87.71%, 79.33%, and 89.11%, respectively.

CONCLUSION

This paper utilizes edge information to characterize the shape and structure of the myocardium and introduces learnable confidence maps to evaluate and refine the segmentation results. Our findings provide strong support and reference for physicians in diagnosis and treatment.

Ensemble fuzzy deep learning for brain tumor detection

This research presents a novel ensemble fuzzy deep learning approach for brain Magnetic Resonance Imaging (MRI) analysis, aiming to improve the segmentation of brain tissues and abnormalities. The method integrates multiple components, including diverse deep learning architectures enhanced with volumetric fuzzy pooling, a model fusion strategy, and an attention mechanism to focus on the most relevant regions of the input data. The process begins by collecting medical data using sensors to acquire MRI images. These data are then used to train several deep learning models that are specifically designed to handle various aspects of brain MRI segmentation. To enhance the model's performance, an efficient ensemble learning method is employed to combine the predictions of multiple models, ensuring that the final decision accounts for different strengths of each individual model. A key feature of the approach is the construction of a knowledge base that stores data from training images and associates it with the most suitable model for each specific sample. During the inference phase, this knowledge base is consulted to quickly identify and select the best model for processing new test images, based on the similarity between the test data and previously encountered samples. The proposed method is rigorously tested on real-world brain MRI segmentation benchmarks, demonstrating superior performance in comparison to existing techniques. Our proposed method achieves an Intersection over Union (IoU) of 95% on the complete Brain MRI Segmentation dataset, demonstrating a 10% improvement over baseline solutions.

BCT-Net: semantic-guided breast cancer segmentation on BUS

Accurately and swiftly segmenting breast tumors is significant for cancer diagnosis and treatment. Ultrasound imaging stands as one of the widely employed methods in clinical practice. However, due to challenges such as low contrast, blurred boundaries, and prevalent shadows in ultrasound images, tumor segmentation remains a daunting task. In this study, we propose BCT-Net, a network amalgamating CNN and transformer components for breast tumor segmentation. BCT-Net integrates a dual-level attention mechanism to capture more features and redefines the skip connection module. We introduce the utilization of a classification task as an auxiliary task to impart additional semantic information to the segmentation network, employing supervised contrastive learning. A hybrid objective loss function is proposed, which combines pixel-wise cross-entropy, binary cross-entropy, and supervised contrastive learning loss. Experimental results demonstrate that BCT-Net achieves high precision, with Pre and DSC indices of 86.12% and 88.70%, respectively. Experiments conducted on the BUSI dataset of breast ultrasound images manifest that this approach exhibits high accuracy in breast tumor segmentation.

Deep Learning in Breast Cancer Imaging: A Decade of Progress and Future Directions

Breast cancer has reached the highest incidence rate worldwide among all malignancies since 2020. Breast imaging plays a significant role in early diagnosis and intervention to improve the outcome of breast cancer patients. In the past decade, deep learning has shown remarkable progress in breast cancer imaging analysis, holding great promise in interpreting the rich information and complex context of breast imaging modalities. Considering the rapid improvement in deep learning technology and the increasing severity of breast cancer, it is critical to summarize past progress and identify future challenges to be addressed. This paper provides an extensive review of deep learning-based breast cancer imaging research, covering studies on mammograms, ultrasound, magnetic resonance imaging, and digital pathology images over the past decade. The major deep learning methods and applications on imaging-based screening, diagnosis, treatment response prediction, and prognosis are elaborated and discussed. Drawn from the findings of this survey, we present a comprehensive discussion of the challenges and potential avenues for future research in deep learning-based breast cancer imaging.

A Cross-scale Attention-Based U-Net for Breast Ultrasound Image Segmentation

Breast cancer remains a significant global health concern and is a leading cause of mortality among women. The accuracy of breast cancer diagnosis can be greatly improved with the assistance of automatic segmentation of breast ultrasound images. Research has demonstrated the effectiveness of convolutional neural networks (CNNs) and transformers in segmenting these images. Some studies combine transformers and CNNs, using the transformer's ability to exploit long-distance dependencies to address the limitations inherent in convolutional neural networks. Many of these studies face limitations due to the forced integration of transformer blocks into CNN architectures. This approach often leads to inconsistencies in the feature extraction process, ultimately resulting in suboptimal performance for the complex task of medical image segmentation. This paper presents CSAU-Net, a cross-scale attention-guided U-Net, which is a combined CNN-transformer structure that leverages the local detail depiction of CNNs and the ability of transformers to handle long-distance dependencies. To integrate global context data, we propose a cross-scale cross-attention transformer block that is embedded within the skip connections of the U-shaped architectural network. To further enhance the effectiveness of the segmentation process, we incorporated a gated dilated convolution (GDC) module and a lightweight channel self-attention transformer (LCAT) on the encoder side. Extensive experiments conducted on three open-source datasets demonstrate that our CSAU-Net surpasses state-of-the-art techniques in segmenting ultrasound breast lesions.

CBAM-RIUnet: Breast Tumor Segmentation With Enhanced Breast Ultrasound and Test-Time Augmentation

This study addresses the challenge of precise breast tumor segmentation in ultrasound images, crucial for effective Computer-Aided Diagnosis (CAD) in breast cancer. We introduce CBAM-RIUnet, a deep learning (DL) model for automated breast tumor segmentation in breast ultrasound (BUS) images. The model, featuring an efficient convolutional block attention module residual inception Unet, outperforms existing models, particularly excelling in Dice and IoU scores. CBAM-RIUnet follows the Unet structure with a residual inception depth-wise separable convolution, and incorporates a convolutional block attention module (CBAM) to eliminate irrelevant features and focus on the region of interest. Evaluation under three scenarios, including enhanced breast ultrasound (EBUS) and test-time augmentation (TTA), demonstrates impressive results. CBAM-RIUnet achieves Dice and IoU scores of 89.38% and 88.71%, respectively, showcasing significant improvements compared to state-of-the-art DL techniques. In conclusion, CBAM-RIUnet presents a highly effective and simplified DL model for breast tumor segmentation in BUS imaging.

A Multi-Task Transformer With Local-Global Feature Interaction and Multiple Tumoral Region Guidance for Breast Cancer Diagnosis

Breast cancer, as a malignant tumor disease, has maintained high incidence and mortality rates over the years. Ultrasonography is one of the primary methods for diagnosing early-stage breast cancer. However, correctly interpreting breast ultrasound images requires massive time from physicians with specialized knowledge and extensive experience. Recently, deep learning-based method have made significant advancements in breast tumor segmentation and classification due to their powerful fitting capabilities. However, most existing methods focus on performing one of these tasks separately, and often failing to effectively leverage information from specific tumor-related areas that hold considerable diagnostic value. In this study, we propose a multi-task network with local-global feature interaction and multiple tumoral region guidance for breast ultrasound-based tumor segmentation and classification. Specifically, we construct a dual-stream encoder, paralleling CNN and Transformer, to facilitate hierarchical interaction and fusion of local and global features. This architecture enables each stream to capitalize on the strengths of the other while preserving its unique characteristics. Moreover, we design a multi-tumoral region guidance module to explicitly learn long-range non-local dependencies within intra-tumoral and peri-tumoral regions from spatial domain, thus providing interpretable cues beneficial for classification. Experimental results on two breast ultrasound datasets show that our network outperforms state-of-the-art methods in tumor segmentation and classification tasks. Compared with the second-best competitive method, our network improves the diagnosis accuracy from 73.64% to 80.21% on a large external validation dataset, which demonstrates its superior generalization capability.

Advances in Medical Image Segmentation: A Comprehensive Review of Traditional, Deep Learning and Hybrid Approaches

Medical image segmentation plays a critical role in accurate diagnosis and treatment planning, enabling precise analysis across a wide range of clinical tasks. This review begins by offering a comprehensive overview of traditional segmentation techniques, including thresholding, edge-based methods, region-based approaches, clustering, and graph-based segmentation. While these methods are computationally efficient and interpretable, they often face significant challenges when applied to complex, noisy, or variable medical images. The central focus of this review is the transformative impact of deep learning on medical image segmentation. We delve into prominent deep learning architectures such as Convolutional Neural Networks (CNNs), Fully Convolutional Networks (FCNs), U-Net, Recurrent Neural Networks (RNNs), Adversarial Networks (GANs), and Autoencoders (AEs). Each architecture is analyzed in terms of its structural foundation and specific application to medical image segmentation, illustrating how these models have enhanced segmentation accuracy across various clinical contexts. Finally, the review examines the integration of deep learning with traditional segmentation methods, addressing the limitations of both approaches. These hybrid strategies offer improved segmentation performance, particularly in challenging scenarios involving weak edges, noise, or inconsistent intensities. By synthesizing recent advancements, this review provides a detailed resource for researchers and practitioners, offering valuable insights into the current landscape and future directions of medical image segmentation.

Attention based UNet model for breast cancer segmentation using BUSI dataset

Breast cancer, a prevalent and life-threatening disease, necessitates early detection for the effective intervention and the improved patient health outcomes. This paper focuses on the critical problem of identifying breast cancer using a model called Attention U-Net. The model is utilized on the Breast Ultrasound Image Dataset (BUSI), comprising 780 breast images. The images are categorized into three distinct groups: 437 cases classified as benign, 210 cases classified as malignant, and 133 cases classified as normal. The proposed model leverages the attention-driven U-Net's encoder blocks to capture hierarchical features effectively. The model comprises four decoder blocks which is a pivotal component in the U-Net architecture, responsible for expanding the encoded feature representation obtained from the encoder block and for reconstructing spatial information. Four attention gates are incorporated strategically to enhance feature localization during decoding, showcasing a sophisticated design that facilitates accurate segmentation of breast tumors in ultrasound images. It displays its efficacy in accurately delineating and segregating tumor borders. The experimental findings demonstrate outstanding performance, achieving an overall accuracy of 0.98, precision of 0.97, recall of 0.90, and a dice score of 0.92. It demonstrates its effectiveness in precisely defining and separating tumor boundaries. This research aims to make automated breast cancer segmentation algorithms by emphasizing the importance of early detection in boosting diagnostic capabilities and enabling prompt and targeted medical interventions.

NeighborNet: Learning Intra- and Inter-Image Pixel Neighbor Representation for Breast Lesion Segmentation

Breast lesion segmentation from ultrasound images is essential in computer-aided breast cancer diagnosis. To alleviate the problems of blurry lesion boundaries and irregular morphologies, common practices combine CNN and attention to integrate global and local information. However, previous methods use two independent modules to extract global and local features separately, such feature-wise inflexible integration ignores the semantic gap between them, resulting in representation redundancy/insufficiency and undesirable restrictions in clinic practices. Moreover, medical images are highly similar to each other due to the imaging methods and human tissues, but the captured global information by transformer-based methods in the medical domain is limited within images, the semantic relations and common knowledge across images are largely ignored. To alleviate the above problems, in the neighbor view, this paper develops a pixel neighbor representation learning method (NeighborNet) to flexibly integrate global and local context within and across images for lesion morphology and boundary modeling. Concretely, we design two neighbor layers to investigate two properties (i.e., number and distribution) of neighbors. The neighbor number for each pixel is not fixed but determined by itself. The neighbor distribution is extended from one image to all images in the datasets. With the two properties, for each pixel at each feature level, the proposed NeighborNet can evolve into the transformer or degenerate into the CNN for adaptive context representation learning to cope with the irregular lesion morphologies and blurry boundaries. The state-of-the-art performances on three ultrasound datasets prove the effectiveness of the proposed NeighborNet.

SC-Unext: A Lightweight Image Segmentation Model with Cellular Mechanism for Breast Ultrasound Tumor Diagnosis

Automatic breast ultrasound image segmentation plays an important role in medical image processing. However, current methods for breast ultrasound segmentation suffer from high computational complexity and large model parameters, particularly when dealing with complex images. In this paper, we take the Unext network as a basis and utilize its encoder-decoder features. And taking inspiration from the mechanisms of cellular apoptosis and division, we design apoptosis and division algorithms to improve model performance. We propose a novel segmentation model which integrates the division and apoptosis algorithms and introduces spatial and channel convolution blocks into the model. Our proposed model not only improves the segmentation performance of breast ultrasound tumors, but also reduces the model parameters and computational resource consumption time. The model was evaluated on the breast ultrasound image dataset and our collected dataset. The experiments show that the SC-Unext model achieved Dice scores of 75.29% and accuracy of 97.09% on the BUSI dataset, and on the collected dataset, it reached Dice scores of 90.62% and accuracy of 98.37%. Meanwhile, we conducted a comparison of the model's inference speed on CPUs to verify its efficiency in resource-constrained environments. The results indicated that the SC-Unext model achieved an inference speed of 92.72 ms per instance on devices equipped only with CPUs. The model's number of parameters and computational resource consumption are 1.46M and 2.13 GFlops, respectively, which are lower compared to other network models. Due to its lightweight nature, the model holds significant value for various practical applications in the medical field.

DFA-UNet: dual-stream feature-fusion attention U-Net for lymph node segmentation in lung cancer diagnosis

In bronchial ultrasound elastography, accurately segmenting mediastinal lymph nodes is of great significance for diagnosing whether lung cancer has metastasized. However, due to the ill-defined margin of ultrasound images and the complexity of lymph node structure, accurate segmentation of fine contours is still challenging. Therefore, we propose a dual-stream feature-fusion attention U-Net (DFA-UNet). Firstly, a dual-stream encoder (DSE) is designed by combining ConvNext with a lightweight vision transformer (ViT) to extract the local information and global information of images; Secondly, we propose a hybrid attention module (HAM) at the bottleneck, which incorporates spatial and channel attention to optimize the features transmission process by optimizing high-dimensional features at the bottom of the network. Finally, the feature-enhanced residual decoder (FRD) is developed to improve the fusion of features obtained from the encoder and decoder, ensuring a more comprehensive integration. Extensive experiments on the ultrasound elasticity image dataset show the superiority of our DFA-UNet over 9 state-of-the-art image segmentation models. Additionally, visual analysis, ablation studies, and generalization assessments highlight the significant enhancement effects of DFA-UNet. Comprehensive experiments confirm the excellent segmentation effectiveness of the DFA-UNet combined attention mechanism for ultrasound images, underscoring its important significance for future research on medical images.

MFMSNet: A Multi-frequency and Multi-scale Interactive CNN-Transformer Hybrid Network for breast ultrasound image segmentation

Breast tumor segmentation in ultrasound images is fundamental for quantitative analysis and plays a crucial role in the diagnosis and treatment of breast cancer. Recently, existing methods have mainly focused on spatial domain implementations, with less attention to the frequency domain. In this paper, we propose a Multi-frequency and Multi-scale Interactive CNN-Transformer Hybrid Network (MFMSNet). Specifically, we utilize Octave convolutions instead of conventional convolutions to effectively separate high-frequency and low-frequency components while reducing computational complexity. Introducing the Multi-frequency Transformer block (MF-Trans) enables efficient interaction between high-frequency and low-frequency information, thereby capturing long-range dependencies. Additionally, we incorporate Multi-scale interactive fusion module (MSIF) to merge high-frequency feature maps of different sizes, enhancing the emphasis on tumor edges by integrating local contextual information. Experimental results demonstrate the superiority of our MFMSNet over seven state-of-the-art methods on two publicly available breast ultrasound datasets and one thyroid ultrasound dataset. In the evaluation of MFMSNet, tests were conducted on the BUSI, BUI, and DDTI datasets, comprising 130 images (BUSI), 47 images (BUI), and 128 images (DDTI) in the respective test sets. Employing a five-fold cross-validation approach, the obtained dice coefficients are as follows: 83.42 % (BUSI), 90.79 % (BUI), and 79.96 % (DDTI). The code is available at https://github.com/wrc990616/MFMSNet.

Spatial and geometric learning for classification of breast tumors from multi-center ultrasound images: a hybrid learning approach

BACKGROUND

Breast cancer is the most common cancer among women, and ultrasound is a usual tool for early screening. Nowadays, deep learning technique is applied as an auxiliary tool to provide the predictive results for doctors to decide whether to make further examinations or treatments. This study aimed to develop a hybrid learning approach for breast ultrasound classification by extracting more potential features from local and multi-center ultrasound data.

METHODS

We proposed a hybrid learning approach to classify the breast tumors into benign and malignant. Three multi-center datasets (BUSI, BUS, OASBUD) were used to pretrain a model by federated learning, then every dataset was fine-tuned at local. The proposed model consisted of a convolutional neural network (CNN) and a graph neural network (GNN), aiming to extract features from images at a spatial level and from graphs at a geometric level. The input images are small-sized and free from pixel-level labels, and the input graphs are generated automatically in an unsupervised manner, which saves the costs of labor and memory space.

RESULTS

The classification AUCROC of our proposed method is 0.911, 0.871 and 0.767 for BUSI, BUS and OASBUD. The balanced accuracy is 87.6%, 85.2% and 61.4% respectively. The results show that our method outperforms conventional methods.

CONCLUSIONS

Our hybrid approach can learn the inter-feature among multi-center data and the intra-feature of local data. It shows potential in aiding doctors for breast tumor classification in ultrasound at an early stage.

Transformative Deep Neural Network Approaches in Kidney Ultrasound Segmentation: Empirical Validation with an Annotated Dataset

Kidney ultrasound (US) images are primarily employed for diagnosing different renal diseases. Among them, one is renal localization and detection, which can be carried out by segmenting the kidney US images. However, kidney segmentation from US images is challenging due to low contrast, speckle noise, fluid, variations in kidney shape, and modality artifacts. Moreover, well-annotated US datasets for renal segmentation and detection are scarce. This study aims to build a novel, well-annotated dataset containing 44,880 US images. In addition, we propose a novel training scheme that utilizes the encoder and decoder parts of a state-of-the-art segmentation algorithm. In the pre-processing step, pixel intensity normalization improves contrast and facilitates model convergence. The modified encoder-decoder architecture improves pyramid-shaped hole pooling, cascaded multiple-hole convolutions, and batch normalization. The pre-processing step gradually reconstructs spatial information, including the capture of complete object boundaries, and the post-processing module with a concave curvature reduces the false positive rate of the results. We present benchmark findings to validate the quality of the proposed training scheme and dataset. We applied six evaluation metrics and several baseline segmentation approaches to our novel kidney US dataset. Among the evaluated models, DeepLabv3+ performed well and achieved the highest dice, Hausdorff distance 95, accuracy, specificity, average symmetric surface distance, and recall scores of 89.76%, 9.91, 98.14%, 98.83%, 3.03, and 90.68%, respectively. The proposed training strategy aids state-of-the-art segmentation models, resulting in better-segmented predictions. Furthermore, the large, well-annotated kidney US public dataset will serve as a valuable baseline source for future medical image analysis research.

Multi-task learning for segmentation and classification of breast tumors from ultrasound images

Segmentation and classification of breast tumors are critical components of breast ultrasound (BUS) computer-aided diagnosis (CAD), which significantly improves the diagnostic accuracy of breast cancer. However, the characteristics of tumor regions in BUS images, such as non-uniform intensity distributions, ambiguous or missing boundaries, and varying tumor shapes and sizes, pose significant challenges to automated segmentation and classification solutions. Many previous studies have proposed multi-task learning methods to jointly tackle tumor segmentation and classification by sharing the features extracted by the encoder. Unfortunately, this often introduces redundant or misleading information, which hinders effective feature exploitation and adversely affects performance. To address this issue, we present ACSNet, a novel multi-task learning network designed to optimize tumor segmentation and classification in BUS images. The segmentation network incorporates a novel gate unit to allow optimal transfer of valuable contextual information from the encoder to the decoder. In addition, we develop the Deformable Spatial Attention Module (DSAModule) to improve segmentation accuracy by overcoming the limitations of conventional convolution in dealing with morphological variations of tumors. In the classification branch, multi-scale feature extraction and channel attention mechanisms are integrated to discriminate between benign and malignant breast tumors. Experiments on two publicly available BUS datasets demonstrate that ACSNet not only outperforms mainstream multi-task learning methods for both breast tumor segmentation and classification tasks, but also achieves state-of-the-art results for BUS tumor segmentation. Code and models are available at https://github.com/qqhe-frank/BUS-segmentation-and-classification.git.

Uncertainty-aware refinement framework for ovarian tumor segmentation in CECT volume

BACKGROUND

Ovarian cancer is a highly lethal gynecological disease. Accurate and automated segmentation of ovarian tumors in contrast-enhanced computed tomography (CECT) images is crucial in the radiotherapy treatment of ovarian cancer, enabling radiologists to evaluate cancer progression and develop timely therapeutic plans. However, automatic ovarian tumor segmentation is challenging due to factors such as inhomogeneous background, ambiguous tumor boundaries, and imbalanced foreground-background, all of which contribute to high predictive uncertainty for a segmentation model.

PURPOSE

To tackle these challenges, we propose an uncertainty-aware refinement framework that aims to estimate and refine regions with high predictive uncertainty for accurate ovarian tumor segmentation in CECT images.

METHODS

To this end, we first employ an approximate Bayesian network to detect coarse regions of interest (ROIs) of both ovarian tumors and uncertain regions. These ROIs allow a subsequent segmentation network to narrow down the search area for tumors and prioritize uncertain regions, resulting in precise segmentation of ovarian tumors. Meanwhile, the framework integrates two guidance modules that learn two implicit functions capable of mapping query features sampled according to their uncertainty to organ or boundary manifolds, guiding the segmentation network to facilitate information encoding of uncertain regions.

RESULTS

Firstly, 367 CECT images are collected from the same hospital for experiments. Dice score, Jaccard, Recall, Positive predictive value (PPV), 95% Hausdorff distance (HD95) and Average symmetric surface distance (ASSD) for the testing group of 77 cases are 86.31%, 73.93%, 83.95%, 86.03%, 15.17  mm and 2.57  mm, all of which are significantly better than that of the other state-of-the-art models. And results of visual comparison shows that the compared methods have more mis-segmentation than our method. Furthermore, our method achieves a Dice score that is at least 20% higher than the Dice scores of other compared methods when tumor volumes are less than 20 cm3 $^3$ , indicating better recognition ability to small regions by our method. And then, 38 CECT images are collected from another hospital to form an external testing group. Our approach consistently outperform the compared methods significantly, with the external testing group exhibiting substantial improvements across key evaluation metrics: Dice score (83.74%), Jaccard (69.55%), Recall (82.12%), PPV (81.61%), HD95 (12.31 mm), and ASSD (2.32 mm), robustly establishing its superior performance.

CONCLUSIONS

Experimental results demonstrate that the framework significantly outperforms the compared state-of-the-art methods, with decreased under- or over-segmentation and better small tumor identification. It has the potential for clinical application.

Slim UNETR: Scale Hybrid Transformers to Efficient 3D Medical Image Segmentation Under Limited Computational Resources

Hybrid transformer-based segmentation approaches have shown great promise in medical image analysis. However, they typically require considerable computational power and resources during both training and inference stages, posing a challenge for resource-limited medical applications common in the field. To address this issue, we present an innovative framework called Slim UNETR, designed to achieve a balance between accuracy and efficiency by leveraging the advantages of both convolutional neural networks and transformers. Our method features the Slim UNETR Block as a core component, which effectively enables information exchange through self-attention mechanism decomposition and cost-effective representation aggregation. Additionally, we utilize the throughput metric as an efficiency indicator to provide feedback on model resource consumption. Our experiments demonstrate that Slim UNETR outperforms state-of-the-art models in terms of accuracy, model size, and efficiency when deployed on resource-constrained devices. Remarkably, Slim UNETR achieves 92.44% dice accuracy on BraTS2021 while being 34.6x smaller and 13.4x faster during inference compared to Swin UNETR. Code: https://github.com/aigzhusmart/Slim-UNETR.

TaiChiNet: Negative-Positive Cross-Attention Network for Breast Lesion Segmentation in Ultrasound Images

Breast lesion segmentation in ultrasound images is essential for computer-aided breast-cancer diagnosis. To improve the segmentation performance, most approaches design sophisticated deep-learning models by mining the patterns of foreground lesions and normal backgrounds simultaneously or by unilaterally enhancing foreground lesions via various focal losses. However, the potential of normal backgrounds is underutilized, which could reduce false positives by compacting the feature representation of all normal backgrounds. From a novel viewpoint of bilateral enhancement, we propose a negative-positive cross-attention network to concentrate on normal backgrounds and foreground lesions, respectively. Derived from the complementing opposites of bipolarity in TaiChi, the network is denoted as TaiChiNet, which consists of the negative normal-background and positive foreground-lesion paths. To transmit the information across the two paths, a cross-attention module, a complementary MLP-head, and a complementary loss are built for deep-layer features, shallow-layer features, and mutual-learning supervision, separately. To the best of our knowledge, this is the first work to formulate breast lesion segmentation as a mutual supervision task from the foreground-lesion and normal-background views. Experimental results have demonstrated the effectiveness of TaiChiNet on two breast lesion segmentation datasets with a lightweight architecture. Furthermore, extensive experiments on the thyroid nodule segmentation and retinal optic cup/disc segmentation datasets indicate the application potential of TaiChiNet.

An efficient framework for lesion segmentation in ultrasound images using global adversarial learning and region-invariant loss

Lesion segmentation in ultrasound images is an essential yet challenging step for early evaluation and diagnosis of cancers. In recent years, many automatic CNN-based methods have been proposed to assist this task. However, most modern approaches often lack capturing long-range dependencies and prior information making it difficult to identify the lesions with unfixed shapes, sizes, locations, and textures. To address this, we present a novel lesion segmentation framework that guides the model to learn the global information about lesion characteristics and invariant features (e.g., morphological features) of lesions to improve the segmentation in ultrasound images. Specifically, the segmentation model is guided to learn the characteristics of lesions from the global maps using an adversarial learning scheme with a self-attention-based discriminator. We argue that under such a lesion characteristics-based guidance mechanism, the segmentation model gets more clues about the boundaries, shapes, sizes, and positions of lesions and can produce reliable predictions. In addition, as ultrasound lesions have different textures, we embed this prior knowledge into a novel region-invariant loss to constrain the model to focus on invariant features for robust segmentation. We demonstrate our method on one in-house breast ultrasound (BUS) dataset and two public datasets (i.e., breast lesion (BUS B) and thyroid nodule from TNSCUI2020). Experimental results show that our method is specifically suitable for lesion segmentation in ultrasound images and can outperform the state-of-the-art approaches with Dice of 0.931, 0.906, and 0.876, respectively. The proposed method demonstrates that it can provide more important information about the characteristics of lesions for lesion segmentation in ultrasound images, especially for lesions with irregular shapes and small sizes. It can assist the current lesion segmentation models to better suit clinical needs.

GLFNet: Global-local fusion network for the segmentation in ultrasound images

Ultrasound imaging, as a portable and radiation-free modality, presents challenges for accurate segmentation due to the variability of lesions and the similar intensity values of surrounding tissues. Current deep learning approaches leverage convolution for extracting local features and self-attention for handling global dependencies. However, traditional CNNs are spatially local, and Vision Transformers lack image specific bias and are computationally demanding. In response, we propose the Global-Local Fusion Network (GLFNet), a hybrid structure addressing the limitations of both CNNs and Vision Transformers. The GLFNet, featuring Global-Local Fusion Blocks (GLFBlocks), integrates global semantic information with local details to improve segmentation. Each GLFBlock comprises Global and Local Branches for feature extraction in parallel. Within the Global and Local Branches, we introduce the Self-Attention Convolution Fusion Block (SACFBlock), which includes a Spatial-Attention Module and Channel-Attention Module. Experimental results show that our proposed GLFNet surpasses its counterparts in the segmentation tasks, achieving the overall best results with an mIoU of 79.58% and Dice coefficient of 74.62% in the DDTI dataset, an mIoU of 76.61% and Dice coefficient of 71.04% in the BUSI dataset, and an mIoU of 86.77% and Dice coefficient of 87.38% in the BUID dataset. The fusion of local and global features contributes to enhanced performance, making GLFNet a promising approach for ultrasound image segmentation.

Curated benchmark dataset for ultrasound based breast lesion analysis

A new detailed dataset of breast ultrasound scans (BrEaST) containing images of benign and malignant lesions as well as normal tissue examples, is presented. The dataset consists of 256 breast scans collected from 256 patients. Each scan was manually annotated and labeled by a radiologist experienced in breast ultrasound examination. In particular, each tumor was identified in the image using a freehand annotation and labeled according to BIRADS features and lexicon. The histopathological classification of the tumor was also provided for patients who underwent a biopsy. The BrEaST dataset is the first breast ultrasound dataset containing patient-level labels, image-level annotations, and tumor-level labels with all cases confirmed by follow-up care or core needle biopsy result. To enable research into breast disease detection, tumor segmentation and classification, the BrEaST dataset is made publicly available with the CC-BY 4.0 license.

BAG-Net: a boundary detection and multiple attention-guided network for liver ultrasound image automatic segmentation in ultrasound guided surgery

Objective.Automated segmentation of targets in ultrasound (US) images during US-guided liver surgery holds the potential to assist physicians in fast locating critical areas such as blood vessels and lesions. However, this remains a challenging task primarily due to the image quality issues associated with US, including blurred edges and low contrast. In addition, studies specifically targeting liver segmentation are relatively scarce possibly since studying deep abdominal organs under US is difficult. In this paper, we proposed a network named BAG-Net to address these challenges and achieve accurate segmentation of liver targets with varying morphologies, including lesions and blood vessels.Approach.The BAG-Net was designed with a boundary detection module together with a position module to locate the target, and multiple attention-guided modules combined with the depth supervision strategy to enhance detailed segmentation of the target area.Main Results.Our method was compared to other approaches and demonstrated superior performance on two liver US datasets. Specifically, the method achieved 93.9% precision, 91.2% recall, 92.4% Dice coefficient, and 86.2% IoU to segment the liver tumor. Additionally, we evaluated the capability of our network to segment tumors on the breast US dataset (BUSI), where it also achieved excellent results.Significance.Our proposed method was validated to effectively segment liver targets with diverse morphologies, providing suspicious areas for clinicians to identify lesions or other characteristics. In the clinic, the method is anticipated to improve surgical efficiency during US-guided surgery.

Rethinking Dual-Stream Super-Resolution Semantic Learning in Medical Image Segmentation

Image segmentation is fundamental task for medical image analysis, whose accuracy is improved by the development of neural networks. However, the existing algorithms that achieve high-resolution performance require high-resolution input, resulting in substantial computational expenses and limiting their applicability in the medical field. Several studies have proposed dual-stream learning frameworks incorporating a super-resolution task as auxiliary. In this paper, we rethink these frameworks and reveal that the feature similarity between tasks is insufficient to constrain vessels or lesion segmentation in the medical field, due to their small proportion in the image. To address this issue, we propose a DS2F (Dual-Stream Shared Feature) framework, including a Shared Feature Extraction Module (SFEM). Specifically, we present Multi-Scale Cross Gate (MSCG) utilizing multi-scale features as a novel example of SFEM. Then we define a proxy task and proxy loss to enable the features focus on the targets based on the assumption that a limited set of shared features between tasks is helpful for their performance. Extensive experiments on six publicly available datasets across three different scenarios are conducted to verify the effectiveness of our framework. Furthermore, various ablation studies are conducted to demonstrate the significance of our DS2F.

DBU-Net: Dual branch U-Net for tumor segmentation in breast ultrasound images

Breast ultrasound medical images often have low imaging quality along with unclear target boundaries. These issues make it challenging for physicians to accurately identify and outline tumors when diagnosing patients. Since precise segmentation is crucial for diagnosis, there is a strong need for an automated method to enhance the segmentation accuracy, which can serve as a technical aid in diagnosis. Recently, the U-Net and its variants have shown great success in medical image segmentation. In this study, drawing inspiration from the U-Net concept, we propose a new variant of the U-Net architecture, called DBU-Net, for tumor segmentation in breast ultrasound images. To enhance the feature extraction capabilities of the encoder, we introduce a novel approach involving the utilization of two distinct encoding paths. In the first path, the original image is employed, while in the second path, we use an image created using the Roberts edge filter, in which edges are highlighted. This dual branch encoding strategy helps to extract the semantic rich information through a mutually informative learning process. At each level of the encoder, both branches independently undergo two convolutional layers followed by a pooling layer. To facilitate cross learning between the branches, a weighted addition scheme is implemented. These weights are dynamically learned by considering the gradient with respect to the loss function. We evaluate the performance of our proposed DBU-Net model on two datasets, namely BUSI and UDIAT, and our experimental results demonstrate superior performance compared to state-of-the-art models.

Boundary-Guided and Region-Aware Network With Global Scale-Adaptive for Accurate Segmentation of Breast Tumors in Ultrasound Images

Breast ultrasound (BUS) image segmentation is a critical procedure in the diagnosis and quantitative analysis of breast cancer. Most existing methods for BUS image segmentation do not effectively utilize the prior information extracted from the images. In addition, breast tumors have very blurred boundaries, various sizes and irregular shapes, and the images have a lot of noise. Thus, tumor segmentation remains a challenge. In this article, we propose a BUS image segmentation method using a boundary-guided and region-aware network with global scale-adaptive (BGRA-GSA). Specifically, we first design a global scale-adaptive module (GSAM) to extract features of tumors of different sizes from multiple perspectives. GSAM encodes the features at the top of the network in both channel and spatial dimensions, which can effectively extract multi-scale context and provide global prior information. Moreover, we develop a boundary-guided module (BGM) for fully mining boundary information. BGM guides the decoder to learn the boundary context by explicitly enhancing the extracted boundary features. Simultaneously, we design a region-aware module (RAM) for realizing the cross-fusion of diverse layers of breast tumor diversity features, which can facilitate the network to improve the learning ability of contextual features of tumor regions. These modules enable our BGRA-GSA to capture and integrate rich global multi-scale context, multi-level fine-grained details, and semantic information to facilitate accurate breast tumor segmentation. Finally, the experimental results on three publicly available datasets show that our model achieves highly effective segmentation of breast tumors even with blurred boundaries, various sizes and shapes, and low contrast.

Prognostic Prediction of Cancer Based on Radiomics Features of Diagnostic Imaging: The Performance of Machine Learning Strategies

Tumor phenotypes can be characterized by radiomics features extracted from images. However, the prediction accuracy is challenged by difficulties such as small sample size and data imbalance. The purpose of the study was to evaluate the performance of machine learning strategies for the prediction of cancer prognosis. A total of 422 patients diagnosed with non-small cell lung carcinoma (NSCLC) were selected from The Cancer Imaging Archive (TCIA). The gross tumor volume (GTV) of each case was delineated from the respective CT images for radiomic features extraction. The samples were divided into 4 groups with survival endpoints of 1 year, 3 years, 5 years, and 7 years. The radiomic image features were analyzed with 6 different machine learning methods: decision tree (DT), boosted tree (BT), random forests (RF), support vector machine (SVM), generalized linear model (GLM), and deep learning artificial neural networks (DL-ANNs) with 70:30 cross-validation. The overall average prediction performance of the BT, RF, DT, SVM, GLM and DL-ANNs was AUC with 0.912, 0.938, 0.793, 0.746, 0.789 and 0.705 respectively. The RF and BT gave the best and second performance in the prediction. The DL-ANN did not show obvious advantage in predicting prognostic outcomes. Deep learning artificial neural networks did not show a significant improvement than traditional machine learning methods such as random forest and boosted trees. On the whole, the accurate outcome prediction using radiomics serves as a supportive reference for formulating treatment strategy for cancer patients.

AAU-Net: An Adaptive Attention U-Net for Breast Lesions Segmentation in Ultrasound Images

Various deep learning methods have been proposed to segment breast lesions from ultrasound images. However, similar intensity distributions, variable tumor morphologies and blurred boundaries present challenges for breast lesions segmentation, especially for malignant tumors with irregular shapes. Considering the complexity of ultrasound images, we develop an adaptive attention U-net (AAU-net) to segment breast lesions automatically and stably from ultrasound images. Specifically, we introduce a hybrid adaptive attention module (HAAM), which mainly consists of a channel self-attention block and a spatial self-attention block, to replace the traditional convolution operation. Compared with the conventional convolution operation, the design of the hybrid adaptive attention module can help us capture more features under different receptive fields. Different from existing attention mechanisms, the HAAM module can guide the network to adaptively select more robust representation in channel and space dimensions to cope with more complex breast lesions segmentation. Extensive experiments with several state-of-the-art deep learning segmentation methods on three public breast ultrasound datasets show that our method has better performance on breast lesions segmentation. Furthermore, robustness analysis and external experiments demonstrate that our proposed AAU-net has better generalization performance in the breast lesion segmentation. Moreover, the HAAM module can be flexibly applied to existing network frameworks. The source code is available on https://github.com/CGPxy/AAU-net.

Graph Flow: Cross-Layer Graph Flow Distillation for Dual Efficient Medical Image Segmentation

With the development of deep convolutional neural networks, medical image segmentation has achieved a series of breakthroughs in recent years. However, high-performance convolutional neural networks always mean numerous parameters and high computation costs, which will hinder the applications in resource-limited medical scenarios. Meanwhile, the scarceness of large-scale annotated medical image datasets further impedes the application of high-performance networks. To tackle these problems, we propose Graph Flow, a comprehensive knowledge distillation framework, for both network-efficiency and annotation-efficiency medical image segmentation. Specifically, the Graph Flow Distillation transfers the essence of cross-layer variations from a well-trained cumbersome teacher network to a non-trained compact student network. In addition, an unsupervised Paraphraser Module is integrated to purify the knowledge of the teacher, which is also beneficial for the training stabilization. Furthermore, we build a unified distillation framework by integrating the adversarial distillation and the vanilla logits distillation, which can further refine the final predictions of the compact network. With different teacher networks (traditional convolutional architecture or prevalent transformer architecture) and student networks, we conduct extensive experiments on four medical image datasets with different modalities (Gastric Cancer, Synapse, BUSI, and CVC-ClinicDB). We demonstrate the prominent ability of our method on these datasets, which achieves competitive performances. Moreover, we demonstrate the effectiveness of our Graph Flow through a novel semi-supervised paradigm for dual efficient medical image segmentation. Our code will be available at Graph Flow.

HCTNet: A hybrid CNN-transformer network for breast ultrasound image segmentation

Automatic breast ultrasound image segmentation helps radiologists to improve the accuracy of breast cancer diagnosis. In recent years, the convolutional neural networks (CNNs) have achieved great success in medical image analysis. However, it exhibits limitations in modeling long-range relations, which is unfavorable for ultrasound images with speckle noise and shadows, resulting in decreased accuracy of breast lesion segmentation. Transformer can obtain sufficient global information, but it is deficient in acquiring local details and needs to be pre-trained on large-scale datasets. In this paper, we propose a Hybrid CNN-Transformer network (HCTNet) for boosting the breast lesion segmentation in ultrasound images. In the encoder of HCTNet, Transformer Encoder Blocks (TEBlocks) are designed to learn the global contextual information, which are combined with CNNs to extract features. In the decoder of HCTNet, a Spatial-wise Cross Attention (SCA) module is developed based on the spatial attention mechanism, which reduces the semantic discrepancy with the encoder. Moreover, residual connection is used between decoder blocks to make the generated features more discriminative by aggregating contextual feature maps at different semantic scales. Extensive experiments on three public breast ultrasound datasets demonstrate that HCTNet outperforms other medical image segmentation methods and the recent semantic segmentation methods on breast ultrasound lesion segmentation.

Residual attention based uncertainty-guided mean teacher model for semi-supervised breast masses segmentation in 2D ultrasonography

Breast tumor is the second deadliest disease among women around the world. Earlier tumor diagnosis is extremely important for improving the survival rate. Recent deep-learning techniques proved helpful in the timely diagnosis of various tumors. However, in the case of breast tumors, the characteristics of the tumors, i.e., low visual contrast, unclear boundary, and diversity in shape and size of breast lesions, make it more challenging to design a highly efficient detection system. Additionally, the scarcity of publicly available labeled data is also a major hurdle in the development of highly accurate and robust deep-learning models for breast tumor detection. To overcome these issues, we propose residual-attention-based uncertainty-guided mean teacher framework which incorporates the residual and attention blocks. The residual for optimizing the deep network by enabling the flow of high-level features and attention modules improves the focus of the model by optimizing its weights during the learning process. We further explore the potential of utilizing unlabeled data during the training process by employing the semi-supervised learning (SSL) method. Particularly, the uncertainty-guided mean-teacher student architecture is exploited to demonstrate the potential of incorporating the unlabeled samples during the training of residual attention U-Net model. The proposed SSL framework has been rigorously evaluated on two publicly available labeled datasets, i.e., BUSI and UDIAT datasets. The quantitative as well as qualitative results demonstrate that the proposed framework achieved competitive performance with respect to the previous state-of-the-art techniques and outperform the existing breast ultrasound masses segmentation techniques. Most importantly, the study demonstrates the potential of incorporating the additional unlabeled data for improving the performance of breast tumor segmentation.

Breast Tumor Classification using Short-ResNet with Pixel-based Tumor Probability Map in Ultrasound Images

Breast cancer is the most common form of cancer and is still the second leading cause of death for women in the world. Early detection and treatment of breast cancer can reduce mortality rates. Breast ultrasound is always used to detect and diagnose breast cancer. The accurate breast segmentation and diagnosis as benign or malignant is still a challenging task in the ultrasound image. In this paper, we proposed a classification model as short-ResNet with DC-UNet to solve the segmentation and diagnosis challenge to find the tumor and classify benign or malignant with breast ultrasonic images. The proposed model has a dice coefficient of 83% for segmentation and achieves an accuracy of 90% for classification with breast tumors. In the experiment, we have compared with segmentation task and classification result in different datasets to prove that the proposed model is more general and demonstrates better results. The deep learning model using short-ResNet to classify tumor whether benign or malignant, that combine DC-UNet of segmentation task to assist in improving the classification results.

A hybrid attentional guidance network for tumors segmentation of breast ultrasound images

PURPOSE

In recent years, breast cancer has become the greatest threat to women. There are many studies dedicated to the precise segmentation of breast tumors, which is indispensable in computer-aided diagnosis. Deep neural networks have achieved accurate segmentation of images. However, convolutional layers are biased to extract local features and tend to lose global and location information as the network deepens, which leads to a decrease in breast tumors segmentation accuracy. For this reason, we propose a hybrid attention-guided network (HAG-Net). We believe that this method will improve the detection rate and segmentation of tumors in breast ultrasound images.

METHODS

The method is equipped with multi-scale guidance block (MSG) for guiding the extraction of low-resolution location information. Short multi-head self-attention (S-MHSA) and convolutional block attention module are used to capture global features and long-range dependencies. Finally, the segmentation results are obtained by fusing multi-scale contextual information.

RESULTS

We compare with 7 state-of-the-art methods on two publicly available datasets through five random fivefold cross-validations. The highest dice coefficient, Jaccard Index and detect rate ([Formula: see text]%, [Formula: see text]%, [Formula: see text]% and [Formula: see text]%, [Formula: see text]%, [Formula: see text]%, separately) obtained on two publicly available datasets(BUSI and OASUBD), prove the superiority of our method.

CONCLUSION

HAG-Net can better utilize multi-resolution features to localize the breast tumors. Demonstrating excellent generalizability and applicability for breast tumors segmentation compare to other state-of-the-art methods.

ATFE-Net: Axial Transformer and Feature Enhancement-based CNN for ultrasound breast mass segmentation

Breast mass is one of the main clinical symptoms of breast cancer. Recently, many CNN-based methods for breast mass segmentation have been proposed. However, these methods have difficulties in capturing long-range dependencies, causing poor segmentation of large-scale breast masses. In this paper, we propose an axial Transformer and feature enhancement-based CNN (ATFE-Net) for ultrasound breast mass segmentation. Specially, an axial Transformer (Axial-Trans) module and a Transformer-based feature enhancement (Trans-FE) module are proposed to capture long-range dependencies. Axial-Trans module only calculates self-attention in width and height directions of input feature maps, which reduces the complexity of self-attention significantly from O(n2) to O(n). In addition, Trans-FE module can enhance feature representation by capturing dependencies between different feature layers, since deeper feature layers have richer semantic information and shallower feature layers have more detailed information. The experimental results show that our ATFE-Net achieved better performance than several state-of-the-art methods on two publicly available breast ultrasound datasets, with Dice coefficient of 82.46% for BUSI and 86.78% for UDIAT, respectively.

Dense-PSP-UNet: A neural network for fast inference liver ultrasound segmentation

Liver Ultrasound (US) or sonography is popularly used because of its real-time output, low-cost, ease-of-use, portability, and non-invasive nature. Segmentation of real-time liver US is essential for diagnosing and analyzing liver conditions (e.g., hepatocellular carcinoma (HCC)), assisting the surgeons/radiologists in therapeutic procedures. In this paper, we propose a method using a modified Pyramid Scene Parsing (PSP) module in tuned neural network backbones to achieve real-time segmentation without compromising the segmentation accuracy. Considering widespread noise in US data and its impact on outcomes, we study the impact of pre-processing and the influence of loss functions on segmentation performance. We have tested our method after annotating a publicly available US dataset containing 2400 images of 8 healthy volunteers (link to the annotated dataset is provided); the results show that the Dense-PSP-UNet model achieves a high Dice coefficient of 0.913±0.024 while delivering a real-time performance of 37 frames per second (FPS).

A multi-task convolutional neural network for classification and segmentation of chronic venous disorders

Chronic Venous Disorders (CVD) of the lower limbs are one of the most prevalent medical conditions, affecting 35% of adults in Europe and North America. Due to the exponential growth of the aging population and the worsening of CVD with age, it is expected that the healthcare costs and the resources needed for the treatment of CVD will increase in the coming years. The early diagnosis of CVD is fundamental in treatment planning, while the monitoring of its treatment is fundamental to assess a patient's condition and quantify the evolution of CVD. However, correct diagnosis relies on a qualitative approach through visual recognition of the various venous disorders, being time-consuming and highly dependent on the physician's expertise. In this paper, we propose a novel automatic strategy for the joint segmentation and classification of CVDs. The strategy relies on a multi-task deep learning network, denominated VENet, that simultaneously solves segmentation and classification tasks, exploiting the information of both tasks to increase learning efficiency, ultimately improving their performance. The proposed method was compared against state-of-the-art strategies in a dataset of 1376 CVD images. Experiments showed that the VENet achieved a classification performance of 96.4%, 96.4%, and 97.2% for accuracy, precision, and recall, respectively, and a segmentation performance of 75.4%, 76.7.0%, 76.7% for the Dice coefficient, precision, and recall, respectively. The joint formulation increased the robustness of both tasks when compared to the conventional classification or segmentation strategies, proving its added value, mainly for the segmentation of small lesions.

BIoMT-ISeg: Blockchain internet of medical things for intelligent segmentation

In the quest of training complicated medical data for Internet of Medical Things (IoMT) scenarios, this study develops an end-to-end intelligent framework that incorporates ensemble learning, genetic algorithms, blockchain technology, and various U-Net based architectures. Genetic algorithms are used to optimize the hyper-parameters of the used architectures. The training process was also protected with the help of blockchain technology. Finally, an ensemble learning system based on voting mechanism was developed to combine local outputs of various segmentation models into a global output. Our method shows that strong performance in a condensed number of epochs may be achieved with a high learning rate and a small batch size. As a result, we are able to perform better than standard solutions for well-known medical databases. In fact, the proposed solution reaches 95% of intersection over the union, compared to the baseline solutions where they are below 80%. Moreover, with the proposed blockchain strategy, the detected attacks reached 76%.

Information bottleneck-based interpretable multitask network for breast cancer classification and segmentation

Breast cancer is one of the most common causes of death among women worldwide. Early signs of breast cancer can be an abnormality depicted on breast images (e.g., mammography or breast ultrasonography). However, reliable interpretation of breast images requires intensive labor and physicians with extensive experience. Deep learning is evolving breast imaging diagnosis by introducing a second opinion to physicians. However, most deep learning-based breast cancer analysis algorithms lack interpretability because of their black box nature, which means that domain experts cannot understand why the algorithms predict a label. In addition, most deep learning algorithms are formulated as a single-task-based model that ignores correlations between different tasks (e.g., tumor classification and segmentation). In this paper, we propose an interpretable multitask information bottleneck network (MIB-Net) to accomplish simultaneous breast tumor classification and segmentation. MIB-Net maximizes the mutual information between the latent representations and class labels while minimizing information shared by the latent representations and inputs. In contrast from existing models, our MIB-Net generates a contribution score map that offers an interpretable aid for physicians to understand the model's decision-making process. In addition, MIB-Net implements multitask learning and further proposes a dual prior knowledge guidance strategy to enhance deep task correlation. Our evaluations are carried out on three breast image datasets in different modalities. Our results show that the proposed framework is not only able to help physicians better understand the model's decisions but also improve breast tumor classification and segmentation accuracy over representative state-of-the-art models. Our code is available at https://github.com/jxw0810/MIB-Net.

Novel automated spinal ultrasound segmentation approach for scoliosis visualization

Scoliosis is a 3D deformity of the spine in which one or more segments of the spine curve laterally, usually with rotation of the vertebral body. Generally, having a Cobb angle (Cobb) greater than 10° can be considered scoliosis. In spine imaging, reliable and accurate identification and segmentation of bony features are crucial for scoliosis assessment, disease diagnosis, and treatment planning. Compared with commonly used X-ray detection methods, ultrasound has received extensive attention from researchers in the past years because of its lack of radiation, high real-time performance, and low price. On the basis of our previous research on spinal ultrasound imaging, this work combines artificial intelligence methods to create a new spine ultrasound image segmentation model called ultrasound global guidance block network (UGBNet), which provides a completely automatic and reliable spine segmentation and scoliosis visualization approach. Our network incorporates a global guidance block module that integrates spatial and channel attention, through which long-range feature dependencies and contextual scale information are learned. We evaluate the performance of the proposed model in semantic segmentation on spinal ultrasound datasets through extensive experiments with several classical learning segmentation methods, such as UNet. Results show that our method performs better than other approaches. Our UGBNet significantly improves segmentation precision, which can reach 74.2% on the evaluation metric of the Dice score.

Ultrasonic breast tumor extraction based on adversarial mechanism and active contour

BACKGROUND AND OBJECTIVE

Breast cancer is a high incidence of gynecological diseases; breast ultrasound screening can effectively reduce the mortality rate of breast cancer. In breast ultrasound images, the localization and segmentation of tumor lesions are important steps for the extraction of lesions, which helps clinicians evaluate breast lesions quantitatively and makes better clinical diagnosis of the disease. However, the segmentation of breast lesions is difficult due to the blurred and uneven edges of some lesions. In this paper, we propose a segmentation framework combining active contour module and deep learning adversarial mechanism and apply it for the segmentation of breast tumor lesions.

METHOD

We use a conditional adversarial network as the main framework. The generator is a segmentation network consisting of a Deformed U-Net and an active contour module. Here, the Deformed U-Net performs pixel-level segmentation for breast ultrasound images. The active contour module refines the tumor lesion edges, and the refined result provides loss information for Deformed U-Net. Therefore, the Deformed U-Net can better classify the edge pixels. The discriminator is the Markov discriminator; this discriminator provides loss feedback for the segmentation network. We cross-train the discriminator and segmentation network to implement Adversarial Mechanism for getting a more optimized segmentation network.

RESULTS

The segmentation performance of the segmentation network for breast ultrasound images is improved by adding a Markov discriminator to provide discriminant loss training. The proposed method for segmenting the tumor lesions in breast ultrasound image obtains dice coefficient: 89.7%, accuracy: 98.1%, precision: 86.3%, mean-intersection-over-union: 82.2%, recall: 94.7%, specificity: 98.5% and F1score: 89.7%.

CONCLUSION

Comparing with traditional methods, the proposed method gives better performance. The experimental results show that the proposed method can effectively segment the lesions in breast ultrasound images, and then assist doctors to realize the diagnosis of breast lesions.