AAU-Net: An Adaptive Attention U-Net for Breast Lesions Segmentation in Ultrasound Images

Semi-Supervised Echocardiography Video Segmentation via Adaptive Spatio-Temporal Tensor Semantic Awareness and Memory Flow

Accurate segmentation of cardiac structures in echocardiography videos is vital for diagnosing heart disease. However, challenges such as speckle noise, low spatial resolution, and incomplete video annotations hinder the accuracy and efficiency of segmentation tasks. Existing video-based segmentation methods mainly utilize optical flow estimation and cross-frame attention to establish pixel-level correlations between frames, which are usually sensitive to noise and have high computational costs. In this paper, we present an innovative echocardiography video segmentation framework that exploits the inherent spatio-temporal correlation of echocardiography video feature tensors. Specifically, we perform adaptive tensor singular value decomposition (t-SVD) on the video semantic feature tensor within a learnable 3D transform domain. By utilizing learnable thresholds, we preserve the principal singular values to reduce redundancy in the high-dimensional spatio-temporal feature tensor and enforce its potential low-rank property. Through this process, we can capture the temporal evolution of the target tissue by effectively utilizing information from limited labeled frames, thus overcoming the constraints of sparse annotations. Furthermore, we introduce a memory flow method that propagates relevant information between adjacent frames based on the multi-scale affinities to precisely resolve frame-to-frame variations of dynamic tissues, thereby improving the accuracy and continuity of segmentation results. Extensive experiments conducted on both public and private datasets validate the superiority of our proposed method over state-of-the-art methods, demonstrating improved performance in echocardiography video segmentation.

AI in Breast Cancer Imaging: An Update and Future Trends

Breast cancer is one of the most common types of cancer affecting women worldwide. Artificial intelligence (AI) is transforming breast cancer imaging by enhancing diagnostic capabilities across multiple imaging modalities including mammography, digital breast tomosynthesis, ultrasound, magnetic resonance imaging, and nuclear medicines techniques. AI is being applied to diverse tasks such as breast lesion detection and classification, risk stratification, molecular subtyping, gene mutation status prediction, and treatment response assessment, with emerging research demonstrating performance levels comparable to or potentially exceeding those of radiologists. The large foundation models are showing remarkable potential in different breast cancer imaging tasks. Self-supervised learning gives an insight into data inherent correlation, and federated learning is an alternative way to maintain data privacy. While promising results have been obtained so far, data standardization from source, large-scale annotated multimodal datasets, and extensive prospective clinical trials are still needed to fully explore and validate deep learning's clinical utility and address the legal and ethical considerations, which will ultimately determine its widespread adoption in breast cancer care. We hereby provide a review of the most up-to-date knowledge on AI in breast cancer imaging.

ICPPNet: A semantic segmentation network model based on inter-class positional prior for scoliosis reconstruction in ultrasound images

OBJECTIVE

Considering the radiation hazard of X-ray, safer, more convenient and cost-effective ultrasound methods are gradually becoming new diagnostic approaches for scoliosis. For ultrasound images of spine regions, it is challenging to accurately identify spine regions in images due to relatively small target areas and the presence of a lot of interfering information. Therefore, we developed a novel neural network that incorporates prior knowledge to precisely segment spine regions in ultrasound images.

MATERIALS AND METHODS

We constructed a dataset of ultrasound images of spine regions for semantic segmentation. The dataset contains 3136 images of 30 patients with scoliosis. And we propose a network model (ICPPNet), which fully utilizes inter-class positional prior knowledge by combining an inter-class positional probability heatmap, to achieve accurate segmentation of target areas.

RESULTS

ICPPNet achieved an average Dice similarity coefficient of 70.83% and an average 95% Hausdorff distance of 11.28 mm on the dataset, demonstrating its excellent performance. The average error between the Cobb angle measured by our method and the Cobb angle measured by X-ray images is 1.41 degrees, and the coefficient of determination is 0.9879 with a strong correlation.

DISCUSSION AND CONCLUSION

ICPPNet provides a new solution for the medical image segmentation task with positional prior knowledge between target classes. And ICPPNet strongly supports the subsequent reconstruction of spine models using ultrasound images.

Semi-supervised medical image segmentation via weak-to-strong perturbation consistency and edge-aware contrastive representation

Despite that supervised learning has demonstrated impressive accuracy in medical image segmentation, its reliance on large labeled datasets poses a challenge due to the effort and expertise required for data acquisition. Semi-supervised learning has emerged as a potential solution. However, it tends to yield satisfactory segmentation performance in the central region of the foreground, but struggles in the edge region. In this paper, we propose an innovative framework that effectively leverages unlabeled data to improve segmentation performance, especially in edge regions. Our proposed framework includes two novel designs. Firstly, we introduce a weak-to-strong perturbation strategy with corresponding feature-perturbed consistency loss to efficiently utilize unlabeled data and guide our framework in learning reliable regions. Secondly, we propose an edge-aware contrastive loss that utilizes uncertainty to select positive pairs, thereby learning discriminative pixel-level features in the edge regions using unlabeled data. In this way, the model minimizes the discrepancy of multiple predictions and improves representation ability, ultimately aiming at impressive performance on both primary and edge regions. We conducted a comparative analysis of the segmentation results on the publicly available BraTS2020 dataset, LA dataset, and the 2017 ACDC dataset. Through extensive quantification and visualization experiments under three standard semi-supervised settings, we demonstrate the effectiveness of our approach and set a new state-of-the-art for semi-supervised medical image segmentation. Our code is released publicly at https://github.com/youngyzzZ/SSL-w2sPC.

A multi-task self-supervised approach for mass detection in automated breast ultrasound using double attention recurrent residual U-Net

Breast cancer is the most common and lethal cancer among women worldwide. Early detection using medical imaging technologies can significantly improve treatment outcomes. Automated breast ultrasound, known as ABUS, offers more advantages compared to traditional mammography and has recently gained considerable attention. However, reviewing hundreds of ABUS slices imposes a high workload on radiologists, increasing review time and potentially leading to diagnostic errors. Consequently, there is a strong need for efficient computer-aided detection, CADe, systems. In recent years, researchers have proposed deep learning-based CADe systems to enhance mass detection accuracy. However, these methods are highly dependent on the number of training samples and often struggle to balance detection accuracy with the false positive rate. To reduce the workload for radiologists and achieve high detection sensitivities with low false positive rates, this study introduces a novel CADe system based on a self-supervised framework that leverages unannotated ABUS datasets to improve detection results. The proposed framework is integrated into an innovative 3-D convolutional neural network called DATTR2U-Net, which employs a multi-task learning approach to simultaneously train inpainting and denoising pretext tasks. A fully convolutional network is then attached to the DATTR2U-Net for the detection task. The proposed method is validated on the TDSCABUS public dataset, demonstrating promising detection results with a recall of 0.7963 and a false positive rate of 5.67 per volume that signifies its potential to improve detection accuracy while reducing workload for radiologists. The code is available at: github.com/Pooryamn/SSL_ABUS.

FRNet V2: A Lightweight Full-Resolution Convolutional Neural Network for OCTA Vessel Segmentation

Optical coherence tomography angiography (OCTA) is an advanced non-invasive imaging technique that can generate three-dimensional images of retinal and choroidal vessels. It is of great value in the diagnosis and monitoring of a variety of ophthalmic diseases. However, most existing methods for blood vessel segmentation in OCTA images rely on an encoder-decoder architecture. This architecture typically involves a large number of parameters and leads to slower inference speeds. To address these challenges and improve segmentation efficiency, this paper proposes a lightweight full-resolution convolutional neural network named FRNet V2 for blood vessel segmentation in OCTA images. FRNet V2 combines the ConvNeXt V2 architecture with deep separable convolution and introduces a recursive mechanism. This mechanism enhances feature representation while reducing the amount of model parameters and computational complexity. In addition, we design a lightweight hybrid adaptive attention mechanism (DWAM) that further improves the segmentation accuracy of the model through the combination of channel self-attention blocks and spatial self-attention blocks. The experimental results show that on two well-known retinal image datasets (OCTA-500 and ROSSA), FRNet V2 can achieve Dice coefficients and accuracy comparable to other methods while reducing the number of parameters by more than 90%. In conclusion, FRNet V2 provides an efficient and lightweight solution for fast and accurate OCTA image blood vessel segmentation in resource-constrained environments, offering strong support for clinical applications.

SAMCT: Segment Any CT Allowing Labor-Free Task-Indicator Prompts

Segment anything model (SAM), a foundation model with superior versatility and generalization across diverse segmentation tasks, has attracted widespread attention in medical imaging. However, it has been proved that SAM would encounter severe performance degradation due to the lack of medical knowledge in training and local feature encoding. Though several SAM-based models have been proposed for tuning SAM in medical imaging, they still suffer from insufficient feature extraction and highly rely on high-quality prompts. In this paper, we propose a powerful foundation model SAMCT allowing labor-free prompts and train it on a collected large CT dataset consisting of 1.1M CT images and 5M masks from public datasets. Specifically, based on SAM, SAMCT is further equipped with a U-shaped CNN image encoder, a cross-branch interaction module, and a task-indicator prompt encoder. The U-shaped CNN image encoder works in parallel with the ViT image encoder in SAM to supplement local features. Cross-branch interaction enhances the feature expression capability of the CNN image encoder and the ViT image encoder by exchanging global perception and local features from one to the other. The task-indicator prompt encoder is a plug-and-play component to effortlessly encode task-related indicators into prompt embeddings. In this way, SAMCT can work in an automatic manner in addition to the semi-automatic interactive strategy in SAM. Extensive experiments demonstrate the superiority of SAMCT against the state-of-the-art task-specific and SAM-based medical foundation models on various tasks. The code, data, and model checkpoints are available at https://github.com/xianlin7/SAMCT.

Syn-Net: A Synchronous Frequency-Perception Fusion Network for Breast Tumor Segmentation in Ultrasound Images

Accurate breast tumor segmentation in ultrasound images is a crucial step in medical diagnosis and locating the tumor region. However, segmentation faces numerous challenges due to the complexity of ultrasound images, similar intensity distributions, variable tumor morphology, and speckle noise. To address these challenges and achieve precise segmentation of breast tumors in complex ultrasound images, we propose a Synchronous Frequency-perception Fusion Network (Syn-Net). Initially, we design a synchronous dual-branch encoder to extract local and global feature information simultaneously from complex ultrasound images. Secondly, we introduce a novel Frequency- perception Cross-Feature Fusion (FrCFusion) Block, which utilizes Discrete Cosine Transform (DCT) to learn all-frequency features and effectively fuse local and global features while mitigating issues arising from similar intensity distributions. In addition, we develop a Full-Scale Deep Supervision method that not only corrects the influence of speckle noise on segmentation but also effectively guides decoder features towards the ground truth. We conduct extensive experiments on three publicly available ultrasound breast tumor datasets. Comparison with 14 state-of-the-art deep learning segmentation methods demonstrates that our approach exhibits superior sensitivity to different ultrasound images, variations in tumor size and shape, speckle noise, and similarity in intensity distribution between surrounding tissues and tumors. On the BUSI and Dataset B datasets, our method achieves better Dice scores compared to state-of-the-art methods, indicating superior performance in ultrasound breast tumor segmentation.

NMTNet: A Multi-task Deep Learning Network for Joint Segmentation and Classification of Breast Tumors

Segmentation and classification of breast tumors are two critical tasks since they provide significant information for computer-aided breast cancer diagnosis. Combining these tasks leverages their intrinsic relevance to enhance performance, but the variability and complexity of tumor characteristics remain challenging. We propose a novel multi-task deep learning network (NMTNet) for the joint segmentation and classification of breast tumors, which is based on a convolutional neural network (CNN) and U-shaped architecture. It mainly comprises a shared encoder, a multi-scale fusion channel refinement (MFCR) module, a segmentation branch, and a classification branch. First, ResNet18 is used as the backbone network in the encoding part to enhance the feature representation capability. Then, the MFCR module is introduced to enrich the feature depth and diversity. Besides, the segmentation branch combines a lesion region enhancement (LRE) module between the encoder and decoder parts, aiming to capture more detailed texture and edge information of irregular tumors to improve segmentation accuracy. The classification branch incorporates a fine-grained classifier that reuses valuable segmentation information to discriminate between benign and malignant tumors. The proposed NMTNet is evaluated on both ultrasound and magnetic resonance imaging datasets. It achieves segmentation dice scores of 90.30% and 91.50%, and Jaccard indices of 84.70% and 88.10% for each dataset, respectively. And the classification accuracy scores are 87.50% and 99.64% for the corresponding datasets, respectively. Experimental results demonstrate the superiority of NMTNet over state-of-the-art methods on breast tumor segmentation and classification tasks.

Mixed attention ensemble for esophageal motility disorders classification

Esophageal motility disorders result from dysfunction of the lower esophageal sphincter and abnormalities in esophageal peristalsis, often presenting symptoms such as dysphagia, chest pain, or heartburn. High-resolution esophageal manometry currently serves as the primary diagnostic method for these disorders, but it has some shortcomings including technical complexity, high demands on diagnosticians, and time-consuming diagnostic process. Therefore, based on ensemble learning with a mixed voting mechanism and multi-dimensional attention enhancement mechanism, a classification model for esophageal motility disorders is proposed and named mixed attention ensemble(MAE) in this paper, which integrates four distinct base models, utilizing a multi-dimensional attention mechanism to extract important features and being weighted with a mixed voting mechanism. We conducted extensive experiments through exploring three different voting strategies and validating our approach on our proprietary dataset. The MAE model outperforms traditional voting ensembles on multiple metrics, achieving an accuracy of 98.48% while preserving a low parameter. The experimental results demonstrate the effectiveness of our method, providing valuable reference to pre-diagnosis for physicians.

BCT-Net: semantic-guided breast cancer segmentation on BUS

Accurately and swiftly segmenting breast tumors is significant for cancer diagnosis and treatment. Ultrasound imaging stands as one of the widely employed methods in clinical practice. However, due to challenges such as low contrast, blurred boundaries, and prevalent shadows in ultrasound images, tumor segmentation remains a daunting task. In this study, we propose BCT-Net, a network amalgamating CNN and transformer components for breast tumor segmentation. BCT-Net integrates a dual-level attention mechanism to capture more features and redefines the skip connection module. We introduce the utilization of a classification task as an auxiliary task to impart additional semantic information to the segmentation network, employing supervised contrastive learning. A hybrid objective loss function is proposed, which combines pixel-wise cross-entropy, binary cross-entropy, and supervised contrastive learning loss. Experimental results demonstrate that BCT-Net achieves high precision, with Pre and DSC indices of 86.12% and 88.70%, respectively. Experiments conducted on the BUSI dataset of breast ultrasound images manifest that this approach exhibits high accuracy in breast tumor segmentation.

Medical image segmentation based on frequency domain decomposition SVD linear attention

Convolutional Neural Networks (CNNs) have achieved remarkable segmentation accuracy in medical image segmentation tasks. However, the Vision Transformer (ViT) model, with its capability of extracting global information, offers a significant advantage in contextual information compared to the limited receptive field of convolutional kernels in CNNs. Despite this, ViT models struggle to fully detect and extract high-frequency signals, such as textures and boundaries, in medical images. These high-frequency features are essential in medical imaging, as targets like tumors and pathological organs exhibit significant differences in texture and boundaries across different stages. Additionally, the high resolution of medical images leads to computational complexity in the self-attention mechanism of ViTs. To address these limitations, we propose a medical image segmentation network framework based on frequency domain decomposition using a Laplacian pyramid. This approach selectively computes attention features for high-frequency signals in the original image to enhance spatial structural information effectively. During attention feature computation, we introduce Singular Value Decomposition (SVD) to extract an effective representation matrix from the original image, which is then applied in the attention computation process for linear projection. This method reduces computational complexity while preserving essential features. We demonstrated the segmentation validity and superiority of our model on the Abdominal Multi-Organ Segmentation dataset and the Dermatological Disease dataset, and on the Synapse dataset our model achieved a score of 82.68 on the Dice metrics and 17.23 mm on the HD metrics. Experimental results indicate that our model consistently exhibits segmentation effectiveness and improved accuracy across various datasets.

PSFHS challenge report: Pubic symphysis and fetal head segmentation from intrapartum ultrasound images

Segmentation of the fetal and maternal structures, particularly intrapartum ultrasound imaging as advocated by the International Society of Ultrasound in Obstetrics and Gynecology (ISUOG) for monitoring labor progression, is a crucial first step for quantitative diagnosis and clinical decision-making. This requires specialized analysis by obstetrics professionals, in a task that i) is highly time- and cost-consuming and ii) often yields inconsistent results. The utility of automatic segmentation algorithms for biometry has been proven, though existing results remain suboptimal. To push forward advancements in this area, the Grand Challenge on Pubic Symphysis-Fetal Head Segmentation (PSFHS) was held alongside the 26th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2023). This challenge aimed to enhance the development of automatic segmentation algorithms at an international scale, providing the largest dataset to date with 5,101 intrapartum ultrasound images collected from two ultrasound machines across three hospitals from two institutions. The scientific community's enthusiastic participation led to the selection of the top 8 out of 179 entries from 193 registrants in the initial phase to proceed to the competition's second stage. These algorithms have elevated the state-of-the-art in automatic PSFHS from intrapartum ultrasound images. A thorough analysis of the results pinpointed ongoing challenges in the field and outlined recommendations for future work. The top solutions and the complete dataset remain publicly available, fostering further advancements in automatic segmentation and biometry for intrapartum ultrasound imaging.

A Neural Network for Segmenting Tumours in Ultrasound Rectal Images

Ultrasound imaging is the most cost-effective approach for the early detection of rectal cancer, which is a high-risk cancer. Our goal was to design an effective method that can accurately identify and segment rectal tumours in ultrasound images, thereby facilitating rectal cancer diagnoses for physicians. This would allow physicians to devote more time to determining whether the tumour is benign or malignant and whether it has metastasized rather than merely confirming its presence. Data originated from the Sichuan Province Cancer Hospital. The test, training, and validation sets were composed of 53 patients with 173 images, 195 patients with 1247 images, and 20 patients with 87 images, respectively. We created a deep learning network architecture consisting of encoders and decoders. To enhance global information capture, we substituted traditional convolutional decoders with global attention decoders and incorporated effective channel information fusion for multiscale information integration. The Dice coefficient (DSC) of the proposed model was 75.49%, which was 4.03% greater than that of the benchmark model, and the Hausdorff distance 95(HD95) was 24.75, which was 8.43 lower than that of the benchmark model. The paired t-test statistically confirmed the significance of the difference between our model and the benchmark model, with a p-value less than 0.05. The proposed method effectively identifies and segments rectal tumours of diverse shapes. Furthermore, it distinguishes between normal rectal images and those containing tumours. Therefore, after consultation with physicians, we believe that our method can effectively assist physicians in diagnosing rectal tumours via ultrasound.

Expansive Receptive Field and Local Feature Extraction Network: Advancing Multiscale Feature Fusion for Breast Fibroadenoma Segmentation in Sonography

Fibroadenoma is a common benign breast disease that affects women of all ages. Early diagnosis can greatly improve the treatment outcomes and reduce the associated pain. Computer-aided diagnosis (CAD) has great potential to improve diagnosis accuracy and efficiency. However, its application in sonography is limited. A network that utilizes expansive receptive fields and local information learning was proposed for the accurate segmentation of breast fibroadenomas in sonography. The architecture comprises the Hierarchical Attentive Fusion module, which conducts local information learning through channel-wise and pixel-wise perspectives, and the Residual Large-Kernel module, which utilizes multiscale large kernel convolution for global information learning. Additionally, multiscale feature fusion in both modules was included to enhance the stability of our network. Finally, an energy function and a data augmentation method were incorporated to fine-tune low-level features of medical images and improve data enhancement. The performance of our model is evaluated using both our local clinical dataset and a public dataset. Mean pixel accuracy (MPA) of 93.93% and 86.06% and mean intersection over union (MIOU) of 88.16% and 73.19% were achieved on the clinical and public datasets, respectively. They are significantly improved over state-of-the-art methods such as SegFormer (89.75% and 78.45% in MPA and 83.26% and 71.85% in MIOU, respectively). The proposed feature extraction strategy, combining local pixel-wise learning with an expansive receptive field for global information perception, demonstrates excellent feature learning capabilities. Due to this powerful and unique local-global feature extraction capability, our deep network achieves superior segmentation of breast fibroadenoma in sonography, which may be valuable in early diagnosis.

LTMSegnet: Lightweight multi-scale medical image segmentation combining Transformer and MLP

Medical image segmentation is currently of a priori guiding significance in medical research and clinical diagnosis. In recent years, neural network-based methods have improved in terms of segmentation accuracy and become the mainstream in the field of medical image segmentation. However, the large number of parameters and computations of prevailing methods currently pose big challenges when employed on mobile devices. While, the lightweight model has great potential to be ported to low-resource hardware devices for its high accuracy. To address the above issues, this paper proposes a lightweight medical image segmentation method combining Transformer and Multi-Layer Perceptron (MLP), aiming to achieve accurate segmentation with lower computational cost. The method consists of a multi-scale branches aggregate module (MBA), a lightweight shift MLP module (LSM) and a feature information share module (FIS). The above three modules are integrated into a U-shaped network. The MBA module learns image features accurately by multi-scale aggregation of global spatial and local detail features. The LSM module introduces shift operations to capture the associations between pixels in different locations in the image. The FIS module interactively fuses multi-stage feature maps acting in skip connections to make the fusion effect finer. The method is validated on ISIC 2018 and 2018 DSB datasets. Experimental results demonstrate that the method outperforms many state-of-the-art lightweight segmentation methods and achieves a balance between segmentation accuracy and computational cost.

A comprehensive benchmarking of a U-Net based model for midbrain auto-segmentation on transcranial sonography

BACKGROUND AND OBJECTIVE

Transcranial sonography-based grading of Parkinson's Disease has gained increasing attention in recent years, and it is currently used for assistive differential diagnosis in some specialized centers. To this end, accurate midbrain segmentation is considered an important initial step. However, current practice is manual, time-consuming, and bias-prone due to the subjective nature. Relevant studies in the literature are scarce and lacks comprehensive model evaluations from application perspectives. Herein, we aimed to benchmark the best-performing U-Net model for objective, stable and robust midbrain auto-segmentation using transcranial sonography images.

METHODS

A total of 584 patients who were suspected of Parkinson's Disease were retrospectively enrolled from Beijing Tiantan Hospital. The dataset was divided into training (n = 416), validation (n = 104), and testing (n = 64) sets. Three state-of-the-art deep-learning networks (U-Net, U-Net+++, and nnU-Net) were utilized to develop segmentation models, under 5-fold cross-validation and three randomization seeds for safeguarding model validity and stability. Model evaluation was conducted in testing set in three key aspects: (i) segmentation agreement using DICE coefficients (DICE), Intersection over Union (IoU), and Hausdorff Distance (HD); (ii) model stability using standard deviations of segmentation agreement metrics; (iii) prediction time efficiency, and (iv) model robustness against various degrees of ultrasound imaging noise produced by the salt-and-pepper noise and Gaussian noise.

RESULTS

The nnU-Net achieved the best segmentation agreement (averaged DICE: 0.910, IoU: 0.836, HD: 2.793-mm) and time efficiency (1.456-s). Under mild noise corruption, the nnU-Net outperformed others with averaged scores of DICE (0.904), IoU (0.827), HD (2.941 mm) in the salt-and-pepper noise (signal-to-noise ratio, SNR = 0.95), and DICE (0.906), IoU (0.830), HD (2.967 mm) in the Gaussian noise (sigma value, σ = 0.1); by contrast, intriguingly, performance of the U-Net and U-Net+++ models were remarkably degraded. Under increasing levels of simulated noise corruption (SNR decreased from 0.95 to 0.75; σ increased from 0.1 to 0.5), the nnU-Net network exhibited marginal decline in segmentation agreement meanwhile yielding decent performance as if there were absence of noise corruption.

CONCLUSIONS

The nnU-Net model was the best-performing midbrain segmentation model in terms of segmentation agreement, stability, time efficiency and robustness, providing the community with an objective, effective and automated alternative. Moving forward, a multi-center multi-vendor study is warranted when it comes to clinical implementation.

A Multi-Task Transformer With Local-Global Feature Interaction and Multiple Tumoral Region Guidance for Breast Cancer Diagnosis

Breast cancer, as a malignant tumor disease, has maintained high incidence and mortality rates over the years. Ultrasonography is one of the primary methods for diagnosing early-stage breast cancer. However, correctly interpreting breast ultrasound images requires massive time from physicians with specialized knowledge and extensive experience. Recently, deep learning-based method have made significant advancements in breast tumor segmentation and classification due to their powerful fitting capabilities. However, most existing methods focus on performing one of these tasks separately, and often failing to effectively leverage information from specific tumor-related areas that hold considerable diagnostic value. In this study, we propose a multi-task network with local-global feature interaction and multiple tumoral region guidance for breast ultrasound-based tumor segmentation and classification. Specifically, we construct a dual-stream encoder, paralleling CNN and Transformer, to facilitate hierarchical interaction and fusion of local and global features. This architecture enables each stream to capitalize on the strengths of the other while preserving its unique characteristics. Moreover, we design a multi-tumoral region guidance module to explicitly learn long-range non-local dependencies within intra-tumoral and peri-tumoral regions from spatial domain, thus providing interpretable cues beneficial for classification. Experimental results on two breast ultrasound datasets show that our network outperforms state-of-the-art methods in tumor segmentation and classification tasks. Compared with the second-best competitive method, our network improves the diagnosis accuracy from 73.64% to 80.21% on a large external validation dataset, which demonstrates its superior generalization capability.

Fast inter-frame motion correction in contrast-free ultrasound quantitative microvasculature imaging using deep learning

Contrast-free ultrasound quantitative microvasculature imaging shows promise in several applications, including the assessment of benign and malignant lesions. However, motion represents one of the major challenges in imaging tumor microvessels in organs that are prone to physiological motions. This study aims at addressing potential microvessel image degradation in in vivo human thyroid due to its proximity to carotid artery. The pulsation of the carotid artery induces inter-frame motion that significantly degrades microvasculature images, resulting in diagnostic errors. The main objective of this study is to reduce inter-frame motion artifacts in high-frame-rate ultrasound imaging to achieve a more accurate visualization of tumor microvessel features. We propose a low-complex deep learning network comprising depth-wise separable convolutional layers and hybrid adaptive and squeeze-and-excite attention mechanisms to correct inter-frame motion in high-frame-rate images. Rigorous validation using phantom and in-vivo data with simulated inter-frame motion indicates average improvements of 35% in Pearson correlation coefficients (PCCs) between motion corrected and reference data with respect to that of motion corrupted data. Further, reconstruction of microvasculature images using motion-corrected frames demonstrates PCC improvement from 31 to 35%. Another thorough validation using in-vivo thyroid data with physiological inter-frame motion demonstrates average improvement of 20% in PCC and 40% in mean inter-frame correlation. Finally, comparison with the conventional image registration method indicates the suitability of proposed network for real-time inter-frame motion correction with 5000 times reduction in motion corrected frame prediction latency.

AI-based automated breast cancer segmentation in ultrasound imaging based on Attention Gated Multi ResU-Net

Breast cancer is a leading cause of death among women worldwide, making early detection and diagnosis critical for effective treatment and improved patient outcomes. Ultrasound imaging is a common diagnostic tool for breast cancer, but interpreting ultrasound images can be challenging due to the complexity of breast tissue and the variability of image quality. This study proposed an Attention Gated Multi ResU-Net model for medical image segmentation tasks, that has shown promising results for breast cancer ultrasound image segmentation. The model's multi-scale feature extraction and attention-gating mechanism enable it to accurately identify and segment areas of abnormality in the breast tissue, such as masses, cysts, and calcifications. The model's quantitative test showed an adequate degree of agreement with expert manual annotations, demonstrating its potential for improving early identification and diagnosis of breast cancer. The model's multi-scale feature extraction and attention-gating mechanism enable it to accurately identify and segment areas of abnormality in the breast tissue, such as masses, cysts, and calcifications, achieving a Dice coefficient of 0.93, sensitivity of 93%, and specificity of 99%. These results underscore the model's high precision and reliability in medical image analysis.

Attention based UNet model for breast cancer segmentation using BUSI dataset

Breast cancer, a prevalent and life-threatening disease, necessitates early detection for the effective intervention and the improved patient health outcomes. This paper focuses on the critical problem of identifying breast cancer using a model called Attention U-Net. The model is utilized on the Breast Ultrasound Image Dataset (BUSI), comprising 780 breast images. The images are categorized into three distinct groups: 437 cases classified as benign, 210 cases classified as malignant, and 133 cases classified as normal. The proposed model leverages the attention-driven U-Net's encoder blocks to capture hierarchical features effectively. The model comprises four decoder blocks which is a pivotal component in the U-Net architecture, responsible for expanding the encoded feature representation obtained from the encoder block and for reconstructing spatial information. Four attention gates are incorporated strategically to enhance feature localization during decoding, showcasing a sophisticated design that facilitates accurate segmentation of breast tumors in ultrasound images. It displays its efficacy in accurately delineating and segregating tumor borders. The experimental findings demonstrate outstanding performance, achieving an overall accuracy of 0.98, precision of 0.97, recall of 0.90, and a dice score of 0.92. It demonstrates its effectiveness in precisely defining and separating tumor boundaries. This research aims to make automated breast cancer segmentation algorithms by emphasizing the importance of early detection in boosting diagnostic capabilities and enabling prompt and targeted medical interventions.

Efficient multi-stage feedback attention for diverse lesion in cancer image segmentation

In the domain of Computer-Aided Diagnosis (CAD) systems, the accurate identification of cancer lesions is paramount, given the life-threatening nature of cancer and the complexities inherent in its manifestation. This task is particularly arduous due to the often vague boundaries of cancerous regions, compounded by the presence of noise and the heterogeneity in the appearance of lesions, making precise segmentation a critical yet challenging endeavor. This study introduces an innovative, an iterative feedback mechanism tailored for the nuanced detection of cancer lesions in a variety of medical imaging modalities, offering a refining phase to adjust detection results. The core of our approach is the elimination of the need for an initial segmentation mask, a common limitation in iterative-based segmentation methods. Instead, we utilize a novel system where the feedback for refining segmentation is derived directly from the encoder-decoder architecture of our neural network model. This shift allows for more dynamic and accurate lesion identification. To further enhance the accuracy of our CAD system, we employ a multi-scale feedback attention mechanism to guide and refine predicted mask subsequent iterations. In parallel, we introduce a sophisticated weighted feedback loss function. This function synergistically combines global and iteration-specific loss considerations, thereby refining parameter estimation and improving the overall precision of the segmentation. We conducted comprehensive experiments across three distinct categories of medical imaging: colonoscopy, ultrasonography, and dermoscopic images. The experimental results demonstrate that our method not only competes favorably with but also surpasses current state-of-the-art methods in various scenarios, including both standard and challenging out-of-domain tasks. This evidences the robustness and versatility of our approach in accurately identifying cancer lesions across a spectrum of medical imaging contexts. Our source code can be found at https://github.com/dewamsa/EfficientFeedbackNetwork.

Progressive Dual Priori Network for Generalized Breast Tumor Segmentation

To promote the generalization ability of breast tumor segmentation models, as well as to improve the segmentation performance for breast tumors with smaller size, low-contrast and irregular shape, we propose a progressive dual priori network (PDPNet) to segment breast tumors from dynamic enhanced magnetic resonance images (DCE-MRI) acquired at different centers. The PDPNet first cropped tumor regions with a coarse-segmentation based localization module, then the breast tumor mask was progressively refined by using the weak semantic priori and cross-scale correlation prior knowledge. To validate the effectiveness of PDPNet, we compared it with several state-of-the-art methods on multi-center datasets. The results showed that, comparing against the suboptimal method, the DSC and HD95 of PDPNet were improved at least by 5.13% and 7.58% respectively on multi-center test sets. In addition, through ablations, we demonstrated that the proposed localization module can decrease the influence of normal tissues and therefore improve the generalization ability of the model. The weak semantic priors allow focusing on tumor regions to avoid missing small tumors and low-contrast tumors. The cross-scale correlation priors are beneficial for promoting the shape-aware ability for irregular tumors. Thus integrating them in a unified framework improved the multi-center breast tumor segmentation performance.

NeighborNet: Learning Intra- and Inter-Image Pixel Neighbor Representation for Breast Lesion Segmentation

Breast lesion segmentation from ultrasound images is essential in computer-aided breast cancer diagnosis. To alleviate the problems of blurry lesion boundaries and irregular morphologies, common practices combine CNN and attention to integrate global and local information. However, previous methods use two independent modules to extract global and local features separately, such feature-wise inflexible integration ignores the semantic gap between them, resulting in representation redundancy/insufficiency and undesirable restrictions in clinic practices. Moreover, medical images are highly similar to each other due to the imaging methods and human tissues, but the captured global information by transformer-based methods in the medical domain is limited within images, the semantic relations and common knowledge across images are largely ignored. To alleviate the above problems, in the neighbor view, this paper develops a pixel neighbor representation learning method (NeighborNet) to flexibly integrate global and local context within and across images for lesion morphology and boundary modeling. Concretely, we design two neighbor layers to investigate two properties (i.e., number and distribution) of neighbors. The neighbor number for each pixel is not fixed but determined by itself. The neighbor distribution is extended from one image to all images in the datasets. With the two properties, for each pixel at each feature level, the proposed NeighborNet can evolve into the transformer or degenerate into the CNN for adaptive context representation learning to cope with the irregular lesion morphologies and blurry boundaries. The state-of-the-art performances on three ultrasound datasets prove the effectiveness of the proposed NeighborNet.

MH2AFormer: An Efficient Multiscale Hierarchical Hybrid Attention With a Transformer for Bladder Wall and Tumor Segmentation

Achieving accurate bladder wall and tumor segmentation from MRI is critical for diagnosing and treating bladder cancer. However, automated segmentation remains challenging due to factors such as comparable density distributions, intricate tumor morphologies, and unclear boundaries. Considering the attributes of bladder MRI images, we propose an efficient multiscale hierarchical hybrid attention with a transformer (MH2AFormer) for bladder cancer and wall segmentation. Specifically, a multiscale hybrid attention and transformer (MHAT) module in the encoder is designed to adaptively extract and aggregate multiscale hybrid feature representations from the input image. In the decoder stage, we devise a multiscale hybrid attention (MHA) module to generate high-quality segmentation results from multiscale hybrid features. Combining these modules enhances the feature representation and guides the model to focus on tumor and wall regions, which helps to solve bladder image segmentation challenges. Moreover, MHAT utilizes the Fast Fourier Transformer with a large kernel (e.g., 224 × 224) to model global feature relationships while reducing computational complexity in the encoding stage. The model performance was evaluated on two datasets. As a result, the model achieves relatively best results regarding the intersection over union (IoU) and dice similarity coefficient (DSC) on both datasets (Dataset A: IoU = 80.26%, DSC = 88.20%; Dataset B: IoU = 89.74%, DSC = 94.48%). These advantageous outcomes substantiate the practical utility of our approach, highlighting its potential to alleviate the workload of radiologists when applied in clinical settings.

Fetal Head and Pubic Symphysis Segmentation in Intrapartum Ultrasound Image Using a Dual-Path Boundary-Guided Residual Network

Accurate segmentation of the fetal head and pubic symphysis in intrapartum ultrasound images and measurement of fetal angle of progression (AoP) are critical to both outcome prediction and complication prevention in delivery. However, due to poor quality of perinatal ultrasound imaging with blurred target boundaries and the relatively small target of the public symphysis, fully automated and accurate segmentation remains challenging. In this paper, we propse a dual-path boundary-guided residual network (DBRN), which is a novel approach to tackle these challenges. The model contains a multi-scale weighted module (MWM) to gather global context information, and enhance the feature response within the target region by weighting the feature map. The model also incorporates an enhanced boundary module (EBM) to obtain more precise boundary information. Furthermore, the model introduces a boundary-guided dual-attention residual module (BDRM) for residual learning. BDRM leverages boundary information as prior knowledge and employs spatial attention to simultaneously focus on background and foreground information, in order to capture concealed details and improve segmentation accuracy. Extensive comparative experiments have been conducted on three datasets. The proposed method achieves average Dice score of 0.908 ±0.05 and average Hausdorff distance of 3.396 ±0.66 mm. Compared with state-of-the-art competitors, the proposed DBRN achieves better results. In addition, the average difference between the automatic measurement of AoPs based on this model and the manual measurement results is 6.157 °, which has good consistency and has broad application prospects in clinical practice.

Development and clinical validation of a deep learning-based knee CT image segmentation method for robotic-assisted total knee arthroplasty

BACKGROUND

This study aimed to develop a novel deep convolutional neural network called Dual-path Double Attention Transformer (DDA-Transformer) designed to achieve precise and fast knee joint CT image segmentation and to validate it in robotic-assisted total knee arthroplasty (TKA).

METHODS

The femoral, tibial, patellar, and fibular segmentation performance and speed were evaluated and the accuracy of component sizing, bone resection and alignment of the robotic-assisted TKA system constructed using this deep learning network was clinically validated.

RESULTS

Overall, DDA-Transformer outperformed six other networks in terms of the Dice coefficient, intersection over union, average surface distance, and Hausdorff distance. DDA-Transformer exhibited significantly faster segmentation speeds than nnUnet, TransUnet and 3D-Unet (p < 0.01). Furthermore, the robotic-assisted TKA system outperforms the manual group in surgical accuracy.

CONCLUSIONS

DDA-Transformer exhibited significantly improved accuracy and robustness in knee joint segmentation, and this convenient and stable knee joint CT image segmentation network significantly improved the accuracy of the TKA procedure.

A multibranch and multiscale neural network based on semantic perception for multimodal medical image fusion

Medical imaging is indispensable for accurate diagnosis and effective treatment, with modalities like MRI and CT providing diverse yet complementary information. Traditional image fusion methods, while essential in consolidating information from multiple modalities, often suffer from poor image quality and loss of crucial details due to inadequate handling of semantic information and limited feature extraction capabilities. This paper introduces a novel medical image fusion technique leveraging unsupervised image segmentation to enhance the semantic understanding of the fusion process. The proposed method, named DUSMIF, employs a multi-branch, multi-scale deep learning architecture that integrates advanced attention mechanisms to refine the feature extraction and fusion processes. An innovative approach that utilizes unsupervised image segmentation to extract semantic information is introduced, which is then integrated into the fusion process. This not only enhances the semantic relevance of the fused images but also improves the overall fusion quality. The paper proposes a sophisticated network structure that extracts and fuses features at multiple scales and across multiple branches. This structure is designed to capture a comprehensive range of image details and contextual information, significantly improving the fusion outcomes. Multiple attention mechanisms are incorporated to selectively emphasize important features and integrate them effectively across different modalities and scales. This approach ensures that the fused images maintain high quality and detail fidelity. A joint loss function combining content loss, structural similarity loss, and semantic loss is formulated. This function not only guides the network in preserving image brightness and texture but also ensures that the fused image closely resembles the source images in both content and structure. The proposed method demonstrates superior performance over existing fusion techniques in objective assessments and subjective evaluations, confirming its effectiveness in enhancing the diagnostic utility of fused medical images.

DFA-UNet: dual-stream feature-fusion attention U-Net for lymph node segmentation in lung cancer diagnosis

In bronchial ultrasound elastography, accurately segmenting mediastinal lymph nodes is of great significance for diagnosing whether lung cancer has metastasized. However, due to the ill-defined margin of ultrasound images and the complexity of lymph node structure, accurate segmentation of fine contours is still challenging. Therefore, we propose a dual-stream feature-fusion attention U-Net (DFA-UNet). Firstly, a dual-stream encoder (DSE) is designed by combining ConvNext with a lightweight vision transformer (ViT) to extract the local information and global information of images; Secondly, we propose a hybrid attention module (HAM) at the bottleneck, which incorporates spatial and channel attention to optimize the features transmission process by optimizing high-dimensional features at the bottom of the network. Finally, the feature-enhanced residual decoder (FRD) is developed to improve the fusion of features obtained from the encoder and decoder, ensuring a more comprehensive integration. Extensive experiments on the ultrasound elasticity image dataset show the superiority of our DFA-UNet over 9 state-of-the-art image segmentation models. Additionally, visual analysis, ablation studies, and generalization assessments highlight the significant enhancement effects of DFA-UNet. Comprehensive experiments confirm the excellent segmentation effectiveness of the DFA-UNet combined attention mechanism for ultrasound images, underscoring its important significance for future research on medical images.

TPA: Two-stage progressive attention segmentation framework for hepatocellular carcinoma on multi-modality MRI

BACKGROUND

Dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) plays a crucial role in the diagnosis and measurement of hepatocellular carcinoma (HCC). The multi-modality information contained in the multi-phase images of DCE-MRI is important for improving segmentation. However, this remains a challenging task due to the heterogeneity of HCC, which may cause one HCC lesion to have varied imaging appearance in each phase of DCE-MRI. In particular, some phases exhibit inconsistent sizes and boundaries will result in a lack of correlation between modalities, and it may pose inaccurate segmentation results.

PURPOSE

We aim to design a multi-modality segmentation model that can learn meaningful inter-phase correlation for achieving HCC segmentation.

METHODS

In this study, we propose a two-stage progressive attention segmentation framework (TPA) for HCC based on the transformer and the decision-making process of radiologists. Specifically, the first stage aims to fuse features from multi-phase images to identify HCC and provide localization region. In the second stage, a multi-modality attention transformer module (MAT) is designed to focus on the features that can represent the actual size.

RESULTS

We conduct training, validation, and test in a single-center dataset (386 cases), followed by external test on a batch of multi-center datasets (83 cases). Furthermore, we analyze a subgroup of data with weak inter-phase correlation in the test set. The proposed model achieves Dice coefficient of 0.822 and 0.772 in the internal and external test sets, respectively, and 0.829, 0.791 in the subgroup. The experimental results demonstrate that our model outperforms state-of-the-art models, particularly within subgroup.

CONCLUSIONS

The proposed TPA provides best segmentation results, and utilizing clinical prior knowledge for network design is practical and feasible.

Adaptive Multi-Dimensional Weighted Network With Category-Aware Contrastive Learning for Fine-Grained Hand Bone Segmentation

Accurately delineating and categorizing individual hand bones in 3D ultrasound (US) is a promising technology for precise digital diagnostic analysis. However, this is a challenging task due to the inherent imaging limitations of the US and the insignificant feature differences among numerous bones. In this study, we have proposed a novel deep learning-based solution for pediatric hand bone segmentation in the US. Our method is unique in that it allows for effective detailed feature mining through an adaptive multi-dimensional weighting attention mechanism. It innovatively implements a category-aware contrastive learning method to highlight inter-class semantic feature differences, thereby enhancing the category discrimination performance of the model. Extensive experiments on the challenging pediatric clinical hand 3D US datasets show the outstanding performance of the proposed method in segmenting thirty-eight bone structures, with the average Dice coefficient of 90.0%. The results outperform other state-of-the-art methods, demonstrating its effectiveness in fine-grained hand bone segmentation. Our method will be globally released as a plugin in the 3D Slicer, providing an innovative and reliable tool for relevant clinical applications.

A Coarse-Fine Collaborative Learning Model for Three Vessel Segmentation in Fetal Cardiac Ultrasound Images

Congenital heart disease (CHD) is the most frequent birth defect and a leading cause of infant mortality, emphasizing the crucial need for its early diagnosis. Ultrasound is the primary imaging modality for prenatal CHD screening. As a complement to the four-chamber view, the three-vessel view (3VV) plays a vital role in detecting anomalies in the great vessels. However, the interpretation of fetal cardiac ultrasound images is subjective and relies heavily on operator experience, leading to variability in CHD detection rates, particularly in resource-constrained regions. In this study, we propose an automated method for segmenting the pulmonary artery, ascending aorta, and superior vena cava in the 3VV using a novel deep learning network named CoFi-Net. Our network incorporates a coarse-fine collaborative strategy with two parallel branches dedicated to simultaneous global localization and fine segmentation of the vessels. The coarse branch employs a partial decoder to leverage high-level semantic features, enabling global localization of objects and suppression of irrelevant structures. The fine branch utilizes attention-parameterized skip connections to improve feature representations and improve boundary information. The outputs of the two branches are fused to generate accurate vessel segmentations. Extensive experiments conducted on a collected dataset demonstrate the superiority of CoFi-Net compared to state-of-the-art segmentation models for 3VV segmentation, indicating its great potential for enhancing CHD diagnostic efficiency in clinical practice. Furthermore, CoFi-Net outperforms other deep learning models in breast lesion segmentation on a public breast ultrasound dataset, despite not being specifically designed for this task, demonstrating its potential and robustness for various segmentation tasks.

MCV-UNet: a modified convolution & transformer hybrid encoder-decoder network with multi-scale information fusion for ultrasound image semantic segmentation

In recent years, the growing importance of accurate semantic segmentation in ultrasound images has led to numerous advances in deep learning-based techniques. In this article, we introduce a novel hybrid network that synergistically combines convolutional neural networks (CNN) and Vision Transformers (ViT) for ultrasound image semantic segmentation. Our primary contribution is the incorporation of multi-scale CNN in both the encoder and decoder stages, enhancing feature learning capabilities across multiple scales. Further, the bottleneck of the network leverages the ViT to capture long-range high-dimension spatial dependencies, a critical factor often overlooked in conventional CNN-based approaches. We conducted extensive experiments using a public benchmark ultrasound nerve segmentation dataset. Our proposed method was benchmarked against 17 existing baseline methods, and the results underscored its superiority, as it outperformed all competing methods including a 4.6% improvement of Dice compared against TransUNet, 13.0% improvement of Dice against Attention UNet, 10.5% improvement of precision compared against UNet. This research offers significant potential for real-world applications in medical imaging, demonstrating the power of blending CNN and ViT in a unified framework.

MEF-UNet: An end-to-end ultrasound image segmentation algorithm based on multi-scale feature extraction and fusion

Ultrasound image segmentation is a challenging task due to the complexity of lesion types, fuzzy boundaries, and low-contrast images along with the presence of noises and artifacts. To address these issues, we propose an end-to-end multi-scale feature extraction and fusion network (MEF-UNet) for the automatic segmentation of ultrasound images. Specifically, we first design a selective feature extraction encoder, including detail extraction stage and structure extraction stage, to precisely capture the edge details and overall shape features of the lesions. In order to enhance the representation capacity of contextual information, we develop a context information storage module in the skip-connection section, responsible for integrating information from adjacent two-layer feature maps. In addition, we design a multi-scale feature fusion module in the decoder section to merge feature maps with different scales. Experimental results indicate that our MEF-UNet can significantly improve the segmentation results in both quantitative analysis and visual effects.

Transformative Deep Neural Network Approaches in Kidney Ultrasound Segmentation: Empirical Validation with an Annotated Dataset

Kidney ultrasound (US) images are primarily employed for diagnosing different renal diseases. Among them, one is renal localization and detection, which can be carried out by segmenting the kidney US images. However, kidney segmentation from US images is challenging due to low contrast, speckle noise, fluid, variations in kidney shape, and modality artifacts. Moreover, well-annotated US datasets for renal segmentation and detection are scarce. This study aims to build a novel, well-annotated dataset containing 44,880 US images. In addition, we propose a novel training scheme that utilizes the encoder and decoder parts of a state-of-the-art segmentation algorithm. In the pre-processing step, pixel intensity normalization improves contrast and facilitates model convergence. The modified encoder-decoder architecture improves pyramid-shaped hole pooling, cascaded multiple-hole convolutions, and batch normalization. The pre-processing step gradually reconstructs spatial information, including the capture of complete object boundaries, and the post-processing module with a concave curvature reduces the false positive rate of the results. We present benchmark findings to validate the quality of the proposed training scheme and dataset. We applied six evaluation metrics and several baseline segmentation approaches to our novel kidney US dataset. Among the evaluated models, DeepLabv3+ performed well and achieved the highest dice, Hausdorff distance 95, accuracy, specificity, average symmetric surface distance, and recall scores of 89.76%, 9.91, 98.14%, 98.83%, 3.03, and 90.68%, respectively. The proposed training strategy aids state-of-the-art segmentation models, resulting in better-segmented predictions. Furthermore, the large, well-annotated kidney US public dataset will serve as a valuable baseline source for future medical image analysis research.

S2DA-Net: Spatial and spectral-learning double-branch aggregation network for liver tumor segmentation in CT images

Accurate liver tumor segmentation is crucial for aiding radiologists in hepatocellular carcinoma evaluation and surgical planning. While convolutional neural networks (CNNs) have been successful in medical image segmentation, they face challenges in capturing long-term dependencies among pixels. On the other hand, Transformer-based models demand a high number of parameters and involve significant computational costs. To address these issues, we propose the Spatial and Spectral-learning Double-branched Aggregation Network (S2DA-Net) for liver tumor segmentation. S2DA-Net consists of a double-branched encoder and a decoder with a Group Multi-Head Cross-Attention Aggregation (GMCA) module, Two branches in the encoder consist of a Fourier Spectral-learning Multi-scale Fusion (FSMF) branch and a Multi-axis Aggregation Hadamard Attention (MAHA) branch. The FSMF branch employs a Fourier-based network to learn amplitude and phase information, capturing richer features and detailed information without introducing an excessive number of parameters. The FSMF branch utilizes a Fourier-based network to capture amplitude and phase information, enriching features without introducing excessive parameters. The MAHA branch incorporates spatial information, enhancing discriminative features while minimizing computational costs. In the decoding path, a GMCA module extracts local information and establishes long-term dependencies, improving localization capabilities by amalgamating features from diverse branches. Experimental results on the public LiTS2017 liver tumor datasets show that the proposed segmentation model achieves significant improvements compared to the state-of-the-art methods, obtaining dice per case (DPC) 69.4 % and global dice (DG) 80.0 % for liver tumor segmentation on the LiTS2017 dataset. Meanwhile, the pre-trained model based on the LiTS2017 datasets obtain, DPC 73.4 % and an DG 82.2 % on the 3DIRCADb dataset.

Label-Decoupled Medical Image Segmentation With Spatial-Channel Graph Convolution and Dual Attention Enhancement

Deep learning-based methods have been widely used in medical image segmentation recently. However, existing works are usually difficult to simultaneously capture global long-range information from images and topological correlations among feature maps. Further, medical images often suffer from blurred target edges. Accordingly, this paper proposes a novel medical image segmentation framework named a label-decoupled network with spatial-channel graph convolution and dual attention enhancement mechanism (LADENet for short). It constructs learnable adjacency matrices and utilizes graph convolutions to effectively capture global long-range information on spatial locations and topological dependencies between different channels in an image. Then a label-decoupled strategy based on distance transformation is introduced to decouple an original segmentation label into a body label and an edge label for supervising the body branch and edge branch. Again, a dual attention enhancement mechanism, designing a body attention block in the body branch and an edge attention block in the edge branch, is built to promote the learning ability of spatial region and boundary features. Besides, a feature interactor is devised to fully consider the information interaction between the body and edge branches to improve segmentation performance. Experiments on benchmark datasets reveal the superiority of LADENet compared to state-of-the-art approaches.

SG-Transunet: A segmentation-guided Transformer U-Net model for KRAS gene mutation status identification in colorectal cancer

Accurately identifying the Kirsten rat sarcoma virus (KRAS) gene mutation status in colorectal cancer (CRC) patients can assist doctors in deciding whether to use specific targeted drugs for treatment. Although deep learning methods are popular, they are often affected by redundant features from non-lesion areas. Moreover, existing methods commonly extract spatial features from imaging data, which neglect important frequency domain features and may degrade the performance of KRAS gene mutation status identification. To address this deficiency, we propose a segmentation-guided Transformer U-Net (SG-Transunet) model for KRAS gene mutation status identification in CRC. Integrating the strength of convolutional neural networks (CNNs) and Transformers, SG-Transunet offers a unique approach for both lesion segmentation and KRAS mutation status identification. Specifically, for precise lesion localization, we employ an encoder-decoder to obtain segmentation results and guide the KRAS gene mutation status identification task. Subsequently, a frequency domain supplement block is designed to capture frequency domain features, integrating it with high-level spatial features extracted in the encoding path to derive advanced spatial-frequency domain features. Furthermore, we introduce a pre-trained Xception block to mitigate the risk of overfitting associated with small-scale datasets. Following this, an aggregate attention module is devised to consolidate spatial-frequency domain features with global information extracted by the Transformer at shallow and deep levels, thereby enhancing feature discriminability. Finally, we propose a mutual-constrained loss function that simultaneously constrains the segmentation mask acquisition and gene status identification process. Experimental results demonstrate the superior performance of SG-Transunet over state-of-the-art methods in discriminating KRAS gene mutation status.

DMAeEDNet: Dense Multiplicative Attention Enhanced Encoder Decoder Network for Ultrasound-Based Automated Breast Lesion Segmentation

Automated and precise segmentation of breast lesions can facilitate early diagnosis of breast cancer. Recent research studies employ deep learning for automatic segmentation of breast lesions using ultrasound imaging. Numerous studies introduce somewhat complex modifications to the well adapted segmentation network, U-Net for improved segmentation, however, at the expense of increased computational time. Towards this aspect, this study presents a low complex deep learning network, i.e., dense multiplicative attention enhanced encoder decoder network, for effective breast lesion segmentation in the ultrasound images. For the first time in this context, two dense multiplicative attention components are utilized in the encoding layer and the output layer of an encoder-decoder network with depthwise separable convolutions, to selectively enhance the relevant features. A rigorous performance evaluation using two public datasets demonstrates that the proposed network achieves dice coefficients of 0.83 and 0.86 respectively with an average segmentation latency of 19ms. Further, a noise robustness study using an in-clinic recorded dataset without pre-processing indicates that the proposed network achieves dice coefficient of 0.72. Exhaustive comparison with some commonly used networks indicate its adeptness with low time and computational complexity demonstrating feasibility in real time.

TaiChiNet: Negative-Positive Cross-Attention Network for Breast Lesion Segmentation in Ultrasound Images

Breast lesion segmentation in ultrasound images is essential for computer-aided breast-cancer diagnosis. To improve the segmentation performance, most approaches design sophisticated deep-learning models by mining the patterns of foreground lesions and normal backgrounds simultaneously or by unilaterally enhancing foreground lesions via various focal losses. However, the potential of normal backgrounds is underutilized, which could reduce false positives by compacting the feature representation of all normal backgrounds. From a novel viewpoint of bilateral enhancement, we propose a negative-positive cross-attention network to concentrate on normal backgrounds and foreground lesions, respectively. Derived from the complementing opposites of bipolarity in TaiChi, the network is denoted as TaiChiNet, which consists of the negative normal-background and positive foreground-lesion paths. To transmit the information across the two paths, a cross-attention module, a complementary MLP-head, and a complementary loss are built for deep-layer features, shallow-layer features, and mutual-learning supervision, separately. To the best of our knowledge, this is the first work to formulate breast lesion segmentation as a mutual supervision task from the foreground-lesion and normal-background views. Experimental results have demonstrated the effectiveness of TaiChiNet on two breast lesion segmentation datasets with a lightweight architecture. Furthermore, extensive experiments on the thyroid nodule segmentation and retinal optic cup/disc segmentation datasets indicate the application potential of TaiChiNet.

Hierarchical attention-guided multiscale aggregation network for infrared small target detection

All man-made flying objects in the sky, ships in the ocean can be regarded as small infrared targets, and the method of tracking them has been received widespread attention in recent years. In search of a further efficient method for infrared small target recognition, we propose a hierarchical attention-guided multiscale aggregation network (HAMANet) in this thesis. The proposed HAMANet mainly consists of a compound guide multilayer perceptron (CG-MLP) block embedded in the backbone net, a spatial-interactive attention module (SiAM), a pixel-interactive attention module (PiAM) and a contextual fusion module (CFM). The CG-MLP marked the width-axis, height-axis, and channel-axis, which can result in a better segmentation effect while reducing computational complexity. SiAM improves global semantic information exchange by increasing the connections between different channels, while PiAM changes the extraction of local key information features by enhancing information exchange at the pixel level. CFM fuses low-level positional information and high-level channel information of the target through coding to improve network stability and target feature utilization. Compared with other state-of-the-art methods on public infrared small target datasets, the results show that our proposed HAMANet has high detection accuracy and a low false-alarm rate.

MLAU-Net: Deep supervised attention and hybrid loss strategies for enhanced segmentation of low-resolution kidney ultrasound

Objective

The precise segmentation of kidneys from a 2D ultrasound (US) image is crucial for diagnosing and monitoring kidney diseases. However, achieving detailed segmentation is difficult due to US images' low signal-to-noise ratio and low-contrast object boundaries.

Methods

This paper presents an approach called deep supervised attention with multi-loss functions (MLAU-Net) for US segmentation. The MLAU-Net model combines the benefits of attention mechanisms and deep supervision to improve segmentation accuracy. The attention mechanism allows the model to selectively focus on relevant regions of the kidney and ignore irrelevant background information, while the deep supervision captures the high-dimensional structure of the kidney in US images.

Results

We conducted experiments on two datasets to evaluate the MLAU-Net model's performance. The Wuerzburg Dynamic Kidney Ultrasound (WD-KUS) dataset with annotation contained kidney US images from 176 patients split into training and testing sets totaling 44,880. The Open Kidney Dataset's second dataset has over 500 B-mode abdominal US images. The proposed approach achieved the highest dice, accuracy, specificity, Hausdorff distance (HD95), recall, and Average Symmetric Surface Distance (ASSD) scores of 90.2%, 98.26%, 98.93%, 8.90 mm, 91.78%, and 2.87 mm, respectively, upon testing and comparison with state-of-the-art U-Net series segmentation frameworks, which demonstrates the potential clinical value of our work.

Conclusion

The proposed MLAU-Net model has the potential to be applied to other medical image segmentation tasks that face similar challenges of low signal-to-noise ratios and low-contrast object boundaries.

U structured network with three encoding paths for breast tumor segmentation

Breast ultrasound segmentation remains challenging because of the blurred boundaries, irregular shapes, and the presence of shadowing and speckle noise. The majority of approaches stack convolutional layers to extract advanced semantic information, which makes it difficult to handle multiscale issues. To address those issues, we propose a three-path U-structure network (TPUNet) that consists of a three-path encoder and an attention-based feature fusion block (AFF Block). Specifically, instead of simply stacking convolutional layers, we design a three-path encoder to capture multiscale features through three independent encoding paths. Additionally, we design an attention-based feature fusion block to weight and fuse feature maps in spatial and channel dimensions. The AFF Block encourages different paths to compete with each other in order to synthesize more salient feature maps. We also investigate a hybrid loss function for reducing false negative regions and refining the boundary segmentation, as well as the deep supervision to guide different paths to capture the effective features under the corresponding receptive field sizes. According to experimental findings, our proposed TPUNet achieves more excellent results in terms of quantitative analysis and visual quality than other rival approaches.

Deep Semi-Supervised Ultrasound Image Segmentation by Using a Shadow Aware Network With Boundary Refinement

Accurate ultrasound (US) image segmentation is crucial for the screening and diagnosis of diseases. However, it faces two significant challenges: 1) pixel-level annotation is a time-consuming and laborious process; 2) the presence of shadow artifacts leads to missing anatomy and ambiguous boundaries, which negatively impact reliable segmentation results. To address these challenges, we propose a novel semi-supervised shadow aware network with boundary refinement (SABR-Net). Specifically, we add shadow imitation regions to the original US, and design shadow-masked transformer blocks to perceive missing anatomy of shadow regions. Shadow-masked transformer block contains an adaptive shadow attention mechanism that introduces an adaptive mask, which is updated automatically to promote the network training. Additionally, we utilize unlabeled US images to train a missing structure inpainting path with shadow-masked transformer, which further facilitates semi-supervised segmentation. Experiments on two public US datasets demonstrate the superior performance of the SABR-Net over other state-of-the-art semi-supervised segmentation methods. In addition, experiments on a private breast US dataset prove that our method has a good generalization to clinical small-scale US datasets.

DARMF-UNet: A dual-branch attention-guided refinement network with multi-scale features fusion U-Net for gland segmentation

Accurate gland segmentation is critical in determining adenocarcinoma. Automatic gland segmentation methods currently suffer from challenges such as less accurate edge segmentation, easy mis-segmentation, and incomplete segmentation. To solve these problems, this paper proposes a novel gland segmentation network Dual-branch Attention-guided Refinement and Multi-scale Features Fusion U-Net (DARMF-UNet), which fuses multi-scale features using deep supervision. At the first three layers of feature concatenation, a Coordinate Parallel Attention (CPA) is proposed to guide the network to focus on the key regions. A Dense Atrous Convolution (DAC) block is used in the fourth layer of feature concatenation to perform multi-scale features extraction and obtain global information. A hybrid loss function is adopted to calculate the loss of each segmentation result of the network to achieve deep supervision and improve the accuracy of segmentation. Finally, the segmentation results at different scales in each part of the network are fused to obtain the final gland segmentation result. The experimental results on the gland datasets Warwick-QU and Crag show that the network improves in terms of the evaluation metrics of F1 Score, Object Dice, Object Hausdorff, and the segmentation effect is better than the state-of-the-art network models.

MsGoF: Breast lesion classification on ultrasound images by multi-scale gradational-order fusion framework

BACKGROUND AND OBJECTIVE

Predicting the malignant potential of breast lesions based on breast ultrasound (BUS) images is a crucial component of computer-aided diagnosis system for breast cancers. However, since breast lesions in BUS images generally have various shapes with relatively low contrast and present complex textures, it still remains challenging to accurately identify the malignant potential of breast lesions.

METHODS

In this paper, we propose a multi-scale gradational-order fusion framework to make full advantages of multi-scale representations incorporating with gradational-order characteristics of BUS images for breast lesions classification. Specifically, we first construct a spatial context aggregation module to generate multi-scale context representations from the original BUS images. Subsequently, multi-scale representations are efficiently fused in feature fusion block that is armed with special fusion strategies to comprehensively capture morphological characteristics of breast lesions. To better characterize complex textures and enhance non-linear modeling capability, we further propose isotropous gradational-order feature module in the feature fusion block to learn and combine multi-order representations. Finally, these multi-scale gradational-order representations are utilized to perform prediction for the malignant potential of breast lesions.

RESULTS

The proposed model was evaluated on three open datasets by using 5-fold cross-validation. The experimental results (Accuracy: 85.32%, Sensitivity: 85.24%, Specificity: 88.57%, AUC: 90.63% on dataset A; Accuracy: 76.48%, Sensitivity: 72.45%, Specificity: 80.42%, AUC: 78.98% on dataset B) demonstrate that the proposed method achieves the promising performance when compared with other deep learning-based methods in BUS classification task.

CONCLUSIONS

The proposed method has demonstrated a promising potential to predict malignant potential of breast lesion using ultrasound image in an end-to-end manner.