H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes

Advancing hierarchical neural networks with scale-aware pyramidal feature learning for medical image dense prediction

BACKGROUND AND OBJECTIVE

Hierarchical neural networks are pivotal in medical imaging for multi-scale representation, aiding in tasks such as object detection and segmentation. However, their effectiveness is often limited by the loss of intra-scale information and misalignment of inter-scale features. Our study introduces the Integrated-Scale Pyramidal Interactive Reconfiguration to Enhance feature learning (INSPIRE).

METHODS

INSPIRE focuses on intra-scale semantic enhancement and precise inter-scale spatial alignment, integrated with a novel spatial-semantic back augmentation technique. We evaluated INSPIRE's efficacy using standard hierarchical neural networks, such as UNet and FPN, across multiple medical segmentation challenges including brain tumors and polyps. Additionally, we extended our evaluation to object detection and semantic segmentation in natural images to assess generalizability.

RESULTS

INSPIRE demonstrated superior performance over standard baselines in medical segmentation tasks, showing significant improvements in feature learning and alignment. In identifying brain tumors and polyps, INSPIRE achieved enhanced precision, sensitivity, and specificity compared to traditional models. Further testing in natural images confirmed the adaptability and robustness of our approach.

CONCLUSIONS

INSPIRE effectively enriches semantic clarity and aligns multi-scale features, achieving integrated spatial-semantic coherence. This method seamlessly integrates with existing frameworks used in medical image analysis, thereby promising to significantly enhance the efficacy of computer-aided diagnostics and clinical interventions. Its application could lead to more accurate and efficient imaging processes, essential for improved patient outcomes.

SSAT-Swin: Deep Learning-Based Spinal Ultrasound Feature Segmentation for Scoliosis Using Self-Supervised Swin Transformer

OBJECTIVE

Scoliosis, a 3-D spinal deformity, requires early detection and intervention. Ultrasound curve angle (UCA) measurement using ultrasound images has emerged as a promising diagnostic tool. However, calculating the UCA directly from ultrasound images remains challenging due to low contrast, high noise, and irregular target shapes. Accurate segmentation results are therefore crucial to enhance image clarity and precision prior to UCA calculation.

METHODS

We propose the SSAT-Swin model, a transformer-based multi-class segmentation framework designed for ultrasound image analysis in scoliosis diagnosis. The model integrates a boundary-enhancement module in the decoder and a channel attention module in the skip connections. Additionally, self-supervised proxy tasks are used during pre-training on 1,170 images, followed by fine-tuning on 109 image-label pairs.

RESULTS

The SSAT-Swin achieved Dice scores of 85.6% and Jaccard scores of 74.5%, with a 92.8% scoliosis bone feature detection rate, outperforming state-of-the-art models.

CONCLUSION

Self-supervised learning enhances the model's ability to capture global context information, making it well-suited for addressing the unique challenges of ultrasound images, ultimately advancing scoliosis assessment through more accurate segmentation.

U3UNet: An accurate and reliable segmentation model for forest fire monitoring based on UAV vision

Forest fires pose a serious threat to the global ecological environment, and the critical steps in reducing the impact of fires are fire warning and real-time monitoring. Traditional monitoring methods, like ground observation and satellite sensing, were limited by monitoring coverage or low spatio-temporal resolution, making it difficult to meet the needs for precise shape of fire sources. Therefore, we propose an accurate and reliable forest fire monitoring segmentation model U3UNet based on UAV vision, which uses a nested U-shaped structure for feature fusion at different scales to retain important feature information. The idea of a full-scale connection is utilized to balance the global information of detailed features to ensure the full fusion of features. We conducted a series of comparative experiments with U-Net, UNet 3＋, U2-Net, Yolov9, FPS-U2Net, PSPNet, DeeplabV3＋ and TransFuse on the Unreal Engine platform and several real forest fire scenes. According to the designed composite metric S, in static scenarios 71. 44% is achieved, which is 0.3% lower than the best method. In the dynamic scenario, it reaches 80.53%, which is 8.94% higher than the optimal method. In addition, we also tested the real-time performance of U3UNet on edge computing device equipped on UAV.

ResTransUNet: A hybrid CNN-transformer approach for liver and tumor segmentation in CT images

BACKGROUND AND OBJECTIVE

Accurate medical tumor segmentation is critical for early diagnosis and treatment planning, significantly improving patient outcomes. This study aims to enhance liver and tumor segmentation from CT and liver images by developing a novel model, ResTransUNet, which combines convolutional and transformer blocks to improve segmentation accuracy.

METHODS

The proposed ResTransUNet model is a custom implementation inspired by the TransUNet architecture, featuring a Standalone Transformer Block and ResNet50 as the backbone for the encoder. The hybrid architecture leverages the strengths of Convolutional Neural Networks (CNNs) and Transformer blocks to capture both local features and global context effectively. The encoder utilizes a pre-trained ResNet50 to extract rich hierarchical features, with key feature maps to preserved it as skip connections. The Standalone Transformer Block, integrated into the model, employs multi-head attention mechanisms to capture long-range dependencies across the image, enhancing segmentation performance in complex cases. The decoder reconstructs the segmentation mask by progressively upsampling encoded features while integrating skip connections, ensuring both semantic information and spatial details are retained. This process culminates in a precise binary segmentation mask that effectively distinguishes liver and tumor regions.

RESULTS

The ResTransUNet model achieved superior Dice Similarity Coefficient (DSC) for liver segmentation (98.3% on LiTS and 98.4% on 3D-IRCADb-01) and for tumor segmentation from CT images (94.7% on LiTS and 89.8% on 3D-IRCADb-01) as well as from liver images (94.6% on LiTS and 91.1% on 3D-IRCADb-01). The model also demonstrated high precision, sensitivity, and specificity, outperforming current state-of-the-art methods in these tasks.

CONCLUSIONS

The ResTransUNet model demonstrates robust and accurate performance in complex medical image segmentation tasks, particularly in liver and tumor segmentation. These findings suggest that ResTransUNet has significant potential for improving the precision of surgical interventions and therapy planning in clinical settings.

HALSR-Net: Improving CNN Segmentation of Cardiac Left Ventricle MRI with Hybrid Attention and Latent Space Reconstruction

Accurate cardiac MRI segmentation is vital for detailed cardiac analysis, yet the manual process is labor-intensive and prone to variability. Despite advancements in MRI technology, there remains a significant need for automated methods that can reliably and efficiently segment cardiac structures. This paper introduces HALSR-Net, a novel multi-level segmentation architecture designed to improve the accuracy and reproducibility of cardiac segmentation from Cine-MRI acquisitions, focusing on the left ventricle (LV). The methodology consists of two main phases: first, the extraction of the region of interest (ROI) using a regression model that accurately predicts the location of a bounding box around the LV; second, the semantic segmentation step based on HALSR-Net architecture. This architecture incorporates a Hybrid Attention Pooling Module (HAPM) that merges attention and pooling mechanisms to enhance feature extraction and capture contextual information. Additionally, a reconstruction module leverages latent space features to further improve segmentation accuracy. Experiments conducted on an in-house clinical dataset and two public datasets (ACDC and LVQuan19) demonstrate that HALSR-Net outperforms state-of-the-art architectures, achieving up to 98% accuracy and F1-score for the segmentation of the LV cavity and myocardium. The proposed approach effectively addresses the limitations of existing methods, offering a more accurate and robust solution for cardiac MRI segmentation, thereby likely to improve cardiac function analysis and patient care.

Multiscale transformers and multi-attention mechanism networks for pathological nuclei segmentation

Pathology nuclei segmentation is crucial of computer-aided diagnosis in pathology. However, due to the high density, complex backgrounds, and blurred cell boundaries, it makes pathology cell segmentation still a challenging problem. In this paper, we propose a network model for pathology image segmentation based on a multi-scale Transformer multi-attention mechanism. To solve the problem that the high density of cell nuclei and the complexity of the background make it difficult to extract features, a dense attention module is embedded in the encoder, which improves the learning of the target cell information to minimize target information loss; Additionally, to solve the problem of poor segmentation accuracy due to the blurred cell boundaries, the Multi-scale Transformer Attention module is embedded between encoder and decoder, improving the transfer of the boundary feature information and makes the segmented cell boundaries more accurate. Experimental results on MoNuSeg, GlaS and CoNSeP datasets demonstrate the network's superior accuracy.

Anatomy guided modality fusion for cancer segmentation in PET CT volumes and images

Segmentation in computed tomography (CT) provides detailed anatomical information, while positron emission tomography (PET) provide the metabolic activity of cancer. Existing segmentation models in CT and PET either rely on early fusion, which struggles to effectively capture independent features from each modality, or late fusion, which is computationally expensive and fails to leverage the complementary nature of the two modalities. This research addresses the gap by proposing an intermediate fusion approach that optimally balances the strengths of both modalities. Our method leverages anatomical features to guide the fusion process while preserving spatial representation quality. We achieve this through the separate encoding of anatomical and metabolic features followed by an attentive fusion decoder. Unlike traditional fixed normalization techniques, we introduce novel "zero layers" with learnable normalization. The proposed intermediate fusion reduces the number of filters, resulting in a lightweight model. Our approach demonstrates superior performance, achieving a dice score of 0.8184 and an [Formula: see text] score of 2.31. The implications of this study include more precise tumor delineation, leading to enhanced cancer diagnosis and more effective treatment planning.

Multi-scale conv-attention U-Net for medical image segmentation

U-Net-based network structures are widely used in medical image segmentation. However, effectively capturing multi-scale features and spatial context information of complex organizational structures remains a challenge. To address this, we propose a novel network structure based on the U-Net backbone. This model integrates the Adaptive Convolution (AC) module, Multi-Scale Learning (MSL) module, and Conv-Attention module to enhance feature expression ability and segmentation performance. The AC module dynamically adjusts the convolutional kernel through an adaptive convolutional layer. This enables the model to extract features of different shapes and scales adaptively, further improving its performance in complex scenarios. The MSL module is designed for multi-scale information fusion. It effectively aggregates fine-grained and high-level semantic features from different resolutions, creating rich multi-scale connections between the encoding and decoding processes. On the other hand, the Conv-Attention module incorporates an efficient attention mechanism into the skip connections. It captures global context information using a low-dimensional proxy for high-dimensional data. This approach reduces computational complexity while maintaining effective spatial and channel information extraction. Experimental validation on the CVC-ClinicDB, MICCAI 2023 Tooth, and ISIC2017 datasets demonstrates that our proposed MSCA-UNet significantly improves segmentation accuracy and model robustness. At the same time, it remains lightweight and outperforms existing segmentation methods.

Large language model-augmented learning for auto-delineation of treatment targets in head-and-neck cancer radiotherapy

BACKGROUND AND PURPOSE

Radiation therapy (RT) is highly effective, but its success depends on accurate, manual target delineation, which is time-consuming, labor-intensive, and prone to variability. Despite AI advancements in auto-contouring normal tissues, accurate RT target volume delineation remains challenging. This study presents Radformer, a novel visual language model that integrates text-rich clinical data with medical imaging for accurate automated RT target volume delineation.

MATERIALS AND METHODS

We developed Radformer, an innovative network that utilizes a hierarchical vision transformer as its backbone and integrates large language models (LLMs) to extract and embed clinical data in text-rich form. The model features a novel visual language attention module (VLAM) to combine visual and linguistic features, enabling language-aware visual encoding (LAVE). The Radformer was evaluated on a dataset of 2985 patients with head-and-neck cancer who underwent RT. Quantitative evaluations were performed utilizing metrics such as the Dice similarity coefficient (DSC), intersection over union (IOU), and 95th percentile Hausdorff distance (HD95).

RESULTS

The Radformer demonstrated superior performance in segmenting RT target volumes compared to state-of-the-art models. On the head-and-neck cancer dataset, Radformer achieved a mean DSC of 0.76 ± 0.09 versus 0.66 ± 0.09, a mean IOU of 0.69 ± 0.08 versus 0.59 ± 0.07, and a mean HD95 of 7.82 ± 6.87 mm versus 14.28 ± 6.85 mm for gross tumor volume delineation, compared to the baseline 3D-UNETR.

CONCLUSIONS

The Radformer model offers a clinically optimal means of RT target auto-delineation by integrating both imaging and clinical data through a visual language model. This approach improves the accuracy of RT target volume delineation, facilitating broader AI-assisted automation in RT treatment planning.

A-Eval: A benchmark for cross-dataset and cross-modality evaluation of abdominal multi-organ segmentation

Although deep learning has revolutionized abdominal multi-organ segmentation, its models often struggle with generalization due to training on small-scale, specific datasets and modalities. The recent emergence of large-scale datasets may mitigate this issue, but some important questions remain unsolved: Can models trained on these large datasets generalize well across different datasets and imaging modalities? If yes/no, how can we further improve their generalizability? To address these questions, we introduce A-Eval, a benchmark for the cross-dataset and cross-modality Evaluation ('Eval') of Abdominal ('A') multi-organ segmentation, integrating seven datasets across CT and MRI modalities. Our evaluations indicate that significant domain gaps persist despite larger data scales. While increased datasets improve generalization, model performance on unseen data remains inconsistent. Joint training across multiple datasets and modalities enhances generalization, though annotation inconsistencies pose challenges. These findings highlight the need for diverse and well-curated training data across various clinical scenarios and modalities to develop robust medical imaging models. The code and pre-trained models are available at https://github.com/uni-medical/A-Eval.

Multi-type stroke lesion segmentation: comparison of single-stage and hierarchical approach

Stroke, a major cause of death and disability worldwide, can be haemorrhagic or ischaemic depending on the type of bleeding in the brain. Rapid and accurate identification of stroke type and lesion segmentation is critical for timely and effective treatment. However, existing research primarily focuses on segmenting a single stroke type, potentially limiting their clinical applicability. This study addresses this gap by exploring multi-type stroke lesion segmentation using deep learning methods. Specifically, we investigate two distinct approaches: a single-stage approach that directly segments all tissue types in one model and a hierarchical approach that first classifies stroke types and then utilises specialised segmentation models for each subtype. Recognising the importance of accurate stroke classification for the hierarchical approach, we evaluate ResNet, ResNeXt and ViT networks, incorporating focal loss and oversampling techniques to mitigate the impact of class imbalance. We further explore the performance of U-Net, U-Net++ and DeepLabV3 models for segmentation within each approach. We use a comprehensive dataset of 6650 images provided by the Ministry of Health of the Republic of Türkiye. This dataset includes 1130 ischaemic strokes, 1093 haemorrhagic strokes and 4427 non-stroke cases. In our comparative experiments, we achieve an AUC score of 0.996 when classifying stroke and non-stroke slices. For lesion segmentation task, while the performance of different architectures is comparable, the hierarchical training approach outperforms the single-stage approach in terms of intersection over union (IoU). The performance of the U-Net model increased significantly from an IoU of 0.788 to 0.875 when the hierarchical approach is used. This comparative analysis aims to identify the most effective approach and deep learning model for multi-type stroke lesion segmentation in brain CT scans, potentially leading to improved clinical decision-making, treatment efficiency and outcomes.

A device-dependent auto-segmentation method based on combined generalized and single-device datasets

BACKGROUND

Although generalized-dataset-based auto-segmentation models that consider various computed tomography (CT) scanners have shown great clinical potential, their application to medical images from unseen scanners remains challenging because of device-dependent image features.

PURPOSE

This study aims to investigate the performance of a device-dependent auto-segmentation model based on a combined dataset of a generalized dataset and single CT scanner dataset.

METHOD

We constructed two training datasets for 21 chest and abdominal organs. The generalized dataset comprised 1203 publicly available multi-scanner data. The device-dependent dataset comprised 1253 data, including the 1203 multi-CT scanner data and 50 single CT scanner data. Using these datasets, the generalized-dataset-based model (GDSM) and the device-dependent-dataset-based model (DDSM) were trained. The models were trained using nnU-Net and tested on ten data samples from a single CT scanner. The evaluation metrics included the Dice similarity coefficient (DSC), the Hausdorff distance (HD), and the average symmetric surface distance (ASSD), which were used to assess the overall performance of the models. In addition, DSCdiff, HDratio, and ASSDratio, which are variations of the three existing metrics, were used to compare the performance of the models across different organs.

RESULT

For the average DSC, the GDSM and DDSM had values of 0.9251 and 0.9323, respectively. For the average HD, the GDSM and DDSM had values of 10.66 and 9.139 mm, respectively; for the average ASSD, the GDSM and DDSM had values of 0.8318 and 0.6656 mm, respectively. Compared with the GDSM, the DDSM showed consistent performance improvements of 0.78%, 14%, and 20% for the DSC, HD, and ASSD metrics, respectively. In addition, compared with the GDSM, the DDSM had better DSCdiff values in 14 of 21 tested organs, better HDratio values in 13 of 21 tested organs, and better ASSDratio values in 14 of 21 tested organs. The three averages of the variant metrics were all better for the DDSM than for the GDSM.

CONCLUSION

The results suggest that combining the generalized dataset with a single scanner dataset resulted in an overall improvement in model performance for that device image.

Prior Knowledge-Guided U-Net for Automatic Clinical Target Volume Segmentation in Postmastectomy Radiation Therapy of Breast Cancer

PURPOSE

This study aimed to design and evaluate a prior-knowledge-guided U-Net (PK-UNet) for automatic clinical target volume (CTV) segmentation in postmastectomy radiation therapy for breast cancer.

METHODS AND MATERIALS

A total of 102 computed tomography (CT) scans from breast cancer patients who underwent postmastectomy were retrospectively collected. Of these, 80 scans were used for training with 5-fold cross-validation, and 22 scans for independent testing. The CTV included the chest wall, supraclavicular region, and axillary group III. The proposed PK-UNet method employs a 2-stage auto-segmentation process. Initially, the localization network categorizes CT slices based on the anatomic information of the CTV and generates prior knowledge labels. These outputs, along with the CT images, were fed into the final segmentation network. Quantitative evaluation was conducted using the mean Dice similarity coefficient (DSC), 95% Hausdorff distance, average surface distance, and surface DSC. A four-level objective scale evaluation was performed by 2 experienced radiation oncologists in a randomized double-blind manner.

RESULTS

Quantitative evaluations revealed that PK-UNet significantly outperformed state-of-the-art segmentation methods (P < .01), with a mean DSC of 0.90 ± 0.02 and a 95% Hausdorff distance of 2.82 ± 1.29 mm. The mean average surface distance of PK-UNet was 0.91 ± 0.22 mm and the surface DSC was 0.84 ± 0.07, significantly surpassing the performance of AdwU-Net (P < .01) and showing comparable results to other models. Clinical evaluation confirmed the efficacy of PK-UNet, with 81.8% of the predicted contours being acceptable for clinical application. The advantages of the auto-segmentation capability of PK-UNet were most evident in the superior and inferior slices and slices with discontinuities at the junctions of different subregions. The average manual correction time was reduced to 1.02 min, compared with 18.20 min for manual contouring leading to a 94.4% reduction in working time.

CONCLUSIONS

This study introduced the pioneering integration of prior medical knowledge into a deep learning framework for postmastectomy radiation therapy. This strategy addresses the challenges of CTV segmentation in postmastectomy radiation therapy and improves clinical workflow efficiency.

Deep graph embedding based on Laplacian eigenmaps for MR fingerprinting reconstruction

Magnetic resonance fingerprinting (MRF) is a promising technique for fast quantitative imaging of multiple tissue parameters. However, the highly undersampled schemes utilized in MRF typically lead to noticeable aliasing artifacts in reconstructed images. Existing model-based methods can mitigate aliasing artifacts and enhance reconstruction quality but suffer from long reconstruction times. In addition, data priors used in these methods, such as low-rank and total variation, make it challenging to incorporate non-local and non-linear redundancies in MRF data. Furthermore, existing deep learning-based methods for MRF often lack interpretability and struggle with the high computational overhead caused by the high dimensionality of MRF data. To address these issues, we introduce a novel deep graph embedding framework based on the Laplacian eigenmaps for improved MRF reconstruction. Our work first models the acquired high-dimensional MRF data and the corresponding parameter maps as graph data nodes. Then, we propose an MRF reconstruction framework based on the graph embedding framework, retaining intrinsic graph structures between parameter maps and MRF data. To improve the accuracy of the estimated graph structure and the computational efficiency of the proposed framework, we unroll the iterative optimization process into a deep neural network, incorporating a learned graph embedding module to adaptively learn the Laplacian eigenmaps. By introducing the graph embedding framework into the MRF reconstruction, the proposed method can effectively exploit non-local and non-linear correlations in MRF data. Numerical experiments demonstrate that our approach can reconstruct high-quality MRF data and multiple parameter maps within a significantly reduced computational cost.

Learning robust medical image segmentation from multi-source annotations

Collecting annotations from multiple independent sources could mitigate the impact of potential noises and biases from a single source, which is a common practice in medical image segmentation. However, learning segmentation networks from multi-source annotations remains a challenge due to the uncertainties brought by the variance of the annotations. In this paper, we proposed an Uncertainty-guided Multi-source Annotation Network (UMA-Net), which guided the training process by uncertainty estimation at both the pixel and the image levels. First, we developed an annotation uncertainty estimation module (AUEM) to estimate the pixel-wise uncertainty of each annotation, which then guided the network to learn from reliable pixels by a weighted segmentation loss. Second, a quality assessment module (QAM) was proposed to assess the image-level quality of the input samples based on the former estimated annotation uncertainties. Furthermore, instead of discarding the low-quality samples, we introduced an auxiliary predictor to learn from them and thus ensured the preservation of their representation knowledge in the backbone without directly accumulating errors within the primary predictor. Extensive experiments demonstrated the effectiveness and feasibility of our proposed UMA-Net on various datasets, including 2D chest X-ray segmentation dataset, 2D fundus image segmentation dataset, 3D breast DCE-MRI segmentation dataset, and the QUBIQ multi-task segmentation dataset. Code will be released at https://github.com/wangjin2945/UMA-Net.

The application of artificial intelligence in upper gastrointestinal cancers

Upper gastrointestinal cancers, mainly comprising esophageal and gastric cancers, are among the most prevalent cancers worldwide. There are many new cases of upper gastrointestinal cancers annually, and the survival rate tends to be low. Therefore, timely screening, precise diagnosis, appropriate treatment strategies, and effective prognosis are crucial for patients with upper gastrointestinal cancers. In recent years, an increasing number of studies suggest that artificial intelligence (AI) technology can effectively address clinical tasks related to upper gastrointestinal cancers. These studies mainly focus on four aspects: screening, diagnosis, treatment, and prognosis. In this review, we focus on the application of AI technology in clinical tasks related to upper gastrointestinal cancers. Firstly, the basic application pipelines of radiomics and deep learning in medical image analysis were introduced. Furthermore, we separately reviewed the application of AI technology in the aforementioned aspects for both esophageal and gastric cancers. Finally, the current limitations and challenges faced in the field of upper gastrointestinal cancers were summarized, and explorations were conducted on the selection of AI algorithms in various scenarios, the popularization of early screening, the clinical applications of AI, and large multimodal models.

PCRFed: personalized federated learning with contrastive representation for non-independently and identically distributed medical image segmentation

Federated learning (FL) has shown great potential in addressing data privacy issues in medical image analysis. However, varying data distributions across different sites can create challenges in aggregating client models and achieving good global model performance. In this study, we propose a novel personalized contrastive representation FL framework, named PCRFed, which leverages contrastive representation learning to address the non-independent and identically distributed (non-IID) challenge and dynamically adjusts the distance between local clients and the global model to improve each client's performance without incurring additional communication costs. The proposed weighted model-contrastive loss provides additional regularization for local models, optimizing their respective distributions while effectively utilizing information from all clients to mitigate performance challenges caused by insufficient local data. The PCRFed approach was evaluated on two non-IID medical image segmentation datasets, and the results show that it outperforms several state-of-the-art FL frameworks, achieving higher single-client performance while ensuring privacy preservation and minimal communication costs. Our PCRFed framework can be adapted to various encoder-decoder segmentation network architectures and holds significant potential for advancing the use of FL in real-world medical applications. Based on a multi-center dataset, our framework demonstrates superior overall performance and higher single-client performance, achieving a 2.63% increase in the average Dice score for prostate segmentation.

LA-ResUNet: Attention-based network for longitudinal liver tumor segmentation from CT images

Longitudinal liver tumor segmentation plays a fundamental role in studying and monitoring the progression of associated diseases. The correlation and differences between longitudinal data can further improve segmentation performance, which are inevitably omitted in single-time-point segmentation. However, there is no research in this field due to the lack of relevant data. To this issue, we collect and annotate the first longitudinal liver tumor segmentation benchmark dataset. A novel strategy that utilizes images from one time point to facilitate the image segmentation from another time point of the same patient is presented. On this basis, we propose a longitudinal attention based residual U-shaped network. Within it, a channel & spatial attention module quantifies both channel-wise and spatial-wise dependencies of each feature to refine feature representations. And a longitudinal co-segmentation module captures cross-temporal correlation to recalibrate the feature at one time point according to another one for enhanced segmentation. Longitudinal segmentation is achieved by plugging these two multi-scale modules into each layer of the backbone network. Extensive experiments on our CT liver tumor dataset and an MRI brain tumor dataset have validated the effectiveness of the established strategy and the longitudinal segmentation ability of our network. Ablation studies have verified the functions of the proposed modules and their respective components.

Ensemble-based multiclass lung cancer classification using hybrid CNN-SVD feature extraction and selection method

Lung cancer (LC) is a leading cause of cancer-related fatalities worldwide, underscoring the urgency of early detection for improved patient outcomes. The main objective of this research is to harness the noble strategies of artificial intelligence for identifying and classifying lung cancers more precisely from CT scan images at the early stage. This study introduces a novel lung cancer detection method, which was mainly focused on Convolutional Neural Networks (CNN) and was later customized for binary and multiclass classification utilizing a publicly available dataset of chest CT scan images of lung cancer. The main contribution of this research lies in its use of a hybrid CNN-SVD (Singular Value Decomposition) method and the use of a robust voting ensemble approach, which results in superior accuracy and effectiveness for mitigating potential errors. By employing contrast-limited adaptive histogram equalization (CLAHE), contrast-enhanced images were generated with minimal noise and prominent distinctive features. Subsequently, a CNN-SVD-Ensemble model was implemented to extract important features and reduce dimensionality. The extracted features were then processed by a set of ML algorithms along with a voting ensemble approach. Additionally, Gradient-weighted Class Activation Mapping (Grad-CAM) was integrated as an explainable AI (XAI) technique for enhancing model transparency by highlighting key influencing regions in the CT scans, which improved interpretability and ensured reliable and trustworthy results for clinical applications. This research offered state-of-the-art results, which achieved remarkable performance metrics with an accuracy, AUC, precision, recall, F1 score, Cohen's Kappa and Matthews Correlation Coefficient (MCC) of 99.49%, 99.73%, 100%, 99%, 99%, 99.15% and 99.16%, respectively, addressing the prior research gaps and setting a new benchmark in the field. Furthermore, in binary class classification, all the performance indicators attained a perfect score of 100%. The robustness of the suggested approach offered more reliable and impactful insights in the medical field, thus improving existing knowledge and setting the stage for future innovations.

P2ED: A four-quadrant framework for progressive prompt enhancement in 3D interactive medical imaging segmentation

Interactive segmentation allows active user participation to enhance output quality and resolve ambiguities. This may be especially indispensable to medical image segmentation to address complex anatomy and customization to varying user requirements. Existing approaches often encounter issues such as information dilution, limited adaptability to diverse user interactions, and insufficient response. To address these challenges, we present a novel 3D interactive framework P2ED that divides the task into four quadrants. It is equipped with a multi-granular prompt encrypted to extract prompt features from various hierarchical levels, along with a progressive hierarchical prompt decrypter to adaptively heighten the attention to the scarce prompt features along three spatial axes. Finally, it is appended by a calibration module to further align the prediction with user intentions. Extensive experiments demonstrate that the proposed P2ED achieves accurate results with fewer user interactions compared to state-of-the-art methods and is effective in promoting the upper limit of segmentation performance. The code will be released in https://github.com/chuyhu/P2ED.

MP-FocalUNet: Multiscale parallel focal self-attention U-Net for medical image segmentation

BACKGROUND AND OBJECTIVE

Medical image segmentation has been significantly improved in recent years with the progress of Convolutional Neural Networks (CNNs). Due to the inherent limitations of convolutional operations, CNNs perform poorly in learning the correlation information between global and long-range features. To solve this problem, some existing solutions rely on building deep encoders and down-sampling operations, but such methods are prone to produce redundant network structures and lose local details. Therefore, medical image segmentation tasks require better solutions to improve the modeling of the global context, while maintaining a strong grasp of the low-level details.

METHODS

We propose a novel multiscale parallel branch architecture (MP-FocalUNet). On the encoder side of MP-FocalUNet, dual-scale sub-networks are used to extract information of different scales. A cross-scale "Feature Fusion" (FF) module was proposed to explore the potential of dual branch networks and fully utilize feature representations at different scales. On the decoder side, combined with the traditional CNN in parallel, focal self-attention is used for long-distance modeling, which can effectively capture the global dependencies and underlying spatial details in a shallower way.

RESULTS

Our proposed method is evaluated on both abdominal organ segmentation datasets and automatic cardiac diagnosis challenge datasets. Our method consistently outperforms several state-of-the-art segmentation methods with an average Dice score of 82.45 % (2.68 % higher than HC-Net) and 91.44 % (0.35 % higher than HC-Net) on the abdominal organ datasets and the automatic cardiac diagnosis challenge datasets, respectively.

CONCLUSIONS

Our MP-FocalUNet is a novel encoder-decoder based multiscale parallel branch Transformer network, which solves the problem of insufficient long-distance modeling in CNNs and fuses image information at different scales. Extensive experiments on abdominal and cardiac medical image segmentation tasks show that our MP-FocalUNet outperforms other state-of-the-art methods. In the future, our work will focus on designing more lightweight Transformer-based models and better learning pixel-level intrinsic structural features generated by patch division in visual Transformers.

Evaluation of stroke sequelae and rehabilitation effect on brain tumor by neuroimaging technique: A comparative study

This study aims at the limitations of traditional methods in the evaluation of stroke sequelae and rehabilitation effect monitoring, especially for the accurate identification and tracking of brain injury areas. To overcome these challenges, we introduce an advanced neuroimaging technology based on deep learning, the SWI-BITR-UNet model. This model, introduced as novel Machine Learning (ML) model, combines the SWIN Transformer's local receptive field and shift mechanism, and the effective feature fusion strategy in the U-Net architecture, aiming to improve the accuracy of brain lesion region segmentation in multimodal MRI scans. Through the application of a 3-D CNN encoder and decoder, as well as the integration of the CBAM attention module and jump connection, the model can finely capture and refine features, to achieve a level of segmentation accuracy comparable to that of manual segmentation by experts. This study introduces a 3D CNN encoder-decoder architecture specifically designed to enhance the processing capabilities of 3D medical imaging data. The development of the 3D CNN model utilizes the ADAM optimization algorithm to facilitate the training process. The Bra2020 dataset is utilized to assess the accuracy of the proposed deep learning neural network. By employing skip connections, the model effectively integrates the high-resolution features from the encoder with the up-sampling features from the decoder, thereby increasing the model's sensitivity to 3D spatial characteristics. To assess both the training and testing phases, the SWI-BITR-Unet model is trained using reliable datasets and evaluated through a comprehensive array of statistical metrics, including Recall (Rec), Precision (Pre), F1 test score, Kappa Coefficient (KC), mean Intersection over Union (mIoU), and Receiver Operating Characteristic-Area Under Curve (ROC-AUC). Furthermore, various machine learning models, such as Random Forest (RF), Support Vector Machine (SVM), Extreme Gradient Boosting (XGBoost), Categorical Boosting (CatBoost), Adaptive Boosting (AdaBoost), and K-Nearest Neighbor (KNN), have been employed to analyze tumor progression in the brain, with performance characterized by Hausdorff distance. In From the performance of ML models, the SWI-BITR-Unet model was more accurate than other models. Subsequently, regarding DICE coefficient values, the segmentation maps (annotation maps of brain tumor distributions) generated by the ML models indicated the models's capability to autonomously delineate areas such as the tumor core (TC) and the enhancing tumor (ET). Moreover, the efficacy of the proposed machine learning models demonstrated superiority over existing research in the field. The computational efficiency and the ability to handle long-distance dependencies of the model make it particularly suitable for applications in clinical Settings. The results showed that the SNA-BITR-UNet model can not only effectively identify and monitor the subtle changes in the stroke injury area, but also provided a new and efficient tool in the rehabilitation process, providing a scientific basis for developing personalized rehabilitation plans.

Multi-dimensional consistency learning between 2D Swin U-Net and 3D U-Net for intestine segmentation from CT volume

PURPOSE

The paper introduces a novel two-step network based on semi-supervised learning for intestine segmentation from CT volumes. The intestine folds in the abdomen with complex spatial structures and contact with neighboring organs that bring difficulty for accurate segmentation and labeling at the pixel level. We propose a multi-dimensional consistency learning method to reduce the insufficient intestine segmentation results caused by complex structures and the limited labeled dataset.

METHODS

We designed a two-stage model to segment the intestine. In stage 1, a 2D Swin U-Net is trained using labeled data to generate pseudo-labels for unlabeled data. In stage 2, a 3D U-Net is trained using labeled and unlabeled data to create the final segmentation model. The model comprises two networks from different dimensions, capturing more comprehensive representations of the intestine and potentially enhancing the model's performance in intestine segmentation.

RESULTS

We used 59 CT volumes to validate the effectiveness of our method. The experiment was repeated three times getting the average as the final result. Compared to the baseline method, our method improved 3.25% Dice score and 6.84% recall rate.

CONCLUSION

The proposed method is based on semi-supervised learning and involves training both 2D Swin U-Net and 3D U-Net. The method mitigates the impact of limited labeled data and maintains consistncy of multi-dimensional outputs from the two networks to improve the segmentation accuracy. Compared to previous methods, our method demonstrates superior segmentation performance.

Multi-label segmentation of carpal bones in MRI using expansion transfer learning

Objective.The purpose of this study was to develop a robust deep learning approach trained with a smallin-vivoMRI dataset for multi-label segmentation of all eight carpal bones for therapy planning and wrist dynamic analysis.Approach.A small dataset of 15 3.0-T MRI scans from five health subjects was employed within this study. The MRI data was variable with respect to the field of view (FOV), wide range of image intensity, and joint pose. Atwo-stagesegmentation pipeline using modified 3D U-Net was proposed. In thefirst stage, a novel architecture, introduced as expansion transfer learning (ETL), cascades the use of a focused region of interest (ROI) cropped around ground truth for pretraining and a subsequent transfer by an expansion to the original FOV for a primary prediction. The bounding box around the ROI generated was utilized in thesecond stagefor high-accuracy, labeled segmentations of eight carpal bones. Different metrics including dice similarity coefficient (DSC), average surface distance (ASD) and hausdorff distance (HD) were used to evaluate performance between proposed and four state-of-the-art approaches.Main results.With an average DSC of 87.8 %, an ASD of 0.46 mm, an average HD of 2.42 mm in all datasets (96.1 %, 0.16 mm, 1.38 mm in 12 datasets after exclusion criteria, respectively), the proposed approach showed an overall strongest performance than comparisons.Significance.To our best knowledge, this is the first CNN-based multi-label segmentation approach for MRI human carpal bones. The ETL introduced in this work improved the ability to localize a small ROI in a large FOV. Overall, the interplay of atwo-stageapproach and ETL culminated in convincingly accurate segmentation scores despite a very small amount of image data.

Lesion segmentation method for multiple types of liver cancer based on balanced dice loss

BACKGROUND

Obtaining accurate segmentation regions for liver cancer is of paramount importance for the clinical diagnosis and treatment of the disease. In recent years, a large number of variants of deep learning based liver cancer segmentation methods have been proposed to assist radiologists. Due to the differences in characteristics between different types of liver tumors and data imbalance, it is difficult to train a deep model that can achieve accurate segmentation for multiple types of liver cancer.

PURPOSE

In this paper, We propose a balance Dice Loss(BD Loss) function for balanced learning of multiple categories segmentation features. We also introduce a comprehensive method based on BD Loss to achieve accurate segmentation of multiple categories of liver cancer.

MATERIALS AND METHODS

We retrospectively collected computed tomography (CT) screening images and tumor segmentation of 591 patients with malignant liver tumors from West China Hospital of Sichuan University. We use the proposed BD Loss to train a deep model that can segment multiple types of liver tumors and, through a greedy parameter averaging algorithm (GPA algorithm) obtain a more generalized segmentation model. Finally, we employ model integration and our proposed post-processing method, which leverages inter-slice information, to achieve more accurate segmentation of liver cancer lesions.

RESULTS

We evaluated the performance of our proposed automatic liver cancer segmentation method on the dataset we collected. The BD loss we proposed can effectively mitigate the adverse effects of data imbalance on the segmentation model. Our proposed method can achieve a dice per case (DPC) of 0.819 (95%CI 0.798-0.841), significantly higher than baseline which achieve a DPC of 0.768(95%CI 0.740-0.796).

CONCLUSIONS

The differences in CT images between different types of liver cancer necessitate deep learning models to learn distinct features. Our method addresses this challenge, enabling balanced and accurate segmentation performance across multiple types of liver cancer.

A frequency attention-embedded network for polyp segmentation

Gastrointestinal polyps are observed and treated under endoscopy, so there presents significant challenges to advance endoscopy imaging segmentation of polyps. Current methodologies often falter in distinguishing complex polyp structures within diverse (mucosal) tissue environments. In this paper, we propose the Frequency Attention-Embedded Network (FAENet), a novel approach leveraging frequency-based attention mechanisms to enhance polyp segmentation accuracy significantly. FAENet ingeniously segregates and processes image data into high and low-frequency components, enabling precise delineation of polyp boundaries and internal structures by integrating intra-component and cross-component attention mechanisms. This method not only preserves essential edge details but also refines the learned representation attentively, ensuring robust segmentation across varied imaging conditions. Comprehensive evaluations on two public datasets, Kvasir-SEG and CVC-ClinicDB, demonstrate FAENet's superiority over several state-of-the-art models in terms of Dice coefficient, Intersection over Union (IoU), sensitivity, and specificity. The results affirm that FAENet's advanced attention mechanisms significantly improve the segmentation quality, outperforming traditional and contemporary techniques. FAENet's success indicates its potential to revolutionize polyp segmentation in clinical practices, fostering diagnosis and efficient treatment of gastrointestinal polyps.

Comparative analysis of the DCNN and HFCNN Based Computerized detection of liver cancer

Liver cancer detection is critically important in the discipline of biomedical image testing and diagnosis. Researchers have explored numerous machine learning (ML) techniques and deep learning (DL) approaches aimed at the automated recognition of liver disease by analysing computed tomography (CT) images. This study compares two frameworks, Deep Convolutional Neural Network (DCNN) and Hierarchical Fusion Convolutional Neural Networks (HFCNN), to assess their effectiveness in liver cancer segmentation. The contribution includes enhancing the edges and textures of CT images through filtering to achieve precise liver segmentation. Additionally, an existing DL framework was employed for liver cancer detection and segmentation. The strengths of this paper include a clear emphasis on the criticality of liver cancer detection in biomedical imaging and diagnostics. It also highlights the challenges associated with CT image detection and segmentation and provides a comprehensive summary of recent literature. However, certain difficulties arise during the detection process in CT images due to overlapping structures, such as bile ducts, blood vessels, image noise, textural changes, size and location variations, and inherent heterogeneity. These factors may lead to segmentation errors and subsequently different analyses. This research analysis compares two advanced methodologies, DCNN and HFCNN, for liver cancer detection. The evaluation of DCNN and HFCNN in liver cancer detection is conducted using multiple performance metrics, including precision, F1-score, recall, and accuracy. This comprehensive assessment provides a detailed evaluation of these models' effectiveness compared to other state-of-the-art methods in identifying liver cancer.

Contour-constrained branch U-Net for accurate left ventricular segmentation in echocardiography

Using echocardiography to assess the left ventricular function is one of the most crucial cardiac examinations in clinical diagnosis, and LV segmentation plays a particularly vital role in medical image processing as many important clinical diagnostic parameters are derived from the segmentation results, such as ejection function. However, echocardiography typically has a lower resolution and contains a significant amount of noise and motion artifacts, making it a challenge to accurate segmentation, especially in the region of the cardiac chamber boundary, which significantly restricts the accurate calculation of subsequent clinical parameters. In this paper, our goal is to achieve accurate LV segmentation through a simplified approach by introducing a branch sub-network into the decoder of the traditional U-Net. Specifically, we employed the LV contour features to supervise the branch decoding process and used a cross attention module to facilitate the interaction relationship between the branch and the original decoding process, thereby improving the segmentation performance in the region LV boundaries. In the experiments, the proposed branch U-Net (BU-Net) demonstrated superior performance on CAMUS and EchoNet-dynamic public echocardiography segmentation datasets in comparison to state-of-the-art segmentation models, without the need for complex residual connections or transformer-based architectures. Our codes are publicly available at Anonymous Github https://anonymous.4open.science/r/Anoymous_two-BFF2/ .

Challenges and solutions of deep learning-based automated liver segmentation: A systematic review

The liver is one of the vital organs in the body. Precise liver segmentation in medical images is essential for liver disease treatment. The deep learning-based liver segmentation process faces several challenges. This research aims to analyze the challenges of liver segmentation in prior studies and identify the modifications made to network models and other enhancements implemented by researchers to tackle each challenge. In total, 88 articles from Scopus and ScienceDirect databases published between January 2016 and January 2022 have been studied. The liver segmentation challenges are classified into five main categories, each containing some subcategories. For each challenge, the proposed technique to overcome the challenge is investigated. The provided report details the authors, publication years, dataset types, imaging technologies, and evaluation metrics of all references for comparison. Additionally, a summary table outlines the challenges and solutions.

Prior-Knowledge Embedded U-Net-Based Fully Automatic Vessel Wall Volume Measurement of the Carotid Artery in 3D Ultrasound Image

The vessel-wall-volume (VWV) measured based on three-dimensional (3D) carotid artery (CA) ultrasound (US) images can help to assess carotid atherosclerosis and manage patients at risk for stroke. Manual involvement for measurement work is subjective and requires well-trained operators, and fully automatic measurement tools are not yet available. Thereby, we proposed a fully automatic VWV measurement framework (Auto-VWV) using a CA prior-knowledge embedded U-Net (CAP-UNet) to measure the VWV from 3D CA US images without manual intervention. The Auto-VWV framework is designed to improve the repeated VWV measuring consistency, which resulted in the first fully automatic framework for VWV measurement. CAP-UNet is developed to improve segmentation accuracy on the whole CA, which composed of a U-Net type backbone and three additional prior-knowledge learning modules. Specifically, a continuity learning module is used to learn the spatial continuity of the arteries in a sequence of image slices. A voxel evolution learning module was designed to learn the evolution of the artery in adjacent slices, and a topology learning module was used to learn the unique topology of the carotid artery. In two 3D CA US datasets, CAP-UNet architecture achieved state-of-the-art performance compared to eight competing models. Furthermore, CAP-UNet-based Auto-VWV achieved better accuracy and consistency than Auto-VWV based on competing models in the simulated repeated measurement. Finally, using 10 pairs of real repeatedly scanned samples, Auto-VWV achieved better VWV measurement reproducibility than intra- and inter-operator manual measurements. The code is available at https://github.com/Yue9603/Auto-VWV.

Generalizability of lesion detection and segmentation when ScaleNAS is trained on a large multi-organ dataset and validated in the liver

BACKGROUND

Tumor assessment through imaging is crucial for diagnosing and treating cancer. Lesions in the liver, a common site for metastatic disease, are particularly challenging to accurately detect and segment. This labor-intensive task is subject to individual variation, which drives interest in automation using artificial intelligence (AI).

PURPOSE

Evaluate AI for lesion detection and lesion segmentation using CT in the context of human performance on the same task. Use internal testing to determine how an AI-developed model (ScaleNAS) trained on lesions in multiple organs performs when tested specifically on liver lesions in a dataset integrating real-world and clinical trial data. Use external testing to evaluate whether ScaleNAS's performance generalizes to publicly available colorectal liver metastases (CRLM) from The Cancer Imaging Archive (TCIA).

METHODS

The CUPA study dataset included patients whose CT scan of chest, abdomen, or pelvis at Columbia University between 2010-2020 indicated solid tumors (CUIMC, n = 5011) and from two clinical trials in metastatic colorectal cancer, PRIME (n = 1183) and Amgen (n = 463). Inclusion required ≥1 measurable lesion; exclusion criteria eliminated 1566 patients. Data were divided at the patient level into training (n = 3996), validation (n = 570), and testing (n = 1529) sets. To create the reference standard for training and validation, each case was annotated by one of six radiologists, randomly assigned, who marked the CUPA lesions without access to any previous annotations. For internal testing we refined the CUPA test set to contain only patients who had liver lesions (n = 525) and formed an enhanced reference standard through expert consensus reviewing prior annotations. For external testing, TCIA-CRLM (n = 197) formed the test set. The reference standard for TCIA-CRLM was formed by consensus review of the original annotation and contours by two new radiologists. Metrics for lesion detection were sensitivity and false positives. Lesion segmentation was assessed with median Dice coefficient, under-segmentation ratio (USR), and over-segmentation ratio (OSR). Subgroup analysis examined the influence of lesion size ≥ 10  mm (measurable by RECIST1.1) versus all lesions (important for early identification of disease progression).

RESULTS

ScaleNAS trained on all lesions achieved sensitivity of 71.4% and Dice of 70.2% for liver lesions in the CUPA internal test set (3,495 lesions) and sensitivity of 68.2% and Dice 64.2% in the TCIA-CRLM external test set (638 lesions). Human radiologists had mean sensitivity of 53.5% and Dice of 73.9% in CUPA and sensitivity of 84.1% and Dice of 88.4% in TCIA-CRLM. Performance improved for ScaleNAS and radiologists in the subgroup of lesions that excluded sub-centimeter lesions.

CONCLUSIONS

Our study presents the first evaluation of ScaleNAS in medical imaging, demonstrating its liver lesion detection and segmentation performance across diverse datasets. Using consensus reference standards from multiple radiologists, we addressed inter-observer variability and contributed to consistency in lesion annotation. While ScaleNAS does not surpass radiologists in performance, it offers fast and reliable results with potential utility in providing initial contours for radiologists. Future work will extend this model to lung and lymph node lesions, ultimately aiming to enhance clinical applications by generalizing detection and segmentation across tissue types.

PFPRNet: A Phase-Wise Feature Pyramid With Retention Network for Polyp Segmentation

Early detection of colonic polyps is crucial for the prevention and diagnosis of colorectal cancer. Currently, deep learning-based polyp segmentation methods have become mainstream and achieved remarkable results. Acquiring a large number of labeled data is time-consuming and labor-intensive, and meanwhile the presence of numerous similar wrinkles in polyp images also hampers model prediction performance. In this paper, we propose a novel approach called Phase-wise Feature Pyramid with Retention Network (PFPRNet), which leverages a pre-trained Transformer-based Encoder to obtain multi-scale feature maps. A Phase-wise Feature Pyramid with Retention Decoder is designed to gradually integrate global features into local features and guide the model's attention towards key regions. Additionally, our custom Enhance Perception module enables capturing image information from a broader perspective. Finally, we introduce an innovative Low-layer Retention module as an alternative to Transformer for more efficient global attention modeling. Evaluation results on several widely-used polyp segmentation datasets demonstrate that our proposed method has strong learning ability and generalization capability, and outperforms the state-of-the-art approaches.

Multi scale multi attention network for blood vessel segmentation in fundus images

Precise segmentation of retinal vasculature is crucial for the early detection, diagnosis, and treatment of vision-threatening ailments. However, this task is challenging due to limited contextual information, variations in vessel thicknesses, the complexity of vessel structures, and the potential for confusion with lesions. In this paper, we introduce a novel approach, the MSMA Net model, which overcomes these challenges by replacing traditional convolution blocks and skip connections with an improved multi-scale squeeze and excitation block (MSSE Block) and Bottleneck residual paths (B-Res paths) with spatial attention blocks (SAB). Our experimental findings on publicly available datasets of fundus images, specifically DRIVE, STARE, CHASE_DB1, HRF and DR HAGIS consistently demonstrate that our approach outperforms other segmentation techniques, achieving higher accuracy, sensitivity, Dice score, and area under the receiver operator characteristic (AUC) in the segmentation of blood vessels with different thicknesses, even in situations involving diverse contextual information, the presence of coexisting lesions, and intricate vessel morphologies.

An Efficient Retinal Fluid Segmentation Network Based on Large Receptive Field Context Capture for Optical Coherence Tomography Images

Optical Coherence Tomography (OCT) is a crucial imaging modality for diagnosing and monitoring retinal diseases. However, the accurate segmentation of fluid regions and lesions remains challenging due to noise, low contrast, and blurred edges in OCT images. Although feature modeling with wide or global receptive fields offers a feasible solution, it typically leads to significant computational overhead. To address these challenges, we propose LKMU-Lite, a lightweight U-shaped segmentation method tailored for retinal fluid segmentation. LKMU-Lite integrates a Decoupled Large Kernel Attention (DLKA) module that captures both local patterns and long-range dependencies, thereby enhancing feature representation. Additionally, it incorporates a Multi-scale Group Perception (MSGP) module that employs Dilated Convolutions with varying receptive field scales to effectively predict lesions of different shapes and sizes. Furthermore, a novel Aggregating-Shift decoder is proposed, reducing model complexity while preserving feature integrity. With only 1.02 million parameters and a computational complexity of 3.82 G FLOPs, LKMU-Lite achieves state-of-the-art performance across multiple metrics on the ICF and RETOUCH datasets, demonstrating both its efficiency and generalizability compared to existing methods.

CAISeg: A Clustering-Aided Interactive Network for Lesion Segmentation in 3D Medical Imaging

Accurate lesion segmentation in medical imaging is critical for medical diagnosis and treatment. Lesions' diverse and heterogeneous characteristics often present a distinct long-tail distribution, posing difficulties for automatic methods. Currently, interactive segmentation approaches have shown promise in improving accuracy, but still struggle to deal with tail features. This triggers a demand of effective utilizing strategies of user interaction. To this end, we propose a novel point-based interactive segmentation model called Clustering-Aided Interactive Segmentation Network (CAISeg) in 3D medical imaging. A customized Interaction-Guided Module (IGM) adopts the concept of clustering to capture features that are semantically similar to interaction points. These clustered features are then mapped to the head regions of the prompted category to facilitate more precise classification. Meanwhile, we put forward a Focus Guided Loss function to grant the network an inductive bias towards user interaction through assigning higher weights to voxels closer to the prompted points, thereby improving the responsiveness efficiency to user guidance. Evaluation across brain tumor, colon cancer, lung cancer, and pancreas cancer segmentation tasks show CAISeg's superiority over the state-of-the-art methods. It outperforms the fully automated segmentation models in accuracy, and achieves results comparable to or better than those of the leading point-based interactive methods while requiring fewer prompt points. Furthermore, we discover that CAISeg possesses good interpretability at various stages, which endows CAISeg with potential clinical application value.

Segmentation and Vascular Vectorization for Coronary Artery by Geometry-Based Cascaded Neural Network

Segmentation of the coronary artery is an important task for the quantitative analysis of coronary computed tomography angiography (CCTA) images and is being stimulated by the field of deep learning. However, the complex structures with tiny and narrow branches of the coronary artery bring it a great challenge. Coupled with the medical image limitations of low resolution and poor contrast, fragmentations of segmented vessels frequently occur in the prediction. Therefore, a geometry-based cascaded segmentation method is proposed for the coronary artery, which has the following innovations: 1) Integrating geometric deformation networks, we design a cascaded network for segmenting the coronary artery and vectorizing results. The generated meshes of the coronary artery are continuous and accurate for twisted and sophisticated coronary artery structures, without fragmentations. 2) Different from mesh annotations generated by the traditional marching cube method from voxel-based labels, a finer vectorized mesh of the coronary artery is reconstructed with the regularized morphology. The novel mesh annotation benefits the geometry-based segmentation network, avoiding bifurcation adhesion and point cloud dispersion in intricate branches. 3) A dataset named CCA-200 is collected, consisting of 200 CCTA images with coronary artery disease. The ground truths of 200 cases are coronary internal diameter annotations by professional radiologists. Extensive experiments verify our method on our collected dataset CCA-200 and public ASOCA dataset, with a Dice of 0.778 on CCA-200 and 0.895 on ASOCA, showing superior results. Especially, our geometry-based model generates an accurate, intact and smooth coronary artery, devoid of any fragmentations of segmented vessels.

MixUNETR: A U-shaped network based on W-MSA and depth-wise convolution with channel and spatial interactions for zonal prostate segmentation in MRI

Magnetic resonance imaging (MRI) plays a pivotal role in diagnosing and staging prostate cancer. Precise delineation of the peripheral zone (PZ) and transition zone (TZ) within prostate MRI is essential for accurate diagnosis and subsequent artificial intelligence-driven analysis. However, existing segmentation methods are limited by ambiguous boundaries, shape variations and texture complexities between PZ and TZ. Moreover, they suffer from inadequate modeling capabilities and limited receptive fields. To address these challenges, we propose a Enhanced MixFormer, which integrates window-based multi-head self-attention (W-MSA) and depth-wise convolution with parallel design and cross-branch bidirectional interaction. We further introduce MixUNETR, which use multiple Enhanced MixFormers as encoder to extract features from both PZ and TZ in prostate MRI. This augmentation effectively enlarges the receptive field and enhances the modeling capability of W-MSA, ultimately improving the extraction of both global and local feature information from PZ and TZ, thereby addressing mis-segmentation and challenges in delineating boundaries between them. Extensive experiments were conducted, comparing MixUNETR with several state-of-the-art methods on the Prostate158, ProstateX public datasets and private dataset. The results consistently demonstrate the accuracy and robustness of MixUNETR in MRI prostate segmentation. Our code of methods is available at https://github.com/skyous779/MixUNETR.git.

Dynamic Subcluster-Aware Network for Few-Shot Skin Disease Classification

This article addresses the problem of few-shot skin disease classification by introducing a novel approach called the subcluster-aware network (SCAN) that enhances accuracy in diagnosing rare skin diseases. The key insight motivating the design of SCAN is the observation that skin disease images within a class often exhibit multiple subclusters, characterized by distinct variations in appearance. To improve the performance of few-shot learning (FSL), we focus on learning a high-quality feature encoder that captures the unique subclustered representations within each disease class, enabling better characterization of feature distributions. Specifically, SCAN follows a dual-branch framework, where the first branch learns classwise features to distinguish different skin diseases, and the second branch aims to learn features, which can effectively partition each class into several groups so as to preserve the subclustered structure within each class. To achieve the objective of the second branch, we present a cluster loss to learn image similarities via unsupervised clustering. To ensure that the samples in each subcluster are from the same class, we further design a purity loss to refine the unsupervised clustering results. We evaluate the proposed approach on two public datasets for few-shot skin disease classification. The experimental results validate that our framework outperforms the state-of-the-art methods by around 2%-5% in terms of sensitivity, specificity, accuracy, and F1-score on the SD-198 and Derm7pt datasets.

Large-scale multi-center CT and MRI segmentation of pancreas with deep learning

Automated volumetric segmentation of the pancreas on cross-sectional imaging is needed for diagnosis and follow-up of pancreatic diseases. While CT-based pancreatic segmentation is more established, MRI-based segmentation methods are understudied, largely due to a lack of publicly available datasets, benchmarking research efforts, and domain-specific deep learning methods. In this retrospective study, we collected a large dataset (767 scans from 499 participants) of T1-weighted (T1 W) and T2-weighted (T2 W) abdominal MRI series from five centers between March 2004 and November 2022. We also collected CT scans of 1,350 patients from publicly available sources for benchmarking purposes. We introduced a new pancreas segmentation method, called PanSegNet, combining the strengths of nnUNet and a Transformer network with a new linear attention module enabling volumetric computation. We tested PanSegNet's accuracy in cross-modality (a total of 2,117 scans) and cross-center settings with Dice and Hausdorff distance (HD95) evaluation metrics. We used Cohen's kappa statistics for intra and inter-rater agreement evaluation and paired t-tests for volume and Dice comparisons, respectively. For segmentation accuracy, we achieved Dice coefficients of 88.3% (±7.2%, at case level) with CT, 85.0% (±7.9%) with T1 W MRI, and 86.3% (±6.4%) with T2 W MRI. There was a high correlation for pancreas volume prediction with R2 of 0.91, 0.84, and 0.85 for CT, T1 W, and T2 W, respectively. We found moderate inter-observer (0.624 and 0.638 for T1 W and T2 W MRI, respectively) and high intra-observer agreement scores. All MRI data is made available at https://osf.io/kysnj/. Our source code is available at https://github.com/NUBagciLab/PaNSegNet.

A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities

Biomedical image analysis is fundamental for biomedical discovery. Holistic image analysis comprises interdependent subtasks such as segmentation, detection and recognition, which are tackled separately by traditional approaches. Here, we propose BiomedParse, a biomedical foundation model that can jointly conduct segmentation, detection and recognition across nine imaging modalities. This joint learning improves the accuracy for individual tasks and enables new applications such as segmenting all relevant objects in an image through a textual description. To train BiomedParse, we created a large dataset comprising over 6 million triples of image, segmentation mask and textual description by leveraging natural language labels or descriptions accompanying existing datasets. We showed that BiomedParse outperformed existing methods on image segmentation across nine imaging modalities, with larger improvement on objects with irregular shapes. We further showed that BiomedParse can simultaneously segment and label all objects in an image. In summary, BiomedParse is an all-in-one tool for biomedical image analysis on all major image modalities, paving the path for efficient and accurate image-based biomedical discovery.

Deep learning-based 3D quantitative total tumor burden predicts early recurrence of BCLC A and B HCC after resection

OBJECTIVES

This study aimed to evaluate the potential of deep learning (DL)-assisted automated three-dimensional quantitative tumor burden at MRI to predict postoperative early recurrence (ER) of hepatocellular carcinoma (HCC).

MATERIALS AND METHODS

This was a single-center retrospective study enrolling patients who underwent resection for BCLC A and B HCC and preoperative contrast-enhanced MRI. Quantitative total tumor volume (cm3) and total tumor burden (TTB, %) were obtained using a DL automated segmentation tool. Radiologists' visual assessment was used to ensure the quality control of automated segmentation. The prognostic value of clinicopathological variables and tumor burden-related parameters for ER was determined by Cox regression analyses.

RESULTS

A total of 592 patients were included, with 525 and 67 patients assigned to BCLC A and B, respectively (2-year ER rate: 30.0% vs. 45.3%; hazard ratio (HR) = 1.8; p = 0.007). TTB was the most important predictor of ER (HR = 2.2; p < 0.001). Using 6.84% as the threshold of TTB, two ER risk strata were obtained in overall (p < 0.001), BCLC A (p < 0.001), and BCLC B (p = 0.027) patients, respectively. The BCLC B low-TTB patients had a similar risk for ER to BCLC A patients and thus were reassigned to a BCLC An stage; whilst the BCLC B high-TTB patients remained in a BCLC Bn stage. The 2-year ER rate was 30.5% for BCLC An patients vs. 58.1% for BCLC Bn patients (HR = 2.8; p < 0.001).

CONCLUSIONS

TTB determined by DL-based automated segmentation at MRI was a predictive biomarker for postoperative ER and facilitated refined subcategorization of patients within BCLC stages A and B.

CLINICAL RELEVANCE STATEMENT

Total tumor burden derived by deep learning-based automated segmentation at MRI may serve as an imaging biomarker for predicting early recurrence, thereby improving subclassification of Barcelona Clinic Liver Cancer A and B hepatocellular carcinoma patients after hepatectomy.

KEY POINTS

Total tumor burden (TTB) is important for Barcelona Clinic Liver Cancer (BCLC) staging, but is heterogenous. TTB derived by deep learning-based automated segmentation was predictive of postoperative early recurrence. Incorporating TTB into the BCLC algorithm resulted in successful subcategorization of BCLC A and B patients.

Labeled-to-unlabeled distribution alignment for partially-supervised multi-organ medical image segmentation

Partially-supervised multi-organ medical image segmentation aims to develop a unified semantic segmentation model by utilizing multiple partially-labeled datasets, with each dataset providing labels for a single class of organs. However, the limited availability of labeled foreground organs and the absence of supervision to distinguish unlabeled foreground organs from the background pose a significant challenge, which leads to a distribution mismatch between labeled and unlabeled pixels. Although existing pseudo-labeling methods can be employed to learn from both labeled and unlabeled pixels, they are prone to performance degradation in this task, as they rely on the assumption that labeled and unlabeled pixels have the same distribution. In this paper, to address the problem of distribution mismatch, we propose a labeled-to-unlabeled distribution alignment (LTUDA) framework that aligns feature distributions and enhances discriminative capability. Specifically, we introduce a cross-set data augmentation strategy, which performs region-level mixing between labeled and unlabeled organs to reduce distribution discrepancy and enrich the training set. Besides, we propose a prototype-based distribution alignment method that implicitly reduces intra-class variation and increases the separation between the unlabeled foreground and background. This can be achieved by encouraging consistency between the outputs of two prototype classifiers and a linear classifier. Extensive experimental results on the AbdomenCT-1K dataset and a union of four benchmark datasets (including LiTS, MSD-Spleen, KiTS, and NIH82) demonstrate that our method outperforms the state-of-the-art partially-supervised methods by a considerable margin, and even surpasses the fully-supervised methods. The source code is publicly available at LTUDA.

Uncertainty Global Contrastive Learning Framework for Semi-Supervised Medical Image Segmentation

In semi-supervised medical image segmentation, the issue of fuzzy boundaries for segmented objects arises. With limited labeled data and the interaction of boundaries from different segmented objects, classifying segmentation boundaries becomes challenging. To mitigate this issue, we propose an uncertainty global contrastive learning (UGCL) framework. Specifically, we propose a patch filtering method and a classification entropy filtering method to provide reliable pseudo-labels for unlabelled data, while separating fuzzy boundaries and high-entropy pixel points as unreliable points. Considering that unreliable regions contain rich complementary information, we introduce an uncertainty global contrast learning method to distinguish these challenging unreliable regions, enhancing intra-class compactness and inter-class separability at the global data level. Within our optimization framework, we also integrate consistency regularization techniques and select unreliable points as targets for consistency. As demonstrated, the contrastive learning and consistency regularization applied to uncertain points enable us to glean valuable semantic information from unreliable data, which enhances segmentation accuracy. We evaluate our method on two publicly available medical image datasets and compare it with other state-of-the-art semi-supervised medical image segmentation methods, and a series of experimental results show that our method has achieved substantial improvements.

Generalizability, robustness, and correction bias of segmentations of thoracic organs at risk in CT images

OBJECTIVE

This study aims to assess and compare two state-of-the-art deep learning approaches for segmenting four thoracic organs at risk (OAR)-the esophagus, trachea, heart, and aorta-in CT images in the context of radiotherapy planning.

MATERIALS AND METHODS

We compare a multi-organ segmentation approach and the fusion of multiple single-organ models, each dedicated to one OAR. All were trained using nnU-Net with the default parameters and the full-resolution configuration. We evaluate their robustness with adversarial perturbations, and their generalizability on external datasets, and explore potential biases introduced by expert corrections compared to fully manual delineations.

RESULTS

The two approaches show excellent performance with an average Dice score of 0.928 for the multi-class setting and 0.930 when fusing the four single-organ models. The evaluation of external datasets and common procedural adversarial noise demonstrates the good generalizability of these models. In addition, expert corrections of both models show significant bias to the original automated segmentation. The average Dice score between the two corrections is 0.93, ranging from 0.88 for the trachea to 0.98 for the heart.

CONCLUSION

Both approaches demonstrate excellent performance and generalizability in segmenting four thoracic OARs, potentially improving efficiency in radiotherapy planning. However, the multi-organ setting proves advantageous for its efficiency, requiring less training time and fewer resources, making it a preferable choice for this task. Moreover, corrections of AI segmentation by clinicians may lead to biases in the results of AI approaches. A test set, manually annotated, should be used to assess the performance of such methods.

KEY POINTS

Question While manual delineation of thoracic organs at risk is labor-intensive, prone to errors, and time-consuming, evaluation of AI models performing this task lacks robustness. Findings The deep-learning model using the nnU-Net framework showed excellent performance, generalizability, and robustness in segmenting thoracic organs in CT, enhancing radiotherapy planning efficiency. Clinical relevance Automatic segmentation of thoracic organs at risk can save clinicians time without compromising the quality of the delineations, and extensive evaluation across diverse settings demonstrates the potential of integrating such models into clinical practice.

High-throughput mesoscopic optical imaging data processing and parsing using differential-guided filtered neural networks

High-throughput mesoscopic optical imaging technology has tremendously boosted the efficiency of procuring massive mesoscopic datasets from mouse brains. Constrained by the imaging field of view, the image strips obtained by such technologies typically require further processing, such as cross-sectional stitching, artifact removal, and signal area cropping, to meet the requirements of subsequent analyse. However, obtaining a batch of raw array mouse brain data at a resolution of 0.65 × 0.65 × 3 μ m 3 can reach 220TB, and the cropping of the outer contour areas in the disjointed processing still relies on manual visual observation, which consumes substantial computational resources and labor costs. In this paper, we design an efficient deep differential guided filtering module (DDGF) by fusing multi-scale iterative differential guided filtering with deep learning, which effectively refines image details while mitigating background noise. Subsequently, by amalgamating DDGF with deep learning network, we propose a lightweight deep differential guided filtering segmentation network (DDGF-SegNet), which demonstrates robust performance on our dataset, achieving Dice of 0.92, Precision of 0.98, Recall of 0.91, and Jaccard index of 0.86. Building on the segmentation, we utilize connectivity analysis for ascertaining three-dimensional spatial orientation of each brain within the array. Furthermore, we streamline the entire processing workflow by developing an automated pipeline optimized for cluster-based message passing interface(MPI) parallel computation, which reduces the processing time for a mouse brain dataset to a mere 1.1 h, enhancing manual efficiency by 25 times and overall data processing efficiency by 2.4 times, paving the way for enhancing the efficiency of big data processing and parsing for high-throughput mesoscopic optical imaging techniques.

Medical Image Segmentation Review: The Success of U-Net

Automatic medical image segmentation is a crucial topic in the medical domain and successively a critical counterpart in the computer-aided diagnosis paradigm. U-Net is the most widespread image segmentation architecture due to its flexibility, optimized modular design, and success in all medical image modalities. Over the years, the U-Net model has received tremendous attention from academic and industrial researchers who have extended it to address the scale and complexity created by medical tasks. These extensions are commonly related to enhancing the U-Net's backbone, bottleneck, or skip connections, or including representation learning, or combining it with a Transformer architecture, or even addressing probabilistic prediction of the segmentation map. Having a compendium of different previously proposed U-Net variants makes it easier for machine learning researchers to identify relevant research questions and understand the challenges of the biological tasks that challenge the model. In this work, we discuss the practical aspects of the U-Net model and organize each variant model into a taxonomy. Moreover, to measure the performance of these strategies in a clinical application, we propose fair evaluations of some unique and famous designs on well-known datasets. Furthermore, we provide a comprehensive implementation library with trained models. In addition, for ease of future studies, we created an online list of U-Net papers with their possible official implementation.

Symmetry of the left and right tibial plafond; a comparison of 75 distal tibia pairs

PURPOSE

Tibia plafond or pilon fractures present a high level of complexity, making their surgical management challenging. Three-Dimensional Virtual Planning (3DVP) can assist in preoperative planning to achieve optimal fracture reduction. This study aimed to assess the symmetry of the left and right tibial plafond and whether left-right mirroring can reliably be used.

METHODS

Bilateral CT scans of the lower limbs of 75 patients without ankle problems or prior fractures of the lower limb were included. The CT images were segmented to create 3D surface models of the tibia. Subsequently, the left tibial models were mirrored and superimposed onto the right tibia models using a Coherent Point Drift surface matching algorithm. The tibias were then cut to create bone models of the distal tibia with a height of 30 mm, and correspondence points were established. The Euclidean distance was calculated between correspondence points and visualized in a boxplot and heatmaps. The articulating surface was selected as a region of interest.

RESULTS

The median left-right difference was 0.57 mm (IQR, 0.38 - 0.85 mm) of the entire tibial plafond and 0.53 mm (IQR, 0.37 - 0.76 mm) of the articulating surface. The area with the greatest left-right differences were the medial malleoli and the anterior tubercle of the tibial plafond.

CONCLUSION

The tibial plafond exhibits a high degree of bilateral symmetry. Therefore, the mirrored unfractured tibial plafond may be used as a template to optimize preoperative surgical reduction using 3DVP techniques in patients with pilon fractures.