Automatic Multi-Organ Segmentation on Abdominal CT With Dense V-Networks

Localization and Classification of Adrenal Masses in Multiphase Computed Tomography: Retrospective Study

BACKGROUND

The incidence of adrenal incidentalomas is increasing annually, and most types of adrenal masses require surgical intervention. Accurate classification of common adrenal masses based on tumor computed tomography (CT) images by radiologists or clinicians requires extensive experience and is often challenging, which increases the workload of radiologists and leads to unnecessary adrenal surgeries. There is an urgent need for a fully automated, noninvasive, and precise approach for the identification and accurate classification of common adrenal masses.

OBJECTIVE

This study aims to enhance diagnostic efficiency and transform the current clinical practice of preoperative diagnosis of adrenal masses.

METHODS

This study is a retrospective analysis that includes patients with adrenal masses who underwent adrenalectomy from January 1, 2021, to May 31, 2023, at Center 1 (internal dataset), and from January 1, 2016, to May 31, 2023, at Center 2 (external dataset). The images include unenhanced, arterial, and venous phases, with 21,649 images used for the training set, 2406 images used for the validation set, and 12,857 images used for the external test set. We invited 3 experienced radiologists to precisely annotate the images, and these annotations served as references. We developed a deep learning-based adrenal mass detection model, Multi-Attention YOLO (MA-YOLO), which can automatically localize and classify 6 common types of adrenal masses. In order to scientifically evaluate the model performance, we used a variety of evaluation metrics, in addition, we compared the improvement in diagnostic efficacy of 6 doctors after incorporating model assistance.

RESULTS

A total of 516 patients were included. In the external test set, the MA-YOLO model achieved an intersection over union of 0.838, 0.885, and 0.890 for the localization of 6 types of adrenal masses in unenhanced, arterial, and venous phase CT images, respectively. The corresponding mean average precision for classification was 0.885, 0.913, and 0.915, respectively. Additionally, with the assistance of this model, the classification diagnostic performance of 6 radiologists and clinicians for adrenal masses improved. Except for adrenal cysts, at least 1 physician significantly improved diagnostic performance for the other 5 types of tumors. Notably, in the categories of adrenal adenoma (for senior clinician: P=.04, junior radiologist: P=.01, and senior radiologist: P=.01) and adrenal cortical carcinoma (junior clinician: P=.02, junior radiologist: P=.01, and intermediate radiologist: P=.001), half of the physicians showed significant improvements after using the model for assistance.

CONCLUSIONS

The MA-YOLO model demonstrates the ability to achieve efficient, accurate, and noninvasive preoperative localization and classification of common adrenal masses in CT examinations, showing promising potential for future applications.

TRUSTED: The Paired 3D Transabdominal Ultrasound and CT Human Data for Kidney Segmentation and Registration Research

Inter-modal image registration (IMIR) and image segmentation with abdominal Ultrasound (US) data have many important clinical applications, including image-guided surgery, automatic organ measurement, and robotic navigation. However, research is severely limited by the lack of public datasets. We propose TRUSTED (the Tridimensional Renal Ultra Sound TomodEnsitometrie Dataset), comprising paired transabdominal 3DUS and CT kidney images from 48 human patients (96 kidneys), including segmentation, and anatomical landmark annotations by two experienced radiographers. Inter-rater segmentation agreement was over 93% (Dice score), and gold-standard segmentations were generated using the STAPLE algorithm. Seven anatomical landmarks were annotated, for IMIR systems development and evaluation. To validate the dataset's utility, 4 competitive Deep-Learning models for kidney segmentation were benchmarked, yielding average DICE scores from 79.63% to 90.09% for CT, and 70.51% to 80.70% for US images. Four IMIR methods were benchmarked, and Coherent Point Drift performed best with an average Target Registration Error of 4.47 mm and Dice score of 84.10%. The TRUSTED dataset may be used freely to develop and validate segmentation and IMIR methods.

A novel hybrid layer-based encoder-decoder framework for 3D segmentation in congenital heart disease

The segmentation of cardiac anatomy represents a crucial stage in accurate diagnosis and subsequent treatment planning for patients with congenital heart disease (CHD). However, the current deep learning-based segmentation networks are ineffective when applied to 3D medical images of CHD because of the limited availability of training datasets and the inherent complexity exhibited by the variability of cardiac and large vessel tissues. To address this challenge, we propose a novel hybrid layer-based encoder-decoder framework for 3D CHD image segmentation. The model incorporates a global volume mixing module and a local volume-based multihead attention module, which uses a self-attention mechanism to explicitly capture the local and global dependencies of the 3D image segmentation process. This enables the model to more effectively learn the shape boundary features of organs, thereby facilitating accurate segmentation of the whole heart (WH) and great vessels. We compare our method with several popular networks on the public ImageCHD and HVSMR-2.0 datasets. The experimental results show that the proposed model achieves excellent performance in WH and great vessel segmentation tasks with high Dice coefficients and IoU indices.

PMFSNet: Polarized multi-scale feature self-attention network for lightweight medical image segmentation

BACKGROUND AND OBJECTIVES

Current state-of-the-art medical image segmentation methods prioritize precision but often at the expense of increased computational demands and larger model sizes. Applying these large-scale models to the relatively limited scale of medical image datasets tends to induce redundant computation, complicating the process without the necessary benefits. These approaches increase complexity and pose challenges for integrating and deploying lightweight models on edge devices. For instance, recent transformer-based models have excelled in 2D and 3D medical image segmentation due to their extensive receptive fields and high parameter count. However, their effectiveness comes with the risk of overfitting when applied to small datasets. It often neglects the vital inductive biases of Convolutional Neural Networks (CNNs), essential for local feature representation.

METHODS

In this work, we propose PMFSNet, a novel medical imaging segmentation model that effectively balances global and local feature processing while avoiding the computational redundancy typical of larger models. PMFSNet streamlines the UNet-based hierarchical structure and simplifies the self-attention mechanism's computational complexity, making it suitable for lightweight applications. It incorporates a plug-and-play PMFS block, a multi-scale feature enhancement module based on attention mechanisms, to capture long-term dependencies.

RESULTS

The extensive comprehensive results demonstrate that our method achieves superior performance in various segmentation tasks on different data scales even with fewer than a million parameters. Results reveal that our PMFSNet achieves IoU of 84.68%, 82.02%, 78.82%, and 76.48% on public datasets of 3D CBCT Tooth, ovarian tumors ultrasound (MMOTU), skin lesions dermoscopy (ISIC 2018), and gastrointestinal polyp (Kvasir SEG), and yields DSC of 78.29%, 77.45%, and 78.04% on three retinal vessel segmentation datasets, DRIVE, STARE, and CHASE-DB1, respectively.

CONCLUSION

Our proposed model exhibits competitive performance across various datasets, accomplishing this with significantly fewer model parameters and inference time, demonstrating its value in model integration and deployment. It strikes an optimal compromise between efficiency and performance and can be a highly efficient solution for medical image analysis in resource-constrained clinical environments. The source code is available at https://github.com/yykzjh/PMFSNet.

A-Eval: A benchmark for cross-dataset and cross-modality evaluation of abdominal multi-organ segmentation

Although deep learning has revolutionized abdominal multi-organ segmentation, its models often struggle with generalization due to training on small-scale, specific datasets and modalities. The recent emergence of large-scale datasets may mitigate this issue, but some important questions remain unsolved: Can models trained on these large datasets generalize well across different datasets and imaging modalities? If yes/no, how can we further improve their generalizability? To address these questions, we introduce A-Eval, a benchmark for the cross-dataset and cross-modality Evaluation ('Eval') of Abdominal ('A') multi-organ segmentation, integrating seven datasets across CT and MRI modalities. Our evaluations indicate that significant domain gaps persist despite larger data scales. While increased datasets improve generalization, model performance on unseen data remains inconsistent. Joint training across multiple datasets and modalities enhances generalization, though annotation inconsistencies pose challenges. These findings highlight the need for diverse and well-curated training data across various clinical scenarios and modalities to develop robust medical imaging models. The code and pre-trained models are available at https://github.com/uni-medical/A-Eval.

Automatic Brain Segmentation for PET/MR Dual-Modal Images Through a Cross-Fusion Mechanism

The precise segmentation of different brain regions and tissues is usually a prerequisite for the detection and diagnosis of various neurological disorders in neuroscience. Considering the abundance of functional and structural dual-modality information for positron emission tomography/magnetic resonance (PET/MR) images, we propose a novel 3D whole-brain segmentation network with a cross-fusion mechanism introduced to obtain 45 brain regions. Specifically, the network processes PET and MR images simultaneously, employing UX-Net and a cross-fusion block for feature extraction and fusion in the encoder. We test our method by comparing it with other deep learning-based methods, including 3DUXNET, SwinUNETR, UNETR, nnFormer, UNet3D, NestedUNet, ResUNet, and VNet. The experimental results demonstrate that the proposed method achieves better segmentation performance in terms of both visual and quantitative evaluation metrics and achieves more precise segmentation in three views while preserving fine details. In particular, the proposed method achieves superior quantitative results, with a Dice coefficient of 85.73% 0.01%, a Jaccard index of 76.68% 0.02%, a sensitivity of 85.00% 0.01%, a precision of 83.26% 0.03% and a Hausdorff distance (HD) of 4.4885 14.85%. Moreover, the distribution and correlation of the SUV in the volume of interest (VOI) are also evaluated (PCC > 0.9), indicating consistency with the ground truth and the superiority of the proposed method. In future work, we will utilize our whole-brain segmentation method in clinical practice to assist doctors in accurately diagnosing and treating brain diseases.

Systematic Review: AI Applications in Liver Imaging with a Focus on Segmentation and Detection

The liver is a frequent focus in radiology due to its diverse pathology, and artificial intelligence (AI) could improve diagnosis and management. This systematic review aimed to assess and categorize research studies on AI applications in liver radiology from 2018 to 2024, classifying them according to areas of interest (AOIs), AI task and imaging modality used. We excluded reviews and non-liver and non-radiology studies. Using the PRISMA guidelines, we identified 6680 articles from the PubMed/Medline, Scopus and Web of Science databases; 1232 were found to be eligible. A further analysis of a subgroup of 329 studies focused on detection and/or segmentation tasks was performed. Liver lesions were the main AOI and CT was the most popular modality, while classification was the predominant AI task. Most detection and/or segmentation studies (48.02%) used only public datasets, and 27.65% used only one public dataset. Code sharing was practiced by 10.94% of these articles. This review highlights the predominance of classification tasks, especially applied to liver lesion imaging, most often using CT imaging. Detection and/or segmentation tasks relied mostly on public datasets, while external testing and code sharing were lacking. Future research should explore multi-task models and improve dataset availability to enhance AI's clinical impact in liver imaging.

Augmenting motion artifacts to enhance auto-contouring of complex structures in cone-beam computed tomography imaging

Objective. To develop an augmentation method that simulates cone-beam computed tomography (CBCT) related motion artifacts, which can be used to generate training-data to increase the performance of artificial intelligence models dedicated to auto-contouring tasks.Approach.The augmentation technique generates data that simulates artifacts typically present in CBCT imaging. The simulated pseudo-CBCT (pCBCT) is created using interleaved sequences of simulated breath-hold and free-breathing projections. Neural networks for auto-contouring of head and neck and bowel structures were trained with and without pCBCT data. Quantitative and qualitative assessment was done in two independent test sets containing CT and real CBCT data focus on four anatomical regions: head, neck, abdomen, and pelvis. Qualitative analyses were conducted by five clinical experts from three different healthcare institutions.Main results.The generated pCBCT images demonstrate realistic motion artifacts comparable to those observed in real CBCT data. Training a neural network with CT and pCBCT data improved Dice similarity coefficient (DSC) and average contour distance (ACD) results on CBCT test sets. The results were statistically significant (p-value ⩽.03) for bone-mandible (model without/with pCBCT: 0.91/0.92 DSC,p⩽ .01; 0.74/0.66 mm ACD,p⩽.01), brain (0.34/0.93 DSC,p⩽ 1 × 10-5; 17.5/2.79 mm ACD,p= 1 × 10-5), oral-cavity (0.81/0.83 DSC,p⩽.01; 5.11/4.61 mm ACD,p= .02), left-submandibular-gland (0.58/0.77 DSC,p⩽.001; 3.24/2.12 mm ACD,p⩽ .001), right-submandibular-gland (0.00/0.75 DSC,p⩽.1 × 10-5; 17.5/2.26 mm ACD,p⩽ 1 × 10-5), left-parotid (0.68/0.78 DSC,p⩽ .001; 3.34/2.58 mm ACD,p⩽.01), large-bowel (0.60/0.75 DSC,p⩽ .01; 6.14/4.56 mm ACD,p= .03) and small-bowel (3.08/2.65 mm ACD,p= .03). Visual evaluation showed fewer false positives, false negatives, and misclassifications in artifact-affected areas. Qualitative analyses demonstrated that, auto-generated contours are clinically acceptable in over 90% of cases for most structures, with only a few requiring adjustments.Significance.The inclusion of pCBCT improves the performance of trainable auto-contouring approaches, particularly in cases where the images are prone to severe artifacts.

Efficient Generative-Adversarial U-Net for Multi-Organ Medical Image Segmentation

Manual labeling of lesions in medical image analysis presents a significant challenge due to its labor-intensive and inefficient nature, which ultimately strains essential medical resources and impedes the advancement of computer-aided diagnosis. This paper introduces a novel medical image-segmentation framework named Efficient Generative-Adversarial U-Net (EGAUNet), designed to facilitate rapid and accurate multi-organ labeling. To enhance the model's capability to comprehend spatial information, we propose the Global Spatial-Channel Attention Mechanism (GSCA). This mechanism enables the model to concentrate more effectively on regions of interest. Additionally, we have integrated Efficient Mapping Convolutional Blocks (EMCB) into the feature-learning process, allowing for the extraction of multi-scale spatial information and the adjustment of feature map channels through optimized weight values. Moreover, the proposed framework progressively enhances its performance by utilizing a generative-adversarial learning strategy, which contributes to improvements in segmentation accuracy. Consequently, EGAUNet demonstrates exemplary segmentation performance on public multi-organ datasets while maintaining high efficiency. For instance, in evaluations on the CHAOS T2SPIR dataset, EGAUNet achieves approximately 2% higher performance on the Jaccard metric, 1% higher on the Dice metric, and nearly 3% higher on the precision metric in comparison to advanced networks such as Swin-Unet and TransUnet.

A multi-modal multi-branch framework for retinal vessel segmentation using ultra-widefield fundus photographs

Background

Vessel segmentation in fundus photography has become a cornerstone technique for disease analysis. Within this field, Ultra-WideField (UWF) fundus images offer distinct advantages, including an expansive imaging range, detailed lesion data, and minimal adverse effects. However, the high resolution and low contrast inherent to UWF fundus images present significant challenges for accurate segmentation using deep learning methods, thereby complicating disease analysis in this context.

Methods

To address these issues, this study introduces M3B-Net, a novel multi-modal, multi-branch framework that leverages fundus fluorescence angiography (FFA) images to improve retinal vessel segmentation in UWF fundus images. Specifically, M3B-Net tackles the low segmentation accuracy caused by the inherently low contrast of UWF fundus images. Additionally, we propose an enhanced UWF-based segmentation network in M3B-Net, specifically designed to improve the segmentation of fine retinal vessels. The segmentation network includes the Selective Fusion Module (SFM), which enhances feature extraction within the segmentation network by integrating features generated during the FFA imaging process. To further address the challenges of high-resolution UWF fundus images, we introduce a Local Perception Fusion Module (LPFM) to mitigate context loss during the segmentation cut-patch process. Complementing this, the Attention-Guided Upsampling Module (AUM) enhances segmentation performance through convolution operations guided by attention mechanisms.

Results

Extensive experimental evaluations demonstrate that our approach significantly outperforms existing state-of-the-art methods for UWF fundus image segmentation.

Automated Deep Learning-Based Detection and Segmentation of Lung Tumors at CT

"Just Accepted" papers have undergone full peer review and have been accepted for publication in Radiology: Artificial Intelligence. This article will undergo copyediting, layout, and proof review before it is published in its final version. Please note that during production of the final copyedited article, errors may be discovered which could affect the content. Background Detection and segmentation of lung tumors on CT scans are critical for monitoring cancer progression, evaluating treatment responses, and planning radiation therapy; however, manual delineation is labor-intensive and subject to physician variability. Purpose To develop and evaluate an ensemble deep learning model for automating identification and segmentation of lung tumors on CT scans. Materials and Methods A retrospective study was conducted between July 2019 and November 2024 using a large dataset of CT simulation scans and clinical lung tumor segmentations from radiotherapy plans. This dataset was used to train a 3D U-Net-based, image-multiresolution ensemble model to detect and segment lung tumors on CT scans. Model performance was evaluated on internal and external test sets composed of CT simulation scans and lung tumor segmentations from two affiliated medical centers, including single primary and metastatic lung tumors. Performance metrics included sensitivity, specificity, false positive rate, and Dice similarity coefficient (DSC). Model-predicted tumor volumes were compared with physician-delineated volumes. Group comparisons were made with Wilcoxon signed-rank test or one-way ANOVA. P < 0.05 indicated statistical significance. Results The model, trained on 1,504 CT scans with clinical lung tumor segmentations, achieved 92% sensitivity (92/100) and 82% specificity (41/50) in detecting lung tumors on the combined 150-CT scan test set. For a subset of 100 CT scans with a single lung tumor each, the model achieved a median model-physician DSC of 0.77 (IQR: 0.65-0.83) and an interphysician DSC of 0.80 (IQR: 0.72-0.86). Segmentation time was shorter for the model than for physicians (mean 76.6 vs. 166.1-187.7 seconds; p<0.001). Conclusion Routinely collected radiotherapy data were useful for model training. The key strengths of the model include a 3D U-Net ensemble approach for balancing volumetric context with resolution, robust tumor detection and segmentation performance, and the ability to generalize to an external site.

Large-scale multi-center CT and MRI segmentation of pancreas with deep learning

Automated volumetric segmentation of the pancreas on cross-sectional imaging is needed for diagnosis and follow-up of pancreatic diseases. While CT-based pancreatic segmentation is more established, MRI-based segmentation methods are understudied, largely due to a lack of publicly available datasets, benchmarking research efforts, and domain-specific deep learning methods. In this retrospective study, we collected a large dataset (767 scans from 499 participants) of T1-weighted (T1 W) and T2-weighted (T2 W) abdominal MRI series from five centers between March 2004 and November 2022. We also collected CT scans of 1,350 patients from publicly available sources for benchmarking purposes. We introduced a new pancreas segmentation method, called PanSegNet, combining the strengths of nnUNet and a Transformer network with a new linear attention module enabling volumetric computation. We tested PanSegNet's accuracy in cross-modality (a total of 2,117 scans) and cross-center settings with Dice and Hausdorff distance (HD95) evaluation metrics. We used Cohen's kappa statistics for intra and inter-rater agreement evaluation and paired t-tests for volume and Dice comparisons, respectively. For segmentation accuracy, we achieved Dice coefficients of 88.3% (±7.2%, at case level) with CT, 85.0% (±7.9%) with T1 W MRI, and 86.3% (±6.4%) with T2 W MRI. There was a high correlation for pancreas volume prediction with R2 of 0.91, 0.84, and 0.85 for CT, T1 W, and T2 W, respectively. We found moderate inter-observer (0.624 and 0.638 for T1 W and T2 W MRI, respectively) and high intra-observer agreement scores. All MRI data is made available at https://osf.io/kysnj/. Our source code is available at https://github.com/NUBagciLab/PaNSegNet.

Autodelineation of Treatment Target Volume for Radiation Therapy Using Large Language Model-Aided Multimodal Learning

PURPOSE

Artificial intelligence-aided methods have made significant progress in the auto-delineation of normal tissues. However, these approaches struggle with the auto-contouring of radiation therapy target volume. Our goal was to model the delineation of target volume as a clinical decision-making problem, resolved by leveraging large language model-aided multimodal learning approaches.

METHODS AND MATERIALS

A vision-language model, termed Medformer, has been developed, employing the hierarchical vision transformer as its backbone and incorporating large language models to extract text-rich features. The contextually embedded linguistic features are seamlessly integrated into visual features for language-aware visual encoding through the visual language attention module. Metrics, including Dice similarity coefficient (DSC), intersection over union (IOU), and 95th percentile Hausdorff distance (HD95), were used to quantitatively evaluate the performance of our model. The evaluation was conducted on an in-house prostate cancer data set and a public oropharyngeal carcinoma data set, totaling 668 subjects.

RESULTS

Our Medformer achieved a DSC of 0.81 ± 0.10 versus 0.72 ± 0.10, IOU of 0.73 ± 0.12 versus 0.65 ± 0.09, and HD95 of 9.86 ± 9.77 mm versus 19.13 ± 12.96 mm for delineation of gross tumor volume on the prostate cancer dataset. Similarly, on the oropharyngeal carcinoma dataset, it achieved a DSC of 0.77 ± 0.11 versus 0.72 ± 0.09, IOU of 0.70 ± 0.09 versus 0.65 ± 0.07, and HD95 of 7.52 ± 4.8 mm versus 13.63 ± 7.13 mm, representing significant improvements (P < 0.05). For delineating the clinical target volume, Medformer achieved a DSC of 0.91 ± 0.04, IOU of 0.85 ± 0.05, and HD95 of 2.98 ± 1.60 mm, comparable with other state-of-the-art algorithms.

CONCLUSIONS

Auto-delineation of the treatment target based on multimodal learning outperforms conventional approaches that rely purely on visual features. Our method could be adopted into routine practice to rapidly contour clinical target volume/gross tumor volume.

Labeled-to-unlabeled distribution alignment for partially-supervised multi-organ medical image segmentation

Partially-supervised multi-organ medical image segmentation aims to develop a unified semantic segmentation model by utilizing multiple partially-labeled datasets, with each dataset providing labels for a single class of organs. However, the limited availability of labeled foreground organs and the absence of supervision to distinguish unlabeled foreground organs from the background pose a significant challenge, which leads to a distribution mismatch between labeled and unlabeled pixels. Although existing pseudo-labeling methods can be employed to learn from both labeled and unlabeled pixels, they are prone to performance degradation in this task, as they rely on the assumption that labeled and unlabeled pixels have the same distribution. In this paper, to address the problem of distribution mismatch, we propose a labeled-to-unlabeled distribution alignment (LTUDA) framework that aligns feature distributions and enhances discriminative capability. Specifically, we introduce a cross-set data augmentation strategy, which performs region-level mixing between labeled and unlabeled organs to reduce distribution discrepancy and enrich the training set. Besides, we propose a prototype-based distribution alignment method that implicitly reduces intra-class variation and increases the separation between the unlabeled foreground and background. This can be achieved by encouraging consistency between the outputs of two prototype classifiers and a linear classifier. Extensive experimental results on the AbdomenCT-1K dataset and a union of four benchmark datasets (including LiTS, MSD-Spleen, KiTS, and NIH82) demonstrate that our method outperforms the state-of-the-art partially-supervised methods by a considerable margin, and even surpasses the fully-supervised methods. The source code is publicly available at LTUDA.

Robust Automated Mouse Micro-CT Segmentation Using Swin UNEt TRansformers

Image-guided mouse irradiation is essential to understand interventions involving radiation prior to human studies. Our objective is to employ Swin UNEt TRansformers (Swin UNETR) to segment native micro-CT and contrast-enhanced micro-CT scans and benchmark the results against 3D no-new-Net (nnU-Net). Swin UNETR reformulates mouse organ segmentation as a sequence-to-sequence prediction task using a hierarchical Swin Transformer encoder to extract features at five resolution levels, and it connects to a Fully Convolutional Neural Network (FCNN)-based decoder via skip connections. The models were trained and evaluated on open datasets, with data separation based on individual mice. Further evaluation on an external mouse dataset acquired on a different micro-CT with lower kVp and higher imaging noise was also employed to assess model robustness and generalizability. The results indicate that Swin UNETR consistently outperforms nnU-Net and AIMOS in terms of the average dice similarity coefficient (DSC) and the Hausdorff distance (HD95p), except in two mice for intestine contouring. This superior performance is especially evident in the external dataset, confirming the model's robustness to variations in imaging conditions, including noise and quality, and thereby positioning Swin UNETR as a highly generalizable and efficient tool for automated contouring in pre-clinical workflows.

DLLabelsCT: Annotation tool using deep transfer learning to assist in creating new datasets from abdominal computed tomography scans, case study: Pancreas

The utilization of artificial intelligence (AI) is expanding significantly within medical research and, to some extent, in clinical practice. Deep learning (DL) applications, which use large convolutional neural networks (CNN), hold considerable potential, especially in optimizing radiological evaluations. However, training DL algorithms to clinical standards requires extensive datasets, and their processing is labor-intensive. In this study, we developed an annotation tool named DLLabelsCT that utilizes CNN models to accelerate the image analysis process. To validate DLLabelsCT, we trained a CNN model with a ResNet34 encoder and a UNet decoder to segment the pancreas on an open-access dataset and used the DL model to assist in annotating a local dataset, which was further used to refine the model. DLLabelsCT was also tested on two external testing datasets. The tool accelerates annotation by 3.4 times compared to a completely manual annotation method. Out of 3,715 CT scan slices in the testing datasets, 50% did not require editing when reviewing the segmentations made by the ResNet34-UNet model, and the mean and standard deviation of the Dice similarity coefficient was 0.82±0.24. DLLabelsCT is highly accurate and significantly saves time and resources. Furthermore, it can be easily modified to support other deep learning models for other organs, making it an efficient tool for future research involving larger datasets.

Deep Learning Image Segmentation Based on Adaptive Total Variation Preprocessing

This article proposes a two-stage image segmentation method based on the MS model, aiming to enhance the segmentation accuracy of images with complex structure and background. In the first stage, in order to obtain the smooth approximate solution of the image by minimizing the energy functional, an anisotropic regularization term formed by the combination of the gradient operator and an adaptive weighted matrix is introduced. Different weights in both horizontal and vertical directions can be provided by the adaptive weighting matrix according to the gradient information, so that the curve diffuses along the directions of local feature tangents of the objects. In addition, information irrelevant to the image target can be filtered out by the adaptive weighting matrix, thus reducing the interference of complex background. The alternating direction method of multipliers (ADMMs) is employed to solve the convex optimization problem in the first stage. In the second stage, the smoothed image obtained in the first stage is segmented by the deep learning method. By comparing with some traditional methods and deep learning methods, the results demonstrate that not only has good perceptual quality been achieved by this segmentation method, but also superior evaluation metrics have been obtained.

DDANet: A deep dilated attention network for intracerebral haemorrhage segmentation

Intracranial haemorrhage (ICH) is an urgent and potentially fatal medical condition caused by brain blood vessel rupture, leading to blood accumulation in the brain tissue. Due to the pressure and damage it causes to brain tissue, ICH results in severe neurological impairment or even death. Recently, deep neural networks have been widely applied to enhance the speed and precision of ICH detection yet they are still challenged by small or subtle hemorrhages. The authors introduce DDANet, a novel haematoma segmentation model for brain CT images. Specifically, a dilated convolution pooling block is introduced in the intermediate layers of the encoder to enhance feature extraction capabilities of middle layers. Additionally, the authors incorporate a self-attention mechanism to capture global semantic information of high-level features to guide the extraction and processing of low-level features, thereby enhancing the model's understanding of the overall structure while maintaining details. DDANet also integrates residual networks, channel attention, and spatial attention mechanisms for joint optimisation, effectively mitigating the severe class imbalance problem and aiding the training process. Experiments show that DDANet outperforms current methods, achieving the Dice coefficient, Jaccard index, sensitivity, accuracy, and a specificity of 0.712, 0.601, 0.73, 0.997, and 0.998, respectively. The code is available at https://github.com/hpguo1982/DDANet.

Semi-supervised abdominal multi-organ segmentation by object-redrawing

BACKGROUND

Multi-organ segmentation is a critical task in medical imaging, with wide-ranging applications in both clinical practice and research. Accurate delineation of organs from high-resolution 3D medical images, such as CT scans, is essential for radiation therapy planning, enhancing treatment outcomes, and minimizing radiation toxicity risks. Additionally, it plays a pivotal role in quantitative image analysis, supporting various medical research studies. Despite its significance, manual segmentation of multiple organs from 3D images is labor-intensive and prone to low reproducibility due to high interoperator variability. Recent advancements in deep learning have led to several automated segmentation methods, yet many rely heavily on labeled data and human anatomy expertise.

PURPOSE

In this study, our primary objective is to address the limitations of existing semi-supervised learning (SSL) methods for abdominal multi-organ segmentation. We aim to introduce a novel SSL approach that leverages unlabeled data to enhance the performance of deep neural networks in segmenting abdominal organs. Specifically, we propose a method that incorporates a redrawing network into the segmentation process to correct errors and improve accuracy.

METHODS

Our proposed method comprises three interconnected neural networks: a segmentation network for image segmentation, a teacher network for consistency regularization, and a redrawing network for object redrawing. During training, the segmentation network undergoes two rounds of optimization: basic training and readjustment. We adopt the Mean-Teacher model as our baseline SSL approach, utilizing labeled and unlabeled data. However, recognizing significant errors in abdominal multi-organ segmentation using this method alone, we introduce the redrawing network to generate redrawn images based on CT scans, preserving original anatomical information. Our approach is grounded in the generative process hypothesis, encompassing segmentation, drawing, and assembling stages. Correct segmentation is crucial for generating accurate images. In the basic training phase, the segmentation network is trained using both labeled and unlabeled data, incorporating consistency learning to ensure consistent predictions before and after perturbations. The readjustment phase focuses on reducing segmentation errors by optimizing the segmentation network parameters based on the differences between redrawn and original CT images.

RESULTS

We evaluated our method using two publicly available datasets: the beyond the cranial vault (BTCV) segmentation dataset (training: 44, validation: 6) and the abdominal multi-organ segmentation (AMOS) challenge 2022 dataset (training:138, validation:16). Our results were compared with state-of-the-art SSL methods, including MT and dual-task consistency (DTC), using the Dice similarity coefficient (DSC) as an accuracy metric. On both datasets, our proposed SSL method consistently outperformed other methods, including supervised learning, achieving superior segmentation performance for various abdominal organs. These findings demonstrate the effectiveness of our approach, even with a limited number of labeled data.

CONCLUSIONS

Our novel semi-supervised learning approach for abdominal multi-organ segmentation addresses the challenges associated with this task. By integrating a redrawing network and leveraging unlabeled data, we achieve remarkable improvements in accuracy. Our method demonstrates superior performance compared to existing SSL and supervised learning methods. This approach holds great promise in enhancing the precision and efficiency of multi-organ segmentation in medical imaging applications.

A flexible 2.5D medical image segmentation approach with in-slice and cross-slice attention

Deep learning has become the de facto method for medical image segmentation, with 3D segmentation models excelling in capturing complex 3D structures and 2D models offering high computational efficiency. However, segmenting 2.5D images, characterized by high in-plane resolution but lower through-plane resolution, presents significant challenges. While applying 2D models to individual slices of a 2.5D image is feasible, it fails to capture the spatial relationships between slices. On the other hand, 3D models face challenges such as resolution inconsistencies in 2.5D images, along with computational complexity and susceptibility to overfitting when trained with limited data. In this context, 2.5D models, which capture inter-slice correlations using only 2D neural networks, emerge as a promising solution due to their reduced computational demand and simplicity in implementation. In this paper, we introduce CSA-Net, a flexible 2.5D segmentation model capable of processing 2.5D images with an arbitrary number of slices. CSA-Net features an innovative Cross-Slice Attention (CSA) module that effectively captures 3D spatial information by learning long-range dependencies between the center slice (for segmentation) and its neighboring slices. Moreover, CSA-Net utilizes the self-attention mechanism to learn correlations among pixels within the center slice. We evaluated CSA-Net on three 2.5D segmentation tasks: (1) multi-class brain MR image segmentation, (2) binary prostate MR image segmentation, and (3) multi-class prostate MR image segmentation. CSA-Net outperformed leading 2D, 2.5D, and 3D segmentation methods across all three tasks, achieving average Dice coefficients and HD95 values of 0.897 and 1.40 mm for the brain dataset, 0.921 and 1.06 mm for the prostate dataset, and 0.659 and 2.70 mm for the ProstateX dataset, demonstrating its efficacy and superiority. Our code is publicly available at: https://github.com/mirthAI/CSA-Net.

Reinforcement learning-based anatomical maps for pancreas subregion and duct segmentation

BACKGROUND

The pancreas is a complex abdominal organ with many anatomical variations, and therefore automated pancreas segmentation from medical images is a challenging application.

PURPOSE

In this paper, we present a framework for segmenting individual pancreatic subregions and the pancreatic duct from three-dimensional (3D) computed tomography (CT) images.

METHODS

A multiagent reinforcement learning (RL) network was used to detect landmarks of the head, neck, body, and tail of the pancreas, and landmarks along the pancreatic duct in a selected target CT image. Using the landmark detection results, an atlas of pancreases was nonrigidly registered to the target image, resulting in anatomical probability maps for the pancreatic subregions and duct. The probability maps were augmented with multilabel 3D U-Net architectures to obtain the final segmentation results.

RESULTS

To evaluate the performance of our proposed framework, we computed the Dice similarity coefficient (DSC) between the predicted and ground truth manual segmentations on a database of 82 CT images with manually segmented pancreatic subregions and 37 CT images with manually segmented pancreatic ducts. For the four pancreatic subregions, the mean DSC improved from 0.38, 0.44, and 0.39 with standard 3D U-Net, Attention U-Net, and shifted windowing (Swin) U-Net architectures, to 0.51, 0.47, and 0.49, respectively, when utilizing the proposed RL-based framework. For the pancreatic duct, the RL-based framework achieved a mean DSC of 0.70, significantly outperforming the standard approaches and existing methods on different datasets.

CONCLUSIONS

The resulting accuracy of the proposed RL-based segmentation framework demonstrates an improvement against segmentation with standard U-Net architectures.

Discrepancy-based diffusion models for lesion detection in brain MRI

Diffusion probabilistic models (DPMs) have exhibited significant effectiveness in computer vision tasks, particularly in image generation. However, their notable performance heavily relies on labelled datasets, which limits their application in medical images due to the associated high-cost annotations. Current DPM-related methods for lesion detection in medical imaging, which can be categorized into two distinct approaches, primarily rely on image-level annotations. The first approach, based on anomaly detection, involves learning reference healthy brain representations and identifying anomalies based on the difference in inference results. In contrast, the second approach, resembling a segmentation task, employs only the original brain multi-modalities as prior information for generating pixel-level annotations. In this paper, our proposed model - discrepancy distribution medical diffusion (DDMD) - for lesion detection in brain MRI introduces a novel framework by incorporating distinctive discrepancy features, deviating from the conventional direct reliance on image-level annotations or the original brain modalities. In our method, the inconsistency in image-level annotations is translated into distribution discrepancies among heterogeneous samples while preserving information within homogeneous samples. This property retains pixel-wise uncertainty and facilitates an implicit ensemble of segmentation, ultimately enhancing the overall detection performance. Thorough experiments conducted on the BRATS2020 benchmark dataset containing multimodal MRI scans for brain tumour detection demonstrate the great performance of our approach in comparison to state-of-the-art methods.

Kidney medicine meets computer vision: a bibliometric analysis

BACKGROUND AND OBJECTIVE

Rapid advances in computer vision (CV) have the potential to facilitate the examination, diagnosis, and treatment of diseases of the kidney. The bibliometric study aims to explore the research landscape and evolving research focus of the application of CV in kidney medicine research.

METHODS

The Web of Science Core Collection was utilized to identify publications related to the research or applications of CV technology in the field of kidney medicine from January 1, 1900, to December 31, 2022. We analyzed emerging research trends, highly influential publications and journals, prolific researchers, countries/regions, research institutions, co-authorship networks, and co-occurrence networks. Bibliographic information was analyzed and visualized using Python, Matplotlib, Seaborn, HistCite, and Vosviewer.

RESULTS

There was an increasing trend in the number of publications on CV-based kidney medicine research. These publications mainly focused on medical image processing, surgical procedures, medical image analysis/diagnosis, as well as the application and innovation of CV technology in medical imaging. The United States is currently the leading country in terms of the quantities of published articles and international collaborations, followed by China. Deep learning-based segmentation and machine learning-based texture analysis are the most commonly used techniques in this field. Regarding research hotspot trends, CV algorithms are shifting toward artificial intelligence, and research objects are expanding to encompass a wider range of kidney-related objects, with data dimensions used in research transitioning from 2D to 3D while simultaneously incorporating more diverse data modalities.

CONCLUSION

The present study provides a scientometric overview of the current progress in the research and application of CV technology in kidney medicine research. Through the use of bibliometric analysis and network visualization, we elucidate emerging trends, key sources, leading institutions, and popular topics. Our findings and analysis are expected to provide valuable insights for future research on the use of CV in kidney medicine research.

Main challenges on the curation of large scale datasets for pancreas segmentation using deep learning in multi-phase CT scans: Focus on cardinality, manual refinement, and annotation quality

Accurate segmentation of the pancreas in computed tomography (CT) holds paramount importance in diagnostics, surgical planning, and interventions. Recent studies have proposed supervised deep-learning models for segmentation, but their efficacy relies on the quality and quantity of the training data. Most of such works employed small-scale public datasets, without proving the efficacy of generalization to external datasets. This study explored the optimization of pancreas segmentation accuracy by pinpointing the ideal dataset size, understanding resource implications, examining manual refinement impact, and assessing the influence of anatomical subregions. We present the AIMS-1300 dataset encompassing 1,300 CT scans. Its manual annotation by medical experts required 938 h. A 2.5D UNet was implemented to assess the impact of training sample size on segmentation accuracy by partitioning the original AIMS-1300 dataset into 11 smaller subsets of progressively increasing numerosity. The findings revealed that training sets exceeding 440 CTs did not lead to better segmentation performance. In contrast, nnU-Net and UNet with Attention Gate reached a plateau for 585 CTs. Tests on generalization on the publicly available AMOS-CT dataset confirmed this outcome. As the size of the partition of the AIMS-1300 training set increases, the number of error slices decreases, reaching a minimum with 730 and 440 CTs, for AIMS-1300 and AMOS-CT datasets, respectively. Segmentation metrics on the AIMS-1300 and AMOS-CT datasets improved more on the head than the body and tail of the pancreas as the dataset size increased. By carefully considering the task and the characteristics of the available data, researchers can develop deep learning models without sacrificing performance even with limited data. This could accelerate developing and deploying artificial intelligence tools for pancreas surgery and other surgical data science applications.

Artificial intelligence techniques in liver cancer

Hepatocellular Carcinoma (HCC), the most common primary liver cancer, is a significant contributor to worldwide cancer-related deaths. Various medical imaging techniques, including computed tomography, magnetic resonance imaging, and ultrasound, play a crucial role in accurately evaluating HCC and formulating effective treatment plans. Artificial Intelligence (AI) technologies have demonstrated potential in supporting physicians by providing more accurate and consistent medical diagnoses. Recent advancements have led to the development of AI-based multi-modal prediction systems. These systems integrate medical imaging with other modalities, such as electronic health record reports and clinical parameters, to enhance the accuracy of predicting biological characteristics and prognosis, including those associated with HCC. These multi-modal prediction systems pave the way for predicting the response to transarterial chemoembolization and microvascular invasion treatments and can assist clinicians in identifying the optimal patients with HCC who could benefit from interventional therapy. This paper provides an overview of the latest AI-based medical imaging models developed for diagnosing and predicting HCC. It also explores the challenges and potential future directions related to the clinical application of AI techniques.

Cross-Modal Consistency for Single-Modal MR Image Segmentation

OBJECTIVE

Multi-modal magnetic resonance (MR) image segmentation is an important task in disease diagnosis and treatment, but it is usually difficult to obtain multiple modalities for a single patient in clinical applications. To address these issues, a cross-modal consistency framework is proposed for a single-modal MR image segmentation.

METHODS

To enable single-modal MR image segmentation in the inference stage, a weighted cross-entropy loss and a pixel-level feature consistency loss are proposed to train the target network with the guidance of the teacher network and the auxiliary network. To fuse dual-modal MR images in the training stage, the cross-modal consistency is measured according to Dice similarity entropy loss and Dice similarity contrastive loss, so as to maximize the prediction similarity of the teacher network and the auxiliary network. To reduce the difference in image contrast between different MR images for the same organs, a contrast alignment network is proposed to align input images with different contrasts to reference images with a good contrast.

RESULTS

Comprehensive experiments have been performed on a publicly available prostate dataset and an in-house pancreas dataset to verify the effectiveness of the proposed method. Compared to state-of-the-art methods, the proposed method can achieve better segmentation.

CONCLUSION

The proposed image segmentation method can fuse dual-modal MR images in the training stage and only need one-modal MR images in the inference stage.

SIGNIFICANCE

The proposed method can be used in routine clinical occasions when only single-modal MR image with variable contrast is available for a patient.

UNETR++: Delving Into Efficient and Accurate 3D Medical Image Segmentation

Owing to the success of transformer models, recent works study their applicability in 3D medical segmentation tasks. Within the transformer models, the self-attention mechanism is one of the main building blocks that strives to capture long-range dependencies, compared to the local convolutional-based design. However, the self-attention operation has quadratic complexity which proves to be a computational bottleneck, especially in volumetric medical imaging, where the inputs are 3D with numerous slices. In this paper, we propose a 3D medical image segmentation approach, named UNETR++, that offers both high-quality segmentation masks as well as efficiency in terms of parameters, compute cost, and inference speed. The core of our design is the introduction of a novel efficient paired attention (EPA) block that efficiently learns spatial and channel-wise discriminative features using a pair of inter-dependent branches based on spatial and channel attention. Our spatial attention formulation is efficient and has linear complexity with respect to the input. To enable communication between spatial and channel-focused branches, we share the weights of query and key mapping functions that provide a complimentary benefit (paired attention), while also reducing the complexity. Our extensive evaluations on five benchmarks, Synapse, BTCV, ACDC, BraTS, and Decathlon-Lung, reveal the effectiveness of our contributions in terms of both efficiency and accuracy. On Synapse, our UNETR++ sets a new state-of-the-art with a Dice Score of 87.2%, while significantly reducing parameters and FLOPs by over 71%, compared to the best method in the literature. Our code and models are available at: https://tinyurl.com/2p87x5xn.

XIOSIS: An X-Ray-Based Intra-Operative Image-Guided Platform for Oncology Smart Material Delivery

Image-guided interventional oncology procedures can greatly enhance the outcome of cancer treatment. As an enhancing procedure, oncology smart material delivery can increase cancer therapy's quality, effectiveness, and safety. However, the effectiveness of enhancing procedures highly depends on the accuracy of smart material placement procedures. Inaccurate placement of smart materials can lead to adverse side effects and health hazards. Image guidance can considerably improve the safety and robustness of smart material delivery. In this study, we developed a novel generative deep-learning platform that highly prioritizes clinical practicality and provides the most informative intra-operative feedback for image-guided smart material delivery. XIOSIS generates a patient-specific 3D volumetric computed tomography (CT) from three intraoperative radiographs (X-ray images) acquired by a mobile C-arm during the operation. As the first of its kind, XIOSIS (i) synthesizes the CT from small field-of-view radiographs;(ii) reconstructs the intra-operative spacer distribution; (iii) is robust; and (iv) is equipped with a novel soft-contrast cost function. To demonstrate the effectiveness of XIOSIS in providing intra-operative image guidance, we applied XIOSIS to the duodenal hydrogel spacer placement procedure. We evaluated XIOSIS performance in an image-guided virtual spacer placement and actual spacer placement in two cadaver specimens. XIOSIS showed a clinically acceptable performance, reconstructed the 3D intra-operative hydrogel spacer distribution with an average structural similarity of 0.88 and Dice coefficient of 0.63 and with less than 1 cm difference in spacer location relative to the spinal cord.

A Review of Advancements and Challenges in Liver Segmentation

Liver segmentation technologies play vital roles in clinical diagnosis, disease monitoring, and surgical planning due to the complex anatomical structure and physiological functions of the liver. This paper provides a comprehensive review of the developments, challenges, and future directions in liver segmentation technology. We systematically analyzed high-quality research published between 2014 and 2024, focusing on liver segmentation methods, public datasets, and evaluation metrics. This review highlights the transition from manual to semi-automatic and fully automatic segmentation methods, describes the capabilities and limitations of available technologies, and provides future outlooks.

Automatic detection of cognitive impairment in patients with white matter hyperintensity and causal analysis of related factors using artificial intelligence of MRI

PURPOSE

White matter hyperintensity (WMH) is a common feature of brain aging, often linked with cognitive decline and dementia. This study aimed to employ deep learning and radiomics to develop models for detecting cognitive impairment in WMH patients and to analyze the causal relationships among cognitive impairment and related factors.

MATERIALS AND METHODS

A total of 79 WMH patients from hospital 1 were randomly divided into a training set (62 patients) and a testing set (17 patients). Additionally, 29 patients from hospital 2 were included as an independent testing set. All participants underwent formal neuropsychological assessments to determine cognitive status. Automated identification and segmentation of WMH were conducted using VB-net, with extraction of radiomics features from cortex, white matter, and nuclei. Four machine learning classifiers were trained on the training set and validated on the testing set to detect cognitive impairment. Model performances were evaluated and compared. Causal analyses were conducted among cortex, white matter, nuclei alterations, and cognitive impairment.

RESULTS

Among the models, the logistic regression (LR) model based on white matter features demonstrated the highest performance, achieving an AUC of 0.819 in the external test dataset. Causal analyses indicated that age, education level, alterations in cortex, white matter, and nuclei were causal factors of cognitive impairment.

CONCLUSION

The LR model based on white matter features exhibited high accuracy in detecting cognitive impairment in WMH patients. Furthermore, the possible causal relationships among alterations in cortex, white matter, nuclei, and cognitive impairment were elucidated.

Dynamic Loss Weighting for Multiorgan Segmentation in Medical Images

Deep neural networks often suffer from performance inconsistency for multiorgan segmentation in medical images; some organs are segmented far worse than others. The main reason might be organs with different levels of learning difficulty for segmentation mapping, due to variations such as size, texture complexity, shape irregularity, and imaging quality. In this article, we propose a principled class-reweighting algorithm, termed dynamic loss weighting, which dynamically assigns a larger loss weight to organs if they are discriminated as more difficult to learn according to the data and network's status, for forcing the network to learn from them more to maximally promote the performance consistency. This new algorithm uses an extra autoencoder to measure the discrepancy between the segmentation network's output and the ground truth and dynamically estimates the loss weight of organs per the contribution of the organ to the new updated discrepancy. It can capture the variation in organs' learning difficult during training, and it is neither sensitive to data's property nor dependent on human priors. We evaluate this algorithm in two multiorgan segmentation tasks: abdominal organs and head-neck structures, on publicly available datasets, with positive results obtained from extensive experiments which confirm the validity and effectiveness. Source codes are available at: https://github.com/YouyiSong/Dynamic-Loss-Weighting.

Radiograph-based rheumatoid arthritis diagnosis via convolutional neural network

OBJECTIVES

Rheumatoid arthritis (RA) is a severe and common autoimmune disease. Conventional diagnostic methods are often subjective, error-prone, and repetitive works. There is an urgent need for a method to detect RA accurately. Therefore, this study aims to develop an automatic diagnostic system based on deep learning for recognizing and staging RA from radiographs to assist physicians in diagnosing RA quickly and accurately.

METHODS

We develop a CNN-based fully automated RA diagnostic model, exploring five popular CNN architectures on two clinical applications. The model is trained on a radiograph dataset containing 240 hand radiographs, of which 39 are normal and 201 are RA with five stages. For evaluation, we use 104 hand radiographs, of which 13 are normal and 91 RA with five stages.

RESULTS

The CNN model achieves good performance in RA diagnosis based on hand radiographs. For the RA recognition, all models achieve an AUC above 90% with a sensitivity over 98%. In particular, the AUC of the GoogLeNet-based model is 97.80%, and the sensitivity is 100.0%. For the RA staging, all models achieve over 77% AUC with a sensitivity over 80%. Specifically, the VGG16-based model achieves 83.36% AUC with 92.67% sensitivity.

CONCLUSION

The presented GoogLeNet-based model and VGG16-based model have the best AUC and sensitivity for RA recognition and staging, respectively. The experimental results demonstrate the feasibility and applicability of CNN in radiograph-based RA diagnosis. Therefore, this model has important clinical significance, especially for resource-limited areas and inexperienced physicians.

Artificial Intelligence in Pancreatic Image Analysis: A Review

Pancreatic cancer is a highly lethal disease with a poor prognosis. Its early diagnosis and accurate treatment mainly rely on medical imaging, so accurate medical image analysis is especially vital for pancreatic cancer patients. However, medical image analysis of pancreatic cancer is facing challenges due to ambiguous symptoms, high misdiagnosis rates, and significant financial costs. Artificial intelligence (AI) offers a promising solution by relieving medical personnel's workload, improving clinical decision-making, and reducing patient costs. This study focuses on AI applications such as segmentation, classification, object detection, and prognosis prediction across five types of medical imaging: CT, MRI, EUS, PET, and pathological images, as well as integrating these imaging modalities to boost diagnostic accuracy and treatment efficiency. In addition, this study discusses current hot topics and future directions aimed at overcoming the challenges in AI-enabled automated pancreatic cancer diagnosis algorithms.

Uncertainty quantification via localized gradients for deep learning-based medical image assessments

Objective.Deep learning models that aid in medical image assessment tasks must be both accurate and reliable to be deployed within clinical settings. While deep learning models have been shown to be highly accurate across a variety of tasks, measures that indicate the reliability of these models are less established. Increasingly, uncertainty quantification (UQ) methods are being introduced to inform users on the reliability of model outputs. However, most existing methods cannot be augmented to previously validated models because they are not post hoc, and they change a model's output. In this work, we overcome these limitations by introducing a novel post hoc UQ method, termedLocal Gradients UQ, and demonstrate its utility for deep learning-based metastatic disease delineation.Approach.This method leverages a trained model's localized gradient space to assess sensitivities to trained model parameters. We compared the Local Gradients UQ method to non-gradient measures defined using model probability outputs. The performance of each uncertainty measure was assessed in four clinically relevant experiments: (1) response to artificially degraded image quality, (2) comparison between matched high- and low-quality clinical images, (3) false positive (FP) filtering, and (4) correspondence with physician-rated disease likelihood.Main results.(1) Response to artificially degraded image quality was enhanced by the Local Gradients UQ method, where the median percent difference between matching lesions in non-degraded and most degraded images was consistently higher for the Local Gradients uncertainty measure than the non-gradient uncertainty measures (e.g. 62.35% vs. 2.16% for additive Gaussian noise). (2) The Local Gradients UQ measure responded better to high- and low-quality clinical images (p< 0.05 vsp> 0.1 for both non-gradient uncertainty measures). (3) FP filtering performance was enhanced by the Local Gradients UQ method when compared to the non-gradient methods, increasing the area under the receiver operating characteristic curve (ROC AUC) by 20.1% and decreasing the false positive rate by 26%. (4) The Local Gradients UQ method also showed more favorable correspondence with physician-rated likelihood for malignant lesions by increasing ROC AUC for correspondence with physician-rated disease likelihood by 16.2%.Significance. In summary, this work introduces and validates a novel gradient-based UQ method for deep learning-based medical image assessments to enhance user trust when using deployed clinical models.

Active learning using adaptable task-based prioritisation

Supervised machine learning-based medical image computing applications necessitate expert label curation, while unlabelled image data might be relatively abundant. Active learning methods aim to prioritise a subset of available image data for expert annotation, for label-efficient model training. We develop a controller neural network that measures priority of images in a sequence of batches, as in batch-mode active learning, for multi-class segmentation tasks. The controller is optimised by rewarding positive task-specific performance gain, within a Markov decision process (MDP) environment that also optimises the task predictor. In this work, the task predictor is a segmentation network. A meta-reinforcement learning algorithm is proposed with multiple MDPs, such that the pre-trained controller can be adapted to a new MDP that contains data from different institutes and/or requires segmentation of different organs or structures within the abdomen. We present experimental results using multiple CT datasets from more than one thousand patients, with segmentation tasks of nine different abdominal organs, to demonstrate the efficacy of the learnt prioritisation controller function and its cross-institute and cross-organ adaptability. We show that the proposed adaptable prioritisation metric yields converging segmentation accuracy for a new kidney segmentation task, unseen in training, using between approximately 40% to 60% of labels otherwise required with other heuristic or random prioritisation metrics. For clinical datasets of limited size, the proposed adaptable prioritisation offers a performance improvement of 22.6% and 10.2% in Dice score, for tasks of kidney and liver vessel segmentation, respectively, compared to random prioritisation and alternative active sampling strategies.

Federated learning with knowledge distillation for multi-organ segmentation with partially labeled datasets

The state-of-the-art multi-organ CT segmentation relies on deep learning models, which only generalize when trained on large samples of carefully curated data. However, it is challenging to train a single model that can segment all organs and types of tumors since most large datasets are partially labeled or are acquired across multiple institutes that may differ in their acquisitions. A possible solution is Federated learning, which is often used to train models on multi-institutional datasets where the data is not shared across sites. However, predictions of federated learning can be unreliable after the model is locally updated at sites due to 'catastrophic forgetting'. Here, we address this issue by using knowledge distillation (KD) so that the local training is regularized with the knowledge of a global model and pre-trained organ-specific segmentation models. We implement the models in a multi-head U-Net architecture that learns a shared embedding space for different organ segmentation, thereby obtaining multi-organ predictions without repeated processes. We evaluate the proposed method using 8 publicly available abdominal CT datasets of 7 different organs. Of those datasets, 889 CTs were used for training, 233 for internal testing, and 30 volumes for external testing. Experimental results verified that our proposed method substantially outperforms other state-of-the-art methods in terms of accuracy, inference time, and the number of parameters.

Towards more precise automatic analysis: a systematic review of deep learning-based multi-organ segmentation

Accurate segmentation of multiple organs in the head, neck, chest, and abdomen from medical images is an essential step in computer-aided diagnosis, surgical navigation, and radiation therapy. In the past few years, with a data-driven feature extraction approach and end-to-end training, automatic deep learning-based multi-organ segmentation methods have far outperformed traditional methods and become a new research topic. This review systematically summarizes the latest research in this field. We searched Google Scholar for papers published from January 1, 2016 to December 31, 2023, using keywords "multi-organ segmentation" and "deep learning", resulting in 327 papers. We followed the PRISMA guidelines for paper selection, and 195 studies were deemed to be within the scope of this review. We summarized the two main aspects involved in multi-organ segmentation: datasets and methods. Regarding datasets, we provided an overview of existing public datasets and conducted an in-depth analysis. Concerning methods, we categorized existing approaches into three major classes: fully supervised, weakly supervised and semi-supervised, based on whether they require complete label information. We summarized the achievements of these methods in terms of segmentation accuracy. In the discussion and conclusion section, we outlined and summarized the current trends in multi-organ segmentation.

Exploring Automated Contouring Across Institutional Boundaries: A Deep Learning Approach with Mouse Micro-CT Datasets

Image-guided mouse irradiation is essential to understand interventions involving radiation prior to human studies. Our objective is to employ Swin UNEt Transformers (Swin UNETR) to segment native micro-CT and contrast-enhanced micro-CT scans and benchmark the results against 3D no-new-Net (nnU-Net). Swin UNETR reformulates mouse organ segmentation as a sequence-to-sequence prediction task, using a hierarchical Swin Transformer encoder to extract features at 5 resolution levels, and connects to a Fully Convolutional Neural Network (FCNN)-based decoder via skip connections. The models were trained and evaluated on open datasets, with data separation based on individual mice. Further evaluation on an external mouse dataset acquired on a different micro-CT with lower kVp and higher imaging noise was also employed to assess model robustness and generalizability. Results indicate that Swin UNETR consistently outperforms nnU-Net and AIMOS in terms of average dice similarity coefficient (DSC) and Hausdorff distance (HD95p), except in two mice of intestine contouring. This superior performance is especially evident in the external dataset, confirming the model's robustness to variations in imaging conditions, including noise and quality, thereby positioning Swin UNETR as a highly generalizable and efficient tool for automated contouring in pre-clinical workflows.

HCA-DAN: hierarchical class-aware domain adaptive network for gastric tumor segmentation in 3D CT images

BACKGROUND

Accurate segmentation of gastric tumors from CT scans provides useful image information for guiding the diagnosis and treatment of gastric cancer. However, automated gastric tumor segmentation from 3D CT images faces several challenges. The large variation of anisotropic spatial resolution limits the ability of 3D convolutional neural networks (CNNs) to learn features from different views. The background texture of gastric tumor is complex, and its size, shape and intensity distribution are highly variable, which makes it more difficult for deep learning methods to capture the boundary. In particular, while multi-center datasets increase sample size and representation ability, they suffer from inter-center heterogeneity.

METHODS

In this study, we propose a new cross-center 3D tumor segmentation method named Hierarchical Class-Aware Domain Adaptive Network (HCA-DAN), which includes a new 3D neural network that efficiently bridges an Anisotropic neural network and a Transformer (AsTr) for extracting multi-scale context features from the CT images with anisotropic resolution, and a hierarchical class-aware domain alignment (HCADA) module for adaptively aligning multi-scale context features across two domains by integrating a class attention map with class-specific information. We evaluate the proposed method on an in-house CT image dataset collected from four medical centers and validate its segmentation performance in both in-center and cross-center test scenarios.

RESULTS

Our baseline segmentation network (i.e., AsTr) achieves best results compared to other 3D segmentation models, with a mean dice similarity coefficient (DSC) of 59.26%, 55.97%, 48.83% and 67.28% in four in-center test tasks, and with a DSC of 56.42%, 55.94%, 46.54% and 60.62% in four cross-center test tasks. In addition, the proposed cross-center segmentation network (i.e., HCA-DAN) obtains excellent results compared to other unsupervised domain adaptation methods, with a DSC of 58.36%, 56.72%, 49.25%, and 62.20% in four cross-center test tasks.

CONCLUSIONS

Comprehensive experimental results demonstrate that the proposed method outperforms compared methods on this multi-center database and is promising for routine clinical workflows.

Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation

Precise delineation of multiple organs or abnormal regions in the human body from medical images plays an essential role in computer-aided diagnosis, surgical simulation, image-guided interventions, and especially in radiotherapy treatment planning. Thus, it is of great significance to explore automatic segmentation approaches, among which deep learning-based approaches have evolved rapidly and witnessed remarkable progress in multi-organ segmentation. However, obtaining an appropriately sized and fine-grained annotated dataset of multiple organs is extremely hard and expensive. Such scarce annotation limits the development of high-performance multi-organ segmentation models but promotes many annotation-efficient learning paradigms. Among these, studies on transfer learning leveraging external datasets, semi-supervised learning including unannotated datasets and partially-supervised learning integrating partially-labeled datasets have led the dominant way to break such dilemmas in multi-organ segmentation. We first review the fully supervised method, then present a comprehensive and systematic elaboration of the 3 abovementioned learning paradigms in the context of multi-organ segmentation from both technical and methodological perspectives, and finally summarize their challenges and future trends.

BucketAugment: Reinforced Domain Generalisation in Abdominal CT Segmentation

Goal: In recent years, deep neural networks have consistently outperformed previously proposed methods in the domain of medical segmentation. However, due to their nature, these networks often struggle to delineate desired structures in data that fall outside their training distribution. The goal of this study is to address the challenges associated with domain generalization in CT segmentation by introducing a novel method called BucketAugment for deep neural networks. Methods: BucketAugment leverages principles from the Q-learning algorithm and employs validation loss to search for an optimal policy within a search space comprised of distributed stacks of 3D volumetric augmentations, termed 'buckets.' These buckets have tunable parameters and can be seamlessly integrated into existing neural network architectures, offering flexibility for customization. Results: In our experiments, we focus on segmenting kidney and liver structures across three distinct medical datasets, each containing CT scans of the abdominal region collected from various clinical institutions and scanner vendors. Our results indicate that BucketAugment significantly enhances domain generalization across diverse medical datasets, requiring only minimal modifications to existing network architectures. Conclusions: The introduction of BucketAugment provides a promising solution to the challenges of domain generalization in CT segmentation. By leveraging Q-learning principles and distributed stacks of 3D augmentations, this method improves the performance of deep neural networks on medical segmentation tasks, demonstrating its potential to enhance the applicability of such models across different datasets and clinical scenarios.

Auto-segmentation of thoraco-abdominal organs in pediatric dynamic MRI

Purpose

Analysis of the abnormal motion of thoraco-abdominal organs in respiratory disorders such as the Thoracic Insufficiency Syndrome (TIS) and scoliosis such as adolescent idiopathic scoliosis (AIS) or early onset scoliosis (EOS) can lead to better surgical plans. We can use healthy subjects to find out the normal architecture and motion of a rib cage and associated organs and attempt to modify the patient's deformed anatomy to match to it. Dynamic magnetic resonance imaging (dMRI) is a practical and preferred imaging modality for capturing dynamic images of healthy pediatric subjects. In this paper, we propose an auto-segmentation set-up for the lungs, kidneys, liver, spleen, and thoraco-abdominal skin in these dMRI images which have their own challenges such as poor contrast, image non-standardness, and similarity in texture amongst gas, bone, and connective tissue at several inter-object interfaces.

Methods

The segmentation set-up has been implemented in two steps: recognition and delineation using two deep neural network (DL) architectures (say DL-R and DL-D) for the recognition step and delineation step, respectively. The encoder-decoder framework in DL-D utilizes features at four different resolution levels to counter the challenges involved in the segmentation. We have evaluated on dMRI sagittal acquisitions of 189 (near-)normal subjects. The spatial resolution in all dMRI acquisitions is 1.46 mm in a sagittal slice and 6.00 mm between sagittal slices. We utilized images of 89 (10) subjects at end inspiration for training (validation). For testing we experimented with three scenarios: utilizing (1) the images of 90 (=189-89-10) different (remaining) subjects at end inspiration for testing, (2) the images of the aforementioned 90 subjects at end expiration for testing, and (3) the images of the aforesaid 99 (=89+10) subjects but at end expiration for testing. In some situations, we can take advantage of already available ground truth (GT) of a subject at a particular respiratory phase to automatically segment the object in the image of the same subject at a different respiratory phase and then refining the segmentation to create the final GT. We anticipate that this process of creating GT would require minimal post hoc correction. In this spirit, we conducted separate experiments where we assume to have the ground truth of the test subjects at end expiration for scenario (1), end inspiration for (2), and end inspiration for (3).

Results

Amongst these three scenarios of testing, for the DL-R, we achieve a best average location error (LE) of about 1 voxel for the lungs, kidneys, and spleen and 1.5 voxels for the liver and the thoraco- abdominal skin. The standard deviation (SD) of LE is about 1 or 2 voxels. For the delineation approach, we achieve an average Dice coefficient (DC) of about 0.92 to 0.94 for the lungs, 0.82 for the kidneys, 0.90 for the liver, 0.81 for the spleen, and 0.93 for the thoraco-abdominal skin. The SD of DC is lower for the lungs, liver, and the thoraco-abdominal skin, and slightly higher for the spleen and kidneys.

Conclusions

Motivated by applications in surgical planning for disorders such as TIS, AIS, and EOS, we have shown an auto-segmentation system for thoraco-abdominal organs in dMRI acquisitions. This proposed setup copes with the challenges posed by low resolution, motion blur, inadequate contrast, and image intensity non-standardness quite well. We are in the process of testing its effectiveness on TIS patient dMRI data.

COSST: Multi-Organ Segmentation With Partially Labeled Datasets Using Comprehensive Supervisions and Self-Training

Deep learning models have demonstrated remarkable success in multi-organ segmentation but typically require large-scale datasets with all organs of interest annotated. However, medical image datasets are often low in sample size and only partially labeled, i.e., only a subset of organs are annotated. Therefore, it is crucial to investigate how to learn a unified model on the available partially labeled datasets to leverage their synergistic potential. In this paper, we systematically investigate the partial-label segmentation problem with theoretical and empirical analyses on the prior techniques. We revisit the problem from a perspective of partial label supervision signals and identify two signals derived from ground truth and one from pseudo labels. We propose a novel two-stage framework termed COSST, which effectively and efficiently integrates comprehensive supervision signals with self-training. Concretely, we first train an initial unified model using two ground truth-based signals and then iteratively incorporate the pseudo label signal to the initial model using self-training. To mitigate performance degradation caused by unreliable pseudo labels, we assess the reliability of pseudo labels via outlier detection in latent space and exclude the most unreliable pseudo labels from each self-training iteration. Extensive experiments are conducted on one public and three private partial-label segmentation tasks over 12 CT datasets. Experimental results show that our proposed COSST achieves significant improvement over the baseline method, i.e., individual networks trained on each partially labeled dataset. Compared to the state-of-the-art partial-label segmentation methods, COSST demonstrates consistent superior performance on various segmentation tasks and with different training data sizes.

Artificial intelligence and machine learning trends in kidney care

BACKGROUND

The integration of artificial intelligence (AI) and machine learning (ML) in kidney care has seen a significant rise in recent years. This study specifically analyzed AI and ML research publications related to kidney care to identify leading authors, institutions, and countries in this area. It aimed to examine publication trends and patterns, and to explore the impact of collaborative efforts on citation metrics.

METHODS

The study used the Science Citation Index Expanded (SCI-EXPANDED) of Clarivate Analytics Web of Science Core Collection to search for AI and machine learning publications related to nephrology from 1992 to 2021. The authors used quotation marks and Boolean operator "or" to search for keywords in the title, abstract, author keywords, and Keywords Plus. In addition, the 'front page' filter was applied. A total of 5425 documents were identified and analyzed.

RESULTS

The results showed that articles represent 75% of the analyzed documents, with an average author to publications ratio of 7.4 and an average number of citations per publication in 2021 of 18. English articles had a higher citation rate than non-English articles. The USA dominated in all publication indicators, followed by China. Notably, the research also showed that collaborative efforts tend to result in higher citation rates. A significant portion of the publications were found in urology journals, emphasizing the broader scope of kidney care beyond traditional nephrology.

CONCLUSIONS

The findings underscore the importance of AI and ML in enhancing kidney care, offering a roadmap for future research and implementation in this expanding field.

A feature-enhanced network for stroke lesion segmentation from brain MRI images

Accurate and expeditious segmentation of stroke lesions can greatly assist physicians in making accurate medical diagnoses and administering timely treatments. However, there are two limitations to the current deep learning methods. On the one hand, the attention structure utilizes only local features, which misleads the subsequent segmentation; on the other hand, simple downsampling compromises task-relevant detailed semantic information. To address these challenges, we propose a novel feature refinement and protection network (FRPNet) for stroke lesion segmentation. FRPNet employs a symmetric encoding-decoding structure and incorporates twin attention gate (TAG) and multi-dimension attention pooling (MAP) modules. The TAG module leverages the self-attention mechanism and bi-directional attention to extract both global and local features of the lesion. On the other hand, the MAP module establishes multidimensional pooling attention to effectively mitigate the loss of features during the encoding process. Extensive comparative experiments show that, our method significantly outperforms the state-of-the-art approaches with 60.16% DSC, 36.20px HD and 85.72% DSC, 27.02px HD on two ischemic stroke datasets that contain all stroke stages and several sequences of stroke images. The excellent results that exceed those of existing methods illustrate the efficacy and generalizability of the proposed method. The source code is released on https://github.com/wu2ze2lin2/FRPNet.

Fully automated deep learning system for osteoporosis screening using chest computed tomography images

Background

Osteoporosis, a disease stemming from bone metabolism irregularities, affects approximately 200 million people worldwide. Timely detection of osteoporosis is pivotal in grappling with this public health challenge. Deep learning (DL), emerging as a promising methodology in the field of medical imaging, holds considerable potential for the assessment of bone mineral density (BMD). This study aimed to propose an automated DL framework for BMD assessment that integrates localization, segmentation, and ternary classification using various dominant convolutional neural networks (CNNs).

Methods

In this retrospective study, a cohort of 2,274 patients underwent chest computed tomography (CT) was enrolled from January 2022 to June 2023 for the development of the integrated DL system. The study unfolded in 2 phases. Initially, 1,025 patients were selected based on specific criteria to develop an automated segmentation model, utilizing 2 VB-Net networks. Subsequently, a distinct cohort of 902 patients was employed for the development and testing of classification models for BMD assessment. Then, 3 distinct DL network architectures, specifically DenseNet, ResNet-18, and ResNet-50, were applied to formulate the 3-classification BMD assessment model. The performance of both phases was evaluated using an independent test set consisting of 347 individuals. Segmentation performance was evaluated using the Dice similarity coefficient; classification performance was appraised using the receiver operating characteristic (ROC) curve. Furthermore, metrics such as the area under the curve (AUC), accuracy, and precision were meticulously calculated.

Results

In the first stage, the automatic segmentation model demonstrated excellent segmentation performance, with mean Dice surpassing 0.93 in the independent test set. In the second stage, both the DenseNet and ResNet-18 demonstrated excellent diagnostic performance in detecting bone status. For osteoporosis, and osteopenia, the AUCs were as follows: DenseNet achieved 0.94 [95% confidence interval (CI): 0.91-0.97], and 0.91 (95% CI: 0.87-0.94), respectively; ResNet-18 attained 0.96 (95% CI: 0.92-0.98), and 0.91 (95% CI: 0.87-0.94), respectively. However, the ResNet-50 model exhibited suboptimal diagnostic performance for osteopenia, with an AUC value of only 0.76 (95% CI: 0.69-0.80). Alterations in tube voltage had a more pronounced impact on the performance of the DenseNet. In the independent test set with tube voltage at 100 kVp images, the accuracy and precision of DenseNet decreased on average by approximately 14.29% and 18.82%, respectively, whereas the accuracy and precision of ResNet-18 decreased by about 8.33% and 7.14%, respectively.

Conclusions

The state-of-the-art DL framework model offers an effective and efficient approach for opportunistic osteoporosis screening using chest CT, without incurring additional costs or radiation exposure.

Age Encoded Adversarial Learning for Pediatric CT Segmentation

Organ segmentation from CT images is critical in the early diagnosis of diseases, progress monitoring, pre-operative planning, radiation therapy planning, and CT dose estimation. However, data limitation remains one of the main challenges in medical image segmentation tasks. This challenge is particularly huge in pediatric CT segmentation due to children's heightened sensitivity to radiation. In order to address this issue, we propose a novel segmentation framework with a built-in auxiliary classifier generative adversarial network (ACGAN) that conditions age, simultaneously generating additional features during training. The proposed conditional feature generation segmentation network (CFG-SegNet) was trained on a single loss function and used 2.5D segmentation batches. Our experiment was performed on a dataset with 359 subjects (180 male and 179 female) aged from 5 days to 16 years and a mean age of 7 years. CFG-SegNet achieved an average segmentation accuracy of 0.681 dice similarity coefficient (DSC) on the prostate, 0.619 DSC on the uterus, 0.912 DSC on the liver, and 0.832 DSC on the heart with four-fold cross-validation. We compared the segmentation accuracy of our proposed method with previously published U-Net results, and our network improved the segmentation accuracy by 2.7%, 2.6%, 2.8%, and 3.4% for the prostate, uterus, liver, and heart, respectively. The results indicate that our high-performing segmentation framework can more precisely segment organs when limited training images are available.

Artificial intelligence in gastrointestinal endoscopy: a comprehensive review

Integrating artificial intelligence (AI) into gastrointestinal (GI) endoscopy heralds a significant leap forward in managing GI disorders. AI-enabled applications, such as computer-aided detection and computer-aided diagnosis, have significantly advanced GI endoscopy, improving early detection, diagnosis and personalized treatment planning. AI algorithms have shown promise in the analysis of endoscopic data, critical in conditions with traditionally low diagnostic sensitivity, such as indeterminate biliary strictures and pancreatic cancer. Convolutional neural networks can markedly improve the diagnostic process when integrated with cholangioscopy or endoscopic ultrasound, especially in the detection of malignant biliary strictures and cholangiocarcinoma. AI's capacity to analyze complex image data and offer real-time feedback can streamline endoscopic procedures, reduce the need for invasive biopsies, and decrease associated adverse events. However, the clinical implementation of AI faces challenges, including data quality issues and the risk of overfitting, underscoring the need for further research and validation. As the technology matures, AI is poised to become an indispensable tool in the gastroenterologist's arsenal, necessitating the integration of robust, validated AI applications into routine clinical practice. Despite remarkable advances, challenges such as operator-dependent accuracy and the need for intricate examinations persist. This review delves into the transformative role of AI in enhancing endoscopic diagnostic accuracy, particularly highlighting its utility in the early detection and personalized treatment of GI diseases.

Multi-organ segmentation of organ-at-risk (OAR's) of head and neck site using ensemble learning technique

INTRODUCTION

This paper presents a novel approach to automate the segmentation of Organ-at-Risk (OAR) in Head and Neck cancer patients using Deep Learning models combined with Ensemble Learning techniques. The study aims to improve the accuracy and efficiency of OAR segmentation, essential for radiotherapy treatment planning.

METHODS

The dataset comprised computed tomography (CT) scans of 182 patients in DICOM format, obtained from an institutional image bank. Experienced Radiation Oncologists manually segmented seven OARs for each scan. Two models, 3D U-Net and 3D DenseNet-FCN, were trained on reduced CT scans (192 × 192 x 128) due to memory limitations. Ensemble Learning techniques were employed to enhance accuracy and segmentation metrics. Testing was conducted on 78 patients from the institutional dataset and an open-source dataset (TCGA-HNSC and Head-Neck Cetuximab) consisting of 31 patient scans.

RESULTS

Using the Ensemble Learning technique, the average dice similarity coefficient for OARs ranged from 0.990 to 0.994, indicating high segmentation accuracy. The 95% Hausdorff distance (mm) ranged from 1.3 to 2.1, demonstrating precise segmentation boundaries.

CONCLUSION

The proposed automated segmentation method achieved efficient and accurate OAR segmentation, surpassing human expert performance in terms of time and accuracy.

IMPLICATIONS FOR PRACTICE

This approach has implications for improving treatment planning and patient care in radiotherapy. By reducing manual segmentation reliance, the proposed method offers significant time savings and potential improvements in treatment planning efficiency and precision for head and neck cancer patients.

Enhancing medical image segmentation with a multi-transformer U-Net

Various segmentation networks based on Swin Transformer have shown promise in medical segmentation tasks. Nonetheless, challenges such as lower accuracy and slower training convergence have persisted. To tackle these issues, we introduce a novel approach that combines the Swin Transformer and Deformable Transformer to enhance overall model performance. We leverage the Swin Transformer's window attention mechanism to capture local feature information and employ the Deformable Transformer to adjust sampling positions dynamically, accelerating model convergence and aligning it more closely with object shapes and sizes. By amalgamating both Transformer modules and incorporating additional skip connections to minimize information loss, our proposed model excels at rapidly and accurately segmenting CT or X-ray lung images. Experimental results demonstrate the remarkable, showcasing the significant prowess of our model. It surpasses the performance of the standalone Swin Transformer's Swin Unet and converges more rapidly under identical conditions, yielding accuracy improvements of 0.7% (resulting in 88.18%) and 2.7% (resulting in 98.01%) on the COVID-19 CT scan lesion segmentation dataset and Chest X-ray Masks and Labels dataset, respectively. This advancement has the potential to aid medical practitioners in early diagnosis and treatment decision-making.