A Review of Deep-Learning-Based Medical Image Segmentation Methods

Automated Cone Photoreceptor Detection in Adaptive Optics Flood Illumination Ophthalmoscopy

Purpose

To develop and validate a deep learning-based model for detecting cone photoreceptor cells in adaptive optics flood illumination ophthalmoscopy (AO-FIO).

Design

Healthy volunteer study.

Participants

A total of 36 healthy participants were included.

Methods

The imaging protocol consisted of 21 AO-FIO images per eye acquired with the rtx1 adaptive optics retinal camera (Imagine Eyes), 4° × 4° each with 2° overlap, imaging a retinal patch 4° nasal (N) to 12° temporal (T) and -5° inferior to 5° superior relative to the fovea. Each image was divided into patches of 128 × 128 pixels, with a 20-pixel overlap. A training set (625 patches) from 18 subjects (32 ± 12 years, 6 males and 12 females) was annotated by a single center, whereas the test set (54 patches) from 18 subjects (40 ± 16 years, 11 males and 7 females) was annotated by graders from 3 different institutions. The deep learning model, based on the U-Net architecture, underwent a parameter search using the tree-structured Parzen estimator.

Main Outcome Measures

The F1 score was used to determine both intragrader and intergrader agreements and to evaluate the model's performance compared with the automated detection by the manufacturer's software (AOdetect Mosaic).

Results

The average intragrader agreement was 0.85 ± 0.06 between 2°N and 2°T, followed by 0.83 ± 0.09 between 3 and 6°T, and 0.80 ± 0.10 between 7 and 10°T. The average intergrader agreement for the 3 centers was 0.84 ± 0.05, 0.79 ± 0.05, and 0.76 ± 0.06 at 2°N-2°T, 3-6°T, and 7-10°T, respectively. The best combination of hyperparameters based on the tree-structured Parzen estimator algorithm achieved an F1 score of 0.89 ± 0.04. The average agreement between the model and the graders was 0.87 ± 0.04, 0.85 ± 0.03, and 0.81 ± 0.03 at 2°N-2°T, 3°-6°T, and 7°-10°T, respectively. These values were higher than those between AOdetect's auto detection without manual correction and the graders (0.84 ± 0.05, 0.79 ± 0.03, and 0.68 ± 0.04, respectively). A reduction in cone density was noted at greater eccentricities, in line with previous research findings, and the model indicated variations in estimating cell density for individuals aged 18 to 30 compared with those aged ≥50 years.

Conclusions

The performance of the developed deep learning-based model, AO-FIO ConeDetect, was comparable to that of graders from 3 medical centers. It outperformed the manufacturers' software auto-detection, particularly at higher eccentricities (7°-10°T). Hence, the model could reduce the need for manual correction and enable faster cone mosaic analyses.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Deep learning-based automated segmentation of cardiac real-time MRI in non-human primates

Advanced imaging techniques, like magnetic resonance imaging (MRI), have revolutionised cardiovascular disease diagnosis and monitoring in humans and animal models. Real-time (RT) MRI, which can capture a single slice during each consecutive heartbeat while the animal or patient breathes continuously, generates large data sets that necessitate automatic myocardium segmentation to fully exploit these technological advancements. While automatic segmentation is common in human adults, it remains underdeveloped in preclinical animal models. In this study, we developed and trained a fully automated 2D convolutional neural network (CNN) for segmenting the left and right ventricles and the myocardium in non-human primates (NHPs) using RT cardiac MR images of rhesus macaques, in the following referred to as PrimUNet. Based on the U-Net framework, PrimUNet achieved optimal performance with a learning rate of 0.0001, an initial kernel size of 64, a final kernel size of 512, and a batch size of 32. It attained an average Dice score of 0.9, comparable to human studies. Testing PrimUNet on additional RT MRI data from rhesus macaques demonstrated strong agreement with manual segmentation for left ventricular end-diastolic volume (LVEDV), left ventricular end-systolic volume (LVESV), and left ventricular myocardial volume (LVMV). It also performs well on cine MRI data of rhesus macaques and acceptably on those of baboons. PrimUNet is well-suited for automatically segmenting extensive RT MRI data, facilitating strain analyses of individual heartbeats. By eliminating human observer variability, PrimUNet enhances the reliability and reproducibility of data analysis in animal research, thereby advancing translational cardiovascular studies.

Multi-scale conv-attention U-Net for medical image segmentation

U-Net-based network structures are widely used in medical image segmentation. However, effectively capturing multi-scale features and spatial context information of complex organizational structures remains a challenge. To address this, we propose a novel network structure based on the U-Net backbone. This model integrates the Adaptive Convolution (AC) module, Multi-Scale Learning (MSL) module, and Conv-Attention module to enhance feature expression ability and segmentation performance. The AC module dynamically adjusts the convolutional kernel through an adaptive convolutional layer. This enables the model to extract features of different shapes and scales adaptively, further improving its performance in complex scenarios. The MSL module is designed for multi-scale information fusion. It effectively aggregates fine-grained and high-level semantic features from different resolutions, creating rich multi-scale connections between the encoding and decoding processes. On the other hand, the Conv-Attention module incorporates an efficient attention mechanism into the skip connections. It captures global context information using a low-dimensional proxy for high-dimensional data. This approach reduces computational complexity while maintaining effective spatial and channel information extraction. Experimental validation on the CVC-ClinicDB, MICCAI 2023 Tooth, and ISIC2017 datasets demonstrates that our proposed MSCA-UNet significantly improves segmentation accuracy and model robustness. At the same time, it remains lightweight and outperforms existing segmentation methods.

Perspectives on lung visualization: Three-dimensional anatomical modeling of computed and micro-computed tomographic data in comparative evolutionary morphology and medicine with applications for COVID-19

The vertebrate respiratory system is challenging to study. The complex relationship between the lungs and adjacent tissues, the vast structural diversity of the respiratory system both within individuals and between taxa, its mobility (or immobility) and distensibility, and the difficulty of quantifying and visualizing functionally important internal negative spaces have all impeded descriptive, functional, and comparative research. As a result, there is a relative paucity of three-dimensional anatomical information on this organ system in all vertebrate groups (including humans) relative to other regions of the body. We present some of the challenges associated with evaluating and visualizing the vertebrate respiratory system using computed and micro-computed tomography and its subsequent digital segmentation. We discuss common mistakes to avoid when imaging deceased and live specimens and various methods for merging manual and threshold-based segmentation approaches to visualize pulmonary tissues across a broad range of vertebrate taxa, with a particular focus on sauropsids (reptiles and birds). We also address some of the recent work in comparative evolutionary morphology and medicine that have used these techniques to visualize respiratory tissues. Finally, we provide a clinical study on COVID-19 in humans in which we apply modeling methods to visualize and quantify pulmonary infection in the lungs of human patients.

Automated Euler number of the alveolar capillary network based on deep learning segmentation with verification by stereological methods

Diseases like bronchopulmonary dysplasia (BPD) affect the development of the pulmonary vasculature, including the alveolar capillary network (ACN). Since pulmonary development is highly dependent on angiogenesis and microvascular maturation, ACN investigations are essential. Therefore, efficient methods are needed for quantitative comparative studies. Here, the suitability of deep learning (DL) for processing serial block-face scanning electron microscopic (SBF-SEM) data by generating ACN segmentations, 3D reconstructions and performing automated quantitative analyses based on them, was tested. Since previous studies revealed inefficient ACN segmentation as the limiting factor in the overall workflow, a 2D DL-based approach was used with existing data, aiming at the reduction of necessary manual interaction. Automated quantitative analyses based on completed segmentations were performed subsequently. The results were compared to stereological estimations, assessing segmentation quality and result reliability. It was shown that the DL-based approach was suitable for generating segmentations on SBF-SEM data. This approach generated more complete initial ACN segmentations than an established method, despite the limited amount of available training data and the use of a 2D rather than a 3D approach. The quality of the completed ACN segmentations was assessed as sufficient. Furthermore, quantitative analyses delivered reliable results about the ACN architecture, automatically obtained contrary to manual stereological approaches. This study demonstrated that ACN segmentation is still the part of the overall workflow that requires improvement regarding the reduction of manual interaction to benefit from available automated software tools. Nevertheless, the results indicated that it could be advantageous taking further efforts to implement a 3D DL-based segmentation approach. As the amount of analysed data was limited, this study was not conducted to obtain representative data about BPD-induced ACN alterations, but to highlight next steps towards a fully automated segmentation and evaluation workflow, enabling larger sample sizes and representative studies.

Self-supervised 3D medical image segmentation by flow-guided mask propagation learning

Despite significant progress in 3D medical image segmentation using deep learning, manual annotation remains a labor-intensive bottleneck. Self-supervised mask propagation (SMP) methods have emerged to alleviate this challenge, allowing intra-volume segmentation with just a single slice annotation. However, the previous SMP methods often rely on 2D information and ignore volumetric contexts. While our previous work, called Vol2Flow, attempts to address this concern, it exhibits limitations, including not focusing enough on local (i.e., slice-pair) information, neglecting global information (i.e., volumetric contexts) in the objective function, and error accumulation during slice-to-slice reconstruction. This paper introduces Flow2Mask, a novel SMP method, developed to overcome the limitations of previous SMP approaches, particularly Vol2Flow. During training, Flow2Mask proposes the Local-to-Global (L2G) loss to learn inter-slice flow fields among all consecutive slices within a volume in an unsupervised manner. This dynamic loss is based on curriculum learning to gradually learn information within a volume from local to global contexts. Additionally, the Inter-Slice Smoothness (ISS) loss is introduced as a regularization term to encourage changes between the slices occur consistently and continuously. During inference, Flow2Mask leverages these 3D flow fields for inter-slice mask propagation in a 3D image, spreading annotation from a single annotated slice to the entire volume. Moreover, we propose an automatic strategy to select the most representative slice as initial annotation in the mask propagation process. Experimental evaluations on different abdominal datasets demonstrate that our proposed SMP method outperforms previous approaches and improves the overall mean DSC of Vol2Flow by +2.1%, +8.2%, and +4.0% for the Sliver, CHAOS, and 3D-IRCAD datasets, respectively. Furthermore, Flow2Mask even exhibits substantial improvements in weakly-supervised and self-supervised few-shot segmentation methods when applied as a mask completion tool. The code and model for Flow2Mask are available at https://github.com/AdelehBitarafan/Flow2Mask, providing a valuable contribution to the field of medical image segmentation.

An improved feature extraction algorithm for robust Swin Transformer model in high-dimensional medical image analysis

The Swin Transformer is recently developed transformer architecture with promising results in various computer vision tasks. Medical image analysis is a complex and critical task that requires high dimensional feature extraction. The significant challenge in medical image analysis is the limited availability of annotated data for training. It has been proposed that a multitask learning scheme be put in place. Swin Transformer can be trained for all the medical image analysis tasks simultaneously so that general features can be learned from the model and used for other new tasks and data. In most cases, the medical images have poor properties such as noise, artifacts, and low contrast. The Swin Transformer presents an adaptive attention mechanism: its attention weights are learned dynamically according to input quality. It could selectively focus on essential regions in an image while discarding noise or irrelevant information. Medical images may have very complex anatomical structures. In this sense, an iterative transformer encoder is proposed to form a hierarchical structure with gradually decreasing dimensionality between layers-so that the attention mechanism is applied at different scales, capturing local and long-range relationships between image patches. This research proposes a robust Swin Transformer architecture for high-dimensional feature extraction in medical images. The proposed algorithm reached 80.76 % accuracy, 80.28 % precision, 78.04 % recall, 76.46 % F1-Score and 73.46 % critical success index.

CCA: Contrastive cluster assignment for supervised and semi-supervised medical image segmentation

Transformers have shown great potential in vision tasks such as semantic segmentation. However, most of the existing transformer-based segmentation models neglect the cross-attention between pixel features and class features which impedes the application of transformers. Inspired by the concept of object queries in k-means Mask Transformer, we develop cluster learning and contrastive cluster assignment (CCA) for medical image segmentation in this paper. The cluster learning leverages the object queries to fit the feature-level cluster centers. The contrastive cluster assignment is introduced to guide the pixel class prediction using the cluster centers. Our method is a plug-in and can be integrated into any model. We design two networks for supervised segmentation tasks and semi-supervised segmentation tasks respectively. We equip the decoder with our proposed modules for the supervised segmentation to improve the pixel-level predictions. For the semi-supervised segmentation, we enhance the feature extraction capability of the encoder by using our proposed modules. We conduct comprehensive comparison and ablation experiments on public medical image datasets (ACDC, LA, Synapse, and ISIC2018), the results demonstrate that our proposed models outperform state-of-the-art models consistently, validating the effectiveness of our proposed method. The source code is accessible at https://github.com/zhujinghua1234/CCA-Seg.

Application of TransUnet Deep Learning Model for Automatic Segmentation of Cervical Cancer in Small-Field T2WI Images

Effective segmentation of cervical cancer tissue from magnetic resonance (MR) images is crucial for automatic detection, staging, and treatment planning of cervical cancer. This study develops an innovative deep learning model to enhance the automatic segmentation of cervical cancer lesions. We obtained 4063 T2WI small-field sagittal, coronal, and oblique axial images from 222 patients with pathologically confirmed cervical cancer. Using this dataset, we employed a convolutional neural network (CNN) along with TransUnet models for segmentation training and evaluation of cervical cancer tissues. In this approach, CNNs are leveraged to extract local information from MR images, whereas Transformers capture long-range dependencies related to shape and structural information, which are critical for precise segmentation. Furthermore, we developed three distinct segmentation models based on coronal, axial, and sagittal T2WI within a small field of view using multidirectional MRI techniques. The dice similarity coefficient (DSC) and mean Hausdorff distance (AHD) were used to assess the performance of the models in terms of segmentation accuracy. The average DSC and AHD values obtained using the TransUnet model were 0.7628 and 0.8687, respectively, surpassing those obtained using the U-Net model by margins of 0.0033 and 0.3479, respectively. The proposed TransUnet segmentation model significantly enhances the accuracy of cervical cancer tissue delineation compared to alternative models, demonstrating superior performance in overall segmentation efficacy. This methodology can improve clinical diagnostic efficiency as an automated image analysis tool tailored for cervical cancer diagnosis.

Metastatic Lung Lesion Changes in Follow-up Chest CT: The Advantage of Deep Learning Simultaneous Analysis of Prior and Current Scans With SimU-Net

PURPOSE

Radiological follow-up of oncology patients requires the detection of metastatic lung lesions and the quantitative analysis of their changes in longitudinal imaging studies. Our aim was to evaluate SimU-Net, a novel deep learning method for the automatic analysis of metastatic lung lesions and their temporal changes in pairs of chest CT scans.

MATERIALS AND METHODS

SimU-Net is a simultaneous multichannel 3D U-Net model trained on pairs of registered prior and current scans of a patient. It is part of a fully automatic pipeline for the detection, segmentation, matching, and classification of metastatic lung lesions in longitudinal chest CT scans. A data set of 5040 metastatic lung lesions in 344 pairs of 208 prior and current chest CT scans from 79 patients was used for training/validation (173 scans, 65 patients) and testing (35 scans, 14 patients) of a standalone 3D U-Net models and 3 simultaneous SimU-Net models. Outcome measures were the lesion detection and segmentation precision, recall, Dice score, average symmetric surface distance (ASSD), lesion matching, and classification of lesion changes from computed versus manual ground-truth annotations by an expert radiologist.

RESULTS

SimU-Net achieved a mean lesion detection recall and precision of 0.93±0.13 and 0.79±0.24 and a mean lesion segmentation Dice and ASSD of 0.84±0.09 and 0.33±0.22 mm. These results outperformed the standalone 3D U-Net model by 9.4% in the recall, 2.4% in Dice, and 15.4% in ASSD, with a minor 3.6% decrease in precision. The SimU-Net pipeline achieved perfect precision and recall (1.0±0.0) for lesion matching and classification of lesion changes.

CONCLUSIONS

Simultaneous deep learning analysis of metastatic lung lesions in prior and current chest CT scans with SimU-Net yields superior accuracy compared with individual analysis of each scan. Implementation of SimU-Net in the radiological workflow may enhance efficiency by automatically computing key metrics used to evaluate metastatic lung lesions and their temporal changes.

Review of Non-Invasive Fetal Electrocardiography Monitoring Techniques

Non-invasive fetal electrocardiography (NIFECG), an emerging technology for fetal health monitoring, has garnered significant attention in recent years. It is considered a promising alternative to traditional Doppler ultrasound methods and has the potential to become the standard approach for fetal monitoring. This paper provides a comprehensive review of the latest advancements in NIFECG technology, including signal acquisition, signal preprocessing, fetal electrocardiogram extraction, and fetal cardiac anomaly classification. Furthermore, the characteristics and limitations of existing NIFECG datasets are analyzed, and improvement suggestions are proposed. Future research directions for NIFECG technology are discussed, with a particular focus on the potential applications of deep learning techniques, multimodal data fusion, and remote monitoring systems. This review offers references and support for advancing the development and application of NIFECG monitoring technology.

Ischemic Stroke Lesion Core Segmentation from CT Perfusion Scans Using Attention ResUnet Deep Learning

Accurate segmentation of ischemic stroke lesions is crucial for refining diagnosis, prognosis, and treatment planning. Manual identification is time-consuming and challenging, especially in urgent clinical scenarios. This paper presents an innovative deep learning-based system for automated segmentation of ischemic stroke lesions from Computed Tomography Perfusion (CTP) datasets. This paper introduces a deep learning-based system designed to segment ischemic stroke lesions from Computed Tomography Perfusion (CTP) datasets. The proposed approach integrates Edge Enhancing Diffusion (EED) filtering as a preprocessing step, acting as a form of hard attention to emphasize affected regions. Besides the Attention ResUnet (AttResUnet) architecture with a modified decoder path, incorporating spatial and channel attention mechanisms to capture long-range dependencies. The system was evaluated using the ISLES challenge 2018 dataset with a fivefold cross-validation approach. The proposed framework achieved a noteworthy average Dice Similarity Coefficient (DSC) score of 59%. This performance underscores the effectiveness of combining EED filtering with attention mechanisms in the AttResUnet architecture for accurate stroke lesion segmentation. The fold-wise analysis revealed consistent performance across different data subsets, with slight variations highlighting the model's generalizability. The proposed approach offers a reliable and generalizable tool for automated ischemic stroke lesion segmentation, potentially improving efficiency and accuracy in clinical settings.

Leveraging paired mammogram views with deep learning for comprehensive breast cancer detection

Employing two standard mammography views is crucial for radiologists, providing comprehensive insights for reliable clinical evaluations. This study introduces paired mammogram view based-network(PMVnet), a novel algorithm designed to enhance breast lesion detection by integrating relational information from paired whole mammograms, addressing the limitations of current methods. Utilizing 1,636 private mammograms, PMVnet combines cosine similarity and the squeeze-and-excitation method within a U-shaped architecture to leverage correlated information. Performance comparisons with single view-based models with VGGnet16, Resnet50, and EfficientnetB5 as encoders revealed PMVnet's superior capability. Using VGGnet16, PMVnet achieved a Dice Similarity Coefficient (DSC) of 0.709 in segmentation and a recall of 0.950 at 0.156 false positives per image (FPPI) in detection tasks, outperforming the single-view model, which had a DSC of 0.579 and a recall of 0.813 at 0.188 FPPI. These findings demonstrate PMVnet's effectiveness in reducing false positives and avoiding missed true positives, suggesting its potential as a practical tool in computer-aided diagnosis systems. PMVnet can significantly enhance breast lesion detection, aiding radiologists in making more precise evaluations and improving patient outcomes. Future applications of PMVnet may offer substantial benefits in clinical settings, improving patient care through enhanced diagnostic accuracy.

Optimizing transformer-based network via advanced decoder design for medical image segmentation

U-Net is widely used in medical image segmentation due to its simple and flexible architecture design. To address the challenges of scale and complexity in medical tasks, several variants of U-Net have been proposed. In particular, methods based on Vision Transformer (ViT), represented by Swin UNETR, have gained widespread attention in recent years. However, these improvements often focus on the encoder, overlooking the crucial role of the decoder in optimizing segmentation details. This design imbalance limits the potential for further enhancing segmentation performance. To address this issue, we analyze the roles of various decoder components, including upsampling method, skip connection, and feature extraction module, as well as the shortcomings of existing methods. Consequently, we propose Swin DER (i.e.,SwinUNETRDecoderEnhanced andRefined), by specifically optimizing the design of these three components. Swin DER performs upsampling using learnable interpolation algorithm called offset coordinate neighborhood weighted up sampling (Onsampling) and replaces traditional skip connection with spatial-channel parallel attention gate (SCP AG). Additionally, Swin DER introduces deformable convolution along with attention mechanism in the feature extraction module of the decoder. Our model design achieves excellent results, surpassing other state-of-the-art methods on both the Synapse dataset and the MSD brain tumor segmentation task. Code is available at:https://github.com/WillBeanYang/Swin-DER.

Multi-scale graph harmonies: Unleashing U-Net's potential for medical image segmentation through contrastive learning

Medical image segmentation is essential for accurately representing tissues and organs in scans, improving diagnosis, guiding treatment, enabling quantitative analysis, and advancing AI-assisted healthcare. Organs and lesion areas in medical images have complex geometries and spatial relationships. Due to variations in the size and location of lesion areas, automatic segmentation faces significant challenges. While Convolutional Neural Networks (CNNs) and Transformers have proven effective in segmentation task, they still possess inherent limitations. Because these models treat images as regular grids or sequences of patches, they struggle to learn the geometric features of an image, which are essential for capturing irregularities and subtle details. In this paper we propose a novel segmentation model, MSGH, which utilizes Graph Neural Network (GNN) to fully exploit geometric representation for guiding image segmentation. In MSGH, we combine multi-scale features from Pyramid Feature and Graph Feature branches to facilitate information exchange across different networks. We also leverage graph contrastive representation learning to extract features through self-supervised learning to mitigate the impact of category imbalance in medical images. Moreover, we optimize the decoder by integrating Transformer to enhance the model's capability in restoring the intricate image details feature. We conducted a comprehensive experimental study on ACDC, Synapse and BraTS datasets to validate the effectiveness and efficiency of MSGH. Our method achieved an improvement of 2.56-13.41%, 1.04-5.11% and 1.77-3.35% of dice on the three segmentation tasks respectively. The results demonstrate that our model consistently performs well compared with state-of-the-art models. The source code is accessible at https://github.com/Dorothywujie/MSGH.

Quantification of training-induced alterations in body composition via automated machine learning analysis of MRI images in the thigh region: A pilot study in young females

The maintenance of an appropriate ratio of body fat to muscle mass is essential for the preservation of health and performance, as excessive body fat is associated with an increased risk of various diseases. Accurate body composition assessment requires precise segmentation of structures. In this study we developed a novel automatic machine learning approach for volumetric segmentation and quantitative assessment of MRI volumes and investigated the efficacy of using a machine learning algorithm to assess muscle, subcutaneous adipose tissue (SAT), and bone volume of the thigh before and after a strength training. Eighteen healthy, young, female volunteers were randomly allocated to two groups: intervention group (IG) and control group (CG). The IG group followed an 8-week strength endurance training plan that was conducted two times per week. Before and after the training, the subjects of both groups underwent MRI scanning. The evaluation of the image data was performed by a machine learning system which is based on a 3D U-Net-based Convolutional Neural Network. The volumes of muscle, bone, and SAT were each examined using a 2 (GROUP [IG vs. CG]) × 2 (TIME [pre-intervention vs. post-intervention]) analysis of variance (ANOVA) with repeated measures for the factor TIME. The results of the ANOVA demonstrate significant TIME × GROUP interaction effects for the muscle volume (F1,16 = 12.80, p = 0.003, ηP 2 = 0.44) with an increase of 2.93% in the IG group and no change in the CG (-0.62%, p = 0.893). There were no significant changes in bone or SAT volume between the groups. This study supports the use of artificial intelligence systems to analyze MRI images as a reliable tool for monitoring training responses on body composition.

Plug-and-play segment anything model improves nnUNet performance

BACKGROUND

The automatic segmentation of medical images has widespread applications in modern clinical workflows. The Segment Anything Model (SAM), a recent development of foundational models in computer vision, has become a universal tool for image segmentation without the need for specific domain training. However, SAM's reliance on prompts necessitates human-computer interaction during the inference process. Its performance on specific domains can also be limited without additional adaptation. In contrast, traditional models like nnUNet are designed to perform segmentation tasks automatically during inference and can work well for each specific domain, but they require extensive training on domain-specific datasets.

PURPOSE

To leverage the advantages of both foundational and domain-specific models and achieve fully automated segmentation with limited training samples, we propose nnSAM, which combines the robust feature extraction capabilities of SAM with the automatic configuration abilities of nnUNet to enhance the accuracy and robustness of medical image segmentation on small datasets.

METHODS

We propose the nnSAM model for small sample medical image segmentation. We made optimizations for this goal via two main approaches: first, we integrated the feature extraction capabilities of SAM with the automatic configuration advantages of nnUNet, which enables robust feature extraction and domain-specific adaptation on small datasets. Second, during the training process, we designed a boundary shape supervision loss based on level set functions and curvature calculations, enabling the model to learn anatomical shape priors from limited annotation data.

RESULTS

We conducted quantitative and qualitative assessments on the performance of our proposed method on four segmentation tasks: brain white matter, liver, lung, and heart segmentation. Our method achieved the best performance across all tasks. Specifically, in brain white matter segmentation using 20 training samples, nnSAM achieved the highest DICE score of 82.77 ( ± $\pm$ 10.12) % and the lowest average surface distance (ASD) of 1.14 ( ± $\pm$ 1.03) mm, compared to nnUNet, which had a DICE score of 79.25 ( ± $\pm$ 17.24) % and an ASD of 1.36 ( ± $\pm$ 1.63) mm. A sample size study shows that the advantage of nnSAM becomes more prominent under fewer training samples.

CONCLUSIONS

A comprehensive evaluation of multiple small-sample segmentation tasks demonstrates significant improvements in segmentation performance by nnSAM, highlighting the vast potential of small-sample learning.

LightAWNet: Lightweight adaptive weighting network based on dynamic convolutions for medical image segmentation

PURPOSE

The complexity of convolutional neural networks (CNNs) can lead to improved segmentation accuracy in medical image analysis but also results in increased network complexity and training challenges, especially under resource limitations. Conversely, lightweight models offer efficiency but often sacrifice accuracy. This paper addresses the challenge of balancing efficiency and accuracy by proposing LightAWNet, a lightweight adaptive weighting neural network for medical image segmentation.

METHODS

We designed LightAWNet with an efficient inverted bottleneck encoder block optimized by spatial attention. A two-branch strategy is employed to separately extract detailed and spatial features for fusion, enhancing the reusability of model feature maps. Additionally, a lightweight optimized up-sampling operation replaces traditional transposed convolution, and channel attention is utilized in the decoder to produce more accurate outputs efficiently.

RESULTS

Experimental results on the LiTS2017, MM-WHS, ISIC2018, and Kvasir-SEG datasets demonstrate that LightAWNet achieves state-of-the-art performance with only 2.83 million parameters. Our model significantly outperforms existing methods in terms of segmentation accuracy, highlighting its effectiveness in maintaining high performance with reduced complexity.

CONCLUSIONS

LightAWNet successfully balances efficiency and accuracy in medical image segmentation. The innovative use of spatial attention, dual-branch feature extraction, and optimized up-sampling operations contribute to its superior performance. These findings offer valuable insights for the development of resource-efficient yet highly accurate segmentation models in medical imaging. The code will be made available at https://github.com/zjmiaprojects/lightawnet upon acceptance for publication.

Understanding the Impact of Evaluation Metrics in Kinetic Models for Consensus-Based Segmentation

In this article, we extend a recently introduced kinetic model for consensus-based segmentation of images. In particular, we will interpret the set of pixels of a 2D image as an interacting particle system that evolves in time in view of a consensus-type process obtained by interactions between pixels and external noise. Thanks to a kinetic formulation of the introduced model, we derive the large time solution of the model. We will show that the parameters defining the segmentation task can be chosen from a plurality of loss functions that characterize the evaluation metrics.

Automatic X-ray teeth segmentation with grouped attention

Detection and teeth segmentation from X-rays, aiding healthcare professionals in accurately determining the shape and growth trends of teeth. However, small dataset sizes due to patient privacy, high noise, and blurred boundaries between periodontal tissue and teeth pose challenges to the models' transportability and generalizability, making them prone to overfitting. To address these issues, we propose a novel model, named Grouped Attention and Cross-Layer Fusion Network (GCNet). GCNet effectively handles numerous noise points and significant individual differences in the data, achieving stable and precise segmentation on small-scale datasets. The model comprises two core modules: Grouped Global Attention (GGA) modules and Cross-Layer Fusion (CLF) modules. The GGA modules capture and group texture and contour features, while the CLF modules combine these features with deep semantic information to improve prediction. Experimental results on the Children's Dental Panoramic Radiographs dataset show that our model outperformed existing models such as GT-U-Net and Teeth U-Net, with a Dice coefficient of 0.9338, sensitivity of 0.9426, and specificity of 0.9821. The GCNet model also demonstrates clearer segmentation boundaries compared to other models.

Advancements in Machine Learning and Artificial Intelligence in the Radiological Detection of Pulmonary Embolism

Pulmonary embolism (PE) is a clinically challenging diagnosis that varies from silent to life-threatening symptoms. Timely diagnosis of the condition is subject to clinical assessment, D-dimer testing and radiological imaging. Computed tomography pulmonary angiogram (CTPA) is considered the gold standard imaging modality, although some cases can be missed due to reader dependency, resulting in adverse patient outcomes. Hence, it is crucial to implement faster and precise diagnostic strategies to help clinicians diagnose and treat PE patients promptly and mitigate morbidity and mortality. Machine learning (ML) and artificial intelligence (AI) are the newly emerging tools in the medical field, including in radiological imaging, potentially improving diagnostic efficacy. Our review of the studies showed that computer-aided design (CAD) and AI tools displayed similar to superior sensitivity and specificity in identifying PE on CTPA as compared to radiologists. Several tools demonstrated the potential in identifying minor PE on radiological scans showing promising ability to aid clinicians in reducing missed cases substantially. However, it is imperative to design sophisticated tools and conduct large clinical trials to integrate AI use in everyday clinical setting and establish guidelines for its ethical applicability. ML and AI can also potentially help physicians in formulating individualized management strategies to enhance patient outcomes.

CADS: A Self-Supervised Learner via Cross-Modal Alignment and Deep Self-Distillation for CT Volume Segmentation

Self-supervised learning (SSL) has long had great success in advancing the field of annotation-efficient learning. However, when applied to CT volume segmentation, most SSL methods suffer from two limitations, including rarely using the information acquired by different imaging modalities and providing supervision only to the bottleneck encoder layer. To address both limitations, we design a pretext task to align the information in each 3D CT volume and the corresponding 2D generated X-ray image and extend self-distillation to deep self-distillation. Thus, we propose a self-supervised learner based on Cross-modal Alignment and Deep Self-distillation (CADS) to improve the encoder's ability to characterize CT volumes. The cross-modal alignment is a more challenging pretext task that forces the encoder to learn better image representation ability. Deep self-distillation provides supervision to not only the bottleneck layer but also shallow layers, thus boosting the abilities of both. Comparative experiments show that, during pre-training, our CADS has lower computational complexity and GPU memory cost than competing SSL methods. Based on the pre-trained encoder, we construct PVT-UNet for 3D CT volume segmentation. Our results on seven downstream tasks indicate that PVT-UNet outperforms state-of-the-art SSL methods like MOCOv3 and DiRA, as well as prevalent medical image segmentation methods like nnUNet and CoTr. Code and pre-trained weight will be available at https://github.com/yeerwen/CADS.

Berendzen K, Wanke F, Gabernet G, Harter K, Nahnsen S. Nf-Root: A Best-Practice Pipeline for Deep-Learning-Based Analysis of Apoplastic pH in Microscopy Images of Developmental Zones in Plant Root Tissue

Hormonal mechanisms associated with cell elongation play a vital role in the development and growth of plants. Here, we report Nextflow-root (nf-root), a novel best-practice pipeline for deep-learning-based analysis of fluorescence microscopy images of plant root tissue from A. thaliana. This bioinformatics pipeline performs automatic identification of developmental zones in root tissue images. This also includes apoplastic pH measurements, which is useful for modeling hormone signaling and cell physiological responses. We show that this nf-core standard-based pipeline successfully automates tissue zone segmentation and is both high-throughput and highly reproducible. In short, a deep-learning module deploys deterministically trained convolutional neural network models and augments the segmentation predictions with measures of prediction uncertainty and model interpretability, while aiming to facilitate result interpretation and verification by experienced plant biologists. We observed a high statistical similarity between the manually generated results and the output of the nf-root.

Automatic Segmentation of Hemorrhages in the Ultra-Wide Field Retina: Multi-Scale Attention Subtraction Networks and an Ultra-Wide Field Retinal Hemorrhage Dataset

Ultra-wide field (UWF) retinal imaging can improve the detection rate of retinal hemorrhage as compared with conventional fundus images. However, hemorrhages in UWF retinal images can also become smaller and more widely distributed, which can be time consuming and labor intensive. With the development of computer technology, automatic segmentation techniques can assist physicians in diagnosis. However, the lack of publicly available UWF retinal hemorrhage segmentation datasets has limited the development of automatic hemorrhage segmentation techniques in UWF retinal images. We present a large-scale high-quality UWF retinal hemorrhage segmentation dataset, named UWF-RHS Dataset, for public use. To the best of our knowledge, we are the first team to make the UWF retinal hemorrhage segmentation dataset publicly available. In addition, we propose a multi-scale attention subtraction network (MASNet) for UWF retinal hemorrhage segmentation. Specifically, highly focused lesion features are extracted by using the proposed multi-scale attention subtraction (MAS) module at the progress of the skip-connection. Several comparative experiments and ablation experiments were performed at the UWF-RHS Dataset, and all experiments state that our proposed method is effective in diagnosing retinal hemorrhages with state-of-the-art results. The proposed UWF-RHS dataset and MASNet will greatly facilitate the development of automated segmentation techniques for UWF retinal hemorrhages.

Breast tumor segmentation via deep correlation analysis of multi-sequence MRI

Precise segmentation of breast tumors from MRI is crucial for breast cancer diagnosis, as it allows for detailed calculation of tumor characteristics such as shape, size, and edges. Current segmentation methodologies face significant challenges in accurately modeling the complex interrelationships inherent in multi-sequence MRI data. This paper presents a hybrid deep network framework with three interconnected modules, aimed at efficiently integrating and exploiting the spatial-temporal features among multiple MRI sequences for breast tumor segmentation. The first module involves an advanced multi-sequence encoder with a densely connected architecture, separating the encoding pathway into multiple streams for individual MRI sequences. To harness the intricate correlations between different sequence features, we propose a sequence-awareness and temporal-awareness method that adeptly fuses spatial-temporal features of MRI in the second multi-scale feature embedding module. Finally, the decoder module engages in the upsampling of feature maps, meticulously refining the resolution to achieve highly precise segmentation of breast tumors. In contrast to other popular methods, the proposed method learns the interrelationships inherent in multi-sequence MRI. We justify the proposed method through extensive experiments. It achieves notable improvements in segmentation performance, with Dice Similarity Coefficient (DSC), Intersection over Union (IoU), and Positive Predictive Value (PPV) scores of 80.57%, 74.08%, and 84.74% respectively.

Refining heart disease prediction accuracy using hybrid machine learning techniques with novel metaheuristic algorithms

Early diagnosis of heart disease is crucial, as it's one of the leading causes of death globally. Machine learning algorithms can be a powerful tool in achieving this goal. Therefore, this article aims to increase the accuracy of predicting heart disease using machine learning algorithms. Five classification models are explored: eXtreme Gradient Boosting (XGBC), Random Forest Classifier (RFC), Decision Tree Classifier (DTC), K-Nearest Neighbors Classifier (KNNC), and Logistic Regression Classifier (LRC). Additionally, four optimizers are evaluated: Slime mold Optimization Algorithm, Forest Optimization Algorithm, Pathfinder algorithm, and Giant Armadillo Optimization. To ensure robust model selection, a feature selection technique utilizing k-fold cross-validation is employed. This method identifies the most relevant features from the data, potentially improving model performance. The top three performing models are then coupled with the optimization algorithms to potentially enhance their generalizability and accuracy in predicting heart failure. In the final stage, the shortlisted models (XGBC, RFC, and DTC) were assessed using performance metrics like accuracy, precision, recall, F1-score, and Matthews Correlation Coefficient (MCC). This rigorous evaluation identified the XGGA hybrid model as the top performer, demonstrating its effectiveness in predicting heart failure. XGGA achieved impressive metrics, with an accuracy, precision, recall, and F1-score of 0.972 in the training phase, underscoring its robustness. Notably, the model's predictions deviated by less than 5.5 % for patients classified as alive and by less than 1.2 % for those classified as deceased compared to the actual outcomes, reflecting minimal error and high predictive reliability. In contrast, the DTC base model was the least effective, with an accuracy of 0.840 and a precision of 0.847. Overall, the optimization using the GAO algorithm significantly enhanced the performance of the models, highlighting the benefits of this approach.

Automated lung segmentation on chest MRI in children with cystic fibrosis

Introduction

Segmentation of lung structures in medical imaging is crucial for the application of automated post-processing steps on lung diseases like cystic fibrosis (CF). Recently, machine learning methods, particularly neural networks, have demonstrated remarkable improvements, often outperforming conventional segmentation methods. Nonetheless, challenges still remain when attempting to segment various imaging modalities and diseases, especially when the visual characteristics of pathologic findings significantly deviate from healthy tissue.

Methods

Our study focuses on imaging of pediatric CF patients [mean age, standard deviation (7.50 ± 4.6)], utilizing deep learning-based methods for automated lung segmentation from chest magnetic resonance imaging (MRI). A total of 165 standardized annual surveillance MRI scans from 84 patients with CF were segmented using the nnU-Net framework. Patient cases represented a range of disease severities and ages. The nnU-Net was trained and evaluated on three MRI sequences (BLADE, VIBE, and HASTE), which are highly relevant for the evaluation of CF induced lung changes. We utilized 40 cases for training per sequence, and tested with 15 cases per sequence, using the Sørensen-Dice-Score, Pearson's correlation coefficient (r), a segmentation questionnaire, and slice-based analysis.

Results

The results demonstrated a high level of segmentation performance across all sequences, with only minor differences observed in the mean Dice coefficient: BLADE (0.96 ± 0.05), VIBE (0.96 ± 0.04), and HASTE (0.95 ± 0.05). Additionally, the segmentation quality was consistent across different disease severities, patient ages, and sizes. Manual evaluation identified specific challenges, such as incomplete segmentations near the diaphragm and dorsal regions. Validation on a separate, external dataset of nine toddlers (2-24 months) demonstrated generalizability of the trained model achieving a Dice coefficient of 0.85 ± 0.03.

Discussion and conclusion

Overall, our study demonstrates the feasibility and effectiveness of using nnU-Net for automated segmentation of lung halves in pediatric CF patients, showing promising directions for advanced image analysis techniques to assist in clinical decision-making and monitoring of CF lung disease progression. Despite these achievements, further improvements are needed to address specific segmentation challenges and enhance generalizability.

Application of U-Net Network Utilizing Multiattention Gate for MRI Segmentation of Brain Tumors

BACKGROUND

Studies have shown that the type of low-grade glioma is associated with its shape. The traditional diagnostic method involves extraction of the tumor shape from MRIs and diagnosing the type of glioma based on corresponding relationship between the glioma shape and type. This method is affected by the MRI background, tumor pixel size, and doctors' professional level, leading to misdiagnoses and missed diagnoses. With the help of deep learning algorithms, the shape of a glioma can be automatically segmented, thereby assisting doctors to focus more on the diagnosis of glioma and improving diagnostic efficiency. The segmentation of glioma MRIs using traditional deep learning algorithms exhibits limited accuracy, thereby impeding the effectiveness of assisting doctors in the diagnosis. The primary objective of our research is to facilitate the segmentation of low-grade glioma MRIs for medical practitioners through the utilization of deep learning algorithms.

METHODS

In this study, a UNet glioma segmentation network that incorporates multiattention gates was proposed to address this limitation. The UNet-based algorithm in the coding part integrated the attention gate into the hierarchical structure of the network to suppress the features of irrelevant regions and reduce the feature redundancy. In the decoding part, by adding attention gates in the fusion process of low- and high-level features, important feature information was highlighted, model parameters were reduced, and model sensitivity and accuracy were improved.

RESULTS

The network model performed image segmentation on the glioma MRI dataset, and the accuracy and average intersection ratio (mIoU) of the algorithm segmentation reached 99.7%, 87.3%, 99.7%, and 87.6%.

CONCLUSIONS

Compared with the UNet, PSPNet, and Attention UNet network models, this network model has obvious advantages in accuracy, mIoU, and loss convergence. It can serve as a standard for assisting doctors in diagnosis.

A multi-scale feature selection module based architecture for the diagnosis of Alzheimer's disease on [18F]FDG PET

OBJECTIVE

Alzheimer's disease (AD) is a prevalent form of dementia worldwide as a cryptic neurodegenerative disease. The symptoms of AD will last for several years, which brings great mental and economic burden to patients and their families. Unfortunately, the complete cure of AD still faces great challenges. Therefore, it is crucial to diagnose the disease in the early stage.

MATERIALS AND METHODS

The Visual Geometry Group (VGG) network serves as the backbone for feature extraction, which could reduce the time cost of network training to a certain extent. In order to better extract image information and pay attention to the association information in the images, the group convolutional module and the multi-scale RNN-based feature selection module are proposed. The dataset employed in the study are drawn from [18F]FDG-PET images within the Alzheimer's Disease Neuroimaging Initiative (ADNI) database.

RESULTS

Comprehensive experimental results show that the proposed model outperforms several competing approaches in AD-related diagnostic tasks. In addition, the model reduces the number of parameters of the model compared to the backbone model, from 134.27 M to 17.36 M. Furthermore, the ablation reaserch is conducted to confirm the effectiveness of the proposed module.

CONCLUSIONS

The paper introduces a lightweight network architecture for the early diagnosis of AD. In contrast to analogous methodologies, the proposed method yields acceptable results.

Automated volumetric analysis of the inner ear fluid space from hydrops magnetic resonance imaging using 3D neural networks

Due to the development of magnetic resonance (MR) imaging processing technology, image-based identification of endolymphatic hydrops (EH) has played an important role in understanding inner ear illnesses, such as Meniere's disease or fluctuating sensorineural hearing loss. We segmented the inner ear, consisting of the cochlea, vestibule, and semicircular canals, using a 3D-based deep neural network model for accurate and automated EH volume ratio calculations. We built a dataset of MR cisternography (MRC) and HYDROPS-Mi2 stacks labeled with the segmentation of the perilymph fluid space and endolymph fluid space of the inner ear to devise a 3D segmentation deep neural network model. End-to-end learning was used to segment the perilymph fluid and the endolymph fluid spaces simultaneously using aligned pair data of the MRC and HYDROPS-Mi2 stacks. Consequently, the segmentation performance of the total fluid space and endolymph fluid space had Dice similarity coefficients of 0.9574 and 0.9186, respectively. In addition, the EH volume ratio calculated by experienced otologists and the EH volume ratio value predicted by the proposed deep learning model showed high agreement according to the interclass correlation coefficient (ICC) and Bland-Altman plot analysis.

MaskAppendix: Backbone-Enriched Mask R-CNN Based on Grad-CAM for Automatic Appendix Segmentation

BACKGROUND

A leading cause of emergency abdominal surgery, appendicitis is a common condition affecting millions of people worldwide. Automatic and accurate segmentation of the appendix from medical imaging is a challenging task, due to its small size, variability in shape, and proximity to other anatomical structures.

METHODS

In this study, we propose a backbone-enriched Mask R-CNN architecture (MaskAppendix) on the Detectron platform, enhanced with Gradient-weighted Class Activation Mapping (Grad-CAM), for precise appendix segmentation on computed tomography (CT) scans. In the proposed MaskAppendix deep learning model, ResNet101 network is used as the backbone. By integrating Grad-CAM into the MaskAppendix network, our model improves feature localization, allowing it to better capture subtle variations in appendix morphology.

RESULTS

We conduct extensive experiments on a dataset of abdominal CT scans, demonstrating that our method achieves state-of-the-art performance in appendix segmentation, outperforming traditional segmentation techniques in terms of both accuracy and robustness. In the automatic segmentation of the appendix region in CT slices, a DSC score of 87.17% was achieved with the proposed approach, and the results obtained have the potential to improve clinical diagnostic accuracy.

CONCLUSIONS

This framework provides an effective tool for aiding clinicians in the diagnosis of appendicitis and other related conditions, reducing the potential for diagnostic errors and enhancing clinical workflow efficiency.

Advances in Medical Image Segmentation: A Comprehensive Review of Traditional, Deep Learning and Hybrid Approaches

Medical image segmentation plays a critical role in accurate diagnosis and treatment planning, enabling precise analysis across a wide range of clinical tasks. This review begins by offering a comprehensive overview of traditional segmentation techniques, including thresholding, edge-based methods, region-based approaches, clustering, and graph-based segmentation. While these methods are computationally efficient and interpretable, they often face significant challenges when applied to complex, noisy, or variable medical images. The central focus of this review is the transformative impact of deep learning on medical image segmentation. We delve into prominent deep learning architectures such as Convolutional Neural Networks (CNNs), Fully Convolutional Networks (FCNs), U-Net, Recurrent Neural Networks (RNNs), Adversarial Networks (GANs), and Autoencoders (AEs). Each architecture is analyzed in terms of its structural foundation and specific application to medical image segmentation, illustrating how these models have enhanced segmentation accuracy across various clinical contexts. Finally, the review examines the integration of deep learning with traditional segmentation methods, addressing the limitations of both approaches. These hybrid strategies offer improved segmentation performance, particularly in challenging scenarios involving weak edges, noise, or inconsistent intensities. By synthesizing recent advancements, this review provides a detailed resource for researchers and practitioners, offering valuable insights into the current landscape and future directions of medical image segmentation.

Breast tumor segmentation using neural cellular automata and shape guided segmentation in mammography images

PURPOSE

Using computer-aided design (CAD) systems, this research endeavors to enhance breast cancer segmentation by addressing data insufficiency and data complexity during model training. As perceived by computer vision models, the inherent symmetry and complexity of mammography images make segmentation difficult. The objective is to optimize the precision and effectiveness of medical imaging.

METHODS

The study introduces a hybrid strategy combining shape-guided segmentation (SGS) and M3D-neural cellular automata (M3D-NCA), resulting in improved computational efficiency and performance. The implementation of Shape-guided segmentation (SGS) during the initialization phase, coupled with the elimination of convolutional layers, enables the model to effectively reduce computation time. The research proposes a novel loss function that combines segmentation losses from both components for effective training.

RESULTS

The robust technique provided aims to improve the accuracy and consistency of breast tumor segmentation, leading to significant improvements in medical imaging and breast cancer detection and treatment.

CONCLUSION

This study enhances breast cancer segmentation in medical imaging using CAD systems. Combining shape-guided segmentation (SGS) and M3D-neural cellular automata (M3D-NCA) is a hybrid approach that improves performance and computational efficiency by dealing with complex data and not having enough training data. The approach also reduces computing time and improves training efficiency. The study aims to improve breast cancer detection and treatment methods in medical imaging technology.

A 25-Year Retrospective of the Use of AI for Diagnosing Acute Stroke: Systematic Review

BACKGROUND

Stroke is a leading cause of death and disability worldwide. Rapid and accurate diagnosis is crucial for minimizing brain damage and optimizing treatment plans.

OBJECTIVE

This review aims to summarize the methods of artificial intelligence (AI)-assisted stroke diagnosis over the past 25 years, providing an overview of performance metrics and algorithm development trends. It also delves into existing issues and future prospects, intending to offer a comprehensive reference for clinical practice.

METHODS

A total of 50 representative articles published between 1999 and 2024 on using AI technology for stroke prevention and diagnosis were systematically selected and analyzed in detail.

RESULTS

AI-assisted stroke diagnosis has made significant advances in stroke lesion segmentation and classification, stroke risk prediction, and stroke prognosis. Before 2012, research mainly focused on segmentation using traditional thresholding and heuristic techniques. From 2012 to 2016, the focus shifted to machine learning (ML)-based approaches. After 2016, the emphasis moved to deep learning (DL), which brought significant improvements in accuracy. In stroke lesion segmentation and classification as well as stroke risk prediction, DL has shown superiority over ML. In stroke prognosis, both DL and ML have shown good performance.

CONCLUSIONS

Over the past 25 years, AI technology has shown promising performance in stroke diagnosis.

ChoroidSeg-ViT: A Transformer Model for Choroid Layer Segmentation Based on a Mixed Attention Feature Enhancement Mechanism

Purpose

To develop a Vision Transformer (ViT) model based on the mixed attention feature enhancement mechanism, ChoroidSeg-ViT, for choroid layer segmentation in optical coherence tomography (OCT) images.

Methods

This study included a dataset of 100 OCT B-scans images. Ground truths were carefully labeled by experienced ophthalmologists. An end-to-end local-enhanced Transformer model, ChoroidSeg-ViT, was designed to segment the choroid layer by integrating the local enhanced feature extraction and semantic feature fusion paths. Standard segmentation metrics were selected to evaluate ChoroidSeg-ViT.

Results

Experimental results demonstrate that ChoroidSeg-ViT exhibited superior segmentation performance (mDice: 98.31, mIoU: 96.62, mAcc: 98.29) compared to other deep learning approaches, thus indicating the effectiveness and superiority of this proposed model for the choroid layer segmentation task. Furthermore, ablation and generalization experiments validated the reasonableness of the module design.

Conclusions

We developed a novel Transformer model to precisely and automatically segment the choroid layer and achieved the state-of-the-art performance.

Translational Relevance

ChoroidSeg-ViT could segment precise and smooth choroid layers and form the basis of an automatic choroid analysis system that would facilitate future choroidal research in ophthalmology.

Brain-GCN-Net: Graph-Convolutional Neural Network for brain tumor identification

BACKGROUND

The intersection of artificial intelligence and medical image analysis has ushered in a new era of innovation and changed the landscape of brain tumor detection and diagnosis. Correct detection and classification of brain tumors based on medical images is crucial for early diagnosis and effective treatment. Convolutional Neural Network (CNN) models are widely used for disease detection. However, they are sometimes unable to sufficiently recognize the complex features of medical images.

METHODS

This paper proposes a fused Deep Learning (DL) model that combines Graph Neural Networks (GNN), which recognize relational dependencies of image regions, and CNN, which captures spatial features, is proposed to improve brain tumor detection. By integrating these two architectures, our model achieves a more comprehensive representation of brain tumor images and improves classification performance. The proposed model is evaluated on a public dataset of 10847 MRI images. The results show that the proposed model outperforms the existing pre-trained models and traditional CNN architectures.

RESULTS

The fused DL model achieves 93.68% accuracy in brain tumor classification. The results indicate that the proposed model outperforms the existing pre-trained models and traditional CNN architectures.

CONCLUSION

The numerical results suggest that the model should be further investigated for potential use in clinical trials to improve clinical decision-making.

MultiTrans: Multi-branch transformer network for medical image segmentation

BACKGROUND AND OBJECTIVE

Transformer, which is notable for its ability of global context modeling, has been used to remedy the shortcomings of Convolutional neural networks (CNN) and break its dominance in medical image segmentation. However, the self-attention module is both memory and computational inefficient, so many methods have to build their Transformer branch upon largely downsampled feature maps or adopt the tokenized image patches to fit their model into accessible GPUs. This patch-wise operation restricts the network in extracting pixel-level intrinsic structural or dependencies inside each patch, hurting the performance of pixel-level classification tasks.

METHODS

To tackle these issues, we propose a memory- and computation-efficient self-attention module to enable reasoning on relatively high-resolution features, promoting the efficiency of learning global information while effective grasping fine spatial details. Furthermore, we design a novel Multi-Branch Transformer (MultiTrans) architecture to provide hierarchical features for handling objects with variable shapes and sizes in medical images. By building four parallel Transformer branches on different levels of CNN, our hybrid network aggregates both multi-scale global contexts and multi-scale local features.

RESULTS

MultiTrans achieves the highest segmentation accuracy on three medical image datasets with different modalities: Synapse, ACDC and M&Ms. Compared to the Standard Self-Attention (SSA), the proposed Efficient Self-Attention (ESA) can largely reduce the training memory and computational complexity while even slightly improve the accuracy. Specifically, the training memory cost, FLOPs and Params of our ESA are 18.77%, 20.68% and 74.07% of the SSA.

CONCLUSIONS

Experiments on three medical image datasets demonstrate the generality and robustness of the designed network. The ablation study shows the efficiency and effectiveness of our proposed ESA. Code is available at: https://github.com/Yanhua-Zhang/MultiTrans-extension.

Deep-learning-based method for the segmentation of ureter and renal pelvis on non-enhanced CT scans

This study aimed to develop a deep-learning (DL) based method for three-dimensional (3D) segmentation of the upper urinary tract (UUT), including ureter and renal pelvis, on non-enhanced computed tomography (NECT) scans. A total of 150 NECT scans with normal appearance of the left UUT were chosen for this study. The dataset was divided into training (n = 130) and validation sets (n = 20). The test set contained 29 randomly chosen cases with computed tomography urography (CTU) and NECT scans, all with normal appearance of the left UUT. An experienced radiologist marked out the left renal pelvis and ureter on each scan. Two types of frameworks (entire and sectional) with three types of DL models (basic UNet, UNet3 + and ViT-UNet) were developed, and evaluated. The sectional framework with basic UNet model achieved the highest mean precision (85.5%) and mean recall (71.9%) on the test set compared to all other tested methods. Compared with CTU scans, this method had higher axial UUT recall than CTU (82.5% vs 69.1%, P < 0.01). This method achieved similar or better visualization of UUT than CTU in many cases, however, in some cases, it exhibited a non-ignorable false-positive rate. The proposed DL method demonstrates promising potential in automated 3D UUT segmentation on NECT scans. The proposed DL models could remarkably improve the efficiency of UUT reconstruction, and have the potential to save many patients from invasive examinations such as CTU. DL models could also serve as a valuable complement to CTU.

Automated peripheral nerve segmentation for MR-neurography

BACKGROUND

Magnetic resonance neurography (MRN) is increasingly used as a diagnostic tool for peripheral neuropathies. Quantitative measures enhance MRN interpretation but require nerve segmentation which is time-consuming and error-prone and has not become clinical routine. In this study, we applied neural networks for the automated segmentation of peripheral nerves.

METHODS

A neural segmentation network was trained to segment the sciatic nerve and its proximal branches on the MRN scans of the right and left upper leg of 35 healthy individuals, resulting in 70 training examples, via 5-fold cross-validation (CV). The model performance was evaluated on an independent test set of one-sided MRN scans of 60 healthy individuals.

RESULTS

Mean Dice similarity coefficient (DSC) in CV was 0.892 (95% confidence interval [CI]: 0.888-0.897) with a mean Jaccard index (JI) of 0.806 (95% CI: 0.799-0.814) and mean Hausdorff distance (HD) of 2.146 (95% CI: 2.184-2.208). For the independent test set, DSC and JI were lower while HD was higher, with a mean DSC of 0.789 (95% CI: 0.760-0.815), mean JI of 0.672 (95% CI: 0.642-0.699), and mean HD of 2.118 (95% CI: 2.047-2.190).

CONCLUSION

The deep learning-based segmentation model showed a good performance for the task of nerve segmentation. Future work will focus on extending training data and including individuals with peripheral neuropathies in training to enable advanced peripheral nerve disease characterization.

RELEVANCE STATEMENT

The results will serve as a baseline to build upon while developing an automated quantitative MRN feature analysis framework for application in routine reading of MRN examinations.

KEY POINTS

Quantitative measures enhance MRN interpretation, requiring complex and challenging nerve segmentation. We present a deep learning-based segmentation model with good performance. Our results may serve as a baseline for clinical automated quantitative MRN segmentation.

AMFP-net: Adaptive multi-scale feature pyramid network for diagnosis of pneumoconiosis from chest X-ray images

Early detection of pneumoconiosis by routine health screening of workers in the mining industry is critical for preventing the progression of this incurable disease. Automated pneumoconiosis classification in chest X-ray images is challenging due to the low contrast of opacities, inter-class similarity, intra-class variation and the existence of artifacts. Compared to traditional methods, convolutional neural networks have shown significant improvement in pneumoconiosis classification tasks, however, accurate classification remains challenging due to mainly the inability to focus on semantically meaningful lesion opacities. Most existing networks focus on high level abstract information and ignore low level detailed object information. Different from natural images where an object occupies large space, the classification of pneumoconiosis depends on the density of small opacities inside the lung. To address this issue, we propose a novel two-stage adaptive multi-scale feature pyramid network called AMFP-Net for the diagnosis of pneumoconiosis from chest X-rays. The proposed model consists of 1) an adaptive multi-scale context block to extract rich contextual and discriminative information and 2) a weighted feature fusion module to effectively combine low level detailed and high level global semantic information. This two-stage network first segments the lungs to focus more on relevant regions by excluding irrelevant parts of the image, and then utilises the segmented lungs to classify pneumoconiosis into different categories. Extensive experiments on public and private datasets demonstrate that the proposed approach can outperform state-of-the-art methods for both segmentation and classification.

Simple Method to Assess Foam Structure and Stability using Hydrophobin and BSA as Model Systems

The properties and arrangement of surface-active molecules at air-water interfaces influence foam stability and bubble shape. Such multiscale-relationships necessitate a well-conducted analysis of mesoscopic foam properties. We introduce a novel automated and precise method to characterize bubble growth, size distribution and shape based on image analysis and using the machine learning algorithm Cellpose. Studying the temporal evolution of bubble size and shape facilitates conclusions on foam stability. The addition of two sets of masks, for tiny bubbles and large bubbles, provides for a high precision of analysis. A python script for analysis of the evolution of bubble diameter, circularity and dispersity is provided in the Supporting Information. Using foams stabilized by bovine serum albumin (BSA), hydrophobin (HP), and blends thereof, we show how this technique can be used to precisely characterize foam structures. Foams stabilized by HP show a significantly increased foam stability and rounder bubble shape than BSA-stabilized foams. These differences are induced by the different molecular structure of the two proteins. Our study shows that the proposed method provides an efficient way to analyze relevant foam properties in detail and at low cost, with higher precision than conventional methods of image analysis.

An Automatic Method for Elbow Joint Recognition, Segmentation and Reconstruction

Elbow computerized tomography (CT) scans have been widely applied for describing elbow morphology. To enhance the objectivity and efficiency of clinical diagnosis, an automatic method to recognize, segment, and reconstruct elbow joint bones is proposed in this study. The method involves three steps: initially, the humerus, ulna, and radius are automatically recognized based on the anatomical features of the elbow joint, and the prompt boxes are generated. Subsequently, elbow MedSAM is obtained through transfer learning, which accurately segments the CT images by integrating the prompt boxes. After that, hole-filling and object reclassification steps are executed to refine the mask. Finally, three-dimensional (3D) reconstruction is conducted seamlessly using the marching cube algorithm. To validate the reliability and accuracy of the method, the images were compared to the masks labeled by senior surgeons. Quantitative evaluation of segmentation results revealed median intersection over union (IoU) values of 0.963, 0.959, and 0.950 for the humerus, ulna, and radius, respectively. Additionally, the reconstructed surface errors were measured at 1.127, 1.523, and 2.062 mm, respectively. Consequently, the automatic elbow reconstruction method demonstrates promising capabilities in clinical diagnosis, preoperative planning, and intraoperative navigation for elbow joint diseases.

Cooperative AI training for cardiothoracic segmentation in computed tomography: An iterative multi-center annotation approach

PURPOSE

Radiological reporting is transitioning to quantitative analysis, requiring large-scale multi-center validation of biomarkers. A major prerequisite and bottleneck for this task is the voxelwise annotation of image data, which is time-consuming for large cohorts. In this study, we propose an iterative training workflow to support and facilitate such segmentation tasks, specifically for high-resolution thoracic CT data.

METHODS

Our study included 132 thoracic CT scans from clinical practice, annotated by 13 radiologists. In three iterative training experiments, we aimed to improve and accelerate segmentation of the heart and mediastinum. Each experiment started with manual segmentation of 5-25 CT scans, which served as training data for a nnU-Net. Further iterations incorporated AI pre-segmentation and human correction to improve accuracy, accelerate the annotation process, and reduce human involvement over time.

RESULTS

Results showed consistent improvement in AI model quality with each iteration. Resampled datasets improved the Dice similarity coefficients for both the heart (DCS 0.91 [0.88; 0.92]) and the mediastinum (DCS 0.95 [0.94; 0.95]). Our AI models reduced human interaction time by 50 % for heart and 70 % for mediastinum segmentation in the most potent iteration. A model trained on only five datasets achieved satisfactory results (DCS > 0.90).

CONCLUSIONS

The iterative training workflow provides an efficient method for training AI-based segmentation models in multi-center studies, improving accuracy over time and simultaneously reducing human intervention. Future work will explore the use of fewer initial datasets and additional pre-processing methods to enhance model quality.

TPA: Two-stage progressive attention segmentation framework for hepatocellular carcinoma on multi-modality MRI

BACKGROUND

Dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) plays a crucial role in the diagnosis and measurement of hepatocellular carcinoma (HCC). The multi-modality information contained in the multi-phase images of DCE-MRI is important for improving segmentation. However, this remains a challenging task due to the heterogeneity of HCC, which may cause one HCC lesion to have varied imaging appearance in each phase of DCE-MRI. In particular, some phases exhibit inconsistent sizes and boundaries will result in a lack of correlation between modalities, and it may pose inaccurate segmentation results.

PURPOSE

We aim to design a multi-modality segmentation model that can learn meaningful inter-phase correlation for achieving HCC segmentation.

METHODS

In this study, we propose a two-stage progressive attention segmentation framework (TPA) for HCC based on the transformer and the decision-making process of radiologists. Specifically, the first stage aims to fuse features from multi-phase images to identify HCC and provide localization region. In the second stage, a multi-modality attention transformer module (MAT) is designed to focus on the features that can represent the actual size.

RESULTS

We conduct training, validation, and test in a single-center dataset (386 cases), followed by external test on a batch of multi-center datasets (83 cases). Furthermore, we analyze a subgroup of data with weak inter-phase correlation in the test set. The proposed model achieves Dice coefficient of 0.822 and 0.772 in the internal and external test sets, respectively, and 0.829, 0.791 in the subgroup. The experimental results demonstrate that our model outperforms state-of-the-art models, particularly within subgroup.

CONCLUSIONS

The proposed TPA provides best segmentation results, and utilizing clinical prior knowledge for network design is practical and feasible.

MRI-Derived Dural Sac and Lumbar Vertebrae 3D Volumetry Has Potential for Detection of Marfan Syndrome

PURPOSE

To assess the feasibility and diagnostic accuracy of MRI-derived 3D volumetry of lower lumbar vertebrae and dural sac segments using shape-based machine learning for the detection of Marfan syndrome (MFS) compared with dural sac diameter ratios (the current clinical standard).

MATERIALS AND METHODS

The final study sample was 144 patients being evaluated for MFS from 01/2012 to 12/2016, of whom 81 were non-MFS patients (46 [67%] female, 36 ± 16 years) and 63 were MFS patients (36 [57%] female, 35 ± 11 years) according to the 2010 Revised Ghent Nosology. All patients underwent 1.5T MRI with isotropic 1 × 1 × 1 mm3 3D T2-weighted acquisition of the lumbosacral spine. Segmentation and quantification of vertebral bodies L3-L5 and dural sac segments L3-S1 were performed using a shape-based machine learning algorithm. For comparison with the current clinical standard, anteroposterior diameters of vertebral bodies and dural sac were measured. Ratios between dural sac volume/diameter at the respective level and vertebral body volume/diameter were calculated.

RESULTS

Three-dimensional volumetry revealed larger dural sac volumes (p < 0.001) and volume ratios (p < 0.001) at L3-S1 levels in MFS patients compared with non-MFS patients. For the detection of MFS, 3D volumetry achieved higher AUCs at L3-S1 levels (0.743, 0.752, 0.808, and 0.824) compared with dural sac diameter ratios (0.673, 0.707, 0.791, and 0.848); a significant difference was observed only for L3 (p < 0.001).

CONCLUSION

MRI-derived 3D volumetry of the lumbosacral dural sac and vertebral bodies is a feasible method for quantifying dural ectasia using shape-based machine learning. Non-inferior diagnostic accuracy was observed compared with dural sac diameter ratio (the current clinical standard for MFS detection).

Using machine learning to distinguish between authentic and imitation Jackson Pollock poured paintings: A tile-driven approach to computer vision

Jackson Pollock's abstract poured paintings are celebrated for their striking aesthetic qualities. They are also among the most financially valued and imitated artworks, making them vulnerable to high-profile controversies involving Pollock-like paintings of unknown origin. Given the increased employment of artificial intelligence applications across society, we investigate whether established machine learning techniques can be adopted by the art world to help detect imitation Pollocks. The low number of images compared to typical artificial intelligence projects presents a potential limitation for art-related applications. To address this limitation, we develop a machine learning strategy involving a novel image ingestion method which decomposes the images into sets of multi-scaled tiles. Leveraging the power of transfer learning, this approach distinguishes between authentic and imitation poured artworks with an accuracy of 98.9%. The machine also uses the multi-scaled tiles to generate novel visual aids and interpretational parameters which together facilitate comparisons between the machine's results and traditional investigations of Pollock's artistic style.

Improved Latin hypercube sampling initialization-based whale optimization algorithm for COVID-19 X-ray multi-threshold image segmentation

Image segmentation techniques play a vital role in aiding COVID-19 diagnosis. Multi-threshold image segmentation methods are favored for their computational simplicity and operational efficiency. Existing threshold selection techniques in multi-threshold image segmentation, such as Kapur based on exhaustive enumeration, often hamper efficiency and accuracy. The whale optimization algorithm (WOA) has shown promise in addressing this challenge, but issues persist, including poor stability, low efficiency, and accuracy in COVID-19 threshold image segmentation. To tackle these issues, we introduce a Latin hypercube sampling initialization-based multi-strategy enhanced WOA (CAGWOA). It incorporates a COS sampling initialization strategy (COSI), an adaptive global search approach (GS), and an all-dimensional neighborhood mechanism (ADN). COSI leverages probability density functions created from Latin hypercube sampling, ensuring even solution space coverage to improve the stability of the segmentation model. GS widens the exploration scope to combat stagnation during iterations and improve segmentation efficiency. ADN refines convergence accuracy around optimal individuals to improve segmentation accuracy. CAGWOA's performance is validated through experiments on various benchmark function test sets. Furthermore, we apply CAGWOA alongside similar methods in a multi-threshold image segmentation model for comparative experiments on lung X-ray images of infected patients. The results demonstrate CAGWOA's superiority, including better image detail preservation, clear segmentation boundaries, and adaptability across different threshold levels.

Few-Shot Learning for Medical Image Segmentation Using 3D U-Net and Model-Agnostic Meta-Learning (MAML)

Deep learning has attained state-of-the-art results in general image segmentation problems; however, it requires a substantial number of annotated images to achieve the desired outcomes. In the medical field, the availability of annotated images is often limited. To address this challenge, few-shot learning techniques have been successfully adapted to rapidly generalize to new tasks with only a few samples, leveraging prior knowledge. In this paper, we employ a gradient-based method known as Model-Agnostic Meta-Learning (MAML) for medical image segmentation. MAML is a meta-learning algorithm that quickly adapts to new tasks by updating a model's parameters based on a limited set of training samples. Additionally, we use an enhanced 3D U-Net as the foundational network for our models. The enhanced 3D U-Net is a convolutional neural network specifically designed for medical image segmentation. We evaluate our approach on the TotalSegmentator dataset, considering a few annotated images for four tasks: liver, spleen, right kidney, and left kidney. The results demonstrate that our approach facilitates rapid adaptation to new tasks using only a few annotated images. In 10-shot settings, our approach achieved mean dice coefficients of 93.70%, 85.98%, 81.20%, and 89.58% for liver, spleen, right kidney, and left kidney segmentation, respectively. In five-shot sittings, the approach attained mean Dice coefficients of 90.27%, 83.89%, 77.53%, and 87.01% for liver, spleen, right kidney, and left kidney segmentation, respectively. Finally, we assess the effectiveness of our proposed approach on a dataset collected from a local hospital. Employing five-shot sittings, we achieve mean Dice coefficients of 90.62%, 79.86%, 79.87%, and 78.21% for liver, spleen, right kidney, and left kidney segmentation, respectively.

Dual-branch feature fusion S3D V-Net network for lung nodules segmentation

BACKGROUND

Accurate segmentation of lung nodules can help doctors get more accurate results and protocols in early lung cancer diagnosis and treatment planning, so that patients can be better detected and treated at an early stage, and the mortality rate of lung cancer can be reduced.

PURPOSE

Currently, the improvement of lung nodule segmentation accuracy has been limited by his heterogeneous performance in the lungs, the imbalance between segmentation targets and background pixels, and other factors. We propose a new 2.5D lung nodule segmentation network model for lung nodule segmentation. This network model can well improve the extraction of edge information of lung nodules, and fuses intra-slice and inter-slice features, which makes good use of the three-dimensional structural information of lung nodules and can more effectively improve the accuracy of lung nodule segmentation.

METHODS

Our approach is based on a typical encoding-decoding network structure for improvement. The improved model captures the features of multiple nodules in both 3-D and 2-D CT images, complements the information of the segmentation target's features and enhances the texture features at the edges of the pulmonary nodules through the dual-branch feature fusion module (DFFM) and the reverse attention context module (RACM), and employs central pooling instead of the maximal pooling operation, which is used to preserve the features around the target and to eliminate the edge-irrelevant features, to further improve the performance of the segmentation of the pulmonary nodules.

RESULTS

We evaluated this method on a wide range of 1186 nodules from the LUNA16 dataset, and averaging the results of ten cross-validated, the proposed method achieved the mean dice similarity coefficient (mDSC) of 84.57%, the mean overlapping error (mOE) of 18.73% and average processing of a case is about 2.07 s. Moreover, our results were compared with inter-radiologist agreement on the LUNA16 dataset, and the average difference was 0.74%.

CONCLUSION

The experimental results show that our method improves the accuracy of pulmonary nodules segmentation and also takes less time than more 3-D segmentation methods in terms of time.