Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved?

SHFormer: Dynamic spectral filtering convolutional neural network and high-pass kernel generation transformer for adaptive MRI reconstruction

Attention Mechanism (AM) selectively focuses on essential information for imaging tasks and captures relationships between regions from distant pixel neighborhoods to compute feature representations. Accelerated magnetic resonance image (MRI) reconstruction can benefit from AM, as the imaging process involves acquiring Fourier domain measurements that influence the image representation in a non-local manner. However, AM-based models are more adept at capturing low-frequency information and have limited capacity in constructing high-frequency representations, restricting the models to smooth reconstruction. Secondly, AM-based models need mode-specific retraining for multimodal MRI data as their knowledge is restricted to local contextual variations within modes that might be inadequate to capture the diverse transferable features across heterogeneous data domains. To address these challenges, we propose a neuromodulation-based discriminative multi-spectral AM for scalable MRI reconstruction, that can (i) propagate the context-aware high-frequency details for high-quality image reconstruction, and (ii) capture features reusable to deviated unseen domains in multimodal MRI, to offer high practical value for the healthcare industry and researchers. The proposed network consists of a spectral filtering convolutional neural network to capture mode-specific transferable features to generalize to deviated MRI data domains and a dynamic high-pass kernel generation transformer that focuses on high-frequency details for improved reconstruction. We have evaluated our model on various aspects, such as comparative studies in supervised and self-supervised learning, diffusion model-based training, closed-set and open-set generalization under heterogeneous MRI data, and interpretation-based analysis. Our results show that the proposed method offers scalable and high-quality reconstruction with best improvement margins of ∼1 dB in PSNR and ∼0.01 in SSIM under unseen scenarios. Our code is available at https://github.com/sriprabhar/SHFormer.

The impact of training image quality with a novel protocol on artificial intelligence-based LGE-MRI image segmentation for potential atrial fibrillation management

BACKGROUND

Atrial fibrillation (AF) is the most common cardiac arrhythmia, affecting up to 2 % of the population. Catheter ablation is a promising treatment for AF, particularly for paroxysmal AF patients, but it often has high recurrence rates. Developing in silico models of patients' atria during the ablation procedure using cardiac MRI data may help reduce these rates.

OBJECTIVE

This study aims to develop an effective automated deep learning-based segmentation pipeline by compiling a specialized dataset and employing standardized labeling protocols to improve segmentation accuracy and efficiency. In doing so, we aim to achieve the highest possible accuracy and generalization ability while minimizing the burden on clinicians involved in manual data segmentation.

METHODS

We collected LGE-MRI data from VMRC and the cDEMRIS database. Two specialists manually labeled the data using standardized protocols to reduce subjective errors. Neural network (nnU-Net and smpU-Net++) performance was evaluated using statistical tests, including sensitivity and specificity analysis. A new database of LGE-MRI images, based on manual segmentation, was created (VMRC).

RESULTS

Our approach with consistent labeling protocols achieved a Dice coefficient of 92.4 % ± 0.8 % for the cavity and 64.5 % ± 1.9 % for LA walls. Using the pre-trained RIFE model, we attained a Dice score of approximately 89.1 % ± 1.6 % for atrial LGE-MRI imputation, outperforming classical methods. Sensitivity and specificity values demonstrated substantial enhancement in the performance of neural networks trained with the new protocol.

CONCLUSION

Standardized labeling and RIFE applications significantly improved machine learning tool efficiency for constructing 3D LA models. This novel approach supports integrating state-of-the-art machine learning methods into broader in silico pipelines for predicting ablation outcomes in AF patients.

Atten-Nonlocal Unet: Attention and Non-local Unet for medical image segmentation

The convolutional neural network(CNN)-based models have emerged as the predominant approach for medical image segmentation due to their effective inductive bias. However, their limitation lies in the lack of long-range information. In this study, we propose the Atten-Nonlocal Unet model that integrates CNN and transformer to overcome this limitation and precisely capture global context in 2D features. Specifically, we utilize the BCSM attention module and the Cross Non-local module to enhance feature representation, thereby improving the segmentation accuracy. Experimental results on the Synapse, ACDC, and AVT datasets show that Atten-Nonlocal Unet achieves DSC scores of 84.15%, 91.57%, and 86.94% respectively, and has 95% HD of 15.17, 1.16, and 4.78 correspondingly. Compared to the existing methods for medical image segmentation, the proposed method demonstrates superior segmentation performance, ensuring high accuracy in segmenting large organs while improving segmentation for small organs.

CNN-Transformer gated fusion network for medical image super-resolution

To solve the problems of image detail blurring and insufficient utilization of global information in the existing medical image super-resolution reconstruction, this paper proposes a dual-branch fusion network based on residual Transformer network and dynamic convolutional neural network (CTGFSR). The network consists of two branches, one is the global branch based on residual Transformer network, and the other is the local branch based on dynamic convolutional neural network. The global branch uses the self-attention mechanism of Transformer network, which can effectively mine the large-scale global information in the image and improve the overall quality of the image. The local branch uses the characteristic of dynamic convolution to adaptively adjust the convolution kernel parameters, which can enhance the feature extraction ability of convolutional neural network for multi-scale information and improve the detail restoration ability of the image without significantly increasing the network model size. The network uses residual skip connections to preserve the detail information in medical image super-resolution reconstruction. Finally, through the bidirectional gated attention mechanism, the two branches are fused to obtain the final super-resolution reconstruction image. This paper evaluates the performance of the network on two medical image datasets, namely ACDC abdominal MR related to medical image segmentation and L2R2022 lung CT related to registration. The experimental results show that compared with the mainstream super-resolution algorithms, CTGFSR has better overall performance. When the magnification factor is 2 or 4, compared with the convolutional neural network based CFIPC, PDCNCF, ESPCN, FSRCNN, VDSR and the Transformer network based ESRT, SwinIR, the structural similarity SSIM and peak signal-to-noise ratio PSNR have a certain improvement.

DVPT: Dynamic Visual Prompt Tuning of large pre-trained models for medical image analysis

Pre-training and fine-tuning have become popular due to the rich representations embedded in large pre-trained models, which can be leveraged for downstream medical tasks. However, existing methods typically either fine-tune all parameters or only task-specific layers of pre-trained models, overlooking the variability in input medical images. As a result, these approaches may lack efficiency or effectiveness. In this study, our goal is to explore parameter-efficient fine-tuning (PEFT) for medical image analysis. To address this challenge, we introduce a novel method called Dynamic Visual Prompt Tuning (DVPT). It can extract knowledge beneficial to downstream tasks from large models with only a few trainable parameters. First, the frozen features are transformed by a lightweight bottleneck layer to learn the domain-specific distribution of downstream medical tasks. Then, a few learnable visual prompts are employed as dynamic queries to conduct cross-attention with the transformed features, aiming to acquire sample-specific features. This DVPT module can be shared across different Transformer layers, further reducing the number of trainable parameters. We conduct extensive experiments with various pre-trained models on medical classification and segmentation tasks. We find that this PEFT method not only efficiently adapts pre-trained models to the medical domain but also enhances data efficiency with limited labeled data. For example, with only 0.5% additional trainable parameters, our method not only outperforms state-of-the-art PEFT methods but also surpasses full fine-tuning by more than 2.20% in Kappa score on the medical classification task. It can save up to 60% of labeled data and 99% of storage cost of ViT-B/16.

SWMA-UNet: Multi-Path Attention Network for Improved Medical Image Segmentation

In recent years, deep learning achieves significant advancements in medical image segmentation. Research finds that integrating Transformers and CNNs effectively addresses the limitations of CNNs in managing long-distance dependencies and understanding global information.However, existing models typically employ a serial approach to combine Transformers and CNNs, which complicates the simultaneous processing of global and local information. To address this, our study proposes a parallel multi-path attention architecture, SWMA-UNET, that integrates Transformers and CNNs. This architecture deeply mines features through parallel strategies while capturing both local details and global context information, thereby enhancing the accuracy of medical image segmentation. Experimental results indicate that our method surpasses all previously reported methods in the literature on the Synapse, ACDC, ISIC 2018 and MoNuSeg datasets.

Deep Learning-based Aligned Strain from Cine Cardiac MRI for Detection of Fibrotic Myocardial Tissue in Patients with Duchenne Muscular Dystrophy

Purpose To develop a deep learning (DL) model that derives aligned strain values from cine (noncontrast) cardiac MRI and evaluate performance of these values to predict myocardial fibrosis in patients with Duchenne muscular dystrophy (DMD). Materials and Methods This retrospective study included 139 male patients with DMD who underwent cardiac MRI at a single center between February 2018 and April 2023. A DL pipeline was developed to detect five key frames throughout the cardiac cycle and respective dense deformation fields, allowing for phase-specific strain analysis across patients and from one key frame to the next. Effectiveness of these strain values in identifying abnormal deformations associated with fibrotic segments was evaluated in 57 patients (mean age [± SD], 15.2 years ± 3.1), and reproducibility was assessed in 82 patients by comparing the study method with existing feature-tracking and DL-based methods. Statistical analysis compared strain values using t tests, mixed models, and more than 2000 machine learning models; accuracy, F1 score, sensitivity, and specificity are reported. Results DL-based aligned strain identified five times more differences (29 vs five; P < .01) between fibrotic and nonfibrotic segments compared with traditional strain values and identified abnormal diastolic deformation patterns often missed with traditional methods. In addition, aligned strain values enhanced performance of predictive models for myocardial fibrosis detection, improving specificity by 40%, overall accuracy by 17%, and accuracy in patients with preserved ejection fraction by 61%. Conclusion The proposed aligned strain technique enables motion-based detection of myocardial dysfunction at noncontrast cardiac MRI, facilitating detailed interpatient strain analysis and allowing precise tracking of disease progression in DMD. Keywords: Pediatrics, Image Postprocessing, Heart, Cardiac, Convolutional Neural Network (CNN) Duchenne Muscular Dystrophy Supplemental material is available for this article. © RSNA, 2025.

DiffuSeg: Domain-Driven Diffusion for Medical Image Segmentation

In recent years, the deployment of supervised machine learning techniques for segmentation tasks has significantly increased. Nonetheless, the annotation process for extensive datasets remains costly, labor-intensive, and error-prone. While acquiring sufficiently large datasets to train deep learning models is feasible, these datasets often experience a distribution shift relative to the actual test data. This problem is particularly critical in the domain of medical imaging, where it adversely affects the efficacy of automatic segmentation models. In this work, we introduce DiffuSeg, a novel conditional diffusion model developed for medical image data, that exploits any labels to synthesize new images in the target domain. This allows a number of new research directions, including the segmentation task that motivates this work. Our method only requires label maps from any existing datasets and unlabelled images from the target domain for image diffusion. To learn the target domain knowledge, a feature factorization variational autoencoder is proposed to provide conditional information for the diffusion model. Consequently, the segmentation network can be trained with the given labels and the synthetic images, thus avoiding human annotations. Initially, we apply our method to the MNIST dataset and subsequently adapt it for use with medical image segmentation datasets, such as retinal fundus images for vessel segmentation and MRI images for heart segmentation. Our approach exhibits significant improvements over relevant baselines in both image generation and segmentation accuracy, especially in scenarios where annotations for the target dataset are unavailable during training. An open-source implementation of our approach can be released after reviewing..

Learnable Prompting SAM-Induced Knowledge Distillation for Semi-Supervised Medical Image Segmentation

The limited availability of labeled data has driven advancements in semi-supervised learning for medical image segmentation. Modern large-scale models tailored for general segmentation, such as the Segment Anything Model (SAM), have revealed robust generalization capabilities. However, applying these models directly to medical image segmentation still exposes performance degradation. In this paper, we propose a learnable prompting SAM-induced Knowledge distillation framework (KnowSAM) for semi-supervised medical image segmentation. Firstly, we propose a Multi-view Co-training (MC) strategy that employs two distinct sub-networks to employ a co-teaching paradigm, resulting in more robust outcomes. Secondly, we present a Learnable Prompt Strategy (LPS) to dynamically produce dense prompts and integrate an adapter to fine-tune SAM specifically for medical image segmentation tasks. Moreover, we propose SAM-induced Knowledge Distillation (SKD) to transfer useful knowledge from SAM to two sub-networks, enabling them to learn from SAM's predictions and alleviate the effects of incorrect pseudo-labels during training. Notably, the predictions generated by our subnets are used to produce mask prompts for SAM, facilitating effective inter-module information exchange. Extensive experimental results on various medical segmentation tasks demonstrate that our model outperforms the state-of-the-art semi-supervised segmentation approaches. Crucially, our SAM distillation framework can be seamlessly integrated into other semi-supervised segmentation methods to enhance performance. The code will be released upon acceptance of this manuscript at https://github.com/taozh2017/KnowSAM.

Spiral cardiac quantitative susceptibility mapping for differential cardiac chamber oxygenation-Initial validation in relation to invasive blood sampling

PURPOSE

To develop a breath-hold cardiac quantitative susceptibility mapping (QSM) sequence for noninvasive measurement of differential cardiac chamber blood oxygen saturation (ΔSO2).

METHODS

A non-gated three-dimensional stack-of-spirals QSM sequence was implemented to continuously sample the data throughout the cardiac cycle. Measurements of ΔSO2 between the right and left heart chamber obtained by the proposed sequence and a previously validated navigator Cartesian QSM sequence were compared in three cohorts consisting of healthy volunteers, coronavirus disease 2019 survivors, and patients with pulmonary hypertension. In the pulmonary-hypertension cohort, Bland-Altman plots were used to assess the agreement of ΔSO2 values obtained by QSM and those obtained by invasive right heart catheterization (RHC).

RESULTS

Compared with navigator QSM (average acquisition time 419 ± 158 s), spiral QSM reduced the scan time on average by over 20-fold to a 20-s breath-hold. In all three cohorts, spiral QSM and navigator QSM yielded similar ΔSO2. Among healthy volunteers and coronavirus disease 2019 survivors, ΔSO2 was 17.41 ± 4.35% versus 17.67 ± 4.09% for spiral and navigator QSM, respectively. In pulmonary-hypertension patients, spiral QSM showed a slightly smaller ΔSO2 bias and narrower 95% limits of agreement than that obtained by navigator QSM (1.09% ± 6.47% vs. 2.79% ± 6.99%) when compared with right heart catheterization.

CONCLUSION

Breath-hold three-dimensional spiral cardiac QSM for measuring differential cardiac chamber blood oxygenation is feasible and provides values in good agreement with navigator cardiac QSM and with reference right heart catheterization.

Medical image translation with deep learning: Advances, datasets and perspectives

Traditional medical image generation often lacks patient-specific clinical information, limiting its clinical utility despite enhancing downstream task performance. In contrast, medical image translation precisely converts images from one modality to another, preserving both anatomical structures and cross-modal features, thus enabling efficient and accurate modality transfer and offering unique advantages for model development and clinical practice. This paper reviews the latest advancements in deep learning(DL)-based medical image translation. Initially, it elaborates on the diverse tasks and practical applications of medical image translation. Subsequently, it provides an overview of fundamental models, including convolutional neural networks (CNNs), transformers, and state space models (SSMs). Additionally, it delves into generative models such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Autoregressive Models (ARs), diffusion Models, and flow Models. Evaluation metrics for assessing translation quality are discussed, emphasizing their importance. Commonly used datasets in this field are also analyzed, highlighting their unique characteristics and applications. Looking ahead, the paper identifies future trends, challenges, and proposes research directions and solutions in medical image translation. It aims to serve as a valuable reference and inspiration for researchers, driving continued progress and innovation in this area.

Lightweight cross-resolution coarse-to-fine network for efficient deformable medical image registration

BACKGROUND

Accurate and efficient deformable medical image registration is crucial in medical image analysis. While recent deep learning-based registration methods have achieved state-of-the-art accuracy, they often suffer from extensive network parameters and slow inference times, leading to inefficiency. Efforts to reduce model size and input resolution can improve computational efficiency but frequently result in suboptimal accuracy.

PURPOSE

To address the trade-off between high accuracy and efficiency, we propose a Lightweight Cross-Resolution Coarse-to-Fine registration framework, termed LightCRCF.

METHODS

Our method is built on an ultra-lightweight U-Net architecture with only 0.1 million parameters, offering remarkable efficiency. To mitigate accuracy degradation resulting from fewer parameters while preserving the lightweight nature of the networks, LightCRCF introduces three key innovations as follows: (1) selecting an efficient cross-resolution coarse-to-fine (C2F) registration strategy and integrating it into the lightweight network to progressively decompose the deformation fields into multiresolution subfields to capture fine-grained deformations; (2) a Texture-aware Reparameterization (TaRep) module that integrates Sobel and Laplacian operators to extract rich textural information; (3) a Group-flow Reparameterization (GfRep) module that captures diverse deformation modes by decomposing the deformation field into multiple groups. Furthermore, we introduce a structural reparameterization technique that enhances training accuracy through multibranch structures of the TaRep and GfRep modules, while maintaining efficient inference by equivalently transforming these multibranch structures into single-branch standard convolutions.

RESULTS

We evaluate LightCRCF against various methods on the three public MRI datasets (LPBA, OASIS, and ACDC) and one CT dataset (abdomen CT). Following the previous data division methods, the LPBA dataset comprises 30 training image pairs and nine testing image pairs. For the OASIS dataset, the training, validation, and testing data consist of 1275, 110, and 660 image pairs, respectively. Similarly, for the ACDC dataset, the training, validation, and testing data include 180, 20, and 100 image pairs, respectively. For intersubject registration of the abdomen CT dataset, there are 380 training pairs, six validation pairs, and 42 testing pairs. Compared to state-of-the-art C2F methods, LightCRCF achieves comparable accuracy scores (DSC, HD95, and MSE), while demonstrating significantly superior performance across all efficiency metrics (Params, VRAM, FLOPs, and inference time). Relative to efficiency-first approaches, LightCRCF significantly outperforms these methods in accuracy metrics.

CONCLUSIONS

Our LightCRCF method offers a favorable trade-off between accuracy and efficiency, maintaining high accuracy while achieving superior efficiency, thereby highlighting its potential for clinical applications. The code will be available at https://github.com/PerceptionComputingLab/LightCRCF.

Medical image segmentation by combining feature enhancement Swin Transformer and UperNet

Medical image segmentation plays a crucial role in assisting clinical diagnosis, yet existing models often struggle with handling diverse and complex medical data, particularly when dealing with multi-scale organ and tissue structures. This paper proposes a novel medical image segmentation model, FE-SwinUper, designed to address these challenges by integrating the strengths of the Swin Transformer and UPerNet architectures. The objective is to enhance multi-scale feature extraction and improve the fusion of hierarchical organ and tissue representations through a feature enhancement Swin Transformer (FE-ST) backbone and an adaptive feature fusion (AFF) module. The FE-ST backbone utilizes self-attention mechanisms to efficiently extract rich spatial and contextual features across different scales, while the AFF module adapts to multi-scale feature fusion, mitigating the loss of contextual information. We evaluate the model on two publicly available medical image segmentation datasets: Synapse multi-organ segmentation dataset and the ACDC cardiac segmentation dataset. Our results show that FE-SwinUper outperforms existing state-of-the-art models in terms of Dice coefficient, pixel accuracy, and Hausdorff distance. The model achieves a Dice score of 91.58% on the Synapse dataset and 90.15% on the ACDC dataset. These results demonstrate the robustness and efficiency of the proposed model, indicating its potential for real-world clinical applications.

Deep Learning-Based Estimation of Myocardial Material Parameters from Cardiac MRI

BACKGROUND

Accurate estimation of myocardial material parameters is crucial to understand cardiac biomechanics and plays a key role in advancing computational modeling and clinical applications. Traditional inverse finite element (FE) methods rely on iterative optimization to infer these parameters, which is computationally expensive and time-consuming, limiting their clinical applicability.

METHODS

This study proposes a deep learning-based approach to rapidly and accurately estimate the left ventricular myocardial material parameters directly from routine cardiac magnetic resonance imaging (CMRI) data. A ResNet18-based model was trained on FEM-derived parameters from a dataset of 1288 healthy subjects.

RESULTS

The proposed model demonstrated high predictive accuracy on healthy subjects, achieving mean absolute errors of 0.0146 for Ca and 0.0139 for Cb, with mean relative errors below 5.00%. Additionally, we evaluated the model on a small pathological subset (including ARV and HCM cases). The results revealed that while the model maintained strong performance on healthy data, the prediction errors in the pathological samples were higher, indicating increased challenges in modeling diseased myocardial tissue.

CONCLUSION

This study establishes a computationally efficient and accurate deep learning framework for estimating myocardial material parameters, eliminating the need for time-consuming iterative FE optimization. While the model shows promising performance on healthy subjects, further validation and refinement are required to address its limitations in pathological conditions, thereby paving the way for personalized cardiac modeling and improved clinical decision-making.

U-shaped network combining dual-stream fusion mamba and redesigned multilayer perceptron for myocardial pathology segmentation

BACKGROUND

Cardiac magnetic resonance imaging (CMR) provides critical pathological information, such as scars and edema, which are vital for diagnosing myocardial infarction (MI). However, due to the limited pathological information in single-sequence CMR images and the small size of pathological regions, automatic segmentation of myocardial pathology remains a significant challenge.

PURPOSE

In the paper, we propose a novel two-stage anatomical-pathological segmentation framework combining Kolmogorov-Arnold Networks (KAN) and Mamba, aiming to effectively segment myocardial pathology in multi-sequence CMR images.

METHODS

First, in the coarse segmentation stage, we employed a multiline parallel MambaUnet as the anatomical structure segmentation network to obtain shape prior information. This approach effectively addresses the class imbalance issue and aids in subsequent pathological segmentation. In the fine segmentation stage, we introduced a novel U-shaped segmentation network, KANMambaNet, which features a Dual-Stream Fusion Mamba module. This module enhances the network's ability to capture long-range dependencies while improving its capability to distinguish different pathological features in small regions. Additionally, we developed a Kolmogorov-Arnold Network-based multilayer perceptron (KAN MLP) module that utilizes learnable activation functions instead of fixed nonlinear functions. This design enhances the network's flexibility in handling various pathological features, enabling more accurate differentiation of the pathological characteristics at the boundary between edema and scar regions. Our method achieves competitive segmentation performance compared to state-of-the-art models, particularly in terms of the Dice coefficient.

RESULTS

We validated our model's performance on the MyoPS2020 dataset, achieving a Dice score of 0.8041  ± $\pm$  0.0751 for myocardial edema and 0.9051  ± $\pm$  0.0240 for myocardial scar. Compared to the baseline model MambaUnet, our edema segmentation performance improved by 0.1420, and scar segmentation performance improved by 0.1081.

CONCLUSIONS

We developed an innovative two-stage anatomical-pathological segmentation framework that integrates KAN and Mamba, effectively segmenting myocardial pathology in multi-sequence CMR images. The experimental results demonstrate that our proposed method achieves superior segmentation performance compared to other state-of-the-art methods.

HALSR-Net: Improving CNN Segmentation of Cardiac Left Ventricle MRI with Hybrid Attention and Latent Space Reconstruction

Accurate cardiac MRI segmentation is vital for detailed cardiac analysis, yet the manual process is labor-intensive and prone to variability. Despite advancements in MRI technology, there remains a significant need for automated methods that can reliably and efficiently segment cardiac structures. This paper introduces HALSR-Net, a novel multi-level segmentation architecture designed to improve the accuracy and reproducibility of cardiac segmentation from Cine-MRI acquisitions, focusing on the left ventricle (LV). The methodology consists of two main phases: first, the extraction of the region of interest (ROI) using a regression model that accurately predicts the location of a bounding box around the LV; second, the semantic segmentation step based on HALSR-Net architecture. This architecture incorporates a Hybrid Attention Pooling Module (HAPM) that merges attention and pooling mechanisms to enhance feature extraction and capture contextual information. Additionally, a reconstruction module leverages latent space features to further improve segmentation accuracy. Experiments conducted on an in-house clinical dataset and two public datasets (ACDC and LVQuan19) demonstrate that HALSR-Net outperforms state-of-the-art architectures, achieving up to 98% accuracy and F1-score for the segmentation of the LV cavity and myocardium. The proposed approach effectively addresses the limitations of existing methods, offering a more accurate and robust solution for cardiac MRI segmentation, thereby likely to improve cardiac function analysis and patient care.

A novel hybrid layer-based encoder-decoder framework for 3D segmentation in congenital heart disease

The segmentation of cardiac anatomy represents a crucial stage in accurate diagnosis and subsequent treatment planning for patients with congenital heart disease (CHD). However, the current deep learning-based segmentation networks are ineffective when applied to 3D medical images of CHD because of the limited availability of training datasets and the inherent complexity exhibited by the variability of cardiac and large vessel tissues. To address this challenge, we propose a novel hybrid layer-based encoder-decoder framework for 3D CHD image segmentation. The model incorporates a global volume mixing module and a local volume-based multihead attention module, which uses a self-attention mechanism to explicitly capture the local and global dependencies of the 3D image segmentation process. This enables the model to more effectively learn the shape boundary features of organs, thereby facilitating accurate segmentation of the whole heart (WH) and great vessels. We compare our method with several popular networks on the public ImageCHD and HVSMR-2.0 datasets. The experimental results show that the proposed model achieves excellent performance in WH and great vessel segmentation tasks with high Dice coefficients and IoU indices.

Neighbor-aware calibration of segmentation networks with penalty-based constraints

Ensuring reliable confidence scores from deep neural networks is of paramount significance in critical decision-making systems, particularly in real-world domains such as healthcare. Recent literature on calibrating deep segmentation networks has resulted in substantial progress. Nevertheless, these approaches are strongly inspired by the advancements in classification tasks, and thus their uncertainty is usually modeled by leveraging the information of individual pixels, disregarding the local structure of the object of interest. Indeed, only the recent Spatially Varying Label Smoothing (SVLS) approach considers pixel spatial relationships across classes, by softening the pixel label assignments with a discrete spatial Gaussian kernel. In this work, we first present a constrained optimization perspective of SVLS and demonstrate that it enforces an implicit constraint on soft class proportions of surrounding pixels. Furthermore, our analysis shows that SVLS lacks a mechanism to balance the contribution of the constraint with the primary objective, potentially hindering the optimization process. Based on these observations, we propose NACL (Neighbor Aware CaLibration), a principled and simple solution based on equality constraints on the logit values, which enables to control explicitly both the enforced constraint and the weight of the penalty, offering more flexibility. Comprehensive experiments on a wide variety of well-known segmentation benchmarks demonstrate the superior calibration performance of the proposed approach, without affecting its discriminative power. Furthermore, ablation studies empirically show the model agnostic nature of our approach, which can be used to train a wide span of deep segmentation networks. The code is available at https://github.com/Bala93/MarginLoss.

Investigating self-supervised image denoising with denaturation

Self-supervised learning for image denoising problems in the presence of denaturation for noisy data is a crucial approach in machine learning. However, theoretical understanding of the performance of the approach that uses denatured data is lacking. To provide better understanding of the approach, in this paper, we analyze a self-supervised denoising algorithm that uses denatured data in depth through theoretical analysis and numerical experiments. Through the theoretical analysis, we discuss that the algorithm finds desired solutions to the optimization problem with the population risk, while the guarantee for the empirical risk depends on the hardness of the denoising task in terms of denaturation levels. We also conduct several experiments to investigate the performance of an extended algorithm in practice. The results indicate that the algorithm training with denatured images works, and the empirical performance aligns with the theoretical results. These results suggest several insights for further improvement of self-supervised image denoising that uses denatured data in future directions.

Dynamic graph consistency and self-contrast learning for semi-supervised medical image segmentation

Semi-supervised medical image segmentation endeavors to exploit a limited set of labeled data in conjunction with a substantial corpus of unlabeled data, with the aim of training models that can match or even exceed the efficacy of fully supervised segmentation models. Despite the potential of this approach, most existing semi-supervised medical image segmentation techniques that employ consistency regularization predominantly focus on spatial consistency at the image level, often neglecting the crucial role of feature-level channel information. To address this limitation, we propose an innovative method that integrates graph convolutional networks with a consistency regularization framework to develop a dynamic graph consistency approach. This method imposes channel-level constraints across different decoders by leveraging high-level features within the network. Furthermore, we introduce a novel self-contrast learning strategy, which performs image-level comparison within the same batch and engages in pixel-level contrast learning based on pixel positions. This approach effectively overcomes traditional contrast learning challenges related to identifying positive and negative samples, reduces computational resource consumption, and significantly improves model performance. Our experimental evaluation on three distinct medical image segmentation datasets indicates that the proposed method demonstrates superior performance across a variety of test scenarios.

Machine Learning to Predict Outcomes of Fetal Cardiac Disease: A Pilot Study

Prediction of outcomes following a prenatal diagnosis of congenital heart disease (CHD) is challenging. Machine learning (ML) algorithms may be used to reduce clinical uncertainty and improve prognostic accuracy. We performed a pilot study to train ML algorithms to predict postnatal outcomes based on clinical data. Specific objectives were to predict (1) in utero or neonatal death, (2) high-acuity neonatal care and (3) favorable outcomes. We included all fetuses with cardiac disease at Sunnybrook Health Sciences Centre, Toronto, Canada, from 2012 to 2021. Prediction models were created using the XgBoost algorithm (tree-based) with fivefold cross-validation. Among 211 cases of fetal cardiac disease, 61 were excluded (39 terminations, 21 lost to follow-up, 1 isolated arrhythmia), leaving a cohort of 150 fetuses. Fifteen (10%) demised (10 neonates) and 65 (48%) of live births required high acuity neonatal care. Of those with clinical follow-up, 60/87 (69%) had a favorable outcome. Prediction models for fetal or neonatal death, high acuity neonatal care and favorable outcome had AUCs of 0.76, 0.84 and 0.73, respectively. The most important predictors for death were the presence of non-cardiac abnormalities combined with more severe CHD. High acuity of postnatal care was predicted by anti Ro antibody and more severe CHD. Favorable outcome was most predicted by no right heart disease combined with genetic abnormalities, and maternal medications. Prediction models using ML provide good discrimination of key prenatal and postnatal outcomes among fetuses with congenital heart disease.

Semi-supervised Strong-Teacher Consistency Learning for few-shot cardiac MRI image segmentation

BACKGROUND AND OBJECTIVE

Cardiovascular disease is a leading cause of mortality worldwide. Automated analysis of heart structures in MRI is crucial for effective diagnostics. While supervised learning has advanced the field of medical image segmentation, it however requires extensive labelled data, which is often limited for cardiac MRI.

METHODS

Drawing on the principle of consistency learning, we introduce a novel semi-supervised Strong-Teacher Consistency Network for few-shot multi-class cardiac MRI image segmentation, leveraging largely available unlabelled data. This model incorporates a student-teacher architecture. A multi-teacher structure is introduced to learn diverse perspectives and avoid local optimals when dealing with largely varying cardiac structures and anatomical features. It employs a hybrid loss that emphasizes consistency between student and teacher representations, alongside supervised losses (e.g., Dice and Cross-entropy), tailored to the challenge of unlabelled data. Additionally, we introduced feature-space virtual adversarial training to enhance robust feature learning and model stability.

RESULTS

Evaluation and ablation studies on the MM-WHS and ACDC benchmark datasets show that the proposed model outperforms nine state-of-the-art semi-supervised methods, particularly with limited annotated data. It achieves 90.14% accuracy on MM-WHS and 78.45% accuracy on ACDC at labelling rates of 25% and 1%, respectively. It also highlights its unique advantages over fully-supervised and single-teacher approaches.

Semi-supervised medical image segmentation via weak-to-strong perturbation consistency and edge-aware contrastive representation

Despite that supervised learning has demonstrated impressive accuracy in medical image segmentation, its reliance on large labeled datasets poses a challenge due to the effort and expertise required for data acquisition. Semi-supervised learning has emerged as a potential solution. However, it tends to yield satisfactory segmentation performance in the central region of the foreground, but struggles in the edge region. In this paper, we propose an innovative framework that effectively leverages unlabeled data to improve segmentation performance, especially in edge regions. Our proposed framework includes two novel designs. Firstly, we introduce a weak-to-strong perturbation strategy with corresponding feature-perturbed consistency loss to efficiently utilize unlabeled data and guide our framework in learning reliable regions. Secondly, we propose an edge-aware contrastive loss that utilizes uncertainty to select positive pairs, thereby learning discriminative pixel-level features in the edge regions using unlabeled data. In this way, the model minimizes the discrepancy of multiple predictions and improves representation ability, ultimately aiming at impressive performance on both primary and edge regions. We conducted a comparative analysis of the segmentation results on the publicly available BraTS2020 dataset, LA dataset, and the 2017 ACDC dataset. Through extensive quantification and visualization experiments under three standard semi-supervised settings, we demonstrate the effectiveness of our approach and set a new state-of-the-art for semi-supervised medical image segmentation. Our code is released publicly at https://github.com/youngyzzZ/SSL-w2sPC.

PointCHD: A Point Cloud Benchmark for Congenital Heart Disease Classification and Segmentation

Congenital heart disease (CHD) is one of the most common birth defects. Due to the lack of data and the difficulty of labeling, CHD datasets are scarce. Previous studies focused on CT and other medical image modalities, while point cloud is still unexplored. Point cloud can intuitively model organ shapes, which has obvious advantages in medical analysis and diagnosis assistance. However, the production of medical point cloud dataset is more complex than that of image dataset, and the 3D modeling of internal organs needs to be reconstructed after scanning by high-precision instruments. We propose PointCHD, the first point cloud dataset for CHD diagnosis, with a large number of high precision-annotated and wide-categorized data. PointCHD includes different types of three-dimensional data with varying degrees of distortion, and supports multiple analysis tasks, i.e., classification, segmentation, reconstruction, etc. We also construct a benchmark on PointCHD with the goal of medical diagnosis, we design the analysis process and compare the performances of mainstream point cloud analysis methods. In view of the complex internal and external structures of heart point cloud, we propose a point cloud representation method based on manifold learning. By introducing normals to consider the surface continuity to construct a manifold learning method of adaptive projection plane, we can fully extract the structural features of heart, and achieve the best performance on each task of PointCHD benchmark. Finally, we summarize the existing problems of CHD point cloud analysis and prospects for potential future research directions.

The state-of-the-art in cardiac MRI reconstruction: Results of the CMRxRecon challenge in MICCAI 2023

Cardiac magnetic resonance imaging (MRI) provides detailed and quantitative evaluation of the heart's structure, function, and tissue characteristics with high-resolution spatial-temporal imaging. However, its slow imaging speed and motion artifacts are notable limitations. Undersampling reconstruction, especially data-driven algorithms, has emerged as a promising solution to accelerate scans and enhance imaging performance using highly under-sampled data. Nevertheless, the scarcity of publicly available cardiac k-space datasets and evaluation platform hinder the development of data-driven reconstruction algorithms. To address this issue, we organized the Cardiac MRI Reconstruction Challenge (CMRxRecon) in 2023, in collaboration with the 26th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). CMRxRecon presented an extensive k-space dataset comprising cine and mapping raw data, accompanied by detailed annotations of cardiac anatomical structures. With overwhelming participation, the challenge attracted more than 285 teams and over 600 participants. Among them, 22 teams successfully submitted Docker containers for the testing phase, with 7 teams submitted for both cine and mapping tasks. All teams use deep learning based approaches, indicating that deep learning has predominately become a promising solution for the problem. The first-place winner of both tasks utilizes the E2E-VarNet architecture as backbones. In contrast, U-Net is still the most popular backbone for both multi-coil and single-coil reconstructions. This paper provides a comprehensive overview of the challenge design, presents a summary of the submitted results, reviews the employed methods, and offers an in-depth discussion that aims to inspire future advancements in cardiac MRI reconstruction models. The summary emphasizes the effective strategies observed in Cardiac MRI reconstruction, including backbone architecture, loss function, pre-processing techniques, physical modeling, and model complexity, thereby providing valuable insights for further developments in this field.

ProCNS: Progressive Prototype Calibration and Noise Suppression for Weakly-Supervised Medical Image Segmentation

Weakly-supervised segmentation (WSS) has emerged as a solution to mitigate the conflict between annotation cost and model performance by adopting sparse annotation formats (e.g., point, scribble, block, etc.). Typical approaches attempt to exploit anatomy and topology priors to directly expand sparse annotations into pseudo-labels. However, due to lack of attention to the ambiguous boundaries in medical images and insufficient exploration of sparse supervision, existing approaches tend to generate erroneous and overconfident pseudo proposals in noisy regions, leading to cumulative model error and performance degradation. In this work, we propose a novel WSS approach, named ProCNS, encompassing two synergistic modules devised with the principles of progressive prototype calibration and noise suppression. Specifically, we design a Prototype-based Regional Spatial Affinity (PRSA) loss to maximize the pair-wise affinities between spatial and semantic elements, providing our model of interest with more reliable guidance. The affinities are derived from the input images and the prototype-refined predictions. Meanwhile, we propose an Adaptive Noise Perception and Masking (ANPM) module to obtain more enriched and representative prototype representations, which adaptively identifies and masks noisy regions within the pseudo proposals, reducing potential erroneous interference during prototype computation. Furthermore, we generate specialized soft pseudo-labels for the noisy regions identified by ANPM, providing supplementary supervision. Extensive experiments on six medical image segmentation tasks involving different modalities demonstrate that the proposed framework significantly outperforms representative state-of-the-art methods.

An enhanced visual state space model for myocardial pathology segmentation in multi-sequence cardiac MRI

BACKGROUND

Myocardial pathology (scar and edema) segmentation plays a crucial role in the diagnosis, treatment, and prognosis of myocardial infarction (MI). However, the current mainstream models for myocardial pathology segmentation have the following limitations when faced with cardiac magnetic resonance(CMR) images with multiple objects and large changes in object scale: the remote modeling ability of convolutional neural networks is insufficient, and the computational complexity of transformers is high, which makes myocardial pathology segmentation challenging.

PURPOSE

This study aims to develop a novel model to address the image characteristics and algorithmic challenges faced in the myocardial pathology segmentation task and improve the accuracy and efficiency of myocardial pathology segmentation.

METHODS

We developed a novel visual state space (VSS)-based deep neural network, MPS-Mamba. In order to accurately and adequately extract CMR image features, the encoder employs a dual-branch structure to extract global and local features of the image. Among them, the VSS branch overcomes the limitations of the current mainstream models for myocardial pathology segmentation by modeling remote relationships through linear computability, while the convolutional-based branch provides complementary local information. Given the unique properties of the dual branches, we design a modular dual-branch fusion module for fusing dual branches to enhance the feature representation of the dual encoder. To improve the ability to model objects of different scales in cardiac magnetic resonance (CMR) images, a multi-scale feature fusion (MSF) module is designed to achieve effective integration and fine expression of multi-scale information. To further incorporate anatomical knowledge to optimize segmentation results, a decoder with three decoding branches is designed to output segmentation results of scar, edema, and myocardium, respectively. In addition, multiple sets of constraint functions are used to not only improve the segmentation accuracy of myocardial pathology but also effectively model the spatial position relationship between myocardium, scar, and edema.

RESULTS

The proposed method was comprehensively evaluated on the MyoPS 2020 dataset, and the results showed that MPS-Mamba achieved an average Dice score of 0.717 ± $\pm$ 0.169 in myocardial scar segmentation, which is superior to the current mainstream methods. In addition, MPS-Mamba also performed well in the edema segmentation task, with an average Dice score of 0.735 ± $\pm$ 0.073. The experimental results further demonstrate the effectiveness of MPS-Mamba in segmenting myocardial pathologies in multi-sequence CMR images, verifying its advantages in myocardial pathology segmentation tasks.

CONCLUSIONS

Given the effectiveness and superiority of MPS-Mamba, this method is expected to become a potential myocardial pathology segmentation tool that can effectively assist clinical diagnosis.

Stagger Network: Rethinking information loss in medical image segmentation with various-sized targets

Medical image segmentation presents the challenge of segmenting various-size targets, demanding the model to effectively capture both local and global information. Despite recent efforts using CNNs and ViTs to predict annotations of different scales, these approaches often struggle to effectively balance the detection of targets across varying sizes. Simply utilizing local information from CNNs and global relationships from ViTs without considering potential significant divergence in latent feature distributions may result in substantial information loss. To address this issue, in this paper, we will introduce a novel Stagger Network (SNet) and argues that a well-designed fusion structure can mitigate the divergence in latent feature distributions between CNNs and ViTs, thereby reducing information loss. Specifically, to emphasize both global dependencies and local focus, we design a Parallel Module to bridge the semantic gap. Meanwhile, we propose the Stagger Module, trying to fuse the selected features that are more semantically similar. An Information Recovery Module is further adopted to recover complementary information back to the network. As a key contribution, we theoretically analyze that the proposed parallel and stagger strategies would lead to less information loss, thus certifying the SNet's rationale. Experimental results clearly proved that the proposed SNet excels comparisons with recent SOTAs in segmenting on the Synapse dataset where targets are in various sizes. Besides, it also demonstrates superiority on the ACDC and the MoNuSeg datasets where targets are with more consistent dimensions.

Lumos: Software for Multi-level Multi-reader Comparison of Cardiovascular Magnetic Resonance Late Gadolinium Enhancement Scar Quantification

Cardiovascular magnetic resonance imaging (CMR) offers state-of-the-art myocardial tissue differentiation. The CMR technique late gadolinium enhancement (LGE) currently provides the noninvasive gold standard for the detection of myocardial fibrosis. Typically, thresholding methods are used for fibrotic scar tissue quantification. A major challenge for standardized CMR assessment is large variations in the estimated scar for different methods. The aim was to improve quality assurance for LGE scar quantification, a multi-reader comparison tool "Lumos" was developed to support quality control for scar quantification methods. The thresholding methods and an exact rasterization approach were implemented, as well as a graphical user interface (GUI) with statistical and case-specific tabs. Twenty LGE cases were considered with half of them including artifacts and clinical results for eight scar quantification methods computed. Lumos was successfully implemented as a multi-level multi-reader comparison software, and differences between methods can be seen in the statistical results. Histograms visualize confounding effects of different methods. Connecting the statistical level with the case level allows for backtracking statistical differences to sources of differences in the threshold calculation. Being able to visualize the underlying groundwork for the different methods in the myocardial histogram gives the opportunity to identify causes for different thresholds. Lumos showed the differences in the clinical results between cases with artifacts and cases without artifacts. A video demonstration of Lumos is offered as supplementary material 1. Lumos allows for a multi-reader comparison for LGE scar quantification that offers insights into the origin of reader differences.

HADCN: a hierarchical ascending densely connected network for enhanced medical image segmentation

Medical image segmentation is a key component in computer-aided diagnostic technology. In the past few years, the U-shaped architecture-based hierarchical model has become the mainstream approach, which however often fails to provide accurate results due to the loss of detailed features. To address this issue, this paper proposes a hierarchical ascending densely connected network, called HADCNet, to capture both local short-range and global long-range pathological features in a hierarchically organized network for more accurate segmentation. First, HADCNet devises a cross-scale ascending densely connected structure with a multi-path attention gate (MAG) to achieve full-scale interaction of global pathological features. Then, spatial-channel reconstruction units (called SRU and CRU) are introduced to decrease redundant computation and facilitate representative feature learning. Finally, multi-scale outputs are aggregated for final imaging. Extensive experiments demonstrate that our method achieves an average DSC of 84.45% and HD95 of 17.55 mm on the Synapse dataset (for multi-organ segmentation), with a similarly impressive performance on the ACDC (for cardiac diagnosis) and ISIC2018 datasets (for lesion segmentation). Additionally, HADCNet can be flexibly incorporated into existing backbone networks for better performance, e.g., combining HADC with TransUnet and SwinUnet, respectively, leads to 3.28% and 2.53% Dice score improvements.

Myocardial perfusion imaging SPECT left ventricle segmentation with graphs

PURPOSE

Various specialized and general collimators are used for myocardial perfusion imaging (MPI) with single-photon emission computed tomography (SPECT) to assess different types of coronary artery disease (CAD). Alongside the wide variability in imaging characteristics, the apriori "learnt" information of left ventricular (LV) shape can affect the final diagnosis of the imaging protocol. This study evaluates the effect of prior information incorporation into the segmentation process, compared to deep learning (DL) approaches, as well as the differences of 4 collimation techniques on 5 different datasets.

METHODS

This study was implemented on 80 patients database. 40 patients were coming from mixed black-box collimators, 10 each, from multi-pinhole (MPH), low energy high resolution (LEHR), CardioC and CardioD collimators. The testing was evaluated on a new continuous graph-based approach, which automatically segments the left ventricular volume with prior information on the cardiac geometry. The technique is based on the continuous max-flow (CMF) min-cut algorithm, which performance was evaluated in precision, recall, IoU and Dice score metrics.

RESULTS

In the testing it was shown that, the developed method showed a good improvement over deep learning reaching higher scores in most of the evaluation metrics. Further investigating the different collimation techniques, the evaluation of receiver operating characterstic (ROC) curves showed different stabilities on the various collimators. Running Wilcoxon signed-rank test on the outlines of the LVs showed differentiability between the collimation procedures. To further investigate these phenomena the model parameters of the LVs were reconstructed and evaluated by the uniform manifold approximation and projection (UMAP) method, which further proved that collimators can be differentiated based on the projected LV shapes alone.

CONCLUSIONS

The results show that prior information incorporation can enhance the performance of segmentation methods and collimation strategies have a high effect on the projected cardiac geometry.

Enhanced nnU-Net Architectures for Automated MRI Segmentation of Head and Neck Tumors in Adaptive Radiation Therapy

The increasing utilization of MRI in radiation therapy planning for head and neck cancer (HNC) highlights the need for precise tumor segmentation to enhance treatment efficacy and reduce side effects. This work presents segmentation models developed for the HNTS-MRG 2024 challenge by the team mic-dkfz, focusing on automated segmentation of HNC tumors from MRI images at two radiotherapy (RT) stages: before (pre-RT) and 2-4 weeks into RT (mid-RT). For Task 1 (pre-RT segmentation), we built upon the nnU-Net framework, enhancing it with the larger Residual Encoder architecture. We incorporated extensive data augmentation and applied transfer learning by pre-training the model on a diverse set of public 3D medical imaging datasets. For Task 2 (mid-RT segmentation), we adopted a longitudinal approach by integrating registered pre-RT images and their segmentations as additional inputs into the nnU-Net framework. On the test set, our models achieved mean aggregated Dice Similarity Coefficient (aggDSC) scores of 81.2 for Task 1 and 72.7 for Task 2. Especially the primary tumor (GTVp) segmentation is challenging and presents potential for further optimization. These results demonstrate the effectiveness of combining advanced architectures, transfer learning, and longitudinal data integration for automated tumor segmentation in MRI-guided adaptive radiation therapy.

LUCF-Net: Lightweight U-Shaped Cascade Fusion Network for Medical Image Segmentation

The performance of modern U-shaped neural networks for medical image segmentation has been significantly enhanced by incorporating Transformer layers. Although Transformer architectures are powerful at extracting global information, its ability to capture local information is limited due to their high complexity. To address this challenge, we proposed a new lightweight U-shaped cascade fusion network (LUCF-Net) for medical image segmentation. It utilized an asymmetrical structural design and incorporated both local and global modules to enhance its capacity for local and global modeling. Additionally, a multi-layer cascade fusion decoding network was designed to further bolster the network's information fusion capabilities. Validation performed on open-source CT, MRI, and dermatology datasets demonstrated that the proposed model outperformed other state-of-the-art methods in handling local-global information, achieving an improvement of 1.46% in Dice coefficient and 2.98 mm in Hausdorff distance on multi-organ segmentation. Furthermore, as a network that combines Convolutional Neural Network and Transformer architectures, it achieves competitive segmentation performance with only 6.93 million parameters and 6.6 gigabytes of floating point operations, without the need for pre-training. In summary, the proposed method demonstrated enhanced performance while retaining a simpler model design compared to other Transformer-based segmentation networks.

Semi-supervised medical image segmentation network based on mutual learning

BACKGROUND

Semi-supervised learning provides an effective means to address the challenge of insufficient labeled data in medical image segmentation tasks. However, when a semi-supervised segmentation model is overfitted and exhibits cognitive bias, its performance will deteriorate. Errors stemming from cognitive bias can quickly amplify and become difficult to correct during the training process of neural networks, resulting in the continuous accumulation of erroneous knowledge.

PURPOSE

To address the issue of error accumulation, a novel learning strategy is required to enhance the accuracy of medical image segmentation.

METHODS

This paper proposes a semi-supervised medical image segmentation network based on mutual learning (MLNet) to alleviate the issue of continuous accumulation of erroneous knowledge. The MLNet adopts a teacher-student network as the backbone framework, training student and teacher networks on labeled data and mutually updating network parameter weights, enabling the two models to learn from each other. Additionally, an image partial exchange algorithm (IPE) as an appropriate perturbation addition method is proposed to reduce the introduction of erroneous information and the disruption to the contextual information of the image.

RESULTS

In the 10% labeled experiment on the ACDC dataset, our Dice coefficient reached 89.48%, a 9.28% improvement over the baseline model. In the 10% labeled experiment on the BraTS2019 dataset, the proposed method still performs exceptionally well, achieving 84.56%, surpassing other comparative methods.

CONCLUSIONS

Compared with other methods, experimental results demonstrate that our approach achieves optimal performance across all metrics, proving its effectiveness and reliability.

CMR-BENet: A confidence map refinement boundary enhancement network for left ventricular myocardium segmentation

BACKGROUND AND OBJECTIVE

Left ventricular myocardium segmentation is of great significance for clinical diagnosis, treatment, and prognosis. However, myocardium segmentation is challenging as the medical image quality is disturbed by various factors such as motion, artifacts, and noise. Its accuracy largely depends on the accurate identification of edges and structures. Most existing encoder-decoder based segmentation methods capture limited contextual information and ignore the awareness of myocardial shape and structure, often producing unsatisfactory boundary segmentation results in noisy scenes. Moreover, these methods fail to assess the reliability of the predictions, which is crucial for clinical decisions and applications in medical tasks. Therefore, this study explores how to effectively combine contextual information with myocardial edge structure and confidence maps to improve segmentation performance in an end-to-end network.

METHODS

In this paper, we propose an end-to-end confidence map refinement boundary enhancement network (CMR-BENet) for left ventricular myocardium segmentation. CMR-BENet has three components: a layer semantic-aware module (LSA), an edge information enhancement module (EIE), and a confidence map-based refinement module (CMR). Specifically, LSA first adaptively fuses high- and low-level semantic information across hierarchical layers to mitigate the bias of single-layer features affected by noise. EIE then improves the edge and structure recognition by designing the edge and mask guidance module (EMG) and the edge structure-aware module (ESA). Finally, CMR provides a simple and efficient way to estimate confidence maps and effectively combines the encoder features to refine the segmentation results.

RESULTS

Experiments on two echocardiography datasets and one cardiac MRI dataset show that the proposed CMR-BENet outperforms its rivals in the left ventricular myocardium segmentation task with Dice (DI) of 87.71%, 79.33%, and 89.11%, respectively.

CONCLUSION

This paper utilizes edge information to characterize the shape and structure of the myocardium and introduces learnable confidence maps to evaluate and refine the segmentation results. Our findings provide strong support and reference for physicians in diagnosis and treatment.

Hybrid deep learning for computational precision in cardiac MRI segmentation: Integrating Autoencoders, CNNs, and RNNs for enhanced structural analysis

Recent advancements in cardiac imaging have been significantly enhanced by integrating deep learning models, offering transformative potential in early diagnosis and patient care. The research paper explores the application of hybrid deep learning methodologies, focusing on the roles of Autoencoders, Convolutional Neural Networks (CNNs), and Recurrent Neural Networks (RNNs) in enhancing cardiac image analysis. The study implements a comprehensive approach, combining traditional algorithms such as Sobel, Watershed, and Otsu's Thresholding with advanced deep learning models to achieve precise and accurate imaging outcomes. The Autoencoder model, developed for image enhancement and feature extraction, achieved a notable accuracy of 99.66% on the test data. Optimized for image recognition tasks, the CNN model demonstrated a high precision rate of 98.9%. The RNN model, utilized for sequential data analysis, showed a prediction accuracy of 98%, further underscoring the robustness of the hybrid framework. The research drew upon a diverse range of academic databases and pertinent publications within cardiac imaging and deep learning, focusing on peer-reviewed articles and studies published in the past five years. Models were implemented using the TensorFlow and Keras frameworks. The proposed methodology was evaluated in the clinical validation phase using advanced imaging protocols, including the QuickScan technique and balanced steady-state free precession (bSSFP) imaging. The validation metrics were promising: the Signal-to-Noise Ratio (SNR) was improved by 15%, the Contrast-to-Noise Ratio (CNR) saw an enhancement of 12%, and the ejection fraction (EF) analysis provided a 95% correlation with manually segmented data. These metrics confirm the efficacy of the models, showing significant improvements in image quality and diagnostic accuracy. The integration of adversarial defense strategies, such as adversarial training and model ensembling, have been analyzed to enhance model robustness against malicious inputs. The reliability and comparison of the model's ability have been investigated to maintain clinical integrity, even in adversarial attacks that could otherwise compromise segmentation outcomes. These findings indicate that integrating Autoencoders, CNNs, and RNNs within a hybrid deep-learning framework is promising for enhancing cardiac MRI segmentation and early diagnosis. The study contributes to the field by demonstrating the applicability of these advanced techniques in clinical settings, paving the way for improved patient outcomes through more accurate and timely diagnoses.

The Role of Artificial Intelligence in the Prediction, Diagnosis, and Management of Cardiovascular Diseases: A Narrative Review

Cardiovascular diseases (CVDs) remain the leading global cause of mortality, and a high prevalence of cardiac conditions, including premature deaths, have increased from decades until today. However, early detection and management of these conditions are challenging, given their complexity, the scale of affected populations, the dynamic nature of the disease process, and the treatment approach. The transformative potential is being brought by Artificial Intelligence (AI), specifically machine learning (ML) and deep learning technologies, to analyze massive datasets, improve diagnostic accuracy, and optimize treatment strategy. The recent advancements in such AI-based frameworks as the personalization of decision-making support systems for customized medicine automated image assessments drastically increase the precision and efficiency of healthcare professionals. However, implementing AI is widely clogged with obstacles, including regulatory, privacy, and validation across populations. Additionally, despite the desire to incorporate AI into clinical routines, there is no shortage of concern about interoperability and clinician acceptance of the system. Despite these challenges, further research and development are essential for overcoming these hurdles. This review explores the use of AI in cardiovascular care, its limitations for current use, and future integration toward better patient outcomes.

A multi-scale information fusion medical image segmentation network based on convolutional kernel coupled updata mechanism

Medical image segmentation is pivotal in disease diagnosis and treatment. This paper presents a novel network architecture for medical image segmentation, termed TransDLNet, which is engineered to enhance the efficiency of multi-scale information utilization. TransDLNet integrates convolutional neural networks and Transformers, facilitating cross-level multi-scale information fusion for complex medical images. Key to its innovation is the attention-dilated depthwise convolution (ADDC) module, utilizing depthwise convolution (DWConv) with varied dilation rates to enhance local detail capture. A convolution kernel coupled update mechanism and channel information compensation method ensure robust feature representation. Furthermore, the cross-level grouped attention merge (CGAM) module in both encoder and decoder enhances feature interaction and integration across scales, boosting comprehensive representation. We conducted a comprehensive experimental analysis and quantitative evaluation on four datasets representing diverse modalities. The results indicate that the proposed method has good segmentation performance and generalization ability.

QMaxViT-Unet+: A query-based MaxViT-Unet with edge enhancement for scribble-supervised segmentation of medical images

The deployment of advanced deep learning models for medical image segmentation is often constrained by the requirement for extensively annotated datasets. Weakly-supervised learning, which allows less precise labels, has become a promising solution to this challenge. Building on this approach, we propose QMaxViT-Unet+, a novel framework for scribble-supervised medical image segmentation. This framework is built on the U-Net architecture, with the encoder and decoder replaced by Multi-Axis Vision Transformer (MaxViT) blocks. These blocks enhance the model's ability to learn local and global features efficiently. Additionally, our approach integrates a query-based Transformer decoder to refine features and an edge enhancement module to compensate for the limited boundary information in the scribble label. We evaluate the proposed QMaxViT-Unet+ on four public datasets focused on cardiac structures, colorectal polyps, and breast cancer: ACDC, MS-CMRSeg, SUN-SEG, and BUSI. Evaluation metrics include the Dice similarity coefficient (DSC) and the 95th percentile of Hausdorff distance (HD95). Experimental results show that QMaxViT-Unet+ achieves 89.1% DSC and 1.316 mm HD95 on ACDC, 88.4% DSC and 2.226 mm HD95 on MS-CMRSeg, 71.4% DSC and 4.996 mm HD95 on SUN-SEG, and 69.4% DSC and 50.122 mm HD95 on BUSI. These results demonstrate that our method outperforms existing approaches in terms of accuracy, robustness, and efficiency while remaining competitive with fully-supervised learning approaches. This makes it ideal for medical image analysis, where high-quality annotations are often scarce and require significant effort and expense. The code is available at https://github.com/anpc849/QMaxViT-Unet.

Effective Semi-Supervised Medical Image Segmentation With Probabilistic Representations and Prototype Learning

Label scarcity, class imbalance and data uncertainty are three primary challenges that are commonly encountered in the semi-supervised medical image segmentation. In this work, we focus on the data uncertainty issue that is overlooked by previous literature. To address this issue, we propose a probabilistic prototype-based classifier that introduces uncertainty estimation into the entire pixel classification process, including probabilistic representation formulation, probabilistic pixel-prototype proximity matching, and distribution prototype update, leveraging principles from probability theory. By explicitly modeling data uncertainty at the pixel level, model robustness of our proposed framework to tricky pixels, such as ambiguous boundaries and noises, is greatly enhanced when compared to its deterministic counterpart and other uncertainty-aware strategy. Empirical evaluations on three publicly available datasets that exhibit severe boundary ambiguity show the superiority of our method over several competitors. Moreover, our method also demonstrates a stronger model robustness to simulated noisy data. Code is available at https://github.com/IsYuchenYuan/PPC.

A 3D medical image segmentation network based on gated attention blocks and dual-scale cross-attention mechanism

In the field of multi-organ 3D medical image segmentation, Convolutional Neural Networks (CNNs) are limited to extracting local feature information, while Transformer-based architectures suffer from high computational complexity and inadequate extraction of spatial and channel layer information. Moreover, the large number and varying sizes of organs to be segmented result in suboptimal model robustness and segmentation outcomes. To address these challenges, this paper introduces a novel network architecture, DS-UNETR++, specifically designed for 3D medical image segmentation. The proposed network features a dual-branch feature encoding mechanism that categorizes images into coarse-grained and fine-grained types before processing them through the encoding blocks. Each encoding block comprises a downsampling layer and a Gated Shared Weighted Pairwise Attention (G-SWPA) submodule, which dynamically adjusts the influence of spatial and channel attention on feature extraction. Additionally, a Gated Dual-Scale Cross-Attention Module (G-DSCAM) is incorporated at the bottleneck stage. This module employs dimensionality reduction techniques to cross-coarse-grained and fine-grained features, using a gating mechanism to dynamically balance the ratio of these two types of feature information, thereby achieving effective multi-scale feature fusion. Finally, comprehensive evaluations were conducted on four public medical datasets. Experimental results demonstrate that DS-UNETR++ achieves good segmentation performance, highlighting the effectiveness and significance of the proposed method and offering new insights for various organ segmentation tasks.

Uncertainty CNNs: A path to enhanced medical image classification performance

The automated detection of tumors using medical imaging data has garnered significant attention over the past decade due to the critical need for early and accurate diagnoses. This interest is fueled by advancements in computationally efficient modeling techniques and enhanced data storage capabilities. However, methodologies that account for the uncertainty of predictions remain relatively uncommon in medical imaging. Uncertainty quantification (UQ) is important as it helps decision-makers gauge their confidence in predictions and consider variability in the model inputs. Numerous deterministic deep learning (DL) methods have been developed to serve as reliable medical imaging tools, with convolutional neural networks (CNNs) being the most widely used approach. In this paper, we introduce a low-complexity uncertainty-based CNN architecture for medical image classification, particularly focused on tumor and heart failure (HF) detection. The model's predictive (aleatoric) uncertainty is quantified through a test-set augmentation technique, which generates multiple surrogates of each test image. This process enables the construction of empirical distributions for each image, which allows for the calculation of mean estimates and credible intervals. Importantly, this methodology not only provides UQ, but also significantly improves the model's classification performance. This paper represents the first effort to demonstrate that test-set augmentation can significantly improve the classification performance of medical images. The proposed DL model was evaluated using three datasets: (a) brain magnetic resonance imaging (MRI), (b) lung computed tomography (CT) scans, and (c) cardiac MRI. The low-complexity design of the model enhances its robustness against overfitting, while it is also easily re-trainable in case out-of-distribution data is encountered, due to the reduced computational resources required by the introduced architecture.

Rethinking Copy-Paste for Consistency Learning in Medical Image Segmentation

Semi-supervised learning based on consistency learning offers significant promise for enhancing medical image segmentation. Current approaches use copy-paste as an effective data perturbation technique to facilitate weak-to-strong consistency learning. However, these techniques often lead to a decrease in the accuracy of synthetic labels corresponding to the synthetic data and introduce excessive perturbations to the distribution of the training data. Such over-perturbation causes the data distribution to stray from its true distribution, thereby impairing the model's generalization capabilities as it learns the decision boundaries. We propose a weak-to-strong consistency learning framework that integrally addresses these issues with two primary designs: 1) it emphasizes the use of highly reliable data to enhance the quality of labels in synthetic datasets through cross-copy-pasting between labeled and unlabeled datasets; 2) it employs uncertainty estimation and foreground region constraints to meticulously filter the regions for copy-pasting, thus the copy-paste technique implemented introduces a beneficial perturbation to the training data distribution. Our framework expands the copy-paste method by addressing its inherent limitations, and amplifying the potential of data perturbations for consistency learning. We extensively validated our model using six publicly available medical image segmentation datasets across different diagnostic tasks, including the segmentation of cardiac structures, prostate structures, brain structures, skin lesions, and gastrointestinal polyps. The results demonstrate that our method significantly outperforms state-of-the-art models. For instance, on the PROMISE12 dataset for the prostate structure segmentation task, using only 10% labeled data, our method achieves a 15.31% higher Dice score compared to the baseline models. Our experimental code will be made publicly available at https://github.com/slhuang24/RCP4CL.

Real world federated learning with a knowledge distilled transformer for cardiac CT imaging

Federated learning is a renowned technique for utilizing decentralized data while preserving privacy. However, real-world applications often face challenges like partially labeled datasets, where only a few locations have certain expert annotations, leaving large portions of unlabeled data unused. Leveraging these could enhance transformer architectures' ability in regimes with small and diversely annotated sets. We conduct the largest federated cardiac CT analysis to date (n = 8, 104) in a real-world setting across eight hospitals. Our two-step semi-supervised strategy distills knowledge from task-specific CNNs into a transformer. First, CNNs predict on unlabeled data per label type and then the transformer learns from these predictions with label-specific heads. This improves predictive accuracy and enables simultaneous learning of all partial labels across the federation, and outperforms UNet-based models in generalizability on downstream tasks. Code and model weights are made openly available for leveraging future cardiac CT analysis.

An algorithm for cardiac disease detection based on the magnetic resonance imaging

In experiments to detect heart disease on cardiac magnetic resonance imaging (MRI) medical images, existing object detection models face several challenges including low accuracy and unreliable detection results. To tackle these issues, this article proposes an innovative method for Object Detection in cardiac MRI medical images called SA-YOLO. This method is based on the YOLOv8 model but introduces several key modifications. Firstly, the standard Spatial Pyramid Pooling Fast module is replaced with a Multi-Channel Spatial Pyramid Pooling module. Secondly, an attention mechanism combining the ideas of Squeeze-Excitation and Coordinate Attention designed, and integrated into the Neck part of the baseline model. Subsequently, the bounding box regression loss function CIoU loss of the model was replaced with the iSD-IoU loss that combines shape loss and distance loss. Finally, comparative experiments were conducted on the Automated Cardiac Diagnosis Challenge cardiac MRI image dataset where it was found that SA-YOLOv8 achieved better results in detecting cardiac pathologies, and improvement of 7.4% in mAP0.5 value and 5.1% in mAP0.5-0.95 value compared to the baseline model.

IEA-Net: Internal and External Dual-Attention Medical Segmentation Network with High-Performance Convolutional Blocks

Currently, deep learning is developing rapidly in the field of image segmentation, and medical image segmentation is one of the key applications in this field. Conventional CNN has achieved great success in general medical image segmentation tasks, but it has feature loss in the feature extraction part and lacks the ability to explicitly model remote dependencies, which makes it difficult to adapt to the task of human organ segmentation. Although methods containing attention mechanisms have made good progress in the field of semantic segmentation, most of the current attention mechanisms are limited to a single sample, while the number of samples of human organ images is large, ignoring the correlation between the samples is not conducive to image segmentation. In order to solve these problems, an internal and external dual-attention segmentation network (IEA-Net) is proposed in this paper, and the ICSwR (interleaved convolutional system with residual) module and the IEAM module are designed in this network. The ICSwR contains interleaved convolution and hopping connection, which are used for the initial extraction of the features in the encoder part. The IEAM module (internal and external dual-attention module) consists of the LGGW-SA (local-global Gaussian-weighted self-attention) module and the EA module, which are in a tandem structure. The LGGW-SA module focuses on learning local-global feature correlations within individual samples for efficient feature extraction. Meanwhile, the EA module is designed to capture inter-sample connections, addressing multi-sample complexities. Additionally, skip connections will be incorporated into each IEAM module within both the encoder and decoder to reduce feature loss. We tested our method on the Synapse multi-organ segmentation dataset and the ACDC cardiac segmentation dataset, and the experimental results show that the proposed method achieves better performance than other state-of-the-art methods.

Computer vision algorithms in healthcare: Recent advancements and future challenges

Computer vision has emerged as a promising technology with numerous applications in healthcare. This systematic review provides an overview of advancements and challenges associated with computer vision in healthcare. The review highlights the application areas where computer vision has made significant strides, including medical imaging, surgical assistance, remote patient monitoring, and telehealth. Additionally, it addresses the challenges related to data quality, privacy, model interpretability, and integration with existing healthcare systems. Ethical and legal considerations, such as patient consent and algorithmic bias, are also discussed. The review concludes by identifying future directions and opportunities for research, emphasizing the potential impact of computer vision on healthcare delivery and outcomes. Overall, this systematic review underscores the importance of understanding both the advancements and challenges in computer vision to facilitate its responsible implementation in healthcare.

CapNet: An Automatic Attention-Based with Mixer Model for Cardiovascular Magnetic Resonance Image Segmentation

Deep neural networks have shown excellent performance in medical image segmentation, especially for cardiac images. Transformer-based models, though having advantages over convolutional neural networks due to the ability of long-range dependence learning, still have shortcomings such as having a large number of parameters and and high computational cost. Additionally, for better results, they are often pretrained on a larger data, thus requiring large memory size and increasing resource expenses. In this study, we propose a new lightweight but efficient model, namely CapNet, based on convolutions and mixing modules for cardiac segmentation from magnetic resonance images (MRI) that can be trained from scratch with a small amount of parameters. To handle varying sizes and shapes which often occur in cardiac systolic and diastolic phases, we propose attention modules for pooling, spatial, and channel information. We also propose a novel loss called the Tversky Shape Power Distance function based on the shape dissimilarity between labels and predictions that shows promising performances compared to other losses. Experiments on three public datasets including ACDC benchmark, Sunnybrook data, and MS-CMR challenge are conducted and compared with other state of the arts (SOTA). For binary segmentation, the proposed CapNet obtained the Dice similarity coefficient (DSC) of 94% and 95.93% for respectively the Endocardium and Epicardium regions with Sunnybrook dataset, 94.49% for Endocardium, and 96.82% for Epicardium with the ACDC data. Regarding the multiclass case, the average DSC by CapNet is 93.05% for the ACDC data; and the DSC scores for the MS-CMR are 94.59%, 92.22%, and 93.99% for respectively the bSSFP, T2-SPAIR, and LGE sequences of the MS-CMR. Moreover, the statistical significance analysis tests with p-value < 0.05 compared with transformer-based methods and some CNN-based approaches demonstrated that the CapNet, though having fewer training parameters, is statistically significant. The promising evaluation metrics show comparative results in both Dice and IoU indices compared to SOTA CNN-based and Transformer-based architectures.

Semi-supervised medical image segmentation with dual-branch mixup-decoupling confidence training

Semi-supervised medical image segmentation algorithms hold significant research and practical value due to their ability to reduce labeling dependency and annotation costs. However, most current algorithms lack diverse regularization methods to effectively exploit robust knowledge from unlabeled data. The pseudo-label filtering methods employed are often overly simplistic, which exacerbates the serious category imbalance problem in medical images. Additionally, these algorithms fail to provide robust semantic representations for comparative learning in multi-scenario settings, making it challenging for the model to learn more discriminative semantic information. To address these issues, we propose a semi-supervised medical image segmentation algorithm that utilizes dual-branch mixup-decoupling confidence training to establish a dual-stream semantic link between labeled and unlabeled images, thereby alleviating semantic ambiguity. Furthermore, we design a bidirectional confidence contrast learning method to maximize the consistency between similar pixels and the distinction between dissimilar pixels in both directions across different feature embeddings in dual views. This enables the model to learn the key features of intra-class similarity and inter-class separability. We conduct a series of experiments on both 2D and 3D datasets, and the experimental results demonstrate that the proposed algorithm achieves notable segmentation performance, outperforming other recent state-of-the-art algorithms.

Learning With Explicit Shape Priors for Medical Image Segmentation

Medical image segmentation is a fundamental task for medical image analysis and surgical planning. In recent years, UNet-based networks have prevailed in the field of medical image segmentation. However, convolutional neural networks (CNNs) suffer from limited receptive fields, which fail to model the long-range dependency of organs or tumors. Besides, these models are heavily dependent on the training of the final segmentation head. And existing methods can not well address aforementioned limitations simultaneously. Hence, in our work, we proposed a novel shape prior module (SPM), which can explicitly introduce shape priors to promote the segmentation performance of UNet-based models. The explicit shape priors consist of global and local shape priors. The former with coarse shape representations provides networks with capabilities to model global contexts. The latter with finer shape information serves as additional guidance to relieve the heavy dependence on the learnable prototype in the segmentation head. To evaluate the effectiveness of SPM, we conduct experiments on three challenging public datasets. And our proposed model achieves state-of-the-art performance. Furthermore, SPM can serve as a plug-and-play structure into classic CNNs and Transformer-based backbones, facilitating the segmentation task on different datasets. Source codes are available at https://github.com/AlexYouXin/Explicit-Shape-Priors.